localStorage data size

I’m trying to decide how to store data in localStorage. If I have data that is being updated every time the user interacts with the app that is stored in localStorage, then I have to do the save to localStorage after each data item is changed (assuming a web app). EG, there is no way to capture the app close so you can dump important memory variables to localStorage, you must save after each change.

First, I’m using JSONstringify to turn the data objects into strings for storage into localStorage.

I could save a large chunk of data, of which the one data point that is changed would be a part of it. Or I could setup a bunch of small chunks and update one of the small chunks. I’m leaning toward the large chunk, as that is how the data is currently stored in memory of the app - eg, it could be one - to - one localStorage to internal memory.

Anyone have any input on chunk sizing in localStorage?

I’ve never heard of a limit but most I’ve stored is about 5k. On iOS Apple will delete localStorage and IndexedDB after 5 days of non-use.

I haven’t seen anything documented on this. I think that iOS has a limit of 5 megs. How big is your chunk?

Chunk size - let’s say 1mb stringified average for the large chunk. A small chuck would be 1k to 5k.

I guess it’s like writing a single record to a file verses rewriting the entire file. When we’re worried about spinning a disk and waiting, or eating up server resources we are charged for, then writing records makes sense. But in this case, these resources are not “ours”, they are the user’s device. Is it important to be efficient? Or just get the job done?

Chunk Size? I believe that most browsers can handle at least 5 megs totaled for all of the individual chunks. I did some testing some time ago and created a single “chunk” very large. I believe I was able to save this chunk to the maximum localstorage limit with no other localStorage chunks. I think it’s possible to use Try/Fail coding if you think you are going to reach a localStorage limit.

Concerning saving data, I use the same database concepts as would be used in say SQL: table, record, and field data. I create record strings separated by a special character to distinguish the individual records. Within each record are the fields with data. Simple function calls are used to read a record, or group of records from a local storage table where the localStorage key is the name of the table. And again, simple functions extract, update, add or delete data fields from each record.

I don’t use xml, json, etc to parse the data tables since I find these use too much space on itty-bitty devices. Comma delimited tables are more efficient but require constant maintenance as fields are added or removed, and fixed positioned and sized fields are well just forget it. If you are interested in what I do use, let me know. I can continue with this thread or start another topic.

John

John,
Thanks for your insight. There’s not a lot of difference from your method of using a large string and substr’ing to get the “record”. Verses inserting new data into an “large chunk” object (which is part of the app internal processing in this case) and then stringify’ing the entire object. The difference is substr speed of execution vs stringify. I’m sure stringify is slower, but back to my last comment, do we really care about resource usage (other than memory) on a user’s device? EG, using stringify is easy in my case, so why not leverage it.
Gary

If stringify works for parsing your data and the user doesn’t suffer a speed penalty, I’d stay with it. Personally, on large data strings with many records and many fields I prefer instr for best speed. And, in a pinch, I’ll create a btree indexing method for best performance. At a cost, of course, lots of programming involved.

For storage: If this is not an issue I would find a multitude of localStorage keys awkward to work with. I use little “chunks” sparingly, mostly for app settings and reentry points. For user generated data I use big “chunks” and parse accordingly. It’s a lot neater because local storage doesn’t get overloaded with a multitude of keys. Plus, internally, updating, adding and deleting these keys, if that is a user requirement, seems a bit more work.

John

Thanks again for the insight. The management of a multitude of keys was concerning to me.

My app design included the use of a single object to hold all of the user’s data. This was done prior to considering localStorage of such data. I was mainly concerned with transferring the data to the server, which in my case, JSON was the best solution. But I only send bits of data upline, in a stream. On the server side, smaller chunks consisting of 1 to 2 dozen data points are stored as JSON. These smaller chunks correspond to the Get Data from server requests, as I didn’t want to download megabytes of data during the user experience. These smaller chunks are 1k to 5k.

Sounds a bit horrible that I have three data organizations, but from an efficiency and speed view, I believe I have a very responsive system,

I do have a token system implemented that allows the app to know if it’s data in localStorage is fresh or not. If not, I want to delete the localStorage data, and then, as the user progresses thru the app, I’ll download the small chunks of data from the server, rebuilding the internal data object. The one data chunk in localStorage makes it easy to manage.