Hi guys, in Volt, what’s the recommended way to achieve the equivalent of appending data to a log file (or in my case a rolling database)?
This is not about a problem but just requesting guidance for best practice.
My requirement is to add data in blocks of approx 6 minutes (small amounts of data usually much less than 1KB). This is for a fixed number (~20) of users, so it will never be masses of data.
Is it normal or OK to simply set a large number of key value pairs? Is their any sensible limit to the number of keys one should consider? Is there a way of grouping them into folders? (virtual or otherwise)? Can or should a key value pair be overwritten with a newer larger amount to “grow” the data?
This is an interesting use case! Currently we don’t have a great solution in place for logging, but it’s something we’ve talked about a bit before. In the interim, I’d suggest choosing a key-value pair per user and managing the log as an array of strings (or objects). Just make sure you manage the data so that it doesn’t grow out of control. How many log entries do you plan to keep? How big is each log entry?
BTW - Sorry about the delay in responding - it’s been rather busy these past few days!
Hi, thanks for that. The need is to duplicate all the data on an 8MB eeProm. Typically, that would be around 8000 entries of under 1KB each (compressed) but with 20 users, that is near 200K entries. The data is changed on a carousel basis approx monthly.
To keep it seemly, I think I will need to write a “housekeeping” and retrieval app as well. The housekeeping app will combine and remove.
The total amount of data after 6 month (max trial phase) would be 960mb if we don’t delete on a monthly basis or similar. The problem with deleting it is we would have to transfer it it to another server as it is supposed to be available to many. Cheers
I read this one and didn’t completely understand what you were trying to do. Not for explanation on your part but most likely I was up too late …lol…
If you are dealing with the much data (~960mb) it may be better to use mySQL or SqlServer? Using JSON calls to some type of middleware that does the heavy lifting will prevent you from worry of the Volt storage being taxed.
Effectively, Volt can also be used as an authentication server as it uses JWT and can pass a token that will give access to those who have logged in.
One question that comes to mind is what kind of retrieval are you going to be looking for? 200K keys are going to be hard to manage. It strikes me that a key value store might not be the most appropriate abstraction for what you are trying to accomplish.
If you would like to email me a detailed description of what you’re hoping to accomplish, I’m sure we can come up with something.
…lol…