[bisq-network/bisq] ConcurrentModificationException FileManager::saveToFile (#3752)

[chimp1984]

Thanks for looking into it. It is a tricky topic....

If mapping to `Userthread` we are thread safe but we lose a part of the threading benefits, specially for frequent writes (sequence number,...) as the mapping itself takes time and if the protof conversion is expensive we get the UI stuck for that moment. 

I think its probably best to first analyse which objects cost a lot of cpu time for the protobuffer conversion (`DaoState`, `TradeStatistics`, `AccountAgeWitness` are likely the 3 top candidates - all the files in the db folder which are rather big). For those objects we need to try to avoid frequent persistence and the keep the conversion to protobuf on a separate thread. Then there are objects which are rather small but gets written frequently. For those we should check which ones are critical if there is a state mismatch. `Navigation` or `Peers` for instance are totally non-critical but others like `SequenceNrMap` are likely more problematic.

To let the storage objects decide how the threading is handled would be good.

For the large objects (`DaoState`, `TradeStatistics`, `AccountAgeWitness`) we will need anyway once a solution which is scalable. Maybe its the right time to think about whats the best options are? One idea was to split them into "historical" static data which gets shipped via resources and never gets updated/written and the dynamic part (as it is now). If we can keep the dynamic part small (e.g. the data from the last release) I think we have reduced the protobuf conversion cost by > 90%.  

If this all turns out to a larger and difficult project we should also consider if a database solution would be better, but that is probably also a bigger project... 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/bisq/issues/3752#issuecomment-562782129
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20191206/bea9f2bf/attachment-0001.html>