[bisq-network/bisq] Change write DAO json files to disk strategy (#4635)

chimp1984 notifications at github.com
Tue Oct 13 04:54:28 UTC 2020


 Change write DAO json files to disk strategy

We have written all files each time at each new block which accumulated to about
1 GB of data, this led to stability issues and very high disk IO for the explorer nodes.

Now we write only the data of the new block. We also remove the deletion of all
files at startup.
The dao state is still written in a monolithic file so that cannot be optimized
but we added a new directly where we add each block named by block height.
Looking up the latest written file tells us our last persisted block.
So all that data is equivalent to the monolithic daoState data which should
be removed once the webapp devs have implemented the change to use the blocks directory.

As this code is affecting only explorer nodes I think it can be merged to 1.4.0 without big risk. It would make it easier for operators to use that version instead of waiting to get it once merged into master. Alternatively they could run from my branch but I think it is more clean that way.
You can view, comment on, or merge this pull request online at:

  https://github.com/bisq-network/bisq/pull/4635

-- Commit Summary --

  * Change write json files to disk strategy

-- File Changes --

    M common/src/main/java/bisq/common/file/JsonFileManager.java (103)
    M core/src/main/java/bisq/core/app/misc/ExecutableForAppWithP2p.java (4)
    M core/src/main/java/bisq/core/dao/node/BsqNode.java (6)
    M core/src/main/java/bisq/core/dao/node/explorer/ExportJsonFilesService.java (200)
    M core/src/main/java/bisq/core/dao/node/full/FullNode.java (2)
    M core/src/main/java/bisq/core/dao/node/lite/LiteNode.java (13)
    M core/src/main/java/bisq/core/offer/OfferBookService.java (2)
    M core/src/main/java/bisq/core/trade/DumpDelayedPayoutTx.java (2)
    M core/src/main/java/bisq/core/trade/statistics/TradeStatisticsManager.java (6)

-- Patch Links --

https://github.com/bisq-network/bisq/pull/4635.patch
https://github.com/bisq-network/bisq/pull/4635.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/bisq/pull/4635
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20201012/cb890e6a/attachment.html>


More information about the bisq-github mailing list