[bisq-network/bisq] [WIP] New persistence model (#4527)

Thu Oct 1 06:29:28 UTC 2020

@chimp1984 pushed 100 commits.

cb9f0f0e448b770b6debd75347918cc023169343  Initial tests
3296db30d22bc1028e460b803c6cf56835f466ee  Version helper uses real Bisq version
e7831d67a505030d4d3e60454480ba865577f4db  Tests use static fixtures
581286e15dd2c951749d8b8193c3ca28d9011742  Migration test succeeds
fc7a43dcac9fbe007e842ea9cf3846fb203c121a  GetMap test succeeds
d111d8ce0eb443b0f0832cae3c43aa6dcc903d0b  GetMapSinceFilter works
6eda9a91162aa6c6ea58d5140ed7d069242c9256  updateScenario works
ff53f516866a448ed7c89b3ec8ea8a9ff55b78d6  freshInstall scenario works
49731f9d738bfabba7101a593d5807edbb7ad220  put scenario works
e123a0b824571bc72e3bd8ef3135fa32eed866a2  Introduce SplitStore and Service to handle BL
da5d41b55f057b0054727b72081eff8367809008  Moved file handling logic to SplitStoreService
c7351a058d1957a45c2ff53c2d5bf7212682c055  Activate other split data stores
e2e80d7ed2b12bea510ba45b1e26f3bd95bafb50  Refactored tests
10c1ecc1fe92427c91e758b08f184a73dfecf855  Initial tests for requests
f6cea15b25638c18e5b27ebca8ed82917bc13a63  Requests are smaller
0d80123b55623d0e872913599b4e0bfb891312db  Prevent adding duplicate data
aad57c3c34775801d917abfc64d8b4d01daa0cd9  Enforce version format and filter faulty ones
faca8e4f96c47609be7fe24d2571e9feafd2597a  Test incoming special keys from the future
72600b15d59ae0cd7f9115d8007016abb2f218e1  Javadoc cosmetics
3ad395b928e72629c497c5906b2c89878f182f29  Init integration test script
25891371f481519f0e25059b90e4c7ec3aa19c8f  Persist live database after split
34abad09dfd4f2678b9e3edb3f8eb0f7ec27176c  Integration test script tests stock
62a77e331dc0a78a087f13145a7764183ad59e96  Use version history array
39fcca5a524cda2cb7d7f23c7351f28c811fca0c  Integration test script tests new system
a3628ffffcad41b019770d18e7bade51ceb97c85  Use full sync between seednodes
a37424901838372c87f8636e2aa7d1fcc595fe00  Cleanup integration test script
8e55f640a4eebcd9a8c0bc83b1270e2072780944  Ignore some tests
e927e6d4cf343684d8d12e84bc76f3b5e13f927a  Adjust P2PDataStorage tests
2fb22321bf5958bdef45ce49fab001c76833624c  Add another migration test
5f8b7cfe8f0c929b351d909b8014f0a272f9d13a  Reactivate ignored tests
f8320fe10a6d493008fe1fc5e601740c10948133  Refactoring and cleaning up
017bbfd5e3118456d95eafa477e2bc3d1148d279  Cleanup
d2a665519af569f61383d1b49cf34d398bd34068  Refactored test helpers
469c374ee7988753aab37dbd23d515f4dfb10ab4  Fixed a potential issue when loading stores
249e185878272c9e994bf5c60c5db923ed774a29  Fixed wrong filename
06ec223f4282d8b4818a92b903b1b8436647eae4  Refactoring
eabadc0f69213b3aa3e705f0ed7dbd3f8a99511c  Make special key platform-independent
755fc03303467636baffa11a535dcbf96c6d6c65  Fixed persisting wrong data store
d218f2e27b1164049c8538c731bb6834b83308df  Renamed data stores
c34bfeee4e9760ef45be18fa111b77cee76bef09  Replace TODO comment
d6f195d7b0822cb68684e065a411cdc78dab1b7a  Fixed a timing issue
d81bdab81fbdf3fa555d9f29b03243b15a47ac18  Added 1.3.6 and 1.3.7 data stores
abbfedc9625af64635286c9e8773c6050cd3a0ba  Update history version list
5f9440639f6e8be333d74f9f378d0c1ebbb1fa68  Removed deprecated test mechanisms
0eccec799098149fed62ce826f1a892ae8c2561e  Cleanup imports
270f1e70f40ffcce98c95d5cb12c0e29ce77aba1  Included PR feedback
c8c63efe4bc9b7970ffacf0ea0bb1d2596e3e877  Included PR feedback
76cc37e6bb3a87cb01ce7c5de11fa81a4156caec  Added 1.3.8 data stores
b4aae51ea504d700706a021d194ebd19ddccb7ac  Use Version.VERSION again
f83c147b45e633d03f55865e497719d67181445d  Merge branch 'master_upstream' into chimp-reduce_initial_request_size
5f8e779bfe868f18da43838ca9d6f9256332258b  Refactor: Add getStore method to be reused by SplitStoreService
3100888b2e3f05a5dda9e12f4f3048f8c80d7022  Use readStore(); instead of store = readStore(getFileName());
591c2fb7c0bf570c22159b0e1ba5e1aeb2699b4c  Apply code inspection suggestions
d5c66449aa38d7b6947ca78da47e9613fba26992  Add mapOfHistoricalStores field
3dac4c4ca53adfb302dfe99b02df820fe637a046  Revert critical data stores to current version.
2cef4119d4e0c06af1585da22e382cd81fe25aea  Add version field
8458a88d7fbf070b002d0f3533e21bafa789003f  Remove methods in MapStoreService and do instead instance of check in AppendOnlyDataStoreService to call the new method only where it makes sense.
2ece512dd8181452ed831f8d24018f437b252ad0  Add current version to version list. Makes it more clear that those are all historical immutable data.
965de7fb928f322e07be478b39801ac4a3e8cd01  Remove file
cfefae2d40d41f9960430d043dcc299e422bcca5  Rename SplitStore to PersistableNetworkPayload
28987f54d32dd605b4213133b6ea07950bb6bfe5  Rename for better clarity
6cf054c17fbd86e18172ee69d642e83d245e3ed1  Fix bug with missing replace -> use existing methods in Version for version compare.
aadd2e43d3395db5775eeae5a969601de9cc9f96  Revert version to 1.3.7 to not break build
7a5ba0bafbdedcfe836e5726c47c11f5469ae664  Fix param in test. Add Ignore to the 2 test classes handling the new historical data store feature.
224eaab1f210b8cfce1138b4e76619092faf774c  Fix param in test.
98877b4f5c011dab81871c2bc48bdd57e01a70f0  Revert changes at tests from prev version
32cf32c4abc197e0a91f25703df4c53d1247602d  Merge branch 'master_upstream' into chimp-reduce_initial_request_size
8f2e4caa3554acea1e0f5cfee1df2742bfe9b3ad  Change dev mode price
22fc97d3e919fa78a1078bc1aaa8a7670826b562  Remove test files from previous version
9c873a641e6a5d429925289383ea493167d93cb8  Add getPersisted method
3252a50975af655f033c349eb0b21a86a6cf4924  remove test files
a3ea54a83146a125f418674ebddc197bdd2badaa  remove test
2b615638171fa9d91dda8774283f7d0a84d3c6bf  Fix issues and refactor
996364e77e243fe5f62878dc0f4e0e7410c913ea  Cleanup
4115c0c2b660b27f2264f0492b0993eb3451e43c  Add data from non-splitStoreService as well
ad95019fd9a7ffcf78ec729de91d3cfaacad7328  Merge branch 'master_upstream' into chimp-reduce_initial_request_size
e680c4d576831b1b6bbe2d6dcfbc3c7c029b971e  Merge branch 'master_upstream' into chimp-reduce_initial_request_size
80bf8bf901a0ba4d42976e15021731bd0058c0ff  Merge branch 'master_upstream' into chimp-reduce_initial_request_size
3a4784e48e8d22dca62a48e93bf483b1e6b42a92  Merge branch 'chimp-reduce_initial_request_size' of https://github.com/chimp1984/bisq into chimp-reduce_initial_request_size
c482a1ba123228759d8273ff0e0a50e7fca4f62c  Cleanups, renaming, add comments
ba0dc1b89eb881e6bdec0a8b4935ed566635b8bb  Fix mock class to reflect real class better.
f132a35a53fc9919be3c2e8b30e6375d607d87f6  Fix incorrect test.
6eb1efb5a8da70fc51e21bebed8fcdae131aaff3  Remove todo, add system line break
ee725bc7c234b50c6d59a3fdb6e3741f3cb0aa9b  Add more logs
82c5b5efc159b836a0761a517d9bfdc4cefb24a5  Add isEmpty check
171be24d6ff700807ead2864e9dd42d2c8dd46dc  Do not call  super.readFromResources(postFix); as we do not have a not-versioned resource file anymore. Instead use readStore(); to create the live store.
bd38097e408326492798f26d62928342949c81a3  Add more logs
0feece65f179d573128cf75252607ac7adcb3aa2  Remove toProtoNetworkEnvelope call used only for debug logs
e402c8a02899f923cb7865a945ceb4afe23b284a  Only call pruneStore if we have created a new historical store from resources.
620dfca6541d08934e5efe802a3625f9e4749fbd  Fix wrong param order in log
90da37fa073adca2738f9b5b2e424a23ee8c66f1  Remove dev test data
9240eec557bcb133ec40fd7340fc1173b6fad758  Add more info in log
ab82286520c3382c86ad8d9e2e3470dea8577489  Rename SplitStoreService to HistoricalDataStoreService
b1cf73e855921e4cb39a3ac6ecccac232553a70d  Merge branch 'master_upstream' into new-persistence-model
9e7180d9030ddd6009df17b031c3615a696425f1  Fix merge issues
7601da237b68139e9818c2e9b59d0f8a875a45d3  Merge branch 'master_upstream' into new-persistence-model-merged
3707ad2bb6c4995f056fb9d3b89bdde86f714de4  Merged master
2ab7c031f82c160d8828bea243d0549c16e25796  Merged initial data request branch
616a4b6b0dbd21bb440c330cf828617e04417adb  Merge remote-tracking branch 'origin/new-persistence-model' into new-persistence-model-merged
698756735bad80d598d4ca2f5271ba7f7ef45e57  Fix merge issues

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/bisq-network/bisq/pull/4527/files/656896d650ac66ee1702530f913c88b09de7a1a0..698756735bad80d598d4ca2f5271ba7f7ef45e57
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20200930/fded1399/attachment-0001.html>