[bisq-network/bisq-markets] Re-implement Markets API service with pre-caching proxy + risq backend (#20)

[wiz]

> I noticed the graphs on specific markets pages of the website can still take a while to load

Yes, pre-caching and caching are both currently disabled, so it's almost as slow as the current backend. The pre-caching will be enabled in a 2nd PR later which uses a cron job every 1 minute to pre-fetch the data into memcache, which makes it super fast as per the PR description since the most popular API endpoint responses is served directly from the cloud.

> I wonder if opcache extension was somehow not enabled when the site was moved to new server. If so, that would degrade performance immensely as the filecache drops back to reading json file for each request.

Thanks @dan-da for pointing this out - yes, we migrated everything to the new server by copying over all the packages and configuration and filesystem as exactly as possible. I've only taken over the project very recently but from what I heard when @emzy and others were maintaining this server, the major issues started after the website was modified to include the fancy new charts and got worse over time as the API got more and more popular. Perhaps some newly added queries for the charts weren't being cached properly. I also heard you had proposed some performance enhancements but for whatever reason they weren't implemented - I'm honestly not sure which is why I made a new thing to solve the instability issues.

> @wiz @bodymindarts Am I right that this PR is the pre-caching proxy and the routing of the requests to the modified risq application? Shouldn't the risq application not also be part of the PR? Otherwise someone couldn't take over and deploy the markets server herself.

Well the risq app is AGPL licensed so you could press the "fork" button on GitHub to copy it over to the Bisq organization, and then we could add it as a submodule to this repo?

> it seems to be that the new markets server has more accurate data

Yeah, I think @bodymindarts already answered the technical questions but my understanding is that the Bisq nodes are "eventually consistent" and our new model of running it with a dedicated seednode seems to have found some trades that are missing from the current backend.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/bisq-markets/pull/20#issuecomment-552497356
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20191111/ce2a3c6f/attachment-0001.html>