[bisq-network/projects] Migrate Bisq pricenodes from earn.com fee estimation API to our own self-hosted mempool fee estimation API service (#27)

wiz notifications at github.com
Thu Mar 12 05:38:14 UTC 2020


> _This is a Bisq Network project. Please familiarize yourself with the [project management process](https://bisq.wiki/Project_management)._

## Description

This is a project proposal to migrate Bisq pricenodes away from using the earn.com fee estimation API in favor of an open-source self-hosted mempool fee estimation API backend, that we can decentralize further by having all Bisq btcnode operators run.

## Rationale

Whenever the Bitcoin market sees major price action, trading volume suddenly spikes, which results in a huge rush of new transactions into the Bitcoin mempool, and "next block" fees rise very quickly:

<img width="1233" alt="Screen Shot 2020-03-12 at 14 09 39" src="https://user-images.githubusercontent.com/232186/76489251-239d2180-646b-11ea-8a0c-4c5292085cce.png">

This sudden rise in mempool fees causes a major issue in Bisq whereby any trades taken during the time when the mempool fees start to rise use far too low of a fee, and they do not get confirmed for several hours or even days later, until the mempool clears out. The cause of this issue is that Bisq currently uses Earn.com fee estimation API which is slow and not well maintained. The https://mempool.space backend uses realtime data from Bitcoin nodes, so it is extremely fast to return realtime fee estimates, compared to the Earn.com API which is always lagging behind:

<img width="468" alt="Screen Shot 2020-01-22 at 1 47 36" src="https://user-images.githubusercontent.com/232186/76489806-cc984c00-646c-11ea-98d2-293f2e4db010.png">

This issue was first reported by @ManfredKarrer in https://github.com/bisq-network/bisq/issues/1077 but at the time, a better mempool fee estimation API service did not exist. But now we have a great open-source and self-hosted solution, the [mempool project](https://github.com/mempool/mempool). Of course I am bias, since I am a maintainer/contributor to this project, but IMO it really is the best mempool project for Bitcoin and we have worked very hard making it so.

Recently during the past few days we have seen this issue reproduce again, and now users are complaining in support that their deposit TX have still not confirmed even after waiting for 12 hours. So therefore I propose that we pull the trigger on ditching Earn.com API once and for all.

## Criteria for delivery

After all Bisq pricenodes upgrade to the new code that will query our federated btcnode mempool fee estimation backend service, we can consider it delivered.

## Measures of success

After all Bisq nodes start receiving the self-hosted mempool fee data, and users no longer experience this issue due to sudden fee spikes, we can consider it a success.

## Risks

I've been running the mempool backend and hosting https://mempool.space for over 6 months, and it has never crashed, so I feel it is quite stable to run in production. However, we could keep Earn.com API as a backup data source to mitigate the risk of any issues with our own self-hosted API service.

## Tasks

- [ ] Modify the Bisq pricenode code to use the mempool fee estimation backend
- [ ] Have all Bisq btcnode operators are running the mempool fee estimation backend
- [ ] Have all Bisq pricenodes upgrade to the new code that will query our federated btcnode mempool fee estimation backend service

## Estimates

Dev: $2000 USD
Ops: $1000 USD


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/projects/issues/27
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20200311/4fe0169d/attachment-0001.html>


More information about the bisq-github mailing list