[bisq-network/bisq] Feasibility for replacing Bitcoinj with Bitcoin Core SPV (#1062)

Oscar Guindzberg notifications at github.com
Wed Dec 12 21:44:06 UTC 2018


I have investigated this subject in detail.

Super short summary: I found no good replacement for bitcoinj that we could use today. bip157/bip158 is where the industry is going, so it is worth keeping an eye on it.
Contributing bip157/bip158 support to bitcoinj would be an interesting option to evaluate if someone decides to invest time on this.

In general light wallet design faces a dilemma: Privacy (prevent others knowing what addresses are mine) and Trust (don’t trust third parties) vs. Low resource consumption (storage, cpu, bandwith).


An ideal bitcoinj replacement for bisq should have these features:
- Light - does not requires huge amounts of bandwidth/cpu/disk (ie no bitcoin core).
- Hierarchical Deterministic/BIP32 support (good example: bitcoinj).
- Tor Support.
- Stable, production tested and actively supported.
- Censorship resistant and Privacy - don’t allow the server owner or MITM to know what txs are mine. Worse example: Coinbase. Bad examples: known-to-be-broken bloom filters/bip37/bitcoinj, electrum,  copay. Good examples: bip157/bip158. Perfect case: bitcoin core.
- High-level API: e.g. bitcoinj's wallet class.
- Mobile support is not a requirement (bisq is just desktop)


Resources to understand bitcoin privacy conscious wallet start-of-the-art 
- Jonas schnelli status of wallets june 2018 https://www.youtube.com/watch?v=XORDEX-RrAI&t=57m15s  and https://bitcoin.jonasschnelli.ch/BOB_jonasschnelli_csatfow.pdf  and https://en.bitnovosti.com/2018/07/06/jonas-schnelli-current-and-future-state-of-wallets 
- Bitcoin as a Privacycoin: This Tech is Making Bitcoin More Private https://bitcoinmagazine.com/articles/bitcoin-privacycoin-tech-making-bitcoin-more-private 
- Full Node Level Privacy Even For Mobile Wallets? Transaction Filtered Full Block Downloading Wallet https://medium.com/@nopara73/full-node-level-privacy-even-for-mobile-wallets-transaction-filtered-full-block-downloading-wallet-16ef1847c21 
- Bitcoin Privacy Landscape In 2017 — Beyond Coin Mixing, General Guidelines And Research: https://hackernoon.com/bitcoin-privacy-landscape-in-2017-zero-to-hero-guidelines-and-research-a10d30f1e034
- Anonymous Bitcoin: https://medium.com/@nopara73/anonymous-bitcoin-1fae5d1e33b7 


Some ideas to make a light wallet:
- Download block headers until wallet creation date and full blocks from wallet creation date. 
- UTXO set commitments (root hash of UTXO set in the block header / coinbase tx).
    - You have to trust previous blocks follow consensus rules.
    - You can do full validation from that point on, if you download the rest of the blocks.
- Checkpoints: Have hardcoded checkpoints in the source code. Don’t download block headers before the checkpoints.
- Encrypt communication between the wallet and the server (BIP 151, electrum, etc).
- Pruning: remove old blocks (or data in general) as you already validated it and it is past the finality threshold (ie 100+ blocks).
- Client-side filtering: (BIP 157/158) 


Now, the projects I evaluated...


Bitcoinj 
- Benefits
    - Already implemented in Bisq.
    - Tor Support.
    - Stable, production tested.
    - Unconfirmed txs friendly.
    - High-level API: e.g. wallet class.
- problems
    - Bad performance for arbitrators. Once you get lots of keys and txs in your wallet things are very slow (probably similar performance to not using bloom filters at all).
    - Unmaintained: no release for 2,5 years
    - No segwit support.
    - Bloom filters problems
        - No privacy at all. 
            - Confidential information is sent unencrypted over the internet, so it is subject to MITM attacks.
            - User addresses could be easily obtained by every blockchain analysis company.
            - See https://groups.google.com/forum/#!msg/bitcoinj/Ys13qkTwcNg/9qxnhwnkeoIJ , https://eprint.iacr.org/2014/763.pdf, and  https://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj 
        - Bitcoinj Bloom filter “seed” generation bug removes more privacy.
        - Server can omit txs.
        - Easy to DoS a bitcoin core node that serves bloom filter, so probably most bitcoin core nodes will deactivate support sometime in the future.


Jonas schinelli’s “hybrid full block / SPV mode” on bitcoin core
- Modify bitcoin core to download block headers until wallet creation date and full blocks from wallet creation date. 
- https://bitcoincore.org/en/2017/03/31//prioritized-block-download-using-hybrid-spv/ 
- Downloading the “old” blocks is optional: Wallet could do it to avoid following a longer but consensus-breaking chain.
- Idea: add pruning support to require less storage.
- Benefits
    - Full node privacy.
    - It’s bitcoin core, so it avoids potential alternative client bugs.
    - Recover from existing wallet works faster and the user can start using the wallet (even if she is not sure the shown balance/tx belong to blocks with correct PoW but invalid consensus).
    - Also makes the process of new wallet creation much faster since just headers and no blocks will be downloaded
- Problems
    - Required resources are similar or equal to full bitcoin node.
    - Unfinished/Abandoned implementation.
- Unmerged PRs
    - Complete hybrid full block SPV mode https://github.com/bitcoin/bitcoin/pull/9483 (jan 2017)
        - New config param “spv”:  When on, core should sync the headers and then request blocks starting from wallet creation date.
        - Overlaps/complements #10794
    - Add simple light-client mode (RPC only) https://github.com/bitcoin/bitcoin/pull/10794 (jul 2017)
        - new config param -autorequestblocks=0. When on, sync the headers, don’t download blocks. 
        - External wallet (or internal wallet in the future) can request the download of specific blocks via RPC 
    - Deprecated PRs.
        - Introduce auxiliary block requests https://github.com/bitcoin/bitcoin/pull/9171 (nov 2016). Closed in favor of #10794
        - Add Hybrid full block SPV mode https://github.com/bitcoin/bitcoin/pull/9076 (nov 2016). Closed in favor of #9483


Neutrino / bip157 / bip158
- https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki 
- https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki 
- How it works
    - Bitcoin full nodes provide a filter for each block.
    - Filter is created deterministically by the full node based on the block. The filter is the same filter for every wallet.
    - Light wallets check client-side whether the filter shows the block might contain a relavant tx for them, if it does, they download the full block.
- Built to be used by lightning network nodes
- Benefits
    - Similar privacy to a full node: Peers nor MITM can find out what addressed belong to the light wallet.
    - Much less bandwith/storage/cpu required compared to a full node.
- Problems
    - Far from production ready technology.
    - No solution for unconfirmed txs (light wallets should download every broadcasted tx or don’t be aware of unconfirmed txs at all)
    - More bandwith required compared to bloom filters or electrum:
        - Filter size is 20k per block, much bigger than once in a while bloom filter updates.
        - It has to download full blocks if a relevant tx might be in the block, no SPV proof usage.
    - To investigate: Are there problems with the false/positive rate?
        - Is one-size-fits-all false/positive rate a problem?
        - Clients will end up downloading full blocks that don’t contain relevant txs to the wallet. The more addresses the wallet has, the more unnecessary blocks it will download. Is this a non-problem for lightning network nodes that have just a few keys to monitor (lots of keys added and discarded frequently) but a real problem for normal wallets that have to monitor a lot of derived-long-time-ago-but-still-valid keys.
    - How to make sure the filter is valid?
        - There is no filter commitment yet. A soft fork to add a filter hash to the coinbase tx / block header is very far away (maybe in 2 years).
        - Currently there is a (weak?) mechanism to find out if a peer is providing fraudulent filters (ie connecting to several peers, requesting information from all of them and eventually downloading full blocks)
- Server side implementations: 
    - btcd
        -  https://github.com/btcsuite/btcd 
        - full node written in golang
        - Reference server implementation for bip157 / bip158. 
    - bitcoin core
        - Not implemented yet
        - Issue asking for bip157 support https://github.com/bitcoin/bitcoin/issues/11863 
        - commit to start bip157 support in core by laanwj https://github.com/bitcoin/bitcoin/commit/ea263e1eb030530d3cfe76542c6322de3d1bfa31 
        - bip158 support(not bip157) https://github.com/bitcoin/bitcoin/pull/12254 
- Client side implementations
    - Neutrino
        - https://github.com/lightninglabs/neutrino  
        - Reference client implementation for bip157 / bip158.
        - golang
        - lower level api compared to bitcoinj
            - no class similar to bitcoinj wallet (no coin selection, method to get balances, key derivation)
        - alpha stage
            - docs say “experimental” / “not recommended to use it with real money”
            - almost no user documentation
        - Created for the needs of the LND client 
    - Wasabi includes a bip157 / bip158 client implementation (written in c#)
    - JVM
        - Acinq/eclair told they plan to port neutrino to scala/kotlin
        - Muun is slowly building a bitcoin lib in java and they plan to add bip157/158 support.
        - A Bisq contributor might add bip157 / bip158 client support to bitcoinj


Bitcoin scala implementations
- Neither of them is really mature nor feature complete
- https://github.com/ACINQ/bitcoin-lib
- https://github.com/bitcoin-s/bitcoin-s-core 


Samurai wallet
- https://samouraiwallet.com/
- This is not a wallet API, but a real wallet focused on privacy. 
- Built for mobile, not Bisq use case.
- Lots of interesting privacy features
    - Whirlpool
        - https://github.com/Samourai-Wallet/Whirlpool - port of Wasabi’s ZeroLink
    - Stonewall  (replaces BIP126)
        - Wallet creates txs that look like coinjoin txs so it confuses chain analyzers.
        - https://support.samourai.io/article/64-what-is-stonewall
        - https://gist.github.com/SamouraiDev/4ced85a29996dd56781e2bf319b93aaf
        - https://blog.samouraiwallet.com/post/173544815052/full-bech32-support-introducing-boltzmann-and 
    - boltzmann
        - Returns the entropy of the transaction which measures the linkability of inputs to outputs.
        - https://github.com/Samourai-Wallet/efficiency-score 
        - https://github.com/Samourai-Wallet/boltzmann 
        - https://gist.github.com/LaurentMT/e758767ca4038ac40aaf
        - https://gist.github.com/LaurentMT/d361bca6dc52868573a2
        - https://gist.github.com/LaurentMT/e8644d5bc903f02613c6  
    - Creates change output to match destination address type (p2pkh, p2sh, p2wpkh)
    - Ricochet
        - Creates 5 txs making the tx dependency chain deeper. That outsmarts very basic chain analysis technics that just look 5 txs before.
    - PayNym/BIP47
        - Share a kind of xpub, others will add the “xpub” to their contact list and every time they send you a tx, it will be a new address just they and you know (requires a 1st tx setup). Others won’t be able to find out that tx is from that xpub
    - Uses its own server to get blockchain data (I guess they used blockchain.info in the past). It has the same privacy problems electrum, copay and mycelium have.
    - You can specify your own bitcoin core node, then uses RPC (but maybe that is just to relay txs)


Wasabi Wallet 
- https://www.wasabiwallet.io/
- This is not a wallet API, but a real wallet focused on privacy. 
- https://github.com/zkSNACKs/WalletWasabi 
- https://zksnacks.com/
- Features
    - Bip 157/ 158 support
    - ZeroLink wallet fungibility framework: https://github.com/nopara73/ZeroLink/
    - Chaumian CoinJoin
- Backend: 
    - Custom server that provides bip157/158 filters. I am Not sure how the wallet syncs the block headers. Not sure if there is only 1 server or anyone can run a server or how the wallet could defend against a malicious server 
    - https://github.com/zkSNACKs/WalletWasabi/tree
- c# language
- author: Nopara73 (Adam Ficsor)


Wallets using centralized servers
- Most prominent examples: Copay, Mycelium, electrum.
- Benefits
    - Almost instant new wallet creation and seed recovery.
    - Lowest bandwidth/storage/cpu alternative available.
    - Stable, production tested and actively supported.
    - Encrypted Client-server communication.
- Problems
    - The server knows exactly my addresses.
    - The server can not inform users of incoming txs.
    - Users should trust the company server or servers run by random people they don’t know.
    - Personal Electrum Server could, in theory, be run by users, but it requires a local bitcoin core which kills the almost instant on and low resource consumption benefits.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/bisq/issues/1062#issuecomment-446755367
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20181212/dc8d3018/attachment-0001.html>


More information about the bisq-github mailing list