<p>I have investigated this subject in detail.</p>
<p>Super short summary: I found no good replacement for bitcoinj that we could use today. bip157/bip158 is where the industry is going, so it is worth keeping an eye on it.<br>
Contributing bip157/bip158 support to bitcoinj would be an interesting option to evaluate if someone decides to invest time on this.</p>
<p>In general light wallet design faces a dilemma: Privacy (prevent others knowing what addresses are mine) and Trust (don’t trust third parties) vs. Low resource consumption (storage, cpu, bandwith).</p>
<p>An ideal bitcoinj replacement for bisq should have these features:</p>
<ul>
<li>Light - does not requires huge amounts of bandwidth/cpu/disk (ie no bitcoin core).</li>
<li>Hierarchical Deterministic/BIP32 support (good example: bitcoinj).</li>
<li>Tor Support.</li>
<li>Stable, production tested and actively supported.</li>
<li>Censorship resistant and Privacy - don’t allow the server owner or MITM to know what txs are mine. Worse example: Coinbase. Bad examples: known-to-be-broken bloom filters/bip37/bitcoinj, electrum, copay. Good examples: bip157/bip158. Perfect case: bitcoin core.</li>
<li>High-level API: e.g. bitcoinj's wallet class.</li>
<li>Mobile support is not a requirement (bisq is just desktop)</li>
</ul>
<p>Resources to understand bitcoin privacy conscious wallet start-of-the-art</p>
<ul>
<li>Jonas schnelli status of wallets june 2018 <a href="https://www.youtube.com/watch?v=XORDEX-RrAI&t=57m15s" rel="nofollow">https://www.youtube.com/watch?v=XORDEX-RrAI&t=57m15s</a> and <a href="https://bitcoin.jonasschnelli.ch/BOB_jonasschnelli_csatfow.pdf" rel="nofollow">https://bitcoin.jonasschnelli.ch/BOB_jonasschnelli_csatfow.pdf</a> and <a href="https://en.bitnovosti.com/2018/07/06/jonas-schnelli-current-and-future-state-of-wallets" rel="nofollow">https://en.bitnovosti.com/2018/07/06/jonas-schnelli-current-and-future-state-of-wallets</a></li>
<li>Bitcoin as a Privacycoin: This Tech is Making Bitcoin More Private <a href="https://bitcoinmagazine.com/articles/bitcoin-privacycoin-tech-making-bitcoin-more-private" rel="nofollow">https://bitcoinmagazine.com/articles/bitcoin-privacycoin-tech-making-bitcoin-more-private</a></li>
<li>Full Node Level Privacy Even For Mobile Wallets? Transaction Filtered Full Block Downloading Wallet <a href="https://medium.com/@nopara73/full-node-level-privacy-even-for-mobile-wallets-transaction-filtered-full-block-downloading-wallet-16ef1847c21" rel="nofollow">https://medium.com/@nopara73/full-node-level-privacy-even-for-mobile-wallets-transaction-filtered-full-block-downloading-wallet-16ef1847c21</a></li>
<li>Bitcoin Privacy Landscape In 2017 — Beyond Coin Mixing, General Guidelines And Research: <a href="https://hackernoon.com/bitcoin-privacy-landscape-in-2017-zero-to-hero-guidelines-and-research-a10d30f1e034" rel="nofollow">https://hackernoon.com/bitcoin-privacy-landscape-in-2017-zero-to-hero-guidelines-and-research-a10d30f1e034</a></li>
<li>Anonymous Bitcoin: <a href="https://medium.com/@nopara73/anonymous-bitcoin-1fae5d1e33b7" rel="nofollow">https://medium.com/@nopara73/anonymous-bitcoin-1fae5d1e33b7</a></li>
</ul>
<p>Some ideas to make a light wallet:</p>
<ul>
<li>Download block headers until wallet creation date and full blocks from wallet creation date.</li>
<li>UTXO set commitments (root hash of UTXO set in the block header / coinbase tx).
<ul>
<li>You have to trust previous blocks follow consensus rules.</li>
<li>You can do full validation from that point on, if you download the rest of the blocks.</li>
</ul>
</li>
<li>Checkpoints: Have hardcoded checkpoints in the source code. Don’t download block headers before the checkpoints.</li>
<li>Encrypt communication between the wallet and the server (BIP 151, electrum, etc).</li>
<li>Pruning: remove old blocks (or data in general) as you already validated it and it is past the finality threshold (ie 100+ blocks).</li>
<li>Client-side filtering: (BIP 157/158)</li>
</ul>
<p>Now, the projects I evaluated...</p>
<p>Bitcoinj</p>
<ul>
<li>Benefits
<ul>
<li>Already implemented in Bisq.</li>
<li>Tor Support.</li>
<li>Stable, production tested.</li>
<li>Unconfirmed txs friendly.</li>
<li>High-level API: e.g. wallet class.</li>
</ul>
</li>
<li>problems
<ul>
<li>Bad performance for arbitrators. Once you get lots of keys and txs in your wallet things are very slow (probably similar performance to not using bloom filters at all).</li>
<li>Unmaintained: no release for 2,5 years</li>
<li>No segwit support.</li>
<li>Bloom filters problems
<ul>
<li>No privacy at all.
<ul>
<li>Confidential information is sent unencrypted over the internet, so it is subject to MITM attacks.</li>
<li>User addresses could be easily obtained by every blockchain analysis company.</li>
<li>See <a href="https://groups.google.com/forum/#!msg/bitcoinj/Ys13qkTwcNg/9qxnhwnkeoIJ" rel="nofollow">https://groups.google.com/forum/#!msg/bitcoinj/Ys13qkTwcNg/9qxnhwnkeoIJ</a> , <a href="https://eprint.iacr.org/2014/763.pdf" rel="nofollow">https://eprint.iacr.org/2014/763.pdf</a>, and <a href="https://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj" rel="nofollow">https://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj</a></li>
</ul>
</li>
<li>Bitcoinj Bloom filter “seed” generation bug removes more privacy.</li>
<li>Server can omit txs.</li>
<li>Easy to DoS a bitcoin core node that serves bloom filter, so probably most bitcoin core nodes will deactivate support sometime in the future.</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>Jonas schinelli’s “hybrid full block / SPV mode” on bitcoin core</p>
<ul>
<li>Modify bitcoin core to download block headers until wallet creation date and full blocks from wallet creation date.</li>
<li><a href="https://bitcoincore.org/en/2017/03/31//prioritized-block-download-using-hybrid-spv/" rel="nofollow">https://bitcoincore.org/en/2017/03/31//prioritized-block-download-using-hybrid-spv/</a></li>
<li>Downloading the “old” blocks is optional: Wallet could do it to avoid following a longer but consensus-breaking chain.</li>
<li>Idea: add pruning support to require less storage.</li>
<li>Benefits
<ul>
<li>Full node privacy.</li>
<li>It’s bitcoin core, so it avoids potential alternative client bugs.</li>
<li>Recover from existing wallet works faster and the user can start using the wallet (even if she is not sure the shown balance/tx belong to blocks with correct PoW but invalid consensus).</li>
<li>Also makes the process of new wallet creation much faster since just headers and no blocks will be downloaded</li>
</ul>
</li>
<li>Problems
<ul>
<li>Required resources are similar or equal to full bitcoin node.</li>
<li>Unfinished/Abandoned implementation.</li>
</ul>
</li>
<li>Unmerged PRs
<ul>
<li>Complete hybrid full block SPV mode <a class="issue-link js-issue-link" data-error-text="Failed to load issue title" data-id="199236636" data-permission-text="Issue title is private" data-url="https://github.com/bitcoin/bitcoin/issues/9483" data-hovercard-type="pull_request" data-hovercard-url="/bitcoin/bitcoin/pull/9483/hovercard" href="https://github.com/bitcoin/bitcoin/pull/9483">bitcoin/bitcoin#9483</a> (jan 2017)
<ul>
<li>New config param “spv”: When on, core should sync the headers and then request blocks starting from wallet creation date.</li>
<li>Overlaps/complements #10794</li>
</ul>
</li>
<li>Add simple light-client mode (RPC only) <a class="issue-link js-issue-link" data-error-text="Failed to load issue title" data-id="242079899" data-permission-text="Issue title is private" data-url="https://github.com/bitcoin/bitcoin/issues/10794" data-hovercard-type="pull_request" data-hovercard-url="/bitcoin/bitcoin/pull/10794/hovercard" href="https://github.com/bitcoin/bitcoin/pull/10794">bitcoin/bitcoin#10794</a> (jul 2017)
<ul>
<li>new config param -autorequestblocks=0. When on, sync the headers, don’t download blocks.</li>
<li>External wallet (or internal wallet in the future) can request the download of specific blocks via RPC</li>
</ul>
</li>
<li>Deprecated PRs.
<ul>
<li>Introduce auxiliary block requests <a class="issue-link js-issue-link" data-error-text="Failed to load issue title" data-id="189643527" data-permission-text="Issue title is private" data-url="https://github.com/bitcoin/bitcoin/issues/9171" data-hovercard-type="pull_request" data-hovercard-url="/bitcoin/bitcoin/pull/9171/hovercard" href="https://github.com/bitcoin/bitcoin/pull/9171">bitcoin/bitcoin#9171</a> (nov 2016). Closed in favor of #10794</li>
<li>Add Hybrid full block SPV mode <a class="issue-link js-issue-link" data-error-text="Failed to load issue title" data-id="187175299" data-permission-text="Issue title is private" data-url="https://github.com/bitcoin/bitcoin/issues/9076" data-hovercard-type="pull_request" data-hovercard-url="/bitcoin/bitcoin/pull/9076/hovercard" href="https://github.com/bitcoin/bitcoin/pull/9076">bitcoin/bitcoin#9076</a> (nov 2016). Closed in favor of #9483</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>Neutrino / bip157 / bip158</p>
<ul>
<li><a href="https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki">https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki</a></li>
<li><a href="https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki">https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki</a></li>
<li>How it works
<ul>
<li>Bitcoin full nodes provide a filter for each block.</li>
<li>Filter is created deterministically by the full node based on the block. The filter is the same filter for every wallet.</li>
<li>Light wallets check client-side whether the filter shows the block might contain a relavant tx for them, if it does, they download the full block.</li>
</ul>
</li>
<li>Built to be used by lightning network nodes</li>
<li>Benefits
<ul>
<li>Similar privacy to a full node: Peers nor MITM can find out what addressed belong to the light wallet.</li>
<li>Much less bandwith/storage/cpu required compared to a full node.</li>
</ul>
</li>
<li>Problems
<ul>
<li>Far from production ready technology.</li>
<li>No solution for unconfirmed txs (light wallets should download every broadcasted tx or don’t be aware of unconfirmed txs at all)</li>
<li>More bandwith required compared to bloom filters or electrum:
<ul>
<li>Filter size is 20k per block, much bigger than once in a while bloom filter updates.</li>
<li>It has to download full blocks if a relevant tx might be in the block, no SPV proof usage.</li>
</ul>
</li>
<li>To investigate: Are there problems with the false/positive rate?
<ul>
<li>Is one-size-fits-all false/positive rate a problem?</li>
<li>Clients will end up downloading full blocks that don’t contain relevant txs to the wallet. The more addresses the wallet has, the more unnecessary blocks it will download. Is this a non-problem for lightning network nodes that have just a few keys to monitor (lots of keys added and discarded frequently) but a real problem for normal wallets that have to monitor a lot of derived-long-time-ago-but-still-valid keys.</li>
</ul>
</li>
<li>How to make sure the filter is valid?
<ul>
<li>There is no filter commitment yet. A soft fork to add a filter hash to the coinbase tx / block header is very far away (maybe in 2 years).</li>
<li>Currently there is a (weak?) mechanism to find out if a peer is providing fraudulent filters (ie connecting to several peers, requesting information from all of them and eventually downloading full blocks)</li>
</ul>
</li>
</ul>
</li>
<li>Server side implementations:
<ul>
<li>btcd
<ul>
<li><a href="https://github.com/btcsuite/btcd">https://github.com/btcsuite/btcd</a></li>
<li>full node written in golang</li>
<li>Reference server implementation for bip157 / bip158.</li>
</ul>
</li>
<li>bitcoin core
<ul>
<li>Not implemented yet</li>
<li>Issue asking for bip157 support <a class="issue-link js-issue-link" data-error-text="Failed to load issue title" data-id="281025324" data-permission-text="Issue title is private" data-url="https://github.com/bitcoin/bitcoin/issues/11863" data-hovercard-type="issue" data-hovercard-url="/bitcoin/bitcoin/issues/11863/hovercard" href="https://github.com/bitcoin/bitcoin/issues/11863">bitcoin/bitcoin#11863</a></li>
<li>commit to start bip157 support in core by laanwj <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/bitcoin/bitcoin/commit/ea263e1eb030530d3cfe76542c6322de3d1bfa31/hovercard" href="https://github.com/bitcoin/bitcoin/commit/ea263e1eb030530d3cfe76542c6322de3d1bfa31">bitcoin/bitcoin@<tt>ea263e1</tt></a></li>
<li>bip158 support(not bip157) <a class="issue-link js-issue-link" data-error-text="Failed to load issue title" data-id="291055625" data-permission-text="Issue title is private" data-url="https://github.com/bitcoin/bitcoin/issues/12254" data-hovercard-type="pull_request" data-hovercard-url="/bitcoin/bitcoin/pull/12254/hovercard" href="https://github.com/bitcoin/bitcoin/pull/12254">bitcoin/bitcoin#12254</a></li>
</ul>
</li>
</ul>
</li>
<li>Client side implementations
<ul>
<li>Neutrino
<ul>
<li><a href="https://github.com/lightninglabs/neutrino">https://github.com/lightninglabs/neutrino</a></li>
<li>Reference client implementation for bip157 / bip158.</li>
<li>golang</li>
<li>lower level api compared to bitcoinj
<ul>
<li>no class similar to bitcoinj wallet (no coin selection, method to get balances, key derivation)</li>
</ul>
</li>
<li>alpha stage
<ul>
<li>docs say “experimental” / “not recommended to use it with real money”</li>
<li>almost no user documentation</li>
</ul>
</li>
<li>Created for the needs of the LND client</li>
</ul>
</li>
<li>Wasabi includes a bip157 / bip158 client implementation (written in c#)</li>
<li>JVM
<ul>
<li>Acinq/eclair told they plan to port neutrino to scala/kotlin</li>
<li>Muun is slowly building a bitcoin lib in java and they plan to add bip157/158 support.</li>
<li>A Bisq contributor might add bip157 / bip158 client support to bitcoinj</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>Bitcoin scala implementations</p>
<ul>
<li>Neither of them is really mature nor feature complete</li>
<li><a href="https://github.com/ACINQ/bitcoin-lib">https://github.com/ACINQ/bitcoin-lib</a></li>
<li><a href="https://github.com/bitcoin-s/bitcoin-s-core">https://github.com/bitcoin-s/bitcoin-s-core</a></li>
</ul>
<p>Samurai wallet</p>
<ul>
<li><a href="https://samouraiwallet.com/" rel="nofollow">https://samouraiwallet.com/</a></li>
<li>This is not a wallet API, but a real wallet focused on privacy.</li>
<li>Built for mobile, not Bisq use case.</li>
<li>Lots of interesting privacy features
<ul>
<li>Whirlpool
<ul>
<li><a href="https://github.com/Samourai-Wallet/Whirlpool">https://github.com/Samourai-Wallet/Whirlpool</a> - port of Wasabi’s ZeroLink</li>
</ul>
</li>
<li>Stonewall (replaces BIP126)
<ul>
<li>Wallet creates txs that look like coinjoin txs so it confuses chain analyzers.</li>
<li><a href="https://support.samourai.io/article/64-what-is-stonewall" rel="nofollow">https://support.samourai.io/article/64-what-is-stonewall</a></li>
<li><a href="https://gist.github.com/SamouraiDev/4ced85a29996dd56781e2bf319b93aaf">https://gist.github.com/SamouraiDev/4ced85a29996dd56781e2bf319b93aaf</a></li>
<li><a href="https://blog.samouraiwallet.com/post/173544815052/full-bech32-support-introducing-boltzmann-and" rel="nofollow">https://blog.samouraiwallet.com/post/173544815052/full-bech32-support-introducing-boltzmann-and</a></li>
</ul>
</li>
<li>boltzmann
<ul>
<li>Returns the entropy of the transaction which measures the linkability of inputs to outputs.</li>
<li><a href="https://github.com/Samourai-Wallet/efficiency-score">https://github.com/Samourai-Wallet/efficiency-score</a></li>
<li><a href="https://github.com/Samourai-Wallet/boltzmann">https://github.com/Samourai-Wallet/boltzmann</a></li>
<li><a href="https://gist.github.com/LaurentMT/e758767ca4038ac40aaf">https://gist.github.com/LaurentMT/e758767ca4038ac40aaf</a></li>
<li><a href="https://gist.github.com/LaurentMT/d361bca6dc52868573a2">https://gist.github.com/LaurentMT/d361bca6dc52868573a2</a></li>
<li><a href="https://gist.github.com/LaurentMT/e8644d5bc903f02613c6">https://gist.github.com/LaurentMT/e8644d5bc903f02613c6</a></li>
</ul>
</li>
<li>Creates change output to match destination address type (p2pkh, p2sh, p2wpkh)</li>
<li>Ricochet
<ul>
<li>Creates 5 txs making the tx dependency chain deeper. That outsmarts very basic chain analysis technics that just look 5 txs before.</li>
</ul>
</li>
<li>PayNym/BIP47
<ul>
<li>Share a kind of xpub, others will add the “xpub” to their contact list and every time they send you a tx, it will be a new address just they and you know (requires a 1st tx setup). Others won’t be able to find out that tx is from that xpub</li>
</ul>
</li>
<li>Uses its own server to get blockchain data (I guess they used blockchain.info in the past). It has the same privacy problems electrum, copay and mycelium have.</li>
<li>You can specify your own bitcoin core node, then uses RPC (but maybe that is just to relay txs)</li>
</ul>
</li>
</ul>
<p>Wasabi Wallet</p>
<ul>
<li><a href="https://www.wasabiwallet.io/" rel="nofollow">https://www.wasabiwallet.io/</a></li>
<li>This is not a wallet API, but a real wallet focused on privacy.</li>
<li><a href="https://github.com/zkSNACKs/WalletWasabi">https://github.com/zkSNACKs/WalletWasabi</a></li>
<li><a href="https://zksnacks.com/" rel="nofollow">https://zksnacks.com/</a></li>
<li>Features
<ul>
<li>Bip 157/ 158 support</li>
<li>ZeroLink wallet fungibility framework: <a href="https://github.com/nopara73/ZeroLink/">https://github.com/nopara73/ZeroLink/</a></li>
<li>Chaumian CoinJoin</li>
</ul>
</li>
<li>Backend:
<ul>
<li>Custom server that provides bip157/158 filters. I am Not sure how the wallet syncs the block headers. Not sure if there is only 1 server or anyone can run a server or how the wallet could defend against a malicious server</li>
<li><a href="https://github.com/zkSNACKs/WalletWasabi/tree">https://github.com/zkSNACKs/WalletWasabi/tree</a></li>
</ul>
</li>
<li>c# language</li>
<li>author: Nopara73 (Adam Ficsor)</li>
</ul>
<p>Wallets using centralized servers</p>
<ul>
<li>Most prominent examples: Copay, Mycelium, electrum.</li>
<li>Benefits
<ul>
<li>Almost instant new wallet creation and seed recovery.</li>
<li>Lowest bandwidth/storage/cpu alternative available.</li>
<li>Stable, production tested and actively supported.</li>
<li>Encrypted Client-server communication.</li>
</ul>
</li>
<li>Problems
<ul>
<li>The server knows exactly my addresses.</li>
<li>The server can not inform users of incoming txs.</li>
<li>Users should trust the company server or servers run by random people they don’t know.</li>
<li>Personal Electrum Server could, in theory, be run by users, but it requires a local bitcoin core which kills the almost instant on and low resource consumption benefits.</li>
</ul>
</li>
</ul>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/bisq-network/bisq/issues/1062#issuecomment-446755367">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AkpZtkLFwyFhthvNf0jQxHWi34zazxSiks5u4XimgaJpZM4Q1Z7Y">mute the thread</a>.<img src="https://github.com/notifications/beacon/AkpZtq4ADIKpVm-cMAXQ5ZamZiEgxoBoks5u4XimgaJpZM4Q1Z7Y.gif" height="1" width="1" alt="" /></p>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/bisq-network/bisq","title":"bisq-network/bisq","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/bisq-network/bisq"}},"updates":{"snippets":[{"icon":"PERSON","message":"@oscarguindzberg in #1062: I have investigated this subject in detail.\r\n\r\nSuper short summary: I found no good replacement for bitcoinj that we could use today. bip157/bip158 is where the industry is going, so it is worth keeping an eye on it.\r\nContributing bip157/bip158 support to bitcoinj would be an interesting option to evaluate if someone decides to invest time on this.\r\n\r\nIn general light wallet design faces a dilemma: Privacy (prevent others knowing what addresses are mine) and Trust (don’t trust third parties) vs. Low resource consumption (storage, cpu, bandwith).\r\n\r\n\r\nAn ideal bitcoinj replacement for bisq should have these features:\r\n- Light - does not requires huge amounts of bandwidth/cpu/disk (ie no bitcoin core).\r\n- Hierarchical Deterministic/BIP32 support (good example: bitcoinj).\r\n- Tor Support.\r\n- Stable, production tested and actively supported.\r\n- Censorship resistant and Privacy - don’t allow the server owner or MITM to know what txs are mine. Worse example: Coinbase. Bad examples: known-to-be-broken bloom filters/bip37/bitcoinj, electrum, copay. Good examples: bip157/bip158. Perfect case: bitcoin core.\r\n- High-level API: e.g. bitcoinj's wallet class.\r\n- Mobile support is not a requirement (bisq is just desktop)\r\n\r\n\r\nResources to understand bitcoin privacy conscious wallet start-of-the-art \r\n- Jonas schnelli status of wallets june 2018 https://www.youtube.com/watch?v=XORDEX-RrAI\u0026t=57m15s and https://bitcoin.jonasschnelli.ch/BOB_jonasschnelli_csatfow.pdf and https://en.bitnovosti.com/2018/07/06/jonas-schnelli-current-and-future-state-of-wallets \r\n- Bitcoin as a Privacycoin: This Tech is Making Bitcoin More Private https://bitcoinmagazine.com/articles/bitcoin-privacycoin-tech-making-bitcoin-more-private \r\n- Full Node Level Privacy Even For Mobile Wallets? Transaction Filtered Full Block Downloading Wallet https://medium.com/@nopara73/full-node-level-privacy-even-for-mobile-wallets-transaction-filtered-full-block-downloading-wallet-16ef1847c21 \r\n- Bitcoin Privacy Landscape In 2017 — Beyond Coin Mixing, General Guidelines And Research: https://hackernoon.com/bitcoin-privacy-landscape-in-2017-zero-to-hero-guidelines-and-research-a10d30f1e034\r\n- Anonymous Bitcoin: https://medium.com/@nopara73/anonymous-bitcoin-1fae5d1e33b7 \r\n\r\n\r\nSome ideas to make a light wallet:\r\n- Download block headers until wallet creation date and full blocks from wallet creation date. \r\n- UTXO set commitments (root hash of UTXO set in the block header / coinbase tx).\r\n - You have to trust previous blocks follow consensus rules.\r\n - You can do full validation from that point on, if you download the rest of the blocks.\r\n- Checkpoints: Have hardcoded checkpoints in the source code. Don’t download block headers before the checkpoints.\r\n- Encrypt communication between the wallet and the server (BIP 151, electrum, etc).\r\n- Pruning: remove old blocks (or data in general) as you already validated it and it is past the finality threshold (ie 100+ blocks).\r\n- Client-side filtering: (BIP 157/158) \r\n\r\n\r\nNow, the projects I evaluated...\r\n\r\n\r\nBitcoinj \r\n- Benefits\r\n - Already implemented in Bisq.\r\n - Tor Support.\r\n - Stable, production tested.\r\n - Unconfirmed txs friendly.\r\n - High-level API: e.g. wallet class.\r\n- problems\r\n - Bad performance for arbitrators. Once you get lots of keys and txs in your wallet things are very slow (probably similar performance to not using bloom filters at all).\r\n - Unmaintained: no release for 2,5 years\r\n - No segwit support.\r\n - Bloom filters problems\r\n - No privacy at all. \r\n - Confidential information is sent unencrypted over the internet, so it is subject to MITM attacks.\r\n - User addresses could be easily obtained by every blockchain analysis company.\r\n - See https://groups.google.com/forum/#!msg/bitcoinj/Ys13qkTwcNg/9qxnhwnkeoIJ , https://eprint.iacr.org/2014/763.pdf, and https://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj \r\n - Bitcoinj Bloom filter “seed” generation bug removes more privacy.\r\n - Server can omit txs.\r\n - Easy to DoS a bitcoin core node that serves bloom filter, so probably most bitcoin core nodes will deactivate support sometime in the future.\r\n\r\n\r\nJonas schinelli’s “hybrid full block / SPV mode” on bitcoin core\r\n- Modify bitcoin core to download block headers until wallet creation date and full blocks from wallet creation date. \r\n- https://bitcoincore.org/en/2017/03/31//prioritized-block-download-using-hybrid-spv/ \r\n- Downloading the “old” blocks is optional: Wallet could do it to avoid following a longer but consensus-breaking chain.\r\n- Idea: add pruning support to require less storage.\r\n- Benefits\r\n - Full node privacy.\r\n - It’s bitcoin core, so it avoids potential alternative client bugs.\r\n - Recover from existing wallet works faster and the user can start using the wallet (even if she is not sure the shown balance/tx belong to blocks with correct PoW but invalid consensus).\r\n - Also makes the process of new wallet creation much faster since just headers and no blocks will be downloaded\r\n- Problems\r\n - Required resources are similar or equal to full bitcoin node.\r\n - Unfinished/Abandoned implementation.\r\n- Unmerged PRs\r\n - Complete hybrid full block SPV mode https://github.com/bitcoin/bitcoin/pull/9483 (jan 2017)\r\n - New config param “spv”: When on, core should sync the headers and then request blocks starting from wallet creation date.\r\n - Overlaps/complements #10794\r\n - Add simple light-client mode (RPC only) https://github.com/bitcoin/bitcoin/pull/10794 (jul 2017)\r\n - new config param -autorequestblocks=0. When on, sync the headers, don’t download blocks. \r\n - External wallet (or internal wallet in the future) can request the download of specific blocks via RPC \r\n - Deprecated PRs.\r\n - Introduce auxiliary block requests https://github.com/bitcoin/bitcoin/pull/9171 (nov 2016). Closed in favor of #10794\r\n - Add Hybrid full block SPV mode https://github.com/bitcoin/bitcoin/pull/9076 (nov 2016). Closed in favor of #9483\r\n\r\n\r\nNeutrino / bip157 / bip158\r\n- https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki \r\n- https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki \r\n- How it works\r\n - Bitcoin full nodes provide a filter for each block.\r\n - Filter is created deterministically by the full node based on the block. The filter is the same filter for every wallet.\r\n - Light wallets check client-side whether the filter shows the block might contain a relavant tx for them, if it does, they download the full block.\r\n- Built to be used by lightning network nodes\r\n- Benefits\r\n - Similar privacy to a full node: Peers nor MITM can find out what addressed belong to the light wallet.\r\n - Much less bandwith/storage/cpu required compared to a full node.\r\n- Problems\r\n - Far from production ready technology.\r\n - No solution for unconfirmed txs (light wallets should download every broadcasted tx or don’t be aware of unconfirmed txs at all)\r\n - More bandwith required compared to bloom filters or electrum:\r\n - Filter size is 20k per block, much bigger than once in a while bloom filter updates.\r\n - It has to download full blocks if a relevant tx might be in the block, no SPV proof usage.\r\n - To investigate: Are there problems with the false/positive rate?\r\n - Is one-size-fits-all false/positive rate a problem?\r\n - Clients will end up downloading full blocks that don’t contain relevant txs to the wallet. The more addresses the wallet has, the more unnecessary blocks it will download. Is this a non-problem for lightning network nodes that have just a few keys to monitor (lots of keys added and discarded frequently) but a real problem for normal wallets that have to monitor a lot of derived-long-time-ago-but-still-valid keys.\r\n - How to make sure the filter is valid?\r\n - There is no filter commitment yet. A soft fork to add a filter hash to the coinbase tx / block header is very far away (maybe in 2 years).\r\n - Currently there is a (weak?) mechanism to find out if a peer is providing fraudulent filters (ie connecting to several peers, requesting information from all of them and eventually downloading full blocks)\r\n- Server side implementations: \r\n - btcd\r\n - https://github.com/btcsuite/btcd \r\n - full node written in golang\r\n - Reference server implementation for bip157 / bip158. \r\n - bitcoin core\r\n - Not implemented yet\r\n - Issue asking for bip157 support https://github.com/bitcoin/bitcoin/issues/11863 \r\n - commit to start bip157 support in core by laanwj https://github.com/bitcoin/bitcoin/commit/ea263e1eb030530d3cfe76542c6322de3d1bfa31 \r\n - bip158 support(not bip157) https://github.com/bitcoin/bitcoin/pull/12254 \r\n- Client side implementations\r\n - Neutrino\r\n - https://github.com/lightninglabs/neutrino \r\n - Reference client implementation for bip157 / bip158.\r\n - golang\r\n - lower level api compared to bitcoinj\r\n - no class similar to bitcoinj wallet (no coin selection, method to get balances, key derivation)\r\n - alpha stage\r\n - docs say “experimental” / “not recommended to use it with real money”\r\n - almost no user documentation\r\n - Created for the needs of the LND client \r\n - Wasabi includes a bip157 / bip158 client implementation (written in c#)\r\n - JVM\r\n - Acinq/eclair told they plan to port neutrino to scala/kotlin\r\n - Muun is slowly building a bitcoin lib in java and they plan to add bip157/158 support.\r\n - A Bisq contributor might add bip157 / bip158 client support to bitcoinj\r\n\r\n\r\nBitcoin scala implementations\r\n- Neither of them is really mature nor feature complete\r\n- https://github.com/ACINQ/bitcoin-lib\r\n- https://github.com/bitcoin-s/bitcoin-s-core \r\n\r\n\r\nSamurai wallet\r\n- https://samouraiwallet.com/\r\n- This is not a wallet API, but a real wallet focused on privacy. \r\n- Built for mobile, not Bisq use case.\r\n- Lots of interesting privacy features\r\n - Whirlpool\r\n - https://github.com/Samourai-Wallet/Whirlpool - port of Wasabi’s ZeroLink\r\n - Stonewall (replaces BIP126)\r\n - Wallet creates txs that look like coinjoin txs so it confuses chain analyzers.\r\n - https://support.samourai.io/article/64-what-is-stonewall\r\n - https://gist.github.com/SamouraiDev/4ced85a29996dd56781e2bf319b93aaf\r\n - https://blog.samouraiwallet.com/post/173544815052/full-bech32-support-introducing-boltzmann-and \r\n - boltzmann\r\n - Returns the entropy of the transaction which measures the linkability of inputs to outputs.\r\n - https://github.com/Samourai-Wallet/efficiency-score \r\n - https://github.com/Samourai-Wallet/boltzmann \r\n - https://gist.github.com/LaurentMT/e758767ca4038ac40aaf\r\n - https://gist.github.com/LaurentMT/d361bca6dc52868573a2\r\n - https://gist.github.com/LaurentMT/e8644d5bc903f02613c6 \r\n - Creates change output to match destination address type (p2pkh, p2sh, p2wpkh)\r\n - Ricochet\r\n - Creates 5 txs making the tx dependency chain deeper. That outsmarts very basic chain analysis technics that just look 5 txs before.\r\n - PayNym/BIP47\r\n - Share a kind of xpub, others will add the “xpub” to their contact list and every time they send you a tx, it will be a new address just they and you know (requires a 1st tx setup). Others won’t be able to find out that tx is from that xpub\r\n - Uses its own server to get blockchain data (I guess they used blockchain.info in the past). It has the same privacy problems electrum, copay and mycelium have.\r\n - You can specify your own bitcoin core node, then uses RPC (but maybe that is just to relay txs)\r\n\r\n\r\nWasabi Wallet \r\n- https://www.wasabiwallet.io/\r\n- This is not a wallet API, but a real wallet focused on privacy. \r\n- https://github.com/zkSNACKs/WalletWasabi \r\n- https://zksnacks.com/\r\n- Features\r\n - Bip 157/ 158 support\r\n - ZeroLink wallet fungibility framework: https://github.com/nopara73/ZeroLink/\r\n - Chaumian CoinJoin\r\n- Backend: \r\n - Custom server that provides bip157/158 filters. I am Not sure how the wallet syncs the block headers. Not sure if there is only 1 server or anyone can run a server or how the wallet could defend against a malicious server \r\n - https://github.com/zkSNACKs/WalletWasabi/tree\r\n- c# language\r\n- author: Nopara73 (Adam Ficsor)\r\n\r\n\r\nWallets using centralized servers\r\n- Most prominent examples: Copay, Mycelium, electrum.\r\n- Benefits\r\n - Almost instant new wallet creation and seed recovery.\r\n - Lowest bandwidth/storage/cpu alternative available.\r\n - Stable, production tested and actively supported.\r\n - Encrypted Client-server communication.\r\n- Problems\r\n - The server knows exactly my addresses.\r\n - The server can not inform users of incoming txs.\r\n - Users should trust the company server or servers run by random people they don’t know.\r\n - Personal Electrum Server could, in theory, be run by users, but it requires a local bitcoin core which kills the almost instant on and low resource consumption benefits."}],"action":{"name":"View Issue","url":"https://github.com/bisq-network/bisq/issues/1062#issuecomment-446755367"}}}</script>
<script type="application/ld+json">[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/bisq-network/bisq/issues/1062#issuecomment-446755367",
"url": "https://github.com/bisq-network/bisq/issues/1062#issuecomment-446755367",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>