[bisq-network/proposals] Integrate HTTP API to bisq-network (#22)

Fri Jun 1 10:07:41 UTC 2018

I have yet to fully catch up on this proposal. Over the last weeks, I've written up and had a number of conversations with @blabno and @mrosseel about doing a proper proposal for getting the HTTP API integrated, and I want to reference some of those discussions quickly here:

 - https://github.com/bisq-network/bisq-desktop/pull/1535#pullrequestreview-118781492
 - https://bisq.slack.com/archives/C75ABAU9J/p1525963260000213
 - https://bisq.slack.com/archives/C701A4NNS/p1527509529000009
 - https://bisq.slack.com/archives/C701A4NNS/p1527510818000073

I have other priorities I'm focused on right now, and @ManfredKarrer I'm glad to see that you're diving in here to get stuff done, but I want to state frankly that I'm worried about this work. We've had a hard time aligning on what should be done and how it should be done, and I don't want to see a lot of further implementation work on the integration itself until we can get high-level agreement about what users are going to experience and what the minimum set of requirements for getting the HTTP API out of incubation and into production are. That's why I've been pushing for a proposal, but the proposal as written above jumps pretty quickly into low-level implementation details. What I think everyone needs to get crystal clear on is:

1. What are users going to experience when this integration is complete? Literally, what will users have to do to enable and interact with the Bisq HTTP API? How will they learn about this new functionality? Where and how will it be documented?
2. Where will the code for the HTTP API live, both from a repository perspective and from a packaging perspective?
3. What are the set of criteria for getting the HTTP API out of incubation and into production? I'm talking here about code review, automated testing, style issues, and so on. I've written about some of this in the links above, but it was all very quickly written, is not necessarily complete, and certainly has not been presented a way worthy of being called a proposal.
4. Precisely how and where will the integration take place, i.e. at the level of code? Which classes will change, which will need to be refactored, etc.

I'm concerned that the proposal as written above focuses mostly on (2) and (4) above, and doesn't address (1) and (3). And really, I don't think that (4) needs to be spelled out in too much detail in a proposal. Those are code changes, and that's what pull requests are for. What's important is that we get aligned on the high-level stuff, the UX stuff, the the security and code review stuff, and the overall quality control stuff here. Actually doing the code changes to integrate should be relatively trivial after that, so please let's back up and deal with first things first. Somebody needs to take the time to write up a proper proposal that deals with all this stuff, so that we don't keep going around in circles about it. As I wrote in https://docs.bisq.network/proposals.html:

> In general, **good proposals take time to research and write.** Every minute you spend clearly and logically articulating your proposal is a minute that you save other contributors in understanding it.

This is why I asked @blabno and @mrosseel to take the time to write this proposal early in the process. In the meantime, there has been a lot of work done based on my initial comments about what should probably go into that proposal, e.g. truncating the bisq-api repository history, repackaging stuff to `network.bisq.*`, writing documentation in the form of step-by-step getting started guides, etc. It's all great that this work has been done, but it's all been done based on one-off conversations, most of which have been in Slack, where stuff is ephemeral, not discoverable, and eventually disappears fully. No one is clear about the exact criteria for the HTTP API coming out of incubation and into production, and it results in a lot of wasted time and energy. It's just no way to manage the integration of a large, brand-new and user-facing software component into the Bisq stack.

In our most recent conversation about this, @blabno made it clear that he would in fact write such a proposal, but it's not clear to me, @ManfredKarrer, whether your proposal here was something you basically did on your own, without awareness of that conversation that @blabno, or if this proposal was in fact intended to be the same, all-inclusive proposal that I've been pushing for all along.

@ManfredKarrer, you wrote above:

> One concern of @blabno is that we get too many different combinations of possible combinations how to run the app.

I am not concerned about there being too many modes / combinations.

I believe that for the first step toward integration, there should be an option in bisq-desktop to enable the HTTP API service, and that's it. No headless mode/option yet. Take things one step at a time. We know it will require further refactoring to get to a place where bisq-core can be run standalone with the HTTP API enabled and no ties whatsoever to bisq-desktop. Until that is possible, bisq-desktop should not try to "fake" any headless mode. When it is possible, people will literally be able to run bisq-core from the command line like a proper unix daemon, and they'll also be able to run bisq-desktop in 'headless server mode', but doing so will just be a very lightweight and convenient shortcut to running bisq-core headlessly in the way I just described above.

@ManfredKarrer, are you familiar with the getting started guide that @blabno has been writing at https://github.com/bisq-network/bisq-docs/pull/54? Here is the latest deployment preview of it: https://deploy-preview-54--bisq-docs.netlify.com/http-api-monitor-offers.html.

Notice how the doc currently tells the user to build mrosseel/bisq-api from source and to start everything up via `mvn exec:java`. That's because the doc is designed to take the reader step by step from zero to writing their own dead-simple but fully working Bisq offer monitoring bot. And currently, without the HTTP API being integrated into bisq-core or bisq-desktop in any way, this workaround with `mvn exec:java` is how things have to be done. One question that we should be trying to answer from a high level in this proposal is how _should_ it be done? When this integration is complete, what steps should the user have to take in that guide to enable the HTTP API? I believe the answer is that they should have to download and install the latest Bisq Desktop client, and instead of double-clicking the Bisq app icon, they should have to start it from the command line with, e.g. a `--http-server` option. That is already pretty onerous, though, so it would be good in addition for the user to be able to specify this same option in the `~/.config/bisq.properties` file (which we now support in bisq-pricenode, and as an aside, I think we should probably rename to `bisq.conf` because .properties is a Java convention, and [.conf](https://en.wikipedia.org/wiki/Configuration_file#UNIX/Linux) is a widely-accepted unix convention, e.g. `bitcoin.conf`). That way, the user can create the config file, add a line that reads `http-server=true` and then double-click the Bisq app icon as usual (this is the exact same user experience as Bitcoin Core and its config file).

For reasons I mentioned above, there doesn't need to be any other option for running headlessly, etc at this point. The question is what is the simplest possible initial integration that we can do that makes it possible for the interact programmatically with their Bisq node via the new HTTP API.

We also need to align, of course, on the exact implementation details of the integration: whether bisq-http-api is a separate repository, or whether it gets integrated into bisq-core; what the package naming should look like; whether we should consider anything at all with gRPC yet; and so forth. All of these details should be fleshed out in this proposal, and IMO preferably before we do much more coding / implementation work on anything. We can save so much time and effort by writing this stuff down and aligning on it first.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/proposals/issues/22#issuecomment-393836171
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20180601/9ccf4cc4/attachment-0002.html>