[bisq-network/bisq] API CLI output formatting that is unix-idiomatic and machine-friendly (#4348)

[dmos62]

The new gRPC API CLI client will be outputting various types of data and we want the output format, just like the rest of the CLI client, to be unix-idiomatic. By unix-idomatic, I mean following the unix philosophy, which, most notably, includes interoperating well with other unix standard tools. The idea is for the output logic to be simple and to be able to achieve complex use cases by piping to other simple and standard tools.

## Table format

Currently, the output of `getoffers` call looks more or less like this (below output is a bit dated; some things like the datetime format has changed, but currently I don't have access to my dev workspace, so using this):

```
Buy/Sell  Price in USD for 1 BTC  BTC(min - max)            USD(min - max)  Payment Method  Creation Date             ID
BUY                   9,485.9500  0.03250000 - 0.06250000        308 - 593  Cash Deposit    Jun 21, 2020 10:49:46 PM  nywdxnx-e77596af-c7e9-4df0-b2cd-86a83eb717d1-135
BUY                   9,675.6690  0.07860000                           761  Zelle           Jun 22, 2020 10:27:40 AM  CCQZCI-fa41db95-f002-4ebe-83b0-c2be2d5573a6-134
```

Above output is not easy to destructure automatically. For example, using spaces to distinguish columns is not possible:

```
#!/bin/bash

function drop_header {
	tail -n +2
}

function squeeze_spaces {
	tr -s " "
}

function space_delimited_table {
	column -t -s " "
}

cat current-getoffers-output.txt \
| drop_header \
| squeeze_spaces \
| space_delimited_table
```
Outputs a garbled table:
```
BUY  9,485.9500  0.03250000  -    0.06250000  308  -    593   Cash      Deposit  Jun                                              21,  2020  10:49:46  PM  nywdxnx-e77596af-c7e9-4df0-b2cd-86a83eb717d1-135
BUY  9,675.6690  0.07860000  761  Zelle       Jun  22,  2020  10:27:40  AM       CCQZCI-fa41db95-f002-4ebe-83b0-c2be2d5573a6-134
```

The motivation behind current formatting is human friendliness, but, after giving it more thought, true developer friendliness requires easy programmatic output interpretation.

We can use a simpler columnar format with explicit delimiters (only whitespace between columns has been changed):

```
Buy/Sell|Price in USD for 1 BTC|BTC(min - max)|USD(min - max)|Payment Method|Creation Date|ID
BUY|9,485.9500|0.03250000 - 0.06250000|308 - 593|Cash Deposit|Jun 21, 2020 10:49:46 PM|nywdxnx-e77596af-c7e9-4df0-b2cd-86a83eb717d1-135    
BUY|9,675.6690|0.07860000|761|Zelle|Jun 22, 2020 10:27:40 AM|CCQZCI-fa41db95-f002-4ebe-83b0-c2be2d5573a6-134
```

To format into a human friendly table:

```
#!/bin/bash

function custom_delimiter_table {
	column -t -s "|"
}

cat proposed-getoffers-output.txt \
| custom_delimiter_table
```
Output:
```
Buy/Sell  Price in USD for 1 BTC  BTC(min - max)           USD(min - max)  Payment Method  Creation Date             ID
BUY       9,485.9500              0.03250000 - 0.06250000  308 - 593       Cash Deposit    Jun 21, 2020 10:49:46 PM  nywdxnx-e77596af-c7e9-4df0-b2cd-86a83eb717d1-135    
BUY       9,675.6690              0.07860000               761             Zelle           Jun 22, 2020 10:27:40 AM  CCQZCI-fa41db95-f002-4ebe-83b0-c2be2d5573a6-134
```

This requires the data to not include the delimiter, which might seem fiddly, but can be solved by allowing the user to specify a custom delimiter (e.g. `|||||`). Unless data in question might contain malicious input, sanitization shouldn't be necessary.

## Column complexity

Another concern is that some columns are difficult to machine parse, because they're optimized for space efficiency and general human readability.

For example, the column titled `BTC(min - max)` can either contain the offer quantity or the offer quantity range, which means it can contain two different types of data: a hyphen-deliminated tuple of floats (`0.03250000 - 0.06250000`) or a single float (`0.03250000`).

The column `BTC(min - max)` can be unpacked into two columns `minimum BTC quantity` and `maximum BTC quantity`. That way we have the same information, but the complexity is reduced (both columns are just floats). When the quantity is a range, the new columns will have different values; otherwise, they will be the same. Same goes for `UTC(min -max)`.

## More data in rows

There's some information, like the offer pair (e.g. BTC and USD), that's embedded in the header, but not in the data rows. That makes it impossible to mix multiple currency pairs in the same table. If pair information was in the rows, you could query `getoffers` for multiple pairs at once, or greatly simplify subscribing to push updates (one subscription for any number of pairs); in short, a table that stores all information in the rows is easier to work with.

Instead of `Price in USD for 1 BTC|BTC(min - max)|USD(min - max)`, we could have:

```Price|Base currency|Base currency quantity(min - max)|Quote currency|Quote currency quantity(min - max)```

or with the above proposed change

```Price|Base currency|Minimum base currency quantity|Maximum base currency quantity|Quote currency|Minimum quote currency quantity|Maximum quote currency quantity```

Some notes:
- some abbreviations might be useful;
- also, notice that `Price in USD for 1 BTC` implies it is the value of base currency being quoted against the quote currency, so I shortened it to `Price`;
- see this article for definitions of currency pairs, base currency and quote currency: https://www.investopedia.com/terms/c/currencypair.asp ;
- notice that the last example is considerably more verbose than the current table format, but it is much easier to machine parse (all cells are simple types with simple semantics); ease and simplicity of setting up machine parsing is the priority.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/bisq/issues/4348
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20200701/14d246f7/attachment.html>