[bisq-network/roles] Seednode Operator (#15)

[Devin Bileck]

## Cycle 3 report

### Summary
Running 2 seed nodes.
- fl3mmribyxgrv63c.onion
- 3f3cu2yw7u457ztq.onion

### Issues Encountered
**Issue 1:**
On July 4th, fl3mmribyxgrv63c was delivering outdated blocks as head blocks. See the [monitor](https://monitor.bisq.network/d/iVpBgXXmk/dao-sync-status?refresh=5m&orgId=1&panelId=16&fullscreen&from=1562259600000&to=1562263200000) for more details. The issue appeared to be caused by failing to receive notifications from bitcoind (see log snippet below). No idea why at this point. I ended up restarting the seed node and it resoved the issue.
```
bisq.core.dao.node.full.RpcException: NotificationHandlerException(super=com.neemre.btcdcli4j.daemon.NotificationHandlerException: Error #1004004: The operation failed due to an unknown IO exception., error=Errors(code=1004004, message=The operation failed due to an unknown IO exception.), code=1004004)
    at bisq.core.dao.node.full.RpcService.lambda$setup$0(RpcService.java:138)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: NotificationHandlerException(super=com.neemre.btcdcli4j.daemon.NotificationHandlerException: Error #1004004: The operation failed due to an unknown IO exception., error=Errors(code=1004004, message=The operation failed due to an unknown IO exception.), code=1004004)
    at com.neemre.btcdcli4j.daemon.notification.worker.NotificationWorker.call(NotificationWorker.java:64)
    at com.neemre.btcdcli4j.daemon.notification.worker.NotificationWorker.call(NotificationWorker.java:22)
    at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
    at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
    at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
    ... 3 more
Caused by: java.net.SocketException: Connection reset
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
    at java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
    at com.neemre.btcdcli4j.daemon.notification.worker.NotificationWorker.call(NotificationWorker.java:46)
    ... 7 more
```

**Issue 2:**
@alexej996 was encountering issues with his seed node. While looking at the logs, it seemed to be hitting the memory limit and restarting:
```
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
We are over our memory limit (1200) and trigger a restart. usedMemory: 1275 MB. freeMemory: 295 MB
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
```

And frequently going over 80%:
```
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
We are over 80% of our memory limit (960) and call the GC. usedMemory: 1156 MB. freeMemory: 414 MB
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
```

There is a PR that was merged recently that may or may not help with memory usage: https://github.com/bisq-network/bisq/pull/2501

If it doesn't help, we may require further improvements or potentially increase the memory limit for now.

### Maintenance Performed
No maintenance performed this month.

### Expenses Incurred
Expenses incurred for the month (USD):
- 2 x $20 for server (2 CPU, 4 GB RAM)
- 2 x $30 for storage (300 GB)

**Total:** $100

/cc [bisq-network/compensation#309](https://github.com/bisq-network/compensation/issues/309)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/roles/issues/15#issuecomment-509364974
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20190708/ef746fab/attachment-0001.html>