[bisq-network/bisq] Gradle witness for 'jtorctrl' not updated for Tor 0.4.5 (Issue #5807)

[cd2357]

TLDR:

This happens because jitpack is serving one of two different versions of the jtorctl jar, depending on the caller's IP.

I came to this based on:
- Own tests. I downloaded the jtorctl jar using VPNs to different locations and I sometimes got one version of the file, sometimes the other.
- The fact that current GitHub builds are successful (i.e. the jtorctl retrieved by GH matches the hash in our `gradle-witness`), while at the same time the build fails for some people like OP (i.e. jitpack serves them a jtorctl jar which doesn't match the `gradle-witness` hash).

Both versions of the jar are attached. Change extension from zip to jar:

[jtorctl-1.5-389.zip](https://github.com/bisq-network/bisq/files/7500423/jtorctl-1.5-389.zip) -- hash `389d61b1b5a85eb2f23c582c3913ede49f80c9f2b553e4762382c836270e57e5`
[jtorctl-1.5-646.zip](https://github.com/bisq-network/bisq/files/7500424/jtorctl-1.5-646.zip) -- hash `6465e0b22b921344a065a6e82a5390ab2f9dac5ab293c38bd8a76baddead6492`

----

### Root-cause analysis

What I suspect happened is this:

- Beginning-Mid October: jitpack had an issue and lost some build artefacts (jtorctl)
  - this resulted in Bisq builds failing with:
  `Could not find artifact com.github.JesusMcCloud:jtorctl:jar:1.5 in jitpack.io (https://jitpack.io)`
- 17 Oct: jitpack issue partially fixed, but artefacts still lost
  - A (new) build of jitpack was triggered
    - Same commit, same code version (as per [build log](https://jitpack.io/com/github/JesusMcCloud/jtorctl/1.5/build.log))
    - hash `6465e0b22b921344a065a6e82a5390ab2f9dac5ab293c38bd8a76baddead6492`

- Turns out Jitpack uses Cloudflare to distribute its artefacts
  - Cloudflare mirrors the different artefacts in different regions and serves them to the caller from the closest-reachable CDN location
  - Depending on the SLAs jitpack has with Cloudflare, new artefact versions can take a while to propagate
    - This is where we are now
      - New version of the jitpack artefact is available (hash `6465..`)
      - This newer jtorctl is distributed to some Cloudflare nodes, not to others (yet?)
      - Depending by which Cloudflare server the download request is served, you can get one file or the other


### Q: If the artefact was simply re-built by jitpack, why does it have a different hash?

@ripcurlx pointed out it could be because the new build used a different JDK version (1.8.0_212 vs. 1.8.0_252).

I checked and this could indeed be the case. I built jtorctl from source using those specific JDK versions (from [here](https://wiki.openjdk.java.net/display/jdk8u/Main)). Indeed, the resulting jars had different hashes. They were also different than the two hashes from jitpack.

### Q: Ok so Jitpack now serves a different jtorctl. How do we know this is entirely due to a different JDK? Can we trust the new jar to not contain code modifications?

I would say yes we can trust it.

I used http://java-decompiler.github.io to decompile both jars and compare the sources.

Using `diff --recursive jtorctl-1.5-389.jar.src jtorctl-1.5-646.jar.src` I can confirm the sources are the same.


### Q: What now? What's the next step?

The only option I can think of is wait until Cloudflare propagates the new jar to all its regional servers.

In other words, wait until downloading the jar from jitpack reliably results in the same jar, no matter from which location its downloaded.

Until then, we can alternatively work with either of the two jtorctl hashes in `gradle-witness`, depending which one is pulled. This means manually updating the hash in `gradle-witness` locally, in case the build complains.

(Not the best solution, but can't think of a better one.)

---

There are other open questions to consider for medium / long-term:
- Why did jitpack lose the artefacts?
  - Temporary outage maybe, but even after they got back online the previous jar is lost? No backups?
    - Will they lose artefacts again?
- Why is jitpack using such an old JDK even for newer builds?
  - `Java version: 1.8.0_252, vendor: Private Build` ( as per [jtorctl build log](https://jitpack.io/com/github/JesusMcCloud/jtorctl/1.5/build.log))
- What alternatives are there, in case jitpack / maven central / etc go down, or experience similar issues?
  - One idea is we host the dependencies ourselves in a Git LFS repo. There are already [some](https://github.com/alexvasilkov/GradleGitDependenciesPlugin) possible [solutions](https://alexvasilkov.com/gradle-git) for this.

---

What do you guys think?

Do you see another way to tackle this at the moment?

cc @bisq-network/bisq-devs 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/bisq-network/bisq/issues/5807#issuecomment-963641322
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bisq.network/pipermail/bisq-github/attachments/20211108/b20345a5/attachment.htm>