tor-reproducer issueshttps://code.briarproject.org/briar/tor-reproducer/-/issues2023-09-29T11:44:31Zhttps://code.briarproject.org/briar/tor-reproducer/-/issues/17Upgrade Tor to 0.4.7.152023-09-29T11:44:31ZakwizgranUpgrade Tor to 0.4.7.15Tor 0.4.7.15 contains a major bugfix for onion services.
https://forum.torproject.org/t/stable-release-0-4-7-15-and-0-4-8-6/9292
> This version contains an important fix for onion service regarding congestion control and its reliabil...Tor 0.4.7.15 contains a major bugfix for onion services.
https://forum.torproject.org/t/stable-release-0-4-7-15-and-0-4-8-6/9292
> This version contains an important fix for onion service regarding congestion control and its reliability. Apart from that, very minor bugfixes. We strongly recommend all onion service operators to update immediately.
>
> Major bugfixes (onion service):
> - Fix a reliability issue where services were expiring their
> introduction points every consensus update. This caused
> connectivity issues for clients caching the old descriptor and
> intro points. Bug reported and fixed by gitlab user
> @hyunsoo.kim676. Fixes bug 40858; bugfix on 0.4.7.5-alpha.akwizgranakwizgranhttps://code.briarproject.org/briar/tor-reproducer/-/issues/16Upgrade Tor to 0.4.7.142023-08-08T17:45:01ZakwizgranUpgrade Tor to 0.4.7.14Tor 0.4.7.14 includes a major bugfix related to onion services:
> Major bugfixes (vanguards):
> - Rotate to a new L2 vanguard whenever an existing one loses theStable or Fast flag. Previously, we would leave these relays in the L2 va...Tor 0.4.7.14 includes a major bugfix related to onion services:
> Major bugfixes (vanguards):
> - Rotate to a new L2 vanguard whenever an existing one loses theStable or Fast flag. Previously, we would leave these relays in the L2 vanguard list but never use them, and if all of our vanguards end up like this we wouldn't have any middle nodes left to choose from so we would fail to make onion-related circuits. Fixes bug 40805; bugfix on 0.4.7.1-alpha.https://code.briarproject.org/briar/tor-reproducer/-/issues/7Test binaries by tor-browser-build on different platforms2023-07-27T07:18:54ZNicoTest binaries by tor-browser-build on different platformsOne of the homework of https://code.briarproject.org/briar/tor-reproducer/-/issues/6 is testing that the binaries produced by [tor-browser-build](https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/master/README) are ...One of the homework of https://code.briarproject.org/briar/tor-reproducer/-/issues/6 is testing that the binaries produced by [tor-browser-build](https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/master/README) are actually working on those platforms.
### Projects to test
* tor
* obfs4
* snowflake
### Platforms to test
* torbrowser-linux-x86_64
* ~~torbrowser-linux-i686~~
* ~~torbrowser-windows-i686~~
* torbrowser-windows-x86_64
* torbrowser-osx-x86_64
Also these? How?
* torbrowser-android-armv7
* torbrowser-android-aarch64
* torbrowser-android-x86
* torbrowser-android-x86_64
### Test instructions
* Unpack the `tor.tar.gz` archive and find the _tor_/_tor.exe_ binary
* Try to run the _tor_ binary on the command-line, e.g., `./tor` or `.\tor.exe` in the respective directory
This should result in Tor starting up, looking something like this:
```
$ ./tor
Jan 04 23:26:33.688 [notice] Tor 0.4.7.2-alpha (git-4e921f5b8856af5c) running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1m, Zlib 1.2.11, Liblzma N/A, Libzstd N/A and Glibc 2.33 as libc.
Jan 04 23:26:33.688 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jan 04 23:26:33.688 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Jan 04 23:26:33.688 [notice] Configuration file "/var/tmp/dist/tor/etc/tor/torrc" not present, using reasonable defaults.
Jan 04 23:26:33.690 [notice] Opening Socks listener on 127.0.0.1:9050
Jan 04 23:26:33.690 [notice] Opened Socks listener connection (ready) on 127.0.0.1:9050
Jan 04 23:26:33.000 [notice] Bootstrapped 0% (starting): Starting
Jan 04 23:26:33.000 [notice] Starting with guard context "default"
Jan 04 23:26:34.000 [notice] Bootstrapped 5% (conn): Connecting to a relay
Jan 04 23:26:34.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
```
Please report back if this works for you and if not, what's the error that is shown.
Please also report back the output of `ldd tor`, if that works on your machine. If your OS supports it, please try to run the binaries like this: `LD_LIBRARY_PATH=. ./tor`https://code.briarproject.org/briar/tor-reproducer/-/issues/6Define requirements for revised tor-reproducer2023-07-13T11:00:15ZNicoDefine requirements for revised tor-reproducerInstead of trying to build all Tor related dependencies ourselves ([tor-reproducer](https://code.briarproject.org/briar/tor-reproducer), [go-reproducer](https://code.briarproject.org/briar/go-reproducer/)), there is the idea of using off...Instead of trying to build all Tor related dependencies ourselves ([tor-reproducer](https://code.briarproject.org/briar/tor-reproducer), [go-reproducer](https://code.briarproject.org/briar/go-reproducer/)), there is the idea of using official Tor tools to build (and reproduce) binaries. There are various motivations for this and also various possible solutions, so with this issue I want clear up the whole picture a little bit.
_(Terminology: Following Tor Project's [glossary](https://support.torproject.org/glossary/), when I write "Tor" I mean the whole ecosystem, including "little-t-tor", pluggable transports like obfs4 and snowflake and other stuff. little-t-tor is the `tor` binary Briar uses to make use of Tor's features.)_
### Motivations
* building Tor is hard, as underlying toolchains evolve and especially when trying to cross-compile for new architectures
* Additionally to _obfs4_ Briar might want to make use of [_snowflake_](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake), another [pluggable transport](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports) that uses WebRTC technology (the one for video calls in browsers) for bridging. Like _obfs4_ it's written in Go, so it would be reproduced in _go-reproducer_ with our current toolchain.
* _tor-reproducer_ here at Briar was never designed to build for that many different OS/arch combinations, slowing down the whole process a lot
### Possible solutions
Instead of figuring out build and compiler commandos and flags ourselves, we can make use of [the tor-browser-build](https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/master/README) infrastructure. This was first recommended [in this forum post](https://forum.torproject.net/t/are-there-any-official-reproducible-tor-obfs4proxy-binaries/831/3) and, e.g., compiling _tor_ for macOS (x86) is as easy as this:
```bash
./rbm/rbm build tor --target release --target torbrowser-osx-x86_64
```
For more information, see [this wiki page](https://code.briarproject.org/briar/briar-desktop-servers/-/wikis/Tor-builds).
Some comments about this:
* adding new OS/architecture combos becomes _super easy_, as long as we [use one of the officially supported combos](https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/master/doc/HACKING.txt#L46)
* really cool about this is that _rbm_ also compiles a lot of the underlying toolchain like, e.g., the cross-compilers and related stuff
* however, this also seems to result in much longer build times. It took 2 hours on a 4 core vps until it started to compile the actual tor dependencies. Before that, it built all the compiler dependencies like ninja and the cross-compiler. Afterwards, it was quick though, building all tor only took 10 to 20 minutes, if I see it correctly.
### Requirements
If we decide to switch to _rbm_ for building all Tor dependencies, we should make clear what we need now, what we might need in the future (so we don't slow down as much as we do currently with _tor-reproducer_) and how a revised [_all-tor-reproducer_](https://code.briarproject.org/briar-staging/all-tor-reproducer) (?) might differ from the current _tor-reproducer_.
The first difference is that rbm makes use of Docker containers itself to provide reproducibility. I imagine this means that we can't/don't want to use Docker ourselves in _all-tor-reproducer_, although this would mean that we need to [install the build dependencies](https://code.briarproject.org/briar/briar-desktop-servers/-/wikis/Tor-builds?version_id=4e9f5c731dde7a52ebee6f7743f1e79cf9c7b272) on the host machine. Does this break reproducibility? Does Docker inside Docker work?
Another point is versioning. With the current version of _all-tor-reproducer_ do we want to reproduce all current and previous versions of Tor stuff, or say that you need the right version of _all-tor-reproducer_ to reproduce older versions of Tor stuff? This is a point already unclear with _tor-reproducer_ that should definitively get cleared for _all-tor-reproducer_.
Related to that, we might want to build different versions than those listed in _tor-browser-build_. For example, with the current configuration tor version `0.4.7.2-alpha` would be built [according to this line](https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/6aaf06936e472c044c99d204244ef925827f8c15/projects/tor/config#L3), if I understood it correctly. We might be able to build other versions using a flag
```bash
./rbm/rbm build tor --target release --target torbrowser-linux-x86_64 --version 0.5.0
```
but we might want to make more/other changes to the build configurations. Do we want to for _tor-browser-build_ then?
### Additional homework
* [ ] actually try out binaries produced by _rbm_ on different platforms (https://code.briarproject.org/briar-staging/all-tor-reproducer/-/issues/1)
* [ ] link to this issue from the [Create a new build target to package tor daemon, pluggable transports and dependencies](https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40397) issue
Pinging @akwizgran @grote @sebkur @ialokimhttps://code.briarproject.org/briar/tor-reproducer/-/issues/4Build Tor for macOS/ARM (M1?)2023-07-13T10:59:34ZNicoBuild Tor for macOS/ARM (M1?)Similar to https://code.briarproject.org/briar/tor-reproducer/-/issues/3, https://code.briarproject.org/briar/tor-reproducer/-/issues/2 and https://code.briarproject.org/briar/tor-reproducer/-/merge_requests/13.Similar to https://code.briarproject.org/briar/tor-reproducer/-/issues/3, https://code.briarproject.org/briar/tor-reproducer/-/issues/2 and https://code.briarproject.org/briar/tor-reproducer/-/merge_requests/13.SebastianSebastianhttps://code.briarproject.org/briar/tor-reproducer/-/issues/3Build Tor for macOS/x862023-07-13T10:59:06ZNicoBuild Tor for macOS/x86Similar to https://code.briarproject.org/briar/tor-reproducer/-/merge_requests/13 and https://code.briarproject.org/briar/tor-reproducer/-/issues/2.Similar to https://code.briarproject.org/briar/tor-reproducer/-/merge_requests/13 and https://code.briarproject.org/briar/tor-reproducer/-/issues/2.SebastianSebastianhttps://code.briarproject.org/briar/tor-reproducer/-/issues/2Build Tor for Windows/x862022-02-15T15:25:42ZNicoBuild Tor for Windows/x86Similar to https://code.briarproject.org/briar/tor-reproducer/-/merge_requests/13.Similar to https://code.briarproject.org/briar/tor-reproducer/-/merge_requests/13.NicoNicohttps://code.briarproject.org/briar/tor-reproducer/-/issues/1Attempt at adding tor 0.3.5.3-alpha2019-02-05T20:39:46ZDan BallardAttempt at adding tor 0.3.5.3-alphaWe are attempting to make use of some new Tor features and follow the cutting edge of tor.
I tried adding to tor-versions.json
```
"0.3.5.3-alpha": {
"tor": "tor-0.3.5.3-alpha",
"libevent": "release-2.0.22-stable",
"openss...We are attempting to make use of some new Tor features and follow the cutting edge of tor.
I tried adding to tor-versions.json
```
"0.3.5.3-alpha": {
"tor": "tor-0.3.5.3-alpha",
"libevent": "release-2.0.22-stable",
"openssl": "OpenSSL_1_0_2p",
"xz": "v5.2.4",
"zlib": "v1.2.11",
"zstd": "v1.3.5",
"tor-android": "e822160b00aaed587965547eacad0566eec7de73",
"tor_android_repo_url": "https://github.com/n8fr8/tor-android",
"ndk": {
"url": "https://dl.google.com/android/repository/android-ndk-r18b-linux-x86_64.zip",
"revision": "18.1.5063045",
"sha256": "4f61cbe4bbf6406aa5ef2ae871def78010eed6271af72de83f8bd0b07a9fd3fd"
}
},
```
but when I ran `docker run briar/tor-reproducer:latest ./build-tor.py 0.3.5.3-alpha` (after building a new docker image)
it crashed with:
```
checking for arm-linux-androideabi-gcc... /opt/tor-reproducer/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc --sysroot=/opt/tor-reproducer/android-ndk/platforms/android-16/arch-arm
checking whether the C compiler works... no
configure: error: in `/opt/tor-reproducer/tor-android/external/xz':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** [xz/Makefile] Error 77
Makefile:200: recipe for target 'xz/Makefile' failed
make: Leaving directory '/opt/tor-reproducer/tor-android/external'
Building Tor tor-0.3.5.3-alpha
Downloading Android NDK...
Unpacking Android NDK...
Checking out tor: tor-0.3.5.3-alpha
Checking out libevent: release-2.0.22-stable
Checking out openssl: OpenSSL_1_0_2p
Checking out xz: v5.2.4
Checking out zlib: v1.2.11
Checking out zstd: v1.3.5
Sha256 hash of tor before zipping tor_linux-x86_64.zip: 3d27eb29ac2d609bb1894d59f09f7ad9d3246a12af85d29f2a595edfcf33fadf
Traceback (most recent call last):
File "./build-tor.py", line 276, in <module>
main()
File "./build-tor.py", line 30, in main
build_android()
File "./build-tor.py", line 148, in build_android
build_android_arch('tor_arm_pie.zip')
File "./build-tor.py", line 168, in build_android_arch
check_call(['make', '-C', 'external', 'clean', 'tor'], cwd=REPO_DIR)
File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-C', 'external', 'clean', 'tor']' returned non-zero exit status 2
```
I'll probably continue to poke at this but you have much more experience with it, if anything jumps out at you, I'd love to hear. thanks!