Define requirements for revised tor-reproducer
Instead of trying to build all Tor related dependencies ourselves (tor-reproducer, go-reproducer), there is the idea of using official Tor tools to build (and reproduce) binaries. There are various motivations for this and also various possible solutions, so with this issue I want clear up the whole picture a little bit.
(Terminology: Following Tor Project's glossary, when I write "Tor" I mean the whole ecosystem, including "little-t-tor", pluggable transports like obfs4 and snowflake and other stuff. little-t-tor is the tor
binary Briar uses to make use of Tor's features.)
Motivations
- building Tor is hard, as underlying toolchains evolve and especially when trying to cross-compile for new architectures
- Additionally to obfs4 Briar might want to make use of snowflake, another pluggable transport that uses WebRTC technology (the one for video calls in browsers) for bridging. Like obfs4 it's written in Go, so it would be reproduced in go-reproducer with our current toolchain.
- tor-reproducer here at Briar was never designed to build for that many different OS/arch combinations, slowing down the whole process a lot
Possible solutions
Instead of figuring out build and compiler commandos and flags ourselves, we can make use of the tor-browser-build infrastructure. This was first recommended in this forum post and, e.g., compiling tor for macOS (x86) is as easy as this:
./rbm/rbm build tor --target release --target torbrowser-osx-x86_64
For more information, see this wiki page.
Some comments about this:
- adding new OS/architecture combos becomes super easy, as long as we use one of the officially supported combos
- really cool about this is that rbm also compiles a lot of the underlying toolchain like, e.g., the cross-compilers and related stuff
- however, this also seems to result in much longer build times. It took 2 hours on a 4 core vps until it started to compile the actual tor dependencies. Before that, it built all the compiler dependencies like ninja and the cross-compiler. Afterwards, it was quick though, building all tor only took 10 to 20 minutes, if I see it correctly.
Requirements
If we decide to switch to rbm for building all Tor dependencies, we should make clear what we need now, what we might need in the future (so we don't slow down as much as we do currently with tor-reproducer) and how a revised all-tor-reproducer (?) might differ from the current tor-reproducer.
The first difference is that rbm makes use of Docker containers itself to provide reproducibility. I imagine this means that we can't/don't want to use Docker ourselves in all-tor-reproducer, although this would mean that we need to install the build dependencies on the host machine. Does this break reproducibility? Does Docker inside Docker work?
Another point is versioning. With the current version of all-tor-reproducer do we want to reproduce all current and previous versions of Tor stuff, or say that you need the right version of all-tor-reproducer to reproduce older versions of Tor stuff? This is a point already unclear with tor-reproducer that should definitively get cleared for all-tor-reproducer.
Related to that, we might want to build different versions than those listed in tor-browser-build. For example, with the current configuration tor version 0.4.7.2-alpha
would be built according to this line, if I understood it correctly. We might be able to build other versions using a flag
./rbm/rbm build tor --target release --target torbrowser-linux-x86_64 --version 0.5.0
but we might want to make more/other changes to the build configurations. Do we want to for tor-browser-build then?
Additional homework
-
actually try out binaries produced by rbm on different platforms (https://code.briarproject.org/briar-staging/all-tor-reproducer/-/issues/1) -
link to this issue from the Create a new build target to package tor daemon, pluggable transports and dependencies issue
Pinging @akwizgran @grote @sebkur @ialokim