Implementation of PT3 Hopping
This is an implementation of the PT3 Hopping PT
It relies on openvpn as a session layer, with obfs4 as a circumvention
inner layer. It allows for a different set of arguments to switch openvpn to
using udp with the --float parameter. That then requires obfsvpn to translate
udp datagram packets from openvpn to the obfs4 tcp streams.
I ended up being fairly opinionated about the docker/docker-compose configuration. Except for the minor inconvenience of having to supply an --env-file to the docker-compose commands I think it works reasonably well.
There are a bunch of things that I think we should talk about and that I was unsure of:
- The way that the openvpn-server and obfsvpn containers have been separated. I chose a more convenient approach for this experimental sandbox, but I suspect there may be real world concerns that conflict
- Logging. I tried to more or less keep the previous patterns, but I didn't take any time to try and do any improvements
- Error handling/cancellation. I've taken a few shortcuts that are maybe less than elegant but seem to work for now
- Some openvpn server/client options. You'll notice I've made a few changes around there and they're probably worth discussing.
Now for the fun part! Baseline non-hopping iperf test:
$ make integration
$ docker-compose exec openvpn-server iperf3 -s --bind-dev tun0
$ docker-compose exec client iperf3 -c 10.8.0.1 --bind-dev tun0
Connecting to host 10.8.0.1, port 5201
[ 5] local 10.8.0.6 port 51398 connected to 10.8.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 96.0 MBytes 805 Mbits/sec 68 599 KBytes
[ 5] 1.00-2.00 sec 101 MBytes 848 Mbits/sec 8 545 KBytes
[ 5] 2.00-3.00 sec 88.0 MBytes 739 Mbits/sec 6 463 KBytes
[ 5] 3.00-4.00 sec 80.4 MBytes 674 Mbits/sec 0 566 KBytes
[ 5] 4.00-5.00 sec 101 MBytes 851 Mbits/sec 2 518 KBytes
[ 5] 5.00-6.00 sec 107 MBytes 896 Mbits/sec 4 470 KBytes
[ 5] 6.00-7.00 sec 101 MBytes 847 Mbits/sec 0 595 KBytes
[ 5] 7.00-8.00 sec 91.6 MBytes 768 Mbits/sec 3 528 KBytes
[ 5] 8.00-9.00 sec 77.6 MBytes 651 Mbits/sec 0 619 KBytes
[ 5] 9.00-10.00 sec 76.2 MBytes 639 Mbits/sec 13 532 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 920 MBytes 772 Mbits/sec 104 sender
[ 5] 0.00-10.01 sec 918 MBytes 770 Mbits/sec receiver
iperf Done.Now with hopping at a minimum of 5 seconds and a hop jitter of 5 seconds (so randomly between 5-10 seconds):
$ make integration-hopping
$ docker-compose --env-file ./.env.hopping exec openvpn-server iperf3 -s --bind-dev tun0
❯ docker-compose --env-file .env.hopping exec client iperf3 -c 10.8.0.1 --bind-dev tun0 -t 60
Connecting to host 10.8.0.1, port 5201
[ 5] local 10.8.0.6 port 51402 connected to 10.8.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 21.4 MBytes 179 Mbits/sec 28 70.9 KBytes
[ 5] 1.00-2.00 sec 20.9 MBytes 175 Mbits/sec 12 93.3 KBytes
[ 5] 2.00-3.00 sec 21.1 MBytes 177 Mbits/sec 12 89.6 KBytes
[ 5] 3.00-4.00 sec 23.4 MBytes 196 Mbits/sec 21 94.6 KBytes
[ 5] 4.00-5.00 sec 22.6 MBytes 190 Mbits/sec 11 98.3 KBytes
[ 5] 5.00-6.00 sec 22.8 MBytes 191 Mbits/sec 22 74.6 KBytes
[ 5] 6.00-7.00 sec 22.2 MBytes 186 Mbits/sec 10 74.6 KBytes
[ 5] 7.00-8.00 sec 20.1 MBytes 168 Mbits/sec 18 92.1 KBytes
[ 5] 8.00-9.00 sec 23.7 MBytes 199 Mbits/sec 20 82.1 KBytes
[ 5] 9.00-10.00 sec 20.3 MBytes 170 Mbits/sec 17 72.2 KBytes
[ 5] 10.00-11.00 sec 20.4 MBytes 172 Mbits/sec 16 94.6 KBytes
[ 5] 11.00-12.00 sec 21.4 MBytes 179 Mbits/sec 16 95.8 KBytes
[ 5] 12.00-13.00 sec 20.8 MBytes 174 Mbits/sec 20 78.4 KBytes
[ 5] 13.00-14.00 sec 17.3 MBytes 145 Mbits/sec 11 99.5 KBytes
[ 5] 14.00-15.00 sec 14.7 MBytes 123 Mbits/sec 24 79.6 KBytes
[ 5] 15.00-16.00 sec 13.8 MBytes 116 Mbits/sec 8 83.4 KBytes
[ 5] 16.00-17.00 sec 13.8 MBytes 115 Mbits/sec 7 90.8 KBytes
[ 5] 17.00-18.00 sec 12.8 MBytes 108 Mbits/sec 20 92.1 KBytes
[ 5] 18.00-19.00 sec 12.1 MBytes 102 Mbits/sec 10 83.4 KBytes
[ 5] 19.00-20.00 sec 12.3 MBytes 103 Mbits/sec 6 84.6 KBytes
[ 5] 20.00-21.00 sec 12.9 MBytes 108 Mbits/sec 21 75.9 KBytes
[ 5] 21.00-22.00 sec 12.5 MBytes 104 Mbits/sec 6 73.4 KBytes
[ 5] 22.00-23.00 sec 13.4 MBytes 113 Mbits/sec 10 72.2 KBytes
[ 5] 23.00-24.00 sec 13.5 MBytes 113 Mbits/sec 15 103 KBytes
[ 5] 24.00-25.00 sec 13.6 MBytes 114 Mbits/sec 12 74.6 KBytes
[ 5] 25.00-26.00 sec 12.4 MBytes 104 Mbits/sec 7 73.4 KBytes
[ 5] 26.00-27.00 sec 10.7 MBytes 89.4 Mbits/sec 4 95.8 KBytes
[ 5] 27.00-28.00 sec 11.4 MBytes 95.6 Mbits/sec 8 84.6 KBytes
[ 5] 28.00-29.00 sec 9.60 MBytes 80.6 Mbits/sec 8 85.8 KBytes
[ 5] 29.00-30.00 sec 15.6 MBytes 131 Mbits/sec 12 74.6 KBytes
[ 5] 30.00-31.00 sec 12.5 MBytes 105 Mbits/sec 17 72.2 KBytes
[ 5] 31.00-32.00 sec 13.1 MBytes 110 Mbits/sec 6 75.9 KBytes
[ 5] 32.00-33.00 sec 13.8 MBytes 116 Mbits/sec 12 84.6 KBytes
[ 5] 33.00-34.00 sec 14.0 MBytes 118 Mbits/sec 6 88.3 KBytes
[ 5] 34.00-35.00 sec 14.4 MBytes 121 Mbits/sec 8 101 KBytes
[ 5] 35.00-36.00 sec 14.3 MBytes 120 Mbits/sec 15 79.6 KBytes
[ 5] 36.00-37.00 sec 14.6 MBytes 123 Mbits/sec 7 95.8 KBytes
[ 5] 37.00-38.00 sec 14.4 MBytes 121 Mbits/sec 13 74.6 KBytes
[ 5] 38.00-39.00 sec 11.8 MBytes 98.7 Mbits/sec 10 102 KBytes
[ 5] 39.00-40.00 sec 6.20 MBytes 51.9 Mbits/sec 3 79.6 KBytes
[ 5] 40.00-41.00 sec 7.68 MBytes 64.5 Mbits/sec 2 103 KBytes
[ 5] 41.00-42.00 sec 11.6 MBytes 97.7 Mbits/sec 17 89.6 KBytes
[ 5] 42.00-43.00 sec 11.8 MBytes 98.8 Mbits/sec 9 83.4 KBytes
[ 5] 43.00-44.00 sec 7.62 MBytes 63.9 Mbits/sec 4 109 KBytes
[ 5] 44.00-45.00 sec 7.37 MBytes 61.9 Mbits/sec 18 93.3 KBytes
[ 5] 45.00-46.00 sec 7.00 MBytes 58.7 Mbits/sec 4 84.6 KBytes
[ 5] 46.00-47.00 sec 9.85 MBytes 82.6 Mbits/sec 12 89.6 KBytes
[ 5] 47.00-48.00 sec 10.5 MBytes 88.4 Mbits/sec 3 75.9 KBytes
[ 5] 48.00-49.00 sec 12.4 MBytes 104 Mbits/sec 7 75.9 KBytes
[ 5] 49.00-50.00 sec 10.0 MBytes 84.1 Mbits/sec 5 94.6 KBytes
[ 5] 50.00-51.00 sec 10.5 MBytes 88.4 Mbits/sec 9 75.9 KBytes
[ 5] 51.00-52.00 sec 10.2 MBytes 85.3 Mbits/sec 3 90.8 KBytes
[ 5] 52.00-53.00 sec 8.61 MBytes 72.2 Mbits/sec 11 87.1 KBytes
[ 5] 53.00-54.00 sec 8.98 MBytes 75.3 Mbits/sec 7 92.1 KBytes
[ 5] 54.00-55.00 sec 12.0 MBytes 101 Mbits/sec 15 82.1 KBytes
[ 5] 55.00-56.00 sec 8.18 MBytes 68.6 Mbits/sec 3 87.1 KBytes
[ 5] 56.00-57.00 sec 8.49 MBytes 71.2 Mbits/sec 4 84.6 KBytes
[ 5] 57.00-58.00 sec 10.1 MBytes 84.7 Mbits/sec 5 94.6 KBytes
[ 5] 58.00-59.00 sec 8.55 MBytes 71.7 Mbits/sec 16 93.3 KBytes
[ 5] 59.00-60.00 sec 6.20 MBytes 52.0 Mbits/sec 8 79.6 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 820 MBytes 115 Mbits/sec 671 sender
[ 5] 0.00-60.01 sec 820 MBytes 115 Mbits/sec receiver
iperf Done.So generally still not as performant as without it, but it's been reliable and this is usable. There are a few places that I think I'd look, but the biggest thing that jumps out at me is the CWND differences, which makes sense considering we're creating new TCP connections every 5-10 sec. We can also talk a bit about whether there's an appropriate balance between hopping interval and speed.
Edited by Max B