ObfsVPN
The obfsvpn module contains a Go package that provides server and client components to
use variants of the obfs4 obfuscation protocol. It is intended to be used as a
drop-in Pluggable Transport for OpenVPN connections (although it can be used
for other, more generic purposes).
A docker container will be provided to facilitate startng an OpenVPN service that is accessible via the obfuscated proxy too.
You can read more online about how obfsvpn is used to provide circumvention tactics to the LEAP VPN Clients, and in particular about the design of the Hopping Pluggable Transport.
Protocol stack
--------------------
application data
--------------------
OpenVPN
--------------------
obfsvpn proxy
--------------------
obfs4
--------------------
wire transport
--------------------- Application data is written to the specified interface (typically a
tundevice started byOpenVPN). -
OpenVPNprovides end-to-end encryption and a reliability layer. We'll be testing with the2.5.xbranch of the reference OpenVPN implementation. -
obfs4is used for an extra layer of encryption and obfuscation. It is a look-like-nothing protocol that also hides the key exchange to the eyes of the censor. -
obfs4requires a stream protocol to write to/read from. The default is TCP, but KCP is a configurable optional.QUICcould be an avenue of further exploration.
Development and Testing
Docker compose
There is an entirely automated docker-compose based network sandbox that can be used for development and testing.
It can also serve as useful documentation as to the correct way to run the services as containers/connect them to eachother.
It's useful to note that w/ docker-compose, we specify environment files which allow for variable expansion within the docker-compose file. That is, we have a variety of .env files: .env, .env.hopping, .env.kcp, etc etc etc which all represent different deployment configurations/topologies. Running docker-compose without specifying the --env file will cause the script to assume using the .env file.
In order to start all of the services in the "default mode", simply run:
$ docker-compose up -dThis will start an openvpn server, 2 instances of the obfsvpn server, and a client which runs both the obfsvpn client and an openvpn client.
You can then use docker-compose to inspect/monitor/run commands on the containers
$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------------
obfsvpn_client_1 dumb-init /usr/bin/start.sh Up
obfsvpn_obfsvpn-1_1 dumb-init /opt/obfsvpn/sta ... Up
obfsvpn_obfsvpn-2_1 dumb-init /opt/obfsvpn/sta ... Up
obfsvpn_openvpn-server_1 dumb-init /opt/openvpn-ser ... Up 5540/tcp, 5540/udpYou can get logs from one, more, or all of the services:
$ docker-compose logs client
$ docker-compose logs client openvpn-server
# to tail all logs:
$ docker-compose logs -fYou can then run arbitrary commands on any of the services to debug, test performance, etc:
$ docker-compose exec client ip route
0.0.0.0/1 via 10.8.0.5 dev tun0
default via 192.168.80.1 dev eth0
10.8.0.1 via 10.8.0.5 dev tun0
10.8.0.5 dev tun0 proto kernel scope link src 10.8.0.6
127.0.0.1 via 192.168.80.1 dev eth0
128.0.0.0/1 via 10.8.0.5 dev tun0
192.168.80.0/20 dev eth0 proto kernel scope link src 192.168.80.3
$ docker-compose exec client ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=109 time=12.495 ms
64 bytes from 8.8.8.8: seq=1 ttl=109 time=13.614 ms
64 bytes from 8.8.8.8: seq=2 ttl=109 time=13.900 ms
$ docker-compose exec openvpn-server iperf3 -s --bind-dev tun0
❯ docker-compose exec client iperf3 -c 10.8.0.1 --bind-dev tun0
Connecting to host 10.8.0.1, port 5201
[ 5] local 10.8.0.6 port 51390 connected to 10.8.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 40.9 MBytes 343 Mbits/sec 206 801 KBytes
[ 5] 1.00-2.00 sec 36.2 MBytes 304 Mbits/sec 106 601 KBytes
[ 5] 2.00-3.00 sec 41.2 MBytes 346 Mbits/sec 6 456 KBytes
[ 5] 3.00-4.00 sec 40.0 MBytes 336 Mbits/sec 0 512 KBytes
[ 5] 4.00-5.00 sec 42.5 MBytes 357 Mbits/sec 0 565 KBytes
[ 5] 5.00-6.00 sec 43.8 MBytes 367 Mbits/sec 0 615 KBytes
[ 5] 6.00-7.00 sec 36.2 MBytes 304 Mbits/sec 22 457 KBytes
[ 5] 7.00-8.00 sec 41.2 MBytes 346 Mbits/sec 0 525 KBytes
[ 5] 8.00-9.00 sec 41.2 MBytes 346 Mbits/sec 0 575 KBytes
[ 5] 9.00-10.00 sec 40.0 MBytes 336 Mbits/sec 0 610 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 403 MBytes 338 Mbits/sec 340 sender
[ 5] 0.00-10.02 sec 401 MBytes 336 Mbits/sec receiver
iperf Done.The PT3 Hopping architecture can be brought up in an almost identical way, except that calls to docker-compose require an --env-file ./.env.hopping parameter to distinguish between the two strategies.
Then when you want to run commands, add the --env-file argument:
❯ docker-compose --env-file ./.env.hopping exec client ping -c 3 -I tun0 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=113 time=12.829 ms
64 bytes from 8.8.8.8: seq=1 ttl=113 time=19.346 ms
64 bytes from 8.8.8.8: seq=2 ttl=113 time=19.013 ms
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 12.829/17.062/19.346 msContainer environment variables
obfsvpn server
Before you can run a obfsvpn server container you need to make sure to set the following environment variables which are required in the start script.
| Variable | Purpose | Example | Comment |
|---|---|---|---|
CONTROL_PORT |
port of the Control Plane | 9090 | required |
OBFSVPN_STATE |
directory of private and public key, certifcate, bridgeline file | ./test_data | required |
OBFSVPN_LOCATION |
location of the OpenVPN gateway the bridge is pointing to | amsterdam | required |
OBFS4_IP |
public IP of the bridge | 123.231.123.21 | required |
OBFS4_PORT |
port the bridge is listening on | 4430 | required |
OBFS4_HOST |
The IP obfsvpn server is gets assigned to | 0.0.0.0 | required |
OPENVPN_HOST |
public IP of the OpenVPN gateway the bridge is pointing to | 231.123.231.12 | required |
OPENVPN_PORT |
port | 80 | required |
OBFS4_DATA_DIR |
same as OBFSVPN_STATE
|
./test_data | required |
OBFSVPN_SEED |
seed to deduce randomized ports from for port hopping | 1 | optional (default 1) |
OBFSVPN_PORT_COUNT |
number of ports to be used for port hopping | 100 | optional (default 100) |
OBFSVPN_MIN_HOP_PORT |
lower limit of port range to use for port hopping | 9095 | optional (default 49152) |
OBFSVPN_MAX_HOP_PORT |
upper limit of port range to use for port hopping | 12095 | optional (default 65535) |
HOP_PT |
run server in hopping pt mode | 1 if true | optional (default 0) |
KCP |
run server in KCP transport mode | 1 if true | optional (default 0) |
TCP |
run server in w/ tcp as the protocol the server accepts for proxying traffic | 1 if true | optional (default 1) |
QUIC |
run server in QUIC transport mode | 1 if true | optional (default 0) |
QUIC_TLS_CERT |
TLS cert used for QUIC transport encryption | ./test_data/pki/private/ca.crt | required for QUIC |
QUIC_TLS_KEY |
TLS private key used for QUIC transport encryption | ./test_data/pki/private/ca.key | required for QUIC |
Integration testing
We additionally have an ./scripts/integration-test.sh script which starts all of the services and then does a set of very small/trivial smoke tests to ensure that the platform is working as expected.
To bring up all of the services in the traditional/non-hopping mode:
$ ./scripts/integration-test.shIf you want to test hopping mode:
$ ./scripts/integration-test.sh hopAnd finally to test KCP:
$ ./scripts/integration-test.sh kcpYou can test other modes with the parameters: hop-kcp, quic, hop-quic.
Running components separately/against live systems
There may be scenarios when you'd prefer to run individual components on their own or targeting live systems.
Each of the individual components can be run separately, though some are easier to configure than others.
In general it's recommended to prioritize trying to work within the docker-compose environment if/when possible.
obfsvpn client
The obfsvpn client is a go binary that connects to an obfsvpn server and as a pair form a proxy through which arbitrary UDP (and in some less common scenarios TCP) traffic can be tunneled.
You can see the arguments that are required by running it by executing it with the --help flag:
$ go run ./cmd/client --help
Usage of /tmp/go-build901008461/b001/exe/client:
-c string
The remote obfs4 certificates separated by commas. If hopping is not enabled only the first cert will be used
-h Connect with openvpn over udp in hopping mode
-i string
The host for the local proxy (default "127.0.0.1")
-j uint
A random range to wait (on top of the minimum) seconds before hopping. Only applicable to hopping (default 5)
-kcp
Enable KCP mode
-kcp-disable-flow-control
KCP DisableFlowControl (default true)
-kcp-interval int
KCP Interval (default 10)
-kcp-mtu int
KCP MTU (default 1400)
-kcp-no-delay
KCP NoDelay (default true)
-kcp-read-buffer int
KCP ReadBuffer (default 16777216)
-kcp-receive-window-size int
KCP ReceiveWindowSize (default 65535)
-kcp-resend int
KCP Resend (default 2)
-kcp-send-window-size int
KCP SendWindowSize (default 65535)
-kcp-write-buffer int
KCP WriteBuffer (default 16777216)
-m uint
The minimun number of seconds to wait before hopping. Only applicable to hopping (default 5)
-max-port uint
The upper limit of the port range used for port hopping (default 65507)
-min-port uint
The lower limit of the port range used for port hopping (default 49152)
-p string
The port for the local proxy (default "8080")
-pc uint
The number of ports to try for each remote. Only applicable to hopping (default 100)
-ps int
The random seed to generate ports from. Only applicable to hopping (default 1)
-quic
Enable QUIC mode
-r string
The remote obfs4 endpoint ips (no port) separated by commas. If hopping is not enabled only the first cert will be used
-rp string
The remote obfs4 endpoint port to use. Only applicable to NON-hopping
-v Enable verbose loggingThe -c flag is for obfs4 certificates. This will be in base64 string form. For our docker testbed, we hard code these in the client Dockerfile.
To get information about obfs4 server bridges to connect to, you can query the menshen service. For our demo.bitmask.net deployment that could look like:
❯ curl -sL https://api.demo.bitmask.net/api/5/bridges | jq ✘ 4
[
{
"healthy": true,
"host": "cod.demo.bitmask.net",
"ip_addr": "37.218.245.94",
"ip6_addr": "",
"load": 0,
"location": "northbrabant",
"overloaded": false,
"port": 443,
"transport": "tcp",
"type": "obfs4",
"options": {
"cert": "k0L4LFg0Wk98v7P66xvgAx2ud+kggvjZX/qul3iFTJGH5X7xSHT+vVL4UZR0WI3SkmDzUg",
"iatMode": "0"
},
"bucket": ""
},
{
"healthy": true,
"host": "cod.demo.bitmask.net",
"ip_addr": "37.218.245.94",
"ip6_addr": "",
"load": 0,
"location": "northbrabant",
"overloaded": false,
"port": 4431,
"transport": "kcp",
"type": "obfs4",
"options": {
"cert": "k0L4LFg0Wk98v7P66xvgAx2ud+kggvjZX/qul3iFTJGH5X7xSHT+vVL4UZR0WI3SkmDzUg",
"iatMode": "0"
},
"bucket": ""
},
{
"healthy": true,
"host": "mullet.demo.bitmask.net",
"ip_addr": "37.218.241.208",
"ip6_addr": "",
"load": 0,
"location": "florida",
"overloaded": false,
"port": 443,
"transport": "tcp",
"type": "obfs4",
"options": {
"cert": "k0L4LFg0Wk98v7P66xvgAx2ud+kggvjZX/qul3iFTJGH5X7xSHT+vVL4UZR0WI3SkmDzUg",
"iatMode": "0"
},
"bucket": ""
},
{
"healthy": true,
"host": "mullet.demo.bitmask.net",
"ip_addr": "37.218.241.208",
"ip6_addr": "",
"load": 0,
"location": "florida",
"overloaded": false,
"port": 4431,
"transport": "kcp",
"type": "obfs4",
"options": {
"cert": "k0L4LFg0Wk98v7P66xvgAx2ud+kggvjZX/qul3iFTJGH5X7xSHT+vVL4UZR0WI3SkmDzUg",
"iatMode": "0"
},
"bucket": ""
}
]So, supposing that you wanted to connect to the cod.demo.bitmask.net obfsvpn server over "normal"/non-KCP, you could run:
$ go run ./cmd/client -c "k0L4LFg0Wk98v7P66xvgAx2ud+kggvjZX/qul3iFTJGH5X7xSHT+vVL4UZR0WI3SkmDzUg" -r 37.218.245.94 -rp 443 -v
2024/08/12 16:16:42 proxyAddr: 127.0.0.1:8080
2024/08/12 16:16:42 obfs4 endpoints: [37.218.245.94:443]
2024/08/12 16:16:42 Update state: STARTING
2024/08/12 16:16:43 Update state: RUNNINGThere should now be a udp listener on the default address/port:
$ ss -ul src 127.0.0.1:8080
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
UNCONN 0 0 127.0.0.1:8080 0.0.0.0:*You can specify a particular listening address with the -i flag and a particular listening port with the -p flag.
If you want to connect via KCP, use the port for the host that's listening w/ KCP and specify the -kcp flag:
$ go run ./cmd/client -c "k0L4LFg0Wk98v7P66xvgAx2ud+kggvjZX/qul3iFTJGH5X7xSHT+vVL4UZR0WI3SkmDzUg" -r 37.218.245.94 -rp 4431 -v -kcp
2024/08/12 16:22:11 proxyAddr: 127.0.0.1:8080
2024/08/12 16:22:11 obfs4 endpoints: [37.218.245.94:4431]
2024/08/12 16:22:11 Update state: STARTING
2024/08/12 16:22:11 Dialing kcp://37.218.245.94:4431
2024/08/12 16:22:11 Update state: RUNNINGIf you wanted to run openvpn through that particular bridge, you'd specify the --remote and --proto udp flags when running the openvpn command:
$ openvpn --remote 127.0.0.1 8080 --proto udp [A BUNCH MORE OPENVPN FLAGS/CONFIGS HERE]Android
Assuming you have the android ndk in place, you can build the bindings for android using gomobile:
go get -u golang.org/x/mobile/cmd/gomobile
gomobile init
gomobile bind -x -target android -o mobile/android/obfsvpn.aar ./client/Testing introducer/invite token
Idea: In a censored environment, the client (code in bitmask-core) can use a private proxy to speak with menshen. This communication is also obfuscated using obfs4. This repository holds the code of the proxy part.
- Run the server with
--state, but without--persist. This generates a certificate and caches it to the state directory (state/obfs4_state.json).state/obfs4_bridgeline.txtshows you the certificate needed for the client side (pinning).
mkdir state/
go run ./cmd/server --addr 127.0.0.1 --port 4430 --remote 127.0.0.1:8443 --state $(pwd)/state -v
...
pea@peabox: cat state/obfs4_bridgeline.txt
# obfs4 torrc client bridge line
#
# This file is an automatically generated bridge line based on
# the current obfs4proxy configuration. EDITING IT WILL HAVE
# NO EFFECT.
#
# Before distributing this Bridge, edit the placeholder fields
# to contain the actual values:
# <IP ADDRESS> - The public IP address of your obfs4 bridge.
# <PORT> - The TCP/IP port of your obfs4 bridge.
# <FINGERPRINT> - The bridge's fingerprint.
Bridge obfs4 <IP ADDRESS>:<PORT> <FINGERPRINT> cert=9922C2bKo6PY4iipssMfOH01eb86dcJZD65dPkdL3vJMMGy7h3CLFUDYK3/Udc6tB2h8aQ iat-mode=0- Then you can run the proxy and just use the the previously generated certificate. As the client pins the certificate, it's nice to have a persistent certificate.
go run ./cmd/server --addr 127.0.0.1 --port 4430 --remote 127.0.0.1:8443 --state $(pwd)/state -v --persist
2024/11/13 10:49:33 Using obfs4 config file: /home/pea/leap/obfsvpn/state/obfs4_state.json
DEBUG 2024/11/13 10:49:33 kcp: false, hop: false, udp: false, quic: false
2024/11/13 10:49:33 Listening on 127.0.0.1:4430…
DEBUG 2024/11/13 10:50:08 accepted connection from 127.0.0.1:46398
2024/11/13 10:50:08 Dialing: 127.0.0.1:8443
2024/11/13 10:50:08 Obfs4 client: 127.0.0.1:46398
--> Entering copy loop.In this case, the proxy is listening on 127.0.0.1:4430 and the upstream menshen instance is listening on 127.0.0.1:8443. To use/test this code, you need the client side part. It's documented here. It's basically a simple curl using the proxy+obfs4.