Integrate bitmask-core/implement v5 #758

This branch is based on !171 (merged).
Currently only a draft for tracking.
Not committing/pushing because code changes often.

current state, changes, TODOs and things to discuss

logging

ISSUE #771

DONE

• moved to zerolog from default "log"
• different log levels, specified by env variable
• pkg/config/logger.go

TODO

• some packages are missing and using the old log

TO DISCUSS

• log level: TRACE, DEBUG, default (INFO)
• logging of qt stuff
• what to log (privacy)
• logrotate?
• rename file?
• log file format sucks a bit with the json
• logging of qt stuff

v5 implementation

DONE

• interface Bitmask, which is implemented twice: Bitmask3 and Bitmask5
  • Bitmask3 implementation moved to bitmask/legacy
  • Bitmask5 implementation lays in bitmask/*.go
  • big v3/v5 switch is in bitmask/init.go (initBitmaskVPN)
    • configuration: new field (apiVersion) in provider settings (providers/vendor.conf)
    • env: API_VERSION
• poc/dummy checkGatewayOverload implementation is also part of this branch (but not called)
• interface.go: InstallHelpers(), useUDP, DoLogin/SetTransport

bitmask-core integration

• code lays in pkg/vpn/menshen/
• real v5 stuff lays in pkg/vpn/menshen/v5_implementation.go
  • there are comments where things will be moved (this is just to see the real v5 changes)
  • tests in pkg/vpn/menshen/v5_implementation_test.go
• changes: less and cleaner code
  • no old old stuff (gatewayV1, gatewayV3)
  • removed the NoDNS/static ip path
  • removed some debugging stuff: LEAP_OPENVPN_EXTRA_CONFIG
  • no second api (geoip), just just use bitmask-core
  • removed the auth code: add if we need it
  • flatten abstraction: removed the gatewayPool struct
  • simplified logic of: giveMeGatewaysToConnect() function
  • use swagger: models.ModelsGateway instead of Gateway
  • for debugging: env for API_URL

Summary

• it's a first version. Some implementation details are missing (needs features in menshen)
  • GOOD: GetPemCertificate() (just ask bitmask-core, future work: maybe add caching (in core?))
  • OK: GetOpenvpnArgs() -> currently, we get it from /service via bitmask-core
    • rename endpoint in menshen (have a dedicated one?)?
    • have it per gateway? Does not make sense if we run openvpn with multiple --remote args
  • OK? - FetchGateways()
    • currently, menshen returns 4 (random) gateways: curl -s localhost:8443/api/5/gateways | jq '.[].host'
    • build m.gwLocations = ["Paris", "Amsterdam"]
    • build m.m.gwsByLocation = { "Paris": gw1, gw2, gw3, "Amsterdam": gw4, gw5}
    • not sending current country code - DoLocationLookup() in bitmask-core
    • menshen TODOs:
      • gateway: add "display name" + countryCode
      • cc "filtering"
  • GetBestLocation() -> return m.gwLocations[0] (Could use smallest rtt)
  • GetLocationQualityMap(): qualityMap["Paris"] = 23 based on rtt
  • choose location and connect:
    • no metric by menshen: get first three two gateways per location
    • menshen TODO:
      • metric for each gateway
      • gateway (better: VPNEndpoint) currently returns a list => will be a single port
        • but: not protocol (udp/tcp)

Things I want to discuss in the future

• non-recoverable errors are just printed. Then it segfaults somewhere
  • can not write management password/cert to disk
  • parsing provider json / can not parse string to int (listen port for OpenVPN management Backend)
• state of UDP: can be enabled in settings, stored in config file => remove everywhere and prefer it always (part of next release?)
• Snowflake support
  • there is code in v3 that works, but needs a dependency ("snowflake-client" binary + tor)
  • it's not fully integrated in the UI (button is disabled, env variable is needed)
  • Check if it's installed => use it (allow to enable it in the settings)?
• Can we remove the vendor dir? I don't see benefits, only problems
• GUI/network race conditions:
  • GetBestLocation() triggered by GUI => what if the fetch from menshen is not yet done/blocked? -> sleep? return an error?, show an error in the GUI? -> show initial splash screen until we are in a good state

v5 TODOs

• get !171 (merged) merged (refactor to get v5 done)
• Make proper commits
• logging: also log date date rein, log filename, close log file/return value?
• improve error handling in GUI
  • race: Trying to get locationQualityMap, but m.Gateways is empty
    • race: menshen is offline, start desktop => I can click connect, even there if there are (not yet) gateways => created an issue for this
  • bitmask-root (if it runs openvpn) returns an error => update gui state (showing pending.../connecting)
  • startVPN => segfault on race condition if ip is blocked/Bitmask struct is not yet initialized
• grep: TODO
• tests: in bitmask/utils_test.go add test files and check functionality of isValidCert Understand/improve helper installation/check
• LEAP_DRYRUN=1 does not work, state is broken, gui...
• implement more v5 stuff, that curerntly needs changes in menshen menshen#31 (comment 1183875)
  • send country code to menshen
  • Location: display name (remove all the string.Title calls)
  • load: calc proper qualityMap for each location
  • logic: calc best location
  • logic: calc best gateways per location

v5 check code

• autostart
• Bitmask5: use gateawy/ptGateway genutzt? move to menshen?
• GetMotd => Fehler => should return dict?
• get stderr/warnings from bitmask-root/openvpn (a lot of deprecation messages)
• Check why/how often toJson is called

debugging/env variables

ISSUE #772

DONE

• env variables
• v3/v5 switch

New/Resutling issues

• Handle reconnect after system goes into sleep/hibernate #795
• Connecting with udp 53 (sometimes?) fail: #796
• Understand/improve helper installation/check: #797 (closed)
• location/gateway/port selection in v5 #788
• Show error messages to user #799