Hello! We are running our annual fundraising. Please consider making a donation if you value this freely available service or want to support people around the world working towards liberatory social change. https://riseup.net/donate.

Unverified Commit ba7699c7 authored by Kali Kaneko's avatar Kali Kaneko
Browse files

Merge branch 'paixu-installer' into develop

Great work by paixu, one step into the direction of winodows
reproducible builds. The installer doesn't work, but this probably has
to do with remaining bugs in the main qt app.
parents 033bb6e2 f3e5121b
......@@ -3,17 +3,32 @@ import sys
block_cipher = None
a = Analysis([os.path.join('bitmask.py')],
a = Analysis(['bitmask.py'],
hiddenimports=[
'zope.interface', 'zope.proxy',
'PySide.QtCore', 'PySide.QtGui', 'PySide.QtWebKit'],
'zope.interface', 'zope.proxy',
'PySide.QtCore', 'PySide.QtGui',
'pysqlcipher', 'service_identity',
'leap.common', 'leap.bitmask'
],
binaries=None,
datas=None,
hookspath=None,
runtime_hooks=None,
excludes=None,
win_no_prefer_redirects=None,
win_private_assemblies=None,
cipher=block_cipher)
pyz = PYZ(a.pure,
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
# Binary files you need to include in the form of:
# (<destination>, <source>, '<TYPE>')
# Data files you want to include, in the form of:
# (<destination>, <source>, '<TYPE>')
data = [
('qt.conf', 'qt.conf', 'DATA')
]
exe = EXE(pyz,
a.scripts,
exclude_binaries=True,
......@@ -21,12 +36,13 @@ exe = EXE(pyz,
debug=False,
strip=False,
upx=True,
console=False )
console=False,
icon='../../data/images/mask-icon.ico')
coll = COLLECT(exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
strip=None,
upx=True,
name='bitmask')
if sys.platform.startswith("darwin"):
......
[Paths]
Plugins=qt4_plugins
\ No newline at end of file
.PHONY: all pkg installer openvpn pyinstaller
all:
docker-compose build
$(MAKE) pkg
pkg:
$(MAKE) openvpn
$(MAKE) pyinstaller
$(MAKE) installer
pyinstaller:
docker-compose run --rm pyinstaller
openvpn:
docker-compose run --rm openvpn
installer:
docker-compose run --rm installer
clean:
docker rmi windows_pyinstaller
docker rmi windows_openvpn
docker rmi windows_installer
rm -rf ../../dist/*.exe
rm -rf ../../build/*
\ No newline at end of file
Environment setup in debian:jessie
==================================
basically you need this to setup your environment:
# apt-get install mingw-w64
# apt-get install wine
# apt-get install nsis
this is a incomplete list of dependencies, review the pyinstaller/Dockerfile
to get a understanding of what needs to be setup in order to have a
environment that builds the installer
Requirements
============
docker-compose
Building the package
====================
make pkg
Reproducible builds
===================
please run the binary and installer builds on a clean machine eg
using docker or any virtual environment that can easily be prepared
by a third party to verify that the binaries are actually what the
sourcecode suggests.
to use reproducible build you need to install docker which then installs
a clean debian:jessie to install nsis or the mingw environment
Installer
=========
NSIS was choosen because it provided a out of the box toolchain to build
installers for the windows platform with minimal dependencies. The downside
of nsis is that it does not produce msi binaries
to build the binary dependencies run:
```
docker-compose run --rm openvpn
docker-compose run --rm pyinstaller
```
the produced binaries will be stored in ${ROOT}/build
to build the installer run:
```
docker-compose run --rm installer
```
the produced installer will be stored in ${ROOT}/dist
Pyinstaller
===========
Pyinstaller is a docker image based on debian:jessie with a cross-compile
toolchain (gcc) for building zlib and openssl in linux and wine (staging)
with installed python and mingw32 for pip/wheel compiling.
All pip installed dependencies are
part of the pyinstaller-build.sh script so they can be re-executed when the
dependencies of the project change. The image should be rebuild when openssl,
python or pyinstaller is updated:
```
docker-compose build pyinstaller
```
To debug or fine-tune the compile process it may be useful to setup the
following software on the development machine:
```
X :1 -listen tcp
DISPLAY=:1 xhost +
docker-compose run --rm pyinstaller /bin/bash
root@0fa19215321f:/# export DISPLAY=${YOUR_LOCAL_IP}:1
root@0fa19215321f:/# wine cmd
Z:\>python
>>>
```
the configured volumes are:
- the (read-only) sourcecode of the bitmask project in /var/src/bitmask
- the result of the builds in /var/build
pyinstaller-build.sh
====================
Contains all steps to build the win32 executables. The project relies on
a read-write source tree which will pollute the development environment and
make it hard to reproduce 'clean' builds. therefore it expects that the source
is freshly checked out and not used to run in the host-environment. Otherwise
pyc and ui elements will mess up the binary in unpredictable ways.
* copy the /var/src/bitmask sources to a read-write location (/var/build)
* execute ```make all``` in wine to build the qt ui and other resources
* execute ```pip install $dependencies``` to have all dependencies available
* execute ```pyinstaller``` in wine to compile the executable for
** bitmask (src/leap/bitmask/app.py)
* cleanup
** remove the read-write copy
** remove wine-dlls from the installer
As the step 'install dependencies' may take long on slow internet connections
during development it is advised to recycle the container and share the
build/executables path with a windows-vm to test the result in short cycles
instead of make pkg, uninstall, install.
```
docker-compose run --rm --entrypoint=/bin/bash pyinstalle
root@0fa19215321f:/# cd /var/src/bitmask/pkg/windows
root@0fa19215321f:/var/src/bitmask/pkg/windows# ./pyinstaller-build.sh
root@0fa19215321f:/var/src/bitmask/pkg/windows# ./pyinstaller-build.sh
root@0fa19215321f:/var/src/bitmask/pkg/windows# ./pyinstaller-build.sh
....
```
and test the result binary (accessible in bitmask/build in a separate vm.
OpenVPN
=======
OpenVPN is a straight forward cross compile image that builds the openvpn
sourcecode from the git-repository to a windows executable that can be
used by bitmask_root to launch eip.
It needs to be rebuild regulary as openssl gets a new version about every
month. PyInstaller uses the openssl that is compiled by this image
Installer
=========
Installer is a straight forward debian image with makensis installed. The
installer-build script lists the previously built files from pyinstaller and
openvpn to pass it as nsh file to makensis. bitmask.nis controls what will
be displayed to the user and how the components are installed and uninstalled
\ No newline at end of file
TODO
====
fix python-code (0.9.1) that fails on windows:
- fix the race condition for the backend/frontend startup.
spawning the backend takes time. with a introduced 15s timeout
it was possible to ensure the backend is up. It would be ideal
if the backend could signal the app to continue loading the frontend
- fix the ~/leap/events/zmq_certificates/public_keys/server.key
- fix logger (& remove hack in pyinstaller-build.sh:228)
- fix pysqlcipher (/LIB:,https get not working in setup.py, remove hack
in pyinstaller-build:164)
- merge bitmask_root from https://github.com/alirezamirzaeiyan/bitmask-root
create similar infrastructure for osx dmg, preferably run on osx for compressed dmg
\ No newline at end of file
!define PKGNAME bitmask
!include .\bitmask.nsh
\ No newline at end of file
# pwd is a ro-mounted source-tree that had all dependencies build into
# package-name directories
!define PKGNAMEPATH ..\..\build\executables\${PKGNAME}
!include ${PKGNAMEPATH}_version.nsh
!include .\bitmask_client_product.nsh
!include ${PKGNAMEPATH}_install_files_size.nsh
RequestExecutionLevel admin ;Require admin rights on NT6+ (When UAC is turned on)
InstallDir "$PROGRAMFILES\${APPNAME}"
LicenseData "..\..\LICENSE"
Name "${COMPANYNAME} - ${APPNAME}"
Icon "..\..\build\executables\mask-icon.ico"
# /var/dist is a rw mounted volume
outFile "/var/dist/${PKGNAME}-${VERSIONMAJOR}.${VERSIONMINOR}.${VERSIONBUILD}${VERSIONSUFFIX}.exe"
!include LogicLib.nsh
# Just three pages - license agreement, install location, and installation
page license
page directory
Page instfiles
!macro VerifyUserIsAdmin
UserInfo::GetAccountType
pop $0
${If} $0 != "admin" ;Require admin rights on NT4+
messageBox mb_iconstop "Administrator rights required!"
setErrorLevel 740 ;ERROR_ELEVATION_REQUIRED
quit
${EndIf}
!macroend
function .onInit
setShellVarContext all
!insertmacro VerifyUserIsAdmin
functionEnd
section "TAP Virtual Ethernet Adapter" SecTAP
SetOverwrite on
SetOutPath "$TEMP"
File /oname=tap-windows.exe "..\..\build\executables\openvpn\tap-windows.exe"
DetailPrint "Installing TAP (may need confirmation)..."
nsExec::ExecToLog '"$TEMP\tap-windows.exe" /S /SELECT_UTILITIES=1'
Pop $R0 # return value/error/timeout
Delete "$TEMP\tap-windows.exe"
WriteRegStr HKLM "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "tap" "installed"
sectionEnd
section "install"
setOutPath $INSTDIR
!include ${PKGNAMEPATH}_install_files.nsh
# Uninstaller - See function un.onInit and section "uninstall" for configuration
writeUninstaller "$INSTDIR\uninstall.exe"
# Start Menu
createDirectory "$SMPROGRAMS\${COMPANYNAME}"
createShortCut "$SMPROGRAMS\${COMPANYNAME}\${APPNAME}.lnk" "$INSTDIR\bitmask.exe" "" "$INSTDIR\bitmask.exe"
!include bitmask_client_registry_install.nsh
sectionEnd
# Uninstaller
function un.onInit
SetShellVarContext all
!insertmacro VerifyUserIsAdmin
functionEnd
section "uninstall"
delete "$SMPROGRAMS\${COMPANYNAME}\${APPNAME}.lnk"
# Try to remove the Start Menu folder - this will only happen if it is empty
rmDir "$SMPROGRAMS\${COMPANYNAME}"
# Remove files
!include ${PKGNAMEPATH}_uninstall_files.nsh
# Remove TAP Drivers
ReadRegStr $R0 HKLM "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "tap"
${If} $R0 == "installed"
DetailPrint "Uninstalling TAP as we installed it..."
ReadRegStr $R0 HKLM "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\TAP-Windows" "UninstallString"
${If} $R0 != ""
DetailPrint "Uninstalling TAP..."
nsExec::ExecToLog '"$R0" /S'
Pop $R0 # return value/error/timeout
${Else}
# on x64 windows the uninstall location needs to be accessed using WOW
SetRegView 64
ReadRegStr $R0 HKLM "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\TAP-Windows" "UninstallString"
SetRegView 32
${If} $R0 != ""
DetailPrint "Uninstalling TAP 64..."
nsExec::ExecToLog '"$R0" /S'
Pop $R0 # return value/error/timeout
${EndIf}
${EndIf}
DeleteRegValue HKLM "SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "tap"
${EndIf}
# Always delete uninstaller as the last action
delete $INSTDIR\uninstall.exe
# Try to remove the install directory - this will only happen if it is empty
rmDir $INSTDIR
# Remove uninstaller information from the registry
DeleteRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}"
sectionEnd
\ No newline at end of file
!define APPNAME "Bitmask"
!define COMPANYNAME "leap.se"
!define DESCRIPTION "With Bitmask VPN, all your traffic is securely routed through your provider before it is decrypted and sent on to the open internet."
# These will be displayed by the "Click here for support information" link in "Add/Remove Programs"
# It is possible to use "mailto:" links in here to open the email client
!define HELPURL "https://bitmask.net/en/help" # "Support Information" link
!define UPDATEURL "https://bitmask.net/en/install" # "Product Updates" link
!define ABOUTURL "https://bitmask.net/" # "Publisher" link
\ No newline at end of file
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "DisplayName" "${COMPANYNAME} - ${APPNAME} - ${DESCRIPTION}"
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "UninstallString" "$\"$INSTDIR\uninstall.exe$\""
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "QuietUninstallString" "$\"$INSTDIR\uninstall.exe$\" /S"
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "InstallLocation" "$\"$INSTDIR$\""
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "DisplayIcon" "$\"$INSTDIR\bitmask.exe$\""
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "Publisher" "${COMPANYNAME}"
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "HelpLink" "${HELPURL}"
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "URLUpdateInfo" "$\"${UPDATEURL}$\""
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "URLInfoAbout" "$\"${ABOUTURL}$\""
WriteRegStr HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "DisplayVersion" "${VERSIONMAJOR}.${VERSIONMINOR}.${VERSIONBUILD}${VERSIONSUFFIX}"
WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "VersionMajor" ${VERSIONMAJOR}
WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "VersionMinor" ${VERSIONMINOR}
# There is no option for modifying or repairing the install
WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "NoModify" 1
WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "NoRepair" 1
# Set the INSTALLSIZE constant (!defined at the top of this script) so Add/Remove Programs can accurately report the size
WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\${COMPANYNAME} ${APPNAME}" "EstimatedSize" ${INSTALLSIZE}
\ No newline at end of file
# mingw environment to build dependency binaries in a reproducible environment
# https://wiki.debian.org/ReproducibleBuilds
# service to build a windows executable using pyinstaller
# utilizes wine and pyinstaller-build.sh to produce
# build/executables/pyinstaller/bitmask/*
# usage: docker-compose run --rm pyinstaller
# non-zero exit code on failure
pyinstaller:
build: pyinstaller
volumes:
# bitmask sources
- ../../:/var/src/bitmask:ro
# produced binaries
- ../../build:/var/build
# service to build a windows-executable from openvpn sources
# uses the openvpn-build infrastructure to produce
# build/executables/openvpn/*
# produces the openvpn.exe and provides openssl that is to be
# used by pyinsaller
# usage: docker-compose run --rm openvpn
# non-zero exit code on failure
openvpn:
build: openvpn
volumes:
# bitmask sources
- ../../:/var/src/bitmask:ro
# produced binaries
- ../../build:/var/build
# service to compile a installer using nullsoft installer
# nsis environment to build installer (exe) that contains all required binaries
# for a clean, just installed windows machine
# utilizes the debian makensis and installer-build to produce
# dist/bitmask-VERSION.exe
# usage: docker-compose run --rm installer
# non-zero exit code on failure
installer:
build: installer
volumes:
# bitmask sources
- ../../:/var/src/bitmask:ro
# produced installers - configured in bitmask.nsh
- ../../dist:/var/dist
#!/bin/bash
# build installer
# ===============
#
# builds several installers from previously compiled binaries
product=bitmask
# the location of the nsis installer nis files dictates the path of the files
relative_executable_path=../../build/executables
source_ro_path=/var/src/${product}
temporary_build_path=/var/tmp/installer
setups=($(ls -1 ${source_ro_path}/pkg/windows | grep '.nis$' | sed 's|.nis$||'))
# generate nsis file references for installer for single directory
# appends File and Remove to files that are later included by makensis
# separate files for install and uninstall statements
#
# directory_root: the tree root that is currently generated
# subdir: any directory in the tree
# setup_name: the name of the setup this nsh entries are generated for
function generateDirectoryNSISStatements() {
directory_root=$1
subdir=$2
setup_name=$3
find ${subdir} -maxdepth 1 -type f -exec echo 'File "'${relative_executable_path}'/{}"' \;>> ${setup_name}_install_files.nsh
find ${subdir} -maxdepth 1 -type f -exec echo 'Delete "$INSTDIR/{}"' \; >> ${setup_name}_uninstall_files.nsh
}
# generate a tree of files into nsis installer definitions
# directory_root: the tree root that is currently generated
# setup_name: the name of the setup this nsh entries are generated for
function generateDirectoryNSISStatementsTree() {
directory_root=$1
setup_name=$2
subdirs=$(find ${directory_root} -type d | sort)
for subdir in ${subdirs[@]}
do
if [ "${directory_root}" != "${subdir}" ]; then
echo 'SetOutPath "$INSTDIR/'${subdir}'"' >> ${setup_name}_install_files.nsh
fi
generateDirectoryNSISStatements ${directory_root} ${subdir} ${setup_name}
done
# again to remove emptied directories on uninstall so reverse
subdirs=$(find ${directory_root} -type d | sort | tac)
for subdir in ${subdirs[@]}
do
if [ "${directory_root}" != "${subdir}" ]; then
echo 'RMDir "$INSTDIR/'${subdir}'"' >> ${setup_name}_uninstall_files.nsh
fi
done
}
# generate installer files for the available setups
# those files include install and uninstall statements and are
# modified (backslashes/source_path) to generate a sane target
# structure
function generateNSISStatements() {
pushd ${temporary_build_path}/build/executables
for setup in "${setups[@]}"
do
echo "setup:" ${setup}
echo "# auto generated by pkg/windows/installer-build.sh please do not modify" > ${setup}_install_files.nsh
echo "# auto generated by pkg/windows/installer-build.sh please do not modify" > ${setup}_uninstall_files.nsh
setup_source_path=${setup}
generateDirectoryNSISStatementsTree ${setup_source_path} ${setup}
# remove the setup_source_path from the nsh files
sed -i "s|INSTDIR/${setup_source_path}/|INSTDIR/|" ${setup}_install_files.nsh
sed -i "s|/${setup_source_path}/|/|" ${setup}_uninstall_files.nsh
# make backslashes
sed -i "s|/|\\\\|g" ${setup}_install_files.nsh ${setup}_uninstall_files.nsh
# make install size
installed_size=$(du -s --block-size=1000 ${setup} | awk '{print $1}')
echo "!define INSTALLSIZE ${installed_size}" > ${setup}_install_files_size.nsh
done
popd
}
# makensis to produce a installer.exe
# the result is placed in /var/dist
function buildInstaller() {
pushd ${temporary_build_path}/pkg/windows
for setup in ${setups[@]}
do
makensis ${setup}.nis || die 'build setup "'${setup}'" failed'
done
popd
}
# prepare build path
# copies files that have been produced by other containers
# merges the product so the nsis files are correct
function prepareBuildPath() {
mkdir -p ${temporary_build_path}/pkg/windows
mkdir -p ${temporary_build_path}/build
cp -r ${source_ro_path}/pkg/windows/* ${temporary_build_path}/pkg/windows
cp -r ${source_ro_path}/build/* ${temporary_build_path}/build
cp -r ${source_ro_path}/LICENSE ${temporary_build_path}/LICENSE
test -d ${temporary_build_path}/build/executables/bitmask || die 'bitmask not available run docker-compose run --rm pyinstaller'
test -d ${temporary_build_path}/build/executables/openvpn || die 'openvpn not available run docker-compose run --rm openvpn'
pushd ${temporary_build_path}/build/executables
cp openvpn/bin/openvpn.exe bitmask
cp openvpn/bin/*.dll bitmask
popd
}
# remove build files to ensure subsequent builds
function cleanup() {
rm -r ${temporary_build_path}
}
# display failure message and emit non-zero exit code
function die() {
echo "die:" $@
exit 1
}
function main() {
prepareBuildPath
generateNSISStatements
buildInstaller
cleanup
}
main $@
\ No newline at end of file
FROM debian:jessie
MAINTAINER paixu@0xn0.de
RUN apt-get update
######
# install packages required to build
RUN apt-get -y install \
nsis
WORKDIR /var/src/bitmask/pkg/windows
######
# set a specific user
# needs external tuning of the /var/dist rights!
# RUN useradd installer
# USER installer
ENTRYPOINT ["/var/src/bitmask/pkg/windows/installer-build.sh"]
\ No newline at end of file
#!/bin/bash
# render openvpn prepared for installer
# ================================================
#
# requires
# - a linux host with mingw installed
# - a rw directory mounted to /var/build
# returns nonzero exit code when failed
#
# clone openvpn-build repository
# runs cross-compile build
# - downloads openvpn dependencies
# - compiles
# copy files to executables so they can be installed
# cleans up (remove read-write copy)
# the location where the openvpn binaries are placed
absolute_executable_path=/var/build/executables
temporary_build_path=/var/build/openvpn
# cleanup the temporary build path for subsequent executes
function cleanup() {
rm -r ${temporary_build_path} 2>/dev/null
}
# build openvpn source
function buildSource() {
pushd ${temporary_build_path}/openvpn-build/generic
CHOST=i686-w64-mingw32 \
CBUILD=i686-pc-linux-gnu \
./build \
|| die 'build openvpn from source failed'
mkdir -p ${absolute_executable_path}
cp -r image/openvpn ${absolute_executable_path}/openvpn
popd
}
# fetch tap-windows.exe as defined in the openvpn vars
function fetchTapWindows() {
pushd ${temporary_build_path}/openvpn-build
source windows-nsis/build-complete.vars
wget ${TAP_WINDOWS_INSTALLER_URL} -O ${absolute_executable_path}/openvpn/tap-windows.exe || die 'tap-windows.exe could not be fetched'
popd
}
# prepare read-write copy
function prepareBuildPath() {
cleanup
mkdir -p ${temporary_build_path}
pushd ${temporary_build_path}
git clone https://github.com/OpenVPN/openvpn-build || die 'openvpn-build could not be cloned'
popd
}
# display failure message and emit non-zero exit code
function die() {
echo "die:" $@
exit 1
}
function main() {
prepareBuildPath
buildSource