High memory usage in soledad-server
Trying to set up bitmask on a new machine, I noticed that while sync receiving ~177,000 items, twistd from soledad-server.service on my provider would consume memory up to around 5GB, before releasing it again after sync ended.
I had accidentally deployed 0.10.0, leap info
yields the following:
sune@jekaterina:~/src/pixelated/leap/mail2$ leap info = [mail2] apt_running => false = [mail2] architecture => amd64 = [mail2] augeasversion => 1.2.0 = [mail2] couchdb_pwhash_alg => pbkdf2 = [mail2] couchdb_version => 1.6.0+dfsg.1-5~leap80+1 = [mail2] debian_codename => jessie = [mail2] debian_lts => false = [mail2] debian_nextcodename => stretch = [mail2] debian_nextrelease => testing = [mail2] debian_release => stable = [mail2] dhcp_enabled => true = [mail2] domain => molgaard.org = [mail2] facterversion => 2.2.0 = [mail2] filesystems => btrfs,ext2,ext3,ext4 = [mail2] fqdn => mail2.molgaard.org = [mail2] hardwaremodel => x86_64 = [mail2] hostname => mail2 = [mail2] interface_127.0.0.1 => lo = [mail2] interface_172.31.1.100 => eth0 = [mail2] interfaces => eth0,lo = [mail2] ipaddress => 172.31.1.100 = [mail2] ipaddress_eth0 => 172.31.1.100 = [mail2] ipaddress_lo => 127.0.0.1 = [mail2] is_pe => false = [mail2] is_virtual => true = [mail2] kernel => Linux = [mail2] kernelmajversion => 3.16 = [mail2] kernelrelease => 3.16.0-4-amd64 = [mail2] kernelversion => 3.16.0 = [mail2] lib => /srv/leap/puppet/modules/apache/lib/facter:/srv/leap/puppet/modules/apt/lib/facter:/srv/leap/puppet/modules/concat/lib/facter:/srv/leap/puppet/modules/couchdb/lib/facter:/srv/leap/puppet/modules/rsyslog/lib/facter:/srv/leap/puppet/modules/site_config/lib/facter:/srv/leap/puppet/modules/sshd/lib/facter:/srv/leap/puppet/modules/stdlib/lib/facter = [mail2] lsbdistcodename => jessie = [mail2] lsbdistdescription => Debian GNU/Linux 8.9 (jessie) = [mail2] lsbdistid => Debian = [mail2] lsbdistrelease => 8.9 = [mail2] lsbmajdistrelease => 8 = [mail2] lsbminordistrelease => 9 = [mail2] memoryfree => 6.66 GB = [mail2] memoryfree_mb => 6822.91 = [mail2] memorysize => 7.82 GB = [mail2] memorysize_mb => 8005.68 = [mail2] mtu_eth0 => 1500 = [mail2] mtu_lo => 65536 = [mail2] netmask => 255.255.255.0 = [mail2] netmask_cidr_eth0 => 24 = [mail2] netmask_cidr_lo => 8 = [mail2] netmask_eth0 => 255.255.255.0 = [mail2] netmask_lo => 255.0.0.0 = [mail2] network_eth0 => 172.31.1.0 = [mail2] network_lo => 127.0.0.0 = [mail2] operatingsystem => Debian = [mail2] os => {"name"=>"Debian", "family"=>"Debian", "release"=>{"major"=>"8", "minor"=>"9", "full"=>"8.9"}, "lsb"=>{"distcodename"=>"jessie", "distid"=>"Debian", "distdescription"=>"Debian GNU/Linux 8.9 (jessie)", "distrelease"=>"8.9", "majdistrelease"=>"8", "minordistrelease"=>"9"}} = [mail2] osfamily => Debian = [mail2] path => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin = [mail2] physicalprocessorcount => 1 = [mail2] processor0 => Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge) = [mail2] processor1 => Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge) = [mail2] processorcount => 2 = [mail2] processors => {"models"=>["Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge)", "Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge)"], "count"=>2, "physicalcount"=>1} = [mail2] ps => ps -ef = [mail2] puppetversion => 3.7.2 = [mail2] root_home => /root = [mail2] rsyslog_version => 8.4.2-1+deb8u2 = [mail2] rubysitedir => /usr/local/lib/site_ruby/2.1.0 = [mail2] rubyversion => 2.1.5 = [mail2] selinux => false = [mail2] ssh_version => 6.7p1 = [mail2] swapfree => 16.00 GB = [mail2] swapfree_mb => 16384.00 = [mail2] swapsize => 16.00 GB = [mail2] swapsize_mb => 16384.00 = [mail2] type => Other = [mail2] virtual => kvm = [mail2] ii couchdb 1.6.0+dfsg.1-5~leap80+1 amd64 RESTful document oriented database = [mail2] ii leap-archive-keyring 2017.05.01 all OpenPGP archive key for the leap.se software repositories = [mail2] ii leap-keymanager 0.5.1+0~20160518160549.37+jessie~1.gbp855324 all LEAP's Key Manager = [mail2] ii leap-mx 0.9.1+0~20170627083954.13239+master+jessie~40.g5b754b0 all Asynchronous, transparently-encrypting remailer for the LEAP platform = [mail2] ii pixelated-server 0.2~162.gbpba0081 all Run pixelated-user-agent as a service in multi-user mode. = [mail2] ii pixelated-user-agent 0.6~699.gbpc8f588 all API to serve the pixelated front-end requests = [mail2] ii python-couchdb 0.10-1.1 all library for working with Apache CouchDB = [mail2] ii python-leap-common 0.6.0+0~20170718204908.14939+master+jessie~3.gc35b1e7 all Common Python files needed by LEAP projects = [mail2] ii soledad-client 0.10.0+0~20170725185050.15650+master+jessie~10.ga4ed183 all Synchronization of locally encrypted data among devices (client files) = [mail2] ii soledad-common 0.10.0+0~20170725185050.15650+master+jessie~10.ga4ed183 all Synchronization of locally encrypted data among devices (common files) = [mail2] ii soledad-server 0.10.0+0~20170725185050.15650+master+jessie~10.ga4ed183 all Synchronization of locally encrypted data among devices (server files) = [mail2] ii stunnel4 3:5.30-1~bpo8+1 amd64 Universal SSL tunnel for network daemons = [mail2] pixelat+ 747 0.0 0.7 1132664 63628 ? SLsl Jul28 0:05 /usr/share/python/pixelated-user-agent/bin/python2 /usr/bin/pixelated-user-agent --multi-user -p molgaard.org --leap-provider-cert /etc/ssl/certs/ca-certificates.crt --port 8080 --host 0.0.0.0 --sslkey /etc/x509/keys/leap_commercial.key --sslcert /etc/x509/certs/leap_commercial.crt --banner /etc/pixelated/pixelated_banner.html = [mail2] leap-mx 750 0.0 0.8 234340 67312 ? Ssl Jul28 0:01 /usr/bin/python /usr/bin/twistd -n --rundir=/var/lib/leap_mx/ --python=/usr/share/app/mx.tac --syslog --prefix=leap-mx --pidfile=/tmp/leap-mx.pid = [mail2] couchdb 770 0.0 0.0 4336 1632 ? S Jul28 0:00 /bin/sh -e /usr/bin/couchdb -a /etc/couchdb/default.ini -a /etc/couchdb/local.ini -a /etc/couchdb/local.d/admin.ini -b -r 0 -p /var/run/couchdb/couchdb.pid -o /dev/null -e /dev/null -R = [mail2] couchdb 779 0.0 0.0 4336 156 ? S Jul28 0:00 /bin/sh -e /usr/bin/couchdb -a /etc/couchdb/default.ini -a /etc/couchdb/local.ini -a /etc/couchdb/local.d/admin.ini -b -r 0 -p /var/run/couchdb/couchdb.pid -o /dev/null -e /dev/null -R = [mail2] couchdb 780 1.4 0.5 574960 47756 ? Sl Jul28 3:26 /usr/lib/erlang/erts-6.2/bin/beam.smp -Bd -K true -A 4 -- -root /usr/lib/erlang -progname erl -- -home /var/lib/couchdb -- -noshell -noinput -os_mon start_memsup false start_cpu_sup false disk_space_check_interval 1 disk_almost_full_threshold 1 -sasl errlog_type error -couch_ini /etc/couchdb/default.ini /etc/couchdb/local.ini /etc/couchdb/local.d/admin.ini /etc/couchdb/default.ini /etc/couchdb/local.ini /etc/couchdb/local.d/admin.ini -s couch -pidfile /var/run/couchdb/couchdb.pid -heart = [mail2] stunnel4 997 0.0 0.0 115500 1044 ? Ss Jul28 0:00 /usr/bin/stunnel4 /etc/stunnel/couch_server.conf = [mail2] couchdb 1006 0.0 0.0 8328 752 ? Ss Jul28 0:00 heart -pid 780 -ht 11 = [mail2] couchdb 1111 0.0 0.0 4336 1564 ? Ss Jul28 0:00 sh -s disksup = [mail2] couchdb 7093 0.0 0.1 110780 11676 ? Ssl Jul28 0:00 /usr/bin/couchjs /usr/share/couchdb/server/main.js = [mail2] soledad 8184 0.0 0.7 530632 65080 ? Ssl Jul28 0:03 /usr/bin/python /usr/bin/twistd --nodaemon --pidfile= --syslog --prefix=soledad-server web --class=leap.soledad.server.entrypoint.SoledadEntrypoint --port=ssl:2323:privateKey=/etc/x509/keys/leap.key:certKey=/etc/x509/certs/leap.crt:sslmethod=SSLv23_METHOD = [mail2] leap-we+ 11419 0.0 1.4 424016 114888 ? Sl 00:36 0:00 Passenger RackApp: /srv/leap/webapp = [mail2] root 22198 0.0 0.0 14468 2944 ? Ss 02:32 0:00 /bin/bash /srv/leap/bin/debug.sh = [mail2] root 22336 0.0 0.0 13924 1732 ? S 02:32 0:00 grep -E (leap|pixelated|stunnel|couch|soledad) = [mail2] Last deploy: = [mail2] Jul 28 23:55:54 mail2: STARTING APPLY {user: sune, platform: 0.10, leap_cli: 1.9} = [mail2] Jul 28 23:57:06 mail2: APPLY COMPLETE (changes made) {user: sune, platform: 0.10, leap_cli: 1.9} sune@jekaterina:~/src/pixelated/leap/mail2$