soledad issueshttps://0xacab.org/leap/soledad/-/issues2018-05-30T09:35:48Zhttps://0xacab.org/leap/soledad/-/issues/8925High memory usage in soledad-server2018-05-30T09:35:48ZSune MølgaardHigh memory usage in soledad-serverTrying to set up bitmask on a new machine, I noticed that while sync receiving ~177,000 items, twistd from soledad-server.service on my provider would consume memory up to around 5GB, before releasing it again after sync ended.
I had ac...Trying to set up bitmask on a new machine, I noticed that while sync receiving ~177,000 items, twistd from soledad-server.service on my provider would consume memory up to around 5GB, before releasing it again after sync ended.
I had accidentally deployed 0.10.0, `leap info` yields the following:
`sune@jekaterina:~/src/pixelated/leap/mail2$ leap info
= [mail2] apt_running => false
= [mail2] architecture => amd64
= [mail2] augeasversion => 1.2.0
= [mail2] couchdb_pwhash_alg => pbkdf2
= [mail2] couchdb_version => 1.6.0+dfsg.1-5~leap80+1
= [mail2] debian_codename => jessie
= [mail2] debian_lts => false
= [mail2] debian_nextcodename => stretch
= [mail2] debian_nextrelease => testing
= [mail2] debian_release => stable
= [mail2] dhcp_enabled => true
= [mail2] domain => molgaard.org
= [mail2] facterversion => 2.2.0
= [mail2] filesystems => btrfs,ext2,ext3,ext4
= [mail2] fqdn => mail2.molgaard.org
= [mail2] hardwaremodel => x86_64
= [mail2] hostname => mail2
= [mail2] interface_127.0.0.1 => lo
= [mail2] interface_172.31.1.100 => eth0
= [mail2] interfaces => eth0,lo
= [mail2] ipaddress => 172.31.1.100
= [mail2] ipaddress_eth0 => 172.31.1.100
= [mail2] ipaddress_lo => 127.0.0.1
= [mail2] is_pe => false
= [mail2] is_virtual => true
= [mail2] kernel => Linux
= [mail2] kernelmajversion => 3.16
= [mail2] kernelrelease => 3.16.0-4-amd64
= [mail2] kernelversion => 3.16.0
= [mail2] lib => /srv/leap/puppet/modules/apache/lib/facter:/srv/leap/puppet/modules/apt/lib/facter:/srv/leap/puppet/modules/concat/lib/facter:/srv/leap/puppet/modules/couchdb/lib/facter:/srv/leap/puppet/modules/rsyslog/lib/facter:/srv/leap/puppet/modules/site_config/lib/facter:/srv/leap/puppet/modules/sshd/lib/facter:/srv/leap/puppet/modules/stdlib/lib/facter
= [mail2] lsbdistcodename => jessie
= [mail2] lsbdistdescription => Debian GNU/Linux 8.9 (jessie)
= [mail2] lsbdistid => Debian
= [mail2] lsbdistrelease => 8.9
= [mail2] lsbmajdistrelease => 8
= [mail2] lsbminordistrelease => 9
= [mail2] memoryfree => 6.66 GB
= [mail2] memoryfree_mb => 6822.91
= [mail2] memorysize => 7.82 GB
= [mail2] memorysize_mb => 8005.68
= [mail2] mtu_eth0 => 1500
= [mail2] mtu_lo => 65536
= [mail2] netmask => 255.255.255.0
= [mail2] netmask_cidr_eth0 => 24
= [mail2] netmask_cidr_lo => 8
= [mail2] netmask_eth0 => 255.255.255.0
= [mail2] netmask_lo => 255.0.0.0
= [mail2] network_eth0 => 172.31.1.0
= [mail2] network_lo => 127.0.0.0
= [mail2] operatingsystem => Debian
= [mail2] os => {"name"=>"Debian", "family"=>"Debian", "release"=>{"major"=>"8", "minor"=>"9", "full"=>"8.9"}, "lsb"=>{"distcodename"=>"jessie", "distid"=>"Debian", "distdescription"=>"Debian GNU/Linux 8.9 (jessie)", "distrelease"=>"8.9", "majdistrelease"=>"8", "minordistrelease"=>"9"}}
= [mail2] osfamily => Debian
= [mail2] path => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
= [mail2] physicalprocessorcount => 1
= [mail2] processor0 => Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge)
= [mail2] processor1 => Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge)
= [mail2] processorcount => 2
= [mail2] processors => {"models"=>["Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge)", "Intel(R) Xeon(R) CPU E5-26xx (Sandy Bridge)"], "count"=>2, "physicalcount"=>1}
= [mail2] ps => ps -ef
= [mail2] puppetversion => 3.7.2
= [mail2] root_home => /root
= [mail2] rsyslog_version => 8.4.2-1+deb8u2
= [mail2] rubysitedir => /usr/local/lib/site_ruby/2.1.0
= [mail2] rubyversion => 2.1.5
= [mail2] selinux => false
= [mail2] ssh_version => 6.7p1
= [mail2] swapfree => 16.00 GB
= [mail2] swapfree_mb => 16384.00
= [mail2] swapsize => 16.00 GB
= [mail2] swapsize_mb => 16384.00
= [mail2] type => Other
= [mail2] virtual => kvm
= [mail2] ii couchdb 1.6.0+dfsg.1-5~leap80+1 amd64 RESTful document oriented database
= [mail2] ii leap-archive-keyring 2017.05.01 all OpenPGP archive key for the leap.se software repositories
= [mail2] ii leap-keymanager 0.5.1+0~20160518160549.37+jessie~1.gbp855324 all LEAP's Key Manager
= [mail2] ii leap-mx 0.9.1+0~20170627083954.13239+master+jessie~40.g5b754b0 all Asynchronous, transparently-encrypting remailer for the LEAP platform
= [mail2] ii pixelated-server 0.2~162.gbpba0081 all Run pixelated-user-agent as a service in multi-user mode.
= [mail2] ii pixelated-user-agent 0.6~699.gbpc8f588 all API to serve the pixelated front-end requests
= [mail2] ii python-couchdb 0.10-1.1 all library for working with Apache CouchDB
= [mail2] ii python-leap-common 0.6.0+0~20170718204908.14939+master+jessie~3.gc35b1e7 all Common Python files needed by LEAP projects
= [mail2] ii soledad-client 0.10.0+0~20170725185050.15650+master+jessie~10.ga4ed183 all Synchronization of locally encrypted data among devices (client files)
= [mail2] ii soledad-common 0.10.0+0~20170725185050.15650+master+jessie~10.ga4ed183 all Synchronization of locally encrypted data among devices (common files)
= [mail2] ii soledad-server 0.10.0+0~20170725185050.15650+master+jessie~10.ga4ed183 all Synchronization of locally encrypted data among devices (server files)
= [mail2] ii stunnel4 3:5.30-1~bpo8+1 amd64 Universal SSL tunnel for network daemons
= [mail2] pixelat+ 747 0.0 0.7 1132664 63628 ? SLsl Jul28 0:05 /usr/share/python/pixelated-user-agent/bin/python2 /usr/bin/pixelated-user-agent --multi-user -p molgaard.org --leap-provider-cert /etc/ssl/certs/ca-certificates.crt --port 8080 --host 0.0.0.0 --sslkey /etc/x509/keys/leap_commercial.key --sslcert /etc/x509/certs/leap_commercial.crt --banner /etc/pixelated/pixelated_banner.html
= [mail2] leap-mx 750 0.0 0.8 234340 67312 ? Ssl Jul28 0:01 /usr/bin/python /usr/bin/twistd -n --rundir=/var/lib/leap_mx/ --python=/usr/share/app/mx.tac --syslog --prefix=leap-mx --pidfile=/tmp/leap-mx.pid
= [mail2] couchdb 770 0.0 0.0 4336 1632 ? S Jul28 0:00 /bin/sh -e /usr/bin/couchdb -a /etc/couchdb/default.ini -a /etc/couchdb/local.ini -a /etc/couchdb/local.d/admin.ini -b -r 0 -p /var/run/couchdb/couchdb.pid -o /dev/null -e /dev/null -R
= [mail2] couchdb 779 0.0 0.0 4336 156 ? S Jul28 0:00 /bin/sh -e /usr/bin/couchdb -a /etc/couchdb/default.ini -a /etc/couchdb/local.ini -a /etc/couchdb/local.d/admin.ini -b -r 0 -p /var/run/couchdb/couchdb.pid -o /dev/null -e /dev/null -R
= [mail2] couchdb 780 1.4 0.5 574960 47756 ? Sl Jul28 3:26 /usr/lib/erlang/erts-6.2/bin/beam.smp -Bd -K true -A 4 -- -root /usr/lib/erlang -progname erl -- -home /var/lib/couchdb -- -noshell -noinput -os_mon start_memsup false start_cpu_sup false disk_space_check_interval 1 disk_almost_full_threshold 1 -sasl errlog_type error -couch_ini /etc/couchdb/default.ini /etc/couchdb/local.ini /etc/couchdb/local.d/admin.ini /etc/couchdb/default.ini /etc/couchdb/local.ini /etc/couchdb/local.d/admin.ini -s couch -pidfile /var/run/couchdb/couchdb.pid -heart
= [mail2] stunnel4 997 0.0 0.0 115500 1044 ? Ss Jul28 0:00 /usr/bin/stunnel4 /etc/stunnel/couch_server.conf
= [mail2] couchdb 1006 0.0 0.0 8328 752 ? Ss Jul28 0:00 heart -pid 780 -ht 11
= [mail2] couchdb 1111 0.0 0.0 4336 1564 ? Ss Jul28 0:00 sh -s disksup
= [mail2] couchdb 7093 0.0 0.1 110780 11676 ? Ssl Jul28 0:00 /usr/bin/couchjs /usr/share/couchdb/server/main.js
= [mail2] soledad 8184 0.0 0.7 530632 65080 ? Ssl Jul28 0:03 /usr/bin/python /usr/bin/twistd --nodaemon --pidfile= --syslog --prefix=soledad-server web --class=leap.soledad.server.entrypoint.SoledadEntrypoint --port=ssl:2323:privateKey=/etc/x509/keys/leap.key:certKey=/etc/x509/certs/leap.crt:sslmethod=SSLv23_METHOD
= [mail2] leap-we+ 11419 0.0 1.4 424016 114888 ? Sl 00:36 0:00 Passenger RackApp: /srv/leap/webapp
= [mail2] root 22198 0.0 0.0 14468 2944 ? Ss 02:32 0:00 /bin/bash /srv/leap/bin/debug.sh
= [mail2] root 22336 0.0 0.0 13924 1732 ? S 02:32 0:00 grep -E (leap|pixelated|stunnel|couch|soledad)
= [mail2] Last deploy:
= [mail2] Jul 28 23:55:54 mail2: STARTING APPLY {user: sune, platform: 0.10, leap_cli: 1.9}
= [mail2] Jul 28 23:57:06 mail2: APPLY COMPLETE (changes made) {user: sune, platform: 0.10, leap_cli: 1.9}
sune@jekaterina:~/src/pixelated/leap/mail2$`Cycle 10https://0xacab.org/leap/soledad/-/issues/9018Enable multi-core usage for the Blobs Backend2018-02-14T15:47:54ZdrebsEnable multi-core usage for the Blobs Backend- [ ] #9021 - Investigate pypy for multi-core Soledad Server
- [x] #9022 - Investigate possibilities of multiprocessing for Soledad Server
- [x] #9025 - Modify the blobs file system backend to account for concurrency
- [x] #9026 - Refact...- [ ] #9021 - Investigate pypy for multi-core Soledad Server
- [x] #9022 - Investigate possibilities of multiprocessing for Soledad Server
- [x] #9025 - Modify the blobs file system backend to account for concurrency
- [x] #9026 - Refactor Soledad Server entrypoint to start several processes
- [ ] #9028 - Prototype a multiprocessing solution using a separate blobs server
- [x] #9029 - Benchmark proxied soledad-blobs-serverCycle 10https://0xacab.org/leap/soledad/-/issues/9032Create a roadmap for 20182018-02-06T19:40:10ZdrebsCreate a roadmap for 2018Things we know we want (in no specific order):
- Integration of blobs into bitmask.mail (on demand download of message bodies).
- Search index.
- Sync heuristics.
- Python 3 port.
- Improve standalone use, modularize auth, create standa...Things we know we want (in no specific order):
- Integration of blobs into bitmask.mail (on demand download of message bodies).
- Search index.
- Sync heuristics.
- Python 3 port.
- Improve standalone use, modularize auth, create standalone example app (password manager?).
- Improve Thunderbird integration.
- Research/implement mobile client app.
- Cleanup, define and apply code standards.
- Integration with the platform (https://0xacab.org/leap/platform/issues/8854).
- Add recovery code, eliminate shared db.
Here is a wiki page where we can come up with a roadmap for 2019: https://0xacab.org/leap/soledad/wikis/2018-roadmaphttps://0xacab.org/leap/soledad/-/issues/9030Improve the blobs server proxy performance2018-02-06T18:51:21ZdrebsImprove the blobs server proxy performanceIn #9029 we can see that the Twisted based proxy proposed in !180 introduces serious overhead. We have to better understand why and come up with an improved solution for now.In #9029 we can see that the Twisted based proxy proposed in !180 introduces serious overhead. We have to better understand why and come up with an improved solution for now.Cycle 10https://0xacab.org/leap/soledad/-/issues/9031soledad server crashed2018-02-05T12:14:21Zkwadronautsoledad server crashedgot loads of test error mails from cdev, leap test told me this, I restarted soledad-server.
```
= [giraffe] FAIL: Soledad > Is Soledad running? [soledad.rb:13]:
= [giraffe] Service 'soledad-server' is not running:
= [giraffe] ● ...got loads of test error mails from cdev, leap test told me this, I restarted soledad-server.
```
= [giraffe] FAIL: Soledad > Is Soledad running? [soledad.rb:13]:
= [giraffe] Service 'soledad-server' is not running:
= [giraffe] ● soledad-server.service - Soledad Server
= [giraffe] Loaded: loaded (/lib/systemd/system/soledad-server.service; enabled)
= [giraffe] Active: inactive (dead) since Wed 2018-01-24 22:10:22 UTC; 1 weeks 3 days ago
= [giraffe] Process: 13203 ExecStart=/usr/bin/soledad-server (code=exited, status=0/SUCCESS)
= [giraffe] Main PID: 13203 (code=exited, status=0/SUCCESS)
= [giraffe] 12 tests: 8 passes, 0 skips, 0 warnings, 2 failures, 0 errors
```
In the logs: Jan 25 13:24:12 giraffe systemd[1]: Stopped Soledad Server.
I didn't investigate further, because I don't know the state of changes in those machines.https://0xacab.org/leap/soledad/-/issues/9019Rewrite of the metadata sync engine to use asynchronous couchdb libraries2018-01-27T19:56:23ZdrebsRewrite of the metadata sync engine to use asynchronous couchdb librariesCycle 10https://0xacab.org/leap/soledad/-/issues/9028Prototype a multiprocessing solution using a separate blobs server2018-01-23T18:35:56ZdrebsPrototype a multiprocessing solution using a separate blobs serverBecause we will not modify the platform now, we want an intermediate solution for the multiprocessing problem that will allow for usage with the current server and will also be usable when we migrate to platform-based routing.
This is t...Because we will not modify the platform now, we want an intermediate solution for the multiprocessing problem that will allow for usage with the current server and will also be usable when we migrate to platform-based routing.
This is the proposed structure for theis solution:
* business as usual, if not configured (everything runs in the same process).
* if soledad-blobs-detached-server configured, then it's responsibility of sysadmin to run `soledad-blobs-server`.
* if `/sync`, do as usual.
* if `/blobs` or `/incoming` or `/stream`, delegate request to local `soledad-blobs-server` port.
Obs: `soledad-blobs-server` is a new entrypoint, very lightweight (no weird dependencies, just Twisted). It is a router that spawns multi process and worries about task coordination.Cycle 10https://0xacab.org/leap/soledad/-/issues/9021Investigate pypy for multi-core Soledad Server2017-12-27T17:50:00ZdrebsInvestigate pypy for multi-core Soledad ServerCycle 10https://0xacab.org/leap/soledad/-/issues/9024Running tests with pypy fails with2017-12-27T17:49:41ZdrebsRunning tests with pypy fails withSteps to reproduce:
* Checkout current master (5edf23deefca71b632b2cdf2b686c478bc7dcce2)
* Set `envlist = py27, pypy` in `tox.ini`.
* Remove `basepython` from `[testenv]` in `tox.ini`.
* Run: `tox -e pypy`.
I get the following:
```
__...Steps to reproduce:
* Checkout current master (5edf23deefca71b632b2cdf2b686c478bc7dcce2)
* Set `envlist = py27, pypy` in `tox.ini`.
* Remove `basepython` from `[testenv]` in `tox.ini`.
* Run: `tox -e pypy`.
I get the following:
```
________________________________________ BlobManagerTestCase.test_get_doesnt_include_unavailable_blobs _________________________________________
NOTE: Incompatible Exception Representation, displaying natively:
FirstError: FirstError[#1, [Failure instance: Traceback: <class 'pysqlcipher.dbapi2.DatabaseError'>: file is encrypted or is not a database
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/internet/defer.py:500:errback
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/internet/defer.py:567:_startRunCallbacks
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/internet/defer.py:653:_runCallbacks
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/internet/defer.py:1442:gotResult
--- <exception caught here> ---
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/internet/defer.py:1384:_inlineCallbacks
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/python/failure.py:408:throwExceptionIntoGenerator
/home/drebs/dev/leap/repos/soledad/src/leap/soledad/client/_db/blobs/sql.py:165:update_sync_status
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/python/threadpool.py:250:inContext
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/python/threadpool.py:266:<lambda>
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/python/context.py:122:callWithContext
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/python/context.py:85:callWithContext
/home/drebs/dev/leap/repos/soledad/src/leap/soledad/client/_db/blobs/sql.py:365:_runInteraction
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/enterprise/adbapi.py:464:_runInteraction
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/enterprise/adbapi.py:36:__init__
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/enterprise/adbapi.py:76:reconnect
/home/drebs/dev/leap/repos/soledad/.tox/pypy/site-packages/twisted/enterprise/adbapi.py:433:connect
/usr/lib/pypy/lib_pypy/_functools.py:45:__call__
/home/drebs/dev/leap/repos/soledad/src/leap/soledad/client/_db/pragmas.py:45:set_init_pragmas
/home/drebs/dev/leap/repos/soledad/src/leap/soledad/client/_db/pragmas.py:59:_set_init_pragmas
/home/drebs/dev/leap/repos/soledad/src/leap/soledad/client/_db/pragmas.py:368:set_write_ahead_logging
]]
```https://0xacab.org/leap/soledad/-/issues/9020Document ciphertext size estimation2017-12-26T10:49:32ZdrebsDocument ciphertext size estimationSoledad Backloghttps://0xacab.org/leap/soledad/-/issues/9005Discuss security of blobs and decide next steps2017-12-24T23:31:36ZdrebsDiscuss security of blobs and decide next stepsThis is about the 2 following items of Activity 4 of [the roadmap](https://0xacab.org/leap/soledad/wikis/2017-roadmap):
- Final analysis of security considerations (analysis of the padding, sanitization of input, etc)
- Address the most...This is about the 2 following items of Activity 4 of [the roadmap](https://0xacab.org/leap/soledad/wikis/2017-roadmap):
- Final analysis of security considerations (analysis of the padding, sanitization of input, etc)
- Address the most important security concerns
We have to send an email to the list to discuss and decide how to proceed.Cycle 10https://0xacab.org/leap/soledad/-/issues/8962Evaluate/decide blobs consistency model and relation with old sync2017-12-22T17:21:52ZdrebsEvaluate/decide blobs consistency model and relation with old syncWe have to define the consistency model for blobs:
- should blobs be associated with documents?
- which level of consistency does soledad guarantee?We have to define the consistency model for blobs:
- should blobs be associated with documents?
- which level of consistency does soledad guarantee?Soledad Backloghttps://0xacab.org/leap/soledad/-/issues/9002Verify exceeded quota pipeline2017-12-22T17:20:35ZdrebsVerify exceeded quota pipelineI found the cdev's MX node stalled while trying to delivery mail to the e2e test user:
```
(...)
Nov 28 12:31:19 lynx leap-mx: [-] Waiting for the process of skipped mail to be done...
Nov 28 12:31:29 lynx leap-mx: [-] Waiting for the p...I found the cdev's MX node stalled while trying to delivery mail to the e2e test user:
```
(...)
Nov 28 12:31:19 lynx leap-mx: [-] Waiting for the process of skipped mail to be done...
Nov 28 12:31:29 lynx leap-mx: [-] Waiting for the process of skipped mail to be done...
Nov 28 12:31:29 lynx leap-mx: [-] Waiting for the process of skipped mail to be done...
```
Looking at Soledad Server, it seems that it was also stalling while trying to reply new blob delivery attempts using the Incoming API. After restarting soledad-server, i saw many delivery attempts and many quota-exceeded messages like this:
```
Nov 28 12:42:00 giraffe soledad-server: [leap.soledad.server._blobs#error] Error 507: Quota exceeded for user: a12ae9727e9ed3511e6464128c72aaa6
Nov 28 12:42:00 giraffe soledad-server: [-] 0.0.0.0 - - [28/Nov/2017:12:41:59 +0000] "PUT /incoming/a12ae9727e9ed3511e6464128c72aaa6/8deb3487-3828-4f4c-b792-3f673aedd249 HTTP/1.1" 507 32 "-" "-"
```
It felt like as if some endpoint was not releasing the connection correctly (don't know if on MX or Soledad Server side), but I am not sure if that is the problem.
We should:
- [ ] ensure that the delivered blobs to e2e are removed after the test runs.
- [ ] double check that the quota-exceeded responses are returning and releasing resources correctly (both in MX and Soledad Server sides).Soledad Backloghttps://0xacab.org/leap/soledad/-/issues/9014Separate configuration of kibana/elasticsearch from benchmarks html website u...2017-12-22T17:20:16ZdrebsSeparate configuration of kibana/elasticsearch from benchmarks html website updateCurrently, there's one script configured in **moose.leap.se**'s cron for both configuring kibana/elasticsearch and updating the benchmarks html website. Because the process is lengthy and they are separate processes, they should be split...Currently, there's one script configured in **moose.leap.se**'s cron for both configuring kibana/elasticsearch and updating the benchmarks html website. Because the process is lengthy and they are separate processes, they should be split in 2 scripts and configured independently in cron (eg: configure kibana/elastic hourly and update the website every 5 minutes).Soledad Backloghttps://0xacab.org/leap/soledad/-/issues/9008Discuss scalable backends and priorities2017-12-22T17:15:07ZdrebsDiscuss scalable backends and prioritiesCycle 10https://0xacab.org/leap/soledad/-/issues/9015CI packaging job fails with "ERROR: Job failed (system failure): aborted: ter...2017-12-18T11:49:23ZdrebsCI packaging job fails with "ERROR: Job failed (system failure): aborted: terminated"From: https://0xacab.org/drebs/soledad/-/jobs/29861
```
00:13:48 Unpacking linux-headers-4.13.0-1-common (4.13.13-1) over (4.13.4-2) ...
ERROR: Job failed (system failure): aborted: terminated
```From: https://0xacab.org/drebs/soledad/-/jobs/29861
```
00:13:48 Unpacking linux-headers-4.13.0-1-common (4.13.13-1) over (4.13.4-2) ...
ERROR: Job failed (system failure): aborted: terminated
```https://0xacab.org/leap/soledad/-/issues/8983Dashboard doesn't recognize failing master build2017-12-04T18:59:58ZVaracDashboard doesn't recognize failing master buildThe dashboard checks the last run pipeline for success. Now that we have scheduled benchmark jobs that run alone by themselves (see https://0xacab.org/leap/soledad/pipelines), the latest pipeline will most always succeed.
We need to che...The dashboard checks the last run pipeline for success. Now that we have scheduled benchmark jobs that run alone by themselves (see https://0xacab.org/leap/soledad/pipelines), the latest pipeline will most always succeed.
We need to check the last *unscheduled* pipeline results.
Atm, the last *unscheduled* pipeline [failed](https://0xacab.org/leap/soledad/pipelines/6070)!https://0xacab.org/leap/soledad/-/issues/8863Limit benchmarking graphs to a certain date range2017-12-04T18:58:33ZdrebsLimit benchmarking graphs to a certain date rangeEvaluate the options for limiting the X axis of the benchmarking graphs by date. The problematic part is that the X "buckets" are defined by a custom search, and concatenate the commit id with the commit date. Also, see if we can clean u...Evaluate the options for limiting the X axis of the benchmarking graphs by date. The problematic part is that the X "buckets" are defined by a custom search, and concatenate the commit id with the commit date. Also, see if we can clean up the labels of the X axis (it is unreadable right now).Soledad Backloghttps://0xacab.org/leap/soledad/-/issues/8934Move ports configuration to soledad server configuration file2017-12-04T18:57:23ZdrebsMove ports configuration to soledad server configuration fileCurrently, the configuration of ports which soledad listen on are passed to the application through environment variables set in the systemd service file. This is inconvenient because other soledad server configurations are read from a c...Currently, the configuration of ports which soledad listen on are passed to the application through environment variables set in the systemd service file. This is inconvenient because other soledad server configurations are read from a configuration file in `/etc`.
It would make deployment easier if all configurations would be in the server configuration file. It would also help in the future when we want to only listen on localhost and delegate TLS and tunelling to something like nginx.
We should also move the certificate and TLS version configurations (all related to the port config) to the server config file.https://0xacab.org/leap/soledad/-/issues/8998Review and standardize Soledad Server environment variables and configuration...2017-12-04T18:57:12ZdrebsReview and standardize Soledad Server environment variables and configuration optionsDifferent environment variables were added during the history of development of Soledad Server that affect the server in different ways. They are all currently documented in the `soledad-server` command man page and in the online documen...Different environment variables were added during the history of development of Soledad Server that affect the server in different ways. They are all currently documented in the `soledad-server` command man page and in the online documentation. Despite that, some variables are prefixed with `SOLEDAD_`, some with `SOLEDAD_SERVER` and some with nothing (i.e. `HTTPS_PORT`). Also, some of them replace specific configuration options that are available in the config file (i.e. `SOLEDAD_COUCH_URL`.
We have to review and standardize the environment variables naming, and maybe it's precedence over configuration options in the config file.