CHANGELOG.rst 11 KB
Newer Older
1
2
3
Changelog
=========

4
0.10.8 - `master`_
5
6
7
8
-------------------

.. note:: This version is not yet released and is under active development.

9
10
11
12
13
14
15
16
17
0.10.7 - Tue 3 Jul, 2018
-------------------------

Misc
~~~~
- Fix errors in setup.py avoiding the package from being uploaded to pypi
- Fix author in debian changelog


Kali Kaneko's avatar
Kali Kaneko committed
18
19
20
21
22
23
24
25
26
27
28
29
0.10.6 - Fri 29 Jun, 2018
-------------------------

Server
~~~~~~
- [feature] Improvements to the Blobs Filesystem backend

Client
~~~~~~
- [feature] Add bandwidth throttling to the blobs client
- [docs] Clarify offline use for tests

30

drebs's avatar
drebs committed
31
32
33
34
35
36
37
38
39
40
0.10.5 -  Wed 22 Nov, 2017
-------------------------------

Server 
~~~~~~

- [feature] add the SOLEDAD_COUCH_URL environment variable on server side
- [bug] properly shutdown server if startup checks fail
- [bug] fix and improve logging of server startup checks

drebs's avatar
drebs committed
41
0.10.4 -  Wed 15 Nov, 2017
drebs's avatar
drebs committed
42
-------------------------------
43

drebs's avatar
drebs committed
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
Server
~~~~~~

- [feature] improve server entrypoint (several refactors and introduction of
  binary in debian package)
- [feature] improve speed of server startup
- [bug] wait for couch schema, configuration and environment checks before
  running servers.
- [bug] debian package now reports the correct version
- [pkg] improve user db creation script (fixes and man page)
- [bug] limit concurrent blob writes in server

Client
~~~~~~

- [feature] support unsynced local only blobs
- [feature] add/fix blob deletion
- [feature] add and control concurrency of blobs local and remote operations
- [feature] add retries for blob transfers
- [bug] fixes in blob download/upload pipeline
- [bug] improve resilience of blobs concurrent access to sqlcipher
- [bug] fix blobs preamble flakiness
- [refactor] split blobs backend in many modules
67

68
69
70
Misc
~~~~

drebs's avatar
drebs committed
71
72
73
74
75
76
77
- [doc] many documentation improvements
- [doc] documentation was mirrored in https://leap.se/en/docs/design/soledad
- [benchmarks] add server scalability tests
- [benchmarks] add outlier detection
- [test] add e2e test for incoming mail pipeline
- [pkg] Add packages for debian buster.
- [pkg] deb: Make soledad-client depend on soledad-common
78
79


80
81
82
0.10.3 - Mon 11 Sep, 2017
----------------------------------

Kali Kaneko's avatar
Kali Kaneko committed
83
84
85
Server
~~~~~~

Victor's avatar
Victor committed
86
- [feat] Finished adding support for Incoming API
87
88
89
90
- [feat] Get config file name from environment variable.
- [bug] Add DELETE method to url mapper.
- [bug] Use correct keyword argument for server state initialization.
- `#8924 <https://0xacab.org/leap/soledad/issues/8924>`_: [bug] FileBodyProducer consumer usage wasn't closing the file
Kali Kaneko's avatar
Kali Kaneko committed
91
92
93
94

Client
~~~~~~

95
- [feat] Add columns for sync state of blobs inside sqlcipher
Kali Kaneko's avatar
Kali Kaneko committed
96
97
- [bug] Several bugfixes for BlobManager initialization.
- [bug] Fix usage of StringIO class in gzip middleware.
98
- `#8924 <https://0xacab.org/leap/soledad/issues/8924>`_: [bug] FileBodyProducer consumer usage wasn't closing the file
Kali Kaneko's avatar
Kali Kaneko committed
99
100
101
102
103
104

Misc
~~~~

- Use latest version of pytest-benchmark.
- Find correct twistd when outside tox envs
Kali Kaneko's avatar
Kali Kaneko committed
105
- Build packages for zesty and stretch.
106
107
- Add benchmark comparing legacy vs blobs sync.
- Add reactor responsiveness tests.
Kali Kaneko's avatar
Kali Kaneko committed
108
109


110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
0.10.2 - Mon 21 Aug, 2017
----------------------------------

Server
~~~~~~

- Enforce namespace to default on server
- Add path partitioning to namespaces

Client
~~~~~~

- Add namespace to local blobs db table
- Track namespace information on blobs client

125
126
127
128

0.10.1 - Mon 07 Aug, 2017
---------------------------------

129
130
131
132
133
134
135
136
137
138
139
140
141
Server
~~~~~~

- Fixes IncomingBox missing preamble separator (space) which causes client to
  fail parsing.

Client
~~~~~~

- Adds IncomingBoxProcessLoop and implement the process flow for IncominBox
  specification.
- Adds IIncomingBoxConsumer interface, which can be used by Soledad apps to
  implement consumers for IncomingBox feature.
142
143
144
145
146
147

0.10.0 - 18 July, 2017
-------------------------------

Server
~~~~~~
drebs's avatar
drebs committed
148
149
150
151
152
153
154
155
156
157
158

- Add an incoming API for email delivery. In the future, this may be used by
  external applications for message delivery.
- Add namespace capability.
- List incoming blobs in chronological order.
- Finish minimal filesystem backend for blobs.
- Update BlobManager to support new server features, such as: namespaces,
  incoming and listing.
- Make the backend configurable for incoming API, so it can use CouchDB now and
  Blobs later.

159
160
161
Client
~~~~~~

drebs's avatar
drebs committed
162
- Use OpenSSL backend for scrypt if OpenSSL >= 1.1
163
164
165
166

Misc
~~~~

drebs's avatar
drebs committed
167
168
- Refactor preamble to account for PGP encryption scheme
- Removes scrypt dependency
169
- Unification of Client, Server and Common in a Single python package.
drebs's avatar
drebs committed
170
171
- Build soledad debian package with git-buildpackage.
- Document deprecation policy.
172
- Documentation is automatically uploaded to: https://soledad.readthedocs.io/
drebs's avatar
drebs committed
173
- Launch benchmarks website: https://benchmarks.leap.se/
174

Kali Kaneko's avatar
Kali Kaneko committed
175
0.9.6 - 31 May, 2017
drebs's avatar
drebs committed
176
-------------------------------
Kali Kaneko's avatar
Kali Kaneko committed
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

Server
~~~~~~

- Minimal Filesystem BlobsBackend implementation, disabled by default.

Client
~~~~~~

- Minimal Blobs manager implementation
- Blobs API
- Ability to generate recovery code.
- Fix deprecated multibackend call (cryptography).

Misc
~~~~~~

- Post benchmark results to elasticsearch
- Build docker image and push it to registry every time the dockerfile used for
  tests is changed
- Fix flaky tests
- Cleanup old documentation.
- Added dependency on treq.
- Improve cpu/memory profiling.
201
202
- Bumped version to upload wheels to pypi, to workaround for dbschema.sql not
  found after installation in virtualenv.
Kali Kaneko's avatar
Kali Kaneko committed
203
204
205


0.9.5 -  17 March, 2017
drebs's avatar
drebs committed
206
-------------------------------
Kali Kaneko's avatar
Kali Kaneko committed
207

Kali Kaneko's avatar
Kali Kaneko committed
208
209
210
211
212
213
214
215
216
217
218
Server
~~~~~~
- Make database creation appear in logs

Client
~~~~~~
- `#8721 <https://0xacab.org/leap/soledad/issues/8721>`_: Remove offline flag
- Fix raising of invalid auth token error
- Add default version when decrypting secrets
- Secrets version defaults to v1

219
220
221
222
Misc
~~~~
- First steps porting soledad to python3

Kali Kaneko's avatar
Kali Kaneko committed
223
0.9.3 -  06 March, 2017
drebs's avatar
drebs committed
224
-------------------------------
Kali Kaneko's avatar
Kali Kaneko committed
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246

Server
~~~~~~
- Refactor authentication code to use twisted credential system.
- Announce server blobs capabilities
- `#8764 <https://0xacab.org/leap/soledad/issues/8764>`_: Allow unauthenticated users to retrieve the capabilties banner.
- `#6178 <https://0xacab.org/leap/soledad/issues/6178>`_: Add robots.txt
- #8762: Add a systemd service file
- Add script to deploy from git

Client
~~~~~~~~
- `#8758 <https://0xacab.org/leap/soledad/issues/8758>`_: Add blob size to the crypto preamble
- Improve secrets generation and storage code
- Add offline status to soledad client api.
- Remove syncable property

Misc
~~~~
- Improvements in performance benchmarks.


drebs's avatar
drebs committed
247
0.9.2 - 22 December, 2016
drebs's avatar
drebs committed
248
-------------------------
drebs's avatar
drebs committed
249
250
251
252

Performance improvements
~~~~~~~~~~~~~~~~~~~~~~~~

drebs's avatar
drebs committed
253
- use AES 256 GCM mode instead of CTR-HMAC.
drebs's avatar
drebs committed
254
255
256
257
258
259
260
261
262
263
264
- streaming encryption/decryption and data transfer.

Server
~~~~~~

- move server to a twisted resource entrypoint.

Client
~~~~~~

- use twisted http agent in the client.
drebs's avatar
drebs committed
265
- maintain backwards compatibility with old crypto scheme (AES 256 CTR-HMAC).
drebs's avatar
drebs committed
266
267
268
269
  No migration for now, only in 0.10.
- remove the encryption/decryption pools, replace for inline streaming crypto.
- use sqlcipher transactions on sync.

drebs's avatar
drebs committed
270
0.9.1 - 27 November, 2016
drebs's avatar
drebs committed
271
-------------------------
drebs's avatar
drebs committed
272
273
274
275
276
277
278
279
280
281

Server side bug fixes
~~~~~~~~~~~~~~~~~~~~~

- fix import on create-user-db script
- patch twisted logger so it works with twistd --syslog
- delay couch state initialization
- improve missing couch config doc error logging
- separate server application into another file

drebs's avatar
drebs committed
282
0.9.0 - 11 November, 2016
drebs's avatar
drebs committed
283
-------------------------
drebs's avatar
drebs committed
284
285
286
287
288
289
290
291
292

Main features
~~~~~~~~~~~~~

- Server-side changes in couch backend schema.
- Use of tox and pytest to run tests.
- Performance tests.

Server
drebs's avatar
drebs committed
293
~~~~~~
drebs's avatar
drebs committed
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319

*** Attention: Migration needed! ***

This version of soledad uses a different database schema in the server couch
backend. The difference from the old schema is that the use of design documents
for storing and accessing soledad db metadata was removed because incurred in
too much memory and time overhead for passing data to the javascript
interpreter.

Because of that, you need to run a migration script on your database. Check the
`scripts/migration/0.9.0/` diretctory for instructions on how to run the
migration script on your database. Don't forget to backup before running the
script!

Bugfixes
~~~~~~~~
- Fix order of multipart serialization when writing to couch.

Features
~~~~~~~~
- Log to syslog.
- Remove usage of design documents in couch backend.
- Use _local couch docs for metadata storage.
- Other small improvements in couch backend.


Kali Kaneko's avatar
Kali Kaneko committed
320
0.8.1 - 14 July, 2016
drebs's avatar
drebs committed
321
---------------------
Kali Kaneko's avatar
Kali Kaneko committed
322
323

Client
drebs's avatar
drebs committed
324
~~~~~~
Kali Kaneko's avatar
Kali Kaneko committed
325
326

Features
drebs's avatar
drebs committed
327
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
328
329
330
331
332
- Add recovery document format version for future migrations.
- Use DeferredLock instead of its locking cousin.
- Use DeferredSemaphore instead of its locking cousin.

Bugfixes
drebs's avatar
drebs committed
333
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
334
335
336
337
338
- `#8180 <https://leap.se/code/issues/8180>`_: Initialize OpenSSL context just once.
- Remove document content conversion to unicode. Users of API are responsible
  for only passing valid JSON to Soledad for storage.

Misc
drebs's avatar
drebs committed
339
++++
Kali Kaneko's avatar
Kali Kaneko committed
340
341
342
343
344
345
346
- Add ability to get information about sync phases for profiling purposes.
- Add script for setting up develop environment.
- Refactor bootstrap to remove shared db lock.
- Removed multiprocessing from encdecpool with some extra refactoring.
- Remove user_id argument from Soledad init.

Common
drebs's avatar
drebs committed
347
~~~~~~
Kali Kaneko's avatar
Kali Kaneko committed
348
349

Features
drebs's avatar
drebs committed
350
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
351
352
353
- Embed l2db, forking u1db.

Misc
drebs's avatar
drebs committed
354
++++
Kali Kaneko's avatar
Kali Kaneko committed
355
356
- Toxify tests.

Kali Kaneko's avatar
Kali Kaneko committed
357
0.8.0 - 18 Apr, 2016
drebs's avatar
drebs committed
358
--------------------
Kali Kaneko's avatar
Kali Kaneko committed
359
360

Client
drebs's avatar
drebs committed
361
~~~~~~
Kali Kaneko's avatar
Kali Kaneko committed
362
363

Features
drebs's avatar
drebs committed
364
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
365
366
367
368
- `#7656 <https://leap.se/code/issues/7656>`_: Emit multi-user aware events.
- Client will now send documents at a limited size batch due to changes on SyncTarget. The default limit is 500kB. Disabled by default.

Bugfixes
drebs's avatar
drebs committed
369
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
370
371
372
373
- `#7503 <https://leap.se/code/issues/7503>`_: Do not signal sync completion if sync failed.
- Handle missing design doc at GET (get_sync_info). Soledad server can handle this during sync.

Misc
drebs's avatar
drebs committed
374
++++
Kali Kaneko's avatar
Kali Kaneko committed
375
376
377
- `#7195 <https://leap.se/code/issues/7195>`_: Use cryptography instead of pycryptopp.

Known Issues
drebs's avatar
drebs committed
378
++++++++++++
Kali Kaneko's avatar
Kali Kaneko committed
379
380
381
382
- Upload phase of client syncs is still quite slow. Enabling size limited batching
  can help, but you have to make sure that your server is compatible.

Server
drebs's avatar
drebs committed
383
~~~~~~
Kali Kaneko's avatar
Kali Kaneko committed
384
385

Features
drebs's avatar
drebs committed
386
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
- General performance improvements.
- `#7509 <https://leap.se/code/issues/7509>`_: Moves config directory from /etc/leap to /etc/soledad.
- Adds a new config parameter 'create_cmd', which allows sysadmin to specify
  which command will create a database. That command was added in
  pkg/create-user-db and debian package automates steps needed for sudo access.
- Read netrc path from configuration file for create-user-db command. 
- 'create-user-db' script now can be configured from soledad-server.conf when
  generating the user's security document.
- Migrating a user's database to newest design documents is now possible by
  using a parameter '--migrate-all' on 'create-user-db' script.
- Remove tsafe monkeypatch from SSL lib, as it was needed for Twisted <12
- Added two methods to start and finish a batch on backend. They can be used to
  change database behaviour, allowing batch operations to be optimized.

Common
drebs's avatar
drebs committed
402
~~~~~~
Kali Kaneko's avatar
Kali Kaneko committed
403
404

Features
drebs's avatar
drebs committed
405
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
406
407
408
409
- Add a sanitized command executor for database creation and re-enable user
  database creation on CouchServerState via command line.

Bugfixes
drebs's avatar
drebs committed
410
++++++++
Kali Kaneko's avatar
Kali Kaneko committed
411
- `#7626 <https://leap.se/code/issues/7626>`_: Subclass a leaky leap.common.couch exception to avoid depending on couch.
412
413
414


.. _`master`: https://0xacab.org/leap/soledad