Hello! We are running our annual fundraising. Please consider making a donation if you value this freely available service or want to support people around the world working towards liberatory social change. https://riseup.net/donate.

  1. 04 Jun, 2015 1 commit
  2. 02 Jun, 2015 1 commit
  3. 01 Jun, 2015 6 commits
  4. 27 May, 2015 1 commit
  5. 26 May, 2015 6 commits
  6. 25 May, 2015 4 commits
    • Victor Shyba's avatar
      [bug] tests now accept any couchdb version · c32a444b
      Victor Shyba authored
      The .ini used before was very old and did not contain all the config
      values that couch needs. Using -n resets the configuration file chain,
      making it use what is provided without including the system defaults
      needed. To make it work with all versions the util will now copy
      default.ini to temp dir and use it as additional config, overwriting
      only what is needed with couchdb.ini.template provided.
      c32a444b
    • Kali Kaneko's avatar
      Merge remote-tracking branch 'leapcode/pr/216' into develop · 745ac11c
      Kali Kaneko authored
      this branch includes many changes that improve the asynchronous
      retrieval of the sync docs, and the parallel decryption of the encrypted
      documents.
      745ac11c
    • drebs's avatar
      [bug] remove client syncer call to close method · 3e6e5164
      drebs authored
      3e6e5164
    • drebs's avatar
      [feature] add pool of http/https connections · 31757168
      drebs authored
      Instead of opening one TCP connection for each HTTP request, we want to reuse
      connections. Also, we need to be able to verify SSL certificates. This commit
      implements both features in the twisted http client sync.
      31757168
  7. 22 May, 2015 1 commit
    • drebs's avatar
      [refactor] remove inline enc/dec from client pool · 478dd0eb
      drebs authored
      The whole idea of the encrypter/decrypter pool is to be able to use multiple
      cores to allow parallel encryption/decryption. Previous to this commit, the
      encryptor/decryptor pools could be configured to not use workers and instead
      do encryption/decryption inline. That was meant for testing purposes and
      defeated the purpose of the pools.
      
      This commit removes the possibility of inline encrypting/decrypting when using
      the pools. It also refactors the enc/dec pool code so any failures while using
      the pool are correctly grabbed and raised to the top of the sync deferred
      chain.
      478dd0eb
  8. 21 May, 2015 1 commit
    • drebs's avatar
      [refactor] remove unneeded proxy for insert_doc_cb · 33fa691e
      drebs authored
      When we initialized the async decrypter pool in the target's init method we
      needed a proxy to ensure we could update the insert doc callback with the
      correct method later on. Now we initialize the decrypter only when we need it,
      so we don't need this proxy anymore. This commit removes the unneeded proxy.
      33fa691e
  9. 20 May, 2015 15 commits
    • drebs's avatar
      [bug] ensure async decryption failures are logged · ce161f96
      drebs authored
      We have to make sure any failures in asynchronous decryption code is grabbed
      and properly transmitted up the deferred chain so it can be logged. This
      commit adds errbacks in the decryption pool that grab any failure and a
      check on the http target the failure if that is the case.
      ce161f96
    • drebs's avatar
      [feature] ensure reactor stops on client db script · d59ac3b5
      drebs authored
      d59ac3b5
    • drebs's avatar
      [bug] wrap unauth errors as invalid token errors · ec55459f
      drebs authored
      ec55459f
    • drebs's avatar
      [bug] ensure sync failures are not ignored · 0c23b1c7
      drebs authored
      0c23b1c7
    • drebs's avatar
      [refactor] cleanup sync, remove unused stuff · 5376b0eb
      drebs authored
      This commit does the following:
      
        * Remove the autocreate parameter from the sync() method.
        * Remove the syncing lock from the sync module because it did the same job
          as the lock in the sqlcipher module.
        * Remove the close/stop methods from sync module as they don't make sense
          after we started to use twisted in client-side sync.
      5376b0eb
    • drebs's avatar
      [feature] use twisted.web.client in client sync · e62dafeb
      drebs authored
      This change uses twisted deferreds for the whole syncing process and paves the
      way to implementing other transport schemes. It removes a lot of threaded code
      that used locks and was very difficult to maintain, and lets twisted to the
      dirty work. Furthermore, all blocking network i/o is now handled
      asynchronously by the twisted.
      
      This commit removes the possibility of interrupting a sync, and we should
      reimplement it using cancellable deferreds if we need it.
      e62dafeb
    • drebs's avatar
      [feature] use twisted adbapi for async encryption · 94cbe24f
      drebs authored
      The access to the sync db was modified to use twisted.enterprise.adbapi, but
      only the asynchronous decryption of incoming documents during sync was
      adapted. This commit modifies the asynchornous encryption of documents to also
      use the adbapi for accessing the sync db.
      94cbe24f
    • drebs's avatar
      [feature] add --create-doc to client db script · 93717f50
      drebs authored
      93717f50
    • drebs's avatar
      [bug] remove unused pending documents exception · d1c39d38
      drebs authored
      When we started implementing the sync db, one of the ideas was to reuse the
      data in the database in the case of a sync interruption. We don't do that now
      and thus the pending documents exception is unneeded. This commit removes that
      exception from the code.
      d1c39d38
    • Kali Kaneko's avatar
      [bug] remove illegal CR from auth header · 67f17cd3
      Kali Kaneko authored and drebs's avatar drebs committed
      The b64 encoding of the auth token was introducing an illegal character
      (\n), which was breaking the authentication step since an exception was
      being raised - when that multi-line header was attempted to be built.
      
      this commit fixes that bug.
      
      - Resolves: #6959
      67f17cd3
    • drebs's avatar
      [bug] fix order of insertion of decrypted docs · eae4468d
      drebs authored
      This commit actually does some different things:
      
        * When doing asynchronous decryption of incoming documents in soledad client
          during a sync, there was the possibility that a document corresponding to
          a newer generation would be decrypted and inserted in the local database
          before a document corresponding to an older generation. When this
          happened, the metadata about the target database (i.e. its locally-known
          generation) would be first updated to the newer generation, and then an
          attempt to insert a document corresponding to an older generation would
          cause the infamous InvalidGeneration error.
      
          To fix that we use the sync-index information that is contained in the
          sync stream to correctly find the insertable docs to be inserted in the
          local database, thus avoiding the problem described above.
      
        * Refactor the sync encrypt/decrypt pool to its own file.
      
        * Fix the use of twisted adbapi with multiprocessing.
      
      Closes: #6757.
      eae4468d
    • drebs's avatar
      [feature] use async adbapi for async decryption · 3a7ddacd
      drebs authored
      Since we started implementing twisted api in soledad, some pieces are missing.
      Accessing the sqlcipher database directly with the twisted adbapi facilities
      is one of them. The async encryption/decryption was touching the database
      directly, and this was causing some difficulties like having different threads
      accessing the same database. This commit implements the twisted adbapi stuff
      for the asynchronous encryption/decryption facilities.
      
      Next steps would be use async adbapi for async encryption and use async adbapi
      for all sqlcipher access.
      3a7ddacd
    • drebs's avatar
      [bug] fix log messages for secrets in storage · b75bedb0
      drebs authored
      b75bedb0
    • drebs's avatar
      [bug] fix log messages when fetching documents · 6ab3fe57
      drebs authored
      We always got a log message saying "canceling sync threads" in the end of the
      sync process, even when there was no error during the sync. This commit
      changes that in a way that we only have that log when the sync was actually
      cancelled because of an error.
      6ab3fe57
    • drebs's avatar
      [bug] always initialize sync db · 3a3f2d8c
      drebs authored
      Both deferred encryption and decryption rely on a special sync db. Previous to
      this fix, the sync db was only initialized if a syncer was configured with
      deferred encryption capabilities. This was a problem when the syncer was not
      configured like so, but the actual sync method was initiated configured to do
      deferred decryption.
      
      This commit fixes this by always initializing the sync db, so we have the
      option of doing all combinations of deferred encryption and decryption.
      3a3f2d8c
  10. 14 May, 2015 1 commit
    • drebs's avatar
      [pkg] bump version of server dependency on common · 340b0dcf
      drebs authored
      soledad-common versions before 0.6.5 do not contain the fix for #6833 and thus
      will not work with most recent server. That is why we have to bump this
      soledad-server dependency on soledad.common.
      340b0dcf
  11. 29 Apr, 2015 3 commits
    • drebs's avatar
      [doc] fix doc on why to re-raise sync exceptions · 68f89f1d
      drebs authored
      68f89f1d
    • drebs's avatar
      [bug] remove old code for sync deferred db init · 0ac9b6a6
      drebs authored
      The database initialization on the client sync module is deferred to another
      thread. As there is only one thread in the thread pool, this should not be a
      problem for now, as operations will actually be queued in that thread. There
      was some old code left from when we had to explicitelly wait for the db to be
      initialize before using it. This commit removes that old code and introduces
      some documentation so we remember to deal with deferred db init if we ever
      change the number of threads in the thread pool.
      0ac9b6a6
    • drebs's avatar
      [refactor] rename sync callbacks on client api · c2866316
      drebs authored
      c2866316