Integrate soledad migration into upgrade test

assigned to @micah

Nov 08 00:12:31 micah https://0xacab.org/micah/platform/-/jobs/25693 -- woo!
Nov 08 02:24:11 ⤷ so thats the upgrade test passing
Nov 08 02:24:56 ⤷ it would be good to run it again, and put a sleep in place to look at the output of the script to make sure it is doing what it is supposed to. i dont have time now, 
and wont until thursday morning again, so if someone wants to take my branch and do that - would be swell

@micah: looks good so far!

I won't have time to work on that but I suggest to use this cmd so we see the migration output in CI as well:

/usr/share/soledad-server/migration/0.9/migrate.py --log-file /dev/stdout --verbose | tee /var/log/leap/soledad_migration.log

Also, it would be good to run a leap test before we actually migrate to have a visual indication that a soledad migration is needed.

closed via commit a0eea43c

Oh damn, I didn't see your comment before I pushed this.

I agree it would be useful to see the migration output in CI, this is what it produces:

15:15:55,291 INFO starting couch schema migration to 0.9
15:15:55,299 INFO [user-7ad4ec02764f51d1126de238751ee07d] starting migration of user db
15:15:55,302 DEBUG [user-7ad4ec02764f51d1126de238751ee07d] creating gen doc: gen-0000000001
15:15:55,303 INFO [user-7ad4ec02764f51d1126de238751ee07d] moving sync docs
15:15:55,305 DEBUG [user-7ad4ec02764f51d1126de238751ee07d] moving sync doc: u1db_sync_0efa944f45a24cb99773cf4ef1d10610 -> _local/sync_0efa944f45a24cb99773cf4ef1d10610
15:15:55,309 INFO [user-7ad4ec02764f51d1126de238751ee07d] deleting design doc: _design/docs
15:15:55,312 INFO [user-7ad4ec02764f51d1126de238751ee07d] deleting design doc: _design/syncs
15:15:55,315 INFO [user-7ad4ec02764f51d1126de238751ee07d] deleting design doc: _design/transactions
15:15:55,318 INFO [user-7ad4ec02764f51d1126de238751ee07d] moving config doc: u1db_config -> _local/config
15:15:55,321 INFO [user-7ad4ec02764f51d1126de238751ee07d] finished migration of user db
15:15:55,321 INFO finished couch schema migration to 0.9

@varac - what do you mean about running a leap test before migration? That will fail the first time, do the migration, and then run the test again?

@varac - what do you mean about running a leap test before migration? That > will fail the first time, do the migration, and then run the test again?

Yes, that's what I meant.

I dont get the command you provided:

/usr/share/soledad-server/migration/0.9/migrate.py --log-file /dev/stdout --verbose | tee /var/log/leap/soledad_migration.log

Not sure why the |tee of that log file is there, because when you pass --log-file /dev/stdout it wont log to /var/log/leap/soledad_migration.log

I've pushed a commit to my repo to test it without the pipe and what follows

@micah I tested this cmd on aardwolf (when the migration already happened it will just tell you it has nothing to do, so it's not harmful).

It works exaclty like I pasted it above (see root history on aardwolf).

@varac seems like it works: https://0xacab.org/micah/platform/-/jobs/25915 -- its weird because the first test succeeded, when it shouldn't... but maybe it was doing that sometimes?

@micah indeed, this is not what I would have expected. We should ask @drebs what he thinks about it.

@drebs, we are confused. We integrated an upgrade_test in the platform CI to make sure upgrades from one to another minor platform version work.

The test installs i.e. platform 0.9 (together with soledad-server 0.8), and upgrades to platform 0.10 (soledad-server 0.10). After this, we run leap test, expecting that it would report soledad-server not running because we thought soledad-server would refuse to start when it finds unmigrated docs. but the test succeeds. After this, we run the migration script and run leap test again. Now the test succeeds which is as it should.

Do you understand why the first leap test doesn't complain about soledad-server ?

This is an example where leap test fails because the migration has not happened: https://0xacab.org/micah/platform/-/jobs/25019

Here is an example of leap test failing because the migration has not been done: https://0xacab.org/micah/platform/-/jobs/25682

But this is after a manual

systemctl stop soledad-server
…
systemctl start soledad-server

The first leap test now comes before that step.

Why the second job you linked to breaks while current jobs now doesn't,I don't know...

I added some debug to the ci-build script to tell us what version of soledad is installed in each step, just in case it was somehow installing the wrong version, you can see the results of that here: https://0xacab.org/micah/platform/-/jobs/25954

Unfortunately, what this shows is that we are doing things correctly. With the original 0.9 deployment, soledad is at 0.8, then when we upgrade, it is at 0.10.3, which is what we expect.

reopened

soledad!159 (merged) has a fix to the problem we are encountering here

closed

mentioned in commit a0eea43c

Integrate soledad migration into upgrade test

Designs

Child items ...

Activity