Soledad-server should create user-dbs
Rationale behind this:
tapicero must go
urgency: unknown difficulty: varies
tapicero is a demon that runs in the background and creates each user's separate database used to store all their email. this is the per-user database that soledad-server uses.
the goal with tapicero is to be able to run webapp and soledad-server without giving them admin permissions. tapicero also waits in the background, so the webapp/api is not slowed down waiting for db creation when a new user is created.
tapicero has had a million stability problems, largely because bigcouch randomly fails a lot of the time for every imaginable thing. the current tapicero is mostly just code to catch all these errors and keep retrying. also, tapicero has to deal with the fact that bigcouch often reports the WRONG RESPONSE when things fail due to contention. ugh.
tapicero is much better now than it used to be, but there are still a lot of open issues for tapicero. the main one is that it can randomly get in a weird state where it is still running but no longer creating new databases. this happens infrequently enough that it has been impossible to debug, but frequently enough that it is considered a serious problem.
another big problem is that tapicero listens to a feed of new user account creations and then creates a new db as needed. it stores a file of how far it was in the past reading this feed, so that when it restarts it does not need to read the entire feed from the beginning of time and verify all over again that the per-user database exists for each user. the problem is that these sequence files can sometimes get destroyed and then tapicero takes forever to start up again. we might be able to speed things up a lot by changing the strategy of how tapicero catches up in these situations.
Options:
(1) shyba just proposed on IRC that soledad-server be given admin access to couchdb and that it simply creates the per-user db as needed if the user has a valid token but the per-user db doesn't exist yet. this is smart approach, because it makes soledad-server more self-contained, is simple with few moving parts, and is doesn't have to deal with contention. it does not fix the problem of how to delete the user databases. for this, we still need tapicero or some other solution. a periodic maintenance script might work, since there is no urgency to deleting the old databases.
(2) in the past, I proposed replacing tapicero with rabbitmq solution, which would help us also be able to deploy mcollective in the future (which would make managing large installations 100x faster. obviously, not high priority now). the advantage of a solution like this is that we could find all kinds of uses for rabbitmq should we get it set up, the disadvantage is that incorporating rabbitmq into the platform will take a lot of work. it is also adding yet another moving part that might fail.
(3) the webapp could be granted admin powers and just create/destroy the per-user db at the moment the user account is created/destroyed. done, and done.
(4) the webapp could issue a simple api command to soledad-server to create/destroy the storage. this would make soledad-server more self contained, and would allow webapp to still not have admin powers.
(5) i could fix the six open tapicero bugs assigned to me. since i cannot reproduce the main bug, i would like to avoid further time wasted fixing tapicero, if i can avoid it.
we decided to go for option (1) and give soledad-server admin priviledges so it can create user-dbs.
see also the "related pixelated-platform issue":https://github.com/pixelated/pixelated-platform/issues/104
(from redmine: created on 2015-09-01, closed on 2015-09-28)
- Relations:
- relates #7509 (closed)