[#498] Resolve contact ids

closes #498 (closed) fixed-up by !601 (merged)

context

• after implementing sealed sender in !593 (merged) (and modifying the data model a bit in !594 (merged) and !596 (merged) to accomodate), we noticed that we were getting repeated "Bad Mac" errors when sending subsequent messages to a channel from admins who had just received a successful (outbound) welcome message or subscribers who were able to subscribe with a successful (inbound) "hello" message
• in both cases, the problem appeared to be related to the fact that libsignal switches from using a contact's phone number as its "identifier" during an unsealed session to using its UUID after a sealed session has been initialized. when the identifier changes, session protocol store lookups cannot find the existing session, a new session is created, and that session does not match the session of the subscriber or admin's protocol store, resulting in "bad mac" errors and decryption failures preventing the session from being able to advance
• in this MR, we provide a "contact store" that works around this bug by storing both the UUID and phone number of a contact to a record that has a universal contact_id to which either UUID or phone number can be resolved, and use the contact_id to track sessions and identities in the protocol store
• futher, we address some bugs that came up when using this store and noticing different behavior based on whether a session was initiated with a channel by a subscriber with (1) a mobile-only account, (2) a 2-device account from which they started the conversation from a mobile client, or (3) a 2-device account from which they started the conversation from their desktop client

changes

data model

• add contacts table that stores a contat's uuid and phone_number, links it to a channel's account_id and stores a contact_id that can be used to resolve signal's identifier (which is ambiguous between UUID and phone number) to a constant id
• modify identities and sessions table to reference this contact_id (instead of the old name field which was an ambiguous uuid/phone-number reference)

store

• add a ContactStore that can:
  • create a contact from any combination of contact fields OR an account/envelope tuple
  • when given an "identifier" string that is ambiguous between a uuid or phone number: [1]
    • resolve the identifier into the contact_id for a previously stored contact, or create a new contact and return its contact_id
    • report whether a contact exists with such an identifier
    • retrieve a profile key for a contact containing such an identifier
    • retrieve the contact containing such an identifier and convert it to a SignalcAddress
    • add the identifier to a contact record that already stored another identifier of the other type (ie: store a phone number if we already have a uuid for the contact or store a uuid if we already have a phone number)

[1] ie: a SignalServiceAddress#identifier or a SignalProtocolAddress#name into a contact_id for a previously-stored contac

• leverage the ContactStore in the ProtocolStore:

  • use the resolver function (ContactStore#resolveContactId) in all lookups in the SignalcIdentityStore and SignalcSessionStore that consume a SignalProtocolAddress#name -- which may be either a uuid or phone number and may switch during the course of the same session -- to resolve the address to a contact_id that will be stable across the session and thus prevent bad mac errors after sealed-sender sessions have been initiated (switching an envelope's #identifier from a phone number to a UUID)

• leverage the ContactStore in the SignalSender:

  • upon receiving a prekey bundle from a subscribing user at the beginning of a session in which the first contact is an inbound "hello" message: create a contact record that knows about both the sender's UUID and phone number
  • upon receiving an unsealed receipt from a new contact (ie: an admin or invitee with whom a session was initiated by an outbound welcome message): store the uuid or phone number contained in the receipt to make up for the fact that we might have lacked either when the channel sent the welcome message, but might need to use either to identify the contact as we seek to continue the session
  • upon receiving an inbound identity exception from an existing contact to lookup the contact by its identifier in order to construct a SignalcAddress to pass back to the signalboost client for re-trusting or deauthorizing. (necessary because in a sealed-sender world, the contact address is no longer available to us on the envelope and if decryption fails due to identify failure, also not available to us on the message contents object)
  • upon receiving any message to store the profile key of its sender if we are missing it (so that we can initiate sealed-sender sessions with new contacts)

• leverage the ContactStore in AccountManager:

  • upon creating a new account, also create an "own contact" for the account, which libsignal will attempt to do on its own, but fail to add both identifiers, which appears correlated with a bug causing new contacts to not find the channel's phone number on signal unless they enter a country code before the number (which normally they dont' have to do on mobile devices)

• add 2 new message-sending functions to SignalSender

  • #sendProfileKey (used to initiate sealed-sender sessions in SignalReceiver as soon as we receive a prekey bundle from a new contact)
  • #sendReceipt used to ensure contacts get 2 checks when conversing with us and messages are synced across devices (see bugs below)

Bugs Resolved during QA

• bug 1: "bad mac" errors from desktop-initiated sessions
  • symptom: after initiating session on desktop with inbound "hello", all subsequent messages trigger bad mac errors
  • fix: revert change instructing libsignal to prefer phone numbers to uuids when evaluating SignalServiceAddress#identifier
• bug 2: subscribers must enter +1 before phone number to subscribe to channel
  • fix: This appears resolved by storing an "own contact" for an account as soon as it is created that contains both the account's uuid and phone number (hypothesis: being unable to connect the UUID to the phone number led to lookup failure for mobile clients?). at any rate: in a series of 15 trials i observed 0 occurrences of the bug after introducing the #createOwnContact fix
• bug 3: after initiating sealed-sender session, all messages (from admins or subscribers) only get one check
  • fix: resolved by implementing read receipts (which we send immediately upon receiving message)
  • note: recipients will only get these receipts if they have read receipts enabled

Bugs not resolved but deemed acceptable

• bug 4: self-healing "bad-mac" on mobile-initiated sessions (non-deterministic)
  • symptom: trying to initiate session on mobile with inbound "hello" triggers "bad mac" error and no response but all subsequent messages create a successful session (with 2 checks and responses)
  • note: this occurs in about 33% of trials. in some trials this did not happen.
  • workaround: if user sends "hello" again, the situation repairs itself
  • possible fix: use the same session lock as the protocol store in the contact store to guard against race conditions?
• bug 5: if subscriber initiates session from a desktop, a channel cannot initiate a sealed-sender session with them until they tap "continue" from a mobile device
  • cause: for whatever reason, only the mobile device can send the profile key that is needed to send sealed-sender messages back to them. (this behavior is also observed outside the context of signalc if a user tries to initiate a conversation with another human signal user, but starts the conversation from their desktop client first)
  • why we care: if users only ever talk to signalboost over their desktop client, we will only ever be able to send unsealed messages to them, which will be counted against the rate limit signal uses to block spam. if too many users only ever use a desktop, we run the risk of being blocked if we continue sending messages to them.
  • workaround: insert a metrics counter to track how many sealed-sender messages we receive that don't contain a profile key (ie: messages matching the signature of a person who has not yet tapped "continue") so that we can assess whether the volume of such messages is high enough to warrant a more aggressive fix
• bug 6: after initiating sealed-sender session, messages sent from mobile device do not sync to desktop (non-deterministic)
  • note: observed in ~66% of trials

QA Process

Outgoing message case:

• ACTION: create channel
• VERIFY: did the channel admin receive a welcome message
• VERIFY: did the channel admin receive an 'ACCPET prompt'
• ACTION: tap 'Accept' on the admin's device
• VERIFY: the channel receives and stores a profile key for the admin
• VERIFY: when the admin sends subsequent INFO messages:
  • no "bad mac" errors
  • continue to get responses (all of which are sealed-sender with 2 checks)

Incoming messages from MOBILE-ONLY ACCOUNT:

• ACTION: subscribe to the channel
• VERIFY: does the subscriber get a response
  • FAILURE: non-deterministically fails in < 50% of trials but self-heals if subscriber sends "hello" again
• VERIFY: does signalc store a profile key for the subscriber
• VERIFY: do subsquent INFO messages:
  • receive sealed-sender responses
  • not trigger a bad mac errors
  • receive 2 checks

Incoming messages from 2-DEVICE-ACCOUNT initiated by MOBILE:

• ACTION: subscribe to the channel
• VERIFY: does the subscriber get a response
  • FAILURE: non-deterministically fails in < 50% of trials but self-heals if subscriber sends "hello" again
• VERIFY: does signalc store a profile key for the subscriber
• VERIFY: do subsquent INFO messages:
  • not trigger a bad mac errors
  • receive 2 checks, appear on desktop when sent by mobile
  • receive sealed-sender responses

Incoming messages from 2-DEVICE-ACCOUNT initiated by MOBILE:

• ACTION: subscribe to the channel
• VERIFY: does the subscriber get a response
• VERIFY: does signalc NOT store a profile key for the subscriber
• VERIFY: do subsquent INFO messages:
  • do not trigger a bad mac errors
  • receive 2 checks
  • receive unsealed-sender responses
• VERIFY: mobile receives message requiring user to tap "Continue" to continue conversation
• ACTION: tap "continue"
• VERIFY: profile key is sent to signalc and stored
• VERIFY: subsequent INFO messages:
  • appear on desktop when sent by mobile
    • FAILURE: fails ~66% of trials
  • receive sealed-sender responses