Figuring out when content from live sources is newer than cached content

So we've hit one of the Two Hard Problems: cache invalidation. We need to be able to tell if content received from live plugins (gun+ipfs, fetch) is fresher than content stashed using a stashing plugin (like cache).

This is not easy because each plugin, each retrieval method used, has different ways of handling metadata:

  • for fetch plugin, we can mostly rely on ETag and Last-Modified headers
  • for ipns+ipfs we could try to rely on sequence numbers and short lifetime; or save the IPFS address and just decide that a different address means fresher content
  • for gun+ipfs we could implement metadata as part of the Gun structure (also see #5); or save the IPFS address and just decide that a different address means fresher content
  • for a Web Archive-based plugin, it could be Memento-Datetime and X-Archive-Orig-Date headers

Problem is mapping it all onto one another so that we can compare content across plugins (like fetch'ed content compared to gun+ipfs'ed content).

Edited Dec 03, 2019 by Michał "rysiek" Woźniak
Assignee Loading
Time tracking Loading