stress test and improve filesystem backend walk
Walk the walk.
Performance metrics to collect:
- how long does it take to list X blobs with FileSystemBackend (assuming all X are in a single namespace)
- how is reactor responsiveness affected? (watchdog)
Ad-hoc ideas (just for documenting them somewhere, metrics will tell us the whole story):
- #8882 (closed) has some discussions
- We could use
scandir
at some point - We could find . > soledad_cache on startup and watch filesystem during runtime
- Also take other actions to avoid spending too much time walking the filesystem, like limiting results
- Yield on walk, so one user can't disrupt the entire service.