SquirrelFS currently has high memory utilization due to its large in-memory indexes. We should reduce its memory footprint.
Some possible optimizations:
- Store index information about files that are not open durably. We would need to be careful to store this in a crash-consistent manner way; we may need to be creative about how this data is stored to make sure the indexes are consistent and in-sync with the actual file tree structure.
- Simplify the current index structures; they are likely more complicated/larger than necessary, and there is some unnecessary repetition (e.g., multiple indexes that use the same keys for different information, which duplicates all of the keys).
- Consider whether we could change SquirrelFS' persistent layout to make lookups faster.
SquirrelFS currently has high memory utilization due to its large in-memory indexes. We should reduce its memory footprint.
Some possible optimizations: