Feature #1949
closed
Feature #2303: file-store enhancements (aka file-store v2): deduplication; hash-based naming; json metadata and cleanup tooling
only write unique files
Added by Duane Howard almost 8 years ago.
Updated almost 7 years ago.
Description
Current behavior for filestore is to extract all. It could be useful to keep state and only write a given file once (maybe per run of Suricata?) For example if 15 users download a popular PE file, we'll end up with 15 copies of the same file on disk. Somewhat related to https://redmine.openinfosecfoundation.org/issues/1948 in that writing to hash for filename would avoid wasted disk space, but not actual time Suricata spends writing files to disk.
Related issues
1 (1 open — 0 closed)
hrm... i meant this to be a feature request, don't appear to be able to change it now?
- Tracker changed from Bug to Feature
The file store already starts writing files that are still being transferred. I'm not sure how we can reliably determine duplicate files before we've seen the whole file. In that case we've already started writing it to disk, except perhaps tiny files.
- Assignee set to Anonymous
- Target version set to TBD
- Status changed from New to Assigned
- Assignee changed from Anonymous to Jason Ish
- Target version changed from TBD to 70
This will be supported if we start using SHA-256 hash for names.
- Related to Task #2309: SuriCon 2017 brainstorm added
- Status changed from Assigned to Closed
- Target version changed from 70 to 4.1beta1
Also available in: Atom
PDF