Suricata

file_magic / file.magic uses libmagic to classify files. It has never been a great match for us. It knows many, many files, and can extract quite a bit of meta data, but it comes with 2 important drawbacks:
1. performance - this has never been great and the impact on suri is considered too great for many scenarios
2. cross platform issues - many OS' ship different versions and different file signature data bases. This made it impossible for general purpose rulesets like ET open to use them reliably.

For the use case of Suricata a much simpler and reduced feature set is probably good enough. Most rules likely only care about detecting PDF, PE32, some compression formats. This type of capability could probably just life inside suricata's codebase itself, either hardcoded or as a new rule capability for file classification.

Feedback is requested on:
- what are the most important file types
- what should the detection granularity be (e.g. generic pdf, or pdf per version, generic window exe or more specific?)
- what if any feature extraction should be done