Suricata

The request was in part to reduce IO pressure, which I understand, but I also feel this is mostly a post-processing function. See the work by the vast.io folks to compress data for example.

I would like to avoid a runtime cost by suricata to do any kind of de-duplication. Perhaps simply having more control over which fields are logged per eve type are enough?