Optimization #6960
openfuzz: target to test signatures compatibility
Description
This idea was proposed by Victor. We can test for how different signatures behave with one another.
1. There should be a few hard coded canary rules say from ET Open.
2. There should be some random rules from fuzzer.
3. When given to the detection engine, rules from (1) and (2), at least the rules from (1) should always alert.
Updated by Shivani Bhardwaj 7 months ago
@Philippe Antoine could you please share your thoughts about this if it makes sense to you?
More questions from me:
1. This pattern (checking for alerts) is not seen in the fuzzer yet, is that intentional?
2. Can fuzzer end up creating some rules that can actually affect/ignore the canary rules leading to no alert?
Updated by Philippe Antoine 7 months ago
I guess I miss some context here :
Was there a bug where one rule should have triggered but did not because some other rule was activated ?
But a rule like pass any any -> any any (sid:1;) should indeed prevent all the other rules from triggering, should it not ?
Updated by Shivani Bhardwaj 7 months ago
Was there a bug where one rule should have triggered but did not because some other rule was activated ?
This is to ensure that newer changes to the core part of the engine does not cause any changes in the behavior of Suricata. One such recent change is the change in the entire algorithm and data structures around Port Grouping. Although we do have several unit tests to cover many edge cases, we wondered if we still missed any and if fuzzing could help.
There can ofc be a way to do some isolated target but then Victor proposed this idea of checking all kinds of rules and not just specific to port grouping.
But a rule like
pass any any -> any any (sid:1;)should indeed prevent all the other rules from triggering, should it not ?
Indeed. Is there a way to tell the fuzzer to skip such kinds of rules? Maybe a seed that only contains alert rules? wdyt?
Updated by Philippe Antoine 6 months ago
But a rule like
pass any any -> any any (sid:1;)should indeed prevent all the other rules from triggering, should it not ?Indeed. Is there a way to tell the fuzzer to skip such kinds of rules? Maybe a seed that only contains alert rules? wdyt?
We can have the fuzz target have only alert rules
But will that be enough ?
Like, there is a maximum alerts per packet, so I could hide an alert, by triggering more alerts on the same packet...
If you want to experiment this, you can start from the fuzz_sigpcap_aware target.
The fuzz input is rules + pcap. You should add to the input a new field interpreted as rules, then check if the alerts count are equal for set1 vs set1+set2 on the same pcap
That looks more complete to me than just using a predefined ruleset+pcap