Actions
Bug #5197
closedfast_pattern assignment of specific content results in FN
Affected Versions:
Effort:
Difficulty:
Label:
Description
Consider the following two rules, designed to detect a specific answer for a DNS TXT record reply cough more dns buffers in suri7 plz! cough. The only difference is that sid:2 has a manual fast_pattern assignment.
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:1;) alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:2;)
In testing against the attached pcap, only sid:1; fires, despite containing the exact same signature logic.
This issue is not present in Suricata 4.0.x
Suricata shows the fast_patterns for each rule as follows
-------------------------------------------------------------------
Date: 19/3/2022 -- 00:14:39
-------------------------------------------------------------------
== Sid: 1 ==
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:1;)
Fast Pattern analysis:
Fast pattern matcher: content
Flags: Within Distance
Fast pattern set: no
Fast pattern only set: no
Fast pattern chop set: no
Original content: \x00\x10\x00\x01\x00\x00\x00\x01
Final content: \x00\x10\x00\x01\x00\x00\x00\x01
== Sid: 2 ==
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:2;)
Fast Pattern analysis:
Fast pattern matcher: content
Flags: Within Distance
Fast pattern set: yes
Fast pattern only set: no
Fast pattern chop set: no
Original content: 456789
Final content: 456789
============
Summary:
============
packet/stream payload, smallest pattern 6 byte(s), longest pattern 8 byte(s), number of patterns 2, avg pattern len 7.00 byte(s)
Files
Updated by Brandon Murphy over 2 years ago
For testing purposes, I created a variation of the rule, each with a different fast_pattern manually assigned, and one without a manual assignment.
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:1;) alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:2;) alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; fast_pattern; content:"456789"; distance:2; within:6; sid:3;) alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; fast_pattern; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:4;) alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; fast_pattern; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:5;) alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; fast_pattern; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:6;) alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; fast_pattern; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:7;)
here are the results.
03/17/2022-10:55:24.304304 [**] [1:1:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304 [**] [1:4:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304 [**] [1:5:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304 [**] [1:6:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304 [**] [1:7:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
Interesting to me that sid:3, and sid:6 actually have the same fast_pattern, just in different positions.
-------------------------------------------------------------------
Date: 19/3/2022 -- 15:44:40
-------------------------------------------------------------------
== Sid: 3 ==
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; fast_pattern; content:"456789"; distance:2; within:6; sid:3;)
Fast Pattern analysis:
Fast pattern matcher: content
Flags: Within Distance
Fast pattern set: yes
Fast pattern only set: no
Fast pattern chop set: no
Original content: 1
Final content: 1
== Sid: 6 ==
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; fast_pattern; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:6;)
Fast Pattern analysis:
Fast pattern matcher: content
Flags: Within Distance
Fast pattern set: yes
Fast pattern only set: no
Fast pattern chop set: no
Original content: 1
Final content: 1
============
Summary:
============
packet/stream payload, smallest pattern 1 byte(s), longest pattern 1 byte(s), number of patterns 2, avg pattern len 1.00 byte(s)
Updated by Victor Julien over 2 years ago
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Victor Julien
- Target version changed from TBD to 7.0.0-beta1
Updated by Shivani Bhardwaj over 2 years ago
Updated by Victor Julien over 2 years ago
- Assignee changed from Victor Julien to Jeff Lucovsky
- Priority changed from Normal to High
I investigated this issue through the rules for #5162, but I think this is the correct ticket. It looks like when I comment out the contents of DetectContentPropagateLimits, it works. This suggests that the offset/depth calculated based on the chain of patterns is off somehow. Jeff, can you check if that is the case for you as well?
Updated by Jeff Lucovsky over 2 years ago
I'm calculating content length of 37. Still looking
Updated by Jeff Lucovsky over 2 years ago
Using the attached pcap, both of the rules (the first 2 posted), fire for me:
jlucovsky@ ~/src/jal/suricata (master) $ sudo rm -rf /tmp/ll/*;src/suricata -c suricata.yaml -S ~/5197.rules -r ~/pcap/redacted_dns.pcapng -l /tmp/ll
[587417] 9/4/2022 -- 09:55:57 - (suricata.c:1141) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[587417] 9/4/2022 -- 09:55:57 - (detect-engine-loader.c:232) <Warning> (ProcessSigFiles) -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /home/jlucovsky/5197.rules
[587417] 9/4/2022 -- 09:55:57 - (detect-engine-loader.c:347) <Warning> (SigLoadSignatures) -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 1 rule files specified, but no rules were loaded!
[587417] 9/4/2022 -- 09:55:57 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 16 FM: 1 FR: 1 Engine started.
[587417] 9/4/2022 -- 09:55:57 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received. Stopping engine.
[587431] 9/4/2022 -- 09:55:57 - (source-pcap-file.c:384) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
jlucovsky@ ~/src/jal/suricata (master) $ sudo rm -rf /tmp/ll/*;src/suricata -c suricata.yaml -S ~/rules//5197.rules -r ~/pcap/redacted_dns.pcapng -l /tmp/ll
[587476] 9/4/2022 -- 09:56:08 - (suricata.c:1141) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[587476] 9/4/2022 -- 09:56:09 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 16 FM: 1 FR: 1 Engine started.
[587476] 9/4/2022 -- 09:56:09 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received. Stopping engine.
[587490] 9/4/2022 -- 09:56:09 - (source-pcap-file.c:384) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
jlucovsky@ ~/src/jal/suricata (master) $ cat /tmp/ll/fast.log
03/17/2022-06:55:24.304304 [**] [1:8:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-06:55:24.304304 [**] [1:9:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
jlucovsky@ ~/src/jal/suricata (master) $ cat ~/rules/5197.rules
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:8;)
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:9;)
Actions
#7
Updated by Brandon Murphy over 2 years ago
- File f92840348b24bc49_eve.json f92840348b24bc49_eve.json added
- File 7139323980e636ee_suricata.yaml 7139323980e636ee_suricata.yaml added
I just tested again, just to make sure and was able to replicate only sid:1 alerting.
Attached are the config, and eve log used.
[2565] 10/4/2022 -- 02:36:36 - (suricata.c:1142) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode [2565] 10/4/2022 -- 02:36:36 - (util-cpu.c:178) <Info> (UtilCpuPrintSummary) -- CPUs/cores online: 4 [2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- fast output device (regular) initialized: dalton-fast.log [2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- eve-log output device (regular) initialized: dalton-eve.json [2565] 10/4/2022 -- 02:36:36 - (output-json-dnp3.c:288) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized. [2565] 10/4/2022 -- 02:36:36 - (output-json-dnp3.c:288) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized. [2565] 10/4/2022 -- 02:36:36 - (runmodes.c:658) <Warning> (RunModeInitializeEveOutput) -- [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - eve module 'ikev2' has been replaced by 'ike' [2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- http-log output device (regular) initialized: dalton-http.log [2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- tls-log output device (regular) initialized: dalton-tls.log [2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- alert-debug output device (regular) initialized: dalton-alert_debug.log [2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- stats output device (regular) initialized: dalton-stats.log [2565] 10/4/2022 -- 02:36:36 - (detect-engine-loader.c:355) <Info> (SigLoadSignatures) -- 1 rule files processed. 2 rules successfully loaded, 0 rules failed [2565] 10/4/2022 -- 02:36:36 - (util-threshold-config.c:1113) <Info> (SCThresholdConfParseFile) -- Threshold config parsed: 0 rule(s) found [2565] 10/4/2022 -- 02:36:36 - (detect-engine-build.c:1471) <Info> (SigAddressPrepareStage1) -- 2 signatures processed. 0 are IP-only rules, 2 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only [2565] 10/4/2022 -- 02:36:36 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 4 FM: 1 FR: 1 Engine started. [2566] 10/4/2022 -- 02:36:36 - (source-pcap-file.c:173) <Info> (ReceivePcapFileLoop) -- Starting file run for /tmp/f92840348b24bc49_Apr-10-2022_02-36-36/pcaps/redacted_dns.pcapng.pcap [2566] 10/4/2022 -- 02:36:36 - (source-pcap-file-helper.c:157) <Info> (PcapFileDispatch) -- pcap file /tmp/f92840348b24bc49_Apr-10-2022_02-36-36/pcaps/redacted_dns.pcapng.pcap end of file reached (pcap err code 0) [2565] 10/4/2022 -- 02:36:36 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received. Stopping engine. [2565] 10/4/2022 -- 02:36:36 - (suricata.c:1161) <Info> (SCPrintElapsedTime) -- time elapsed 0.080s [2566] 10/4/2022 -- 02:36:36 - (source-pcap-file.c:389) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes [2567] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests [2568] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests [2569] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests [2570] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests [2565] 10/4/2022 -- 02:36:36 - (counters.c:857) <Info> (StatsLogSummary) -- Alerts: 1 [2565] 10/4/2022 -- 02:36:36 - (detect-engine-build.c:1773) <Info> (SigAddressCleanupStage1) -- cleaning up signature grouping structure... complete
Updated by Jeff Lucovsky over 2 years ago
When Suricata is configured to not use hyperscan, I can reproduce the results (one, rather than two, alerts).
mpm-algo: ac spm-algo: bm
Updated by Victor Julien over 2 years ago
- Status changed from Assigned to Closed
- Assignee changed from Jeff Lucovsky to Victor Julien
- Priority changed from High to Normal
This has been fixed as part of #5162, so its essentially a duplicate.
Updated by Victor Julien over 2 years ago
- Is duplicate of Bug #5162: inspection of smb traffic without smb/dcerpc doesn't work correct. added
Actions