Bug #2769
closedFalse positive alerts firing after upgrade suricata 3.0 -> 4.1.0
Description
Hello, oisf, could use your input on problem.
My corporate network has been running suricata 3.0. Recently upgraded to 4.1.0 then 4.1.1 and encountered issue.
Environment is CentOS release 7.4.1708 (Core) 3.10.0-862.14.4.el7.x86_64 on ~400 Dells
suricata version: suricata-4.1.0-ESG_1.el7.centos.x86_64, suricata-debuginfo-4.1.0-ESG_1.el7.centos.x86_64
pcre version: pcre-devel-8.32-17.el7.x86_64, pcre-8.32-17.el7.x86_64
lua version: lua-5.1.4-15.el7.x86_64
In Suricata 4.1.0 (and 4.1.1) (following recent upgrade from suricata 3.0) we noticed that under certain conditions false positive alerts are firing that should not be. In short, rules looking for HTTP packets are firing on ICMP data. It appears that the issue occurs on rules with http content modifiers where another rule in the ruleset uses an alert ip prefix and any content match. The packets must include an HTTP session followed by ICMP type packets (not that the rule should not match on the http session as the pcre does not match). I’ve also tested on suricata 4.1.2 and found that this issue is there as well. I’m not sure when the issue was introduced.
Rules in my local.rules:
alert ip any any -> any any (msg:"IP Any With Content"; content:"blahblah"; classtype:bad-unknown; priority:102; sid:5002203; rev:10;)
alert http $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Test Rule no http"; flow:to_server,established; content:"POST"; http_method; pcre:"/\/.+\/[a-zA_Z0-9]{4}\_confirm\.php/"; classtype:bad-unknown; priority:102; sid:9999991; rev:1;)
Test output:
#> docker run -it --rm --volume $PWD/data:/data sensor -c /data/suricata.yaml -r ./data/out.pcap
7/1/2019 -- 15:25:55 - <Notice> - This is Suricata version 4.1.0 RELEASE
7/1/2019 -- 15:25:55 - <Notice> - using flow hash instead of active packets
7/1/2019 -- 15:25:55 - <Warning> - [ERRCODE: SC_ERR_DEPRECATED_CONF(274)] - deprecated 'force-md5' option found. Please use 'force-hash: [md5]' instead
7/1/2019 -- 15:25:55 - <Warning> - [ERRCODE: SC_ERR_EVENT_ENGINE(210)] - can't suppress sid 2015985, gid 1: unknown rule
7/1/2019 -- 15:25:55 - <Notice> - all 7 packet processing threads, 4 management threads initialized, engine started.
7/1/2019 -- 15:25:55 - <Notice> - Signal Received. Stopping engine.
7/1/2019 -- 15:25:55 - <Notice> - Pcap-file module read 1 files, 72 packets, 10078 bytes
#> cat ./data/fast.log
01/02/2019-17:57:19.054415 [**] [1:9999991:1] Test Rule no http [**] [Classification: Potentially Bad Traffic] [Priority: 102] {ICMP} 3.20.244.244:2563 -> 10.228.203.106:0
Note that the rule which his supposed to match on an HTTP session is alerting on an ICMP packet.
Eve.json alert also has http data for an icmp protocol:
{
"timestamp": "2019-01-02T17:57:19.054415+0000",
"flow_id": 1426130925450708,
"pcap_cnt": 22,
"event_type": "alert",
"src_ip": "<redacted>",
"dest_ip": "<redacted>",
"proto": "ICMP",
"icmp_type": 3,
"icmp_code": 10,
"tx_id": 0,
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 9999991,
"rev": 1,
"signature": "Test Rule no http",
"category": "Potentially Bad Traffic",
"severity": 102
},
"http": {
"hostname": "<redacted>",
"url": "/em/event",
"http_user_agent": "Wget/1.11.4 Red Hat modified",
"http_method": "POST",
"protocol": "HTTP/1.0",
"length": 0
},
"app_proto": "http",
"flow": {
"pkts_toserver": 4,
"pkts_toclient": 0,
"bytes_toserver": 598,
"bytes_toclient": 0,
"start": "2019-01-02T17:56:06.194004+0000"
},
"payload": "<redacted>",==",
"payload_printable": "<redacted>",
"stream": 0
}
Any thoughs on how to proceed on this would help.
Talked with Andreas Herz on this via oisf forum.
Files
Updated by Victor Julien almost 6 years ago
Do you also see this issue with 4.1.2? If so, can you share a pcap to preproduce the issue?
Updated by Charles Dillard almost 6 years ago
Victor Julien wrote:
Do you also see this issue with 4.1.2? If so, can you share a pcap to preproduce the issue?
Issue is seen with 4.1.2 (in a lab here). Working out details of sending you a pcap file, since information is sensitive. Let you know. Thanks.
Updated by Charles Dillard almost 6 years ago
- File issue_no_icmp-1.pcap issue_no_icmp-1.pcap added
Victor Julien wrote:
Do you also see this issue with 4.1.2? If so, can you share a pcap to preproduce the issue?
Here is pcap file for issue described. Thanks.
Updated by Victor Julien almost 6 years ago
- Assignee set to Victor Julien
I can't seem to reproduce this with 4.1.2. Are you sure it's still an issue? The timestamps of the alert record posted above is a lot older than the actual run output, so maybe its a old alert?
Updated by Charles Dillard almost 6 years ago
Victor Julien wrote:
I can't seem to reproduce this with 4.1.2. Are you sure it's still an issue? The timestamps of the alert record posted above is a lot older than the actual run output, so maybe its a old alert?
Will check with my team, let you know Tuesday.
Updated by Charles Dillard almost 6 years ago
Charles Dillard wrote:
Victor Julien wrote:
I can't seem to reproduce this with 4.1.2. Are you sure it's still an issue? The timestamps of the alert record posted above is a lot older than the actual run output, so maybe its a old alert?
Will check with my team, let you know Tuesday.
Will send you a new pcap in a few
Updated by Charles Dillard almost 6 years ago
Charles Dillard wrote:
Charles Dillard wrote:
Victor Julien wrote:
I can't seem to reproduce this with 4.1.2. Are you sure it's still an issue? The timestamps of the alert record posted above is a lot older than the actual run output, so maybe its a old alert?
Will check with my team, let you know Tuesday.
Will send you a new pcap in a few
On 10 sensors we've deployed suricata 4.1.0, our incident response team seeing false positive for rule below. This rule works and does not fire false positives on servers with suricata 3.0
can your team test with the following rules as they other rules aren't valid for this one:
local.rules
alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Test Bug"; flow:to_server,established; content:"functions"; pcre:"/\/[0-9]{2,8}\.functions/Ui"; classtype:bad-unknown; priority:102; sid:1111111; rev:2;) alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Test Bug something else"; flow:to_server,established; content:"functions"; pcre:"/somethingelse/Ui"; classtype:bad-unknown; priority:102; sid:1111112; rev:2;)
fast.log results:
01/14/2019-18:48:03.115139 [**] [1:1111111:2] Test Bug [**] [Classification: Potentially Bad Traffic] [Priority: 102] {TCP} ip-goes-here:59921 -> 3.235.60.34:8080
01/14/2019-18:48:03.115139 [**] [1:1111112:2] Test Bug something else [**] [Classification: Potentially Bad Traffic] [Priority: 102] {TCP} ip-goes-here:59921 -> target-ip-goes-here:8080
Looking at these rules and PCAP file, these detections should never have occurred because the PCRE negates the the rules from matching. Our incident response team has tested in 4.1.0 and 4.1.2 and get the same results.
Updated by Charles Dillard almost 6 years ago
Charles Dillard wrote:
Charles Dillard wrote:
Charles Dillard wrote:
Victor Julien wrote:
I can't seem to reproduce this with 4.1.2. Are you sure it's still an issue? The timestamps of the alert record posted above is a lot older than the actual run output, so maybe its a old alert?
Will check with my team, let you know Tuesday.
Will send you a new pcap in a few
On 10 sensors we've deployed suricata 4.1.0, our incident response team seeing false positive for rule below. This rule works and does not fire false positives on servers with suricata 3.0
can your team test with the following rules as they other rules aren't valid for this one:
local.rules
alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Test Bug"; flow:to_server,established; content:"functions"; pcre:"/\/[0-9]{2,8}\.functions/Ui"; classtype:bad-unknown; priority:102; sid:1111111; rev:2;) alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Test Bug something else"; flow:to_server,established; content:"functions"; pcre:"/somethingelse/Ui"; classtype:bad-unknown; priority:102; sid:1111112; rev:2;)fast.log results:
01/14/2019-18:48:03.115139 [**] [1:1111111:2] Test Bug [**] [Classification: Potentially Bad Traffic] [Priority: 102] {TCP} ip-goes-here:59921 -> 3.235.60.34:8080
01/14/2019-18:48:03.115139 [**] [1:1111112:2] Test Bug something else [**] [Classification: Potentially Bad Traffic] [Priority: 102] {TCP} ip-goes-here:59921 -> target-ip-goes-here:8080Looking at these rules and PCAP file, these detections should never have occurred because the PCRE negates the the rules from matching. Our incident response team has tested in 4.1.0 and 4.1.2 and get the same results.
Also, here is configuration arguments involved in the build of suricata 4.1 This is all built with rpmbuild.
LIBS="-lnuma" ./configure --enable-hiredis --enable-profiling --enable-pfring --with-libnet-libraries=/usr/lib64/libnet --enable-geoip --enable-libjansson --with-libjansson-libraries=/usr/local/include --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var/run --with-pfring-libraries=/usr/local/lib --enable-unix-socket --enable-luajit --with-libluajit-libraries=/usr/local/lib --with-libnss-libraries=/usr/lib64 --with-libnss-includes=/usr/include/nss3 --with-libnspr-libraries=/usr/lib64 --with-libnspr-includes=/usr/include/nspr4 --disable-gccmarch-native
and
BuildRequires: libyaml-devel automake autoconf libtool numactl GeoIP GeoIP-devel libyaml libyaml-devel libcap-ng-devel file-devel lua lua-devel nss-devel nspr-devel hiredis-devel
Requires: lua libyaml GeoIP numactl flex bison LuaJIT hiredis pfring
Thanks again for any insight on this issue.
Updated by Charles Dillard almost 6 years ago
- File issue_no_icmp.pcap issue_no_icmp.pcap added
Victor Julien wrote:
Did you intend to add a new pcap?
Here is pcap "issue_no_icmp.pcap" related to this issue, thanks
Updated by Victor Julien over 5 years ago
- Status changed from New to Feedback
- Target version set to TBD
Was that the correct pcap? Both attached pcaps are identical.
Updated by Charles Dillard over 5 years ago
Victor Julien wrote:
Was that the correct pcap? Both attached pcaps are identical.[1:35 PM]
Yes, that should be correct....use this rule to test with and thanks:
alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Test Bug"; flow:to_server,established; content:"functions"; pcre:"/\/[0-9]{2,8}\.functions/Ui"; classtype:bad-unknown; priority:102; sid:1111111; rev:2;) alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"Test Bug something else"; flow:to_server,established; content:"functions"; pcre:"/somethingelse/Ui"; classtype:bad-unknown; priority:102; sid:1111112; rev:2;)
Updated by Victor Julien over 5 years ago
- Status changed from Feedback to Assigned
- Target version changed from TBD to 4.1.3
I can reproduce it now, thanks.
Updated by Charles Dillard over 5 years ago
Victor Julien wrote:
I can reproduce it now, thanks.
Great, let us know what you find.
Updated by Victor Julien over 5 years ago
Btw if you can also share a test case that shows the issue reported originally, involving icmp, it would be very helpful.
Updated by Charles Dillard over 5 years ago
Victor Julien wrote:
Btw if you can also share a test case that shows the issue reported originally, involving icmp, it would be very helpful.
I'll get that for you.
Updated by Victor Julien over 5 years ago
The issue in note 13 is fixed by:
https://github.com/OISF/suricata/pull/3673/commits/2bd23bc1d511801469d4736ef85efeb373992411
Updated by Victor Julien over 5 years ago
- Target version changed from 4.1.3 to 5.0beta1
Updated by Victor Julien over 5 years ago
- Copied to Bug #2841: False positive alerts firing after upgrade suricata 3.0 -> 4.1.0 (4.1.x) added
Updated by Charles Dillard over 5 years ago
Victor Julien wrote:
The issue in note 13 is fixed by:
https://github.com/OISF/suricata/pull/3673/commits/2bd23bc1d511801469d4736ef85efeb373992411
Victor Julien wrote:
The issue in note 13 is fixed by:
https://github.com/OISF/suricata/pull/3673/commits/2bd23bc1d511801469d4736ef85efeb373992411
Victor Julien wrote:
The issue in note 13 is fixed by:
https://github.com/OISF/suricata/pull/3673/commits/2bd23bc1d511801469d4736ef85efeb373992411
Updated ticket 2841
Updated by Victor Julien over 5 years ago
Are you able to test the icmp issue against the latest git master? I made a fix for an issue that is possibly related. Thanks!
Updated by Victor Julien over 5 years ago
- Status changed from Assigned to Feedback
- Target version changed from 5.0beta1 to TBD
Updated by Charles Dillard over 5 years ago
Hi Victor, our security team tells us after enabling the github version you provided on a server (that originally had suricata 3.0) the team tells me that the problem is still there "....Using the same test as before I got the alert for HTTP traffic but ICMP protocol..." Let me check with them and see if I can provide output details, stand by.
Updated by Andreas Herz over 5 years ago
Did you have a chance to get more details?
Updated by Victor Julien about 5 years ago
- Status changed from Feedback to Assigned
- Target version changed from TBD to 4.1.5
I received a test case and will be working on a fix.
Updated by Victor Julien about 5 years ago
- Status changed from Assigned to Closed
- Target version changed from 4.1.5 to 5.0rc1