Bug #1275
closedET Rule 2003927 not matchin in suricata
Description
I have the following rule (from ET) included in my suricata.yaml:
drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET TROJAN Suspicious User-Agent (HTTPTEST) - Seen used by downloaders"; flow:to_server,established; content:"User-Agent|3a| HTTPTEST"; nocase; http_header; threshold: type limit, count 2, track by_src, seconds 300;reference:url,doc.emergingthreats.net/bin/view/Main/2003927; classtype:trojan-activity; sid:2003927; rev:9;)
This rule is detected by snort but not by suricata 2.0.3 when i use the following command:
exec 80<>/dev/tcp/10.0.13.134/80 && ( echo -e 'GET / HTTP/1.0\r\nUser-Agent: Autotest httpRequest\r\nUser-Agent: HTTPTEST\r\n\r\n' >&80; while read -t 10 <&80; do echo $REPLY; done; exec 80<&- 80>&- )
It's working as soon as i remove the first User-Agent. But IMHO the rule should even match with the additional User-Agent in front of the HTTPTEST.
Victor said it could be that it's rejected in an earlier stage, but that means such traffic could go through although i wanted to drop it.
Files
Updated by Andreas Herz about 10 years ago
- File log.pcap.1410875272 log.pcap.1410875272 added
The related pcap file:
Updated by Victor Julien about 10 years ago
- Status changed from New to Feedback
Please add results of http-events.rules against this traffic.
Updated by Andreas Herz about 10 years ago
As requested i testet it against the http-events.rules, but they don't trigger either.
Updated by Victor Julien about 10 years ago
Interestingly, it seems that the 2 user-agent headers are merged:
09/16/2014-15:47:52.901650 <hostname unknown> [**] / [**] Autotest httpRequest, HTTPTEST [**] 10.0.13.39:37150 -> 10.0.13.134:80
Updated by Victor Julien about 10 years ago
Looks like libhtp does this unconditionally: https://github.com/ironbee/libhtp/blob/0.5.15/htp/htp_request_generic.c#L66
Updated by Andreas Herz about 10 years ago
So is this intended or something that should be "fixed"?
Updated by Andreas Herz almost 10 years ago
Shall i commit a bug report on github for libhtp or is this already known in upstream libhtp?
Updated by Andreas Herz almost 9 years ago
- Assignee set to Andreas Herz
- Target version set to TBD
Updated by Andreas Herz over 8 years ago
Still the same with 3.0 and libhtp 0.5.18 so worth reporting to libhtp upstream?
Updated by Andreas Herz about 6 years ago
- Assignee changed from Andreas Herz to Anonymous
Updated by Victor Julien about 5 years ago
- Status changed from Feedback to Assigned
- Assignee changed from Community Ticket to Philippe Antoine
Philippe, could you see what we can do about this?
Updated by Philippe Antoine about 5 years ago
Well, I see this as about Suricata rule keywords http_header and http_raw_header definition
The rule triggers when we use http_raw_header instead of http_header
The rule does not trigger with http_header keyword because there is a normalization step first
That normalization includes concatenating together the different values for a single header (This is done by libhtp).
That is transforming
User-Agent: Autotest httpRequest
User-Agent: HTTPTEST
into
User-Agent: Autotest httpRequest, HTTPTEST
Maybe the best solution would be to have some keywords name and value that we could use with http_header
Updated by Victor Julien about 5 years ago
I think the normalization is not working in our advantage. I think we should disable the normalization and instead consider this buffer to have 1-N values, instead of just a single one.
This will be quite involved, as it would first require libhtp changes to store this correctly somehow, then in suri we would need to iterate over the 1-N values in detection and logging. The logging (in eve) means we need to change it from scalar to array probably.
Updated by Philippe Antoine about 5 years ago
then in suri we would need to iterate over the 1-N values in detection
Do you mean that we should then reconstruct N buffers concatenating the one buffered header name with each of the N buffered values ?
Or is there already a motor to do detection across multiple buffers ?
And should we introduce new keywords so as not to break compatibility ?
Updated by Victor Julien almost 5 years ago
At suricon we briefly discussed this issue. I tend to see it as a bug rather than a feature. But I think Philippe you mentioned its documented as a feature?
Updated by Philippe Antoine almost 5 years ago
But I think Philippe you mentioned its documented as a feature?
We have the documentation :
https://suricata.readthedocs.io/en/suricata-5.0.0/rules/http-keywords.html#http-header-and-http-raw-header
which says that the header buffer is normalized, but does not explicitly says that this includes concatenating values
But the concatenation is mentioned explicitly for user agent :
https://suricata.readthedocs.io/en/suricata-4.1.4/rules/http-keywords.html#notes
the values will be concatenated in the http_user_agent buffer, in the order seen from top to bottom, with a comma and space (“, “) between each of them.
What do you think about having http_header_name and http_header_value keywords ?
(Maybe complicated for development, but seems useful for signatures writers)
Updated by Philippe Antoine over 4 years ago
- Status changed from Assigned to Feedback
Updated by Philippe Antoine over 4 years ago
- Status changed from Feedback to In Review
- Assignee changed from Philippe Antoine to Andreas Herz
Updated by Philippe Antoine over 4 years ago
- Assignee changed from Andreas Herz to Philippe Antoine
Updated by Philippe Antoine about 4 years ago
- Target version changed from TBD to 7.0.0-beta1
Updated by Philippe Antoine almost 4 years ago
- Target version changed from 7.0.0-beta1 to 6.0.1
Updated by Victor Julien almost 4 years ago
- Status changed from In Review to Closed
Merged pr/5173. I still would like to consider if this behavior is correct, but for now its enough that it is documented.