Feature #2695
closedwebsocket support
Description
At Suricon2018 support for WebSockets was requested.
Jason Ish, Danny Browning and Matt offered to work on this.
Rust would be preferred.
Files
Updated by Victor Julien almost 6 years ago
- Related to Task #2685: SuriCon 2018 brainstorm added
Updated by Bryant Smith over 5 years ago
I have a Lua script I've developed to unmask websockets. I can add this to the git repo while a rust parser is being created.
Updated by Jason Ish almost 5 years ago
- Related to Feature #3285: rules: XOR keyword added
Updated by Brandon Murphy over 2 years ago
just wanted to bump this request WebSockets is still used by malware, recently case of it is documented here https://isc.sans.edu/forums/diary/Keep+an+Eye+on+WebSockets/28430/
Updated by Brandon Murphy over 2 years ago
just another bump - APT actor using websockets
https://cert.gov.ua/article/37704
Translated to English -- https://cert-gov-ua.translate.goog/article/37704?_x_tr_sl=uk&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp
I was able to confirm this traffic using WebSockets - can share a pcap privately if needed, but looks like standard websockets use here.
Updated by Brandon Murphy almost 2 years ago
redteam tooling using WebSockets
https://github.com/dobin/antnium
Updated by Brandon Murphy about 1 year ago
just another example of a RAT using websockets
https://asec.ahnlab.com/en/52899/
Source Code: https://github.com/XZB-1248/Spark/
Updated by Jason Ish about 1 year ago
- Status changed from Assigned to New
- Assignee changed from Jason Ish to Community Ticket
Updated by Philippe Antoine 12 months ago
- Related to Task #6443: Suricon 2023 brainstorm added
Updated by Philippe Antoine 12 months ago
@Brandon Murphy do you have pcaps for this ?
Updated by Brandon Murphy 12 months ago
- File basic_websockets.pcap basic_websockets.pcap added
yeah sure thing! Keep in mind this pcap might not be representative of all websocket complexities, it's pretty basic, but should demonstrate it well enough.
I can share some SparkRAT (mentioned above) pcaps privately.
Updated by Peter Manev 12 months ago
Also this pcap - thanks to AnyRun - https://app.any.run/tasks/caf8a256-6249-47ec-8de0-80b5d6049874/
Updated by Peter Manev 12 months ago
Also private pcap shared, thanks Brandon !
Updated by Victor Julien 12 months ago
What would be the expectation wrt detection?
Minimum I can think of is keywords for:
- message header
- message payload raw
- message payload unmasked
But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?
Updated by Victor Julien 12 months ago
- Assignee changed from Community Ticket to OISF Dev
- Priority changed from Normal to High
- Target version changed from TBD to 8.0.0-beta1
Updated by Victor Julien 11 months ago
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Philippe Antoine
Updated by Brandon Murphy 11 months ago
Victor Julien wrote in #note-15:
What would be the expectation wrt detection?
Minimum I can think of is keywords for:
- message header
- message payload raw
- message payload unmasked
message payload unmasked would be key to detection.
would be nice to see logic that handles to_client/to_server in these messages as well.
outside of what you have, i could see the actual key being a buffer (while typically dynamic, actors do stupid things sometimes)
But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?
I'm not sure I understand this question. like a single buffer with the contents of all messages?
Updated by Philippe Antoine 11 months ago
- Status changed from Assigned to In Progress
But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?
I'm not sure I understand this question. like a single buffer with the contents of all messages?
There can be a websocket reassembly : each PDU indicates if it is final.
Should we do this reassembly (as the one in DCERPC for instance) ?
Cf wireshark filter websocket.fin == False
Updated by Philippe Antoine 11 months ago
Updated by Jane Ostin 11 months ago
- File websocket_slo88.pcap websocket_slo88.pcap added
Hello, I have attached another traffic dump.
Suggest that if this is not a single buffer, then adding the payload length. Something like dsize only for websocket.
Updated by Victor Julien 11 months ago
Thanks @Jane Ostin. In the pcap the websocket messages look like self contained JSON documents to me, so merging together several messages does not seem needed here.
Wrt the dsize like logic, each new buffer will automatically support bsize:<size>
.
Updated by Brandon Murphy 11 months ago
Philippe Antoine wrote in #note-20:
But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?
I'm not sure I understand this question. like a single buffer with the contents of all messages?
There can be a websocket reassembly : each PDU indicates if it is final.
Should we do this reassembly (as the one in DCERPC for instance) ?Cf wireshark filter
websocket.fin == False
Ahh, yes this helped my understanding a lot. Took me awhile to find some malware samples where this is behavior was found. Yes, I believe this should be reassembled. It looks like some implementations are unique in their behavior as well, attached is one example where all messages are continued, but then a completed message is sent along with it.
Updated by Brandon Murphy 11 months ago
Today presented itself with a malware sample using websocket, so I compiled the latest branch of this and did some testing.
combined elements of HTTP and Websocket Payload¶
As mentioned within https://github.com/OISF/suricata/pull/10014#issuecomment-1847804663, it would be nice if there is some ability to have elements of the HTTP request within the rule. For example
alert websocket ... http.user_agent; content:"foo"; websocket.payload; content:"bar"; ...
I attempted to do this with the branch (2695v4) but received the following error
[108 - Suricata-Main] 2023-12-18 18:43:48 Notice: suricata: This is Suricata version 8.0.0-dev (77d1c7787 2023-12-14) running in SYSTEM mode [snip] [109 - Suricata-Main] 2023-12-18 18:43:48 Error: detect-parse: can't set rule app proto to http_any: already set to websocket [109 - Suricata-Main] 2023-12-18 18:43:48 Error: detect: error parsing signature "alert websocket any any -> any any (msg:"does this work"; flow:established; http.method; content:"GET"; websocket.payload; content:"bar"; sid:1;)" from file /tmp/57483caf51bb5b43_Dec-18-2023_18-43-48/dalton-custom.rules at line 1
the only way I've been able to combine the elements so far is with flowbits.
alert http any any -> any any (msg:"flowbit set"; flow:established,to_server; flowbits:set,test1; http.method; content:"GET"; http.uri; content:"/socket.io/"; sid:1;) alert websocket any any -> any any (msg:"flowbit isset"; flow:established,to_server; flowbits:isset,test1; websocket.payload; content:"bar"; sid:2;)
However, this presents it's own issues, as mentioned in https://redmine.openinfosecfoundation.org/issues/2486, there is not always a good fast_pattern in a single element (in this case either the HTTP request, or the websocket payload). This often results in the rules getting disabled by default, or not created at all. Having a single rule which combines the elements (alert http ... http.method; content:"GET"; http.uri; content:"/socket.io/"; websocket.payload; content:"bar"; ...
allows for a rule writer to select the best fast_pattern between both elements.
Message Payload Logging¶
Is it possible to include an option to log message payloads, perhaps like the http body logging option for HTTP that exists.
payload: yes # enable dumping payload in Base64 payload-buffer-size: 4kb # max size of payload buffer to output in eve-log payload-printable: yes # enable dumping payload in printable (lossy) format
edited* add reason why flowbit solution can lead to rules not being created/enabled.
Updated by Philippe Antoine 11 months ago
- Status changed from In Progress to In Review
Updated by Brandon Murphy 11 months ago
Interesting, I didn't realize until today that websocket over HTTP/2 is a bit different. it might impact protocol detection? i'm not sure, but noticed it's missing from this convo, so though I'd point it out.
Updated by Philippe Antoine 10 months ago
combined elements of HTTP and Websocket Payload
Thanks, I get the fast_pattern argument against flow bits.
This looks hard to implement though.
To be revisited when PR is ready to see if this is a next ticket...
Updated by Philippe Antoine 10 months ago
Brandon Murphy wrote in #note-27:
Interesting, I didn't realize until today that websocket over HTTP/2 is a bit different. it might impact protocol detection? i'm not sure, but noticed it's missing from this convo, so though I'd point it out.
@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?
The one there https://github.com/ckreibich/httplab/tree/master/pcaps does not seem to successfully switch to websocket
Updated by Philippe Antoine 10 months ago
Message Payload Logging
This is only for alerts ?
Or for websocket events as well ?
Cf http-body-printable
in suricata.yaml
Updated by Brandon Murphy 10 months ago
Philippe Antoine wrote in #note-29:
@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?
I don’t, but I can look at creating one if it would help
Philippe Antoine wrote in #note-30:
Message Payload Logging
This is only for alerts ?
Or for websocket events as well ?
Ideally both. I’d just mirror whatever the behavior/options are for HTTP. I think for HTTP it is configured at both the alert (via the metadata option?) and the HTTP logging level.
Updated by Philippe Antoine 10 months ago
@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?
I don’t, but I can look at creating one if it would help
It would help very much :-)
Message Payload Logging
This is only for alerts ?
Or for websocket events as well ?Ideally both. I’d just mirror whatever the behavior/options are for HTTP. I think for HTTP it is configured at both the alert (via the metadata option?) and the HTTP logging level.
I do not see the option for HTTP. I only see it for alerts (http-body-printable
in suricata.yaml)
Updated by Brandon Murphy 10 months ago
- File wss_h2.pcap wss_h2.pcap added
I could have sworn I replied to this yeterday, but can't find it...
Philippe Antoine wrote in #note-32:
@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?
I don’t, but I can look at creating one if it would help
It would help very much :-)
As requested see "wss_h2.pcap"
Message Payload Logging
This is only for alerts ?
Or for websocket events as well ?Ideally both. I’d just mirror whatever the behavior/options are for HTTP. I think for HTTP it is configured at both the alert (via the metadata option?) and the HTTP logging level.
I do not see the option for HTTP. I only see it for alerts (
http-body-printable
in suricata.yaml)
Yup, you're right. Idk where I was getting that from... maybe the old "forensics" logging that eric was working on...
So, the only reason I could see a use for putting it in the websocket events is because getting the unmasked messages via wireshark is a slow process and it'd be nice to have something parse them all out quickly. But Suricata doesn't need to be that solution/tool. I"m sure there's a tshark way of doing it. So, I'd be fine with just leaving it at the alert level
Updated by Philippe Antoine 9 months ago
- Blocked by Optimization #3827: clean up logging initialization code added
Updated by Philippe Antoine 9 months ago
- Related to Feature #6729: websockets: support over HTTP/2 added
Updated by Philippe Antoine 7 months ago
- Status changed from In Review to Closed
Updated by Philippe Antoine 6 months ago
- Related to Bug #7025: websocket: wrong value for opcode ping/pong added
Updated by Victor Julien 4 months ago
- Blocks Task #7118: tracking: add support for new protocols added
Updated by Victor Julien 4 months ago
- Blocks Story #7119: protocols: protocol additions added