Project

General

Profile

Actions

Feature #2695

closed

websocket support

Added by Victor Julien almost 6 years ago. Updated 7 months ago.

Status:
Closed
Priority:
High
Target version:
Effort:
Difficulty:
Label:
Protocol

Description

At Suricon2018 support for WebSockets was requested.

Jason Ish, Danny Browning and Matt offered to work on this.

Rust would be preferred.


Files

basic_websockets.pcap (2.91 KB) basic_websockets.pcap Brandon Murphy, 11/10/2023 05:52 AM
websocket_slo88.pcap (265 KB) websocket_slo88.pcap Jane Ostin, 12/10/2023 07:59 AM
websockets_continued.pcap (10.1 KB) websockets_continued.pcap Brandon Murphy, 12/18/2023 06:34 PM
wss_h2.pcap (71.8 KB) wss_h2.pcap Brandon Murphy, 01/05/2024 06:35 PM

Related issues 8 (5 open3 closed)

Related to Suricata - Task #2685: SuriCon 2018 brainstormAssignedVictor JulienActions
Related to Suricata - Feature #3285: rules: XOR keywordClosedPhilippe AntoineActions
Related to Suricata - Task #6443: Suricon 2023 brainstormAssignedVictor JulienActions
Related to Suricata - Feature #6729: websockets: support over HTTP/2NewPhilippe AntoineActions
Related to Suricata - Bug #7025: websocket: wrong value for opcode ping/pongClosedPhilippe AntoineActions
Blocked by Suricata - Optimization #3827: clean up logging initialization codeClosedPhilippe AntoineActions
Blocks Suricata - Task #7118: tracking: add support for new protocolsAssignedVictor JulienActions
Blocks Suricata - Story #7119: protocols: protocol additionsNewVictor JulienActions
Actions #1

Updated by Victor Julien almost 6 years ago

  • Related to Task #2685: SuriCon 2018 brainstorm added
Actions #2

Updated by Bryant Smith over 5 years ago

I have a Lua script I've developed to unmask websockets. I can add this to the git repo while a rust parser is being created.

Actions #3

Updated by Jason Ish almost 5 years ago

Actions #4

Updated by Victor Julien almost 5 years ago

  • Label Protocol added
Actions #5

Updated by Brandon Murphy over 2 years ago

just wanted to bump this request WebSockets is still used by malware, recently case of it is documented here https://isc.sans.edu/forums/diary/Keep+an+Eye+on+WebSockets/28430/

Actions #6

Updated by Brandon Murphy over 2 years ago

just another bump - APT actor using websockets

https://cert.gov.ua/article/37704
Translated to English -- https://cert-gov-ua.translate.goog/article/37704?_x_tr_sl=uk&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp

I was able to confirm this traffic using WebSockets - can share a pcap privately if needed, but looks like standard websockets use here.

Actions #7

Updated by Brandon Murphy almost 2 years ago

redteam tooling using WebSockets
https://github.com/dobin/antnium

Actions #8

Updated by Brandon Murphy about 1 year ago

just another example of a RAT using websockets

https://asec.ahnlab.com/en/52899/

Source Code: https://github.com/XZB-1248/Spark/

Actions #9

Updated by Jason Ish about 1 year ago

  • Status changed from Assigned to New
  • Assignee changed from Jason Ish to Community Ticket
Actions #10

Updated by Philippe Antoine 12 months ago

  • Related to Task #6443: Suricon 2023 brainstorm added
Actions #11

Updated by Philippe Antoine 12 months ago

@Brandon Murphy do you have pcaps for this ?

Actions #12

Updated by Brandon Murphy 12 months ago

yeah sure thing! Keep in mind this pcap might not be representative of all websocket complexities, it's pretty basic, but should demonstrate it well enough.

I can share some SparkRAT (mentioned above) pcaps privately.

Actions #13

Updated by Peter Manev 12 months ago

Actions #14

Updated by Peter Manev 12 months ago

Also private pcap shared, thanks Brandon !

Actions #15

Updated by Victor Julien 12 months ago

What would be the expectation wrt detection?

Minimum I can think of is keywords for:
- message header
- message payload raw
- message payload unmasked

But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?

Actions #17

Updated by Victor Julien 12 months ago

  • Assignee changed from Community Ticket to OISF Dev
  • Priority changed from Normal to High
  • Target version changed from TBD to 8.0.0-beta1
Actions #18

Updated by Victor Julien 11 months ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Philippe Antoine
Actions #19

Updated by Brandon Murphy 11 months ago

Victor Julien wrote in #note-15:

What would be the expectation wrt detection?

Minimum I can think of is keywords for:
- message header
- message payload raw
- message payload unmasked

message payload unmasked would be key to detection.

would be nice to see logic that handles to_client/to_server in these messages as well.

outside of what you have, i could see the actual key being a buffer (while typically dynamic, actors do stupid things sometimes)

But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?

I'm not sure I understand this question. like a single buffer with the contents of all messages?

Actions #20

Updated by Philippe Antoine 11 months ago

  • Status changed from Assigned to In Progress

But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?

I'm not sure I understand this question. like a single buffer with the contents of all messages?

There can be a websocket reassembly : each PDU indicates if it is final.
Should we do this reassembly (as the one in DCERPC for instance) ?

Cf wireshark filter websocket.fin == False

Actions #22

Updated by Jane Ostin 11 months ago

Hello, I have attached another traffic dump.
Suggest that if this is not a single buffer, then adding the payload length. Something like dsize only for websocket.

Actions #23

Updated by Victor Julien 11 months ago

Thanks @Jane Ostin. In the pcap the websocket messages look like self contained JSON documents to me, so merging together several messages does not seem needed here.

Wrt the dsize like logic, each new buffer will automatically support bsize:<size>.

Actions #24

Updated by Brandon Murphy 11 months ago

Philippe Antoine wrote in #note-20:

But would there be a requirement to reassemble (unmasked) message payloads into a continues buffer as well?

I'm not sure I understand this question. like a single buffer with the contents of all messages?

There can be a websocket reassembly : each PDU indicates if it is final.
Should we do this reassembly (as the one in DCERPC for instance) ?

Cf wireshark filter websocket.fin == False

Ahh, yes this helped my understanding a lot. Took me awhile to find some malware samples where this is behavior was found. Yes, I believe this should be reassembled. It looks like some implementations are unique in their behavior as well, attached is one example where all messages are continued, but then a completed message is sent along with it.

Actions #25

Updated by Brandon Murphy 11 months ago

Today presented itself with a malware sample using websocket, so I compiled the latest branch of this and did some testing.

combined elements of HTTP and Websocket Payload

As mentioned within https://github.com/OISF/suricata/pull/10014#issuecomment-1847804663, it would be nice if there is some ability to have elements of the HTTP request within the rule. For example

alert websocket ... http.user_agent; content:"foo"; websocket.payload; content:"bar"; ...

I attempted to do this with the branch (2695v4) but received the following error

[108 - Suricata-Main] 2023-12-18 18:43:48 Notice: suricata: This is Suricata version 8.0.0-dev (77d1c7787 2023-12-14) running in SYSTEM mode
[snip]
[109 - Suricata-Main] 2023-12-18 18:43:48 Error: detect-parse: can't set rule app proto to http_any: already set to websocket
[109 - Suricata-Main] 2023-12-18 18:43:48 Error: detect: error parsing signature "alert websocket any any -> any any (msg:"does this work"; flow:established; http.method; content:"GET"; websocket.payload; content:"bar"; sid:1;)" from file /tmp/57483caf51bb5b43_Dec-18-2023_18-43-48/dalton-custom.rules at line 1

the only way I've been able to combine the elements so far is with flowbits.

alert http any any -> any any (msg:"flowbit set"; flow:established,to_server; flowbits:set,test1; http.method; content:"GET"; http.uri; content:"/socket.io/"; sid:1;)
alert websocket any any -> any any (msg:"flowbit isset"; flow:established,to_server; flowbits:isset,test1; websocket.payload; content:"bar"; sid:2;)

However, this presents it's own issues, as mentioned in https://redmine.openinfosecfoundation.org/issues/2486, there is not always a good fast_pattern in a single element (in this case either the HTTP request, or the websocket payload). This often results in the rules getting disabled by default, or not created at all. Having a single rule which combines the elements (alert http ... http.method; content:"GET"; http.uri; content:"/socket.io/"; websocket.payload; content:"bar"; ... allows for a rule writer to select the best fast_pattern between both elements.

Message Payload Logging

Is it possible to include an option to log message payloads, perhaps like the http body logging option for HTTP that exists.

            payload: yes             # enable dumping payload in Base64
            payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
            payload-printable: yes   # enable dumping payload in printable (lossy) format

edited* add reason why flowbit solution can lead to rules not being created/enabled.

Actions #26

Updated by Philippe Antoine 11 months ago

  • Status changed from In Progress to In Review
Actions #27

Updated by Brandon Murphy 11 months ago

Interesting, I didn't realize until today that websocket over HTTP/2 is a bit different. it might impact protocol detection? i'm not sure, but noticed it's missing from this convo, so though I'd point it out.

https://www.rfc-editor.org/rfc/rfc8441

Actions #28

Updated by Philippe Antoine 10 months ago

combined elements of HTTP and Websocket Payload

Thanks, I get the fast_pattern argument against flow bits.
This looks hard to implement though.
To be revisited when PR is ready to see if this is a next ticket...

Actions #29

Updated by Philippe Antoine 10 months ago

Brandon Murphy wrote in #note-27:

Interesting, I didn't realize until today that websocket over HTTP/2 is a bit different. it might impact protocol detection? i'm not sure, but noticed it's missing from this convo, so though I'd point it out.

https://www.rfc-editor.org/rfc/rfc8441

@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?

The one there https://github.com/ckreibich/httplab/tree/master/pcaps does not seem to successfully switch to websocket

Actions #30

Updated by Philippe Antoine 10 months ago

Message Payload Logging

This is only for alerts ?
Or for websocket events as well ?

Cf http-body-printable in suricata.yaml

Actions #31

Updated by Brandon Murphy 10 months ago

Philippe Antoine wrote in #note-29:

@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?

I don’t, but I can look at creating one if it would help

Philippe Antoine wrote in #note-30:

Message Payload Logging

This is only for alerts ?
Or for websocket events as well ?

Ideally both. I’d just mirror whatever the behavior/options are for HTTP. I think for HTTP it is configured at both the alert (via the metadata option?) and the HTTP logging level.

Actions #32

Updated by Philippe Antoine 10 months ago

@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?

I don’t, but I can look at creating one if it would help

It would help very much :-)

Message Payload Logging

This is only for alerts ?
Or for websocket events as well ?

Ideally both. I’d just mirror whatever the behavior/options are for HTTP. I think for HTTP it is configured at both the alert (via the metadata option?) and the HTTP logging level.

I do not see the option for HTTP. I only see it for alerts (http-body-printable in suricata.yaml)

Actions #33

Updated by Brandon Murphy 10 months ago

I could have sworn I replied to this yeterday, but can't find it...

Philippe Antoine wrote in #note-32:

@Brandon Murphy would you have a Websockets over/after HTTP/2 pcap ?

I don’t, but I can look at creating one if it would help

It would help very much :-)

As requested see "wss_h2.pcap"

Message Payload Logging

This is only for alerts ?
Or for websocket events as well ?

Ideally both. I’d just mirror whatever the behavior/options are for HTTP. I think for HTTP it is configured at both the alert (via the metadata option?) and the HTTP logging level.

I do not see the option for HTTP. I only see it for alerts (http-body-printable in suricata.yaml)

Yup, you're right. Idk where I was getting that from... maybe the old "forensics" logging that eric was working on...

So, the only reason I could see a use for putting it in the websocket events is because getting the unmasked messages via wireshark is a slow process and it'd be nice to have something parse them all out quickly. But Suricata doesn't need to be that solution/tool. I"m sure there's a tshark way of doing it. So, I'd be fine with just leaving it at the alert level

Actions #34

Updated by Philippe Antoine 10 months ago

Thank you for this awesome pcap :-)

Actions #35

Updated by Philippe Antoine 9 months ago

Actions #36

Updated by Philippe Antoine 9 months ago

  • Related to Feature #6729: websockets: support over HTTP/2 added
Actions #37

Updated by Philippe Antoine 7 months ago

  • Status changed from In Review to Closed
Actions #38

Updated by Philippe Antoine 6 months ago

  • Related to Bug #7025: websocket: wrong value for opcode ping/pong added
Actions #39

Updated by Victor Julien 4 months ago

  • Blocks Task #7118: tracking: add support for new protocols added
Actions #40

Updated by Victor Julien 4 months ago

  • Blocks Story #7119: protocols: protocol additions added
Actions

Also available in: Atom PDF