Project

General

Profile

Actions

Bug #6959

closed

improve handling of content encoding: gzip but request_body not actually compressed

Added by Brandon Murphy 7 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

Looking for a little help here.

Context

We have a rule (simplified here) designed to alert on traffic contained within 2038629_single_fn_synth.pcap (attached)

alert http $HOME_NET any -> $EXTERNAL_NET 8100 (msg:"ET MOBILE_MALWARE Android Dropper Checkin Activity (POST)"; flow:established,to_server; http.request_body; content:"|22|sdk_version|22 3a|"; content:"|22|app_package|22 3a 22|"; distance:0; content:"|22|app_version|22 3a 22|"; distance:0; content:"|22|device_id|22 3a 22|"; distance:0; classtype:trojan-activity; sid:2038629; rev:1;)

The pcap contains the following HTTP transaction, please note the Content-Encoding: gzip header, though the request body is not actually compressed.

POST /api/sdk/init2 HTTP/1.1
Content-Encoding: gzip
Charset: UTF-8
Accept-Encoding: gzip,deflate
Content-Type: application/json
User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; Moto G (7) Build/OPPS28.65-37-7-11)
Host: sdk.xinxiansk.com:8100
Connection: Keep-Alive
Content-Length: 106

{"sdk_version":11026,"app_package":"com.xcm.huasheng","app_version":"1.7.4","device_id":"XXXX"}

Current Behavior

This signature fires using 6.0.18, but does not fire on 7.0.4. This discrepancy was found while testing the Suricata 7.0 Emerging Threats ruleset.

Analysis

Upon examination, with 7.0.4, it appears the condition of the Content-Encoding: gzip header is present while the content is not compressed produces an unexpected population of the http.request_body buffer.

Using the lua logging module to log the contents of the http.request_body buffer the following rule which alerts on 7.0.4 was produced.

alert http $HOME_NET any -> $EXTERNAL_NET 8100 (msg:"TEST"; flow:established,to_server; http.request_body; content:"|e4 1c c4 b5 8e 6b 79 63 83 75 8d 6d 61 83 75 8d|"; startswith; classtype:trojan-activity; sid:2; rev:1;)

Suricata 7.0.4 does produce the "GZIP_DECOMPRESSION_FAILED" anomaly event while 6.0.18 does not.

Outstanding Questions

- Is the observed difference in behavior between 6.0.18 and 7.0.4 expected?
- From where is the data within http.request_body; coming?

Desired Behavior

It would be my preference that the engine behave as it does in 6.0.18 as I currently have no method of writing the signature for 7.0.4 given the buffer is populated with data from an unknown source.


Files

2038629_single_fn_synth.pcap (1.29 KB) 2038629_single_fn_synth.pcap Brandon Murphy, 04/16/2024 02:02 AM

Subtasks 1 (0 open1 closed)

Bug #6966: improve handling of content encoding: gzip but request_body not actually compressed (7.0.x backport)ClosedPhilippe AntoineActions

Related issues 1 (0 open1 closed)

Related to Suricata - Task #6965: libhtp 0.5.48ClosedVictor JulienActions
Actions #1

Updated by Victor Julien 7 months ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Philippe Antoine

@Philippe Antoine can you check this?

Actions #2

Updated by Brandon Murphy 7 months ago

  • Description updated (diff)
Actions #3

Updated by Philippe Antoine 7 months ago

  • Status changed from Assigned to In Review
  • Target version changed from TBD to 8.0.0-beta1

Thanks for the report Brandon

Fix is in https://github.com/OISF/libhtp/pull/420

Actions #4

Updated by Victor Julien 7 months ago

Actions #5

Updated by Victor Julien 7 months ago

  • Label Needs backport to 7.0 added

Fix is in libhtp, but adding backport ticket for the Changelog update.

Actions #6

Updated by OISF Ticketbot 7 months ago

  • Subtask #6966 added
Actions #7

Updated by OISF Ticketbot 7 months ago

  • Label deleted (Needs backport to 7.0)
Actions #8

Updated by Victor Julien 7 months ago

  • Status changed from In Review to Closed
Actions #9

Updated by Victor Julien 6 months ago

  • Subject changed from behavior discrepancy between 6.0.18 and 7.0.4 when encountering Content-Encoding: gzip but request_body not actually compressed to improve handling of content encoding: gzip but request_body not actually compressed
Actions

Also available in: Atom PDF