Bug #428: Segfault when processing NSA pcap with http-events.rules - Suricata - Open Information Security Foundation

Actions

Copy link

Bug #428

closed

Segfault when processing NSA pcap with http-events.rules

Added by Xavier Lange over 12 years ago. Updated over 11 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Target version:

Affected Versions:

Effort:

Difficulty:

Label:

Description

I am running a file processing run against the NSA data from 2009-04-21-04-06-191 and only "http-events.rules" loaded.

Starting program: /Users/xavierlange/code/suricata/src/.libs/suricata -c suricata.yaml -r 2009-04-21-04-06-191
[3595] 21/3/2012 -- 00:36:52 - (suricata.c:1151) <Info> (main) -- This is Suricata version 1.3dev (rev 8350fdd)
[3595] 21/3/2012 -- 00:36:52 - (util-cpu.c:171) <Info> (UtilCpuPrintSummary) -- CPUs/cores online: 8
[3595] 21/3/2012 -- 00:36:52 - (suricata.c:1588) <Info> (main) -- preallocated 50 packets. Total memory 216900
[3595] 21/3/2012 -- 00:36:52 - (flow.c:930) <Info> (FlowInitConfig) -- allocated 4718592 bytes of memory for the flow hash... 65536 buckets of size 72
[3595] 21/3/2012 -- 00:36:52 - (flow.c:950) <Info> (FlowInitConfig) -- preallocated 10000 flows of size 376
[3595] 21/3/2012 -- 00:36:52 - (flow.c:952) <Info> (FlowInitConfig) -- flow memory usage: 8478592 bytes, maximum: 33554432
[3595] 21/3/2012 -- 00:36:52 - (util-classification-config.c:329) <Info> (SCClassConfParseFile) -- Added "34" classification types from the classification file
[3595] 21/3/2012 -- 00:36:52 - (util-reference-config.c:306) <Info> (SCRConfParseFile) -- Added "12" reference types from the reference.config file
[3595] 21/3/2012 -- 00:36:52 - (util-magic.c:62) <Info> (MagicInit) -- using magic-file /usr/share/file/magic
[3595] 21/3/2012 -- 00:36:52 - (detect.c:660) <Info> (SigLoadSignatures) -- 1 rule files processed. 24 rules succesfully loaded, 0 rules failed
[3595] 21/3/2012 -- 00:36:52 - (detect.c:2500) <Info> (SigAddressPrepareStage1) -- 24 signatures processed. 0 are IP-only rules, 0 are inspecting packet payload, 24 inspect application layer, 0 are decoder event only
[3595] 21/3/2012 -- 00:36:52 - (detect.c:2503) <Info> (SigAddressPrepareStage1) -- building signature grouping structure, stage 1: adding signatures to signature source addresses... complete
[3595] 21/3/2012 -- 00:36:52 - (detect.c:3127) <Info> (SigAddressPrepareStage2) -- building signature grouping structure, stage 2: building source address list... complete
[3595] 21/3/2012 -- 00:36:52 - (detect.c:3787) <Info> (SigAddressPrepareStage3) -- building signature grouping structure, stage 3: building destination address lists... complete
[3595] 21/3/2012 -- 00:36:52 - (util-threshold-config.c:135) <Warning> (SCThresholdConfInitContext) -- [ERRCODE: SC_ERR_FOPEN(44)] - Error opening file: "threshold.config": No such file or directory
[3595] 21/3/2012 -- 00:36:52 - (util-coredump-config.c:122) <Info> (CoredumpLoadConfig) -- Core dump size set to unlimited.
[3595] 21/3/2012 -- 00:36:52 - (util-logopenfile.c:168) <Info> (SCConfLogOpenGeneric) -- fast output device (regular) initialized: fast.log
[3595] 21/3/2012 -- 00:36:52 - (alert-unified2-alert.c:1212) <Info> (Unified2AlertInitCtx) -- Unified2-alert initialized: filename unified2.alert, limit 32 MB
[3595] 21/3/2012 -- 00:36:52 - (util-logopenfile.c:168) <Info> (SCConfLogOpenGeneric) -- http-log output device (regular) initialized: http.log
[5635] 21/3/2012 -- 00:36:52 - (source-pcap-file.c:212) <Info> (ReceivePcapFileThreadInit) -- reading pcap file 2009-04-21-04-06-191
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:349) <Info> (StreamTcpInitConfig) -- stream "max-sessions": 262144
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:361) <Info> (StreamTcpInitConfig) -- stream "prealloc-sessions": 32768
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:377) <Info> (StreamTcpInitConfig) -- stream "memcap": 33554432
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:383) <Info> (StreamTcpInitConfig) -- stream "midstream" session pickups: disabled
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:389) <Info> (StreamTcpInitConfig) -- stream "async-oneside": disabled
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:406) <Info> (StreamTcpInitConfig) -- stream "checksum-validation": enabled
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:416) <Info> (StreamTcpInitConfig) -- stream."inline": disabled
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:434) <Info> (StreamTcpInitConfig) -- stream.reassembly "memcap": 67108864
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:452) <Info> (StreamTcpInitConfig) -- stream.reassembly "depth": 1048576
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:493) <Info> (StreamTcpInitConfig) -- stream.reassembly "toserver-chunk-size": 2560
[3595] 21/3/2012 -- 00:36:52 - (stream-tcp.c:495) <Info> (StreamTcpInitConfig) -- stream.reassembly "toclient-chunk-size": 2560
[3595] 21/3/2012 -- 00:36:52 - (tm-threads.c:1825) <Info> (TmThreadWaitOnThreadInit) -- all 14 packet processing threads, 3 management threads initialized, engine started.
[5635] 21/3/2012 -- 00:37:11 - (source-pcap-file.c:189) <Info> (ReceivePcapFileLoop) -- pcap file end of file reached (pcap err code 0)
[3595] 21/3/2012 -- 00:37:11 - (suricata.c:1742) <Info> (main) -- stopping engine, waiting for outstanding packets
[3595] 21/3/2012 -- 00:37:11 - (suricata.c:1777) <Info> (main) -- all packets processed by threads, stopping engine
[9219] 21/3/2012 -- 00:37:11 - (flow-manager.c:293) <Info> (FlowManagerThread) -- 0 new flows, 0 established flows were timed out, 0 flows in closed state
[3595] 21/3/2012 -- 00:37:11 - (suricata.c:1806) <Info> (main) -- time elapsed 18.471s
[5635] 21/3/2012 -- 00:37:11 - (source-pcap-file.c:278) <Info> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 972863 packets, 984435426 bytes
[5635] 21/3/2012 -- 00:37:11 - (stream-tcp.c:3995) <Info> (StreamTcpExitPrintStats) -- Stream TCP processed 913123 TCP packets
[8963] 21/3/2012 -- 00:37:11 - (alert-fastlog.c:331) <Info> (AlertFastLogExitPrintStats) -- Fast log output wrote 0 alerts
[8963] 21/3/2012 -- 00:37:11 - (alert-unified2-alert.c:1132) <Info> (Unified2AlertThreadDeinit) -- Alert unified2 module wrote 0 alerts
[8963] 21/3/2012 -- 00:37:11 - (log-httplog.c:397) <Info> (LogHttpLogExitPrintStats) -- HTTP logger logged 47 requests
[3595] 21/3/2012 -- 00:37:11 - (stream-tcp-reassemble.c:363) <Info> (StreamTcpReassembleFree) -- Max memuse of the stream reassembly engine 11292544 (in use 0)
[3595] 21/3/2012 -- 00:37:11 - (stream-tcp.c:540) <Info> (StreamTcpFreeConfig) -- Max memuse of stream engine 6029312 (in use 0)
suricata(71519,0x7fff71e9c960) malloc: *** error for object 0x10271fa50: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

Breakpoint 1, 0x00007fff829f96c0 in malloc_error_break ()
(gdb) bt
#0  0x00007fff829f96c0 in malloc_error_break ()
#1  0x00007fff829f9805 in free ()
#2  0x000000010008157b in SigGroupHeadFree (sgh=0x7fff5fbe5798) at detect-engine-siggroup.c:181
#3  0x000000010007b7c5 in DetectPortFree (dp=0x1027119c0) at detect-engine-port.c:90
#4  0x000000010007b7f1 in DetectPortCleanupList [inlined] () at /Users/xavierlange/code/suricata/src/detect-engine-port.c:176
#5  0x000000010007b7f1 in DetectPortFree (dp=0x102711640) at detect-engine-port.c:95
#6  0x000000010007b841 in DetectPortCleanupList (head=Cannot access memory at address 0x0
) at detect-engine-port.c:176
#7  0x000000010006a951 in DetectAddressFree (ag=0x7fff5fbe5798) at detect-engine-address.c:118
#8  0x000000010006c121 in DetectAddressCleanupList [inlined] () at /Users/xavierlange/code/suricata/src/detect-engine-address.c:257
#9  0x000000010006c121 in DetectAddressHeadCleanup (gh=0x10271f800) at detect-engine-address.c:1313
#10 0x000000010006a936 in DetectAddressHeadFree [inlined] () at /Users/xavierlange/code/suricata/src/detect-engine-address.c:1333
#11 0x000000010006a936 in DetectAddressFree (ag=0x7fff5fbe5798) at detect-engine-address.c:110
#12 0x000000010006c121 in DetectAddressCleanupList [inlined] () at /Users/xavierlange/code/suricata/src/detect-engine-address.c:257
#13 0x000000010006c121 in DetectAddressHeadCleanup (gh=0x1027179a0) at detect-engine-address.c:1313
#14 0x000000010006a990 in DetectAddressHeadFree (gh=Cannot access memory at address 0x0
) at detect-engine-address.c:1333
#15 0x000000010004a509 in SigAddressCleanupStage1 (de_ctx=0x7fff5fbe5798) at detect.c:3806
#16 0x000000010004a5f9 in SigGroupCleanup (de_ctx=Cannot access memory at address 0x0
) at detect.c:4438
#17 0x0000000100006871 in main (argc=1606416960, argv=0x7fff5fbffa40) at suricata.c:1837

Files

suricata.yaml (31.9 KB) suricata.yaml

Xavier Lange, 03/21/2012 03:15 AM

Actions

Copy link

Updated by Xavier Lange over 12 years ago

Description updated (diff)

Actions

Copy link

Updated by Xavier Lange over 12 years ago

File suricata.yaml suricata.yaml added

Actions

Copy link

Updated by Peter Manev over 12 years ago

I can investigate that later today and hopefully give you a much smaller pcap where we can reproduce the issue.
is that ok?

Can you please send me/post a link to the pcap file?
Is there any special way that you compile/run Suricata with?

thanks

Actions

Copy link

Updated by Xavier Lange over 12 years ago

Hi Peter,

I am using the dataset linked to in https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Public_Data_Sets, found at http://www.itoc.usma.edu/research/dataset/. Direct link: http://www.itoc.usma.edu/research/dataset/data/2009-04-21-04-06-191 .It is a 953MB PCAP with the MD5 790fa7d06392944e6e760aabb0bb6ba7 (my local file MD5 checksummed correctly).

xavierlange $> uname -a
Darwin unknown68a86d237024 11.2.0 Darwin Kernel Version 11.2.0: Tue Aug 9 20:54:00 PDT 2011; root:xnu-1699.24.8~1/RELEASE_X86_64 x86_64
AKA, OSX 10.7.

I can reproduce the error with a regular "./configure && make clean all" and a "./configure --enable-debug && make clean all". The stacktraces are the same.

Actions

Copy link

Updated by Xavier Lange over 12 years ago

I just realized that the built in GCC on Mac OS X 10.6 (Lion) is not actually GCC. Apple supports all the GCC compiler flags but maps them to the CLANG compiler. I will install GCC from brew and recheck bug.

Actions

Copy link

Updated by Victor Julien over 12 years ago

Status changed from New to Assigned
Assignee set to Xavier Lange

Xavier, can you retest this (with clang if necessary)? Some issues that are possibly related were fixed.

Actions

Copy link

Updated by Xavier Lange over 12 years ago

Victor: sorry I missed this. I finally figured out what kind of instructions those were (AVX) and also found out how to disable them. No assembler I've built has been aware of AVX instructions.

CFLAGS="-mno-avx -O2" CC=gcc-4.7 ./configure

Builds without error. Now I need to test it on the pcap.

Actions

Copy link

Updated by Victor Julien over 12 years ago

No segv or even valgrind errors when I run that here. Is this a OSX only issue?

Actions

Copy link

Updated by Victor Julien over 12 years ago

Priority changed from Normal to High

Actions

Copy link

#10

Updated by Victor Julien over 12 years ago

Due date set to 06/15/2012
Target version changed from 1.3beta2 to 1.3rc1

Actions

Copy link

#11

Updated by Victor Julien over 12 years ago

Priority changed from High to Normal
Target version changed from 1.3rc1 to TBD

Actions

Copy link

#12

Updated by Victor Julien over 11 years ago

Status changed from Assigned to Closed
Assignee deleted (~~Xavier Lange~~)
Target version deleted (~~TBD~~)

Actions

Copy link

Also available in: Atom PDF

Project

General

Profile

Suricata

Custom queries

Bug #428

Segfault when processing NSA pcap with http-events.rules

Updated by Xavier Lange over 12 years ago

Updated by Xavier Lange over 12 years ago

Updated by Peter Manev over 12 years ago

Updated by Xavier Lange over 12 years ago

Updated by Xavier Lange over 12 years ago

Updated by Victor Julien over 12 years ago

Updated by Xavier Lange over 12 years ago

Updated by Victor Julien over 12 years ago

Updated by Victor Julien over 12 years ago

Updated by Victor Julien over 12 years ago

Updated by Victor Julien over 12 years ago

Updated by Victor Julien over 11 years ago