Bug #23
closedSegv occurs occasionally inside of DetectHttpCookieMatch
Description
I was occasionally get segmentation faults inside of DetectHttpCookieMatch. I created a perl script to run the engine in a loop until an irregular exit value was detect. Here we ran 69 times without an issue and then on the 70th run we had a segmentation fault. I have attached the script the rules file and the pcap. Maybe a threading issue as it only happens occasionally.
we have run with success 68 times
/home/coz/downloads/dc17ctf-httpcookie-segv.pcap
running ulimit c unlimited; src/suricata -c suricata117.yaml -r /home/coz/downloads/dc17ctf-httpcookie-segv.pcap -l ./ -s /home/coz/downloads/current-all-blah.rules >connp->conn->transactions, 0);
exit value 0
we have run with success 69 times
/home/coz/downloads/dc17ctf-httpcookie-segv.pcap
running ulimit -c unlimited; src/suricata -c suricata117.yaml -r /home/coz/downloads/dc17ctf-httpcookie-segv.pcap -l ./ -s /home/coz/downloads/current-all-blah.rules
exit value 139
core dump found core processesing
warning: Can't read pathname for load map: Input/output error.
core dump
GNU gdb (GDB) 7.0-ubuntu
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Reading symbols from /home/coz/downloads/oisfnew/src/suricata...
done.
[New Thread 19673]
[New Thread 19668]
[New Thread 19666]
[New Thread 19676]
[New Thread 19656]
[New Thread 19667]
[New Thread 19672]
[New Thread 19674]
[New Thread 19670]
[New Thread 19675]
[New Thread 19671]
Reading symbols from /usr/lib/libhtp-0.1.so.1...
done.
Loaded symbols for /usr/lib/libhtp-0.1.so.1
Reading symbols from /usr/lib/libpcap.so.0.8...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libpcap.so.0.8
Reading symbols from /usr/local/lib/libpfring.so...
done.
Loaded symbols for /usr/local/lib/libpfring.so
Reading symbols from /usr/lib/libnet.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libnet.so.1
Reading symbols from /lib/libpthread.so.0...
Reading symbols from /usr/lib/debug/lib/libpthread-2.10.1.so...
done.
(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/libyaml-0.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libyaml-0.so.1
Reading symbols from /lib/libpcre.so.3...
(no debugging symbols found)...done.
Loaded symbols for /lib/libpcre.so.3
Reading symbols from /lib/libc.so.6...
Reading symbols from /usr/lib/debug/lib/libc-2.10.1.so...
done.
(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/lib/ld-2.10.1.so...
done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/libgcc_s.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Core was generated by `src/suricata -c suricata117.yaml -r /home/coz/downloads/dc17ctf-httpcookie-segv'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000447475 in DetectHttpCookieMatch (t=0x7f97240128f0, det_ctx=0x7f9724012ba0, f=0x2806ae0, flags=4 '', state=0x29e8a470, s=0x4019590, m=0x4019e70) at detect-http-cookie.c:80
80 htp_tx_t *tx = list_get(htp_state
#0 0x0000000000447475 in DetectHttpCookieMatch (t=0x7f97240128f0, det_ctx=0x7f9724012ba0, f=0x2806ae0, flags=4 '', state=0x29e8a470, s=0x4019590, m=0x4019e70) at detect-http-cookie.c:80
co = 0x4019d90
htp_state = 0x29e8a470
ret = 0
tx = 0x7f972a460f00
h = 0x2a298e0
#1 0x000000000041991e in SigMatchSignaturesAppLayer (th_v=0x7f97240128f0, de_ctx=0x2a298e0, det_ctx=0x7f9724012ba0, sgh=0x4651130, p=0x23f6cb0) at detect.c:527
match = 1
fmatch = 0
s = 0x4019590
sm = 0x4019e70
idx = 8731
sig = 11913
flags = 4 ''
alstate = 0x29e8a470
#2 0x000000000041a2b3 in SigMatchSignatures (th_v=0x7f97240128f0, de_ctx=0x2a298e0, det_ctx=0x7f9724012ba0, p=0x23f6cb0) at detect.c:786
match = 0
fmatch = 1
s = 0x40ffcb0
sm = 0x0
idx = 9413
sig = 12613
#3 0x000000000041a35a in Detect (tv=0x7f97240128f0, p=0x23f6cb0, data=0x7f9724012ba0, pq=0x7f97240129f0) at detect.c:823
det_ctx = 0x7f9724012ba0
de_ctx = 0x2a298e0
r = 32663
#4 0x000000000046842b in TmThreadsSlot1 (td=0x7f97240128f0) at tm-threads.c:325
tv = 0x7f97240128f0
s = 0x7f97240129c0
p = 0x23f6cb0
run = 1 ''
r = TM_ECODE_OK
#5 0x00007f972c942a04 in start_thread (arg=<value optimized out>) at pthread_create.c:300
__res = <value optimized out>
pd = 0x7f972a461910
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {
140287226026256,
-5743550190939853706,
140736003484000,
0,
0,
3,
5720867868372315254,
5720863042702617718},
mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
0x0}, data = {
prev = 0x0, cleanup = 0x0,
canceltype = 0}}}
not_first_call = <value optimized out>
robust = <value optimized out>
#6 0x00007f972c25d7bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
No locals.
#7 0x0000000000000000 in ?? ()
Files
Updated by Gurvinder Singh almost 15 years ago
- File 0002-fixed-23-bug.patch 0002-fixed-23-bug.patch added
- File bug23-htp.patch bug23-htp.patch added
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Gurvinder Singh
The issue seems to me too as related to the threading. The error seems to affect other parts too such as detect-http-method, brian please update the method task too. To be on the safe side, I have added a check in HTP library where we are facing the segv (patch attached).
I have run the engine after modification for 100 times and no segv. I hope you have the patch for bug 21 is already applied to the code.
Updated by Will Metcalf almost 15 years ago
I did already apply the patch from bug 21 already. I will try adding this patch and update the ticket if needed.
Regards,
Will
Updated by Will Metcalf almost 15 years ago
- File anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08 anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08 added
- File wirefuzz.pl wirefuzz.pl added
I'm still seeing this issue I'm going to attach a larger pcap as it appears to take less time to segv with this pcap.
I have applied the patches from this bug along with the patches from bug 21, to both the engine and r63 of htp.
coz@coz-desktop:~/downloads/oisfnew$ ./wirefuzz.pl f="/home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08" -r=/home/coz/downloads/current-all-blah.rules -c=suricata117.yaml>connp->conn == NULL) {
Name "main::wday" used only once: possible typo at ./wirefuzz.pl line 84.
Name "main::isdst" used only once: possible typo at ./wirefuzz.pl line 84.
Name "main::yday" used only once: possible typo at ./wirefuzz.pl line 84.
looping forever or until we have an error
rules file /home/coz/downloads/current-all-blah.rules
rules file /home/coz/downloads/current-all-blah.rules
not fuzzing pcap(s)
/home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08
running ulimit -c unlimited; src/suricata -c suricata117.yaml -r /home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08 -l ./ -s /home/coz/downloads/current-all-blah.rules
exit value 0
we have run with success 1 times
/home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08
running ulimit -c unlimited; src/suricata -c suricata117.yaml -r /home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08 -l ./ -s /home/coz/downloads/current-all-blah.rules
exit value 0
we have run with success 2 times
/home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08
running ulimit -c unlimited; src/suricata -c suricata117.yaml -r /home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08 -l ./ -s /home/coz/downloads/current-all-blah.rules
exit value 0
we have run with success 3 times
/home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08
running ulimit -c unlimited; src/suricata -c suricata117.yaml -r /home/coz/downloads/pcaps/evil-fingers/anon_sid_2002773_2002955_2006394_2002707_2003408_2002957.pcap-fuzz-2009-12-27-07-43-08 -l ./ -s /home/coz/downloads/current-all-blah.rules
exit value 139
core dump found core processesing
warning: Can't read pathname for load map: Input/output error.
core dump
GNU gdb (GDB) 7.0-ubuntu
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Reading symbols from /home/coz/downloads/oisfnew/src/suricata...
done.
[New Thread 26876]
[New Thread 26874]
[New Thread 26863]
[New Thread 26883]
[New Thread 26878]
[New Thread 26879]
[New Thread 26875]
[New Thread 26884]
[New Thread 26880]
[New Thread 26881]
[New Thread 26882]
[New Thread 26873]
Reading symbols from /usr/lib/libhtp-0.1.so.1...
done.
Loaded symbols for /usr/lib/libhtp-0.1.so.1
Reading symbols from /usr/lib/libpcap.so.0.8...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libpcap.so.0.8
Reading symbols from /usr/local/lib/libpfring.so...
done.
Loaded symbols for /usr/local/lib/libpfring.so
Reading symbols from /usr/lib/libnet.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libnet.so.1
Reading symbols from /lib/libpthread.so.0...
Reading symbols from /usr/lib/debug/lib/libpthread-2.10.1.so...
done.
(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/libyaml-0.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libyaml-0.so.1
Reading symbols from /lib/libpcre.so.3...
(no debugging symbols found)...done.
Loaded symbols for /lib/libpcre.so.3
Reading symbols from /lib/libc.so.6...
Reading symbols from /usr/lib/debug/lib/libc-2.10.1.so...
done.
(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/lib/ld-2.10.1.so...
done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Core was generated by `src/suricata -c suricata117.yaml -r /home/coz/downloads/pcaps/evil-fingers/anon'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000485562 in DetectHttpCookieMatch (t=0x3d9d760, det_ctx=0x7f24fc000c40, f=0x18a8bc0, flags=4 '', state=0x1524a750, s=0x30b4590, m=0x30b4e70) at detect-http-cookie.c:78
78 if (htp_state
#0 0x0000000000485562 in DetectHttpCookieMatch (t=0x3d9d760, det_ctx=0x7f24fc000c40, f=0x18a8bc0, flags=4 '', state=0x1524a750, s=0x30b4590, m=0x30b4e70) at detect-http-cookie.c:78
co = 0x30b4d90
htp_state = 0x1524a750
ret = 0
tx = 0x7f2508b1ef00
h = 0x1ac48e0
#1 0x00000000004228cb in SigMatchSignaturesAppLayer (th_v=0x3d9d760, de_ctx=0x1ac48e0, det_ctx=0x7f24fc000c40, sgh=0x6892df0, p=0x1806cb0) at detect.c:527
match = 1
fmatch = 0
s = 0x30b4590
sm = 0x30b4e70
idx = 8984
sig = 11913
flags = 4 ''
alstate = 0x1524a750
#2 0x0000000000423260 in SigMatchSignatures (th_v=0x3d9d760, de_ctx=0x1ac48e0, det_ctx=0x7f24fc000c40, p=0x1806cb0) at detect.c:786
match = 0
fmatch = 1
s = 0x319acb0
sm = 0x0
idx = 9672
sig = 12613
#3 0x0000000000423307 in Detect (tv=0x3d9d760, p=0x1806cb0, data=0x7f24fc000c40, pq=0x3d9d860) at detect.c:823
det_ctx = 0x7f24fc000c40
de_ctx = 0x1ac48e0
r = 0
#4 0x00000000004bbdca in TmThreadsSlot1 (td=0x3d9d760) at tm-threads.c:325
tv = 0x3d9d760
s = 0x3d9d830
p = 0x1806cb0
run = 1 ''
r = TM_ECODE_OK
#5 0x00007f250b801a04 in start_thread (arg=<value optimized out>) at pthread_create.c:300
__res = <value optimized out>
pd = 0x7f2508b1f910
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {
139797036398864,
-8962179890314330272,
140736220004960,
0,
0,
3,
9067401624406343520,
9067408372617068384},
mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
0x0}, data = {
prev = 0x0, cleanup = 0x0,
canceltype = 0}}}
not_first_call = <value optimized out>
robust = <value optimized out>
#6 0x00007f250b11c7bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
No locals.
#7 0x0000000000000000 in ?? ()
Updated by Will Metcalf almost 15 years ago
I hope this helps, valgrind output makes it appear as if there is an invalid free.
30244 Thread 6:30244 Invalid read of size 8
30244 at 0x485562: DetectHttpCookieMatch (detect-http-cookie.c:78)
30244 by 0x4228CA: SigMatchSignaturesAppLayer (detect.c:527)
30244 by 0x42325F: SigMatchSignatures (detect.c:786)
30244 by 0x423306: Detect (detect.c:823)
30244 by 0x4BBDC9: TmThreadsSlot1 (tm-threads.c:325)
30244 by 0x5692A03: start_thread (pthread_create.c:300)
30244 by 0x5DD67BC: clone (clone.S:112)
30244 Address 0x4537e540 is 16 bytes inside a block of size 320 free'd
30244 at 0x4C24D68: free (vg_replace_malloc.c:325)
30244 by 0x4E80FD: HTPStateFree (app-layer-htp.c:70)
30244 by 0x4E74CF: AppLayerParserCleanupState (app-layer-parser.c:870)
30244 by 0x4C4F57: StreamTcpSessionPktFree (stream-tcp.c:152)
30244 by 0x4C944D: StreamTcpPacketStateTimeWait (stream-tcp.c:2213)
30244 by 0x4C967D: StreamTcpPacket (stream-tcp.c:2273)
30244 by 0x4C974B: StreamTcp (stream-tcp.c:2304)
30244 by 0x4BBDC9: TmThreadsSlot1 (tm-threads.c:325)
30244 by 0x5692A03: start_thread (pthread_create.c:300)
30244 by 0x5DD67BC: clone (clone.S:112)
30244
30244
30244 1 errors in context 2 of 196:
30244 Invalid read of size 8
30244 at 0x48555F: DetectHttpCookieMatch (detect-http-cookie.c:78)
30244 by 0x4228CA: SigMatchSignaturesAppLayer (detect.c:527)
30244 by 0x42325F: SigMatchSignatures (detect.c:786)
30244 by 0x423306: Detect (detect.c:823)
30244 by 0x4BBDC9: TmThreadsSlot1 (tm-threads.c:325)
30244 by 0x5692A03: start_thread (pthread_create.c:300)
30244 by 0x5DD67BC: clone (clone.S:112)
30244 Address 0x4537e4e0 is 0 bytes inside a block of size 16 free'd
30244 at 0x4C24D68: free (vg_replace_malloc.c:325)
30244 by 0x4E8109: HTPStateFree (app-layer-htp.c:72)
30244 by 0x4E74CF: AppLayerParserCleanupState (app-layer-parser.c:870)
30244 by 0x4C4F57: StreamTcpSessionPktFree (stream-tcp.c:152)
30244 by 0x4C944D: StreamTcpPacketStateTimeWait (stream-tcp.c:2213)
30244 by 0x4C967D: StreamTcpPacket (stream-tcp.c:2273)
30244 by 0x4C974B: StreamTcp (stream-tcp.c:2304)
30244 by 0x4BBDC9: TmThreadsSlot1 (tm-threads.c:325)
30244 by 0x5692A03: start_thread (pthread_create.c:300)
30244 by 0x5DD67BC: clone (clone.S:112)
30244
30244
Updated by Victor Julien almost 15 years ago
I think the next master which I'm about to push out will fix this...
Updated by Victor Julien almost 15 years ago
- Status changed from Assigned to Closed