Suricata

Creating per Victor's request.

Suricata is core dumping and seg faulting when calling the socket control command "reopen-log-files" in an aggressive manner. Tested on Suricata 5.0.1.

This may be a race condition between the output handlers doing their "reopen" action after receiving a signal and the pcap processing thread still using (or reusing) those same files. From a security point of view, if a bad actor can already issue socket control commands, he can shutdown Suricata if he wants (with the "shutdown" command) and doesn't have to go about crashing it if he want to DoS it.

See attached sctest.py file (tested with Python3.7) for a script that reproduces the issue. (Adjust accordingly.) The pcap and ruleset shouldn't really matter and the config is what ships with Suricata except it has "unix-command" configured like this:

unix-command:
  enabled: yes
  filename: test.sock

Suricata is run like this:

-c /src/suricata-current/suricata.yaml -k none --runmode single --unix-socket=test.sock

Some GDB output from crashes:

[Thread 0x7ffff3f32700 (LWP 1696) exited]
free(): invalid pointer

Thread 2 "US" received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff4733700 (LWP 1639)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51    ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff5adf801 in __GI_abort () at abort.c:79
#2  0x00007ffff5b28897 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff5c55b9a "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007ffff5b2f90a in malloc_printerr (str=str@entry=0x7ffff5c53d88 "free(): invalid pointer") at malloc.c:5350
#4  0x00007ffff5b36e1c in _int_free (have_lock=0, p=0x7fffe51bb3c0, av=0x7fffe4000020) at malloc.c:4157
#5  __GI___libc_free (mem=0x7fffe51bb3d0) at malloc.c:3124
#6  0x00007ffff697cfe5 in json_delete () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#7  0x00007ffff697741e in ?? () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#8  0x00007ffff69774c9 in ?? () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#9  0x00007ffff697cf91 in json_delete () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#10 0x00007ffff697741e in ?? () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#11 0x00007ffff69774c9 in ?? () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#12 0x00007ffff697cf91 in json_delete () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#13 0x0000555555736e7b in json_decref (json=0x7fffdc785880) at /usr/include/jansson.h:129
#14 UnixCommandExecute (this=0x555555c987a0 <command>, client=0x7fffe4d8ce20, command=0x7ffff4730ad0 "{\"command\": \"command-list\"}") at unix-manager.c:531
#15 UnixCommandRun (client=client@entry=0x7fffe4d8ce20, this=0x555555c987a0 <command>) at unix-manager.c:622
#16 0x00005555557380f8 in UnixMain (this=0x555555c987a0 <command>) at unix-manager.c:673
#17 UnixManager (th_v=0x555556e491e0, thread_data=<optimized out>) at unix-manager.c:1122
#18 0x0000555555730c9e in TmThreadsManagement (td=0x555556e491e0) at tm-threads.c:722
#19 0x00007ffff675c6db in start_thread (arg=0x7ffff4733700) at pthread_create.c:463
#20 0x00007ffff5bc088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) continue
Continuing.
[Thread 0x7ffff7feb680 (LWP 1635) exited]

Program terminated with signal SIGABRT, Aborted.
The program no longer exists.

[Thread 0x7ffff3f32700 (LWP 1744) exited]

Thread 2 "US" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff4733700 (LWP 1702)]
__memset_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:200
200    ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: No such file or directory.
(gdb) bt
#0  __memset_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:200
#1  0x00007ffff5b393a2 in __libc_calloc (n=<optimized out>, elem_size=<optimized out>) at malloc.c:3488
#2  0x0000555555585c30 in AlertFastLogInitCtx (conf=0x555555eea290) at alert-fastlog.c:240
#3  0x0000555555704221 in RunModeInitializeOutputs () at runmodes.c:796
#4  0x000055555572c6b3 in PreRunPostPrivsDropInit (runmode=runmode@entry=2) at suricata.c:2322
#5  0x00005555556fec6c in UnixSocketPcapFilesCheck (data=<optimized out>) at runmode-unix-socket.c:575
#6  0x0000555555737f26 in UnixCommandBackgroundTasks (this=0x555555c987a0 <command>) at unix-manager.c:448
#7  UnixManager (th_v=0x555556e48f60, thread_data=<optimized out>) at unix-manager.c:1138
#8  0x0000555555730c9e in TmThreadsManagement (td=0x555556e48f60) at tm-threads.c:722
#9  0x00007ffff675c6db in start_thread (arg=0x7ffff4733700) at pthread_create.c:463
#10 0x00007ffff5bc088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) continue
Continuing.
[Thread 0x7ffff7feb680 (LWP 1701) exited]

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.

[Thread 0x7ffff3f32700 (LWP 1949) exited]

Thread 2 "US" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff4733700 (LWP 1867)]
_int_malloc (av=av@entry=0x7fffe4000020, bytes=bytes@entry=24) at malloc.c:3647
3647    malloc.c: No such file or directory.
(gdb) bt
#0  _int_malloc (av=av@entry=0x7fffe4000020, bytes=bytes@entry=24) at malloc.c:3647
#1  0x00007ffff5b390b1 in __libc_calloc (n=<optimized out>, elem_size=<optimized out>) at malloc.c:3436
#2  0x00005555556c42bd in OutputRegisterFileRotationFlag (flag=flag@entry=0x7fffe51fa2c8) at output.c:873
#3  0x000055555575f9b8 in SCConfLogOpenGeneric (conf=conf@entry=0x555555eea290, log_ctx=log_ctx@entry=0x7fffe51fa210, default_filename=default_filename@entry=0x5555559681df "fast.log", rotate=rotate@entry=1)
    at util-logopenfile.c:441
#4  0x0000555555585c1d in AlertFastLogInitCtx (conf=0x555555eea290) at alert-fastlog.c:235
#5  0x0000555555704221 in RunModeInitializeOutputs () at runmodes.c:796
#6  0x000055555572c6b3 in PreRunPostPrivsDropInit (runmode=runmode@entry=2) at suricata.c:2322
#7  0x00005555556fec6c in UnixSocketPcapFilesCheck (data=<optimized out>) at runmode-unix-socket.c:575
#8  0x0000555555737f26 in UnixCommandBackgroundTasks (this=0x555555c987a0 <command>) at unix-manager.c:448
#9  UnixManager (th_v=0x555556e491e0, thread_data=<optimized out>) at unix-manager.c:1138
#10 0x0000555555730c9e in TmThreadsManagement (td=0x555556e491e0) at tm-threads.c:722
#11 0x00007ffff675c6db in start_thread (arg=0x7ffff4733700) at pthread_create.c:463
#12 0x00007ffff5bc088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) continue
Continuing.
[Thread 0x7ffff7feb680 (LWP 1866) exited]

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.

[Thread 0x7ffff3f32700 (LWP 2025) exited]

Thread 2 "US" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff4733700 (LWP 1958)]
0x00007ffff6977367 in ?? () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
(gdb) bt
#0  0x00007ffff6977367 in ?? () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#1  0x00007ffff6977668 in ?? () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#2  0x00007ffff697d219 in json_object_set_new_nocheck () from /usr/lib/x86_64-linux-gnu/libjansson.so.4
#3  0x00005555557355b7 in UnixManagerReopenLogFiles (cmd=<optimized out>, server_msg=0x7fffe55bfca0, data=<optimized out>) at unix-manager.c:914
#4  0x0000555555736e06 in UnixCommandExecute (this=0x555555c987a0 <command>, client=0x7fffe5964af0, command=0x7ffff4730ad0 "{\"command\": \"reopen-log-files\"}") at unix-manager.c:504
#5  UnixCommandRun (client=client@entry=0x7fffe5964af0, this=0x555555c987a0 <command>) at unix-manager.c:622
#6  0x00005555557380f8 in UnixMain (this=0x555555c987a0 <command>) at unix-manager.c:673
#7  UnixManager (th_v=0x555556e491e0, thread_data=<optimized out>) at unix-manager.c:1122
#8  0x0000555555730c9e in TmThreadsManagement (td=0x555556e491e0) at tm-threads.c:722
#9  0x00007ffff675c6db in start_thread (arg=0x7ffff4733700) at pthread_create.c:463
#10 0x00007ffff5bc088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) continue
Continuing.
[Thread 0x7ffff7feb680 (LWP 1956) exited]

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.