Suricata

Running Ubuntu 14.04.5 64 bit system on a Dell R630 1U (dual E5-2660 v3 @2.60Ghz processors with 128Gb of memory) with a Napatech NT20E2-PTP capture card (one port active) with Suricata 3.1.1. Upgraded our environment to use Napatech's latest driver set (10.0.4 - Huntington Beach 3) and Suricata 4.0.3. Running Suricata as in daemon mode with command: /usr/bin/suricata -c /etc/suricata/suricata.yaml --napatech --runmode workers -D.

Suricata will run for some time then will see one CPU (CPU defined as a worker) hit 100% and stay there, while one Napatech host buffer (seen by running the Napatech "profiling" command) will hit 100% and drop packets. This will continue without stopping. Then a second CPU (again a CPU that is a worker) hit 100% and another Napatech host buffer will hit 100% and drop packets. This will continue, seeing many CPUs and host buffers pegged, until I issue a "kill `pidof suricata`". Many times this will gracefully end Suricata - but will take 5-10 minutes to do so. But when Suricata ends, it does not remove the /var/run/suricata.pid file.

Attached is the stats.log from a running Suricata 4.0.3 session. The first time a packet drop was seen was at the 12:20:51 mark, and with "nt12.drop" incrementing. During this time one of the CPUs acting as a "worker" was at 100%. But these drops recovered at the 12:20:58 mark, where "nt12.drop" stays constant at 13803. The big issue triggered at the 12:27:05 mark in the file - where one worker CPU was stuck at 100% followed by packet drops in host buffer "nt3.drop". Then came a second CPU at 100% (another "worker" CPU) and packet drops in buffer "nt2.drop" at 12:27:33. Suricata was killed via "kill `pidof suricata`" just before 12:27:54, where you see all host buffers beginning to drop packets.

Also attached is the suricata.yaml configuration file as well as the output from a "suricata --dump-config" command.