Bug #2226
closedimprove error message if stream memcaps too low
Description
It seems af-packet and Napatech (reported and investigated by Philip Young) runmodes can not go beyond 117 threads.
sudo suricata -S /dev/null -vvv -i eth0 --set af-packet.0.threads=118 ... [25950] 11/10/2017 -- 11:16:21 - (runmode-af-packet.c:491) <Config> (ParseAFPConfig) -- eth0: enabling zero copy mode by using data release call [25950] 11/10/2017 -- 11:16:21 - (util-runmodes.c:288) <Info> (RunModeSetLiveCaptureWorkersForDevice) -- Going to use 118 thread(s) [26068] 11/10/2017 -- 11:16:22 - (util-pool.c:169) <Error> (PoolInit) -- [ERRCODE: SC_ERR_POOL_INIT(66)] - alloc error [26068] 11/10/2017 -- 11:16:22 - (util-pool-thread.c:119) <Error> (PoolThreadGrow) -- [ERRCODE: SC_ERR_POOL_INIT(66)] - pool grow failed
Suricata info -
This is Suricata version 4.0.0-dev (rev 11806875) Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS MAGIC SIMD support: SSE_4_2 SSE_4_1 SSE_3 Atomic intrisics: 1 2 4 8 16 byte(s) 64-bits, Little-endian architecture GCC version 6.3.0 20170516, C version 199901 compiled with _FORTIFY_SOURCE=0 L1 cache line size (CLS)=64 thread local storage method: __thread compiled with LibHTP v0.5.25, linked against LibHTP v0.5.25 Suricata Configuration: AF_PACKET support: yes PF_RING support: no NFQueue support: no NFLOG support: no IPFW support: no Netmap support: no DAG enabled: no Napatech enabled: no Unix socket enabled: yes Detection enabled: yes Libmagic support: yes libnss support: yes libnspr support: yes libjansson support: yes hiredis support: no hiredis async with libevent: no Prelude support: no PCRE jit: yes LUA support: yes, through luajit libluajit: yes libgeoip: yes Non-bundled htp: no Old barnyard2 support: no CUDA enabled: no Hyperscan support: no Libnet support: yes Rust support (experimental): no Experimental Rust parsers: no Rust strict mode: no Suricatasc install: yes Profiling enabled: no Profiling locks enabled: no Development settings: Coccinelle / spatch: yes Unit tests enabled: no Debug output enabled: no Debug validation enabled: no Generic build parameters: Installation prefix: /usr Configuration directory: /etc/suricata/ Log directory: /var/log/suricata/ --prefix /usr --sysconfdir /etc --localstatedir /var Host: x86_64-pc-linux-gnu Compiler: gcc (exec name) / gcc (real) GCC Protect enabled: no GCC march native enabled: yes GCC Profile enabled: no Position Independent Executable enabled: no CFLAGS -g -O2 -march=native PCAP_CFLAGS -I/usr/include SECCFLAGS
Updated by Victor Julien about 7 years ago
Does this go away if the stream.memcap and/or stream.reassembly.memcap is increased?
Updated by Peter Manev about 7 years ago
In the case of af-packet (v3):
When the memcaps are adjusted - stream.memcap=1GB and stream.reassembly.memcap=2GB
It helps - it spins up the threads.
In the case of Napatech (--napatech):
The stream.memcap=12GB and stream.reassembly.memcap=24GB but that did not help.
How are the stream memcaps related in the two separate cases?
Updated by Peter Manev about 7 years ago
I am reconfirming the config used in the napatech run - as i suspect the issue goes away after the memcaps are increased. I will feedback as soon as I have more info.
Updated by Peter Manev about 7 years ago
The issue goes away with Napatech (as confirmed by Philip) when increasing the stream and reassembly memcaps.
Updated by Victor Julien about 7 years ago
- Subject changed from af-packet / napatech - thread pool grow failed to improve error message if stream memcaps too low
So this problem is mostly about poor error reporting.
Updated by Andreas Herz about 7 years ago
- Assignee set to OISF Dev
- Target version set to TBD
Updated by Victor Julien about 7 years ago
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Victor Julien
- Target version changed from TBD to 70
Updated by Victor Julien about 7 years ago
- Status changed from Assigned to Closed
- Target version changed from 70 to 4.0.1