Actions
Bug #7048
closedaf-packet: failure to start up on many threads plus high load
Affected Versions:
Effort:
Difficulty:
Label:
Description
Suricata can fail to start up with this error:
Error: threads: thread "W#01-ens6f0" failed to start in time: flags 0003 [TmThreadWaitOnThreadRunning:tm-threads.c:1840]
This happens when there are many threads (>100) and they have a large ring-size in af-packet and the network is very loaded. The start up and AFPSynchronizeStart
appear to take too much time, and the TmThreadWaitOnThreadRunning
function fails with the error above. This timer allows for 60s for the threads to start up and synchronize.
Almost all time is spent in the kernel, it seems:
96.16% [kernel.kallsyms] 2.37% libc.so.6 1.37% suricata 0.05% libhs.so.5.4.0 0.03% [vdso] 0.01% ld-linux-x86-64.so.2 0.00% libpcre2-8.so.0.10.4 0.00% libjansson.so.4.13.0 0.00% libstdc++.so.6.0.30 0.00% libyaml-0.so.2.0.6
Updated by Victor Julien 5 months ago
- Status changed from New to In Review
- Assignee changed from OISF Dev to Victor Julien
- Target version changed from TBD to 8.0.0-beta1
- Label Needs backport to 7.0 added
Reviewing the af-packet logic doesn't show me anything being wrong, just that it takes a lot of time to start the threads sequentially. So the PR https://github.com/OISF/suricata/pull/11169 increases the time based on the number of threads.
Updated by Victor Julien 5 months ago
- Status changed from In Review to Resolved
Updated by Victor Julien about 1 month ago
- Related to Optimization #7272: af-packet: improve startup time added
Actions