Suricata

afpacket already starts processing packets on threads while others are still starting up. This time window can be significant if a sensor uses large bpfs or very large buffers or if there are many capture threads.

The synchronize start logic should prevent this, but does not work. It is broken for tpacket-v3 and disabled for tpacket-v2.

The effect can be that the hashing for load balancing is not stable during the start up phase. If a socket has 4 listening threads a certain flow may end up in queue 1, but when there are 5 threads the same flow may end up in queue 3.

I've observed this on several sensors, where the 'wrong_thread' counter is huge right after startup, but then stops increasing after that.

It can be easily simulated in the afpacket startup, by adding a sleep(tv->id) per thread which lets the threads sleep for increasingly longer times (t1 for 1s, t2 for 2s, etc). Before all threads are started packets are already being processed.