Optimization #3829
openpcap source: Counters, counters and... counters
Description
There seem to be a lot of similar counters being tracked in source-pcap. Mentioned in #2845.
We shall revisit all the counters in source-pcap.
Comment from @victorjulien:
Also, we should consider the places where we count packets. I think I count 3 here: in livedev, in pkts/bytes and then in the stats api through capture_kernel*. And then we have of course the intermediate last_stats64 that you're introducing (for good reason). Thats a lot of stats keeping. W/o looking into why we do all this, my gut feel is that we can reduce some of this.
I agree that some of them seem redundant.
Updated by Roland Fischer over 4 years ago
Current Counters¶
Are:
- pkts/bytes is the thread-local count incremented with each packet
- stats API through capture_kernel_* are the thread-local counts in tv->perf_private_ctx. Periodically updated once a second with counts from last_stats64
- livedev atomic counters for pkts and drops among other things. Seems to be process-wide counts if I got that right. Mixed update rate.
- last_stats64 provides the thread-local 64bit pcap stats. It tracks the 32bit pcap counters and detects roll-overs. Periodically updated once a second with stats from pcap API.
pkts/bytes¶
Always provides the current view for the thread.
Tracks:
- ptv->pkts
Updated for every packet. Used for thread exit stats.
However, also used if CHECKSUM_VALIDATION_AUTO enabled in ChecksumAutoModeCheck() to optimize those checks. Cannot replace with stats counters as the overhead would be too much, imho. - ptv->bytes
Updated for every packet. Only used for thread exit stats.
stats API through capture_kernel_*¶
Provides an snapshot view for the thread in general. Only current when stats are polled, after that it's outdated but that does not matter. Updated once a second during stats polling from last_stats64
Tracks:
- ptv->capture_kernel_packets
- ptv->capture_kernel_drops
- ptv->capture_kernel_ifdrops
livedev¶
Since these counts are atomic, and from the usage in the "auto checksum feature", I infer these are counters across all threads.
Tracks:
- ptv->livedev->pkts
Always provides the current view across all threads. Atomically updated for every packet. Used if CHECKSUM_VALIDATION_AUTO enabled. - ptv->livedev->drop
Provides a snapshot view across all threads. Atomically updated once per second as part of the stats polling with ptv->last_stats64.ps_drop
last_stats64¶
Provides a snapshot view for the thread in general. Only current when stats are polled, after that it's outdated but that does not matter. Updated once a second during stats polling from 32bit pcap API.
Handles the 32bit roll-over. More efficient than misusing the stats API's counts for the same thing through StatsGetLocalCounterValue() and StatsAddUI64().
Will eventually go away once libpcap has a 64bit API.
Tracks:- ptv->last_stats64.ps_recv (packets)
- ptv->last_stats64.ps_drop (dropped by libpcap)
- ptv->capture_kernel_ifdrops (dropped by interface)
Updated by Philippe Antoine over 1 year ago
- Assignee set to Community Ticket
- Target version set to TBD