Bug #706
closeddetect.alert - not incrementing in daemon mode
Added by Peter Manev almost 12 years ago. Updated almost 12 years ago.
Description
On our teams test box - even though we receive a big number of alerts,
the detect.alert counter in stats.log is not incrementing.
Files
| Bug706.txt (65.7 KB) Bug706.txt | Peter Manev, 02/24/2013 01:45 PM |
Updated by Victor Julien almost 12 years ago
Basic info is missing: version (rev), runmode, etc.
Updated by Peter Manev almost 12 years ago
Please find the basic info needed below:
suricata --build-info [8174] 14/1/2013 -- 14:59:20 - (suricata.c:560) <Info> (SCPrintBuildInfo) -- This is Suricata version 1.4dev (rev 5f4c528) [8174] 14/1/2013 -- 14:59:20 - (suricata.c:633) <Info> (SCPrintBuildInfo) -- Features: PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 PF_RING AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK HAVE_HTP_TX_GET_RESPONSE_HEADERS_RAW HAVE_NSS PROFILING [8174] 14/1/2013 -- 14:59:20 - (suricata.c:647) <Info> (SCPrintBuildInfo) -- 64-bits, Little-endian architecture [8174] 14/1/2013 -- 14:59:20 - (suricata.c:649) <Info> (SCPrintBuildInfo) -- GCC version 4.6.3, C version 199901 [8174] 14/1/2013 -- 14:59:20 - (suricata.c:655) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 [8174] 14/1/2013 -- 14:59:20 - (suricata.c:658) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 [8174] 14/1/2013 -- 14:59:20 - (suricata.c:661) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 [8174] 14/1/2013 -- 14:59:20 - (suricata.c:664) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 [8174] 14/1/2013 -- 14:59:20 - (suricata.c:667) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 [8174] 14/1/2013 -- 14:59:20 - (suricata.c:671) <Info> (SCPrintBuildInfo) -- compiled with -fstack-protector [8174] 14/1/2013 -- 14:59:20 - (suricata.c:677) <Info> (SCPrintBuildInfo) -- compiled with _FORTIFY_SOURCE=2 [8174] 14/1/2013 -- 14:59:20 - (suricata.c:680) <Info> (SCPrintBuildInfo) -- compiled with libhtp 0.2.11, linked against 0.2.11
using af_packet/workers
anything else that might be helpful?
Updated by Victor Julien almost 12 years ago
Are other counters being updated normally?
Updated by Peter Manev almost 12 years ago
funny you say that...
actually no - it does not look like all of them are incremented - some are, but most are not (Ex):
capture.kernel_packets | AFPacketeth316 | 0
capture.kernel_drops | AFPacketeth316 | 0
decoder.pkts | AFPacketeth316 | 0
decoder.bytes | AFPacketeth316 | 0
decoder.ipv4 | AFPacketeth316 | 0
decoder.ipv6 | AFPacketeth316 | 0
decoder.ethernet | AFPacketeth316 | 0
decoder.raw | AFPacketeth316 | 0
decoder.sll | AFPacketeth316 | 0
decoder.tcp | AFPacketeth316 | 0
decoder.udp | AFPacketeth316 | 0
decoder.sctp | AFPacketeth316 | 0
decoder.icmpv4 | AFPacketeth316 | 0
decoder.icmpv6 | AFPacketeth316 | 0
decoder.ppp | AFPacketeth316 | 0
decoder.pppoe | AFPacketeth316 | 0
decoder.gre | AFPacketeth316 | 0
decoder.vlan | AFPacketeth316 | 0
decoder.teredo | AFPacketeth316 | 0
decoder.ipv4_in_ipv6 | AFPacketeth316 | 0
decoder.ipv6_in_ipv6 | AFPacketeth316 | 0
decoder.avg_pkt_size | AFPacketeth316 | 0
decoder.max_pkt_size | AFPacketeth316 | 0
defrag.ipv4.fragments | AFPacketeth316 | 0
defrag.ipv4.reassembled | AFPacketeth316 | 0
defrag.ipv4.timeouts | AFPacketeth316 | 0
defrag.ipv6.fragments | AFPacketeth316 | 0
defrag.ipv6.reassembled | AFPacketeth316 | 0
defrag.ipv6.timeouts | AFPacketeth316 | 0
defrag.max_frag_hits | AFPacketeth316 | 0
tcp.sessions | AFPacketeth316 | 0
tcp.ssn_memcap_drop | AFPacketeth316 | 0
tcp.pseudo | AFPacketeth316 | 0
tcp.invalid_checksum | AFPacketeth316 | 0
tcp.no_flow | AFPacketeth316 | 0
tcp.reused_ssn | AFPacketeth316 | 0
tcp.memuse | AFPacketeth316 | 0
tcp.syn | AFPacketeth316 | 0
tcp.synack | AFPacketeth316 | 0
tcp.rst | AFPacketeth316 | 0
tcp.segment_memcap_drop | AFPacketeth316 | 0
tcp.stream_depth_reached | AFPacketeth316 | 0
tcp.reassembly_memuse | AFPacketeth316 | 0
tcp.reassembly_gap | AFPacketeth316 | 0
detect.alert | AFPacketeth316 | 0
flow_mgr.closed_pruned | FlowManagerThread | 717373974
flow_mgr.new_pruned | FlowManagerThread | 1260698120
flow_mgr.est_pruned | FlowManagerThread | 216749232
flow.memuse | FlowManagerThread | 362540720
flow.spare | FlowManagerThread | 1089093
flow.emerg_mode_entered | FlowManagerThread | 0
flow.emerg_mode_over | FlowManagerThread | 0
thanks
Updated by Victor Julien almost 12 years ago
@Eric Liu could we be staying in AFPReadFromRing all the time? We need to call SCPerfSyncCountersIfSignalled(tv, 0); regularly for counters to work. It is called from ReceiveAFPLoop, but maybe we never leave AFPReadFromRing?
@Peter Pan, are the counters updated when you stop suri?
Updated by Peter Manev almost 12 years ago
@Peter, are the counters updated when you stop suri?
no, they are not.
Updated by Peter Manev almost 12 years ago
Here the issue is similar to
https://redmine.openinfosecfoundation.org/issues/714
in -D mode it does not write counters info in stats.log - but in regular mode (non daemon) it updates the stats.log no problem.
root@suricata:/var/data/regit/log/suricata# suricata --build-info [23104] 19/1/2013 -- 13:01:43 - (suricata.c:560) <Info> (SCPrintBuildInfo) -- This is Suricata version 1.4dev (rev 5f4c528) [23104] 19/1/2013 -- 13:01:43 - (suricata.c:633) <Info> (SCPrintBuildInfo) -- Features: PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 PF_RING AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK HAVE_HTP_TX_GET_RESPONSE_HEADERS_RAW HAVE_NSS PROFILING [23104] 19/1/2013 -- 13:01:43 - (suricata.c:647) <Info> (SCPrintBuildInfo) -- 64-bits, Little-endian architecture [23104] 19/1/2013 -- 13:01:43 - (suricata.c:649) <Info> (SCPrintBuildInfo) -- GCC version 4.6.3, C version 199901 [23104] 19/1/2013 -- 13:01:43 - (suricata.c:655) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 [23104] 19/1/2013 -- 13:01:43 - (suricata.c:658) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 [23104] 19/1/2013 -- 13:01:43 - (suricata.c:661) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 [23104] 19/1/2013 -- 13:01:43 - (suricata.c:664) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 [23104] 19/1/2013 -- 13:01:43 - (suricata.c:667) <Info> (SCPrintBuildInfo) -- __GCC_HAVE_SYNC_COMPARE_AND_SWAP_16 [23104] 19/1/2013 -- 13:01:43 - (suricata.c:671) <Info> (SCPrintBuildInfo) -- compiled with -fstack-protector [23104] 19/1/2013 -- 13:01:43 - (suricata.c:677) <Info> (SCPrintBuildInfo) -- compiled with _FORTIFY_SOURCE=2 [23104] 19/1/2013 -- 13:01:43 - (suricata.c:680) <Info> (SCPrintBuildInfo) -- compiled with libhtp 0.2.11, linked against 0.2.11
Updated by Victor Julien almost 12 years ago
- Subject changed from detect.alert - not incrementing to detect.alert - not incrementing in daemon mode
- Assignee set to OISF Dev
- Target version set to 1.4.1
Is 1.3.5 also affected?
Updated by Victor Julien almost 12 years ago
Peter, how did you start suricata?
I noticed the following: when testing like this: "suricata -i eth0 -c suricata.yaml -S local.rules -D" it didn't work. "detect.alert" not incrementing, fast.log empty.
When I looked at Suricata's output however, I noticed what causes it. local.rules was not found. When not in daemon mode, the CWD is where ever you start suri, in daemon mode the CWD is /. So local.rules was loaded from /local.rules, and it wasn't there.
When I started like this "suricata -c suricata.yaml -i eth0 -S /full/path/to/local.rules -D" it works. Both detect.alert and fast.log are fine in this case.
Updated by Peter Manev almost 12 years ago
suricata -c /etc/suricata.yaml --af-packet=eth3 -D
is how i started it.
Updated by Victor Julien almost 12 years ago
Hmm, then I can't reproduce the issue. AF_PACKET + workers, in daemon mode:
detect.alert | AFPacketeth01 | 0 detect.alert | AFPacketeth02 | 0 detect.alert | AFPacketeth03 | 1 detect.alert | AFPacketeth04 | 0 detect.alert | AFPacketeth05 | 1 detect.alert | AFPacketeth06 | 0 detect.alert | AFPacketeth07 | 1 detect.alert | AFPacketeth08 | 1 detect.alert | AFPacketeth09 | 1 detect.alert | AFPacketeth010 | 0 detect.alert | AFPacketeth011 | 0 detect.alert | AFPacketeth012 | 0 detect.alert | AFPacketeth013 | 0 detect.alert | AFPacketeth014 | 0 detect.alert | AFPacketeth015 | 1 detect.alert | AFPacketeth016 | 0 detect.alert | AFPacketeth017 | 0 detect.alert | AFPacketeth018 | 0 detect.alert | AFPacketeth019 | 0 detect.alert | AFPacketeth020 | 0 detect.alert | AFPacketeth021 | 0 detect.alert | AFPacketeth022 | 0 detect.alert | AFPacketeth023 | 1 detect.alert | AFPacketeth024 | 0 detect.alert | AFPacketeth025 | 1 detect.alert | AFPacketeth026 | 0 detect.alert | AFPacketeth027 | 2 detect.alert | AFPacketeth028 | 1 detect.alert | AFPacketeth029 | 0 detect.alert | AFPacketeth030 | 0 detect.alert | AFPacketeth031 | 0 detect.alert | AFPacketeth032 | 0
Can you send over your yaml?
Updated by Peter Manev almost 12 years ago
- File Bug706.txt Bug706.txt added
shared the yaml.
I also attached some more info - after running for about 8:45 min ... all counters(except 4) are 0..
which is puzzling.
Updated by Eric Leblond almost 12 years ago
- % Done changed from 0 to 90
https://github.com/inliniac/suricata/pull/298 is implementing a workaround based on Victor's analysis. This has been deployed on our test box and seems to work.
Updated by Peter Manev almost 12 years ago
The patch seems to do the job correctly.
Updated by Victor Julien almost 12 years ago
- Status changed from New to Closed
- % Done changed from 90 to 100
Merged https://github.com/inliniac/suricata/pull/298, thanks guys!
Updated by Victor Julien almost 12 years ago
- Assignee changed from OISF Dev to Eric Leblond