Project

General

Profile

Actions

Feature #5816

open

Exception policy stats counters

Added by Jamie Lavigne almost 2 years ago. Updated 7 months ago.

Status:
Resolved
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

Exception policies support applying a default action to packets in various exceptional cases, but don't have related stats counters. I am interested in counters for each of the exception policies counting the number of times they are applied. The visibility these counters provide is important for understanding the source of these drops since they can not be reported in things like alert logs.

I think a single counter per exception policy counting the number of times it is invoked would be enough for the visibility we need. Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.


Subtasks 2 (1 open1 closed)

Feature #5890: Exception policy stats counters (6.0.x backport)RejectedActions
Feature #6509: Exception policy stats counters (7.0.x backport)In ReviewJeff LucovskyActions

Related issues 6 (4 open2 closed)

Related to Suricata - Feature #6230: stats: add drop reason countersClosedVictor JulienActions
Related to Suricata - Task #6443: Suricon 2023 brainstormAssignedVictor JulienActions
Related to Suricata - Task #6929: eve/stats: hide zero-values for counters individuallyIn ProgressJuliana Fajardini ReichowActions
Related to Suricata - Feature #6215: Exception policy log outputAssignedJuliana Fajardini ReichowActions
Related to Suricata - Optimization #7185: exceptions: use search-friendly log outputNewOISF DevActions
Has duplicate Suricata - Feature #5828: exceptions: add statsRejectedJuliana Fajardini ReichowActions
Actions #1

Updated by Jamie Lavigne almost 2 years ago

Jamie Lavigne wrote:

Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.

I suppose alternately an exception policy value like "count" would work for this.

Actions #2

Updated by Juliana Fajardini Reichow over 1 year ago

Actions #3

Updated by Juliana Fajardini Reichow over 1 year ago

  • Assignee changed from OISF Dev to Juliana Fajardini Reichow
Actions #4

Updated by Juliana Fajardini Reichow over 1 year ago

  • Target version changed from TBD to 7.0.0-rc2
Actions #5

Updated by Juliana Fajardini Reichow over 1 year ago

  • Status changed from New to In Progress

Jamie Lavigne wrote in #note-1:

Jamie Lavigne wrote:

Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.

I suppose alternately an exception policy value like "count" would work for this.

Do you mean like besides `pass-flow`, `pass-packet`, `bypass`, `reject`, `drop-flow`, `drop-packet`, and `ignore`, there could also be a `count` value?
I'm thinking that the counters could be enabled, maybe under the Exception-policy master-switch.

I was thinking that the `exception-policy` stats counter could look something like:

{
   "exception_policy":{
      "totals":{
         "drop-flow":4,
         "drop-pkt":4,
         "pass-flow":3,
         "pass-pkt":0,
         "bypass":0,
         "ignore":15
      },
      "stream":{
         "memcap":{
            "policy":"drop-flow",
            "counter":0
         },
         "midstream":{
            "policy":"pass-flow",
            "counter":3
         },
         "reassembly_memcap":{
            "policy":"ignore",
            "counter":8
         }
      },
      "app_layer_error":{
         "policy":"drop-pkt",
         "counter":4
      },
      "defrag_memcap":{
         "policy":"drop-flow",
         "counter":4
      },
      "flow_memcap":{
         "policy":"ignore",
         "counter":7
      }
   }
}

Would this be: useful? Or too verbose?

Actions #6

Updated by Jamie Lavigne over 1 year ago

It would work - I'm not sure we would need the policy values like "policy":"drop-flow" echoed in the stats output though, as we will know the configured values of those already. I assume that the _delta counters will be supported for these as well?

Actions #7

Updated by Juliana Fajardini Reichow over 1 year ago

Jamie Lavigne wrote in #note-6:

It would work - I'm not sure we would need the policy values like "policy":"drop-flow" echoed in the stats output though, as we will know the configured values of those already. I assume that the _delta counters will be supported for these as well?

Thanks for the quick feedback :)

About delta counters: At the moment, I don't see a reason why we can't have them. Thanks for the reminder!

Actions #8

Updated by OISF Ticketbot over 1 year ago

  • Subtask #5890 added
Actions #9

Updated by OISF Ticketbot over 1 year ago

  • Label deleted (Needs backport to 6.0)
Actions #10

Updated by Juliana Fajardini Reichow over 1 year ago

Current PR under review/discussion: https://github.com/OISF/suricata/pull/8735

Actions #11

Updated by Juliana Fajardini Reichow over 1 year ago

To add: if exception policy is `ignore`, don't output to log events.

Actions #12

Updated by Juliana Fajardini Reichow over 1 year ago

Actions #13

Updated by Juliana Fajardini Reichow over 1 year ago

Actions #14

Updated by Juliana Fajardini Reichow over 1 year ago

  • Target version changed from 7.0.0-rc2 to 8.0.0-beta1
Actions #15

Updated by Victor Julien over 1 year ago

  • Related to Feature #6230: stats: add drop reason counters added
Actions #16

Updated by Juliana Fajardini Reichow 12 months ago

  • Related to Task #6443: Suricon 2023 brainstorm added
Actions #17

Updated by Juliana Fajardini Reichow 12 months ago

Mentioned during brainstorming as part of a larger discussion on how to debug stuff.

Actions #18

Updated by Juliana Fajardini Reichow 12 months ago

  • Label Needs backport to 7.0 added
Actions #19

Updated by OISF Ticketbot 12 months ago

  • Subtask #6509 added
Actions #20

Updated by OISF Ticketbot 12 months ago

  • Label deleted (Needs backport to 7.0)
Actions #21

Updated by Philippe Antoine 7 months ago

Actions #22

Updated by Juliana Fajardini Reichow 7 months ago

  • Related to Task #6929: eve/stats: hide zero-values for counters individually added
Actions #23

Updated by Juliana Fajardini Reichow 7 months ago

  • Status changed from In Progress to Resolved
Actions #24

Updated by Juliana Fajardini Reichow 7 months ago

Actions #25

Updated by Philippe Antoine 5 months ago

Actions #26

Updated by Juliana Fajardini Reichow 3 months ago

Actions

Also available in: Atom PDF