Project

General

Profile

Actions
Copy link

Bug #1487

closed

Configuration parser depends on key ordering

Added by Steve McMaster over 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Jason Ish
Target version:
3.1rc1
Affected Versions:
Effort:
Difficulty:
Label:

Description

According to the YAML 1.1 spec (http://yaml.org/spec/1.1/#id861435):

The content of a mapping node is an unordered set of key: value node pairs

However, Suricata behaves differently depending on the ordering of at least one mapping node: af-packet. For example:

af-packet:
- interface: default
- cluster-id: '99'
  interface: eth2
  threads: 8
  defrag: 'true'
  cluster-type: cluster_flow

Yields the following with suricata --dump-config:

af-packet = (null)
af-packet.0 = interface
af-packet.0.interface = default
af-packet.1 = cluster-id
af-packet.1.cluster-id = 99
af-packet.1.interface = eth2
af-packet.1.threads = 8
af-packet.1.defrag = true
af-packet.1.cluster-type = cluster_flow

And the following output at startup:

15/6/2015 -- 07:03:22 - <Info> - Adding interface eth2 from config file
15/6/2015 -- 07:03:22 - <Info> - Using 4 AF_PACKET threads for interface eth2
15/6/2015 -- 07:03:22 - <Error> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - Could not get cluster-id from config
15/6/2015 -- 07:03:22 - <Error> - [ERRCODE: SC_ERR_GET_CLUSTER_TYPE_FAILED(35)] - Could not get cluster-type from config

Whereas, this config:

af-packet:
- interface: default
- interface: eth2
  cluster-id: '99'
  threads: 8
  defrag: 'true'
  cluster-type: cluster_flow

produces this output from suricata --dump-config:

af-packet = (null)
af-packet.0 = interface
af-packet.0.interface = default
af-packet.1 = interface
af-packet.1.interface = eth2
af-packet.1.cluster-id = 99
af-packet.1.threads = 8
af-packet.1.defrag = true
af-packet.1.cluster-type = cluster_flow

And this output at startup:

15/6/2015 -- 07:04:38 - <Info> - Using flow cluster mode for AF_PACKET (iface eth2)
15/6/2015 -- 07:04:38 - <Info> - Using defrag kernel functionality for AF_PACKET (iface eth2)
15/6/2015 -- 07:04:38 - <Info> - Generic Receive Offload is unset on eth2
15/6/2015 -- 07:04:38 - <Info> - Large Receive Offload is unset on eth2
15/6/2015 -- 07:04:38 - <Info> - Going to use 8 thread(s)

Version info:
root@hlbinddevids01:~# lsb_release -cs
trusty
root@hlbinddevids01:~# suricata -V
This is Suricata version 2.1beta4 RELEASE
root@hlbinddevids01:~# apt-cache policy suricata
suricata:
Installed: 2.1~beta4-0ubuntu12
Candidate: 2.1~beta4-0ubuntu12
Version table: *** 2.1~beta4-0ubuntu12 0
500 http://ppa.launchpad.net/oisf/suricata-beta/ubuntu/ trusty/main amd64 Packages
100 /var/lib/dpkg/status
2.0.8-0ubuntu8 0
500 http://ppa.launchpad.net/oisf/suricata-stable/ubuntu/ trusty/main amd64 Packages
1.4.7-1ubuntu1.1 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/universe amd64 Packages
500 http://security.ubuntu.com/ubuntu/ trusty-security/universe amd64 Packages
1.4.7-1 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
root@hlbinddevids01:~#

This prevents the use of other tools for modifying the suricata configuration file (such as Puppet) because the ordering is not guaranteed.

Please let me know if I can provide any further information.

Actions
Copy link
 #1

Updated by Victor Julien over 9 years ago

There are a number of places where we're not using the yaml structure correctly. I think this is one of them.

Actions
Copy link
 #2

Updated by Jason Ish over 9 years ago

Yes, it important to note that:

af-packet:
  - threads: auto
    interface: eth0
    cluster-id: 99
    cluster-type: cluster_flow
  - interface: eth0
    threads: auto
    cluster-id: 99
    cluster-type: cluster_flow

are basically equivalent.

This becomes a little more apparent when you convert the YAML to JSON:

{
  "af-packet": [
    {
      "threads": "auto",
      "cluster-type": "cluster_flow",
      "cluster-id": 99,
      "interface": "eth0" 
    },
    {
      "threads": "auto",
      "cluster-type": "cluster_flow",
      "cluster-id": 99,
      "interface": "eth0" 
    }
  ]
}

Now the conf API is probably part of the problem here, making it easy to lookup the first seen value in a map contained in a sequence.

Now, just brainstorming here, but a better YAML section may have been:

af-packet:
  defaults:
    threads: auto
    use-mmap: yes
  eth1:
    cluster-id: 99
  eth0:
    cluster-id: 98

or perhaps:

af-packet:
  defaults:
    threads: auto
    use-mmap: yes
  interfaces:
    eth1:
      cluster-id: 99
    eth0:
      cluster-id: 98

which clearly separates the defaults from interface values, and then would allow one to to iterate over the interfaces map.

Probably the best solution for now is to fix the AFPParse code to pluck out the correct item for the interface.

Actions
Copy link
 #3

Updated by Peter Manev over 9 years ago

What if the defaults are changed for a particular interface? Would that interfere?

Actions
Copy link
 #4

Updated by Jason Ish over 9 years ago

I don't see why. The end result should be the same, just easier, and more straight forward to parse.

Actions
Copy link
 #5

Updated by Victor Julien over 9 years ago

  • Assignee set to Jason Ish
  • Target version changed from TBD to 3.0RC1
Actions
Copy link
 #6

Updated by Victor Julien over 9 years ago

  • Status changed from New to Assigned
Actions
Copy link
 #7

Updated by Victor Julien almost 9 years ago

  • Target version changed from 3.0RC1 to 70
Actions
Copy link
 #8

Updated by Victor Julien over 8 years ago

  • Status changed from Assigned to Closed
  • Target version changed from 70 to 3.1rc1
Actions
Copy link

Also available in: Atom PDF