Bug #1780: VLAN tags not forwarded in afpacket inline mode - Suricata - Open Information Security Foundation

Actions

Copy link

Bug #1780

closed

VLAN tags not forwarded in afpacket inline mode

Added by Alexander Gozman over 8 years ago. Updated over 7 years ago.

Status:

Closed

Priority:

High

Assignee:

Eric Leblond

Target version:

3.2.1

Affected Versions:

Effort:

Difficulty:

Label:

Description

It seems that VLAN tags are not forwarded when suricata runs in afpacket inline mode: incoming packets contain VLAN tag, but outgoing do not. The test was performed on 2.1beta4, but I suppose that the problem remains in more recent versions.
Here's the excerpt from suricata.yaml:

vlan:
  use-for-tracking: true
runmode: workers
af-packet:
- interface: eth1
  copy-iface: eth5
  cluster-id: 3
  cluster-type: cluster_flow
  use-mmap: yes
  copy-mode: ips
  threads: 4

- interface: eth5
  copy-iface: eth1
  cluster-id: 4
  cluster-type: cluster_flow
  use-mmap: yes
  copy-mode: ips
  threads: 4

All offloading features that may affect seem to be turned off. First interface:

# ethtool -k eth1
Features for eth1:
rx-checksumming: on
tx-checksumming: on
    tx-checksum-ipv4: on
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: on
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: on
scatter-gather: off
    tx-scatter-gather: off
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
    tx-tcp-segmentation: off
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]

And the second one:

# ethtool -k eth5
Features for eth5:
rx-checksumming: on
tx-checksumming: on
    tx-checksum-ipv4: on
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: on
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: on
scatter-gather: off
    tx-scatter-gather: off
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
    tx-tcp-segmentation: off
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]

Can send PCAPs privately if needed.

Related issues 2 (2 open — 0 closed)

Actions

Copy link

Updated by Victor Julien over 8 years ago

Status changed from New to Assigned
Assignee set to Eric Leblond
Target version set to 70

Actions

Copy link

Updated by Alexander Gozman over 8 years ago

The problem occurs on kernel 3.14.43. Probably the problem is that sll_protocol field is not set in AFPWritePacket and kernel just ignores VLAN info, but it's just a supposition.

Actions

Copy link

Updated by Alexander Gozman over 8 years ago

Addition: both network cards use igb drivers (i82576). Maybe it's a driver issue... We also try to understand what's going on.

Actions

Copy link

Updated by Alexander Gozman over 8 years ago

The problem can be reproduced only on real network traffic. Locally tcpreplayed packets perfectly pass through suricata without VLAN truncation. We need to go deeper...

Actions

Copy link

Updated by Alexander Gozman over 8 years ago

It seems that the problem is in how suricata works in afpacket mode. We've done some research about it and got the following result:

AFRead() ignores VLAN info - no auxilary data is processed.
AFPReadFromRing() gets VLAN info from auxilary data.
AFPWritePacket() sends plain ethernet packet without [auxilary] VLAN data. If we create a custom packet in AFPWritePacket() and put VLAN info there, everything is passed without any problems. For example (quick'n'dirty, just to test the idea):

if (p->vlan_idx == 1) {
    size_t offset = ETH_ALEN * 2;
    char buffer[1600] = {0};

    memcpy(buffer, GET_PKT_DATA(p), GET_PKT_LEN(p));
    memmove(buffer + offset + 4, buffer + offset, GET_PKT_LEN(p)-offset);

    uint16_t val = htons(0x8100);
    memcpy(buffer + offset, &val, 2);
    val = htons(p->vlan_id[0]);
    memcpy(buffer + offset + 2, &val, 2);
    sendto(socket, buffer, GET_PKT_LEN(p) + 4, 0, (struct sockaddr*) &socket_address, sizeof(struct sockaddr_ll));
} else {
    ... // Send as usual
}

Hope you find this useful.

Actions

Copy link

Updated by Victor Julien over 8 years ago

Thanks for the analysis. Does this mean that if you force the code to use AFPRead() it works correctly?

Actions

Copy link

Updated by Alexander Gozman over 8 years ago

We tried to use AFPRead(), but it didn't help. Actually, this function is called only when 'use-mmap' is set to 'no'. And it only gets data from socket, while AFPReadFromRing() also reads a auxilary data like VLAN id.

So we've concluded that there're two buggy functions:
- AFPRead: it ignores auxilary data. It's bad, but not critical - we have AFPReadFromRing.
- AFPWritePacket(): it always writes plain ethernet, no 802.1q or smth. And this is a real problem.

Actions

Copy link

Updated by Victor Julien over 8 years ago

The problem is that the kernel always strips the vlan header from the data passed to Suricata. Kernel folks suggested we forge a vlan header and add it to the out going packet. Quite ugly :(

Actions

Copy link

Updated by Eric Leblond almost 8 years ago

I've got an implementation here that seems to work https://github.com/inliniac/suricata/pull/2404

Actions

Copy link

#10

Updated by Victor Julien over 7 years ago

Status changed from Assigned to Closed
Target version changed from 70 to 3.2.1

https://github.com/inliniac/suricata/pull/2544

Actions

Copy link

#11

Updated by Victor Julien over 5 years ago

Related to Bug #2934: VLAN tags stripped when saving pcap log added

Actions

Copy link

#12

Updated by Andreas Herz over 5 years ago

Related to Bug #2478: PCAP logging does not include 802.1q header when using af-packet added

Actions

Copy link

Also available in: Atom PDF

	Related to Suricata - Bug #2934: VLAN tags stripped when saving pcap log	New	OISF Dev				Actions
	Related to Suricata - Bug #2478: PCAP logging does not include 802.1q header when using af-packet	Feedback	OISF Dev				Actions

Project

General

Profile

Suricata

Custom queries

Bug #1780

VLAN tags not forwarded in afpacket inline mode

Updated by Victor Julien over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Victor Julien over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Victor Julien over 8 years ago

Updated by Eric Leblond almost 8 years ago

Updated by Victor Julien over 7 years ago

Updated by Victor Julien over 5 years ago

Updated by Andreas Herz over 5 years ago