Bug #4280: Suricata is not fully reading or loading the iprep files - Suricata - Open Information Security Foundation

Actions

Copy link

Bug #4280

closed

Suricata is not fully reading or loading the iprep files

Added by Manuel Forte almost 4 years ago. Updated over 3 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Victor Julien

Target version:

7.0.0-beta1

Affected Versions:

6.0.1

Effort:

Difficulty:

Label:

Needs backport to 5.0, Needs backport to 6.0

Description

Hi,

I have been trying to use Suricata as IPS. I decided to use L2 approaching with AFP. My goal is to use IP Reputation mechanism to block lot of IPs from different blacklists.
I managed to configure everything and I could confirm iprep works like charm with a small custom iprep list, but it looks like the same mechanism fails when the list grows large or there are many reputation lists to load. After some tests it looks like Suricata is not fully reading the iprep files, or that there is a limit to the number of lines it can read/load.

Please check more details in the following posts:
- https://github.com/StamusNetworks/SELKS/issues/289
- https://forum.suricata.io/t/suricata-and-ip-blacklist/972/19

If there is a way to easily fix this issue or if you need more details, please let me know.

Please help!

Thank you

Files

4280.tar.xz (136 KB) 4280.tar.xz

Peter Manev, 02/05/2021 07:49 AM

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by Victor Julien almost 4 years ago

Priority changed from High to Normal
Target version changed from 6.0.1 to TBD

Actions

Copy link

Updated by Manuel Forte over 3 years ago

After few tests I have narrowed the working range up to 1170 lines of IPs within the iprep list. If IP falls beyond that point, it won’t be blocked. That’s not a large number of IPs to check.

In this https://blog.inliniac.net/2012/11/21/ip-reputation-in-suricata/, somebody was testing iprep with data sets up to million entries with positive results! Having a small list with only 1170 entries vs a million entries working fine, makes me think that perhaps Suricata is not reading values properly from the Host table or Host table mechanism is not working properly.

Actions

Copy link

Updated by Peter Manev over 3 years ago

Did you try increasing the memcaps and hash size as suggested by Victor here - https://forum.suricata.io/t/suricata-and-ip-blacklist/972/19?

Actions

Copy link

Updated by Manuel Forte over 3 years ago

I did until values grew huge and Suricata got stuck on start. Please check one of my post in the same thread.
https://forum.suricata.io/t/suricata-and-ip-blacklist/972/28?u=manuelfff

Actions

Copy link

Updated by Manuel Forte over 3 years ago

I'm sorry,this is the right post
https://forum.suricata.io/t/suricata-and-ip-blacklist/972/32?u=manuelfff

Actions

Copy link

Updated by Peter Manev over 3 years ago

File 4280.tar.xz 4280.tar.xz added

I can confirm there is a problem in alerting/detecting depending on the IP position in the list.
In the example crafted with pcap attached the first IP from the iprep list always alerts and the last one does not.
Even if the memcaps are adjusted to numbers plenty to provide for the ipreplist

Actions

Copy link