Suricata

Do I understand correctly the challenges ?

We want to support detection
- unidirectional (for both client and server sides)
- midstream
- port independent
- similar FTP-IRC-SMTP
- performant

What about evasion techniques ?
Such as my evil FTP client connects to a FTP server, but then first sends a EHLO request (rejected by the server) and then speaks regular FTP

Correct. It should be robust against evasions too. One way we do this (for TCP) is to check if client and server protos match. If not, an event is set.

While port independent is the goal, it might not always be achievable. I could imagine that in some ambiguous cases, like SMTP/FTP, we would ultimately fall back to ports or at least assign some weight to the ports.

• There is clearly the server and client protos to match (and not stop on first detection as is done now calling TCPProtoDetect)
• Support when multiple protocols have patterns matching (such as USER for FTP and IRC)
• Check completeness for the current protocols in terms of patterns, probes, both client and server side
• Add a configurable behavior to run all probing parsers on unrecognized trafic whatever the port (and then add lots of protocols in terms of recognition only)

There is clearly the server and client protos to match (and not stop on first detection as is done now calling TCPProtoDetect)

In facts, it seems to be done already for TCP (whatever the comment says)
But it seems not done over UDP

About completeness, here are the protocols referenced in app-layer-protos.h but not implemented :
- IMAP
- MSN
- JABBER
- IRC

Should we implement at least detection for them ?

And here are the other partially (ie TO_CLIENT) missing protocol detections (and current pull requests) :
SMTP : https://github.com/OISF/suricata/pull/3832
DNS : https://github.com/OISF/suricata/pull/4058
MODBUS : https://github.com/OISF/suricata/pull/4082
DNP3 : https://github.com/OISF/suricata/pull/4082
TFTP : https://github.com/OISF/suricata/pull/4082

For all the other protocols and directions, we have have at least one mechanism (whose relevance I did not check here) :
HTTP,TOSERVER,PM
HTTP,TOCLIENT,PM
FTP,TOSERVER,PM
FTP,TOCLIENT,PM
FTPDATA,FLOW,EXPECTATION
SMTP,TOSERVER,PM
TLS,TOSERVER,PM+PP
TLS,TOCLIENT,PM
SSH,TOSERVER,PM
SSH,TOCLIENT,PM
SMB,TOSERVER,PM+PP
SMB,TOCLIENT,PM+PP
DCERPC,TOSERVER,PM+PP+hack?
DCERPC,TOCLIENT,PM+hack?
DNS,TOSERVER,PP
MODBUS,TOSERVER,PP
ENIP,TOSERVER,PP
ENIP,TOCLIENT,PP
DNP3,TOSERVER,PP
NFS,TOSERVER,PP
NFS,TOCLIENT,PP
TFTP,TOSERVER,PP
DHCP,TOSERVER,PP
DHCP,TOCLIENT,PP
IKEV2,TOSERVER,PP
IKEV2,TOCLIENT,PP
KRB5,TOSERVER,PP
KRB5,TOCLIENT,PP
NTP,TOSERVER,PP
NTP,TOCLIENT,PP

About TCP in the case of client and server protocol mismatch, the one taken into consideration is the first one to have sent data...
Here is a problematic case :
https://github.com/OISF/suricata-verify/pull/117

I think that, in the case of a protocol mismatch between client and server, we should parse further to confirm which is the protocol...
Any ideas on this ?

To sum up a bit my point of view, we need :
- completeness : what is wished ? (ie should we add IRC, MSN and such ?)
- match client and server protocols : for UDP https://github.com/OISF/suricata/pull/4058
- match client and server protocols : do not always trust the server in case of mismatch cf https://github.com/OISF/suricata-verify/pull/117
- multiple protocol recognition : discussion in https://github.com/OISF/suricata/pull/4152

Merged https://github.com/OISF/suricata/pull/5773 : debug validation that we cannot have multiple patterns matching at once

Now that https://github.com/OISF/suricata/pull/7320 got merged, this ticket requires more discussion about what needs to be done
- Code cleaning ?
- match client and server protocols : do not always trust the server in case of mismatch (retry multiple times) ?
- completeness (detect more protocols) ?

1125 is next