Project

General

Profile

Actions

Bug #2886

closed

IMAP protocol detection is incomplete

Added by Allen Ma almost 6 years ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Target version:
Affected Versions:
Effort:
Difficulty:
low
Label:
Beginner, Good First Issue, Outreachy, Protocol

Description

I tried both reading offline pcap file and capturing imap packet lively, but failed.
The same testing methods are used to test POP3 successfully, which I extend on my own version.

1.reading offline pcap file, I've attached the ones I tested.
sudo suricata -c ~/suricata/suricata.yaml -r ~/pcap/imap/imap_aliyun_outlook_short.pcapng -k none --init-errors-fatal

2.captuing imap packets lively.
sudo suricata -c ~/suricata/suricata.yaml -i eno16777736 --init-errors-fatal

/** IMAP */
AppLayerProtoDetectRegisterProtocol(ALPROTO_IMAP, "imap");
if (AppLayerProtoDetectConfProtoDetectionEnabled("tcp", "imap")) {
// Is AppLayerProtoDetectPMRegisterPatternCS OK as it's case-sensitive? Will it detect upper-cased CAPABILITY?
if (AppLayerProtoDetectPMRegisterPatternCS(IPPROTO_TCP, ALPROTO_IMAP,
"1|20|capability", 12, 0, STREAM_TOSERVER) < 0) {
SCLogInfo("imap proto registration failure\n");
exit(EXIT_FAILURE);
}
} else {
SCLogInfo("Protocol detection and parser disabled for %s protocol.",
"imap");
}

Files

imap-ssl.pcapng (9.91 KB) imap-ssl.pcapng Allen Ma, 03/18/2019 04:38 AM
imap.cap (30.7 KB) imap.cap Allen Ma, 03/18/2019 04:38 AM

Subtasks 3 (0 open3 closed)

Bug #6506: IMAP protocol detection is incomplete (6.0.x backport)RejectedActions
Bug #6514: IMAP protocol detection is incomplete (6.0.x backport)RejectedActions
Bug #6526: IMAP protocol detection is incomplete (7.0.x backport)RejectedActions

Related issues 2 (2 open0 closed)

Related to Suricata - Task #2757: improve protocol detectionIn ReviewPhilippe AntoineActions
Related to Suricata - Feature #3244: IMAP Support NewCommunity TicketActions
Actions #1

Updated by Allen Ma almost 6 years ago

[coapk@localhost ~]$ sudo suricata --build-info
This is Suricata version 5.0.0-dev (rev e710b06)
Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LIBJANSSON TLS MAGIC RUST
SIMD support: SSE_4_2 SSE_4_1 SSE_3
Atomic intrisics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 4.8.5 20150623 (Red Hat 4.8.5-36), C version 199901
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.28, linked against LibHTP v0.5.28

Suricata Configuration:
  AF_PACKET support:                       yes
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  liblzma support:                         no
  hiredis support:                         no
  hiredis async with libevent:             no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes
  libluajit:                               no
  libgeoip:                                no
  Non-bundled htp:                         no
  Old barnyard2 support:                   no
  Hyperscan support:                       no
  Libnet support:                          yes
  liblz4 support:                          no

  Rust support:                            yes (default)
  Rust strict mode:                        no
  Rust debug mode:                         no
  Rust compiler:                           rustc 1.32.0
  Rust cargo:                              cargo 1.32.0

  Install suricatasc:                      yes
  Install suricata-update:                 no

  Profiling enabled:                       no
  Profiling locks enabled:                 no

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var
  --datarootdir                            /usr/share

  Host:                                    x86_64-unknown-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -march=native -I${srcdir}/../rust/gen/c-headers
  PCAP_CFLAGS
  SECCFLAGS
Actions #2

Updated by Victor Julien over 5 years ago

  • Target version changed from 5.0beta1 to TBD
Actions #3

Updated by Andreas Herz over 5 years ago

  • Assignee set to Community Ticket
Actions #4

Updated by Victor Julien almost 5 years ago

  • Label Protocol added
Actions #5

Updated by Philippe Antoine over 1 year ago

  • Subject changed from IMAP fail to register app layer protocol detection to IMAP protocol detection is incomplete
  • Label Good First Issue added

Problem is that IMAP is not recognized by Suricata on classic pcaps as available here
https://wiki.wireshark.org/IMAP

Actions #6

Updated by Victor Julien over 1 year ago

  • Status changed from New to Assigned
  • Assignee changed from Community Ticket to Philippe Antoine
  • Target version changed from TBD to 8.0.0-beta1
  • Label Needs backport to 6.0, Needs backport to 7.0 added
Actions #7

Updated by Philippe Antoine over 1 year ago

  • Related to Task #2757: improve protocol detection added
Actions #8

Updated by Victor Julien over 1 year ago

  • Status changed from Assigned to New
  • Assignee changed from Philippe Antoine to Community Ticket
Actions #9

Updated by Victor Julien over 1 year ago

  • Label Outreachy added
Actions #10

Updated by Victor Julien over 1 year ago

Good issue for outreachy contribution phase.

Actions #11

Updated by Bruno Franca about 1 year ago

  • Assignee changed from Community Ticket to Bruno Franca
Actions #12

Updated by Bruno Franca about 1 year ago

  • Status changed from New to In Progress
Actions #13

Updated by Bruno Franca about 1 year ago

Hello people,

I am trying to do this task and I have some doubt. Is it supposed to me finalize the implementation of IMAP protocol ?

Actions #14

Updated by Juliana Fajardini Reichow about 1 year ago

Bruno Franca wrote in #note-13:

Hello people,

I am trying to do this task and I have some doubt. Is it supposed to me finalize the implementation of IMAP protocol ?

Sorry for not answering this before, I thought I had - maybe I did in some other channel.
Yes, that's the idea. Do you have further questions about the process?

Actions #15

Updated by Bruno Franca about 1 year ago

I am working on it

Actions #16

Updated by OISF Ticketbot about 1 year ago

  • Subtask #6506 added
Actions #17

Updated by OISF Ticketbot about 1 year ago

  • Label deleted (Beginner, Good First Issue, Needs backport to 6.0, Needs backport to 7.0, Outreachy, Protocol)
Actions #18

Updated by Shivani Bhardwaj about 1 year ago

  • Label Beginner, Good First Issue, Needs backport to 6.0, Outreachy, Protocol added
Actions #19

Updated by OISF Ticketbot about 1 year ago

  • Subtask #6514 added
Actions #20

Updated by OISF Ticketbot about 1 year ago

  • Label deleted (Needs backport to 6.0)
Actions #21

Updated by Shivani Bhardwaj about 1 year ago

  • Label Needs backport to 7.0 added
Actions #22

Updated by OISF Ticketbot about 1 year ago

  • Subtask #6526 added
Actions #23

Updated by OISF Ticketbot about 1 year ago

  • Label deleted (Needs backport to 7.0)
Actions #24

Updated by Juliana Fajardini Reichow about 1 year ago

  • Status changed from In Progress to New
  • Assignee changed from Bruno Franca to Community Ticket

Hi there, according to our guidelines for stale tickets, I'm unassigning this ticket.

Thanks for your interest in our project, and feel free to reach out in case you have time and want to contribute to Suricata again, or if you're still working on this task! :) :)

Refer to:
https://forum.suricata.io/t/important-outreachy-contribution-phase-wrap-up-prs-claimed-tickets-and-more
https://docs.suricata.io/en/latest/devguide/contributing/contribution-process.html#stale-tickets-policy

Actions #25

Updated by Juliana Fajardini Reichow 11 months ago

Actions #26

Updated by Mahmoud Maatuq 11 months ago

  • Status changed from New to Assigned
  • Assignee changed from Community Ticket to Mahmoud Maatuq

Hi team,
I'm going to work on this ticket.
just to make sure I understood the scope of this ticket correctly, in this ticket we should complete the detection patterns for IMAP, no dissection/parsing right?
e.g for now this "1|20|capability" is the only pattern implemented to detect IMAP, and we should extend/modify this to be able to comprehensively detect flows as imap.

Actions #27

Updated by Juliana Fajardini Reichow 11 months ago

Mahmoud Maatuq wrote in #note-26:

Hi team,
I'm going to work on this ticket.
just to make sure I understood the scope of this ticket correctly, in this ticket we should complete the detection patterns for IMAP, no dissection/parsing right?
e.g for now this "1|20|capability" is the only pattern implemented to detect IMAP, and we should extend/modify this to be able to comprehensively detect flows as imap.

Hi, I think that can be a good start. If we manage to have Suricata recognize IMAP traffic from the source shared by Philippe on one of the comments above, that'd be good!
And add a simple Suricata-verify test to go with the contribution, too. :)

We may want more in the future, but then we'll communicate and we can agree together if you'd be available to work on that or not.

Actions #28

Updated by Mahmoud Maatuq 11 months ago

Hey,
Unless i missed something, what is mentioned here https://wiki.wireshark.org/IMAP doesn't say much about detection patterns for imap, also by looking at proto_reg_handoff_imap() here https://gitlab.com/wireshark/wireshark/-/blob/master/epan/dissectors/packet-imap.c?ref_type=heads, I think dissector handoff is done based on port 143, 993 for imap, imaps respectively, there is no pattern, I think this wireshark's way for most of the dissectors is using port and continue with the dissection phase and through dissection it will confirm/reject the protocol.
Here we need a different approach, anyway I'm looking now at this rfc https://datatracker.ietf.org/doc/html/rfc9051, hopefully will get some time to implement protocol patterns.

Actions #29

Updated by Shivani Bhardwaj 11 months ago

Hi!
I'm unaware of the details of IMAP so far but Suricata offers port based protocol registering as well. Look for `AppLayerProtoDetectPPRegister`, the implementation is followed by protocols like TFTP and others.

Actions #30

Updated by Mahmoud Maatuq 11 months ago

  • Status changed from Assigned to In Progress

Hi,
will use port to register the protocol, but won't depend on this as it's a very weak way to detect protocols, but i got the idea from what is implemented in TFTP, i can go ahead with the implementation, thank you.

Actions #31

Updated by Philippe Antoine 9 months ago

  • Status changed from In Progress to In Review
Actions #33

Updated by Philippe Antoine 5 months ago

  • Status changed from In Review to Resolved

https://github.com/OISF/suricata/pull/11219

I do not think we want to backport this, do we ?

Actions #34

Updated by Victor Julien 5 months ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF