Bug #2886: IMAP protocol detection is incomplete - Suricata - Open Information Security Foundation

Custom queries

8.0.0 Stories
Good First Issues
OISF community

Actions

Copy link

Bug #2886

closed

IMAP protocol detection is incomplete

Added by Allen Ma over 5 years ago. Updated 3 months ago.

Status:

Closed

Priority:

Normal

Assignee:

Mahmoud Maatuq

Target version:

8.0.0-beta1

Affected Versions:

4.1.3, 5.0beta1, 5.0rc1, 5.0.0

Effort:

Difficulty:

low

Label:

Beginner, Good First Issue, Outreachy, Protocol

Description

I tried both reading offline pcap file and capturing imap packet lively, but failed.
The same testing methods are used to test POP3 successfully, which I extend on my own version.

1.reading offline pcap file, I've attached the ones I tested.
sudo suricata -c ~/suricata/suricata.yaml -r ~/pcap/imap/imap_aliyun_outlook_short.pcapng -k none --init-errors-fatal

2.captuing imap packets lively.
sudo suricata -c ~/suricata/suricata.yaml -i eno16777736 --init-errors-fatal

/** IMAP */
    AppLayerProtoDetectRegisterProtocol(ALPROTO_IMAP, "imap");
    if (AppLayerProtoDetectConfProtoDetectionEnabled("tcp", "imap")) {
        // Is AppLayerProtoDetectPMRegisterPatternCS OK as it's case-sensitive? Will it detect upper-cased CAPABILITY?
        if (AppLayerProtoDetectPMRegisterPatternCS(IPPROTO_TCP, ALPROTO_IMAP,
                                  "1|20|capability", 12, 0, STREAM_TOSERVER) < 0)
    {
            SCLogInfo("imap proto registration failure\n");
            exit(EXIT_FAILURE);
        }
    } else {
        SCLogInfo("Protocol detection and parser disabled for %s protocol.",
                  "imap");
    }

Files

Download all files

imap-ssl.pcapng (9.91 KB) imap-ssl.pcapng		Allen Ma, 03/18/2019 04:38 AM
imap.cap (30.7 KB) imap.cap		Allen Ma, 03/18/2019 04:38 AM

Subtasks 3 (0 open — 3 closed)

Bug #6506: IMAP protocol detection is incomplete (6.0.x backport)	Rejected	Actions
Bug #6514: IMAP protocol detection is incomplete (6.0.x backport)	Rejected	Actions
Bug #6526: IMAP protocol detection is incomplete (7.0.x backport)	Rejected	Actions

Related issues 2 (2 open — 0 closed)

	Related to Suricata - Task #2757: improve protocol detection	In Review	Philippe Antoine				Actions
	Related to Suricata - Feature #3244: IMAP Support	New	Community Ticket				Actions

Issue # Delay: days Cancel

History
Notes
Property changes

Actions

Copy link

Updated by Allen Ma over 5 years ago

[coapk@localhost ~]$ sudo suricata --build-info
This is Suricata version 5.0.0-dev (rev e710b06)
Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LIBJANSSON TLS MAGIC RUST
SIMD support: SSE_4_2 SSE_4_1 SSE_3
Atomic intrisics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 4.8.5 20150623 (Red Hat 4.8.5-36), C version 199901
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.28, linked against LibHTP v0.5.28

Suricata Configuration:
  AF_PACKET support:                       yes
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  liblzma support:                         no
  hiredis support:                         no
  hiredis async with libevent:             no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes
  libluajit:                               no
  libgeoip:                                no
  Non-bundled htp:                         no
  Old barnyard2 support:                   no
  Hyperscan support:                       no
  Libnet support:                          yes
  liblz4 support:                          no

  Rust support:                            yes (default)
  Rust strict mode:                        no
  Rust debug mode:                         no
  Rust compiler:                           rustc 1.32.0
  Rust cargo:                              cargo 1.32.0

  Install suricatasc:                      yes
  Install suricata-update:                 no

  Profiling enabled:                       no
  Profiling locks enabled:                 no

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var
  --datarootdir                            /usr/share

  Host:                                    x86_64-unknown-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -march=native -I${srcdir}/../rust/gen/c-headers
  PCAP_CFLAGS
  SECCFLAGS

Actions

Copy link

Updated by Victor Julien over 5 years ago

Target version changed from 5.0beta1 to TBD

Actions

Copy link

Updated by Andreas Herz over 5 years ago

Assignee set to Community Ticket

Actions

Copy link

Updated by Victor Julien almost 5 years ago

Label Protocol added

Actions

Copy link

Updated by Philippe Antoine about 1 year ago

Subject changed from IMAP fail to register app layer protocol detection to IMAP protocol detection is incomplete
Label Good First Issue added

Problem is that IMAP is not recognized by Suricata on classic pcaps as available here
https://wiki.wireshark.org/IMAP

Actions

Copy link

Updated by Victor Julien about 1 year ago

Status changed from New to Assigned
Assignee changed from Community Ticket to Philippe Antoine
Target version changed from TBD to 8.0.0-beta1
Label Needs backport to 6.0, Needs backport to 7.0 added

Actions

Copy link

Updated by Philippe Antoine about 1 year ago

Related to Task #2757: improve protocol detection added

Actions

Copy link

Updated by Victor Julien about 1 year ago

Status changed from Assigned to New
Assignee changed from Philippe Antoine to Community Ticket

Actions

Copy link

Updated by Victor Julien about 1 year ago

Label Outreachy added

Actions

Copy link

#10

Updated by Victor Julien about 1 year ago

Good issue for outreachy contribution phase.

Actions

Copy link

#11

Updated by Bruno Franca about 1 year ago

Assignee changed from Community Ticket to Bruno Franca

Actions

Copy link

#12

Updated by Bruno Franca about 1 year ago

Status changed from New to In Progress

Actions

Copy link

#13

Updated by Bruno Franca about 1 year ago

Hello people,

I am trying to do this task and I have some doubt. Is it supposed to me finalize the implementation of IMAP protocol ?

Actions

Copy link

#14

Updated by Juliana Fajardini Reichow about 1 year ago

Bruno Franca wrote in #note-13:

Hello people,

I am trying to do this task and I have some doubt. Is it supposed to me finalize the implementation of IMAP protocol ?

Sorry for not answering this before, I thought I had - maybe I did in some other channel.
Yes, that's the idea. Do you have further questions about the process?

Actions

Copy link

#15

Updated by Bruno Franca 12 months ago

I am working on it

Actions

Copy link

#16

Updated by OISF Ticketbot 12 months ago

Subtask #6506 added

Actions

Copy link

#17

Updated by OISF Ticketbot 12 months ago

Label deleted (~~Beginner, Good First Issue, Needs backport to 6.0, Needs backport to 7.0, Outreachy, Protocol~~)

Actions

Copy link

#18

Updated by Shivani Bhardwaj 12 months ago

Label Beginner, Good First Issue, Needs backport to 6.0, Outreachy, Protocol added

Actions

Copy link

#19

Updated by OISF Ticketbot 12 months ago

Subtask #6514 added

Actions

Copy link

#20

Updated by OISF Ticketbot 12 months ago

Label deleted (~~Needs backport to 6.0~~)

Actions

Copy link

#21

Updated by Shivani Bhardwaj 12 months ago

Label Needs backport to 7.0 added

Actions

Copy link

#22

Updated by OISF Ticketbot 12 months ago

Subtask #6526 added

Actions

Copy link

#23

Updated by OISF Ticketbot 12 months ago

Label deleted (~~Needs backport to 7.0~~)

Actions

Copy link

#24

Updated by Juliana Fajardini Reichow 11 months ago

Status changed from In Progress to New
Assignee changed from Bruno Franca to Community Ticket

Hi there, according to our guidelines for stale tickets, I'm unassigning this ticket.

Thanks for your interest in our project, and feel free to reach out in case you have time and want to contribute to Suricata again, or if you're still working on this task! :) :)

Refer to:
https://forum.suricata.io/t/important-outreachy-contribution-phase-wrap-up-prs-claimed-tickets-and-more
https://docs.suricata.io/en/latest/devguide/contributing/contribution-process.html#stale-tickets-policy

Actions

Copy link

#25

Updated by Juliana Fajardini Reichow 10 months ago

Related to Feature #3244: IMAP Support added

Actions

Copy link

#26

Updated by Mahmoud Maatuq 10 months ago

Status changed from New to Assigned
Assignee changed from Community Ticket to Mahmoud Maatuq

Hi team,
I'm going to work on this ticket.
just to make sure I understood the scope of this ticket correctly, in this ticket we should complete the detection patterns for IMAP, no dissection/parsing right?
e.g for now this "1|20|capability" is the only pattern implemented to detect IMAP, and we should extend/modify this to be able to comprehensively detect flows as imap.

Actions

Copy link

#27

Updated by Juliana Fajardini Reichow 9 months ago

Mahmoud Maatuq wrote in #note-26:

Hi team,
I'm going to work on this ticket.
just to make sure I understood the scope of this ticket correctly, in this ticket we should complete the detection patterns for IMAP, no dissection/parsing right?
e.g for now this "1|20|capability" is the only pattern implemented to detect IMAP, and we should extend/modify this to be able to comprehensively detect flows as imap.

Hi, I think that can be a good start. If we manage to have Suricata recognize IMAP traffic from the source shared by Philippe on one of the comments above, that'd be good!
And add a simple Suricata-verify test to go with the contribution, too. :)

We may want more in the future, but then we'll communicate and we can agree together if you'd be available to work on that or not.

Actions

Copy link

#28

Updated by Mahmoud Maatuq 9 months ago

Hey,
Unless i missed something, what is mentioned here https://wiki.wireshark.org/IMAP doesn't say much about detection patterns for imap, also by looking at proto_reg_handoff_imap() here https://gitlab.com/wireshark/wireshark/-/blob/master/epan/dissectors/packet-imap.c?ref_type=heads, I think dissector handoff is done based on port 143, 993 for imap, imaps respectively, there is no pattern, I think this wireshark's way for most of the dissectors is using port and continue with the dissection phase and through dissection it will confirm/reject the protocol.
Here we need a different approach, anyway I'm looking now at this rfc https://datatracker.ietf.org/doc/html/rfc9051, hopefully will get some time to implement protocol patterns.

Actions

Copy link

#29

Updated by Shivani Bhardwaj 9 months ago

Hi!
I'm unaware of the details of IMAP so far but Suricata offers port based protocol registering as well. Look for `AppLayerProtoDetectPPRegister`, the implementation is followed by protocols like TFTP and others.

Actions

Copy link

#30

Updated by Mahmoud Maatuq 9 months ago

Status changed from Assigned to In Progress

Hi,
will use port to register the protocol, but won't depend on this as it's a very weak way to detect protocols, but i got the idea from what is implemented in TFTP, i can go ahead with the implementation, thank you.

Actions

Copy link

#31