Project

General

Profile

Actions
Copy link

Support #4661

closed

Why calculate the MD5 value of the file at each packet

Added by yida zhang over 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Affected Versions:
Label:

Description

Why is the MD5 of the file calculated in the AppendData function? 

static int AppendData(File *file, const uint8_t *data, uint32_t data_len)
{
    if (StreamingBufferAppendNoTrack(file->sb, data, data_len) != 0) {
        SCReturnInt(-1);
    }

#ifdef HAVE_NSS
    if (file->md5_ctx) {
        HASH_Update(file->md5_ctx, data, data_len);
    }
    if (file->sha1_ctx) {
        HASH_Update(file->sha1_ctx, data, data_len);
    }
    if (file->sha256_ctx) {
        HASH_Update(file->sha256_ctx, data, data_len);
    }
#endif
    SCReturnInt(0);
}

In this way, MD5 must be calculated for each packet, and there are many thread lock conflicts.

Is it not possible to calculate the MD5 when the file is closed?

Actions
Copy link
 #1

Updated by Victor Julien almost 3 years ago

  • Tracker changed from Optimization to Support
  • Status changed from New to Closed
  • Assignee deleted (Victor Julien)

The files are processing in streaming mode, so at the end of the file we most likely won't have all the data. In recent code we switched to rust hashing which should be lockless.

Actions
Copy link
 #2

Updated by yida zhang over 2 years ago

Victor Julien wrote in #note-1:

The files are processing in streaming mode, so at the end of the file we most likely won't have all the data. In recent code we switched to rust hashing which should be lockless.

Great! Very much looking forward to it!

Actions
Copy link

Also available in: Atom PDF