Feature #1201: file-store metadata in JSON format - Suricata - Open Information Security Foundation

Actions

Feature #1201

closed

Feature #2303: file-store enhancements (aka file-store v2): deduplication; hash-based naming; json metadata and cleanup tooling

file-store metadata in JSON format

Added by Peter Manev over 10 years ago. Updated almost 7 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Jason Ish

Target version:

4.1beta1

Effort:

Difficulty:

Label:

Description

Currently we write metadata for filestore like so:

root@LTS-64-1:~# cat  /var/log/suricata/files/file.2.meta 
TIME:              06/08/2014-14:15:08.392536
SRC IP:            31.186.225.23
DST IP:            10.0.2.15
PROTO:             6
SRC PORT:          80
DST PORT:          53064
HTTP URI:          /a/11016/26510/105352-2.js?&cb=0.15413070828462816&tk_st=1&rf=http://edition.cnn.com/&rp_s=c&tg_i.site=cnn_international&tg_i.rollup=homepage&tg_i.pagetype=main&p_pos=btf&p_screen_res=1680x945
HTTP HOST:         optimized-by.rubiconproject.com
HTTP REFERER:      http://ads.cnn.com/html.ng/site=cnn_international&cnn_intl_pagetype=main&cnn_intl_position=728x90_bot&cnn_intl_rollup=homepage&page.allowcompete=no&params.styles=fs&Params.User.UserID=53944fdb05ba670a3c6b805990008512&transactionID=14022297068343779055472671&tile=895079222045&domId=6c5b4c103152e6e3&kxid=ojke0w8tp&kxseg=
HTTP USER AGENT:   Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:29.0) Gecko/20100101 Firefox/29.0
FILENAME:          /a/11016/26510/105352-2.js
MAGIC:             HTML document, ASCII text, with very long lines
STATE:             CLOSED
MD5:               2a5d49f36faaf44d1e115f01bee3f499
SIZE:              2175
root@LTS-64-1:~#

It would be beneficial if we can do JSON format logging as well for the meta files.

Actions

Copy link

Updated by Victor Julien over 10 years ago

Assignee set to Anonymous
Target version set to TBD

Actions

Copy link

Updated by Andreas Moe over 10 years ago

Was just about to write an issue about this problem. Seeing that the general direction (with the eve-format) is trending towards JSON this would be a good thing to move to the JSON format aswell.

Actions

Copy link

Updated by Andreas Moe about 10 years ago

Was looking through the source of suricata to see if this ticket could be solved quickly. Then i say the function "LogFileWriteJsonRecord" in log-file.c. Isnt this an implementation for this feature ticket?

Actions

Copy link

Updated by Victor Julien about 10 years ago

LogFileWriteJsonRecord() isn't being used for creating/writing an 'meta' file. It's used to generate files-json.log lines. It could be used to generate the meta file with relatively little effort I think.

Actions

Copy link