Suricata

Thinking of another option: most signatures don't have a threshold setting and most setups will not use large number of thresholds through the thresholds.conf file. What this means is that the number of threshold rules is relatively low.

We can also do a separate "threshold id" per rule, that is unique per "gid + sid". During the init stage we can set this by having a simple hash. Due to the small number of threshold enabled rules the global lookup array can be small. Each threshold entry would have it's own lock. The only contention would happen when 2 or more threads set/check a threshold for the same rule, which should be relatively rare.

Then on a rule reload, we can use the same simple hash to lookup the existing id's. Then we'd assign new id's to new rules and resize the lookup array somehow. The resize would require some form of locking though, this might bring back the big evil lock... Maybe a rwlock would be an option:

Lookup(Signature *s) {
  read_lock(global_table);
  entry = s->threshold_id;
  mutex_lock(entry);
  // check/update
  mutex_unlock(entry);
  read_unlock(global_table);
}

Update() {
  write_lock(global_table);
  // resize and init new entries
  write_unlock(global_table);
}

Thoughts?

Is this issue about this code ?
detect-engine-threshold.c :

    } else if (td->track == TRACK_RULE) {
        SCMutexLock(&de_ctx->ths_ctx.threshold_table_lock);
        ret = ThresholdHandlePacketRule(de_ctx,p,td,s,pa);
        SCMutexUnlock(&de_ctx->ths_ctx.threshold_table_lock);
    }