During the "Rule Detection Engine (MPSE)" phase, the URI and the header
will be offloaded to the worker thread
After the second pseudo packet is run through the packet-processing pipe (but perhaps before the MPSE work is done by the worker thread), the packet thread finishes processing the ACK packet (which will be trivial since this ACK packet did not contain payload). Finally, at some later point, the worker thread will finish with its MPSE work and the packet thread will finish processing this second pseudo packet.
Let's look in detail how this works.
ContextSwitcher and IpsContext
We've just seen how Snort++ packet threads offload a pseudo packet's MPSE work to worker threads. We've also seen how Snort++ interrupts the processing of an ACK packet to work on the pseudo packet
that is generated as a result of the ACK packet. In order to keep track of which packet (real or pseudo) that a packet thread is processing and which pseudo packet has been handed off to worker threads, Snort++ uses ContextSwitcher
objects. Each packet thread has an associated ContextSwitcher
and each ContextSwitcher
has 20 IpsContext
objects associated with it
This number - 20
- is apparently arbitrary.
has 3 vectors in which it keeps the IpsContext
objects - idle
, and hold
Unfortunately, a packet thread and RegexOffload both have idle and busy vectors and it's sometimes difficult to keep them straight.
are used as stacks (Last-In First-Out queues) and hold
contains "slots" in which IpsContext
objects can be placed (we'll see how the hold
vector is used in a moment).
After Snort++'s initialization, all IpsContext objects are in the idle vector.
worker Threads Example
Let's revisit our HTTP example above to see how these vectors are used. When the first ACK that acknowledges payload arrives, the ACK packet is associated with an IpsContext
object and moved to the busy
stack. As long as a packet (for example, this ACK packet) is at the back
The "back" of the stack is the top of the stack in my diagrams. I use the term "back" of the stack since that is the term that is used in the code
of the busy
stack, it is moving through the pipeline.
I described the processing of an ACK packet above
but let me repeat it here. The ACK packet acknowledging the first packet with payload triggers the Wizard
, which attempts to find the appropriate service Inspector
. In this example, that will be HttpInspect
since it finds the "GET" string. HttpInspect
's associated splitter (HttpInspectSplitter
) then breaks the data up into pseudo packets (PDU's). The first pseudo packet contains the request.
Before Snort++ sends this pseudo packet through the pipeline, it associates an IpsContext
object from the idle
vector with this pseudo packet ("pseudo packet 1" in the diagram below) and moves the object to the back of the busy vector
Since Snort++ creates the headers for the different protocols (e.g., Ethernet, IP, TCP) of a pseudo packet, Snort++ knows that the headers are acceptable and so the Decoders are not called to look at these headers.
is interested in the pseudo packets and will therefore inspect the data within the pseudo packet
. It's important to understand that the service inspectors inspect the pseudo packet within the packet thread. This is in contrast to the detection
"Detect" is a fairly ambiguous term in the Snort++ code. The method DetectionEngine::detect()
invokes the MPSE to search for the fast-patterns from all the rules and then evaluates the remaining options for the rules from the MPSE fast-pattern matches. On the other hand, the DetectionEngine class
encompasses both the service Inspector
s and the MPSE work. Here I use the term "detection" to refer to the MPSE work.
, which is initially done in a worker thread and completed back in the packet thread. The work that is done in the worker thread is the MPSE fast-pattern search using the Aho-Corasick method and the work that is done in the packet thread is the later evaluation of the remaining options for the rules from the MPSE fast-pattern matches.
That being said, pseudo packet 1 is not sent to the MPSEs for reasons explained elsewhere
and so will be removed from the busy vector
after it's done in the pipeline.
After pseudo packet 1 has gone through the pipeline, the IpsContext associated with pseudo packet 1 is bumped off the busy vector and the IpsContext at the back of the busy vector (which corresponds to the ACK packet) becomes active (i.e., continues through the pipeline). Earlier we saw that pseudo packet 1 was created when the ACK packet was active. Since there is still payload that was ACK'ed by the ACK packet, pseudo packet 2 will also be created. Pseudo packet 2 is then associated with an IpsContext and since it is at the back of the busy vector, pseudo packet 2 will become active.
Pseudo packet 2 will work its way through the pipeline just as pseudo packet 1 did
but when the "Rule Detection Engine (MPSE)" phase is reached and the payload from the pseudo packets is offloaded to a worker thread for MPSE work, the IpsContext associated with pseudo packet 2 will be moved to the hold queue. At this point, since the IpsContext associated with the ACK packet is at the back of the busy vector, the ACK packet can continue through the pipeline.
RegexRequest must be moved from the offloader's idle vector to the busy vector
As mentioned earlier, both s_switcher and offloader have idle and busy vectors and it's easy to get them confused. and must then be associated with pseudo packet 2.