Implemented AI based "saccades" engine to improve scanning efficiency. The Saccades engine borrows from vision systems research and allows SNF to learn important message structures and adapt it's scanning technique in real-time. Using this engine SNF can often avoid scanning large portions of each message without missing important content resulting in potentially 10 fold improvements in scanner efficiency.
Fixed a bug in scanMessageFile where the XHDRInjectOn flag was being interpreted before it had been set by the configuration This caused large messages to be rewritten shorter than they needed to be when header injection was turned on because the flag would appear to be off and the MessageFileSize would be recalculated downward. Later, after the flag was set, the headers would be injected into the shortened file.
Added a function to snf_RulebaseHandler that safely peeks at the current configuration to facilitate the above bug fix.
Bumbed the minor revision number for the engine.
git-svn-id: https://svn.microneil.com/svn/SNFMulti/trunk@44 dc71a809-1921-45c4-985c-09c81d0142d9
Changed boolean to bool at 986 and 987 - bool is the correct type specifier for C++ and this was causing errors on one of the labrat compilers. No idea why it worked perfectly on mingw!
Reworked rule voting code to clarify and bring the code closer to current standards. This was part of researching a potential problem involving the voting of panicked rules and this rebuild has verified that there is no problem --- panicked rules are indeed inert as intended.
Fixed bug where only one of multiple source header directives with the same source header definition would be recognized. The fix was to improve the comparator function for HeaderFinderPattern so that it included context. Prior to that only one source header definition would be recognized because the ordinal is always 0 for source headers. As a result, only the first context could be registered.
Refactored HeaderFinder::CheckContent() to improve the coding style and make use of a switch().
Corrected a bug in HeaderFinder::CheckContent that would allow a header forced source IP to be set multiple times. Now it will only be set when the first event where all of the conditions are satisfied. Bug was that the test that the source had already been set was looking at the source IP which is actually never set until the actual message scan begins (after header directives have already been evaluated).
Changed SNFMulti revision to 16.
git-svn-id: https://svn.microneil.com/svn/SNFMulti/trunk@34 dc71a809-1921-45c4-985c-09c81d0142d9
Fixed bug in FilterChainBase64 where an extra character would be returned if the module ran out of data in SCANNING mode. Now if this occurs a flag is set and "No More Data" is thrown instead of returning the extra byte.
Adjusted DEFUNKER preamble in code to indicate spaces in front and back since newlines would be converted to spaces by the outer shell of FilterChainDefunker.
Adjusted engine update number to .15 - now at Version 3.0.15
git-svn-id: https://svn.microneil.com/svn/SNFMulti/trunk@33 dc71a809-1921-45c4-985c-09c81d0142d9
Changed engine update number to .14 (now Version 3.0.14).
Modified FilterChainDefunker to avoid recursion when eating (X)HTML tags. Now there is a new mechanism that allows deeper states to return nothing (no conversion) by setting a boolean flag. The result is that the root state will skip that result and try again with the next available byte in a simple loop (no recursion). This eliminates a vulnerability where a message constructed with deeply embeded / chained (X)HTML tags could cause the system to run out of stack space under some conditions.
git-svn-id: https://svn.microneil.com/svn/SNFMulti/trunk@32 dc71a809-1921-45c4-985c-09c81d0142d9
Fixed valgrind complaint about using uninitialized data when generating a OneTimePad. The algorithm uses unsigned char x as a register. Each next random byte in the pad is based on encrypting the last-- so a first byte is needed. Original code used an uninitialized x on purpose in order to get a supposedly unpredictable byte out of RAM. To make valgrind happy (and truly to make the algorithm better) x is now initialized by encrypting a 0 with the pad generator. This value is not directly exposed in the generated pad. The first byte in the one time pad is not the result of encrypting 0 but instead the result of encrypting the result of that operation. So, the first byte is based on the state of the pad generator just prior to making the OneTimePad. This is probably harder to predict than the state of the stack (where x would have come from) anyway.
Fixed persistent state data bug. Persistent state data is now saved once per second after updating status logs.
Improved snfLOGmgr d'tor so that final persistent state data is only saved (attempted) if snfLOGmgr has been Configured.
git-svn-id: https://svn.microneil.com/svn/SNFMulti/trunk@22 dc71a809-1921-45c4-985c-09c81d0142d9
