Introduction
Setup notes and important instructions
Code Samples
API Reference - Functions listed in the order they should be used
Miscellaneous
This SDK contains a complete Message Sniffer engine. You MUST be familiar with that engine before you can effectively use this SDK. If you do not already have an OEM license or trial SNF rule base license then please visit our web site first and get one! The engine won't run without it.
We highly recommend that you have some experience with our SNFServer and SNFClient applications before you begin writing your own or begin integrating the SNF engine into your own software.
We're not saying it's hard to do -- quite the opposite actually, BUT SNF is a sophisticated piece of software with a lot of capabilities and options. You will have much better results if you are familiar with these before you begin ;-)
This SDK contains the following:
To make the SDK available to other users, copy the the contents of SNFMultiSDK_Windows to C:\SNF.
Before running the sample programs, copy the following files to C:\SNF (if the contents of SNFMultiSDK_Windows haven't been copied to C:\SNF):
The SNF SDK for Win* is based on a DLL. The DLLs for both the 32 bit and 64 bit versions are included here.
The DLL contains the entire SNFServer engine and provides a simple API for starting the engine, scanning messages, and retrieving the results. Since the entire SNFServer engine is included you also have the option of using the SNFClient utility once your application has started the SNF engine. You can also make calls to the engine using the XCI protocol (which is how SNFClient does it's work)
Of course the best way to use the DLL is to perform scans directly through the API. The best performance can be achieved by scanning a message in memory via the scanBuffer() function since this can be done at the full speed of the processor without waiting for IO operations.
The DLL is fully thread-safe so you can perform as many concurrent scans as you wish. Also, just in case it will make things easier for you, the DLL provides a throttling function which can limit the number of concurrent scans. (It won't unless you ask it to.)
The general form of an application using the DLL will first start the engine, then optionally set the throttle, then perform scans (perform a scan, get results, close the scan, repeat), and finally it will shutdown the engine.
Since the DLL contains the entire SNFServer engine, it can (and must) be configured in exactly the same way as SNFServer. Documentation for configuring SNFServer can be found on our web site.
New in snfmulti.dll V3.4!
The SNF engine has been updated to remove theSaccades algorithm, and to use the SNFMulti 3.2.2 and latest CodeDweller libraries.
New in snfmulti.dll V3.0!
OEM developers can now protect their licenseID and Authentication string by providing it directly to the SNF engine at run-time. When combined with an internal mechanism for downloading rule base files this makes it practical to control SNF license information entirely within the OEM's application. See startupSNFAuthenticated() for details.
This function initializes the SNF scanning engine using the configuration file provided. The configuration file identifies all of the operational parameters for the SNF engine including the location of the working directories and rule base file, SNF license information, and much more. See our web site for details on configuring the SNF engine.
This function initializes the SNF scanning engine using the configuration file and authentication information provided. When SNF is started with this function the identity.xml file can be omitted and the identity= attribute of the <node/> element in snf_engine.xml can be omitted. This allows OEM developers to protect their authentication string by retrieving it from an encrypted source at run-time and providing it directly to the SNF engine.
In all other ways the SNF engine is configured in the same as when using startupSNF() (see above)
Note that if you intend to use this mechanism to protect your SNF license information you will also need to address the mechanism you use to download and verify rule base files. Either build a mechanism to download and authenticate your rule base file without exposing your authentication string or you might modify the existing getRulebase script to remove the snf2check operation and the associated authentication string. The SNF engine will check all rule base files before they are loaded for scanning and will refuse to load a rule base file that does not authenticate properly.
This function establishes a limit on the number of concurrent scans that can run. Any additional threads will block until at least one of the active scans is completed.
The default value for the throttle setting is zero. When the throttle is set to zero then no limits are placed on the number of concurrent scans. In this mode the application must limit the number of concurrent scans.
This function tests an IP against the GBUdb. This function returns very quickly and can be called as often as required without any follow-up actions as long as the SNF Engine is active (between startupSNF() and shutdownSNF()). This function is thread-safe and does not interfere with other scanning functions.
GBUdb gathers it's statistics based on the message scans that are performed. Information about those scans is also shared with other SNF nodes approximately once every minute. No external queries are performed to gather GBUdb data. As a result GBUdb can only provide an IP reputation for IPs that sourced messages scanned by this SNF node.
Put another way - GBUdb does not work like a conventional real-time black list. Message scans must be performed in order for GBUdb to provide IP reputation information.
For more information on how GBUdb works visit the GBUdb Technology section of our web site.
This function returns a number representing the overall reputation of the IP based on local GBUdb statistics. This number (Reputation Figure) can be easily manipulated to provide additional weight values in systems that combine multiple tests using a weight based scoring system. The Reputation Figure is calculated by combining the Probability figure and the Confidence figure using the formula:
R = sign(P) * sqrt(abs(P * C))
This function returns very quickly and can be called as often as required without any follow-up actions as long as the SNF Engine is active (between startupSNF...() and shutdownSNF()). This function is thread-safe and does not interfere with other scanning functions.
There are a number of ways to convert a Reputation figure to a weight value. The simplest is to simply multiply the Reputation figure by the maximum weight you wish to give to this test.
SimpleWeight = R * MaxReputationWeight
Since many legitimate ISPs also produce a lot of spam it might be useful to apply a bias to this weight so that these systems appear closer to zero. For example if you applied a maximum weight of 10 and found that many ISPs regularly scored 5 or more then you might add a Bias of -5 to bring those systems toward zero.
BiasedWeight = (R * MaxReputationWeight) + Bias
A more sophisticated system might allow for different weights on the positive and negative going Reputation figures so that the amount of negative or positive weight that can be applied can be adjusted independently. Such a system might also wish to apply a bias directly to the reputation figure before doing that calculation so that the zero point can be adjusted to compensate for averages.
In a system like this if legitimate ISPs tended to get a Reputation Figure of 0.5 then the bias might be -0.5 so that this would become the zero point. Then the positive and negative weight factors could be adjusted so that the desired maximum and minimum weights can be achieved... Note that in this scenario the positive and negative weight settings are not maximum values.
SplitWeight = (0 > (R + Bias)) ? ((R + Bias) * NegativeWeightFactor) : ((R + Bias) * PositiveWeightFactor) MaximumNegativeWeight is given by (-1.0 + Bias) * NegativeWeightFactor MaximumPositiveWeight is given by (+1.0 + Bias) * PositiveWeightFactor When R + Bias == 0.0, the weight will be 0.
The most sophisticated system might provide a graphic interface that maps the reputation figure directly to a desired weight. This would allow the user to shape the effect of the Reputation figure any way they wish in order to gain very tight control over their systems accuracy.
This function scans an SMTP message from a buffer. A scan result block is allocated for the scan and a handle representing the scan result block is returned. The application can then use this handle to retrieve the scan results using the get...() functions. When the application is finished it MUST release the scan result block with a call to closeScan().
The message buffer is expected to contain the raw SMTP data for the message with the local Received: header at the top. The message should not be broken into MIME segments before it is scanned by SNFMulti. This is important because Message Sniffer examines the entire message as well as how the message was assembled by the originating system. Any additional processing is both unnecessary and may remove subtle defects and artifacts that will help Message Sniffer classify the message.
If the message is particularly large it is acceptable to scan only the first 32K bytes of the message. This means that if the calling application wants to scan a large incoming message before it has received all of the DATA during the SMTP connection then it can scan the first 32K of the message and potentially reject the remainder based on the scan result.
When the application is finished with the results from this scan it must release the scan result block with a call to closeScan(). Scan result blocks are allocated as needed and then recycled in order to improve performance. If the application fails to close the scan result blocks then the DLL will continue to allocate additional blocks until there is no more RAM.
The entire scan is performed before this function returns. After that the scanning thread is no longer considered to be active. The resulting scan result buffer may be accessed as often as needed to gather results data from this scan without impacting any other scan operations.
Any number of scan...() operations may be active concurrently up to the limits of the platform.
The scanBuffer() function accepts two additional parameters that are passed on to the logging system to aid in debugging.
The Name parameter is a null terminated string containing an identifier for the message being scanned. This can be any name that can be used later to identify this particular message in the log files such as a serial number, unique hash, or the message-id. For example, when the scanFile() function is called this parameter is filled in with the path to the file that was scanned.
The Setup parameter is an integer representing the number of milliseconds spent so far setting up the message to be scanned. This can be any useful metric - but generally it should represent how much time the system has spent working on preparing and evaluating the message so far. For example, when the scanFile() function is called this value is automatically established with the amount of time spent opening and reading the message file.
IMPORTANT: SNF expects to identify the source IP for the message by searching Received: headers in the message. The application must ensure that the local Received: header is present as the first Received: header in the message in order for this search to be accurate. If necessary the calling application can simulate the local received header using the following minimal form:
Received: from connecting.mta.example.com [12.34.56.78] by this.scanning.system
where connecting.mta.example.com is the optional reverse DNS resolved for the connecting MTA; 12.34.56.78 is the IP of the connecting MTA; and by this.scanning.system is an optional reference to the calling application.
In general a result > 0 indicates a valid scan handle and a result <= 0 indicates an error. NOTE that the scan may have failed even if you get a valid handle. The scan result code you retrieve from get...() may indicate an error. See Result Codes for details.
This function scans an SMTP message from a file. A scan result block is allocated for the scan and a handle representing the results is returned. The application can then use this handle to retrieve the scan results using the get...() functions. When the application is finished it MUST release the scan result block with a call to closeScan().
The scanFile() function is nearly identical to the scanBuffer() function (see above) except that this function accepts the path to a file (null terminated string) instead of a pointer to a message buffer.
One other key difference with between scanBuffer() and scanFile() is that the SNF engine can be configured to inject it's X- headers when scanFile() is used. These same X- headers are available to the calling application when using scanBuffer(), however if the application wishes to pass the message file on to other additional scanners and external processes then scanFile() might be more convenient.
NOTE: There are significant performance implications to scanning files and injecting headers. Each time headers are injected into a message the message file must be rewritten. For optimal performance it is best to collect headers from scanning tools before writing the message to disk so that the message only needs to be written once. Extra file IO is the cost of the convenience of passing a message file to external processes.
The Setup time passed to scanFile() will be added to the time required to open and read the message file prior to scanning. This value will be passed on to the logging system. For example, the calling application might include the number of milliseconds required to perform any previous message testing and the time it has taken to create a temporary message file for scanning. The log will then reflect the complete setup time separately from the time required to perform the SNF message scan.
The FilePath will be passed on to the logging system to identify this message scan in the logs.
In general a result > 0 indicates a valid scan handle and a result <= 0 indicates an error. NOTE that the scan may have failed even if you get a valid handle. The scan result code you retrieve from get...() may indicate an error. See Result Codes for details.
This function returns the scan result code (see Result Codes) and a pointer to a buffer containing any X- headers that were produced for the scan associated with the ScanHandle.
The function is passed a valid ScanHandle which identifies the scan result block to query; the address of a char* which will be changed to point to a buffer containing any X- headers that were produced; and the address of an integer which will be changed to the length of the X- headers buffer.
If no X- headers were produced for the scan then the pointer Bfr will point to a safe empty string and Length will be set to zero. Put another way, Bfr and Length will be consistent with an empty null terminated string.
The char* Bfr and the int Length will remain valid until closeScan() is called for this ScanHandle.
In order for X- headers to be produced the engine must be configured properly. For details visit the XHeader configuration documentation on our web site.
This function returns the scan result code (see Result Codes) and a pointer to a buffer containing any XML Log Data that was produced for the scan associated with the ScanHandle.
The function is passed a valid ScanHandle which identifies the scan result block to query; the address of a char* which will be changed to point to a buffer containing any XML Log Data that was produced; and the address of an integer which will be changed to the length of the XML Log Data buffer.
If no XML Log Data was produced for the scan then the pointer Bfr will point to a safe empty string and Length will be set to zero. Put another way, Bfr and Length will be consistent with an empty null terminated string.
The char* Bfr and the int Length will remain valid until closeScan() is called for this ScanHandle.
In order for XML Log Data to be produced the engine must be configured properly. For details visit the XML Log configuration documentation on our web site.
This function returns the scan result code (see Result Codes) and a pointer to a buffer containing any Classic Log Data that was produced for the scan associated with the ScanHandle.
The function is passed a valid ScanHandle which identifies the scan result block to query; the address of a char* which will be changed to point to a buffer containing any Classic Log Data that was produced; and the address of an integer which will be changed to the length of the Classic Log Data buffer.
If no Classic Log Data was produced for the scan then the pointer Bfr will point to a safe empty string and Length will be set to zero. Put another way, Bfr and Length will be consistent with an empty null terminated string.
The char* Bfr and the int Length will remain valid until closeScan() is called for this ScanHandle.
In order for XML Log Data to be produced the engine must be configured properly. For details visit the Classic Log configuration documentation on our web site.
This function returns the scan result code (see Result Codes) for the scan associated with the ScanHandle.
This function closes a ScanHandle and releases the associated Scan Result Block to the pool. This function MUST be called once for each ScanHandle produced in a scan...() function. Once this function is called the ScanHandle is no longer valid and any pointers returned by previous calls to get...() functions should be discarded (forgotten, not freed!)
This function shuts down the SNFMulti engine. All previously allocated ScanHandles MUST be closed before this function is called. This should be the last function in the DLL that is called by your application (call no other SNFMultiDLL functions after this).
See our web site for more detailed descriptions of these error codes.
Scan results codes 0 through 63 represent normal scan results. By convention a result of 0 indicates ham (not spam); a result of 1 indicates a white-ruled message; and other non-zero result values that are less than 64 indicate some kind of spam or malware was detected. For more details on message scan result codes please see our web site.