SNF4CGP/Misc/Notes.txt

20091009.1549 _M

Copyright (C) 2009 ARM Research Labs, LLC

SNF4CGP provides a high-performance, multi-threaded spam filter plugin for
Communigate Pro using the Message Sniffer (SNFMulti) engine.

http://www.communigate.com/CommuniGatePro/VirusScan.html
http://www.communigate.com/CommuniGatePro/Helpers.html
http://www.communigate.com/CommuniGatePro/Data.html#String

The general design goals are (in no particular order):

1. Provide a customizable match between each possible SNF result code and
   each possible communigate response.

2. Provide comprehensive debugging and performance logging capabilities
   consistent with the way Communigate operates.

3. Provide a flexible message-hold capability to allow for additional processing
   by other software for quarantine operations or further message handing.

4. Provide the highest possible message handling performance.

5. Create an architecture that is easy to extend and support on multiple
   platforms.

See Architecture.jpg for a drawing of the architecture. Note that the snap-shot
is slightly out of spec. When the Architecture.jpg was snapped the thinking was
that each Job in the pool would contain it's own dedicated scanner. Later during
development it was decided that scanners should also be pooled and conserved and
only bound to a job for as short a time as possible. The latest drawing shows
this by connecting a Scanner Pool to the scan operation with a short loop and
indicating that the scanner pool resides inside the Job pool with a small symbol
in each. When we redraw this architectural diagram we will include these changes.

The application uses threads in this way:

* One thread sets up the application, reads requests, and ultimately closes
down the application.

* Another thread processes all finished jobs by pumping their output through
cout and then recycling job objects.

* Worker threads from a pool process each request.

Resource contention is minimized wherever possible:

- Ram allocation is conserved by pre-allocating and reusing buffers.

- Scanners, Worker threads, and Jobs are allocated as needed and pooled.

- Scanners are borrowed from a pool only long enough to scan a message and
  capture the results.

- Configuration data is parsed and cached only when it has changed.

- Worker threads are released back to the pool without waiting for IO to cout.

- As much as possible thread interactions are reduced to reference passing
  in very short mutex protected functions.