SNFServer Installation and Configuration Copyright (C) 2008 ARM Research Labs, LLC. See www.armresearch.com for the copyright terms. Installing SNFServer to filter email involves the following steps: 1) Check prerequisites. 2) Create the snfuser user and group. 3) Build and install the SNFServer package (using a tarball or a package). 4) Configure the SNFServer package. 5) Interface with the MTA (postfix or sendmail). 6) Configure the OS to start and stop SNFServer on bootup and shutdown. The following sections describes each of these steps for a default installation. Any OS-specific issues are described at the end of each section. Prerequisites ************* Before installing SNFServer, make sure that: 1) The program curl must be installed. Creating the snfuser user and group *********************************** Before installing, the snfuser user and group must be created. For increased security, snfuser user has no shell. OS-specific issues-- The commands to create the snfuser user and group are OS dependent. For your convenience, the commands for creating the user and group for varous OSes are listed here. However, no guarantee is made that these commands will work on your system; please refer to your system documentation. 1) OpenBSD: a) 'useradd -g =uid -m -c "Sniffer Account" -s /bin/false snfuser'. 2) Ubuntu: a) 'adduser --gecos "Sniffer Account" --no-create-home --shell /bin/false snfuser'. 3) RedHat (and variants such as Fedora and CentOS): a) 'adduser --comment "Sniffer Account" -M --shell /bin/false snfuser' 4) Suse: a) 'groupadd snfuser' b) 'useradd -c "Sniffer Account" -s /bin/false -g snfuser snfuser' 5) FreeBSD: a) 'pw user add -c "Sniffer Account" -n snfuser -w no -s /bin/false' Building and installing SNFServer ********************************* For some operating systems, configuration files are normally installed in /etc, and the installation of the remaining files are in /usr (i.e. /usr/sbin, /usr/share/snf-server, etc). In other operating systems, the files are installed in /usr/local/etc or /usr/local. The directories to install are specified by the '--prefix' and '--sysconfdir' options to the configure script (see below). To build and install from the tarball-- 1) Extract the distribution from the tarball: 'tar xvzf snf-server-X.Y.Z.tar.gz', where X.Y.Z is the version of the distribution. 1) 'cd' to the source directory. 2) Run the configure script. See the section on OS-specific issues for the options to specify. This configures the package. 3) Type 'make'. This builds the package. 4) Type 'make install'. This installs the package in the default directories. The following files are installed: Executables and scripts in sbin: SNFServer SNFClient SNF2Check getRulebase.sample snfSniffer.sample snfSnifferFilter.sample snfscan-standalone.sample snfServerControl.sample Sample configuration files in etc/snf-server: SNFServer.xml.sample identity.xml.sample In share/snf-server GBUdbIgnoreList.txt.sample During operation of SNFServer, the following files are created: In share/snf-server: *.snf Log files UpdateReady.txt Temporary files OS-specific options for ./configure: 1) OpenBSD: ./configure --enable-os-type=OpenBSD --sysconfdir=/etc 2) FreeBSD: ./configure --enable-os-type=FreeBSD 3) Ubuntu: ./configure --enable-os-type=Ubuntu --sysconfdir=/etc --prefix=/usr 4) RedHat (and variants such as Fedora and CentOS): ./configure --enable-os-type=RedHat --sysconfdir=/etc --prefix=/usr 5) Suse: ./configure --enable-os-type=Suse --sysconfdir=/etc --prefix=/usr Configuration ************* Configuration consists of creating the configuration files used by SNFServer from the sample configuration files of the distribution. There are three configuration files. The location of the files described in this section are for the installation in /usr and /etc. Two of the files are in /etc/snf-server: 1) SNFServer.xml (sample configuration file is SNFServer.xml.sample). 2) identity.xml (sample configuration file is identity.xml.sample). and one in /usr/share/snf-server: 1) GBUdbIngoreList.txt (sample configuration file is GBUdbIgnoreList.txt.sample). To configure SNFServer, do the following: 1) Copy SNFServer.xml.sample to SNFServer.xml, and edit as follows: a) Specify the directories. The default directories are: The paths need to be changed only if the default directories are not used. If desired for security purposes, restrict the permissions of SNFServer.xml. For example, to make SNFServer.xml readonly by only the snfuser user and snfuser group, enter the following: chmod 440 SNFServer.xml 2) Copy identity.xml.sample to identity.xml, and edit to include the license ID and authentication attributes of the element. If desired for security purposes, restrict the permissions of identity.xml. For example, to make identity.xml readonly by only snfuser, enter the following: chmod 400 identity.xml 3) Copy GBUdbIngoreList.txt.sample to GBUdbIgnoreList.txt, and edit as appropriate (the file format is described in the comments in the file). See README for more information on this file. 4) In the directory for the update script specified in the configuration file (default: /usr/sbin): a) Copy getRulebase.sample to getRulebase, and edit as follows: i) In the line AUTHENTICATION=authenticationxx replace authenticationxx with the authentication for the SNFServer license. ii) In the line LICENSE_ID=licenseid replace licenseid with the license ID of the SNFServer license. iii) Any other changes as necessary if the default directories are not used. b) Ensure that getRulebase is executable by the snfuser user. This can be done with the command: chmod +x getRulebase 5) Ensure that the snfuser user has read/write access to the files in workspace (default: /usr/share/snf-server or /usr/local/share/snf-server) and configuration directory (default: /etc/snf-server). To grant this access, enter the following command, as the root user: chown -R snfuser:snfuser /usr/share/snf-server chown -R snfuser:snfuser /etc/snf-server As you modify files in these directories, please ensure that the read/write permissions for snfuser is maintained. 6) Create the logfile directory, and ensure the snfuser user has read/write access to it: mkdir /var/log/snf-server chown snfuser:snfuser /var/log/snf-server chmod 755 /var/log/snf-server 7) Download the rulebase file: a) 'cd /usr/share/snf-server'. If the workspace specified in the configuration file is not the default, this command should be changed accordingly. b) 'touch UpdateReady.txt'. c) 'chown snfuser UpdateReady.txt'. d) 'su -m snfuser -c "/usr/sbin/getRulebase"'. If getRulebase is in a different directory, this command should be changed accordingly. OS-specific issues-- None. Configuring the OS ****************** The OS can be configured to automatically start and stop SNFServer on system startup and shutdown. The following gives the procedure for each OS: OpenBSD: 1) Add the following line to /etc/rc.local: /usr/local/sbin/snf-server start 2) Add the following line to /etc/rc.shutdown: /usr/local/sbin/snf-server stop 3) Run '/usr/local/sbin/snf-server start' to start the server. FreeBSD: 1) Create the directory /etc/rc.conf.d (if it doesn't exist). 2) Create the file /etc/rc.conf.d/snfsrver with the contents: snfserver_enable="YES" 3) Run '/usr/local/etc/rc.d/snf-server start' to start the server. Ubuntu: 1) Set the VERBOSE variable in /etc/default/rcS to control the script output. 2) Configure the system to run the script. With sysv-rc-conf: sysv-rc-conf -level 2345 snf-server on will start the server for runlevels 2, 3, 4, and 5. And sysv-rc-conf --list snf-server will list the runlevels for which the server is started. 3) Run '/etc/init.d/snf-server start' to start the server. RedHat (and variants such as Fedora) and Suse: 1) Run 'chkconfig --add snf-server'. This sonfigures the system to run the script on startup and shutdown. 2) Run 'chkconfig --list snf-server' to display the runlevels 3) Run 'service snf-server start' to start the SNFServer. Integration with MTAs ********************* The section assumes you will be using SNFServer with an MTA and simply injecting headers that will be used later to remove, quarantine, or otherwise redirect messages detected as spam. There are as many ways to use SNFServer as there are systems using it -- so the following is just a good starting point. It is presumed that SNFServer is configured with x-header injection turned on and that the x-headers have been customized to suit your needs. Check the section of your /etc/snf-server/SNFServer.xml file to verify that SNFServer is configured to do what you want. Integration with postfix ------------------------ One way to integrate with postfix is to configure postfix to use the snfSniffer script (described below) as a content filter. The snfSniffer script passes the message through SNFServer, and then reinjects the message into the mail system. 1) In /usr/sbin (or /usr/local/sbin), copy the script snfSniffer.sample to snfSniffer set the correct access rights: cp snfSniffer.sample snfSniffer chown snfuser snfSniffer chmod 550 snfSniffer The default snfSniffer script creates a temporary copy of the message, scans it with SNFServer, and then reinjects the message. 2) Change /etc/postfix/master.cf as follows (LEADING WHITE SPACES ARE IMPORTANT WHEN MAKING THIS CHANGE): change (each line is enclosed in single quotes; add only the text between the single quotes): 'smtp inet n - n - - smtpd' to: 'smtp inet n - n - - smtpd' ' -o content_filter=snfilter:dummy' also add: 'snfilter unix - n n - 10 pipe' ' flags=Rq user=snfuser argv=/usr/sbin/snfSniffer' ' -f ${sender} -- ${recipient}' to master.cf. Specify the directory snfSniffer is in if not /usr/sbin. At this point you could just restart postfix, and hope nothing goes wrong. Instead, it would be smarter to first test the installation from the command line by injecting a message directly into the filter script "snfSniffer". We can issue a command like (in directory /usr/share/snf-server): /usr/sbin/snfSniffer -f sender recipient < junkmsg.txt Where junkmsg.txt is a spam test message. We should also test a clean message to make sure that this script is working as we expect it to. In this case we would issue a command like /usr/sbin/snfSniffer -f sender recipient < cleanmsg.txt If you've done everything correctly then all you have to do is reload postfix to start the content_filter working. 4) 'postfix reload'. Restart postfix. 5) To stop using the snfSniffer filter: i) Comment out or remove the '-o content_filter=snfilter:dummy' line in the /etc/postfix/master.cf file. ii) 'postsuper -r ALL' to remove content filter request records from existing queue files. iii) 'postfix reload' to reload the postfix configuration. OS-specific issues-- 1) OpenBSD: In the default postfix configuration for OpenBSD, the postfix programs run chrooted in /var/spool/postfix. For that configuration, the /etc/pwd.db file needs to be made available to the postfix programs. This can be done by copying pwd.db: cp /etc/pwd.db /var/spool/postfix/etc If the /etc/pwd.db file is modified after copying, the postfix system issues a warning. To avoid this warning, copy the /etc/pwd.db file to /var/spool/postfix/etc directory whenever /etc/pwd.db is modified. Integration with sendmail ------------------------- One way to integrate with send is to configure sendmail to use the snfSnifferFilter script (described below) as a filter. The snfSniffer script passes the message through SNFServer, and then sends the message to stdout. The integration is done by having sendmail use procmail to filter all mail (which is normally the default), and configuring procmail to use snfSnifferFilter as a filter. 1) In /usr/sbin (or /usr/local/sbin), copy the script snfSnifferFilter.sample to snfSnifferFilter set the correct access rights: cp snfSnifferFilter.sample snfSnifferFilter chown snfuser snfSnifferFilter chmod 550 snfSnifferFilter The default snfSnifferFilter script creates a temporary copy of the message, scans it with SNFServer, and then send the message to stdout. 2) Verify that sendmail is configured to use procmail to deliver mail. Please see the sendmail and procmail documentation for how to do this. 3) Create the file /etc/procmailrc (or whatever the system procmail rcfile is; see the procmail documentation for the path on your system) to contain the following lines: ':0 fw' '| /usr/sbin/snfSnifferFilter' where the single quotes are not to be included. At this point mail should be filtered by SNFServer. To check this, issue a command like (in directory /usr/share/snf-server): /usr/sbin/snfSnifferFilter < junkmsg.txt Where junkmsg.txt is a spam test message. We should also test a clean message to make sure that this script is working as we expect it to. In this case we would issue a command like /usr/sbin/snfSnifferFilter < cleanmsg.txt In either case you should see the message with the headers added by SNFServer. 4) To stop using the snfSnifferFilter: i) Comment out or remove the two lines you added to /etc/procmailrc. OS-specific issues-- 1) OpenBSD: The default sendmail confiugration might not use procmail. To configure sendmail to use procmail, please ensure that procmail is installed and see the procmail documentation (/usr/local/share/examples/procmail/advanced) for the procedure to integrate procmail as an integrated local mail delivery agent. By default, procmail is installed in /usr/local/bin, and the system procmail rcfile is /etc/procmailrc. After you make the changes, stop sendmail with 'pkill sendmail', and start it with 'sendmail -bd'. 2) FreeBSD: The default sendmail confiugration might not use procmail. To configure sendmail to use procmail, please ensure that procmail is installed and see the procmail documentation (/usr/local/share/examples/procmail/advanced) for the procedure to integrate procmail as an integrated local mail delivery agent. By default, procmail is installed in /usr/local/bin, and the system procmail rcfile is /usr/local/etc/procmailrc. After you make the changes, restart sendmail with '/etc/rc.d/sendmail restart'. 3) RedHat, Fedora, Centos, Suse, and Ubuntu: Restart sendmail with '/etc/init.d/sendmail restart'. Debugging --------- To obtain additional information for debugging, SNFServer can be run in debug mode. In debug mode, additional messages are output to a file, and core dumps are enabled. Debug mode is enabled and disabled using the snf-server startup script. Please see the section "Configuring the OS" the the location of the startup script for each OS. To enable debug mode, run: snf-server debug_mode Note: Running this command creates a file /usr/share/snf-server/debug_mode that indicates to the snf-server startup script to start SNFServer in debug mode. This file contains brief documentation about running in debug mode. To enable production mode (non-debug mode), run: snf-server production_mode Note: Running this command creates a file /usr/share/snf-server/production_mode that indicates to the snf-server startup script to start SNFServer in production mode. This file has the same content as /usr/share/snf-server/debug_mode. After the desired mode is enabled, you can start or restart SNFServer using the standard procedure. In debug mode for all OSes except OpenBSD, SNFDebugServer is run with strace (with the '-r -tt -v' options), and all the output is written to /var/log/snf-server/debug.log. Also, any core dumps are written to /usr/share/snf-server. In debug mode for OpenBSD, SNFDebugServer is run with ktrace. The output of SNFDebugServer is written to /var/log/snf-server/debug.log, and the output of strace is written to /var/log/snf-server/ktrace.out. OS-specific issues-- 1) FreeBSD: You'll need to ensure that core dumps are enabled. On FreeBSD 7.0, the command to do this is: sysctl kern.coredump=1 You'll also need to ensure that /proc is mounted. You can mount /proc with the following command: mount -t procfs proc /proc When running in debug mode, 'snf-server start' displays the PID of the strace process instead of the SNFDebugServer process. 'snf-server status' displays the PID values correctly. 2) OpenBSD: Use kdump to view the kernal trace file: kdump /var/log/snf-server/ktrace.out