Scan Spam for Legitimate Mail

Posted by Filed Under Spam Control with Comments Off

Checking for Legitimate Mail in the Spam Folder
One of the major issues as a mail administrator is trying to determine if legitimate mail was actually sent to the spam folder.  So one the ways to verify that issue is to scan the spam contents for legitimate terms.  The first problem that you will see is that the spam may be compressed so you will need to uncompress it before you scan.  DO NOT access anything in the spam folder with root access.  Move the spam to a different location and perform all tasks as an unprivileged user.

spam-9B6i8B9rD+id.gz  spam-EeFAaMeaMx1G.gz  spam-ihuIgFpirfUo.gz  spam-NXXzcRNixkii.gz  spam-tf65NhbFJcEu.gz  spam-ZmogC5vZyJlk.gz
spam-9EfOeKyIb6sf.gz  spam-eip3gM+DilfI.gz  spam-IQpbO2KMp8l5.gz  spam-ObJ1xedNLU26.gz  spam-TfpV+yyYsjAB.gz  spam-ZmteJrzYUCBY.gz
spam-9GHo7x7DmOW6.gz  spam-ekJaDB7htlKH.gz  spam-IRC5D5UIEjMk.gz  spam-ocKT1ji46idY.gz  spam-Tg8mub5yGGwn.gz  spam-Zpi4JatgssEL.gz
spam-9HX9P6ajL6Gq.gz  spam-el1WVuh47t9B.gz  spam-IUEwPi8iYgfJ.gz  spam-ODyC3cxIVbZx.gz  spam-Th0SgW4269qG.gz  spam-zQcDemaSYlRj

The spam can all be uncompressed with this command:

gunzip spam*

spam-8AbERQ2zlWnW  spam-CVTlacjyZDm8  spam-gVVz+mQE3IUP  spam-LizqVOW-U8cS  spam-Qk2jzhSjXnQh  spam-TQVW1CzGrPT8  spam-ZmogC5vZyJlk
spam-8BVfclh+5uVl  spam-CxYWRK3g4kwg  spam-G-wjm7cpVWs3  spam-lJwHwY48bCzL  spam-qLWKQzvEFWwp  spam-tSY7hIK5O5Sc  spam-ZmteJrzYUCBY
spam-8EvgnhDx-VNk  spam-cxZbQ8Uw88q6  spam-gWqLRYA3QxAN  spam-loZE8MzZ0SVZ  spam-qM+-EWOF95aP  spam-Tui6Dq-2vnc7  spam-Zpi4JatgssEL
spam-8vINTJLzfwlB  spam-d2eRqmy-4pRL  spam-H1qp0lVdM8dK  spam-LqmKtErj2CvA  spam-qowVrXuhXp-5  spam-TyH60Cn1kMZw  spam-zQcDemaSYlRj
spam-8VZvPZ2aJlAi  spam-DIzzAzS7BXIa  spam-h2fuyznd3PTC  spam-lQmRHTcThADD  spam-Qqq5tl2Stsqe  spam-TyvpEZteK5nw

Now scan for a text strings that may indicate legitimate mail.  The example demonstrates mail that you do not want and was correctly plaged in the spam folder.

grep betting *
spam-2AEQl8mQ9rag:X-Envelope-From: <>

Adjusting Your Spam Rules

Posted by Filed Under Spam Control with Comments Off

Adjusting Spamassassin Rules
In this example the headers from 4 emails are captured and used to help adjust rules that have allowed email that needs to be eliminated, so rule adjustments need to be made.

First do some research on what the rules mean so that you are not adjusting rules that could potentially cause you a lot of problems.

Here are the examples of headers:
Yes, score=5.33 tagged_above=2 required=4.2 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_IMAGE_ONLY_28=0.726, HTML_MESSAGE=0.001, RCVD_IN_BRBL_LASTEXT=1.644, RCVD_IN_RP_RNBL=1.284, URIBL_BLACK=1.775] autolearn=no

Yes, score=5.267 tagged_above=2 required=4.2 tests=[DATE_IN_PAST_12_24=0.804, HTML_MESSAGE=0.001, HTML_MIME_NO_HTML_TAG=0.635, MIME_HTML_ONLY=1.105, RDNS_NONE=1.274, TO_NO_BRKTS_DIRECT=1.448] autolearn=no

No, score=3.118 tagged_above=2 required=4.2 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HK_RANDOM_ENVFROM=0.626, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.504, SINGLE_HEADER_1K=0.597, SUSPICIOUS_RECIPS=2.497] autolearn=no

No, score=2.787 tagged_above=2 required=4.2 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLYTO=2.775, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, T_TO_NO_BRKTS_FREEMAIL=0.01] autolearn=no

Adjusting Spamassassin Rules

This is a rule that is based on the Barracuda Reputation System which generates a list of IPs that have a reputation of sending Spam.  This rule would then be a good candidated to increase the score.  In fact, you may want to increase the score dramatically.

There is no reverse DNS available for the host.  If there are several relays this could mean that the first relay did not have a reverse DNS option.  This is a good indication of a spammer.

This rule searches for similarities if Cc: abd Bcc: are found in the header fields.  The similarities things like all of the email start with joe@.  This is very likely Spam.  You can see it has been given a high number modification.

Edit /etc/mail/spamassassin/ and add the rule adjustments you want to implement.

##### Score Adjustments #####
score RDNS_NONE 2.1

Sending non-FQDN Mail to Your Mail Server

Posted by Filed Under Mail Server Basics with Comments Off

Sending Mail from Server Scripts

Often when an administrator tries to send the results of a script from a server to a user on the mail server for the company the email is rejected. One of the reasons this happens is that the server that is sending the mail does not have

a Fully Qualified Domain Name (FQDN), either because it was never set or because DNS cannot resolve to this IP Address as it is behind a firewall. The other reason for the failure is that the mail server specifically is designed to drop both non-FQDN hostnames or drop invalid hostnames.

The rules are that each valid hostname should contain a top level domain (com,net,com, etc), a domain, like example in and it should contain a “.” separating the two. Both of these settings are indications of SPAM. Here is what the smtpd restrictions look like on a Postfix mail server.  Note: All lines must be indented after the “=”, WordPress just will not recognize an empty space.

smtpd_recipient_restrictions =

This image illustrates the problem. The database server sending the mail from a generated script is a legitimate server to send mail, but since it does not have a Fully Qualified Domain Name (FQDN) the mail is rejected because of the two recipient restrictions:


Log Entries Demonstrating Rejection

Oct 4 11:32:42 mail postfix/smtpd[30870]: connect from[]
Oct 4 11:32:42 mail postfix/smtpd[30870]: warning: Unable to look up MX host for Host not found
Oct 4 11:32:42 mail postfix/cleanup[30874]: B0966D0C8A: message-id=<>
Oct 4 11:32:42 mail postfix/qmgr[22286]: B0966D0C8A: from=<>, size=298, nrcpt=1 (queue active)
Oct 4 11:32:42 mail postfix/smtp[30875]: B0966D0C8A: to=<>, relay=none, delay=0.01, delays=0/0.01/0/0, dsn=5.4.4, status=undeliverable (Host or domain name not found. Name service error for type=A: Host not found)
Oct 4 11:32:42 mail postfix/qmgr[22286]: B0966D0C8A: removed
Oct 4 11:32:45 mail postfix/smtpd[30870]: NOQUEUE: reject: RCPT from[]: 450 4.1.7 <>: Sender address rejected: undeliverable address: Host or domain name not found. Name service error for type=A: Host not found; from=<> to=<> proto=ESMTP helo=<>
Oct 4 11:32:45 mail postfix/smtpd[30870]: disconnect from[]

The solution is easy to perform. Add the database server IP Address to the mynetworks parameter in

mynetworks =,

Restart Postfix and you are ready to go.

Whether the mail server is Postfix, Sendmail, Exim, or Exchange they all have settings which allow exceptions like the above mentioned.

Log Entries Demonstrating Solution

Oct 4 11:36:17 mail postfix/smtpd[30902]: connect from localhost[]
Oct 4 11:36:17 mail postfix/smtpd[30902]: 18AF2D0C8D: client=localhost[]
Oct 4 11:36:17 mail postfix/cleanup[30897]: 18AF2D0C8D: message-id=<>
Oct 4 11:36:17 mail postfix/qmgr[30892]: 18AF2D0C8D: from=<>, size=1188, nrcpt=1 (queue active)
Oct 4 11:36:17 mail amavis[9320]: (09320-01) Passed CLEAN, [] [] <> -> <>, Message-ID: <>, mail_id: sw1H7xdMgUHv, Hits: 1.179, size: 682, queued_as: 18AF2D0C8D, 9490 ms
Oct 4 11:36:17 mail postfix/smtp[30898]: 45DCAD0C8A: to=<>, relay=[]:10024, delay=10, delays=0.11/0.01/1.1/8.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA([]:10025): 250 2.0.0 Ok: queued as 18AF2D0C8D)
Oct 4 11:36:17 mail postfix/qmgr[30892]: 45DCAD0C8A: removed
Oct 4 11:36:17 mail postfix/local[30903]: 18AF2D0C8D: to=<>, relay=local, delay=0.64, delays=0.08/0.52/0/0.04, dsn=2.0.0, status=sent (delivered to mailbox)
Oct 4 11:36:17 mail postfix/qmgr[30892]: 18AF2D0C8D: removed

Installing Dovecot 2 on CentOS

Posted by Filed Under Dovecot with Comments Off

Dovecot version 2 has been available for some time but it may not be in the repositories for CentOS for a long time to come.  This tutorial will show you how to perform a basic install of Dovecot version 2 and get it working for one domain…other tutorials will follow with addtional configurations.
Download a pre-built RPM

yum install openssl-devel

The repository for CentOS 5.5 does not have the version required for postgresql-libs.  As a result you will need to enable the testing repository for CentOS.  Warning: Any time you install from the testing repository you will be taking on additional risk, so think twice about using it on a production machine.  Create access to the CentOS testing repo.  Add the Centos-Test.repo in /etc/yum.repos.d.

name=CentOS-5 Testing

Only enable the testing repository for this one package.

yum --enablerepo=c5-testing install postgresql-libs

rpm -ivh dovecot-2.0.9-1_125.el5.x86_64.rpm

Configuration: One Domain

The first example is setting up Dovecot for one domain and a local user.  The structure for Dovecot 2 is completely different from Dovecot 1, early versions, but the configuration is similar.  Now you have a directory inside of /etc/dovecot called conf.d which contains a number of smaller config files.

10-auth.conf      10-ssl.conf   90-acl.conf                  auth-ldap.conf.ext        auth-system.conf.ext
10-director.conf  15-lda.conf   90-plugin.conf               auth-master.conf.ext      auth-vpopmail.conf.ext
10-logging.conf   20-imap.conf  90-quota.conf                auth-passwdfile.conf.ext
10-mail.conf      20-lmtp.conf  auth-checkpassword.conf.ext  auth-sql.conf.ext
10-master.conf    20-pop3.conf  auth-deny.conf.ext           auth-static.conf.ext

These configuration files are read by Dovecot in order.

##### Basic Dovecot 2 Configuration #####
protocols = imap pop3 lmtp
login_greeting = Dovecot Available
mail_privileged_group = mail
!include conf.d/*.conf

The second file that must be edited is the mailbox location.

Mail Location
In order to get some idea of where the actual mailbox is, login as the user and run this command.

set | grep -i mail

Edit the 10-mail.conf  to allow the correct mail location.
This example, which uses mutt for the local user, will collect the mail in /var/spool/mail.  The “%u” represents the user name of the mail recipient.  When the mail has been read it can be moved to the user’s home directory and the mail folder.

mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u

dovecot -n
# 2.0.9: /etc/dovecot/dovecot.conf
# OS: Linux x86_64 CentOS release 5.5 (Final)
auth_verbose = yes
disable_plaintext_auth = no
login_greeting = Dovecot Available
mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u
mbox_write_locks = fcntl
passdb {
driver = pam
ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
ssl_key = </etc/pki/dovecot/private/dovecot.pem
userdb {
driver = passwd

Mail File Lock
Verify the permissions and location of the mail directory and then provide additional permissions for Dovecot in this situation for the dotlock files.

ls -ld /var/spool/mail
drwxrwxr-x 2 root mail 4096 Mar 22 07:58 /var/spool/mail

In order to make sure that the mail file can be locked add this line.

mail_privileged_group = mail

Bot Connection Attempts to Postfix

Posted by Filed Under Filters with Comments Off

There are obviously many different kinds of bots that systematically search the Internet for vulnerabilities.  At times you may see bot connection attempts to your Postfix mail server in the logs.  Here is an example of what that may look like.

As you examine the logs (/var/log/maillog centOS) you see that the smtpd daemon receives a connection from a machine at  Immediately the smptd daemon which listens on port 25 recognizes that the connection is a command that is not related to a mail server “GET / HTTP/1.1″ and issues the report “warning: non-SMTP command” and disconnects from the client.

The anvil daemon which is a defense against denial-of-service attacks reports that the max connection rate is one connection per 60 seconds “max connection rate 1/60s”.

Feb 20 10:31:59 mail postfix/smtpd[891]: connect from[]
Feb 20 10:31:59 mail postfix/smtpd[891]: warning: non-SMTP command from[]: GET / HTTP/1.1
Feb 20 10:31:59 mail postfix/smtpd[891]: disconnect from[]
Feb 20 10:35:19 mail postfix/anvil[893]: statistics: max connection rate 1/60s for (smtp: at Feb 20 10:31:59
Feb 20 10:35:19 mail postfix/anvil[893]: statistics: max connection count 1 for (smtp: at Feb 20 10:31:59
Feb 20 10:35:19 mail postfix/anvil[893]: statistics: max cache size 1 at Feb 20 10:31:59

As you review this information remember that anvil is designed to be used as a defense mechanism protecting against attacks, it is not designed to regulate legitimate traffic.

Speed Up Your Mail Server

Posted by Filed Under Performance with Comments Off

One of the most important factors of speed for a mail server is the ability to query DNS as quickly as possible.  DNS queries can be made faster by using one of two options.  The first option is to provide a DNS server in the /etc/resolv.conf file that is local and that provides recursive lookups.  Proximity for DNS means speed.  The transfer of queries over long distances just increases the delay so use a DNS server that is close to the mail server.  Also list at least two nameservers in /etc/resolv.conf.

nameser ver

Note in this example one DNS server is local the other is outside the local network.  By providing two mail servers the mail server can still function if one DNS server is not available.

The second necessary option is to make sure that the DNS server you use for the mail server will allow the mail server to make recursive requests, not just iterative requests.  When a machine is able to make recursive requests of a DNS server, that DNS server is required to find a definitive answer to any queries requested.  In other words, the DNS server must come up with “the answer” to any queries.  If a request is only iterative, it means the DNS server can provide it’s best guess, it is not required to do the research for a definitive answer.   Below is an options line found in a DNS server that indicates that a subnet, the localhost and a single IP Address have the access to make recursive requests.

options {
allow-recursion {; localhost;; };

If enough speed cannot be attained by using a DNS server, then a caching-nameserver can be installed on the mail server.  A caching  nameserver that is either located on the Postfix mail server itself or very close on the network is one the the most significant options you can use.  Because mail is closely tied to DNS, the faster you can resolve domains the more efficient everything will be.  The cache is significant because once a domain is in the cache the lookup is almost instant.

yum install -y caching-nameserver
cd /etc
cp named.caching-nameserver.conf named.conf
chown root:named named.conf
service named start

Note the configuration of the file that was copied to named.conf allows the localhost (the mail server) recursive queries and a cache.

options {
listen-on port 53 {; };
listen-on-v6 port 53 { ::1; };
directory       “/var/named”;
dump-file       “/var/named/data/cache_dump.db”;
statistics-file “/var/named/data/named_stats.txt”;
memstatistics-file “/var/named/data/named_mem_stats.txt”;

allow-query     { localhost; };
allow-query-cache { localhost; };
logging {
channel default_debug {
file “data/”;
severity dynamic;
view localhost_resolver {
match-clients      { localhost; };
match-destinations { localhost; };
recursion yes;
include “/etc/named.rfc1912.zones”;

Edit /etc/resolv.conf and make sure the first nameserver is the localhost.

You can add a second and third nameserver if you want redundancy.

Test your caching nameserver by installing bind-utils so you can so some tests.

yum install -y bind-utils

After you have installed the caching-nameserver correctly perform perform a query for a domain and note the time it takes (highlighted).  Then perform it again and note how much it has changed as the second query comes from the cache.


; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_5.3 <<>>
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49530
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 4, ADDITIONAL: 0

;            IN    A

;; ANSWER SECTION:        300    IN    A        300    IN    A        300    IN    A        300    IN    A        300    IN    A        300    IN    A

;; AUTHORITY SECTION:        172800    IN    NS        172800    IN    NS        172800    IN    NS        172800    IN    NS

;; Query time: 144 msec
;; WHEN: Sat Feb 12 12:27:37 2011
;; MSG SIZE  rcvd: 196

;; Query time: 0 msec
;; WHEN: Sat Feb 12 12:28:07 2011
;; MSG SIZE  rcvd: 196

Manage Mail Server Connections

Posted by Filed Under Performance with Comments Off

One aspect of managing mail server connections is managing Keep-Alives.  Managing Keep-Alives with TCP connections may increase reliability of connections or save resources on the server.

Once a connection is made with a mail server, the TCP protocol does not determine that data must be exchanged in order to maintain the connection.  It is possible for a connection to remain open for a long period of time without exchanging data.  Keep-Alive helps the server determine if the connection is no longer available as there is no point in maintaining resources if the connection is not available.

Resource Management
Here is an example of a client connected to a mail server.  Note how many connections are made to the secure IMAP.  Depending upon how many folders in your IMAP account and depending on how many accounts, you will have multiple connections to manage.

tcp        0      0        ESTABLISHED
tcp        0      0        ESTABLISHED
tcp        0      0        ESTABLISHED
tcp        0      0        ESTABLISHED
tcp        0      0        ESTABLISHED

The problem with so many connections to the mail server is  to manage resources for the mail server when you have a lot of  users and many connections.  Keep-Alives is one aspect of managing server resources.

By managing Keep-Alive settings you can either save resources that are being wasted or increase the Keep-Alive settings to insure more stable connections.

Keep-Alive Settings
There are three variables that refer to keep alives.
This setting is the interval between subsequential keepalive tests.  This setting occurs regardless of what is happening on the connection.

This setting is the interval between the last data packet sent and the first keepalive test.  Once the connection is marked as keepalive, the counter is not used.  Note, ACKs are not going to be considered data.

This setting is the number of unacknowledged tests to send before considering the connection dead and then notifiying the application layer.

Here are default settings.
cat /proc/sys/net/ipv4/tcp_keepalive_intvl
cat /proc/sys/net/ipv4/tcp_keepalive_time
cat /proc/sys/net/ipv4/tcp_keepalive_probes

These settings allow for connection getting dropped after 2 hours and 11 seconds.  Adjusting these settings can allow for longer connection times or lesser connection times to save on system resources.

Changing Keep-Alive Settings
For testing purposes the best thing to do is to echo a setting the current setting.  This will go away on restart.  For example, if your connections were not as reliable as you needed, clients complained about dropped connections, then increase your Keep-Alive settings.

echo 15 > /proc/sys/net/ipv4/tcp_keepalive_probes

If you were more interested in saving resources on the mail server, then decrease the time for Keep-Alive.

echo 6000 > /proc/sys/net/ipv4/tcp_keepalive_time

Whatever you do test and listen to clients to verify your settings.

PostfixAdmin and Fetchmail

Posted by Filed Under Postfixadmin with Comments Off

Getting Fetchmail to Work with Postfix Admin
The default set up with Postfixadmin looks like it should work but it will not work correctly until you modify a few things.  This tutorial will help you get it working correctly.

WARNING: Use this at your own risk.  It has been found to work however, it may pull down all email from an account and remove it on the remote server.  Test to verify that it is working like you want.

First go to PostfixAdmin and set up the account you want to pull mail to.

Mailbox            this is the local mailbox
Server               this is the remote server for the account you want to pull from
Auth Type      usually you will supply a password
User                 user_name or remote account
Password        password for account
Folder             the remote folder you want to retrieve
Poll                 Does not work, you must use cron jobs for script
Fetch All       click for all messages
Keep              Be sure to keep on both servers until you are satisfied
Protocol         IMAP usually
SSL                 if it is required

Ubuntu 10.04
Install several programs in preparation.

apt-get install fetchmail liblockfile-simple-perl

Create a directory and lockfile for fetchmail.

mkdir /var/run/fetchmail
touch /var/run/fetchmail/fetchmail-all.lock

You will need to install several perl modules.  Here is a list:


perl -MCPAN -e shell


When you install them they should return an “OK” at the end or you will need to fix dependencies.

Edit the script.  The key here is to put your database name, user name and password in the script.  Note several lines have been commented out as the file in /etc/mail/postfixadmin/ did not work as well as actually changing the script.

use DBI;
use MIME::Base64;
# use Data::Dumper;
use File::Temp qw/ mkstemp /;
use Sys::Syslog;
#require “liblockfile-simple-perl”;
use LockFile::Simple qw(lock trylock unlock);

########## Change the following variables to fit your needs ##########

# database settings

# database backend – uncomment one of these
#our $db_type = ‘Pg’;
my $db_type = ‘mysql’;

# host name
our $db_host=”″;
# database name
our $db_name=”postfix”;
# database username
our $db_username=”postfix_user”;
# database password
our $db_password=”database_password”;

# instead of changing this script, you can put your settings to /etc/mail/postfixadmin/fetchmail.conf
# just use perl syntax there to fill the variables listed above (without the “our” keyword). Example:
# $db_username = ‘mail’;
#if (-f “/etc/mail/postfixadmin/fetchmail.conf”) {
#       require “/etc/mail/postfixadmin/fetchmail.conf”;

Make the script executable.
chmod 755 /var/www/postfixadmin/ADDITIONS/

It may choke if you do not run it directly from perl.
sh /var/www/postfixadmin/ADDITIONS/
/var/www/postfixadmin/ADDITIONS/ 3: use: not found
/var/www/postfixadmin/ADDITIONS/ 4: use: not found
/var/www/postfixadmin/ADDITIONS/ 6: use: not found
/var/www/postfixadmin/ADDITIONS/ 7: use: not found
/var/www/postfixadmin/ADDITIONS/ 9: Syntax error: “(” unexpected

Call it from perl and it works…it will complain running as root.
/usr/bin/perl /var/www/postfixadmin/ADDITIONS/

Put it in a cron job if you want to run it continually.

It finally works correctly and is a great way to move mail to a new account.

« Older Entries