Archive for the Spam Control Category

Scan Spam for Legitimate Mail

Posted by Filed Under Spam Control with Comments Off

Checking for Legitimate Mail in the Spam Folder
One of the major issues as a mail administrator is trying to determine if legitimate mail was actually sent to the spam folder.  So one the ways to verify that issue is to scan the spam contents for legitimate terms.  The first problem that you will see is that the spam may be compressed so you will need to uncompress it before you scan.  DO NOT access anything in the spam folder with root access.  Move the spam to a different location and perform all tasks as an unprivileged user.

spam-9B6i8B9rD+id.gz  spam-EeFAaMeaMx1G.gz  spam-ihuIgFpirfUo.gz  spam-NXXzcRNixkii.gz  spam-tf65NhbFJcEu.gz  spam-ZmogC5vZyJlk.gz
spam-9EfOeKyIb6sf.gz  spam-eip3gM+DilfI.gz  spam-IQpbO2KMp8l5.gz  spam-ObJ1xedNLU26.gz  spam-TfpV+yyYsjAB.gz  spam-ZmteJrzYUCBY.gz
spam-9GHo7x7DmOW6.gz  spam-ekJaDB7htlKH.gz  spam-IRC5D5UIEjMk.gz  spam-ocKT1ji46idY.gz  spam-Tg8mub5yGGwn.gz  spam-Zpi4JatgssEL.gz
spam-9HX9P6ajL6Gq.gz  spam-el1WVuh47t9B.gz  spam-IUEwPi8iYgfJ.gz  spam-ODyC3cxIVbZx.gz  spam-Th0SgW4269qG.gz  spam-zQcDemaSYlRj

The spam can all be uncompressed with this command:

gunzip spam*

spam-8AbERQ2zlWnW  spam-CVTlacjyZDm8  spam-gVVz+mQE3IUP  spam-LizqVOW-U8cS  spam-Qk2jzhSjXnQh  spam-TQVW1CzGrPT8  spam-ZmogC5vZyJlk
spam-8BVfclh+5uVl  spam-CxYWRK3g4kwg  spam-G-wjm7cpVWs3  spam-lJwHwY48bCzL  spam-qLWKQzvEFWwp  spam-tSY7hIK5O5Sc  spam-ZmteJrzYUCBY
spam-8EvgnhDx-VNk  spam-cxZbQ8Uw88q6  spam-gWqLRYA3QxAN  spam-loZE8MzZ0SVZ  spam-qM+-EWOF95aP  spam-Tui6Dq-2vnc7  spam-Zpi4JatgssEL
spam-8vINTJLzfwlB  spam-d2eRqmy-4pRL  spam-H1qp0lVdM8dK  spam-LqmKtErj2CvA  spam-qowVrXuhXp-5  spam-TyH60Cn1kMZw  spam-zQcDemaSYlRj
spam-8VZvPZ2aJlAi  spam-DIzzAzS7BXIa  spam-h2fuyznd3PTC  spam-lQmRHTcThADD  spam-Qqq5tl2Stsqe  spam-TyvpEZteK5nw

Now scan for a text strings that may indicate legitimate mail.  The example demonstrates mail that you do not want and was correctly plaged in the spam folder.

grep betting *
spam-2AEQl8mQ9rag:X-Envelope-From: <>

Adjusting Your Spam Rules

Posted by Filed Under Spam Control with Comments Off

Adjusting Spamassassin Rules
In this example the headers from 4 emails are captured and used to help adjust rules that have allowed email that needs to be eliminated, so rule adjustments need to be made.

First do some research on what the rules mean so that you are not adjusting rules that could potentially cause you a lot of problems.

Here are the examples of headers:
Yes, score=5.33 tagged_above=2 required=4.2 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_IMAGE_ONLY_28=0.726, HTML_MESSAGE=0.001, RCVD_IN_BRBL_LASTEXT=1.644, RCVD_IN_RP_RNBL=1.284, URIBL_BLACK=1.775] autolearn=no

Yes, score=5.267 tagged_above=2 required=4.2 tests=[DATE_IN_PAST_12_24=0.804, HTML_MESSAGE=0.001, HTML_MIME_NO_HTML_TAG=0.635, MIME_HTML_ONLY=1.105, RDNS_NONE=1.274, TO_NO_BRKTS_DIRECT=1.448] autolearn=no

No, score=3.118 tagged_above=2 required=4.2 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HK_RANDOM_ENVFROM=0.626, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.504, SINGLE_HEADER_1K=0.597, SUSPICIOUS_RECIPS=2.497] autolearn=no

No, score=2.787 tagged_above=2 required=4.2 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLYTO=2.775, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, T_TO_NO_BRKTS_FREEMAIL=0.01] autolearn=no

Adjusting Spamassassin Rules

This is a rule that is based on the Barracuda Reputation System which generates a list of IPs that have a reputation of sending Spam.  This rule would then be a good candidated to increase the score.  In fact, you may want to increase the score dramatically.

There is no reverse DNS available for the host.  If there are several relays this could mean that the first relay did not have a reverse DNS option.  This is a good indication of a spammer.

This rule searches for similarities if Cc: abd Bcc: are found in the header fields.  The similarities things like all of the email start with joe@.  This is very likely Spam.  You can see it has been given a high number modification.

Edit /etc/mail/spamassassin/ and add the rule adjustments you want to implement.

##### Score Adjustments #####
score RDNS_NONE 2.1

Spamassassin Blacklists

Posted by Filed Under Spam Control with Comments Off

Blacklists are a little easier to understand in that you will find domains or senders that you simply do not want to receive mail from them ever.
There are two blacklist directives.  The first directive, blacklist_from will specify a a sender address that address is what will appear in the Resent-From, From, Envelope-Sender, Resent-Sender or X-Envelope-From. When the sender address matches a score of 100 is added to the sender, which effectively blocks mail from that user.   Edit the /etc/mail/spamassassin/ file to place these entries.

blacklist_from *

The wildcard “*” can be used for including multiple characters to eliminate all users on the domain.

You can remove a user from the list by using the the unblacklist_from directive.


The other method of blacklisting users is the blacklist_to which will blacklist the recipient address.  This users may be found in the Resent-To, Resent-Cc, To, Apparently-To, Delivered-To, Envelope-Recipients, Apparently-Resent-To, X-Envelope-To, Envelope-To, X-Delivered-To, X-Original-To, X-Rcpt-To, X-Real-To, or Cc.


This would be used to eliminate mail coming to possibly forged To header by adding 10 points to the score.

Here is what a looks like with some blacklist entries.
# These should be safe assumptions and allow for simple visual sifting
# without risking lost emails.

required_hits 5
report_safe 0
rewrite_header Subject [SPAM]

blacklist_from *
blacklist_from *
blacklist_from *
blacklist_from *

Spamassassin Whitelists

Posted by Filed Under Spam Control with Comments Off

Global Manual Whitelist
Once you have set up Spamassassin so it is working and you have run it for some time you may want to tune it to provide automatic whitelists and blacklists.  The whitelist will provide a way to insure that the mail from a particular source will never get rejected.  This may be important clients, users on the system or messages from servers that do not necessarily have the right credentials for sending mail. Your company may have important clients that send email that has characteristics that put it on the edge with Spamassassin and you may want to ensure that you do not loose an email.  Or you may have users who are on the system that do not always follow the best protocol for how they construct messages but you want to make sure that their mail is not rejected.  An important use for whitelists is when you have servers that are sending logs, error messages, or monitoring information to a mail account and this mail does not have the format or the sender format that Spamassassin needs to see.  If you are looking for documentation of this information check out our Postfix Mail Server Manual.

The whitelist will automatically reduce the spam score for a user  which is listed in the whitelist.  The score is reduced by 100 points so that would suggest there is very little chance of the mail being rejected.
The procedure for using a whitelist is straightforward.  Use the directive whitelist_from to list any sources that you want to ensure get whitelisted.
Edit the /etc/mail/spamassassin/ file to place these entries.  Listed is the default contents of this file.

required_hits 5
report_safe 0
rewrite_header Subject [SPAM]

Add your whitelists to the file, save and restart Spamassassin.

whitelist_from * *

When you list a user or source, you will list what you see in the Resent-From, From, Envelope-Sender, Recent-Sender or X-Envelope-From headers.  The wildcard “*” can be used to provide all of the users for a domain as you see in the examples above.

You can remove users that you place in the whitelist with unwhitelist_from.


You may want to review the information found in which Spamassassin uses automatically as there may be some users listed that you want to remove.

You do have an additional option for sender whitelisting.  You can use whitelist_from_rcvd which does a reverse lookup with DNS to verify the IP Address of the last trusted relay.  So basically what it is doing is looking up the IP of where the mail came from to make sure it came from an IP Address on the senders network.  Now, this has several problems related to it.  First, you are using precious resources doing reverse DND lookups.  And second, depending how the sender network is designed, you may have problems verifying the IP Address.  Here is how you can use it to drop the score 100 points.


Note that what you are requiring is that the mail sent by tom must come from a mail server on the domain.

Whitelist Recipients
There are three separate levels of whitelisting you can perform for recipients.    If you had a user that did not want any spam checks on their account you would use this method of checking.  The whitelist_to directive can use the Resent-To, Resent-Cc, To, Apparently-To, Delivered-To, Envelope-Recipients, Apparently-Resent-To, X-Envelope-To, Envelope-To, X-Delivered-To, X-Original-To, X-Rcpt-To, X-Real-To, or Cc.

The three levels can be used like this:

whitelist_to   (lowers score by 6)
more_spam_to   (lowers score by 20)
all_spam_to   (lowers score by 100)

Postfix: Bayesian Learning System

Posted by Filed Under Spam Control with Comments Off

Learning System
You are able to additionally tune SpamAssassin to learn about your email.  Two programs are used together to create this learning system; autowhitelisting and Bayesian filtering.  Autowhitelisting is an algorithm that learns about each senders history and modifies the spam score of their subsequent mail.  This should reduce false positives.  Autowhitelisting develops a database for each sender’s mail address and IP address.  Each time a message is received from that sender the score is added to the database score for that sender.  The average score divided by the number of messages is used to modify any new messages.

The most important issue with autowhitelisting is the weight you place on the sender history.  The auto_whitelist_factor is the directive that sets the multiplier between 0-1.  The default is .5 which will make the final score halfway between the message spam score.  If you wanted to increase the weight set the factor to 1.

The system-wide autowhitelist with amavisd.
Edit the /etc/mail/spamassassin/


Sitewide Bayesian Filtering for Amavisd
The idea behind Bayesian filtering is that it will learn aspects of email which will determine how to distinguish between spam and non-spam.  The advantage is that it can help facilitate a more accurate Spam filtering process.  The Bayesian rules sets up baselines that determine how much each rule should change the possibility that the email is Spam.  These rules have features that are likely to be Spam, thus increasing the probability, and they have rules that typically are not in Spam, thus reducing the probablity of Spam.
Edit the /etc/mail/spamassassin/

use_bayes 1
bayes_path /var/amavisd/bayes/bayes

Create the directories you need in /var both amavisd and the subdirectory bayes.  Be sure to chmod 700 the database file so no others can access it.  The user is vscan as is set up in the /etc/amavisd.conf file so that user must have access to the file.  Now with the new version of Spamassassin the line for bayes_pay must not end in a folder, so add the name bayes to it per the example.

chown -R vscan:vscan /var/amavisd/

ls -la /var/amavisd/bayes/
total 8
drwx—— 2 vscan vscan 4096 May 11 07:32 .
drwx—— 3 vscan vscan 4096 May 11 07:32 ..

Using Amavisd and Spamassassin to Block Spam

Posted by Filed Under Spam Control with Comments Off

There will be times when you have you have your Postfix Mail Server set up and Spam is under control and all of a sudden you will see a new wave of Spam hit your site.  This article will help you see how you can make some small adjustments to cut down on new waves of Spam.  Here is an example of the new wave of Spam I started getting.  I will take you through a few steps I used to eliminate this new wave.

Prospector specializes in delivering results for brokers, lenders as well as mortgage products and services
companies nationwide. If you need to jump-start your company’s sales or originations, we can help.

We understand the industry from the inside out and are the only B2B marketer that can GUARANTEE results
with every campaign. An industry leader for almost a decade, Prospector has the largest active network of loan
producers in the nation who are actively seeking help with their businesses.

We specialize in the following areas;

*    FHA, Commercial, Hard Money, Reverse, Conventional Programs
*    Loan Modification Networks and Affiliates
*    MTG Training and Education
*    Lead Generators
*    Loan Processing and Compliance

Step #1: Check Your Logs

When you check your logs you are looking for several things.  One thing that is important is to see what level this particular email was rated at by Spamassassin.  You can see the Hits at 6.353.  Now because my set up is using Amavisd-new, the hits and what results from those hits is listed in the /etc/amavisd.conf file.  The other thing I pick up from the logs is the IP Address of the mail server that is sending the Spam.

Aug 14 12:23:48 ns amavis[30026]: (30026-11) Passed SPAMMY, [] [] <> -> <>, Message-ID: <>, mail_id: OhHzJmCU7qmf, Hits: 6.353, size: 2637, queued_as: A710E207B83, 5129 ms
Aug 14 12:23:54 ns postfix/smtpd[7279]: < unknown[]: EHLO
Aug 14 12:23:54 ns postfix/smtpd[7279]: < unknown[]: MAIL FROM:<>
Aug 14 12:23:54 ns postfix/smtpd[7279]: extract_addr: input: <noreply@hyperbiz1.

Step #2: Drop Hit Levels

Here is the amavisd.conf file hit levels and you can see that the hit level above was 6.3 and the trigger to block the email is at 6.8.  Now an easy solution when you start seeing new Spam is to start slowly reducing the hit level.  So what I did is reduce the 6.8 down to 6.0 and then reduce the “spam detected” level from 6.2 to 5.8.  This is a small adjustment but made a big difference.

$sa_tag_level_deflt  = 2.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.2;  # add ‘spam detected’ headers at that level
$sa_kill_level_deflt = 6.8;  # triggers spam evasive actions (e.g. blocks mail)

Be sure to reload amavisd when you are done making changes.  Remember, amavisd is what controls Spamassassin in this set up.
./amavisd reload
Daemon [28054] terminated by SIGTERM, waiting for dust to settle…
becoming a new daemon…

Now one question you may ask is why not write a header check or some other check for regular expressions.  The answer is that the last thing you want to do is write a lot of special rules.  Try to control Spam by using general princicples that will help reduce Spam because if one wave of Spam is gettign through…another is on the way from someone else.

Postfix: Whitelists and Blacklists

Posted by Filed Under Spam Control, Uncategorized with Comments Off

Whitelists / Blacklists
You can set up whitelists and blacklists to modify the settings to make sure certain email addresses never get blocked or always get blocked.

Prevent any Spam Checking
In order to create a situation where you have no Spam checking you can use the bypass option.  These options are added to amavisd.conf

@bypass_spam_checks_acl = qw(;

The spam lovers option makes sure that if you do a check the email is not tagged as spam and is not quarantined.

@spam_lovers_acl = (‘’, ‘’);

Sender Whitelist and Blacklist
This is built based on the sender address, the FROM in the SMTP connection.  In amavisd if an address is both on the blacklist and on the whitelist both actions take place.

@blacklist_sender_acl = (‘’, ‘’);

@whitelist_sender_acl = (‘’, ‘’);

You can set up a regular expression option that looks like this.

$blacklist_sender_re = new_RE(

Here are the default blacklist/whitelist options in amavisd.conf.  Notice that now amavisd will increase the blacklist score so it is more likely to be Spam.  The score option helps reduce false positives if that is an issue.  In addition, you can add a “-” to decrease the Spam score.

## site-wide opinions about senders (the ‘.’ matches any recipient)
‘.’ => [  # the _first_ matching sender determines the score boost

new_RE(  # regexp-type lookup table, just happens to be all soft-blacklist
[qr'^(bulkmail|offers|cheapbenefits|earnmoney|foryou)@'i         => 5.0],
[qr'^(greatcasino|investments|lose_weight_today|market\.alert)@'i=> 5.0],
[qr'^(money2you|MyGreenCard|new\.tld\.registry|opt-out|opt-in)@'i=> 5.0],
[qr'^(optin|saveonlsmoking2002k|specialoffer|specialoffers)@'i   => 5.0],
[qr'^(stockalert|stopsnoring|wantsome|workathome|yesitsfree)@'i  => 5.0],
[qr'^(your_friend|greatoffers)@'i                                => 5.0],
[qr'^(inkjetplanet|marketopt|MakeMoney)\d*@'i                    => 5.0],

#  read_hash(“/var/amavis/sender_scores_sitewide”),

{ # a hash-type lookup table (associative array)
‘’                        => -3.0,
‘’              => -3.0,
‘’                    => -3.0,
‘’                  => -3.0,
‘’                      => -3.0,
‘’       => -3.0,
‘’      => -3.0,
‘’      => -3.0,
‘’=> -3.0,
‘’ => -3.0,
‘’                => -3.0,
‘’   => -3.0,
‘’        => -3.0,
‘’     => -3.0,
‘’   => -3.0,
‘’ => -3.0,
‘’                => -3.0,
‘’               => -3.0,
‘’                  => -3.0,
‘’          => -3.0,
‘’           => -3.0,
‘’       => -3.0,
‘’          => -3.0,
‘’            => -3.0,
‘’            => -3.0,
‘’                => -5.0,
‘’           => -3.0,
‘’               => -3.0,
‘’           => -3.0,
lc(‘’)    => -3.0,
lc(‘owner-textbreakingnews@CNNIMAIL12.CNN.COM’) => -5.0,

# soft-blacklisting (positive score)
‘’                     =>  3.0,
‘’                           =>  1.0,

],  # end of site-wide tables

You certainly can modify the default lists that are in amavisd.conf.

Quarantine Spam with Amavisd

Posted by Filed Under Spam Control with Comments Off

Amavisd-new acts as a connecting point between Spamassassin, Clamav and Postfix.  This is important to  remember because much of the configuration that would seem to be done on Spamassasin directly, actually occurs in the amavisd-new configuration file.

When amavisd detects spam using Spamassassin it will log it to the log file and it also is able to perform several other actions.  It is possible to send it to a quarantine.  The quarantine will be where you placed it but typically it will be /var/virusmails.  Here is a sample of the spam messages that get collected there.


The quarantine directory is set in /etc/amavisd.conf

$QUARANTINEDIR = ‘/var/virusmails’;  # -Q

You can see from above that when mail is placed in the quarantine directory it  will tag it and compress it.  So to view it run gunzip -d and then review the email.

This example shows Spamassassin tagged this email with a score of 17.454.  It also shows that the email was delivered to the spam-quarantine.  Notice that the Spam-Status shows yo exactly why it was tagged with such a high score.

# gunzip -d /home/spam-FY4ONy4piwUl.gz
# cat /home/spam-FY4ONy4piwUl
Return-Path: <>
Delivered-To: spam-quarantine
X-Envelope-From: <>
X-Envelope-To: <>
X-Quarantine-ID: <FY4ONy4piwUl>
X-Spam-Flag: YES
X-Spam-Score: 17.454
X-Spam-Level: *****************
X-Spam-Status: Yes, score=17.454 tag=2 tag2=6.2 kill=6.9
Received: from[])
by localhost ([]) (amavisd-new, port 10024)
with ESMTP id FY4ONy4piwUl for <>;
Tue, 23 Sep 2008 01:18:23 -0700 (PDT)
Received: from (unknown [])
by with SMTP id 08584207D90
for <>; Tue, 23 Sep 2008 01:18:21 -0700 (PDT)
Message-Id: <>
To: <>
Subject: RE: SALE 89% OFF
MIME-Version: 1.0
Content-Type: text/html
Date: Tue, 23 Sep 2008 01:18:21 -0700 (PDT)

Here are the settings fro Spamassassin found in /etc/amavisd.conf.  You can see that with a tag score of 6.9 or more an email is sent to the quarantine, blocked from the user.

$sa_tag_level_deflt  = 2.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.2;  # add ‘spam detected’ headers at that level
$sa_kill_level_deflt = 6.9;  # triggers spam evasive actions (e.g. blocks mail)

$sa_dsn_cutoff_level = 10;   # spam level beyond which a DSN is not sent

Delivery Status Notification (DSN) Messages
Delivery Status Notification refers to OUTBOUND emails that get a return status that it was not deliverable.   You can see that at or above level 10 no messages will be sent back to the sender.

# $sa_quarantine_cutoff_level = 25; # spam level beyond which quarantine is off

If users are complaining about mail that is marked Spam in their mailboxes, you can drop the numbers.  For example if users are getting email that is marked as Spam and has a number of 5, then you could change to this configuration to put those email in quarantine instead of the user’s mailbox.

$sa_tag_level_deflt  = 2.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 4.8;  # add ‘spam detected’ headers at that level
$sa_kill_level_deflt = 5.0;  # triggers spam evasive actions (e.g. blocks mail)

Of course any time that you adjust these you need to verify that you are not losing mail that is not Spam, but it should be in your quarantine.

« Older Entries