Archive for the Statistics Category

Automating Log Statistics

Posted by Filed Under Statistics with Comments Off

This tutorial will show you how to set up your log statistics to be mailed to you every night so you know what is happening with your Postfix Mail Server.

Set up a cron job so that it will run at 11:55 PM, just before midnight as you want to get all of the logs for that day.  Be sure to give enough time for the script to run before the next day.  Here is the line to use if you have install pflogsumm at /usr/pflogsumm, see this article for installation.

As root run:

crontab -e

Now add this line with your email at the end.  Note the \ which indicates a line break.

55 23 * * * /usr/pflogsumm/./pflogsumm.pl -u 5 -h 5 –problems_first -d today \
/var/log/maillog | mail -s “PostFix Report  `date`” \someemail@somewhere.com

Here is an example

Postfix log summaries for Nov 13

Grand Totals
————
messages

19   received
16   delivered
0   forwarded
1   deferred  (5  deferrals)
1   bounced
16   rejected (50%)
0   reject warnings
0   held
0   discarded (0%)

69610   bytes received
66259   bytes delivered
10   senders
9   sending hosts/domains
6   recipients
6   recipient hosts/domains

message deferral detail
———————–
smtp (total: 5)
3   hjeigb.info[69.64.157.16]: Connection refused
2   conversation with hjeigb.info[216.52.184.243] timed out while …

message bounce detail (by relay)
——————————–
none (total: 1)
1   Host not found

message reject detail
———————
cleanup
header (total: 2)
1   Content-Type: text/html; charset=”iso-2022-jp”
1   Content-Type: text/plain;??charset=”gb2312″
RCPT
blocked using sbl-xbl.spamhaus.org (total: 2)
1   125.187.32.174
1   konstantynow.mm.pl
Helo command rejected: need fully-qualified hostname (total: 8)
2   122.198.44.5
1   59.151.193.207
1   123.131.179.188
1   221.205.192.210
1   59.25.194.72
1   ppp85-141-130-24.pppoe.mtu-net.ru
1   117.104.245.29
Relay access denied (total: 4)
1   parestaurant.org
1   92.80.72.159
1   92.113.111.226
1   122.198.44.5

message reject warning detail: none

message hold detail: none

message discard detail: none

smtp delivery failures
———————-
connection refused (total: 3)
3   hjeigb.info

Warnings
——–
smtpd (total: 5)
1   99.178.220.164: hostname adsl-99-178-220-164.dsl.irvnca.sbcglob…
1   92.113.111.226: hostname 226-111-113-92.pool.ukrtel.net verific…
1   117.104.245.29: hostname 29.245.104.117.ids.service.eastern-tel…
1   Unable to look up MX host for pure1-mail.net: Host not found
1   Unable to look up MX host for amota8.co.il: Host not found

Fatal Errors: none

Panics: none

Master daemon messages: none

Per-Hour Traffic Summary
time          received  delivered   deferred    bounced     rejected
——————————————————————–
0000-0100           2          1          1          1          1
0100-0200           3          3          1          0          3
0200-0300           3          2          1          0          2
0300-0400           4          4          1          0          0
0400-0500           3          2          0          0          4
0500-0600           4          4          1          0          6
0600-0700           0          0          0          0          0
0700-0800           0          0          0          0          0
0800-0900           0          0          0          0          0
0900-1000           0          0          0          0          0
1000-1100           0          0          0          0          0
1100-1200           0          0          0          0          0
1200-1300           0          0          0          0          0
1300-1400           0          0          0          0          0
1400-1500           0          0          0          0          0
1500-1600           0          0          0          0          0
1600-1700           0          0          0          0          0
1700-1800           0          0          0          0          0
1800-1900           0          0          0          0          0
1900-2000           0          0          0          0          0
2000-2100           0          0          0          0          0
2100-2200           0          0          0          0          0
2200-2300           0          0          0          0          0
2300-2400           0          0          0          0          0

Host/Domain Summary: Message Delivery (top 5)
sent cnt  bytes   defers   avg dly max dly host/domain
——– ——-  ——-  ——- ——- ———–
3     8124        0     5.8 s    7.9 s  yahoo.com
7     4175        0     3.2 s    3.2 s  news.espacopublico.com.br

Host/Domain Summary: Messages Received (top 5)
msg cnt   bytes   host/domain
——– ——-  ———–
3     6481   gmail.com
2    14562   alicensing.com
2     3238   chasey.com

top 5 Senders by message count
——————————
3   from=<>
2   c19remove@alicensing.com
2   ash22@chasey.com

top 5 Recipients by message count
———————————
1   noreply@news.espacopublico.com.br

top 5 Senders by message size
—————————–
17328   sophiew@ubi.com
14562   c19remove@alicensing.com
11353   from=<>
5238   noreply@news.espacopublico.com.br

top 5 Recipients by message size
——————————–

4175   noreply@news.espacopublico.com.br

Postfix Statistics

Posted by Filed Under Statistics with 1 Comment

It is important to understand what your mail server is doing.  Statistics can be warnings about problems that are developing but also provide you with information that will help you make decisions about hardware and management.  One program that provides some insight into the hard work your mail server is doing is pflogsumm.

pflogsumm
pflogsumm.pl is designed to provide an over-view of postfix activity and produce a log summary of the data that is in the system logs.

You can download the script from here:

http://jimsun.linxnet.com/postfix_contrib.html

Pflogsumm requires the Date::Calc module.  You can download and install the Date::Calc module from CPAN.  It can be found at:

http://search.cpan.org/search?module=Date::Calc

Usage
When you want to run the program just invoke the command and provide log that it should evaluate.

./pflogsumm.pl /var/log/maillog

Example Results

Grand Totals
————
messages

144   received
159   delivered
0   forwarded
0   deferred
24   bounced
357   rejected (69%)
0   reject warnings
0   held
0   discarded (0%)

1219k  bytes received
1070k  bytes delivered
127   senders
108   sending hosts/domains
5   recipients
4   recipient hosts/domains

Per-Day Traffic Summary
date          received  delivered   deferred    bounced     rejected
——————————————————————–
Sep  7 2008        33         41          0          4        102
Sep  8 2008        70         76          0         10        202
Sep  9 2008        41         42          0         10         53

Per-Hour Traffic Daily Average
time          received  delivered   deferred    bounced     rejected
——————————————————————–
0000-0100           2          2          0          0          2
0100-0200           2          2          0          1          3
0200-0300           0          0          0          0          3
0300-0400           2          3          0          0          3
0400-0500           1          1          0          1          3
0500-0600           2          2          0          0          4
0600-0700           3          3          0          1          6
0700-0800           4          5          0          0          6
0800-0900           4          5          0          0          5
0900-1000           2          2          0          1          6
1000-1100           3          4          0          1          6
1100-1200           4          4          0          1          9
1200-1300           3          4          0          1          6
1300-1400           2          2          0          1          6
1400-1500           2          2          0          0          5
1500-1600           2          2          0          0          4
1600-1700           2          2          0          0          3
1700-1800           2          1          0          1          7
1800-1900           1          1          0          0          5
1900-2000           2          2          0          1          7
2000-2100           1          1          0          0          6
2100-2200           1          1          0          0          3
2200-2300           1          1          0          1          7
2300-2400           1          1          0          0          4

Host/Domain Summary: Message Delivery
sent cnt  bytes   defers   avg dly max dly host/domain
——– ——-  ——-  ——- ——- ———–
73   325657        0    11.9 s   57.0 s  somedomain.net
71      679k       0    11.1 s    1.3 m  gmail.com
14    72076        0     4.6 s   21.0 s  hotmail.com
1     2053        0     1.0 s    1.0 s  anotherdomain.com

Host/Domain Summary: Messages Received
msg cnt   bytes   host/domain
——– ——-  ———–
10    10580   mail.goo.ne.jp
8    21551   itextron.com
5    14784   paypal.com
3    12234   yeah.net
3     7758   ebay.com
3     6151   gmail.com
3     5754   yahoo.com
2    13567   yahoo.com.cn
2    10367   sales12.com
2     7080   hotmail.com
2     2607   waiuj.info
2     2470   gojen.info
2     1924   greatezleads.com
2     1767   infoseek.jp
1   170371   mail333.com

Senders by message count
————————
8   govclaimdpt1@itextron.com
3   aw-confirm@ebay.com
3   payment@paypal.com
2   cjyoujsd@yahoo.com.cn
2   noreply@greatezleads.com
2   service@paypal.com
2   kosher@sales12.com
2   kamishinmeiki@infoseek.jp
1   hulawrence@yahoo.com.ar

Recipients by message count
—————————
73   tom@somedomain.net
44   some_user@gmail.com
27   another_user@gmail.com
14   some_user@hotmail.com

Senders by message size
———————–
170371   avrsem@mail333.com
147883   KingCity@kingscity6.com
107654   Sharon@korinet2.com
68373   mefike@muzar6.com
59184   MAROM_TEUFA@marom2.com
45149   root@mast1.nettransactions.com
43467   test@relay05.reunion.com
39630   Dell_Small_Business@busenetwork.net
29044   sintera@mitug12.com
28028   return_0_17181861818_17179873120@tauckworlddiscovery.fbmta.com
26748   zt_mailer_17@zt40.broadcasttoemail.com
23793   newsbites@sans.org
23547   no_reply@gruppolife.net
21551   govclaimdpt1@itextron.com

Recipients by message size
————————–
402222   some_user@gmail.com
325657   some_user@somedomain.net
293876   another_user@gmail.com
72076   some_user@hotmail.com

message deferral detail: none
message bounce detail (by relay)
——————————–
somedomain.net.s6a1.psmtp.com[164.18.15.160]:25 (total: 11)
2   MAIL FROM:<kosher@sales12.com> (in reply to RCPT TO command)
1   <arc_sakurapv6@ss-td.yoshi-ss.info> (in reply to RCPT TO command)
1   MAIL FROM:<tatumi-2@doreamtf.info> (in reply to RCPT TO command)
1   MAIL FROM:<Sharon@korinet2.com> (in reply to RCPT TO command)
1   MAIL FROM:<KingCity@kingscity6.com> (in reply to RCPT TO command)
1   MAIL FROM:<sintera@mitug12.com> (in reply to RCPT TO command)
1   MAIL FROM:<fondorger@windson.info> (in reply to RCPT TO command)
1   MAIL FROM:<MAROM_TEUFA@marom2.com> (in reply to RCPT TO command)
1   MAIL FROM:<mefike@muzar6.com> (in reply to RCPT TO command)
1   <theophiluso@mackone.freeserve.co.uk> (in reply to RCPT TO comm…

none (total: 12)
9   Host not found
3   Host found but no data record of requested type

message reject detail
———————
RCPT
blocked using sbl-xbl.spamhaus.org (total: 234)
16   125.187.32.174
11   rr.com
9   verizon.net
5   qwest.net
4   66.199.236.122
4   charter.com

Helo command rejected: need fully-qualified hostname (total: 46)
3   117.104.245.36
2   219.130.135.222
2   24.139.224.228
2   117.104.245.37

Recipient address rejected: User unknown in virtual alias table (total: 32)
4   buhgalter@somedomain.com
4   office@somedomain.com
3   director@somedomain.com
2   buhg@somedomain.com
2   buhgalteria@somedomain.com
Recipient address rejected: User unknown in virtual mailbox table (total: 1)
1   contact@somedomain.com
Relay access denied (total: 39)
4   samsung.co.kr
3   hundred-acre-retrievers.com
3   ppp83-237-120-194.pppoe.mtu-net.ru
2   147.202.65.172
2   google.com
2   adsl-dyn213.78-98-40.t-com.sk
1   64.18.134.189
1   220.248.128.162

Sender address rejected: Bad Network (total: 5)
1   ghwjbotsecg@brandbox.com
1   mag@cybr01.com
1   sylvester@ciberaula.infase.es

message reject warning detail: none

message hold detail: none

message discard detail: none

smtp delivery failures: none

Warnings
——–
smtpd (total: 152)
4   66.199.236.122: hostname 66-199-236-122.reverse.ezzi.net verifi…
4   61.29.115.50: hostname dsl-61-29-115-50.request.com.au verifica…
4   Unable to look up MX host for yamabiko-do.net: Host not found
3   117.104.245.36: hostname 36.245.104.117.ids.service.eastern-tel…
3   Unable to look up MX host for pure1-mail.net: Host not found
3   Unable to look up MX host for treasure-boxes.jp: Host not found
1   Unable to look up MX host imail.interainc.com for Sender addres…

trivial-rewrite (total: 423)

virtual (total: 2)
1   database /etc/postfix/virtual_gid_map.db is older than source f…
1   database /etc/postfix/virtual_uid_map.db is older than source f…

Fatal Errors: none

Panics: none

Master daemon messages
———————-
1   reload configuration /etc/postfix