Manage Mail In the Deferred Queue
February 18, 2009 Postfix Configuration
Deferred queue
If Postfix cannot deliver a message to a recipient it is placed in the deferred queue. The queue manager will scan the deferred queue to see it if can place mail back into the active queue. How often this scan occurs is determined by the queue_run_delay. Postfix will scan the incoming queue at the same time as the deferred queue just to make sure that one does not take all the resources and so each can continue to move messages.
The queue_run_delay is by default 300s or 300 seconds. Each time the deferred queue is scanned it will only reinject a portion of the messages.
If you have a very busy mail server you may see a large deferred queue. The first instinct is to flush the queue but that actually may be counter productive. The only reason you would flush the deferred queue is if you think that the messages now have a good chance of delivery. If they still do not have a good chance of delivery it will only slow down an already busy server.
The real question is, What is causing messages to be deferred? One of the major reasons that messages are deferred is that your server is going to place mail to “unknown recipients” into the deferred queue if they do not have a legitimate user to go to.
Here is the process to view and analyze why mail is deferred. The fist warning is that you see deferred mail in your messages logs. In the example there are 2 listed.
Postfix log summaries for Feb 17
Grand Totals
————
messages
101 received
106 delivered
0 forwarded
2 deferred (35 deferrals)
0 bounced
104 rejected (49%)
0 reject warnings
0 held
0 discarded (0%)
1263k bytes received
1331k bytes delivered
32 senders
23 sending hosts/domains
17 recipients
11 recipient hosts/domains
Once you know that mail is in the deferred queue you need to locate the message Ids so you can read the mail in the queue. Use the command postqueue to view mail Ids.
postqueue -p
-Queue ID- –Size– —-Arrival Time—- -Sender/Recipient——-
9DF7520804A 3726 Mon Feb 16 03:06:41 MAILER-DAEMON
(connect to hydra.udag.de[89.31.140.33]: Connection timed out)
www-data@hydra.udag.de
CC1D4208048 3786 Mon Feb 16 02:39:50 MAILER-DAEMON
(connect to bootes.caixa.gov.br[200.201.166.138]: Connection timed out)
servicos@caixa.gov.br
– 8 Kbytes in 2 Requests.
Once you have the message ID you can use postcat to open the message in the queue. The first line shows that it is one of the deferred messages. As you view the contents of the email you can see the reason it is deferred, in this example someone is trying to send mail as if it is from the real user, thus this is SPAM and has been detected.
postcat -q 9DF7520804A
*** ENVELOPE RECORDS deferred/9/9DF7520804A ***
message_size: 3726 589 1 0
message_arrival_time: Mon Feb 16 03:06:41 2009
create_time: Mon Feb 16 03:06:41 2009
named_attribute: rewrite_context=local
named_attribute: envelope_id=AM..20090216T110641Z@ns.example.org
sender:
named_attribute: log_client_name=ns.example.org
named_attribute: log_client_address=127.0.0.1
named_attribute: log_message_origin=ns.example.org[127.0.0.1]
named_attribute: log_helo_name=localhost
named_attribute: log_protocol_name=ESMTP
named_attribute: client_name=ns.example.org
named_attribute: reverse_client_name=ns.example.org
named_attribute: client_address=127.0.0.1
named_attribute: helo_name=localhost
named_attribute: client_address_type=2
named_attribute: dsn_orig_rcpt=rfc822;www-data@hydra.udag.de
original_recipient: www-data@hydra.udag.de
recipient: www-data@hydra.udag.de
*** MESSAGE CONTENTS deferred/9/9DF7520804A ***
Received: from localhost (ns.example.org [127.0.0.1])
by ns.example.org (Postfix) with ESMTP id 9DF7520804A
for <www-data@hydra.udag.de>; Mon, 16 Feb 2009 03:06:41 -0800 (PST)
Content-Type: multipart/report; report-type=delivery-status;
boundary=”———-=_1234782401-3999-0″
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Subject: Considered UNSOLICITED BULK EMAIL, apparently from you
In-Reply-To: <9d798f4d2f18e5879bdfe20b2504d376@www.team-koeln.de>
Message-ID: <SSlkPbEmzalURU@ns.example.org>
From: “Content-filter at ns.example.org” <postmaster@example.org>
To: <www-data@hydra.udag.de>
Date: Mon, 16 Feb 2009 03:06:34 -0800 (PST)
This is a multi-part message in MIME format…
————=_1234782401-3999-0
Content-Type: text/plain; charset=”iso-8859-1″
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
A message from <www-data@hydra.udag.de> to:
-> user@gmail.com
was considered unsolicited bulk e-mail (UBE).
Our internal reference code for your message is 03999-12/lkPbEmzalURU
The message carried your return address, so it was either a genuine mail
from you, or a sender address was faked and your e-mail address abused
by third party, in which case we apologize for undesired notification.
We do try to minimize backscatter for more prominent cases of UBE and
for infected mail, but for less obvious cases of UBE some balance
between losing genuine mail and sending undesired backscatter is sought,
and there can be some collateral damage on both sides.
First upstream SMTP client IP address: [89.31.140.33]
According to a ‘Received:’ trace, the message originated at: [89.31.140.33],
Return-Path: <www-data@hydra.udag.de>
From: Caixa Economica Federal <servicos@caixa.gov.br>
Message-ID: <9d798f4d2f18e5879bdfe20b2504d376@www.team-koeln.de>
Subject: Atualizacao Caixa
Delivery of the email was stopped!
————=_1234782401-3999-0
Content-Type: message/delivery-status; name=”dsn_status”
Content-Disposition: inline; filename=”dsn_status”
Content-Transfer-Encoding: 7bit
Content-Description: Delivery error report
Reporting-MTA: dns; ns.example.org
Received-From-MTA: smtp; ns.example.org ([127.0.0.1])
Arrival-Date: Mon, 16 Feb 2009 03:06:34 -0800 (PST)
Original-Recipient: rfc822;fred@example.com
Final-Recipient: rfc822;user@gmail.com
Action: failed
Status: 5.7.0
Diagnostic-Code: smtp; 554 5.7.0 Reject, id=03999-12 – SPAM
Last-Attempt-Date: Mon, 16 Feb 2009 03:06:34 -0800 (PST)
Final-Log-ID: 03999-12/lkPbEmzalURU
————=_1234782401-3999-0
Content-Type: text/rfc822-headers; name=”header”
Content-Disposition: inline; filename=”header”
Content-Transfer-Encoding: 7bit
Content-Description: Message header section
Return-Path: <www-data@hydra.udag.de>
Received: from hydra.udag.de (hydra.udag.de [89.31.140.33])
by ns.example.org (Postfix) with ESMTP id 4CA89208029
for <fred@example.com>; Mon, 16 Feb 2009 03:06:33 -0800 (PST)
Received: by hydra.udag.de (Postfix, from userid 33)
id 32D3E45C463; Mon, 16 Feb 2009 12:21:10 +0100 (CET)
To: fred@example.com
Subject: Atualizacao Caixa
Date: Mon, 16 Feb 2009 12:21:07 +0100
From: Caixa Economica Federal <servicos@caixa.gov.br>
Reply-to: Caixa Economica Federal <servicos@caixa.gov.br>
Message-ID: <9d798f4d2f18e5879bdfe20b2504d376@www.team-koeln.de>
X-Priority: 3
X-Mailer: Microsoft Outlook Express 6.00.2800.1409
X-MSMail-Priority: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V6.10.2800.1409.518512323.rg.sm31
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/html; charset=”iso-8859-1″
————=_1234782401-3999-0–
*** HEADER EXTRACTED deferred/9/9DF7520804A ***
*** MESSAGE FILE END deferred/9/9DF7520804A ***
Comments (2)
Hi,
Interesting post, made me come up with these questions:
1) I see you have a nice “Postfix log summaries for Feb 17″, is this generated manually or do you have a script for this? Can we have it?
I know of tools to analyze the current postfix queues (qshape and pfqueue), but I haven’t used any tools yet to gather historical stats.
2) You say:
“One of the major reasons that messages are deferred is that your server is going to place mail to “unknown recipients” into the deferred queue if they do not have a legitimate user to go to.”
Maybe I’m wrong, but if there isn’t a legitimate user to deliver the mail to, wouldn’t postfix bounce the mail instead of putting it into the deferred queue? I think the deferred queue is only for mail which has permanent problems (eg because of greylisting), while a permanent problem causes a bounce.
Regards,
Stijn
In the example you can see that the mail is in the deferred queue because it was sent as the user, not necessarily to the right usr;
“The message carried your return address, so it was either a genuine mail
from you, or a sender address was faked and your e-mail address abused
by third party, in which case we apologize for undesired notification.”
Postfix is recognizing that the origin of the mail was an attempt to act as if the mail came from the user it was intended for.