Bounce Processor Concepts
- Table of Contents
- Overview
- SimpleMH Requirements
- Raw Injection Requirements
- Events Triggered
- Types of Bounces (Hard/Soft/Other)
-
Bounce Codes
- 0 - Non-bounce
- 10 - Bad email address
- 20 - Temp failure - General
- 21 - Temp failure - DNS Failure
- 22 - Temp failure - Mailbox full
- 23 - Temp failure - Message size too large
- 29 - Temp failure - Unable to connect
- 30 - Bounce with no email address
- 40 - General bounce
- 50 - Mail block - General
- 51 - Mail block - Known Spammer
- 52 - Mail block - Spam detected
- 53 - Mail block - Attachment detected
- 54 - Mail block - Relay denied
- 55 - Mail block - SPF/DKIM/DMARC
- 59 - Mail block - Unable to connect
- 60 - Auto-reply
- 70 - Transient bounce
- 90 - Unsubscribe request
- 100 - Challenge response
- 110 - Over GreenArrow throttle limit
- 120 - Queue dumped
- 121 - Delivery paused then message expired
- 122 - Suppressed by System due to Subscriber Inactivity
- 123 - Blocked due to System policy enforcement
- 124 - Blocked due to daily throttling limit
- Repeat Bounce Counting System
- Viewing Data
- Synchronous and Asynchronous Bounce Messages
- Service and Logfile
- Bounce Processor Queue
Overview
This page describes GreenArrow’s bounce processor from a conceptual point of view. See the Bounce Processor Configuration page for configuration instructions.
The bounce processor is responsible for processing bounce messages received at Bounce Mailboxes. The bounces are parsed to determine the reason for the bounce, along with the recipient address, ListID, and the SendID of the message. Each processed bounce triggers a bounce_all
event.
Bounces are attributed to the email address that GreenArrow sent the email to. This means that if GreenArrow sends an email to AddressA, which forwards to AddressB, and AddressB generates a bounce message, GreenArrow will interpret the bounce as being for AddressA.
The bounce processor tracks the number of bounces to an email address, and the amount of time the email address has been bouncing. It combines this information with its configuration to determine if the email address is no longer valid. If the email address is no longer valid, then the bounce processor issues a bounce_bad_address
event so you can deactivate the email address in your database.
When bounce_all
and bounce_bad_address
events occur, the Event Notification System can notify you via an SQL Query, HTTP Post, or one of its other event delivery methods.
The bounce processor works hard to avoid deactivating valid addresses, as well as to make sure that all invalid addresses are identified.
Proper bounce processing is essential for deliverability, as a sender can be blocked for sending to a high percentage of invalid addresses.
SimpleMH Requirements
These settings are not required for SimpleMH. SimpleMH automatically sets the ListID and the Return-Path
based on the configuration of the Mail Class.
Raw Injection Requirements
If you are using Raw Injection, you must configure a couple of things in the messages you send.
Set the ListID
You must set the ListID to a correct and consistent value. The repeat bounce counting is done on a per-ListID basis, so the ListID must be consistent. The ListID is compared case-insensitively for repeat bounce counting.
The ListID is described on the GreenArrow Concepts, and the X-GreenArrow-ListID
header, which is used to set the ListID is described on the Raw Injection Method Headers page.
Set the Return-Path
The Return-Path
is the field in an email message that determines where bounces are sent. You must correctly set the Return-Path
on outgoing messages for the bounce processor to work.
The Return-Path
should be set to a valid Bounce Mailbox’s address.
Details of how to set the Return-Path
are in the Raw Injection Method Headers page.
Events Triggered
There are two event types triggered by the bounce processor:
-
bounce_all
-- Triggered for every bounce received. (Do not use this to deactivate email addresses in your database.) -
bounce_bad_address
-- Triggered when an address is detected as bad and should be deactivated in your database.
Types of Bounces (Hard/Soft/Other)
GreenArrow breaks bounce messages down into three different types:
Type | Description |
---|---|
Hard |
A hard bounce is where the ISP is explicitly saying that this email address is not valid. Typically this is a user does not exist error message. |
Soft |
A soft bounce is an error condition, which if it continues, the email address should be considered invalid. An example is a DNS Failure (code 21 ) bounce message. This can happen because the domain name no longer exists, or this could happen because the DNS registration expired and will be renewed tomorrow, or there was a temporary DNS lookup error. If the DNS failure messages persist, then we know the address is bad. |
Other |
An other bounce is an error condition where the delivery failed which does not mean that the email address is bad. Typically this is a spam blocking. Instead of removing the email addresses from your list, you would want to solve whatever caused the blocking and restore delivery to these addresses. |
About other
bounces: In the future, we may build a quarantine facility for email addresses that consistently fail with other
bounces. The reason is as follows: If an email address bounces with an other
bounce every time we send to it, then there is no point in continuing to send to it, which may just bother the ISP. To solve this problem, instead of removing the email address from the list, the email address would be placed in a quarantine that would prevent you from sending to that email address. When this happens, you would be notified of the quarantine, and after fixing the root delivery problem, you could easily release the email addresses from quarantine.
Bounce Codes
Bounce messages are categorized into different bounce codes. Each code has a default type (Hard
, Soft
or Other
).
Code | Description | Default type | Example SMTP Response |
---|---|---|---|
0 |
Non-bounce | Other |
|
10 |
Bad email address | Hard |
550 User Unknown |
20 |
Temp failure - General | Soft |
|
21 |
Temp failure - DNS Failure | Soft |
|
22 |
Temp failure - Mailbox full | Soft |
|
23 |
Temp failure - Message size too large | Other |
|
29 |
Temp failure - Unable to connect | Soft |
|
30 |
Bounce with no email address | Other |
|
40 |
General bounce | Other |
Sorry, I wasn't able to establish an SMTP connection. (#4.4.1)/I'm not going to try again; this message has been in the queue too long. |
50 |
Mail block - General | Other |
|
51 |
Mail block - Known Spammer | Other |
|
52 |
Mail block - Spam detected | Other |
|
53 |
Mail block - Attachment detected | Other |
|
54 |
Mail block - Relay denied | Other |
|
55 |
Mail block - SPF/DKIM/DMARC | Other |
|
59 |
Mail block - Unable to connect | Other |
|
60 |
Auto-reply | Other |
|
70 |
Transient bounce | Other |
|
90 |
Unsubscribe request | Hard |
|
100 |
Challenge response | Other |
|
110 |
Over GreenArrow throttle limit | Other |
|
120 |
Queue dumped | Other |
|
121 |
Delivery paused then message expired | Other |
|
122 |
Suppressed by System due to Subscriber Inactivity | Other |
|
123 |
Blocked due to System policy enforcement | Other |
|
124 |
Blocked due to daily throttling limit | Other |
|
200 |
Spam complaint | NA | NA |
0 - Non-bounce
This occurs when the bounce processor receives an email that it cannot classify
as a bounce or any other non-bounce code (e.g. 60 - Auto-reply
).
10 - Bad email address
This is a hard bounce where we were able to positively identify that delivery attempts to this email address should no longer be attempted. This usually indicates that the requested mailbox does not exist.
20 - Temp failure - General
This is a soft bounce which indicates a possibly temporary failure in delivery of the message. The email address itself may or may not be invalid. This is often the result of ambiguous messages such as a mailbox being “inactive” or “unavailable”.
21 - Temp failure - DNS Failure
This is a soft bounce caused by a failed DNS lookup. This can be caused by failures in DNS lookup either by GreenArrow itself or subsequent relay servers.
22 - Temp failure - Mailbox full
This is a soft bounce caused by the recipient’s mailbox or quota being full.
23 - Temp failure - Message size too large
This is an other bounce caused by the destination email server rejecting the message due to its size.
29 - Temp failure - Unable to connect
This is a soft bounce where we were unable to connect to the remote server. If there are many bounces with this code for a single domain, this usually means that the domain is not working or that the domain is blocking your mail server.
30 - Bounce with no email address
This is an other bounce caused by the bounce message itself containing no email address. Since our return paths include sufficient information to identify the recipient’s email address, GreenArrow no longer categorizes any bounces as a 30.
40 - General bounce
This is an other bounce that could not otherwise be categorized. GreenArrow positively identified that this is a bounce message, but was unable to determine a more specific category.
50 - Mail block - General
This is an other bounce caused by the remote server rejecting the message for reasons that are not otherwise covered by a more specific “Mail block” category.
51 - Mail block - Known Spammer
This is an other bounce caused by the remote server rejecting the message because it considers the sender to be a spammer. These bounces are not specific to the individual message, but rather the sending server itself. This can be caused by listing on various public or private block lists.
52 - Mail block - Spam detected
This is an other bounce caused by the remote server rejecting the individual message content as spam.
53 - Mail block - Attachment detected
This is an other bounce caused by the remote server rejecting the individual message due to its inclusion of one or more attachments.
54 - Mail block - Relay denied
This is an other bounce caused by the remote server refusing to serve as a relay for delivery of the message.
55 - Mail block - SPF/DKIM/DMARC
This is an other bounce caused by the remote server refusing delivery of the message due to SPF, DKIM, or DMARC policies. Resolving the specific complaint in the bounce message can often improve deliverability.
59 - Mail block - Unable to connect
This is an other bounce where we were unable to connect to the remote server (much
like code 29
), however, this is to a large domain name, so we are reasonably
confident that the domain name did not cease to exist, but that they are
blocking all connections from your IP address. Some large domains, when
blacklisting, will completely block your IP from connecting to their servers.
60 - Auto-reply
These are messages received by the bounce processor that were generated by an auto-reply system such as a vacation autoresponder. These are not actual bounces.
70 - Transient bounce
This is an other bounce caused by a temporary delay in message transmission. The remote server will usually continue trying to deliver the message for a limited period of time.
90 - Unsubscribe request
This is a hard bounce caused by a request to unsubscribe from the mailing list.
100 - Challenge response
This is an other bounce caused by the remote server issuing a challenge-response probe. GreenArrow does not respond to these challenges.
110 - Over GreenArrow throttle limit
This bounce happens when a message has been in the queue past the queue_lifetime limit, is scheduled for a delivery attempt, but that delivery attempt can not be made because it would go over the configured throttle settings for that IP or Relay Server (a connmaxout failure).
120 - Queue dumped
This is a bounce where GreenArrow was told to dump the messages in the queue. All the messages dumped from the queue will bounce with this code.
121 - Delivery paused then message expired
This is a bounce where GreenArrow was told to pause delivery of messages in a queue. After the maximum retry time, the messages are bounced, and they are bounced with this code.
122 - Suppressed by System due to Subscriber Inactivity
This is a bounce where GreenArrow suppressed sending to the recipient because they haven’t taken an action, such as clicking a link or loading images which signals engagement.
123 - Blocked due to System policy enforcement
This is a bounce where GreenArrow suppressed sending to the recipient because of an AUP violation or another policy issue.
124 - Blocked due to daily throttling limit
This is a bounce where GreenArrow suppressed sending this message because the sender has reached its daily throttling limit.
Repeat Bounce Counting System
With soft bounces, we want to remove the email address if the email address continues to soft bounce over a period of time. GreenArrow Engine’s repeat bounce counting system implements this.
You can configure the how many times an email address / ListID combination must soft bounce before it is marked as bad, and therefore removed from your list. You can also configure similar requirements for hard bounces. (See note below on setting this for hard bounces.)
The system is designed so that it does the right thing with the typical settings regardless of your mailing frequency -- if you are sending daily, weekly, monthly, or only occasionally.
Name | Description | Typical Setting |
---|---|---|
soft_req_count |
Required number of soft bounces that must happen sequentially before an address is marked as bad. If you want a single soft bounce to cause the address to be marked as bad, then set this to 1 . |
2 |
soft_req_days |
Minimum number of days between first soft bounce and last soft bounce, in order for an address to be marked as bad. | 10 |
soft_allow_missing |
Number of emails that may be sent without receiving a soft bounce, without interrupting a sequence of soft bounces. | 0 |
hard_req_count |
Same as soft_req_count except for hard bounces. |
1 |
hard_req_days |
Same as soft_req_days except for hard bounces. |
0 |
hard_allow_missing |
same as soft_allow_missing except for hard bounces. |
0 |
These requirements must be met for an email address to be marked as bad:
- The required number of soft or hard bounces must be received for the email address and ListID combination. (This is the
hard_req_count
andsoft_req_count
settings.) - All emails to that email address and ListID combination must have bounced, except that a configurable number may be allowed not to have bounced, (which is the
hard_allowed_missing
andsoft_allowed_missing
settings). - The first bounce and the last bounce must be at least a certain number of days apart. (This is the
hard_req_days
andsoft_req_days
settings.)
Rule #1 is required because we want to make sure that the email address has bounced a certain number of times before we mark it as bad.
Rule #2 is required because we want to make sure that the email address is consistently bouncing, not just occasionally bouncing. For example, if the only requirement was that the email address bounce 10
times, and if we sent daily emails but the email address bounced once per week, without this rule the address would get removed after 10
weeks (because it would have reached 10
bounces). With the hard_allow_missing
and soft_allow_missing
parameters, you can allow a few messages to be sent without producing bounces. This can be helpful if you require a large number of bounces because email is not perfect and sometimes messages get lost without producing bounces.
Rule #3 is required because we want to make sure that the email address is consistently bouncing over a long enough period to be sure it is bad. For example, if you send daily mailings, and if a user goes over quota for one week, you don’t want the 7
bounces generated to trigger a removal. By having soft_req_days
set to 14
, you would require that the error condition exists for at least two weeks.
Regardless of whether or not stateless event tracking is configured, repeat bounce counting only takes into account the bounces for the particular server on which they are processed.
Examples
With these settings:
Parameter | Value |
---|---|
soft_req_count |
3 |
soft_req_days |
7 |
soft_allow_missing |
1 |
Example #1
This bounce history would not cause the email to get removed because we have four bounces (which qualifies) but only four days between the first and last bounce (which does not qualify):
- Email sent Jan 1 – bounced
- Email sent Jan 3 – bounced
- Email sent Jan 4 – bounced
- Email sent Jan 5 – bounced
Example #2
This bounce history would not cause the email to get removed because we have only two bounces (which does not qualify) and eight days between the first and last bounce (which does qualify):
- Email sent Jan 1 – bounced
- Email sent Jan 9 – bounced
Example #3
This bounce history would cause the email to get removed because we have four bounces (which qualifies) and 8 days between the first and last bounce (which qualifies):
- Email sent Jan 1 – bounced
- Email sent Jan 3 – bounced
- Email sent Jan 6 – bounced
- Email sent Jan 9 – bounced
Example #4
This bounce history would cause the email to get removed, because we have four bounces (which qualifies) and 8 days between the first and last bounce (which qualifies), and only one missing bounce (the email sent on Jan 7) (which qualifies):
- Email sent Jan 1 – bounced
- Email sent Jan 3 – bounced
- Email sent Jan 6 – bounced
- Email sent Jan 7 – did not bounce
- Email sent Jan 9 – bounced
Example #5
This bounce history would not cause the email to get removed, because we have four bounces (which qualifies) and 8 days between the first and last bounce (which qualifies), but two missing bounces (the email sent on Jan 5 and Jan 7) (which does not qualify):
- Email sent Jan 1 – bounced
- Email sent Jan 3 – bounced
- Email sent Jan 5 – did not bounce
- Email sent Jan 6 – bounced
- Email sent Jan 7 – did not bounce
- Email sent Jan 9 – bounced
Considerations for Hard Bounces
GreenArrow allows you to set up repeat bounce counting for hard bounces, but we recommend being very careful with this.
The reason you may want to require more than one hard bounce is that very rarely an ISP will send a hard bounce for an email address that actually exists. Often this is a software error at the ISP. It sounds strange, but we have seen this happen.
The caution is that if you are starting to send to a list that has not had correct bounce processing done (perhaps imported from other software that was not processing bounces correctly), it’s imperative for deliverability that you get the hard bouncing addresses removed as quickly as possible, so you want to deactivate addresses after a single hard bounce.
If your list is already clean, and your hard bounce percentage is low, then to protect against this you can set hard_req_count=2
and hard_req_days=2
.
Viewing Data
You can view statistical information about bounces by looking at a Send’s Statistics in GreenArrow Engine’s web interface. For a particular send, click the Bounce stats
tab to view these stats.
Synchronous and Asynchronous Bounce Messages
There are two kinds of bounce messages, depending on when the failure happens:
Synchronous Bounce Messages
This is where a remote SMTP server rejects the message in the SMTP conversation with a 5xx
error code. (Or if the message is continually deferred, once it reaches the maximum retry time, GreenArrow Engine will consider this an SMTP failure.) Internally, GreenArrow creates a bounce message and sends this to the bounce processor. This bounce message is synchronous.
“Synchronous” means same time -- these are bounce messages were the failure happens at the same time as the SMTP conversation.
Asynchronous Bounce Messages
This is where a remote SMTP server accepts the message via SMTP, and then later determines that it can not deliver the email, and returns a bounce message back to the Return-Path
address of the message. This bounce message is an asynchronous bounce message.
This message will show accepted
via SMTP, but then later a bounce will be logged.
“Asynchronous” means NOT at the same time -- these are bounce messages were the failure happens NOT at the same time as the SMTP conversation.
Considerations
The GreenArrow bounce processor processes both synchronous and asynchronous bounce messages.
In the reporting, synchronous bounces contribute to (a) the SMTP failures in the GreenArrow stats (the SMTP Server Stats
tab, the TOTAL
row, and the Failure
column), and (b) the total number of bounces (the Bounce stats
tab, the Total Bounces
number. Asynchronous bounces only contribute to the total number of bounces (b). This is because asynchronous bounces don’t have a failure in the SMTP conversation.
Service and Logfile
Bounces and spam complaints are processed and logged by the same service. To monitor this service’s log file, execute the following command:
tail -F /var/hvmail/log/bounce-processor/current | tai64nlocal
The bounce processor is run from the /service/hvmail-bounce-processor
service.
To temporarily turn off the bounce processor, run:
svc -d /service/hvmail-bounce-processor
To turn it back on run:
svc -u /service/hvmail-bounce-processor
To check whether the service is on, run:
hvmail_init status | grep hvmail-bounce-processor
Bounce Processor Queue
As bounces occur, they get delivered to GreenArrow’s bounce processing queue.
Normally GreenArrow’s bounce processor can keep up with the rate at which bounces are queued. You can verify that it is by running the hvmail_status status
command, then looking at the “Bounce processor queue” section of the output. For example, the following output shows that there are 0 messages in the bounce processor’s queue:
Bounce processor queue: 0% used ( 0/ 3000) soft limit
0% used ( 0/ 5000) hard limit
The “hard” and “soft” limits referenced in the above output are described in the Bounce Processor Configuration page.