Queue Management

• Table of Contents
• How are email queues managed for efficient delivery?
• Are email messages queued to an IP Address?
• What queue settings are configurable?
• What tools are available to show visibility into queues?
• Are there features for prioritizing or throttling messages?
• Can messages be purged from the queue?

How are email queues managed for efficient delivery?

GreenArrow’s queues are typically managed based on the recipient domain (or MX) and the Sending IP.

Delivery throttling, backoff modes, and retry schedules (to name a few) all are managed on this level.

Certain other queue-management tools can also key off of other parameters.

For example, the domain pause feature pauses deliveries to a recipient domain from a particular sending domain (either RFC5321.MailFrom domain or RFC5322.From domain) for a configurable period of time, across all IP addresses, based on a regex match of a SMTP response. This can be used to automatically throttle down deliveries for customer domains that are hitting domain throttles at mailbox providers.

Are email messages queued to an IP Address?

Messages are not queued to an “IP Address”, but are queued in GreenArrow into one large pool of messages where each message has a text metadata field named “VirtualMTA” which, along with the recipient domain name (including MX rollups and wildcard rules), is used to determine the IP used for delivery. This determination is performed every time that an email message needs to be delivered, so updates to the delivery configuration (such as Routing Rules used to define an IP pool) are automatically applied. (More on the Mail Routing and Traffic Shaping page.)

This means that email messages don’t need to be re-queued to another IP when you change a Routing Rule that is used to define an IP pool. Rather, the updated Routing Rule immediately takes effect.

GreenArrow’s reporting that shows the email messages queued for later retry can aggregate based on the IP address that was last used for at attempted delivery of the email message or the VirtualMTA metadata field. This report can also be aggregated by individual recipient domain names or domains grouped together (through explicitly listing multiple domains, MX rollup rules, and wildcarding rules)

Many users think of an “IP Address” and “Recipient Domain” (including multiple domains that are grouped together, MX rollup rules, and wildcarding rules) as a “queue” since the delivery throttling is applied on that level. GreenArrow calls these “throttles” – they can be accessed through the Throttles API, each throttle has a unique ID that is reported in the delivery_attempt event as the throttleid parameter, and each throttle has a page in the UI.

What queue settings are configurable?

The following settings applicable to “queues”/”throttles” are all configurable:

• Retry period/intervals
• Message expiration
• Rerouting messages, such as from one VirtualMTA to another
• STARTTLS for delivery
• Delivery overrides
• Custom behavior triggered based on SMTP response messages and codes
• Any configuration directive that is configurable in the “domain (within ip_address)” scope.

What tools are available to show visibility into queues?

Both our Engine UI and our Engine API offer visibility into queues. There are also command line tools available.

The Throttles API and UI shows which queues are in backoff mode.

Are there features for prioritizing or throttling messages?

Email delivery can be throttled based off of max delivery attempts per hour and/or maximum concurrent connections:

Our online documentation contains an entire section on Throttling as well.

Can messages be purged from the queue?

There are several ways to purge messages from queues.

For example, you can use header match in the header_pattern directive to direct email to a VirtualMTA that dumps all email.

Here is example configuration for that:

header_pattern X-Header /foo/ set_virtualmta=dump_from_queue


ip_address dump_from_queue {
  smtp_source 127.0.0.1 localhost.localdomain
  delivery_override perm_failure "This message has been dumped from the queue"
}

Additionally, to dump email injected for a particular IP address, you could directly use the delivery_override directive:

ip customer_a {
  domain * {
    delivery_override perm_failure "This message has been dumped from the queue"
  }
}

Messages for a given send can also be dumped from the queue from the Engine UI