Enthusiasm never stops

Leave a comment

Force Exim to process outgoing queue quickly

There are times when a lot of messages queue up in Exim. For example, it could be due to an intermittent network problem.

I always struggled to force Exim to process the outgoing queue with lots of parallel connections to the remote SMTP servers. Note that my use-case is rather special where all mails are delivered to the same recipient domain.

There are suggestions to use "queue_run_max = 30" or "remote_max_parallel = 30" to increase the maximum parallel outgoing SMTP connections. When I execute "exim -qf" or "exim -Rf domain" to process the mail queue immediately, the parallel SMTP connections are still capped to just about 5.

Today I found a way to control the parallelism for SMTP deliveries when we want to process the queue manually (forced):

exiqgrep -ir example.com|xargs -P 30 -n 10 exim -M

This effectively launches 30 parallel SMTP connections and a queue with thousands of messages gets processed in a few minutes. If you want to process all messages regardless of their domain name, use only "exiqgrep -i". The command “exiqgrep” has other filtering capabilities to help you with the selection of messages to process.

Note that if the messages have queued up because the remote MX server was marked as unreachable, you will have to clear the “retry” database before you start to process the queue manually. Additionally, I also clear the database about messages waiting for remote SMTP hosts:

exim_tidydb -t 1s /var/spool/exim4 retry
exim_tidydb -t 1s /var/spool/exim4 wait-remote_smtp

The man page of “exim_tidydb” explains in detail what this command does.