5.1. Queue management
These sub-commands operate on the queue (which contains queued email messages) in various ways. Please see the queue section for more information about what the queue is, and how it works. Because the metadata for all messages in queue is loaded into memory, the MTA can quickly summarize and modify large number of messages. The list, groupby and update sub-commands share the same condition arguments, which are used to filter the result:
$ halonctl queue list -h
...
Conditions:
--transportid Filter on transport ID (repeatable) <string>
--jobid Filter on job ID (repeatable) <string>
--senderdomain Filter on sender domain (repeatable) <domain>
--recipientdomain Filter on recipient domain (repeatable) <domain>
--sender Filter on sender (repeatable) <address>
--recipient Filter on recipient (repeatable) <address>
--id Filter on ID (repeatable) <id>
--state Filter on state (repeatable) <state>
--retrycount Filter on retry count (operators, repeatable) <number>
--retrydelay Filter on next retry (operators, repeatable) <seconds>
--age Filter on age (operators, repeatable) <seconds>
--ts Filter on received time (operators, repeatable) <iso8601>
--metadata Filter on metadata <key=value>
Available states are:
queue: ACTIVE, DEFER
working: SENDING, UNKNOWN
freeze: HOLD, UPDATE
Number and date format:
operators: >, >=, <, <= and = (eg ">2020-01-01T01:02:03")
range: .. (eg "10..20")
...
5.1.1. List messages
This command shows information about messages in queue, filtered by the conditions described above. There are a few pre-defined views for showing the defer queue, active queue and open connections with relevant table fields pre-selected.
$ halonctl queue list --age '>3600' --state DEFER -F id -F retrycount -F retrydelay -F recipient
┌────────────────────────────────────────┬────────────┬────────────┬─────────────────┐
│ id │ retrycount │ retrydelay │ recipient │
├────────────────────────────────────────┼────────────┼────────────┼─────────────────┤
│ c6321f4c-1671-11ea-885e-005056914940:1 │ 20 │ 243 │ [email protected] │
│ f640a91a-1671-11ea-b6ea-005056914940:1 │ 21 │ 70 │ [email protected] │
...
5.1.2. Export messages
This command exports a messages in queue in “message/rfc822” format. If only a messageid (and no :queueid is provided) the original message (as received) will be exported.
$ halonctl queue export c6321f4c-1671-11ea-885e-005056914940
...
$ halonctl queue export c6321f4c-1671-11ea-885e-005056914940:1
...
5.1.3. Take actions on messages
The queue update command can take various actions on messages,
filtered by the conditions described above.
This command can perform the updates synchronously or asynchronously, see the --background and --progress options.
--deleteremove messages, without sending bounces
--reasonset the DSN diagnosticcode when deleting the message (“Message could not be delivered in the allotted time frame”)
--bounceremove messages, and generate bounce messages to the sender
--dsn-statusset the DSN status when bouncing the message (5.4.7)
--dsn-diagnosticcodeset the DSN diagnosticcode when bouncing the message (“Message could not be delivered in the allotted time frame”)
--activemove messages (that are deferred or on hold) to the active queue for imminent delivery--defermove messages (that are in active queue or on hold) to the defer queue for the specified number of seconds (number)
--retrytsjitterapply an optional jitter so that messages are spread evenly between the defer time and an additional jitter in seconds
--holdput queued messages on hold--updatetransportidchanges the messages’ transport ID to the one specified by the argument (string)--updatemetadatamodify or add auxiliary metadata to the messages as specified by the argument (in format field=value)--updateprioritychange the priority of the message (0 - normal, 1 - high, number)
$ halonctl queue update --age '>3600' --state DEFER --state ACTIVE --bounce
230 messages affected
$ halonctl queue update --transportid t1 --state DEFER --updatetransportid t2
27854 messages affected
5.1.3.1. Actions and working states
Messages in a working state
(delivery attempts, script execution, DNS resolution, etc)
cannot be directly modified, and they will be reported as unaffected by the command.
Sometimes it is important to perform a task on all matching messages, even those that are in a working state.
For example, you might want to delete every possible queued message from a certain sender domain.
To do this, re-run the command with the --freeze flag.
This will capture those messages once they are no longer in a working state
and put them in the so-called UPDATE state
(given that they still in in queue;
messages that have been delivered or deleted can not be captured).
No messages will be reported as unaffected;
they will instead be reported as freezing.
You should then monitor the queue.freeze.update.pending process counter
and wait for it to reach 0.
When it does, every messages that could not be immediately modified (but which is still in queue)
will be in the UPDATE state, ready for you to modify them.
$ halonctl queue update --delete --senderdomain example.com --all-states
100 messages affected
30 messages unaffected
$ halonctl queue update --delete --senderdomain example.com --all-states --freeze
4 messages affected
26 messages freezing
$ halonctl process-stats | grep queue.freeze.update
queue.freeze.update.size: 6
queue.freeze.update.pending: 20
$ halonctl process-stats | grep queue.freeze.update
queue.freeze.update.size: 22
queue.freeze.update.pending: 0
$ halonctl queue update --delete --senderdomain example.com --state UPDATE
22 messages affected
5.1.4. Show messages distribution
This command shows distribution of messages in queue, filtered by the conditions described above. There are a few pre-defined views for showing the distribution by for example age.
$ halonctl queue groupby --view state
┌─────────────┬─────────────────┬───────┬────────┬───────┬──────┬─────────┬─────────┐
│ transportid │ recipientdomain │ total │ ACTIVE │ DEFER │ HOLD │ UNKNOWN │ SENDING │
├─────────────┼─────────────────┼───────┼────────┼───────┼──────┼─────────┼─────────┤
│ inbound │ example.com │ 19 │ 0 │ 17 │ 1 │ 0 │ 1 │
...
5.1.5. Active queue policies
These sub-commands can add, list and delete dynamic
active queue policies that overrides
the ones in the environment.policyconf configuration,
as well as reload that configuration file.
$ halonctl queue policy add -F remotemx --remotemx *.example.com --rate 10/600 --ttl 3600
30be002e-7436-11eb-ae42-5254004d77d3
$ halonctl queue policy list
┌──────────────────────────────────────┬──────────┬─────────────┬─────────┬──────────┬────────────────┬─────────────────┬───────┬──────────┬─────────────┬────────┬─────────┬─────┐
│ id │ fields │ transportid │ localip │ remoteip │ remotemx │ recipientdomain │ jobid │ grouping │ concurrency │ rate │ ttl │ tag │
├──────────────────────────────────────┼──────────┼─────────────┼─────────┼──────────┼────────────────┼─────────────────┼───────┼──────────┼─────────────┼────────┼─────────┼─────┤
│ 30be002e-7436-11eb-ae42-5254004d77d3 │ remotemx │ │ │ │ *.example.com │ │ │ │ │ 10/600 │ 3592.44 │ │
└──────────────────────────────────────┴──────────┴─────────────┴─────────┴──────────┴────────────────┴─────────────────┴───────┴──────────┴─────────────┴────────┴─────────┴─────╯
$ halonctl queue policy delete 30be002e-7436-11eb-ae42-5254004d77d3
It is also possible to clear or partially refill active queue policy rates, for a specific policy rate “bucket” (matched exactly by the optional condition arguments). If you run it without specifying a bucket, it will clear all rate buckets, created by any policy, and return the number of buckets it cleared/refilled.
$ halonctl queue policy rate refill --localip 198.51.100.4 --remotemx mx1.example.org
1
5.1.6. Active queue suspensions
These sub-commands can add, list and delete dynamic
active queue suspension that overrides
the ones in the environment.suspendconf configuration,
as well as reload that configuration file.
$ halonctl queue suspend add --remotemx *.example.com --ttl 60
803c41d5-4a8e-424a-97e9-f8db55f69179
$ halonctl queue suspend list
┌──────────────────────────────────────┬─────────┬─────────────┬─────────┬──────────┬───────────────┬─────────────────┬───────┬──────────┬─────────┬─────┐
│ id │ source │ transportid │ localip │ remoteip │ remotemx │ recipientdomain │ jobid │ grouping │ ttl │ tag │
├──────────────────────────────────────┼─────────┼─────────────┼─────────┼──────────┼───────────────┼─────────────────┼───────┼──────────┼─────────┼─────┤
│ 803c41d5-4a8e-424a-97e9-f8db55f69179 │ dynamic │ │ │ │ *.example.com │ │ │ │ 57.7753 │ │
└──────────────────────────────────────┴─────────┴─────────────┴─────────┴──────────┴───────────────┴─────────────────┴───────┴──────────┴─────────┴─────╯
$ halonctl queue suspend delete 803c41d5-4a8e-424a-97e9-f8db55f69179
5.1.7. Active queue delivery settings
This sub-command can reload the
active queue delivery settings configuration file
environment.deliveryconf.
$ halonctl queue delivery reload
5.1.8. Moving the queue files
If you need to move the queued message files (for example from one MTA to another)
it is important to first “unload” them. If you want to move the entire queue, the easiest way is to simply
shut down the server process and use the hqfmove tool. To move specific messages, it is recommended to use the
queue unload command, filtered by the conditions described above.
--partialmatch a queued message, even if only one of its recipients matches--freezefreeze messages that are in a working state
$ halonctl queue unload --transportid t1
/storage/mail/spool/13/13ef04db-1e06-11ea-927e-0050569a4c9c.hqf unloaded
/storage/mail/spool/1b/1b5a5e80-1da2-11ea-927e-0050569a4c9c.hqf unloaded
...
$ halonctl queue unload
No messages unloaded
5.1.9. Trace messages
This command allows you to get debug or trace information regarding message deliveries in queue,
filtered by the queue pickup fields available (eg. --remotemx '*.example.com') or a specific transaction id.
Additional options are available to control the pooling (--connect) and PIPELINING (--no-pipelining) behavior.
$ halonctl queue trace
2023-03-28 21:59:23.076 Begin trace of message 18096a8d-cba3-11ed-b464-930f4c7d94ae:1
2023-03-28 21:59:23.076 Connecting from [1.2.3.4] to [5.6.7.8]:25 (mx.example.com) (DNSSEC)
2023-03-28 21:59:23.079 Connected
2023-03-28 21:59:23.083 [SMTP] < 220 mx.example.com ESMTP
2023-03-28 21:59:23.083 [SMTP] > EHLO mx.example.org
...
5.1.10. Loading queue files
If you need to manually import a queue file into a running MTA instance you can use the queue load command. The “load” command will copy the .hqf specified to the spool folder leaving the original file in place. The message ID (in the HQF file’s ID field) has to be an unique and will be used as the target file name (regardless of the imported source file name). If the HQF file’s ID field is an all zero UUID (00000000-0000…) then a new UUID will be generated.
--hqfpath to the queue file
$ halonctl queue load --hqf 13ef04db-1e06-11ea-927e-0050569a4c9c.hqf
Message loaded