1.2. Running configuration

The SMTP server (smtpd) handles all email going in and out, including the queue subsystem. As described in the SMTP server script manual it is primaily configured using hooks on the SMTP command phases; HELO, MAIL FROM, RCPT TO, etc. Similarily, the queue can be configured using pre- and post-delivery hooks as described in the queue script manual. Virtual queues are defined by the active queue policy (using concurrency and rate).

This is the reloadable part of the configuration, which contains the bulk of the configuration data. The default start configuration’s environment.appconf directive loads it from /etc/halon/smtpd-app.yaml. It is described by, and can be validated with, the smtpd-app.schema.json.

It can be softly reloaded and deployed in a blue-green fashion using per-connection conditions from the integrated web admin, command line interface or control sockets.

1.2.1. Script directives

The script is normally edited as individual files using the integrated package’s IDE, the Visual Studio Code plugin or simply a text editor. Those are then checked and “packed” into an actual configuration file by the halonconfig script. For example, two script files called src/hooks/rcptto/smtplookup.hsl and src/hooks/queue/predelivery.hsl will result in:

scripting:
  hooks:
    rcptto:
      - id: smtplookup
        data: |
          ...
    predelivery: |
      ...

and the “smtplookup” RCPT TO script can then be hooked into a server with:

servers:
  - id: inbound
    phases:
      rcptto:
        hook: smtplookup

The default configuration in /opt/halon/share/examples contains the folder structure for all the different hook types.

Since the SMTP server can have multiple virtual servers[] (one inbound on port 25 and one outbound on port 587 for example), the script hooks are designed so that there can be multiple scripts of a given hook type which are then mapped to servers. There is however just one set of pre- and post-delivery scripts controlling the queue.

scripting.hooks.connect[]

An array of connect scripts. Attached to a virtual server via servers[].phases.connect.hook.

scripting.hooks.helo[]

An array of HELO/EHLO scripts. Attached to a virtual server via servers[].phases.helo.hook.

scripting.hooks.auth[]

An array of AUTH scripts. Attached to a virtual server via servers[].phases.auth.hook.

scripting.hooks.mailfrom[]

An array of MAIL FROM scripts. Attached to a virtual server via servers[].phases.mailfrom.hook.

scripting.hooks.rcptto[]

An array of RCPT TO scripts. Attached to a virtual server via servers[].phases.rcptto.hook.

scripting.hooks.eod[]

An array of end-of-DATA scripts. Attached to a virtual server via servers[].phases.eod.hook.

scripting.hooks.proxy[]

An array of proxy scripts. Attached to a virtual server via servers[].phases.proxy.hook.

scripting.hooks.predelivery

The pre-delivery script executed as the first step of loading an email into the active queue.

scripting.hooks.postdelivery

The post-delivery script executed as the last step after a delivery attempt.

scripting.files[]

An array of virtual files used by the script hooks. It is most commonly used for imported script modules, include files, CSV lists and plain text files.

1.2.2. Server directives

The virtual servers first need to be defined in the startup configuration, as they specify which port(s) and address(es) to listen to (which cannot be softly reloaded). All other properties are then defined in this reloadable configuration.

servers[]

An array of virtual servers, which for example define which script hooks should be used. They are connected to the directives of the startup configuration via their IDs, as per the example below:

smtpd-app.yaml
servers:
  - id: default
    transport: mailserver
    phases:
      rcptto:
        hook: smtplookup
      eod:
        hook: inbound
    concurrency:
      total: 10000
      ip: 10
  - id: relay
    transport: mx
    phases:
      rcptto:
        hook: relaytrusted
      eod:
        hook: outbound
smtpd.yaml
servers:
  - id: default
    listeners:
      - port: 25
        backlog: 2048
  - id: relay
    listeners:
      - port: 587
        address: 192.168.0.100
servers[].concurrency.ip

The maximum total number of SMTP connections that the server accepts from a single IP address. The default is unlimited.

servers[].concurrency.total

The maximum total number of SMTP connections that the server accepts. Halon uses an asynchronous model, and can therefore support a much larger number compared to servers that use a process- or thread-based model. Make sure that the number of files resource limit environment.rlimit.nofile is a 2-3 times larger than this number. For 20000 connections, something like 50000 open files are needed. The default is 1000.

smtpd-app.yaml
servers:
  - id: default
    concurrency:
      total: 20000
smtpd.yaml
environment:
  rlimit:
    nofile: 50000
servers[].phases.connect.hook

Which scripting.hooks.connect[] to run on connect. The default is no script; to accept the connection.

servers[].phases.connect.remoteptr

Should a forward-confirmed reverse DNS lookup be performed on the connecting IP, for use in script and trace information (Received header). The default is false.

servers[].phases.helo.hook

Which scripting.hooks.helo[] to run on HELO/EHLO. The default is no script, to send a standard response.

servers[].phases.helo.required

Should the client be required to send HELO/EHLO before proceeding. The default is false.

servers[].phases.auth.hook

Which scripting.hooks.auth[] to run on AUTH. The default is no script, which disables AUTH.

servers[].phases.auth.mechanisms

Which SASL mechanisms to announce. The default is the built-in types LOGIN and PLAIN. If additional mechanisms are announced, they need to be implemented by the scripting.hooks.auth[] script.

servers[].phases.auth.tlsrequired

Should TLS be required for AUTH. The default is false.

servers[].phases.mailfrom.hook

Which scripting.hooks.mailfrom[] to run on MAIL FROM. The default is no script; to accept the sender.

servers[].phases.mailfrom.unqualified

If unqualified addresses (local part only) should be accepted. The default is false.

servers[].phases.rcptto.hook

Which scripting.hooks.rcptto[] to run on RCPT TO. The default is no script; to accept the recipient.

servers[].phases.rcptto.unqualified

If unqualified addresses (local part only) should be accepted. The default is false.

servers[].phases.data.maxsize

How large DATA response to accept. The default is 10 485 760 bytes.

servers[].phases.eod.hook

Which scripting.hooks.eod[] to run on end-of-DATA. The default is no script; to queue the email for all recipients.

servers[].phases.proxy.patterns[]

An array of SMTP command “patterns” before which to run the servers[].phases.proxy.hook script. The SMTP commands are matched up to the length of the pattern, case-insensitive. For example, specifying just one letter (such as “q”) will run the proxy script before all SMTP commands starting with that letter (such as QUIT). The default is to run the proxy script before all commands.

servers[].phases.proxy.hook

Which scripting.hooks.proxy[] to run on before commands matched by servers[].phases.proxy.patterns[]. The default is no script.

servers[].hostname

The hostname shown in the banner and HELO/EHLO. The default is to use the system hostname. This can be overridden by the scripting.hooks.connect[] and scripting.hooks.helo[] scripts.

servers[].extensions.smtputf8

Enable support for the SMTPUTF8 extension. The default is false.

servers[].extensions.xclient

An array of IP addresses to allow XCLIENT from, or true to allow from anyone. The default is false; not from anyone.

servers[].logging.protocol

Enable SMTP protocol logging (more verbose). The default is false.

servers[].timeout.idle

Number of seconds of inactivity before disconnecting a client. The default is 300.

Note

Halon uses an asynchronous model, and can support a much larger number of connected clients compared to servers that use a process- or thread-based model. Therefore, there is usually no reason to lower this setting.

servers[].tls.certs.cert

Which pki.private[] to use for TLS. The default is no TLS.

servers[].tls.certs.sni

An array with pki.private[] for use with SNI. The CN/SANs of the certificate will be used for matching. An optional list of subject names to use instead of the one(s) in the certificate can be provided. The default is no SNI.

servers:
  - id: default
    tls:
      certs:
        cert: defaultpki
        sni:
          - cert: otherpki
          - cert: yetanotherpki
            subjects:
              - "test2.example.org"
servers[].tls.protocols

Which TLS protocols to support. The default is !SSLv2,!SSLv3.

servers[].tls.ciphers

Which TLS protocols to support. The default is aNULL:-aNULL:HIGH:MEDIUM:+RC4:@STRENGTH.

servers[].tls.clientcert

An array of IP addresses to request client certificates from, or true to request from anyone. The default is false; not from anyone.

servers[].tls.implicit

An array of startup configuration servers[].listeners[] IDs to enable implicit TLS for, or true enable implicit TLS for all listeners on this virtual server. The default is false; no implicit TLS.

To enable implcit TLS on port 465, but not 587, you need reference the listener on port 465 by its ID idof465:

smtpd.yaml
  - id: relay
    listeners:
      - port: 465
        id: idof465
      - port: 587
smtpd-app.yaml
servers:
  - id: relay
    tls:
      implicit:
        - idof465
servers[].transport

The default transportgroups[].transports[] to queue the email on. This can be overridden by the scripting.hooks.eod[] script.

1.2.3. Transport directives

In Halon, every email is queued with a text label called “transport”, which normally exists in the configuration as a definition for how the email should be delivered. The parameters defined by the configured transport can be overridden in the pre- and post-delivery script, as in the addresses[] example.

transportgroups[]

The transport groups are simply a logical grouping of transports in the configuration, and transports inherit parameters from the group. In addition to all the transport directives listed below, transport groups have an id, and an array of transports.

transportgroups[].transports[]

Each of the transports have an id in addition to all the transport directives. The example below have to “outbound” MX transports; one with and other without DANE encryption. Then there are two other transports called “smarthost1” and “inbound” in a group called “default”.

transportgroups:
- id: mxes
  retry:
    count: 30
    intervals:
      - interval: 60
      - interval: 900
      - interval: 3600
        notify: true
      - interval: 7200
      - interval: 10800
  dsn:
    transport: mx
  connection:
    sourceip:
      ipv4:
        - "out1"
        - "out2"
      ipv6:
        - "out1v6"
        - "out2v6"
  transports:
    - id: mx
      session:
          tls:
            mode: optional
    - id: mxdane
      session:
          tls:
            mode: dane
- id: default
  transports:
    - id: smarthost1
      connection:
        server: "smtp-out.example.org"
        port: 587
        sourceip:
          ipv6: false
      session:
        authentication:
          username: foo
          password: bar
    - id: inbound
      connection:
        server: "dovecot.example.com"
        port: 24
      session:
        protocol: lmtp

The transport directives listed below are valid for both transport groups and transports.

transportgroups[].connection.server

By default, an MX lookup is performed to determine the next hop destination. By setting the server directive, you can specify an IPv4, IPv6 or hostname as destination. This is normally the case for “inbound” traffic to a mailbox server (maybe over LMTP) or “outbound” delivery via a so-called smarthost.

transportgroups[].connection.port

The TCP port to use for the SMTP/LMTP connection. The default is 25.

transportgroups[].connection.sourceip.ipv4

An array with one or more IPv4 addresses[] IDs to use as local IP(s), or false to disable IPv4.

transportgroups[].connection.sourceip.ipv6

An array with one or more IPv6 addresses[] IDs to use as local IP(s), or false to disable IPv6.

transportgroups[].session.protocol

Only needed if lmtp is to be used. The default is smtp.

transportgroups[].session.hostname

The HELO name to use. The default is to use the hostname of the addresses[] chosen as source IP, and to fall back to the system hostname.

transportgroups[].session.tls.mode

How to handle TLS for the connection. The default is not to use TLS (plain text). To do custom TLS verification per destination, use the pre-delivery script’s tls_X parameters. The script library contains an MTA-STS implementation.

  • optional: Use opportunistic, unverified TLS (fall back to plain text).
  • require: Require TLS, but don’t verify the peer.
  • dane: Verifiy the peer using DANE (DNSSEC). The most secure option.
  • dane_require: Require DANE. Only makes sense for specific destinations.
transportgroups[].session.authentication.username

The username to use for password-based authentication (SASL). Normally used for sending via smarthosts. The default is to not authenticate.

transportgroups[].session.authentication.password

See transportgroups[].session.authentication.username.

transportgroups[].retry.count

The number of times to attempt re-delivery of an email via the defer queue. Determines the time in queue together with the transportgroups[].retry.intervals[].

transportgroups[].retry.intervals[]

An array of intervals (in seconds) for how long the email should stay in the defer queue before retry. If there are fewer intevals than transportgroups[].retry.count, the last interval will be used for consecutive defers. As with most other transport parameters, this behvariour can be overridden via pre- and post-delivery script. By setting notify to true on an interval, a delayed notification will be delivered to the sender at this time.

transportgroups:
- id: mxes
  retry:
    count: 30
    intervals:
      - interval: 60
      - interval: 900
      - interval: 3600
        notify: true
      - interval: 7200
      - interval: 10800
transportgroups[].dsn.transport

Enable bounces on delivery failures via the specified transport ID.

1.2.4. Queue directives

queues.concurrency.total

The maximum total number of outbound SMTP connections that the queue subsystem will make. This should be the basis for the virtual queues you define. Halon uses an asynchronous model, and can therefore support a much larger number compared to servers that use a process- or thread-based model. Make sure that the number of files resource limit environment.rlimit.nofile is a 3-4 times larger than this number. For 20000 connections, something like 70000 open files are needed. The default is 10000.

smtpd-app.yaml
queues:
  concurrency:
    total: 20000
smtpd.yaml
environment:
  rlimit:
    nofile: 70000
queues.pooling.size

The maximum number of idle connections that the queue subsystem will hold in its pooling cache for connection reuse before eviction takes place. This requires one open file resource per connection, similar to queues.concurrency.total. Unlimited is 0 (not recommended). The default is 1000.

postmaster.name

The name used in the From header of system generated email such as delivery reports.

postmaster.address

The email address used in the From header of system generated email such as delivery reports.

1.2.5. General directives

addresses[]

This directive is used to define “named” IP addresses so that they can be referenced by their ID from configuration and script. This is not only useful for giving IP addresses descriptive IDs, but also since you can share script between multiple MTAs in a cluster without having to reference specific IPs that only exists on one instance. The hostname directive will be used as HELO name if defined.

addresses:
  - id: "out1"
    address: 198.51.100.5
    hostname: smtp-out1.example.com
  - id: "out1v6"
    address: 2001:db8:85a3::8a2e:370:7334
    hostname: smtp-out1.example.com
  - id: "out2"
    address: 198.51.100.6
    hostname: smtp-out1.example.com
  - id: "out2v6"
    address: 2001:db8:85a3::8a2e:370:7335
    hostname: smtp-out1.example.com
  - id: "bulk"
    address: 203.0.113.67
    hostname: smtp-out2.example.com

They can then be referenced from for example script, like this pre-delivery snippet:

$sourceip = ["out", "out2", "outv6", "out2v6"];
if (GetMetaData()["spam"] == "yes")
  $sourceip = ["bulk"];
Try(["sourceip" => $sourceip]);

1.2.6. Other directives

resolver.concurrency

The maximum number of concurrent (pending) DNS queries. This should be set according to the DNS server used. The default is 100.

resolver.cache.size

The number of DNS queries to store in the in-memory LRU cache. The default is no caching.

resolver.cache.ttl.min

Normalize the time-to-live values in the cache to have at least this value.

resolver.cache.ttl.max

Normalize the time-to-live values in the cache to have at most this value.

pki.private[]

Array of private keys, possibly with X.509 certificates, for use with servers[] and script functions such as PKCS7, RSA, DKIM, client certificates, etc.

The id and privatekey properties are required, and certificate is optional. The privatekey can however be defined in the startup configuration for the same ID, in which case it is not required. The private key and certificate should have either a path or data property.

pki:
  private:
    - id: selfsigned
      certificate:
        data: |-
          -----BEGIN CERTIFICATE-----
          ...
      privatekey:
        data: |-
          -----BEGIN PRIVATE KEY-----
          ...

Note

For privilege separation reasons, it’s normally recommended to define the private keys in the startup configuration instead of here, as it is read before the privilege drop.

1.2.7. Example

Below is an unrealistically simple example of a running configuration with a few transports. The actual scripting is omitted, since it is normally packed by the tools.

servers:
  - id: default
    transport: mailserver
    phases:
      mailfrom:
        hook: ldaplookup
      eod:
        hook: inbound
    concurrency:
      total: 10000
      ip: 10
  - id: relay
    transport: mx
    phases:
      mailfrom:
        hook: relaylist
      eod:
        hook: outbound
transportgroups:
  - id: default
    transports:
      - id: mx
        session:
          tls:
            mode: dane
        dsn:
          transport: mx
        retry:
          count: 30
          intervals:
            - interval: 60
            - interval: 300
            - interval: 3600
              notify: true
            - interval: 7200
      - id: mailserver
        connection:
          server: mail.example.com
          port: 25
        retry:
          count: 30
          intervals:
            - interval: 10
            - interval: 60
            - interval: 3600
queueus:
  concurrency:
    total: 10000
postmaster:
  address: [email protected]
  name: Postmaster

1.2.8. Additional configurations

There are several other running configuration files, listed below.

1.2.8.1. Queue policy

Specified by the environment.policyconf directive and documented in the queue section.

1.2.8.2. Queue suspend

Specified by the environment.suspendconf directive and documented in the queue section.

1.2.8.3. Queue delivery

Specified by the environment.deliveryconf directive and documented in the queue section.

1.2.8.4. Rate subsystem

Configuration for the rated program that is used for the rate() script function. Specified by rated.yaml’s environment.appconf directive. It is described by, and can be validated with, the rated-app.schema.json JSON schema (included in our Visual Studio Code plugin).

1.2.8.5. Content inspection

Configuration for the dlpd program that is used for the DLP script function. Specified by dlpd.yaml’s environment.appconf directive. It is described by, and can be validated with, the dlpd-app.schema.json JSON schema (included in our Visual Studio Code plugin).