3.1. Startup configuration

This is the non-reloadable part of the configuration. By default, smtpd loads it from /etc/halon/smtpd.yaml. It is described by, and can be validated with, the smtpd.schema.json.

It most importantly contains the server listen sockets (bind addresses and ports), PROXY protocol and thread settings.

Note

The example configuration in /opt/halon/share/examples/ that is usually copied to /etc/halon/ during installation contains reasonable defaults. Normally you only need to modify it when adding additional virtual servers, or changing performance related settings such as the number of threads or open files resource limit.

3.1.1. Server directives

The virtual servers[] are configured in the running configuration, but which port(s) and address(es) to listen to needs to be specified in this file.

Most of the properties applies to servers of both smtp, http, https and plugin type, therefor the only which applies to either are explicitly mentioned.

Below is an example for adding a virtual server called “relay”, listening to any IP on port 587:

servers:
  - id: relay
    listeners:
      - port: 587

servers[].id: The id of the server. It must match the same type as configured in the running configuration.

servers[].type: The type of the server, a server can either be an smtp server, an http / https submission endpoint or plugin inject plugin. The default is smtp. The https server type requires a servers[].tls.certs.cert to be configured in the running configuration.

servers[].listeners[]

^{Only one listener is supported for servers of type http. Not supported for type plugin}

Each virtual server must have one or more listen directives, which specify which TCP port and address to listen on.

servers[].listeners[].port: Which TCP port to listen on. Required.

servers[].listeners[].address: IPv4 or IPv6 address to listen on. The default is to listen to all IPv4 and IPv6 addresses.

servers[].listeners[].backlog: The kernel connection backlog. The default is the system default.

servers[].listeners[].id: An optional ID that can be used for referencing a listen directive from the running configuration so that for example implicit TLS can be enabled on a per-listener basis using servers[].tls.implicit.

servers[].proxyprotocol[]

^{Only applies to smtp servers}

A list of IP addresses or networks to allow the PROXY protocol (v1 and v2) from. Set it to true to allow it from all IP addresses.

servers[].threads.event

^{Not supported for type plugin}

The number of servers[] event loop threads, allowing the event loop to take advantage of multiple CPUs. If given as a number (and not an object with an ID), a event thread pool will be created named after the servers[].id (prefixed with an underscore). The default is 4. If configured as an object you may specify an id of a servers[].threads.event.id or an object with servers[].threads.event.count and servers[].threads.event.priority.

servers[].threads.event.id

^{Only applies to smtp servers}

The name of the threads.events[].id. This property is required if configured with custom event threads.

servers[].threads.event.count: The number of servers[] event loop threads, allowing the event loop to take advantage of multiple CPUs. The default is 4.

servers[].threads.event.priority

^{Only applies to smtp servers}

The priority (nice value) of the thread. The less nice it is the higher priority (-20 to 19). The default is 0.

Note

Don’t forget to update this value as well environment.rlimit.nice.

servers[].threads.script: The number of servers[] script threads running the hooks such as servers[].phases.eod.hook. If given as a number (and not an object), a script thread pool will be created named after the servers[].id (prefixed with an underscore) with the servers[].scripting.concurrency and servers[].scripting.stacksize. The default is 32.

servers[].threads.script.id: The name of the default threads.scripts[].id. This property is required if configured with custom hook threads, and it will be used for any un-configured hooks.

servers[].threads.script.hooks.connect

^{Only applies to smtp servers}

The name of the threads.scripts[].id for the connect hook.

servers[].threads.script.hooks.proxy

^{Only applies to smtp servers}

The name of the threads.scripts[].id for the proxy hook.

servers[].threads.script.hooks.helo

^{Only applies to smtp servers}

The name of the threads.scripts[].id for the helo hook.

servers[].threads.script.hooks.auth

^{Only applies to smtp servers}

The name of the threads.scripts[].id for the auth hook.

servers[].threads.script.hooks.mailfrom

^{Only applies to smtp servers}

The name of the threads.scripts[].id for the mailfrom hook.

servers[].threads.script.hooks.rcptto

^{Only applies to smtp servers}

The name of the threads.scripts[].id for the rcptto hook.

servers[].threads.script.hooks.eod: The name of the threads.scripts[].id for the eod hook.

servers[].threads.script.hooks.disconnect

^{Only applies to smtp servers}

The name of the threads.scripts[].id for the disconnect hook.

servers[].scripting.concurrency: The number of concurrent hooks that may be running concurrently on the servers[].threads.script. The default is servers[].threads.script (1:1). However using a plugin or function supporting the HSL suspend functionality more may be running concurrently using cooperative multitasking (M:N).

servers[].scripting.stacksize: The stack size of a hook. The default is 2097152 bytes (2 MiB). This setting shouldn’t be changed without a strong reason and understanding for doing so.

3.1.2. Queue directives

queues.threads.event: The queue event loop threads, allowing the event loop to take advantage of multiple CPUs. If given as a number (and not an object without an ID), a event thread pool will be created named “_queues”. The default is 4. If configured as an object you may specify an id of a queues.threads.event.id or an object with queues.threads.event.count and queues.threads.event.priority.

queues.threads.event.id: The name of the threads.events[].id. This property is required if configured with custom event threads.

queues.threads.event.count: The number of queue event loop threads, allowing the event loop to take advantage of multiple CPUs. The default is 4.

queues.threads.event.priority: The priority (nice value) of the thread. The less nice it is the higher priority (-20 to 19). The default is 0.

Note

Don’t forget to update this value as well environment.rlimit.nice.

queues.threads.script: The number of queue script threads running the scripting.hooks.predelivery and scripting.hooks.postdelivery hooks. If given as a number (and not an object), a script thread pool will be created named “_queues” with the queues.scripting.concurrency and queues.scripting.stacksize. The default is 32.

queues.threads.script.id: The name of the default threads.scripts[].id. This property is required if configured with custom hook threads, and it will be used for any un-configured hooks.

queues.threads.script.hooks.predelivery: The name of the threads.scripts[].id for the predelivery hook.

queues.threads.script.hooks.postdelivery: The name of the threads.scripts[].id for the postdelivery hook.

queues.threads.pickup: If given as a number of threads preparing messages to be delivered from the active queue. The default is 1. If configured as an object see queues.threads.pickup.count.

queues.threads.pickup.count: The number of threads preparing messages to be delivered from the active queue. The default is 1.

queues.threads.pickup.priority: The priority (nice value) of the thread. The less nice it is the higher priority (-20 to 19). The default is 0.

Note

Don’t forget to update this value as well environment.rlimit.nice.

queues.threads.fetch.priority: The priority (nice value) of the thread fetching messages from the queue. The less nice it is the higher priority (-20 to 19). The default is 0.

queues.threads.rate.priority: The priority (nice value) of the thread refilling queue policy rates. The less nice it is the higher priority (-20 to 19). The default is 0.

queues.threads.release.priority: The priority (nice value) of the thread flagging delivered messages (somewhat like the opposite of queues.threads.pickup). The less nice it is the higher priority (-20 to 19). The default is 0.

queues.scripting.concurrency: The number of concurrent hooks that may be running concurrently on the queues.threads.script. The default is queues.threads.script (1:1). However using a plugin or function supporting the HSL suspend functionality more may be running concurrently using cooperative multitasking (M:N).

queues.scripting.stacksize: The stack size of a hook. The default is 2097152 bytes (2 MiB). This setting shouldn’t be changed without a strong reason and understanding for doing so.

spool.path: The email queue spool path. This option is mutually exclusive with spool.paths[].path. The default is /var/spool/halon/queue.

spool.paths[].path: The email queue spool paths (multiple paths). The paths are used in a round-robin fashion to spread load evenly. This option is mutually exclusive with spool.path. The default is /var/spool/halon/queue. During spool in messages are spooled in the order they are specified in the configuration.

spool.paths[].write: If the path should used when receiving messages (or just consuming existing messages in that path). The default is true.

spool.minfree.inodes: The minimum required free inodes on the spool.path disk. The default is 0.

spool.minfree.bytes: The minimum required free bytes on the spool.path disk. The default is 0.

spool.threads.loader: If given as a number of threads that read the spool files into memory during startup. Those are terminated once the spool is loaded. The default is 32. If configured as an object see spool.threads.loader.count.

spool.threads.loader.count: The number of threads that read the spool files into memory during startup. The default is 32.

spool.threads.loader.priority: The priority (nice value) of the thread. The less nice it is the higher priority (-20 to 19). The default is 0.

Note

Don’t forget to update this value as well environment.rlimit.nice.

spool.threads.loader.wait: If the startup process should wait for the loader to spool in all messages. This allows quotas etc to be properly updated before new messages are accepted. For faster startup times, this can be disabled. The default is true.

spool.threads.update: The maximum number of worker threads for API or CLI queue update of the spool files. The default is 32.

spool.fsync: If fsync operations should be used for the email queue. The default is true.

spool.corrupt: How to handle incomplete or corrupt queue (.hqf) files upon startup. The default action is unlink. But it is also possible to rename those (appending .bad) or ignore to do nothing.

spool.rate: Specify the rate (files/seconds) of which messages will be spooled in from disk. This (in combination with spool.threads.loader.wait) allows you to reduce the load caused by spool in while receiving new messages. The default is no limitation.

3.1.3. Threads directives

3.1.3.1. Scripts

Custom script thread pools allows you to control on which script thread pools a script / hook is executed.

threads.scripts[].id: The name of the script thread pool

threads.scripts[].count: The number of scripts threads running a hooks such as servers[].phases.eod.hook. The default is 32.

threads.scripts[].priority: The priority (nice value) of the thread. The less nice it is the higher priority (-20 to 19). The default is 0.

Note

Don’t forget to update this value as well environment.rlimit.nice.

threads.scripts[].concurrency: The number of concurrent hooks that may be running concurrently. The default is threads.scripts[].count (1:1). However using a plugin or function supporting the HSL suspend functionality more may be running concurrently using cooperative multitasking (M:N).

threads.scripts[].stacksize: The stack size of a hook. The default is 2097152 bytes (2 MiB). This setting shouldn’t be changed without a strong reason and understanding for doing so.

3.1.3.2. Events

Custom event thread pools allows you to control on which thread pools a network event is processed.

threads.events[].id: The name of the events thread pool

threads.events[].count: The number of events threads running network events. The default is 4.

threads.events[].priority: The priority (nice value) of the thread. The less nice it is the higher priority (-20 to 19). The default is 0.

Note

Don’t forget to update this value as well environment.rlimit.nice.

3.1.4. Other directives

resolver.threads.event: Number of DNS resolver event pool threads. The default is 1.

plugins[]

Load the following plugins into the MTA. The id and path properties are required. An optional config object or YAML path may be specified for the plugin’s configuration. A similar config property is also available in the running configuration.

plugins:
  - id: test
    path: "/opt/halon/plugins/test.so"
    config:
      myval: true

plugins[].config: The startup configuration. This property may be either a string or an YAML object. If this property is given as a string it must point to a YAML file on disk.

pki.private[]

Array of private keys, possibly with X.509 certificates, for use with servers[] and script functions such as PKCS7, RSA, DKIM, client certificates, etc.

The id and privatekey properties are required, and certificate is optional. The private key and certificate should have either a path or data property.

pki:
  private:
    - id: selfsigned
      certificate:
        data: |-
          -----BEGIN CERTIFICATE-----
          ...
      privatekey:
        data: |-
          -----BEGIN PRIVATE KEY-----
          ...

Note

It is also possible to add those to the running configuration, but for privilege separation reasons it’s normally recommended to define private keys here instead, as this startup configuration is read before the privilege drop. It is however possible to load the private key from a path in this startup configuration, and load the certificate from a path in running configuration, which allows you to softly reload the certificate when it changes, as long as the private key stays the same.

scripting.ffi: Enable the use of FFI functions from the script language. The default is false.

scripting.rootpath: Enable accessing files from disk via the File class and the scripting.files[] path, relative from the specified root path. The default is to not allow accessing files on disk.

scripting.cache.scope: Configure the script language cache [] statement scope. It can either be global (a persistent cache between config reloads) or config (to be per config reload or live stage, shared between all hooks). The default scope is per config.

3.1.5. Clustering directives

It’s possible to configure clustering for sharing smtpd-policy.yaml’s concurrency, connectinterval and rate in a cluster.

cluster.address: The peer address of the clusterd service.

cluster.port: The peer port of the clusterd service. The default is 2530.

cluster.policy.sharedconcurrency: If the concurrency should be shared in the cluster (this is the opposite of evenly divide based on the number of hosts connected). The default is true.

cluster.pki.id: The id of the client certificate PKI certificate to use.

cluster.pki.verify: If the server peer certificate should be verified.

3.1.6. Environment directives

The default startup configuration that came with the installation package contains reasonable defaults for your platform. Some settings should however be revised.

3.1.6.1. Performance and log

Those settings are typically configured depending on your system and use case.

environment.rlimit.nofile

Set the max number of open file descriptors (this changes the current process limit from system default ulimit -n). This value should be calculated from other limits to allow the server to allocate system resources as needed. There is no downside of specifying a value larger than needed, so there is no reason to be conservative (just make sure it fits within system limits). The most dominant values in the configuration has the biggest impact on the result, so for simplicity in most cases you may use a quite simple formula like this.

The sum of servers[].concurrency.total, queues.concurrency.total and queues.pooling.size multiplying by 5 and adding a fixed value of eg. 10000 (to safely cover other limits such a script thread usage and DNS etc). Even if this value comes out twice or more than what is actual needed it should be used as a safety margin.

((1000 + 10000 + 1000) * 5) + 10000 = 70000

Given the potential complexity of the calculation, which also depends on how the scripting is used, we cannot safely list all other values that need to be considered. In most cases we estimate that these should all fit within the margin of the calculation (both in multiplication factor and also the fixed value added). However, for transparency here is a few.

smtpd’s file descriptors, used for various things
servers[].listeners[]
servers[].threads.script
queues.threads.script
spool.threads.update
resolver.threads.event

The Linux kernel has two limits, the sysctl fs.file-max which is the upper limit of fd in the OS kernel and sysctl fs.nr_open which controls the upper limit of ulimit command. It’s important that they are larger than the environment.rlimit.nofile specified.

environment.rlimit.nice: Set the the highest priority (lowest nice value) a thread can request (this changes the current process limit from system default ulimit -e). The less nice it is the higher priority (-20 to 19). The default is 0.

environment.syslog.mask: If you are using systemd-journald for syslog(), we strongly recommend masking away LOG_INFO (non-error email transaction) for performance reasons by setting this option to 191. For transaction logging you can use a module from our script library such as Elastic, libjlog or syslog directly to rsyslog. The default is no mask.

environment.syslog.ident: The syslog identity. The default is the program name.

environment.syslog.facility: The syslog facility. Valid options are daemon, mail, user, local0, local1, local2, local3, local4, local5, local6 and local7. The default is mail.

environment.syslog.pid: Log the process ID. The default is false.

3.1.6.2. Configuration paths

Those paths can normally be left unchanged.

environment.appconf: From where to load the running configuration. The default is /etc/halon/smtpd-app.yaml.

environment.policyconf: From where to load the active queue policies. The default is /etc/halon/smtpd-policy.yaml.

environment.suspendconf: From where to load the active queue suspends. The default is /etc/halon/smtpd-suspend.yaml.

environment.deliveryconf: From where to load the active queue delivery settings. The default is /etc/halon/smtpd-delivery.yaml.

environment.licensekey: From where to load the license. The default is /etc/halon/license.key. A license key may also be loaded from the environment variable HALON_LICENSEKEY, the should contain the license file content base64 encoded. The environment variable has precedence over this environment.licensekey setting.

3.1.6.3. Other environment

Those settings should normally be left unchanged. The default startup configuration in /opt/halon/examples that came with the installation package should contain correct parameters for your operating system or distribution. Those settings are described in the programs section.

environment.controlsocket.path: Have the control socket on a UNIX socket. This option is mutually exclusive with the environment.controlsocket.port setting.

environment.controlsocket.owner: This option is only supported with the environment.controlsocket.path setting.

environment.controlsocket.group: This option is only supported with the environment.controlsocket.path setting.

environment.controlsocket.chmod: This option is only supported with the environment.controlsocket.path setting.

environment.controlsocket.port: Have the control socket on a TCP port. This option is mutually exclusive with the environment.controlsocket.path setting.

environment.controlsocket.address: Have the control socket listen on a specific IP address. This option is only supported with the environment.controlsocket.port setting.

environment.controlsocket.backlog: The kernel connection backlog. The default is the system default.

environment.privdrop.user: The user to run the process as.

environment.privdrop.group: The group to run the process as.

environment.publicsuffix: The path to a publicsuffix list (publicsuffix.org).

environment.umask: Change the umask for file creation. The default is often 022.

environment.uuid.version: There are two supported methods of UUID generation; time based version 1 and random version 4. The default is 1 and it requires safe UUIDs to be generated (which is tested for upon smtpd startup). Safe in this context means it has to be thread-safe; assisted by the operating system/implementation (either by a syscall, a daemon such as uuidd or file locking). Depending on the environment the preferred way may differ. If you cannot use any of these, we also offer version 4 which is random based. The probably of collision of random UUIDs are so low they should be considered safe.