1. Introduction

The purpose of the halon-policyd service is to orchestrate the flow of email from the cluster, enabling source IPs to be warmed up with selected traffic according to planned schedules.

The service is hosted separately to the Halon smtpd instances, in a similar manner to the halon-clusterd service. Because these services impose a small load on the host, they can be co-located; in a containerized environment it is usual to dedicate a pod to each service.

The warmup schedules are easily defined by the user on the Halon web UI Delivery Guru / IP Warmup page. You can also directly edit the /etc/halon/policyd-app.yaml Running configuration, or use the HTTP API.

For each IP address being warmed, the user selects one or more recipient groupings (&yahoo, &gmail, &microsoft and so on).

For each grouping, the user can define:

  • The schedule mode (automatic or manual)

  • The start date for the warmup

  • How many days to run

  • The target volume per day, when the warmup is done.

In manual mode, the user can adjust additional settings as the warmup progresses.

Daily warmup progress can be monitored by the user on the web UI, in both graphical and tabular form.

1.1. Connectivity and state

graph LR subgraph halon-policyd B[halon-policyd] E[(Local Stats Files)] end A[(Elasticsearch)] D[Halon Engage Web UI] C[smtpd] F[Filestore Service] D -->|reads/writes| B D -->|reads| A B -->|reads/writes| A C -->|writes| A B -->|writes| E E -->|uploads to| F C -->|subscribes to events| B
Components:

  • Elasticsearch: Stores history and action events (rate policies and suspensions) for warmup progress tracking and analytics.

  • halon-policyd: The main service that manages warmup schedules, policies, and state.

  • smtpd: The SMTP daemon responsible for sending email, subscribing to policy updates from halon-policyd.

  • Halon Engage Web UI: The user interface for monitoring and configuring warmup schedules and viewing progress.

  • Local Stats Files: Files generated by halon-policyd to record state and decisions, later uploaded for persistence.

  • Filestore Service: Central storage for stats files and other persistent artifacts.

Each smtpd subscribes to events on a websocket, which provides a bidirectional stream of updates to rate policies and suspensions to achieve warmup goals. The Halon plugin policyd-client is configured in the startup configuration plugins section, and must match the halon-policyd startup configuration WebSocket directives, see example configuration.

Elasticsearch is a required component. Indexes are used to hold:

  • history events - the history of warmup progress, used to display the warmup progress on the web UI

  • action events - a record of the actions taken by the halon-policyd service.

To set up Elasticsearch, see History and action events.

The halon-policyd service requires persistent state of the schedules over many days. This is held in the Running configuration. The halon-policyd service updates the running schedules in auto mode every 15 minutes and does bigger adjustments to the schedules every 24 hours. This is used to ensure that the warmup schedules are always up-to-date and reflect the current state of the warmup process. Schedules are also updated when the user makes changes to the warmup schedules on the web UI.

The local stats files are used to hold the state in which the schedules adjustment decisions were made. These are created by the halon-policyd service, and uploaded to the filestore service for persistence and analysis.

The Halon web UI connects to halon-policyd via an API listener. This can be secured with listeners[].pki , and access control with authentication.apikeys[].

1.2. Tips for successful warmup

Important

Add the cold IPs into a warmup schedule before those IPs are active for delivery, otherwise they will be used immediately which can lead to poor warmup results.

When adding a new IP, define it first in the warmup menu. Set the grouping schedules to begin on a day in the future. Then add the IP to a transport in your smtpd configuration. This prevents the IP from being used before you are ready.

To ensure that messages are delivered in a timely manner during warming, there must be an alternative, already warm IP in the same transport to carry overflow email.

Effective warmup needs enough eligible messages in the queue, for most of the day, to reach the day’s target volume. If not enough messages are available, then the deliveries made via the warming IP will naturally fall below the daily target, taking more days to reach the desired volume.

1.2.2. Alternate approach

You can also add the warmup IP first in the transportgroups[].connection.sourceip.ipv4 list or transportgroups[].connection.sourceip.ipv6 list and then disable transportgroups[].connection.sourceip.random (defaults true). Then it will always try the warmup IP first if it is allowed to deliver traffic.

If you have more than one already warm IP (in addition to the one being warmed-up) disabling random is not ideal, as it may never use the IPs further down in the list, unless you have a lot of traffic.

1.3. Example configuration

Note

This example uses wss:// to establish a secure (TLS) WebSocket connection from smtpd to halon-policyd. Use ws:// for an unencrypted (plain) WebSocket connection.

smtpd.yaml
plugins:
- config:
  id: policyd-client
  address: wss://my-cluster.company.com:8091 # talk to the halon-policyd websocket listener
policyd.yaml
version: "1.0"
listeners:
- port: 8090 # talk to the web UI
  # omit address: to listen on all interfaces, ipv4 and ipv6
  pki:
    privatekey:
      # your private key here
    certificate:
      # your certificate here
websocket:
  listener:
    port: 8091 # talk to the smtpd policyd-clients
    # omit address: to listen on all interfaces, ipv4 and ipv6
  pki:
    privatekey:
      # your private key here
    certificate:
      # your certificate here

elasticsearch:
  index:
    history: "halon-delivery-attempts" # written by smtpd
    actions: "halon-policyd-actions" # written by halon-policyd
  nodes:
    - url: https://your-elastichost.com:9200 # adjust for your ElasticSearch service
  auth:
    username: elastic
    password: # adjust for your ElasticSearch service
  tls:
    verify: false # set if your ElasticSearch has self-signed certs
stats:
  path: "/var/lib/halon/policyd/"
policyd-app.yaml
version: "1.0"
authentication:
  apikeys:
  - badsecret # matches policyd.secret[] in web.yaml
web.yaml
elasticsearch:
  index:
    history: "halon-delivery-attempts"
    policyd: "halon-policyd-actions"
  nodes:
    - url: https://your-elastichost.com:9200
  auth:
    username: elastic
    password: # adjust for your ElasticSearch service
  tls:
    rejectUnauthorized: false # set if your ElasticSearch has self-signed certs

policyd:
  address: your.policyd.com #  where halon-policyd is running
  port: 8090 # talk to halon-policyd
  secret: badsecret # matches authentication.apikeys[] in policyd-app.yaml
  tls:
    enabled: true
    verify: false