<a id="introduction"></a>

# Introduction

The purpose of the `halon-policyd` service is to orchestrate the flow of email from the cluster, enabling source IPs to be warmed up with selected traffic according to planned schedules.

The service is hosted separately to the Halon [smtpd](https://docs.halon.io/manual/) instances, in a similar manner to the [halon-clusterd](https://docs.halon.io/clusterd/) service.
Because these services impose a small load on the host, they can be co-located; in a [containerized environment](https://github.com/halon/halon-docker/) it is usual to dedicate a pod to each service.

The warmup schedules are easily defined by the user on the [Halon web UI](https://docs.halon.io/web/) Delivery Guru / IP Warmup page.
You can also directly edit the `/etc/halon/policyd-app.yaml` [Running configuration](config_running.html.md), or use the [HTTP API](api.html.md).

For each IP address being warmed, the user selects one or more recipient groupings (`&yahoo`, `&gmail`, `&microsoft` and so on).

For each grouping, the user can define:

* The schedule mode (automatic or manual)
* The start date for the warmup
* How many days to run
* The target volume per day, when the warmup is done.

In manual mode, the user can adjust additional settings as the warmup progresses.

Daily warmup progress can be monitored by the user on the web UI, in both graphical and tabular form.

<a id="connectivity-and-state"></a>

## Connectivity and state

```mermaid
graph LR
  B[halon-policyd]

  A[(Elasticsearch)]
  D[Halon Engage Web UI]

  C[smtpd]

  D -->|reads/writes| B
  D -->|reads| A
  B -->|writes| A
  C -->|writes| A
  C -->|subscribes to events| B
```

Components:
: - **Elasticsearch:** Stores action events (rate policies and suspensions) for warmup progress tracking and analytics.
  - **halon-policyd:** The main service that manages warmup schedules, policies, and state.
  - **smtpd:** The SMTP daemon responsible for sending email, subscribing to policy updates from `halon-policyd`.
  - **Halon Engage Web UI:** The user interface for monitoring and configuring warmup schedules and viewing progress.

Each `smtpd` subscribes to events on a websocket, which provides a bidirectional stream of updates to rate policies and suspensions to achieve warmup goals.
The Halon smtpd plugin [policyd-client](https://github.com/halon-extras/policyd-client) is configured in the startup configuration `plugins` section, and must
match the `halon-policyd` startup configuration [WebSocket directives](config_startup.html.md#websocket), see [example configuration](#example).

Elasticsearch is a required component. Indexes are used to hold:

- **action events**  - a record of the actions taken by the `halon-policyd` service.

To set up Elasticsearch, see [Elasticsearch](elastic.html.md).

The `halon-policyd` service requires persistent state of the schedules over many days. This is held in the [Running configuration](config_running.html.md).
The `halon-policyd` service updates the running schedules every 24 hours.
This is used to ensure that the warmup schedules are always up-to-date and reflect the current state of the warmup process.
Schedules are also updated when the user makes changes to the warmup schedules on the web UI.

The Halon web UI connects to `halon-policyd` via an API listener. This can be secured with [`listeners[].pki`](config_startup.html.md#confval-listeners-.pki) , and access control with [`authentication.apikeys[]`](config_running.html.md#confval-authentication.apikeys).

<a id="tips-for-successful-warmup"></a>

## Tips for successful warmup

> **Important**
> 
> Add the cold IPs into a warmup schedule before those IPs are active for delivery, otherwise they will be used immediately which can lead to poor warmup results.

When adding a new IP, define it first in the warmup menu. Set the grouping schedules to begin on a day in the future.
Then add the IP to a transport in your `smtpd` configuration.
This prevents the IP from being used before you are ready.

To ensure that messages are delivered in a timely manner during warming, there must be an alternative, **already warm IP** in the same transport to carry overflow email.

Effective warmup needs enough eligible messages in the queue, for most of the day, to reach the day’s target volume.
If not enough messages are available, then the deliveries made via the warming IP will naturally fall below the daily target, taking more days to reach the desired volume.

<a id="recommended-approach"></a>

### Recommended approach

As the already-warm IP will have a fast delivery rate, adjust this down a little in static policy, so that some queued email is eligible for attempts via the warming IPs.

<a id="alternate-approach"></a>

### Alternate approach

You can also add the warmup IP first in the [transportgroups[].connection.sourceip.ipv4](https://docs.halon.io/manual/config_smtpd.html#confval-transportgroups-.connection.sourceip.ipv4) list
or [transportgroups[].connection.sourceip.ipv6](https://docs.halon.io/manual/config_smtpd.html#confval-transportgroups-.connection.sourceip.ipv6) list
and then disable [transportgroups[].connection.sourceip.random](https://docs.halon.io/manual/config_smtpd.html#confval-transportgroups-.connection.sourceip.random) (defaults `true`).
Then it will always **try the warmup IP first** if it is allowed to deliver traffic.

If you have more than one already warm IP (in addition to the one being warmed-up) disabling `random` is not ideal, as it may never use the IPs further down in the list, unless you have a lot of traffic.

<a id="example"></a>

<a id="example-configuration"></a>

## Example configuration

> **Note**
> 
> This example uses `wss://` to establish a secure (TLS) WebSocket connection from `smtpd` to `halon-policyd`. Use `ws://` for an unencrypted (plain) WebSocket connection.

*smtpd.yaml*

```yaml
plugins:
- id: policyd-client
  config:
    address: wss://my-cluster.company.com:8091 # talk to the halon-policyd websocket listener
```

*policyd.yaml*

```yaml
version: "1.1"
listeners:
- port: 8090 # talk to the web UI
  # omit address: to listen on all interfaces, ipv4 and ipv6
  pki:
    privatekey:
      # your private key here
    certificate:
      # your certificate here
websocket:
  listener:
    port: 8091 # talk to the smtpd policyd-clients
    # omit address: to listen on all interfaces, ipv4 and ipv6
  pki:
    privatekey:
      # your private key here
    certificate:
      # your certificate here

elasticsearch:
  index:
    actions: "halon-policyd-actions" # written by halon-policyd
  nodes:
    - url: https://your-elastichost.com:9200 # adjust for your ElasticSearch service
  auth:
    username: elastic
    password: # adjust for your ElasticSearch service
  tls:
    verify: false # set if your ElasticSearch has self-signed certs
```

*policyd-app.yaml*

```yaml
version: "1.1"
authentication:
  apikeys:
  - badsecret # matches policyd.secret[] in web.yaml
```

*web.yaml*

```yaml
elasticsearch:
  index:
    policyd: "halon-policyd-actions"
  nodes:
    - url: https://your-elastichost.com:9200
  auth:
    username: elastic
    password: # adjust for your ElasticSearch service
  tls:
    rejectUnauthorized: false # set if your ElasticSearch has self-signed certs

policyd:
  address: your.policyd.com #  where halon-policyd is running
  port: 8090 # talk to halon-policyd
  secret: badsecret # matches authentication.apikeys[] in policyd-app.yaml
  tls:
    enabled: true
    verify: false
```