4. History logging
To enable history logging there are some steps that need to be completed which are outlined below.
4.1. Install Elasticsearch
Elastic has an article on installing Elasticsearch here.
4.2. Import an index template
Import one of the index templates below (Delivery attempts or Transactions) into Elasticsearch. The differences between these index templates are described in each section.
4.2.1. Delivery attempts
This template will log all delivery attempts and the same message may appear multiple times, once for each delivery attempt.
For this template you should set the value for elasticsearch.index
to halon-delivery-attempts
in the web.yaml
file.
curl -k -X PUT -H "Content-Type: application/json" -d @/opt/halon/web/share/history/elasticsearch/delivery-attempts/halon_timeseries_policy.json http://elastic:[email protected]:9200/_ilm/policy/halon-timeseries-policy curl -k -X PUT -H "Content-Type: application/json" -d @/opt/halon/web/share/history/elasticsearch/delivery-attempts/halon_delivery_attempts.json http://elastic:[email protected]:9200/_index_template/halon-delivery-attempts
4.2.2. Transactions
This template will log all transactions and update the state for them instead of creating a new log entry after each delivery attempt.
For this template you should set the value for elasticsearch.index
to halon-transactions-*
in the web.yaml
file.
curl -k -X PUT -H "Content-Type: application/json" -d @/opt/halon/web/share/history/elasticsearch/transactions/halon_transactions.json http://elastic:[email protected]:9200/_index_template/halon-transactions
When using the transactions template you also need to periodically remove old indices in Elasticsearch, you can use Elasticsearch’s Curator CLI as a cron job. This sample configuration will remove indices older than 120 days based on the index and date syntax (e.g. halon-transactions-%Y-%m-%d).
Curator configuration ~/.curator/curator.yml
sample file:
elasticsearch: client: hosts: - http://127.0.0.1:9200 ca_certs: client_cert: client_key: verify_certs: False request_timeout: 30 other_settings: username: elastic password: elastic master_only: False logging: loglevel: INFO logfile: logformat: default blacklist: ['elastic_transport', 'urllib3']
Curator action ~/.curator/delete_indices.yml
sample file:
actions: 1: action: delete_indices description: >- Delete indices older than 120 days (based on index name) options: ignore_empty_list: True disable_action: False filters: - filtertype: pattern kind: prefix value: halon-transactions- - filtertype: age source: name direction: older timestring: '%Y-%m-%d' unit: days unit_count: 120
Run the action file with this command:
curator delete_indices.yml
4.3. Add configuration
The history feature requires an additional configuration file. This configuration file is used to define fields in Elasticsearch and how they should be rendered.
halon-web loads the non-reloadable configuration from /etc/halon/web-history.yaml
.
It is described by, and can be validated with, the web-history.schema.json JSON schema.
The default startup configuration file in /opt/halon/web/share
can normally be used as a base (only choose one of them):
The following configuration is based on the Elasticsearch index template called halon-delivery-attempts:
# cp /opt/halon/web/share/history/delivery-attempts.yaml /etc/halon/web-history.yaml
The following configuration is based on the Elasticsearch index template called halon-transactions:
# cp /opt/halon/web/share/history/transactions.yaml /etc/halon/web-history.yaml
Use the elasticsearch
directives to enable history logging in the web administration.
Sample web.yaml config using the Delivery attempts template:
elasticsearch:
index: 'halon-delivery-attempts'
nodes:
- url: http://127.0.0.1:9200
auth:
username: elastic
password: elastic
tls:
rejectUnauthorized: false
4.4. Start sending data
To start sending data to Elasticsearch from the smtpd process using HSL you can use the history-elastic plugin (supports both index templates).