3. History logging
The history logging configuration controls the integration with Elasticsearch and the MTA, this enables users to for example release messages from quarantine and interact with the Halon queue directly.
To enable history logging there are some steps that need to be completed which are outlined below.
3.1. Install Elasticsearch
Instructions for how to install Elasticsearch can be found here.
3.2. Import an index template
Import one of the index templates below (Delivery attempts or Transactions) into Elasticsearch. The differences between these index templates are described in each section.
3.2.1. Delivery attempts
This template will log all delivery attempts and the same message may appear multiple times, once for each delivery attempt.
For this template you should set the value for servers[].history.elasticsearch.index
to halon-delivery-attempts
in msui.yaml
.
$ curl -k -X PUT -H "Content-Type: application/json" -d @/opt/halon/web/share/history/elasticsearch/delivery-attempts/halon_timeseries_policy.json http://elastic:[email protected]:9200/_ilm/policy/halon-timeseries-policy $ curl -k -X PUT -H "Content-Type: application/json" -d @/opt/halon/web/share/history/elasticsearch/delivery-attempts/halon_delivery_attempts.json http://elastic:[email protected]:9200/_index_template/halon-delivery-attempts
3.2.2. Transactions
This template will log all transactions and update the state for them instead of creating a new log entry after each delivery attempt.
For this template you should set the value for servers[].history.elasticsearch.index
to halon-transactions-*
in msui.yaml
.
$ curl -k -X PUT -H "Content-Type: application/json" -d @/opt/halon/web/share/history/elasticsearch/transactions/halon_transactions.json http://elastic:[email protected]:9200/_index_template/halon-transactions
When using the transactions template you also need to periodically remove old indices in Elasticsearch, you can use Elasticsearch’s Curator CLI as a cron job. This sample configuration will remove indices older than 120 days based on the index and date syntax (e.g. halon-transactions-%Y-%m-%d).
Curator configuration ~/.curator/curator.yml
sample file:
elasticsearch: client: hosts: - http://127.0.0.1:9200 ca_certs: client_cert: client_key: verify_certs: False request_timeout: 30 other_settings: username: elastic password: elastic master_only: False logging: loglevel: INFO logfile: logformat: default blacklist: ['elastic_transport', 'urllib3']
Curator action ~/.curator/delete_indices.yml
sample file:
actions: 1: action: delete_indices description: >- Delete indices older than 120 days (based on index name) options: ignore_empty_list: True disable_action: False filters: - filtertype: pattern kind: prefix value: halon-transactions- - filtertype: age source: name direction: older timestring: '%Y-%m-%d' unit: days unit_count: 120
Run the action file with this command:
$ curator delete_indices.yml
3.3. Elasticsearch configuration
Once Elasticsearch is installed and the selected index template has been added, the settings must be configured to connect MSUI:
Copy one of the following files to
/etc/halon/
(depending on the index template used in the previous step):For transactions logging
# cp /opt/halon/msui/share/examples/msui-history-transactions.yaml /etc/halon/msui-history.yaml
For delivery attempts logging
# cp /opt/halon/msui/share/examples/msui-history-delivery-attempts.yaml /etc/halon/msui-history.yaml
Configure the Elasticsearch directives in the
/etc/halon/msui.yaml
configuration file:
servers:
- [...]
history:
elasticsearch:
index: 'halon-transactions-*' # or 'halon-delivery-attempts'
nodes:
- url: https://elasticsearch:9200
auth:
username: elastic
password: badpassword
Configure the MTA directives in the
/etc/halon/msui.yaml
configuration file:
servers:
- [...]
history:
mta:
nodes:
- id: "hostname"
url: http://smtpd:8080
apikey: badsecret
auth:
username: admin
password: admin