Upgrade From Legacy Event Processor
- Table of Contents
- Introduction
-
Questions About Upgrading
- What are the advantages to upgrading?
- Can I upgrade to the new event processor?
- What versions of GreenArrow support the new event processor?
- What if I’m using the legacy event processor’s ability to filter events by mail class?
- What if I’m using the legacy event processor’s ability to write events to a logfile?
- Upgrade Procedure (complete conversion – no logfile usage)
- Upgrade Procedure (continue logfile usage)
Introduction
This guide will walk you through what you need to do to upgrade from the Legacy Event Processor to the “latest and greatest.” For an introduction to the event processor, and to see a description of what’s different about it – see the Event Processor documentation.
The legacy event processor was configured using a variety of different files:
/var/hvmail/control/record_events.simplemh.scomp
/var/hvmail/control/record_events.simplemh.bounce_all
/var/hvmail/control/record_events.simplemh.bounce_bad_address
/var/hvmail/control/record_events.simplemh.unsub
/var/hvmail/control/record_events.simplemh.open
/var/hvmail/control/record_events.simplemh.click
/var/hvmail/control/record_events.delivery_attempt.json
/var/hvmail/control/event_processor.json
And for Studio, events were controlled using database fields.
While the legacy event processor will continue to be supported in its current state (i.e. new features will not be added to it, but we will maintain support for it in the form of any relevant bug fixes), we encourage all customers to adapt to the new event processor as they are ready to make the migration.
The new event processor is controlled by our single configuration file:
/var/hvmail/control/greenarrow.conf
This guide will walk you through converting to the new event processor.
Questions About Upgrading
What are the advantages to upgrading?
-
Deliver the same events to multiple destinations.
-
Deliver in batches, leading to event delivery efficiency.
-
Configure in
greenarrow.conf
, as opposed to many different configuration files. -
Events are delivered faster – typically within a few seconds of event occurrence.
Can I upgrade to the new event processor?
You can upgrade to the new event processor if none of the following is true:
-
You are using an event destination that delivers using a database connection (
type=custom_sql
). -
You are writing events to a logfile.
What versions of GreenArrow support the new event processor?
GreenArrow v4.270.0 and beyond support the new event processor.
What if I’m using the legacy event processor’s ability to filter events by mail class?
The automatic configuration converter documented below does not handle this case. However, the new event processor does support this. See the event_delivery_filter_mail_class directive for more information, or contact GreenArrow Technical Support for assistance.
What if I’m using the legacy event processor’s ability to write events to a logfile?
We recommend that you continue using the legacy event processor for logfile generation, but migrate to the new event processor for any HTTP endpoints you have configured.
To accomplish this, you’ll follow the upgrade procedure below marked “Upgrade Procedure (continue logfile usage)”.
The switchover from the legacy event processor to the new event processor is not “atomic” if you continue to use the logfile event delivery. This means that there will likely be some duplicate events that reach the HTTP endpoint during the transition.
Upgrade Procedure (complete conversion – no logfile usage)
You can upgrade to the new event processor “atomically” – meaning that if you follow this procedure, no duplicate events will be generated (and no events will be missed).
(1) Determine your new greenarrow.conf
configuration
Start by running the following command. If you are a candidate for the new
event processor, it will emit the greenarrow.conf
syntax needed to deliver
your events using the new event processor.
$ greenarrow events convert_legacy_configuration
event_delivery_destination destination-1 {
event_delivery_url `http://example.com`
event_delivery_events engine_open, bounce_bad_address.studio, studio_subscriber_created
event_delivery_disable_batching yes
event_delivery_format json
event_delivery_serialize_all_columns yes
}
If the above command gives you an error, it’ll describe the reason your configuration isn’t compatible for easy conversion to the new event processor.
Otherwise, copy and paste the event_delivery_destination block(s) to your
/var/hvmail/control/greenarrow.conf
configuration file.
Do not reload configuration at this time.
(2) Disable legacy event creation
Run the following to disable legacy event creation:
echo 0 > /var/hvmail/control/record_events.simplemh.scomp
echo 0 > /var/hvmail/control/record_events.simplemh.bounce_all
echo 0 > /var/hvmail/control/record_events.simplemh.bounce_bad_address
echo 0 > /var/hvmail/control/record_events.simplemh.unsub
echo 0 > /var/hvmail/control/record_events.simplemh.open
echo 0 > /var/hvmail/control/record_events.simplemh.click
$ /var/hvmail/postgres/default/bin/psql -U greenarrow greenarrow -c '
UPDATE s_system_configs SET
event_notification_studio_click=false,
event_notification_studio_open=false,
event_notification_studio_unsub=false,
event_notification_bounce_all=false,
event_notification_bounce_bad_address=false,
event_notification_scomp=false,
event_notification_studio_subscriber_created=false,
event_notification_studio_subscriber_updated=false;
'
Edit the following file and set the enabled
key to false
:
/var/hvmail/control/record_events.delivery_attempt.json
(3) Reload configuration to activate the above changes
Run GreenArrow’s standard configuration reload to activate the changes you made above:
greenarrow_config reload
(4) Wait for the legacy event processor’s queue to drain
Monitor the following command, waiting for the __legacy__
event destination’s queue to drain to 0:
$ greenarrow events status
+------------------+-----------------------------+-----------------------+
| Destination Name | Event Count (First Attempt) | Event Count (Retries) |
+------------------+-----------------------------+-----------------------+
| destination-1 | 0 | 0 |
| __legacy__ | 100 | n/a |
+------------------+-----------------------------+-----------------------+
(5) Remove the legacy event processor’s configuration
Now you can edit the following file:
/var/hvmail/control/event_processor.json
Typically you can replace its contents with this in order to disable event delivery:
{
"event_destinations": [
{
"matches": {
"all": true
},
"destination": {
"type": "drop_from_queue"
}
}
],
"use_json_for_http_post": true
}
Upgrade Procedure (continue logfile usage)
The switchover from the legacy event processor to the new event processor is not “atomic” if you continue to use the logfile event delivery. This means that there will likely be some duplicate events that reach the HTTP endpoint during the transition.
(1) Determine your new greenarrow.conf
configuration
Start by running the following command. If you are a candidate for the new
event processor, it will emit the greenarrow.conf
syntax needed to deliver
your events using the new event processor.
$ greenarrow events convert_legacy_configuration
event_delivery_destination destination-1 {
event_delivery_url `http://example.com`
event_delivery_events engine_open, bounce_bad_address.studio, studio_subscriber_created
event_delivery_disable_batching yes
event_delivery_format json
event_delivery_serialize_all_columns yes
}
If the above command gives you an error, it’ll describe the reason your configuration isn’t compatible for easy conversion to the new event processor.
Otherwise, copy and paste the event_delivery_destination block(s) to your
/var/hvmail/control/greenarrow.conf
configuration file.
(2) Reload configuration to activate the above changes
Run GreenArrow’s standard configuration reload to activate the changes you made above:
greenarrow_config reload
(3) Wait for the legacy event processor to be caught up
Because you need to continue generating legacy events (in order to continue
delivering to the logfile), the __legacy__
destination may not fully drain of
events at this point.
Run the following command to determine the current time as a UNIX timestamp:
/var/hvmail/postgres/default/bin/psql -U greenarrow greenarrow -c "
SELECT EXTRACT(EPOCH FROM NOW() AT TIME ZONE 'UTC')::bigint AS now_int
"
The above command will render an output like this:
now_int
------------
1667925710
(1 row)
This tells you the UNIX timestamp of a time after you ran greenarrow_config reload
.
Now, run the following command to determine the time of the oldest legacy event in your queue.
/var/hvmail/postgres/default/bin/psql -U greenarrow greenarrow -c "
SELECT COALESCE(MIN(event_time), 0) AS oldest_unwritten_event
FROM events
WHERE events.destination_id = 1
"
Here’s an example of what the output to the above command will look like:
oldest_unwritten_event
------------------------
1666267942
(1 row)
Run the above command until it reports either 0
(indicating that the queue
has drained) or a timestamp greater than the timestamp reported as now_int
above. This tells us that we’re ready to move onto the next step and disable
HTTP event delivery from the legacy event processor.
(4) Update the legacy event processor’s configuration
Now you can edit the following file:
/var/hvmail/control/event_processor.json
Since you’re going to continue using the legacy event processor for logfile event
delivery, your /var/hvmail/control/event_processor.json
file will look
something like this (although your filename
and filename_append_date
values
may be different than below):
{
"event_destinations": [],
"logfile": {
"filename": "/var/log/greenarrow-events.log",
"filename_append_date": false
},
"use_json_for_http_post": true
}