GreenArrow Email Software Documentation

Upgrade From Legacy Event Processor

Introduction

This guide will walk you through what you need to do to upgrade from the Legacy Event Processor to the “latest and greatest.” For an introduction to the event processor, and to see a description of what’s different about it – see the Event Processor documentation.

The legacy event processor was configured using a variety of different files:

/var/hvmail/control/record_events.simplemh.scomp
/var/hvmail/control/record_events.simplemh.bounce_all
/var/hvmail/control/record_events.simplemh.bounce_bad_address
/var/hvmail/control/record_events.simplemh.unsub
/var/hvmail/control/record_events.simplemh.open
/var/hvmail/control/record_events.simplemh.click
/var/hvmail/control/record_events.delivery_attempt.json
/var/hvmail/control/event_processor.json

And for Studio, events were controlled using database fields.

While the legacy event processor will continue to be supported in its current state (i.e. new features will not be added to it, but we will maintain support for it in the form of any relevant bug fixes), we encourage all customers to adapt to the new event processor as they are ready to make the migration.

The new event processor is controlled by our single configuration file:

/var/hvmail/control/greenarrow.conf

This guide will walk you through converting to the new event processor.

Questions About Upgrading

What are the advantages to upgrading?

  1. Deliver the same events to multiple destinations.

  2. Deliver in batches, leading to event delivery efficiency.

  3. Configure in greenarrow.conf, as opposed to many different configuration files.

  4. Events are delivered faster – typically within a few seconds of event occurrence.

Can I upgrade to the new event processor?

You can upgrade to the new event processor if none of the following is true:

  1. You are using an event destination that delivers using a database connection (type=custom_sql).

  2. You are writing events to a logfile.

What versions of GreenArrow support the new event processor?

GreenArrow v4.270.0 and beyond support the new event processor.

What if I’m using the legacy event processor’s ability to filter events by mail class?

The automatic configuration converter documented below does not handle this case. However, the new event processor does support this. See the event_delivery_filter_mail_class directive for more information, or contact GreenArrow Technical Support for assistance.

What if I’m using the legacy event processor’s ability to write events to a logfile?

We recommend that you continue using the legacy event processor for logfile generation, but migrate to the new event processor for any HTTP endpoints you have configured.

To accomplish this, you’ll follow the upgrade procedure below marked “Upgrade Procedure (continue logfile usage)”.

The switchover from the legacy event processor to the new event processor is not “atomic” if you continue to use the logfile event delivery. This means that there will likely be some duplicate events that reach the HTTP endpoint during the transition.

Upgrade Procedure (complete conversion – no logfile usage)

You can upgrade to the new event processor “atomically” – meaning that if you follow this procedure, no duplicate events will be generated (and no events will be missed).

(1) Determine your new greenarrow.conf configuration

Start by running the following command. If you are a candidate for the new event processor, it will emit the greenarrow.conf syntax needed to deliver your events using the new event processor.

$ greenarrow events convert_legacy_configuration
event_delivery_destination destination-1 {
  event_delivery_url `http://example.com`
  event_delivery_events engine_open, bounce_bad_address.studio, studio_subscriber_created
  event_delivery_disable_batching yes
  event_delivery_format json
  event_delivery_serialize_all_columns yes
}

If the above command gives you an error, it’ll describe the reason your configuration isn’t compatible for easy conversion to the new event processor.

Otherwise, copy and paste the event_delivery_destination block(s) to your /var/hvmail/control/greenarrow.conf configuration file.

Do not reload configuration at this time.

(2) Disable legacy event creation

Run the following to disable legacy event creation:

echo 0 > /var/hvmail/control/record_events.simplemh.scomp
echo 0 > /var/hvmail/control/record_events.simplemh.bounce_all
echo 0 > /var/hvmail/control/record_events.simplemh.bounce_bad_address
echo 0 > /var/hvmail/control/record_events.simplemh.unsub
echo 0 > /var/hvmail/control/record_events.simplemh.open
echo 0 > /var/hvmail/control/record_events.simplemh.click


$ /var/hvmail/postgres/default/bin/psql -U greenarrow greenarrow -c '
  UPDATE s_system_configs SET
    event_notification_studio_click=false,
    event_notification_studio_open=false,
    event_notification_studio_unsub=false,
    event_notification_bounce_all=false,
    event_notification_bounce_bad_address=false,
    event_notification_scomp=false,
    event_notification_studio_subscriber_created=false,
    event_notification_studio_subscriber_updated=false;
'

Edit the following file and set the enabled key to false:

/var/hvmail/control/record_events.delivery_attempt.json

(3) Reload configuration to activate the above changes

Run GreenArrow’s standard configuration reload to activate the changes you made above:

greenarrow_config reload

(4) Wait for the legacy event processor’s queue to drain

Monitor the following command, waiting for the __legacy__ event destination’s queue to drain to 0:

$ greenarrow events status
+------------------+-----------------------------+-----------------------+
| Destination Name | Event Count (First Attempt) | Event Count (Retries) |
+------------------+-----------------------------+-----------------------+
| destination-1    |                           0 |                     0 |
| __legacy__       |                         100 |                   n/a |
+------------------+-----------------------------+-----------------------+

(5) Remove the legacy event processor’s configuration

Now you can edit the following file:

/var/hvmail/control/event_processor.json

Typically you can replace its contents with this in order to disable event delivery:

{
  "event_destinations": [
    {
      "matches": {
        "all": true
      },
      "destination": {
        "type": "drop_from_queue"
      }
    }
  ],
  "use_json_for_http_post": true
}

Upgrade Procedure (continue logfile usage)

The switchover from the legacy event processor to the new event processor is not “atomic” if you continue to use the logfile event delivery. This means that there will likely be some duplicate events that reach the HTTP endpoint during the transition.

(1) Determine your new greenarrow.conf configuration

Start by running the following command. If you are a candidate for the new event processor, it will emit the greenarrow.conf syntax needed to deliver your events using the new event processor.

$ greenarrow events convert_legacy_configuration
event_delivery_destination destination-1 {
  event_delivery_url `http://example.com`
  event_delivery_events engine_open, bounce_bad_address.studio, studio_subscriber_created
  event_delivery_disable_batching yes
  event_delivery_format json
  event_delivery_serialize_all_columns yes
}

If the above command gives you an error, it’ll describe the reason your configuration isn’t compatible for easy conversion to the new event processor.

Otherwise, copy and paste the event_delivery_destination block(s) to your /var/hvmail/control/greenarrow.conf configuration file.

(2) Reload configuration to activate the above changes

Run GreenArrow’s standard configuration reload to activate the changes you made above:

greenarrow_config reload

(3) Wait for the legacy event processor to be caught up

Because you need to continue generating legacy events (in order to continue delivering to the logfile), the __legacy__ destination may not fully drain of events at this point.

Run the following command to determine the current time as a UNIX timestamp:

/var/hvmail/postgres/default/bin/psql -U greenarrow greenarrow -c "
  SELECT EXTRACT(EPOCH FROM NOW() AT TIME ZONE 'UTC')::bigint AS now_int
"

The above command will render an output like this:

  now_int
------------
 1667925710
(1 row)

This tells you the UNIX timestamp of a time after you ran greenarrow_config reload.

Now, run the following command to determine the time of the oldest legacy event in your queue.

/var/hvmail/postgres/default/bin/psql -U greenarrow greenarrow -c "
  SELECT COALESCE(MIN(event_time), 0) AS oldest_unwritten_event
  FROM events
  WHERE events.destination_id = 1
"

Here’s an example of what the output to the above command will look like:

 oldest_unwritten_event
------------------------
             1666267942
(1 row)

Run the above command until it reports either 0 (indicating that the queue has drained) or a timestamp greater than the timestamp reported as now_int above. This tells us that we’re ready to move onto the next step and disable HTTP event delivery from the legacy event processor.

(4) Update the legacy event processor’s configuration

Now you can edit the following file:

/var/hvmail/control/event_processor.json

Since you’re going to continue using the legacy event processor for logfile event delivery, your /var/hvmail/control/event_processor.json file will look something like this (although your filename and filename_append_date values may be different than below):

{
  "event_destinations": [],
  "logfile": {
    "filename": "/var/log/greenarrow-events.log",
    "filename_append_date": false
  },
  "use_json_for_http_post": true
}

 Legacy Event Processor
Network Services 

Copyright © 2012–2023 GreenArrow Email