Server Migrations
- Table of Contents
- Intro
- Prerequisites
- Non-GreenArrow Files and Services
- Migration Procedure
- Post-Migration Configuration
- Final Testing
- Disable Old Server
- Let Us Know You’ve Migrated
Intro
This document describes how to migrate a GreenArrow installation from one server to another. Please contact GreenArrow technical support if you would rather have GreenArrow migrate your GreenArrow installation.
Prerequisites
The migration procedure described on this page has some prerequisites:
-
An experienced Linux systems administrator who can execute the migration procedure.
-
The server that you’re migrating from must be running GreenArrow 4.200.0 or later. If you’re using an older release, then see these instructions.
Here’s a command you can run to check what version is installed:
rpm -q greenarrow 2> /dev/null || dpkg -l greenarrow 2> /dev/null || echo "GreenArrow is either not installed, or is older than version 4.200.0."
-
The server that you’re migrating to must run a Linux distribution from the same family as the server that you’re migrating from. Here are two examples:
- If your existing server runs CentOS 8, then the migration must be made to a Red Hat-based distribution, such as Alma Linux 9.
- If your existing server runs Ubuntu 18.04, then the migration must be made to a Debian-based distribution, such as Ubuntu 22.04 or Debian 12.
-
Most GreenArrow installations use PostgreSQL 9.5, which can run on all of our supported Linux distributions. Some installations use PostgreSQL 8.3, though, which comes with Linux distribution restrictions.
You can check which version of PostgreSQL your existing server is running by checking where the
/var/hvmail/postgres/default
symlink points to. If it points to/var/hvmail/postgres/9.5
, then you’re running PostgreSQL 9.5. If it points to/var/hvmail/postgres/8.3
, then you’re running PostgreSQL 8.3:ls -l /var/hvmail/postgres/default
If you’re running PostgreSQL 8.3, then the following restrictions apply:
-
PostgreSQL 8.3 is only supported by Red Hat-based distributions, like CentOS 7 and 8.
-
PostgreSQL 8.3 can run on RedHat 8.x-based Linux distributions like CentOS 8, but will not support using TLS for incoming connections. If you wish to use both PostgreSQL 8.3 and TLS for incoming PostgreSQL connections, then you must run a RedHat 7.x or earlier Linux distribution, like CentOS 7.
Please contact GreenArrow technical support if you’d like us to migrate you from PostgreSQL 8.3 to 9.5.
-
-
The server should have enough disk space to hold the migrated data. The vast majority of GreenArrow’s data is located within the
/var/hvmail
directory and the migrated data will occupy approximately the same amount of disk space after the migration as it did before.If the specs of the destination server are identical to or greater than the server that you’re migrating from, and you didn’t have any disk space issues before the migration, then you’re covered.
-
The
/var/hvmail/control/UPGRADE_BLOCKER.txt
file must not exist. This file is only present on GreenArrow installations where non-standard steps are required to perform updates of GreenArrow’s software. Those same non-standard steps will need to be performed during the migration. Contact GreenArrow technical support to arrange to have those steps applied.
Non-GreenArrow Files and Services
These migration instructions do not take into account any non-GreenArrow files or services that you may also wish to migrate.
Migration Procedure
Some of the steps below are Linux distribution specific. For the sake of brevity, we refer to Red Hat-based Linux distributions, such as CentOS, Red Hat Enterprise Linux or Scientific Linux as “CentOS”, and Debian-based distributions, such as Ubuntu and Debian as “Debian”.
Here’s how to migrate GreenArrow to a new server. All steps should be completed on the new server unless otherwise indicated:
-
If you’re adding, removing or changing any IPs as part of the migration, then consider configuring their DNS records and feedback loop registrations in advance. Instructions for adding new IPs are here.
-
Authorize the new server’s
root
account to SSH into the old server asroot
. -
Log in to the new server as the
root
user using the bash shell. -
Install packages that are used by the migration process:
On CentOS systems, run:
yum install ruby rsync
On Debian systems, run:
apt-get install ruby wget gnupg rsync
-
Make an entry in the new server’s
/etc/hosts
file, pointing theoldserver
hostname to an IP on the old server that has an SSH server running. For example:echo "1.2.3.4 oldserver" >> /etc/hosts
This entry is needed because we’re going to run some commands later on which connect to
oldserver
. -
Verify that you can SSH into the old server from the new server by running:
ssh oldserver
If a non-default port is used for ssh then you can tell the SSH client to use it by creating or updating
/root/.ssh/config
. For example:Host oldserver Port 2222
-
Take a backup of the old server. A migration normally only uses some of the files that get created by a backup, but we recommend that you hold onto the entire backup directory to increase the odds of being able to recover if something goes wrong during the migration. For extra safety, you may want to copy this directory to some media that’s not attached to either server.
-
Copy the following files from the backup directory to the
/var/migrate/
directory on the new server:- greenarrow-packages.txt
- greenarrow-repository-access-key.txt
- hvmail_reinstall_packages
- postgresql-tablespaces.txt
-
Check the
/var/migrate/postgresql-tablespaces.txt
file for any tablespaces which specify a location.Here’s an example of what this file might look like. In this example, the tablespace named “data2” is the only one which has a location (
/media/data2
) specified:List of tablespaces Name | Owner | Location ------------+------------+------------------------------------ data2 | greenarrow | /media/data2 pg_default | postgres | pg_global | postgres | (3 rows)
If there are no tablespaces which specify a location, then you can move onto the next step.
If there are any tablespaces which specify a location, then re-create their directories on the new server. For example:
mkdir /media/data2
Also,
rsync
each of the directories identified above from the old server to the new server. You should both run eachrsync
invocation now, and record it so that it can be run again later afterhvmail_server_migration
has run to completion. For example:rsync -a --delete oldserver:/media/data2/ /media/data2
-
View the contents of the
/var/migrate/greenarrow-repository-access-key.txt
file. This key is needed for the next step. -
Complete the Prerequisites section of our Installation Guide. If you get to the “Install GreenArrow” step in the Installation Guide, you’ve gone too far.
-
Patch
hvmail_reinstall_packages
:sed -i 's|yum -y --enablerepo=greenarrow install|yum -y --setopt=obsoletes=0 --enablerepo=greenarrow install|g' /var/migrate/hvmail_reinstall_packages
-
Install GreenArrow’s packages on the new server:
/var/migrate/hvmail_reinstall_packages /var/migrate/greenarrow-packages.txt
If the above command encounters an error that you’re able to correct, then it’s safe to run it again.
-
GreenArrow packages make some changes to environment settings. To activate the changes, either log out, then back into the server or run the following commands. If you ’re logged into the server in multiple windows, then you’ll need to complete this step in each window:
. /etc/profile.d/greenarrow-engine.sh . /etc/profile.d/greenarrow-ruby.sh . /etc/profile.d/greenarrow-studio.sh . /etc/profile.d/greenarrow-passenger.sh
-
Run the migration script for the first time. This can safely be done while GreenArrow is still running on the old server:
/var/hvmail/libexec/hvmail_server_migration --no-exit-on-error
This migration script could take anywhere from minutes to hours to run, depending on how much data there is to transfer. It’s safe to run the script multiple times, as long as only one instance is running at any given point in time.
rsync
is used for most of the data transfer, so all other things being equal, the closer together two invocations of this script are to each other, the faster the second invocation will run. -
Stop GreenArrow on the old server:
systemctl stop greenarrow.service systemctl disable greenarrow.service
-
Run the migration script again. This will synchronize any files which changed since the last time the script was run:
/var/hvmail/libexec/hvmail_server_migration
If the migration script completes without error, then its last line of output will read
No errors were detected
. If the last line contains anything else, review the remaining output for issues. Once any issues have been corrected, it’s safe to re-runhvmail_server_migration
. -
If you identified any additional
rsync
invocations to run while you were inspectingpostgresql-tablespaces.txt
earlier, run them now. -
If there are any IP addresses to move from the old server to the new one, move them now.
-
This step is not needed for Let’s Encrypt certificates that are configured automatically.
If you’re using the Legacy Let’s Encrypt configuration method, then remove all of that method’s certificates from the HTTP TLS configuration file. These certificates are stored outside of GreenArrow’s directories, so they are not migrated by this procedure. Take note of which certificates you’ve removed in case you wish to add them back in later. There’s a step later in this procedure for doing that.
-
Start GreenArrow on the new server:
systemctl start greenarrow.service systemctl enable greenarrow.service
-
On Debian systems, disable the GreenArrow package repos:
sed -i 's/^deb /#deb /' /etc/apt/sources.list.d/greenarrow.list
-
Check whether
greenarrow.conf
has dns_cache_service_run set:grep dns_cache_service_run /var/hvmail/control/greenarrow.conf
-
If
dns_cache_service_run
is set totrue
oryes
, then configure the/etc/resolv.conf
file so that the onlynameserver
line is set to127.0.0.1
. For example:echo "nameserver 127.0.0.1" > /etc/resolv.conf
-
If
dns_cache_service_run
is set tofalse
orno
, or is not specified, then configure the/etc/resolv.conf
file to query one or more recursive DNS resolvers. For example, if you would like to query the Google and Cloudflare public DNS servers, you could populate the file with:echo "nameserver 8.8.8.8 nameserver 8.8.4.4 nameserver 1.1.1.1" > /etc/resolv.conf
-
-
Verify that different nameservers aren’t configured outside of
/etc/resolv.conf
. If the commands below return any output, then verify that it matches your desired configuration:On CentOS systems, run:
find /etc/sysconfig/network-scripts/ifcfg-* /etc/sysconfig/network -type f | xargs grep 'DNS=' | grep -v 'PEERDNS='
On Debian systems, run:
grep dns-nameservers /etc/network/interfaces
-
Check if the
/var/hvmail/control/UPGRADE_BLOCKER.txt
file exists:cat /var/hvmail/control/UPGRADE_BLOCKER.txt
This file is only present on GreenArrow installations where non-standard steps are required to perform updates of GreenArrow’s software.
If the file is missing, then you can move onto the next step.
If the file is present, then contact GreenArrow technical support to have those non-standard steps applied.
-
Complete the Performance Tuning section of our Installation Guide.
Post-Migration Configuration
-
If you’re adding, removing or changing any IPs as part of the migration, then they should be added into GreenArrow’s configuration now. Instructions for adding new IPs are here.
-
This step is not needed for Let’s Encrypt certificates that are configured automatically.
Optionally, add Legacy Let’s Encrypt certificates to the HTTP Server’s TLS configuration.
-
If there are any other configuration changes desired as part of this migration, make them now.
Final Testing
-
Verify that some common misconfigurations are not present. Each test gets a simple pass/fail result:
hvmail_check_config
-
Verify that all services are running normally. Any services with an abnormal state are shown in red:
service greenarrow status
The
hvmail-qmail-smtpd3
andhvmail-dnscache
services are down by default. All other services should be reported as up unless you shut them down intentionally. -
Send yourself a test email (replacing [email protected] with your actual email address):
date | /var/hvmail/bin/mailsubj "GreenArrow Test Message" [email protected]
-
Remove the entry that you made earlier for
oldserver
in the new server’s/etc/hosts
file.
Congratulations, you’re finished migrating GreenArrow to the new server! As a next step, we recommend following up with GreenArrow technical support to request that we review the migrated GreenArrow installation for issues.
Continue to the next section to prevent the old server from coming back online, and attempting to send duplicate emails and/or use the same license key in two locations at the same time.
Disable Old Server
-
Remove GreenArrow’s licenses from the old server:
rm -f /var/hvmail/control/license_key /var/hvmail/control/bounce.boogie_studio_api_license_key
-
Delete the queued messages on the old server:
find /var/hvmail/qmail-ram/savedqueue -type f -delete find /var/hvmail/qmail-bounce/savedqueue -type f -delete cd /var/hvmail/qmail-disk/queue/ find info -type f -delete find mess -type f -delete find remote -type f -delete find intd -type f -delete find local -type f -delete find todo -type f -delete find bounce -type f -delete find batches -type f -delete
Let Us Know You’ve Migrated
Contact GreenArrow technical support to update your server information. This is not required, but it helps prevent potential licensing issues.