cPanel Conference: Migrating Data Between cPanel Servers

Posted on

A few weeks back, I had the opportunity to speak at the cPanel Conference in Houston and presented an in-depth look at strategy and techniques for migrating data between cPanel servers at all scopes. I talked about ways to prepare for a server migration to ensure site functionality, security, and performance upon arrival, as well as account completeness, common sense defaults, and pitfalls.

My speech was very well received, and a lot of folks found the presentation to be very insightful. So, I wanted to share some of that information with you as well, and hope you find it useful.

Enjoy!

Considerations for cPanel Migrations

Often, the focus during cPanel migrations is on the wrong elements. The most common request is to ensure that every file and database is copied over to the new server accurately, which by itself is a relatively trivial task. However, the real magic of migrations is in ensuring service compatibility, which will allow sites to load on the new server just as they did on the old server.

One of the first things to do when considering a migration is to consider DNS. For most migrations, particularly those which technology or host changes, a change in IP address is required. Because of this, you will need to understand where your nameservers lie for each domain involved. You should also take the time to lower the TTL values for your zone files as low as possible so that when the time does come to update DNS, changes will propagate quickly. Another item to consider is where your nameservers are hosted. If they are on the old machine, then once it is terminated, the nameservers will no longer be accessible, so plans should be made to move the DNS to the new server.

The sites to be migrated can also be prepared for migration in an optional step. Take out the trash to reduce the quantity of data to migrate, and update any sites that are using an out-of-date Content Management System (CMS). This will help ensure compatibility with new default minimum versions on new servers, just in case matching versions are unavailable.

Check Mark Like this content? Subscribe now to get the latest content straight to your inbox.

Ensuring Compatibility

Now, work can begin on ensuring compatibility, a critical step in migration. A good look should be taken at the core services of the servers and determine whether the upgrade path will remove or change any functionality. Some of this determination can only come with experience and knowledge of these core services, but generally, your CMS vendor will be able to tell you minimum or maximum versions of certain software.

cpanel considerations

For instance, modern WordPress has a recommended PHP version of 7.2 but will work with versions as old as 5.2.4. However, very old versions of WordPress are programmed with functions that are not compatible with PHP 5.6 or greater, which is why updating your code and plugins is recommended when starting a migration.

Some websites have strict requirements for older versions of software, and upgrades are not always possible. For these types of legacy applications, support for End Of Life software is necessary, so consider adding CloudLinux to your server, which, among other features, allows installation of hardened PHP versions to augment those available through your control panel.

What Should Be Checked First, and How is All of this Checked on Your Source Server?

Many favor to use the command line to investigate this information:

  • cat /etc/redhat-release
  • df -h
  • df -i
  • du -hs /var/lib/mysql
  • cat /proc/cpuinfo
  • free -m

These commands will check, in order, the version of CentOS on the server, indicating the general age of the system; the total disk usage of all partitions; the inode (file and folder) count; the disk usage of the /var/lib/mysql folder, as well as its subfolders, hinting at any very large databases; the processor architecture and core count; and the total memory on the server, in megabytes.

This look at hardware helps us size up the target system and make sure there is enough room and power for the data we want to move.

Check Mark Need to migrate but not sure which solution works best for your needs? Download our Buyer’s Guide to help you make the right choice.

LAMP Software

This set of commands will look at the remainder of the LAMP stack:

  • php -v
  • php -m
  • httpd -v
  • httpd -L
  • httpd -S
  • mysql --version

 

Here, the output shows the primary PHP version and its installed modules; the Apache version, modules, and running sites; and the MySQL version. These are compared to similar output from the target server and matched up where possible.

commands for cpanel migrations

Because WHM utilizes EasyApache to make things easy, the act of copying a running EasyApache configuration into another server is made even easier by use of a certain set of scripts. When utilizing EasyApache 4 on the source server, the following command will export a single JSON file listing out all of the installed packages:

  • /usr/local/bin/ea_current_to_profile --output=/home/ea4-migration-profile.json

If you happen to be using the older EasyApache 3, the necessary information is stored in a YAML file instead and can be converted using a utility on the new server, which should be running EasyApache 4. Bring the file /var/cpanel/easy/apache/profile/_main.yaml over to the target machine, and then use this command to turn it into an EasyApache 4 profile JSON file:

  • /scripts/convert_ea3_profile_to_ea4 /home/_main.yaml /home/ea4-migration-profile.json

Both of the output files can then be restored as a profile on the target server by using the same command:

  • ea_install_profile --install /home/ea4-migration-profile.json

For good reason, WHM disallows downgrades of the MySQL server, but generally, database dumps from mostly any version can be made to work with modern MySQL. The issue that gets encountered more frequently is old password hash support being removed in version MySQL 5.6 or greater; passwords stored in the old hash format will have to be re-entered manually before they can work.

WHM Settings

Just as your clients would expect consistent behavior from the server, you would expect consistent behavior from WHM. So, the settings in WHM will need to be researched and duplicated. WHM has a built-in tool to help with this import and export procedure that can be leveraged to copy the configuration for one WHM server into another.

  • /usr/local/cpanel/bin/cpconftool --backup

The backup is stored in the /home/ directory on the old server, and can then be copied to the new server and restored with this command:

  • /usr/local/cpanel/bin/cpconftool --restore=/path/to/filename.tar.gz

There are other items outside of this backup script that should also be attended to, including the Packages and Features levels created inside of WHM, Communication preferences, Backup preferences, and any Plugins installed in WHM. Packages and Features are rather easy to move: their respective folders can be directly copied to the new server in the appropriate location. These two commands run on the target server will accomplish that, assuming 123.45.67.89 is your old server’s IP address:

  • rsync -aqHz 123.45.67.89:/var/cpanel/packages /var/cpanel/
  • rsync -aqHz 123.45.67.89:/var/cpanel/features /var/cpanel/

Communication and Backup preferences should be checked from inside the WHM interface. Though they are both available from the command line, the commands are a bit outside the scope of this post, and they are simple enough to copy manually.

Other Services

With Linux, it is possible to install anything you want anywhere on the system you want, making researching extra software or services difficult. Ideally, you will already know what software your sites depend on and what is installed on the source server to support them, though this is not always the case.

There are a few important commands to remember:

  • ps faux
  • netstat -plunt

The first command shows all running processes on a server, which can be selectively parsed for particular lines such as Varnish or cPHulk, and the second command shows listening services, which should export a few dozen lines that can be scanned for alternative listeners like Nginx, Litespeed, or Memcache.

LAMP Configurations

There are a variety of settings that can be modified from within WHM in regards to MySQL, Apache, and PHP. It is not recommended to copy the configuration files for which the settings are saved outright to the new server, as there may be minor variations between builds or versions that can cause applications to not start.

lamp stack - showing a server

There is also the matter of changes in resource levels. If an upgrade in servers is being performed, then larger or smaller limits may be necessary to avoid memory issues or utilizing hardware effectively.

To be safe, I recommend that any minimum variables be set for your site software, and then tuning be performed after the new server is taken live.

Duplicating Accounts

Now that the environments are in sync, we have finally arrived at the core commands for moving data!

There are two core commands that need to be run to restore the accounts. In essence, we are packaging up the needed cPanel accounts, copying the backups, and restoring them on the new server. But, to save space and time, we will use a few extra tricks.

First, it is recommended to making a working directory:

  • mkdir /home/migrationtemp

For each user (we will use username in our example), the following three steps will need to be taken:

  • /scripts/pkgacct --skiphomedir username /home/migrationtemp/

This will package up the account, skipping the very large home directory, and place the backup in /home/migrationtemp/.

Why skip the home directory?

It takes a lot of CPU resources and disk space to copy the home directory into the backup, compress it, then copy it over to the new server, decompress it, and copy it once again into the appropriate location. Since the folder is not changing the structure, it can be copied directly to the new server separately and directly, skipping all the compression steps, and saving disk space on both servers.

Once the backup is ready (it should be very quick), it can be copied to the new server using your favorite method (for single files, I prefer scp), and then restored:

  • /scripts/restorepkg /home/migrationtemp/cpmove-username.tar.gz

This will bring back the entirety of the cPanel account on the new server, save the home directory. Let’s address that now. This command is run on the new server:

  • rsync -avHPz 123.45.67.89:/home/username /home/

Make sure that you pay attention to those trailing slashes! This command syncs the /home/username folder and its contents into the /home/ folder on the new server, preserving permissions, acting verbosely, and compressing data.

We also have more information on the rsync command if you need it.

We’re done! On to testing!

Testing Your Migration

There is really one way to test a migrated website accurately: by altering the hosts file on your workstation. This will essentially ‘trick’ your browser into thinking that DNS is already updated by bypassing nameserver lookups. Public traffic will continue to be routed to the original server, but your computer will be browsing the new server.

testing your migration

The new server is configured to accept requests for the real domain name as soon as the cPanel account is restored, so it will respond as is would as if it were live. As far as your computer and the new server are concerned, this is ‘real’ traffic.

We have an article available for modifying your hosts file that is geared towards several different operating systems since the procedure varies by version and vendor.

What Are You Testing For?

Anything weird. We recommend looking at the main page, a few different subpages, pages that parse important information or have high visibility, order forms or carts, and the backend of your site as well.

You can be as thorough here as you like, and if you should run into any issues, you can follow normal troubleshooting techniques for the target server. Knowing that the errors are not yet live really takes the stress off.

The Final Sync

For cPanel to cPanel migrations with root access, updating any changed information is very straightforward. New database dumps should be taken and imported, and the home directories should be resynced one more time. This is all followed by a DNS update.

As you recall, we looked at the inode count on the old server as well as the disk usage of the MySQL directory /var/lib/mysql. These two items together help us project the length of time the final sync might take.

Unless you have databases that are over, say, 50GB, or /home directory inodes well over 2 million (indicating a lot of files to compare), a final sync can normally be executed in less than an hour, or two at most.

Because the point of the final sync is to bring everything up-to-date on the new server that may have changed during testing, it is important to ensure that no further changes are made on the origin server, either by content creators (posts) or by visitors (orders or comments). This will aid in the accuracy of the content that is made live on the new machine. Therefore, it is best to interrupt services that allow sites to be visited, incurring downtime to the websites and email.

These commands will stop the necessary services on the old server:

  • /usr/local/cpanel/bin/tailwatchd --disable=Cpanel::TailWatch::ChkServd
  • service httpd stop
  • service cpanel stop
  • service exim stop

This will prevent websites from being viewed, email from being delivered, and cPanel or WHM from being accessed. You can see why it is important to select a time when traffic will be low! MySQL will need to be kept running, as we will need it to perform database dumps.

There are now just a handful of commands to consider, all of which are run on the target machine:

  • rsync -avHPz 123.45.67.89:/home/username /home/ --update
  • mysqldump username_db | gzip > \
    /home/migrationtemp/username_db.sql.gz
  • ssh -C 123.45.67.89 “mysqldump username_db” | mysql username_db

 

The first command modifies our original rsync command to add the –update flag, which will compare timestamps on files to be synced and only copy those files which are newer on the source server.

The next command backs up the database on the target server to a zipped file, just in case something goes awry, and the final command will stream a dump of a database called “username_db” from the old server directly into the new server.

Update DNS

The final step in the migration is to let your visitors know to access the new server. This is done with a change in the A record in your DNS zone files, wherever they may be. Since we lowered the TTL values at the start of the migration, this new A record should propagate around the internet very quickly.

Once DNS is updated and propagated, your new server will be serving content, and your migration is complete! Make sure to turn on your local and/or remote backup solution, and terminate your old server. If you need to move your nameservers off of the old server, updates at the domain registrar will be needed to change the glue records.

Need Help Migrating Your Server?

Learn more about our migrations and download our free Buyer’s Guide to help you choose which solution works best for your needs. Contact us today to see how we can help!

Want more news and updates like this straight to your inbox?

Keep up to date with the latest​ ​Hosting news.