Install Rsync and Lsync on CentOS, Fedora or Red Hat

Have you ever needed to copy files from your local computer over to your web server? You may have previously used File Transfer Protocol (FTP) applications for this task, but FTP is prone to being insecure and can be challenging to work with over the command line. What if there was a better way? In this tutorial, we’ll be covering two popular utilities in the Linux world to securely assist in file transfers, rsync and lsyncd. We’ll show you how to install and use both in this article. Let’s dig in!

 

What is Rsync?

The first utility we’ll look at is called rsync, and this command is a real powerhouse! Most simply, it is used for copying files between folders, but it has some extra features that make it very useful. We’ll start with the most basic usage for rsync, and work into more complicated examples to show you how versatile it can be.

 

Install Rsync on CentOS, Fedora, or Red Hat

If you are using CentOS, Fedora, or Red Hat, you can use the yum package manager to install:

Note:
You’ll need to be the “root” user to install packages!

yum install rsync

 

Install Rsync on Ubuntu or Debian

If you are using Ubuntu or Debian, you can use the apt-get package manager to install:

Note:
You’ll need to be the “root” user to install packages!

apt-get install rsync

 

How to Use Rsync

The syntax for running rsync looks like this:

rsync -options <source> <destination>

You are not required to specify “options”, but you’ll always need to tell rsync a source and destination.  In this example, we’re using rsync to copy “file.txt” into the “/path/of/destination/” folder:

rsync /path/of/source/file.txt /path/of/destination/file.txt

Important
When you run rsync, remember always to put the “source” first, and “destination” second!

Now that you have the basics let’s try another common task. In this example, we’re going to use rsync to copy a directory from our local computer over to our web server “192.168.0.100”.

rsync -avHl /path/of/source/folder root@192.168.0.100:/path/to/destination/folder

Notice how just as before, we specified our source first, and destination second. One of the great things about rsync is that it performs remote transfers of data securely, through SSH. Using SSH is fantastic from a security point of view, and it allows you to use SSH keys to avoid typing passwords.

As one last example, let’s try copying something from your remote server “192.168.0.100” over to your local machine. Once again, rsync is the tool for the job!

rsync -avH  root@192.168.0.100:/path/of/remote/folder /path/of/destination/folder

We used some special options in that last example. Let’s break them down.

-a = archive mode (includes several commonly used options: -rlptgoD, check the rsync man page for detailed info.)

-v = print verbose messages to screen, (very helpful!)

-H = preserve hard links when copying

One of the great things about rsync is that it intelligently copies files. If only the last few bits of a file has changed, rsync solely copies the changes, rather than the whole file. Transferring only the changed parts of a file can be a huge time saver, but especially when copying files remotely like in that last example.

 

Introducing Lsyncd

Finally, we’ll talk about lsyncd. The utility lsyncd is somewhat similar to rsync in that it is used to synchronize two folders. Unlike rsync, which has to be run manually, lsyncd is a daemon. Sounds scary, but in computer terminology, daemons are merely applications that run as a background process. You usually don’t have to manually run daemons every time you want to use them, as they are typically configured to start automatically when your server boots. When you configure lsycnd correctly, it can automatically synchronize folders between two computers. Imagine if you didn’t have to manually create backups of your website on your web server every time you made a small change? That could be a real time saver! Let’s dig in.

 

Install Lsyncd on CentOS, Fedora, or Red Hat

If you are using CentOS, Fedora, or Red Hat, you can use the yum package manager to install:

Note
You’ll need to be the “root” user to install packages!

yum install lsyncd

 

Install Lsyncd on Ubuntu or Debian

If you are using Ubuntu or Debian, you can use the apt-get package manager to install:

Note
You’ll need to be the “root” user to install packages!

apt-get install lsyncd

 

How to Use Lsyncd

Unlike rsync, lsyncd runs as a daemon. You don’t run it directly. Instead, it starts automatically with your server at boot time, and runs silently in the background. It’s a great automated way to sync folders on your server! All we need to do is configure lsyncd, so it knows what folders to sync.

First, we’ll create some files and folders.

Create the configuration folder location
mkdir /etc/lsyncd

Create a folder to sync FROM, feel free to name it what you would like
mkdir /source_folder

Create the folder to sync TO
mkdir /destination_folder

Create the log folder location
mkdir /var/log/lsyncd

Make the log file
touch /var/log/lsyncd.log

Create the status file
touch /var/log/lsyncd.status

This is an example file for our sync tutorial
touch /source_folder/hello_world.txt

Next, we need to configure lsyncd to use our newly created files. Open a new file for editing using your favorite Linux text editor.

vi /etc/lsyncd/lsyncd.conf.lua

Paste in the following configuration, and save the file.

settings = {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status"
}
sync {
default.rsync,
source = "/source_folder",
target = "/destination_folder",
}

Now all that remains is to start the program! First, we’ll tell lsyncd to start automatically when your server boots:

systemctl enable lsyncd

Next, we’ll start lsyncd manually, (this only needs to be done once.)

systemctl start lsyncd

We’ve successfully installed lsyncd, but it is good practice to double check your work. We’ll check to see if it is running using this command:

systemctl status lsyncd

If you see a line that reads: active (running), it is running correctly!

Finally, we check the contents of /destination_folder to make sure that it contains our “hello_world.txt” file.

ls -l  /destination_folder
total 0
-rw-r--r-- 1 root root 0 Nov 17 07:15 hello_world.txt

 

You should see that “hello_world.txt” has been automatically synchronized over to “/destination_folder”. Everything is working! In practice, you can set “/source_folder” and “/destination_folder” to any folders you need synchronizing.

As you can see, these two utilities rsync and lsyncd are great tools for copying files and folders in Linux. Have questions about how to use these tools on your Liquid Web web server? Reach out to the Most Helpful Humans in Hosting. We’re here to help, 24×7!

 

Setup a Development Environment in Ubuntu

Often we want to edit our domain’s code, but on a production website, this can be dangerous. Making changes to the production site would not only allow all of the Internet to see unfinished changes but could also cause errors to display. As a workaround, we’ll create a testing domain or “dev” domain to work out any bugs and changes to the site.

As a warning, this is advanced technical work. It’s possible to make mistakes and cause downtime on your live domain. If you are not 100% confident, it may be a good idea to hire a system admin or developer to copy the domain for you.

Pre-flight

Step 1: After logging in as root, back up the domain using the command below to save a compressed version. It’s essential to have a copy of your site just in case you run into any issues. Replace the brackets and the “domainname” with your site’s name.

tar cfv backup.[domainname].tar /document/root/of/the/domain/

 

Step 2: We will be moving our backup file, .tar, somewhere off that document root, This way we can still access the data while keeping it safe.

mv backup.[domainname].tar /another/location/on/the/server

 

Step 3: The tar command created a file that comprises only half of the site, which holds the files. Next, you will want to copy the database for the primary site.

mysqldump [database_name] > /another/location/on/the/server/backup.[database_name].sql

 

Step 4: Create the new dev account, just like you were adding a new domain to your server. However, you will be creating a subdomain which you can name as dev.[domain].com.

 

Step 5: Once you’ve created the dev subdomain, copy the files from the main document root to the newly created dev document root. The document root is the location of the files housing your domain, also known as the “path”.

To find the document root location, for either domain, run the following command. Find the document root by replacing “exampledomain.com” for each both the live and dev domain.

conf=$(apache2ctl -S | grep exampledomain.com | perl -n -e '/\((\/.*)\:/ && print "$1\n"'); grep -i documentroot $conf

 

Step 6: After you have found the document roots for both the domain and dev domain, you will need to copy the files. With the output of the last command, you’ll insert the path for the document roots.

rsync -avh /document/root/of/the/domain/ /document/root/of/the/new/dev/domain

 

Step 7: Give the correct ownership to the dev domain, as the previous username remains in place. The ‘dev_username’ is a default user, one generated by creating the new account. The following command will change the ownership for you.

chown -R [dev_username]: /document/root/of/the/new/domain

 

Step 8: After you have done this you will need to create a new database and database user for the dev domain, be sure to keep all this information including the password you set. First, enter the mysql command prompt by running the command:

mysql

Next, create the database itself.

CREATE DATABASE [new_database_name];

In our next command, we create the user and their password. Be sure this is a unique username, unused up till now.

CREATE USER '[select_new_username]'@'localhost' IDENTIFIED BY '[password]';

 

Step 9: Grant privileges for the user on the database.

GRANT ALL PRIVILEGES ON [new_database_name] TO '[new_username]'@'localhost' WITH GRANT OPTION;

Then exit out of mysql by typing “quit” and hitting enter:

quit

 

Step 10: Once back at the root command prompt, you can start copying the original database into the new database.

mysqldump [new_database_name] < /home/temp/backup.[database_name].sql

With the database dumped you will still need to edit the configuration files for your domain. Typically, there are files which need to access the database and will reference the database, database user, and password. These credentials need updating to the ones created earlier in step 8 of this tutorial. If you are unsure of where these files contact a developer. If you are working with a WordPress site continue on to the next section.  Otherwise, if you have updated your dev configuration files with your new database info continue onto step 11.

Editing WordPress Configurations

Fortunately, WordPress is one of the most commonly used content management systems. WordPress is easy to configure we’ll provide a short tutorial on how to change the database and database user in the wp-config.php. First, move to the new document root.

cd /document/root/to/the/dev/domain

There you will edit the wp-config.php with your favorite text editor such as vim or nano.

nano wp-config.php

In the wp-config.php file, you will see a section that looks like the below. From there you will edit the highlighted characters with the information you used to create the database in the tutorial.

// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', 'new_database_name');
/** MySQL database username */
define('DB_USER', ‘new_database_user');
/** MySQL database password */
define('DB_PASSWORD', 'password');

In the database, clear all mentions of the original domain and replace them with the dev domain. For example with WordPress, in the _options table you need to change two entries of ‘home’ and ‘siteurl’. These can be quickly changed using WP-ClI, which is a is a command line tool for interacting with and managing WordPress sites.  To install WP-CLI follow these instructions and continue onto the next step.

If you do have a WordPress website, once you have installed WP-CLI you will want to run the following commands:

su - [dev_username] cp public_html
wp option update siteurl https://dev.domain.com
wp option update home https://dev.domain.com
exit

Sometimes plugins or themes mention the original domain in the database. If some parts of the dev domain are not working, particularly plugins or themes, you may need to contact a developer to see if the original domain name is still active in the database.

After replacing the names using WP-CLI, you’ll have officially created a dev domain.

Step 11: To complete this tutorial you have two choices: add an A record to your DNS to view your dev site online or edit your local hosts file to view solely on your computer. For our Liquid Web customers feel free to contact The Most Helpful Humans™ with questions you may have in setting up a development environment.

How To Install lsyncd on Fedora 23

Lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check

  • These instructions are intended specifically for installing the lsyncd on Fedora 23. For Fedora 22, visit this tutorial. If you’re on another operating system, you can find instructions for installing lsyncd on Ubuntu 14.04 and CentOS 7
  • We’ll be working as root on a Liquid Web Self Managed Fedora 23 server.

Step #1: Install lsyncd

First, we’ll adhere to best practices by ensuring that the list of available packages is up to date:

dnf -y update

Now, we can install lsyncd with:

dnf -y install lsyncd

Step #2: Example Configuration File Locations

Example configuration files can be found in the following directory:

/usr/share/doc/lsyncd/examples

A basic configuration can be found in lrsync.lua:

/usr/share/doc/lsyncd/examples/lrsync.lua

To view or edit lrsync.lua:

vim /usr/share/doc/lsyncd/examples/lrsync.lua

In vim, remember to press “a” to enter text insertion mode; pressing the escape key (Esc) on your keyboard returns you to command mode. For a refresher on editing files with vim, see New User Tutorial: Overview of the Vim Text Editor. If vim is not installed on your OS, you can follow our tutorial on installing vim at How to Install VIM (Visual editor IMproved) on Fedora 23.

The lsyncd configuration file is written in Lua. Learn more about the Lua programming language at http://www.lua.org/docs.html.

How to Install lsyncd on Fedora 22

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check

  • These instructions are intended specifically for installing the lsyncd on Fedora 22.
  • I’ll be working from a Liquid Web Self Managed Fedora 22 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Fedora 22”

How to Install lsyncd on Fedora 21

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check

  • These instructions are intended specifically for installing the lsyncd on Fedora 21.
  • I’ll be working from a Liquid Web Self Managed Fedora 21 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Fedora 21”

How to Install lsyncd on Fedora 20

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on Fedora 20.
  • I’ll be working from a Liquid Web Core Managed Fedora 20 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Fedora 20”

How to Install lsyncd on CentOS 7

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on CentOS 7.
  • I’ll be working from a Liquid Web Core Managed CentOS 7 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on CentOS 7”

How to Install lsyncd on CentOS 6

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on CentOS 6.
  • I’ll be working from a Liquid Web Core Managed CentOS 6 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on CentOS 6”

How to Install lsyncd on Ubuntu 12.04 LTS

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on Ubuntu 12.04 LTS.
  • I’ll be working from a Liquid Web Core Managed Ubuntu 12.04 LTS server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Ubuntu 12.04 LTS”

How to Install lsyncd on Ubuntu 14.04 LTS

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on Ubuntu 14.04 LTS.
  • I’ll be working from a Liquid Web Core Managed Ubuntu 14.04 LTS server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Ubuntu 14.04 LTS”