Install Rsync and Lsync on CentOS, Fedora or Red Hat

Have you ever needed to copy files from your local computer over to your web server? You may have previously used File Transfer Protocol (FTP) applications for this task, but FTP is prone to being insecure and can be challenging to work with over the command line. What if there was a better way? In this tutorial, we’ll be covering two popular utilities in the Linux world to securely assist in file transfers, rsync and lsyncd. We’ll show you how to install and use both in this article. Let’s dig in!

 

What is Rsync?

The first utility we’ll look at is called rsync, and this command is a real powerhouse! Most simply, it is used for copying files between folders, but it has some extra features that make it very useful. We’ll start with the most basic usage for rsync, and work into more complicated examples to show you how versatile it can be.

 

Install Rsync on CentOS, Fedora, or Red Hat

If you are using CentOS, Fedora, or Red Hat, you can use the yum package manager to install:

Note:
You’ll need to be the “root” user to install packages!

yum install rsync

 

Install Rsync on Ubuntu or Debian

If you are using Ubuntu or Debian, you can use the apt-get package manager to install:

Note:
You’ll need to be the “root” user to install packages!

apt-get install rsync

 

How to Use Rsync

The syntax for running rsync looks like this:

rsync -options <source> <destination>

You are not required to specify “options”, but you’ll always need to tell rsync a source and destination.  In this example, we’re using rsync to copy “file.txt” into the “/path/of/destination/” folder:

rsync /path/of/source/file.txt /path/of/destination/file.txt

Important
When you run rsync, remember always to put the “source” first, and “destination” second!

Now that you have the basics let’s try another common task. In this example, we’re going to use rsync to copy a directory from our local computer over to our web server “192.168.0.100”.

rsync -avHl /path/of/source/folder root@192.168.0.100:/path/to/destination/folder

Notice how just as before, we specified our source first, and destination second. One of the great things about rsync is that it performs remote transfers of data securely, through SSH. Using SSH is fantastic from a security point of view, and it allows you to use SSH keys to avoid typing passwords.

As one last example, let’s try copying something from your remote server “192.168.0.100” over to your local machine. Once again, rsync is the tool for the job!

rsync -avH  root@192.168.0.100:/path/of/remote/folder /path/of/destination/folder

We used some special options in that last example. Let’s break them down.

-a = archive mode (includes several commonly used options: -rlptgoD, check the rsync man page for detailed info.)

-v = print verbose messages to screen, (very helpful!)

-H = preserve hard links when copying

One of the great things about rsync is that it intelligently copies files. If only the last few bits of a file has changed, rsync solely copies the changes, rather than the whole file. Transferring only the changed parts of a file can be a huge time saver, but especially when copying files remotely like in that last example.

 

Introducing Lsyncd

Finally, we’ll talk about lsyncd. The utility lsyncd is somewhat similar to rsync in that it is used to synchronize two folders. Unlike rsync, which has to be run manually, lsyncd is a daemon. Sounds scary, but in computer terminology, daemons are merely applications that run as a background process. You usually don’t have to manually run daemons every time you want to use them, as they are typically configured to start automatically when your server boots. When you configure lsycnd correctly, it can automatically synchronize folders between two computers. Imagine if you didn’t have to manually create backups of your website on your web server every time you made a small change? That could be a real time saver! Let’s dig in.

 

Install Lsyncd on CentOS, Fedora, or Red Hat

If you are using CentOS, Fedora, or Red Hat, you can use the yum package manager to install:

Note
You’ll need to be the “root” user to install packages!

yum install lsyncd

 

Install Lsyncd on Ubuntu or Debian

If you are using Ubuntu or Debian, you can use the apt-get package manager to install:

Note
You’ll need to be the “root” user to install packages!

apt-get install lsyncd

 

How to Use Lsyncd

Unlike rsync, lsyncd runs as a daemon. You don’t run it directly. Instead, it starts automatically with your server at boot time, and runs silently in the background. It’s a great automated way to sync folders on your server! All we need to do is configure lsyncd, so it knows what folders to sync.

First, we’ll create some files and folders.

Create the configuration folder location
mkdir /etc/lsyncd

Create a folder to sync FROM, feel free to name it what you would like
mkdir /source_folder

Create the folder to sync TO
mkdir /destination_folder

Create the log folder location
mkdir /var/log/lsyncd

Make the log file
touch /var/log/lsyncd.log

Create the status file
touch /var/log/lsyncd.status

This is an example file for our sync tutorial
touch /source_folder/hello_world.txt

Next, we need to configure lsyncd to use our newly created files. Open a new file for editing using your favorite Linux text editor.

vi /etc/lsyncd/lsyncd.conf.lua

Paste in the following configuration, and save the file.

settings = {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status"
}
sync {
default.rsync,
source = "/source_folder",
target = "/destination_folder",
}

Now all that remains is to start the program! First, we’ll tell lsyncd to start automatically when your server boots:

systemctl enable lsyncd

Next, we’ll start lsyncd manually, (this only needs to be done once.)

systemctl start lsyncd

We’ve successfully installed lsyncd, but it is good practice to double check your work. We’ll check to see if it is running using this command:

systemctl status lsyncd

If you see a line that reads: active (running), it is running correctly!

Finally, we check the contents of /destination_folder to make sure that it contains our “hello_world.txt” file.

ls -l  /destination_folder
total 0
-rw-r--r-- 1 root root 0 Nov 17 07:15 hello_world.txt

 

You should see that “hello_world.txt” has been automatically synchronized over to “/destination_folder”. Everything is working! In practice, you can set “/source_folder” and “/destination_folder” to any folders you need synchronizing.

As you can see, these two utilities rsync and lsyncd are great tools for copying files and folders in Linux. Have questions about how to use these tools on your Liquid Web web server? Reach out to the Most Helpful Humans in Hosting. We’re here to help, 24×7!

 

Protecting against CVE-2018-14634 (Mutagen Astronomy)

There is a new exploit, rated as 7.8 severity level,  that affects major Linux distributions of RedHat Enterprise Linux, Debian 8 and CentOS named Mutagen Astronomy. Mutagen Astronomy exploits an integer overflow vulnerability in the Linux kernel and supplies root access (admin privileges) to unauthorized users on the intended server. This exploit affects Linux kernel version dating back from July 2007 to July 2017.  Living in the kernel, the memory table can be manipulated to overflow using the create_tables_elf() function. After overwhelming the server, the hacker can then overtake the server with its malicious intents.

As mentioned this vulnerability is present in RedHat, Debian 8 (Debian 9 are not vulnerable), and CentOS 6 and 7 but its limited to affecting only 64-bit versions. The 32-bit versions do not have the address space to overwhelm the server and thus no patch is needed.  Along with 64-bit versions, the exploit is limited to Linux kernel versions 2.6.x, 3.10.x, and 4.14.x. (Read our article How To Check the Kernel Version to see which version you are running)  Proof of concept reported on August 31, 2018, and remediation from a one-year-old patch was backported to most LTS (long-term support) kernels, CentOS and Debian 8 remain vulnerable. Luckily, patches CentOS 6 /7 and RHEL 7 are below but the world waits for RHEL 6 and Debian 8 remedy.  Check back here for updates as they become available.

 

CentOS 6 and 7 Patch for Mutagen Astronomy

Step 1: Utilizing SystemTap.

Use SystemTap to extract, filter and summarize data to diagnosis performance or functional problems.

yum install systemtap systemtap-runtime

Step 2: Create a File

Using your preferred text editor create a file name mutagenastronomypatch.stp and copy and paste the following info:

// CVE-2018-14634
//
// Theory of operations: adjust the thread's # rlimit-in-effect around
// calls to the vulnerable get_arg_page() function so as to encompass
// the newly required _STK_LIM / 4 * 3 maximum.
// Complication: the rlimit is stored in a current-> structure that
// is shared across the threads of the process. They may concurrently
// invoke this operation.
function clamp_stack_rlim_cur:long ()
%{
struct rlimit *rlim = current->signal->rlim;
unsigned long rlim_cur = READ_ONCE(rlim[RLIMIT_STACK].rlim_cur);
unsigned long limit = _STK_LIM / 4 * 3;
limit *= 4; // multiply it back up, to the scale used by rlim_cur
if (rlim_cur > limit) {
WRITE_ONCE(rlim[RLIMIT_STACK].rlim_cur, limit);
STAP_RETURN(limit);
} else
STAP_RETURN(0);
%}
probe kernel.function("copy_strings").call
{
l = clamp_stack_rlim_cur()
if (l)
printf("lowered process %s(%d) STACK rlim_cur to %p\n",
execname(), pid(), l)
}
probe begin {
printf("CVE-2018-14634 mitigation loaded\n")
}
probe end {
printf("CVE-2018-14634 mitigation unloaded\n")
}
Step 3: Run the Script

Lastly, execute the newly created script:

stap -g mutagenastronomypatch.stp

RHEL 7 Patch for Mutagen Astronomy

Step 1: Log in to your CentOS server

ssh root@ip

Step 2: Yum Updates to the Kernel

yum update kernel

 

 

Tutorial: An Introduction to Managing a Linux Server with systemd

What is systemd?

Systemd is the System Management Daemon, which provides a standard for controlling the Linux boot process. It is named per the UNIX convention of adding ‘d’ to the end of daemon’s name. Systemd is intended as a drop-in replacement for the very common init start-up scripts for the boot process (which is also referred to as System V or SysV).

Though there is some debate regarding the benefits of systemd, the advantages do include: Service Reporting (failed? suspended? error?), Process Monitoring (kills user processes at logout), and Parallel Processes (multiple services are able to start at same time, improving boot times).

Continue reading “Tutorial: An Introduction to Managing a Linux Server with systemd”