Create a Cron Task in Ubuntu 16.04

Cron jobs are an incredibly useful Linux tool aimed at saving you time by scheduling tasks within your server. A programmed cron task will execute commands within a script by the minute, day, week or month. They can be scheduled to do many tasks including backing up your server’s files nightly, updating inventory orders in a database or even compressing files for migrating. Repetitive tasks become a cinch when incorporating a cron job. While there are numerous ways to run a cron task, we will be using the crontab option that is inherent within Ubuntu to set up a nightly backup of our website.

Pre-flight

Log in to your Ubuntu 16.04 LTS server, preferably not as root if you aren’t altering anything on the server level.

Setting Up a Website Backup through Cron

Step 1: Update your server. As a best practice, we will update and upgrade our server with the following command.

apt-get update && apt-get upgrade

 

Step 2: Verify if the cron package is installed.
dpkg -l cronOur example output let’s us know that the cron package is installed, along with its version:
||/ Name Version Architecture Description
+++-=========================-=================-=================-========================================================
ii cron 3.0pl1-128ubuntu2 amd64 process scheduling daemon

Install cron package if necessary.
sudo apt-get install cron

Ensure that the cron service is running with the following command:
systemctl status cronExample Output:
* cron.service - Regular background program processing daemon
Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2018-10-27 02:53:20 EDT; 5 days ago

 

Step 3: Configure the cron job. When you are logged in as your user, you are creating a cron job under that user. Creating a cron jobs owner is helpful when to know who is in charge of the cron as well as how to alter the cron job in the future.

crontab -e

The system asks which editor you’d like to use; this tutorial is using option 2 (vim.basic).

tom@host2:~$ crontab -e
no crontab for tom - using an empty one
Select an editor. To change later, run 'select-editor'.
1. /bin/ed
2. /usr/bin/vim.basic
3. /usr/bin/vim.tiny
Choose 1-3 []: 2

In this file, you’ll see # signs followed by the direction on how to use the file. Allow us a minute to explain the syntax needed to create a cron task.

# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed

Each set of digits designates the time for that section and follows the time set in your server The first set represents minutes, the next set hours, the next set day of the month and so on. You may be asking yourself what the asterisk mean. They represent every or all value for that set. So, if you put an asterisk on the month column, you’ll be telling the system to perform the tasks every month.

After the digits section is the /bin/sh and the type of file that should be executed, lastly, the final column assigns the command of which to run.

Backup Cron:
Let’s say you want to backup your website files daily. Our example uses 4 am since server traffic is low during this time. Take note of the .sh extension, indicating a bash script. Use the appropriate extension for your script. Be sure to save the script when exiting with :wq

crontab -e

After altering the /path/to/script/backup-script.sh path to match the location of your script, set the command in a new line, below the hashtag (#), like so:

# m h dom mon dow command
0 4 * * * /bin/sh /path/to/script/backup-script.sh

 

Step 4: Place Script in Path. Most importantly is the creation and placement of your script. With our example script we are creating a backup that will compress files and directories, indicated by the /SOURCE/DIRECTORY/. The /SOURCE/DIRECTORY/ needs to be alter to the directory and files you wish to save (sometimes a website’s default directory path is /var/www/html), while the /DESTINATION/DIRECTORY/file.zip indicates where the compressed backup are stored. We will name the script backup-script.sh, and afterwards, we will create the script in the location of /path/to/script/backup-script.sh.

vim backup-script.sh

#!/bin/sh
zip -9pr /DESTINATION/DIRECTORY/file.zip /SOURCE/DIRECTORY/

 

Note:
If you are running this script multiple times it will overwrite the file.zip each time, thus it runs it will only store one backup.

Set execute permissions this allows the system permission to run the script.

chmod +x backup-script.sh

When the time of execution has passed, check your /DESTINATION/DIRECTORY/ to see file.zip. YYou now have backups of your files running daily! It’s best to come up with a plan to offload your backups to another location other than your server. Pushing backups to an offsite location not only adds a layer of security but aids in saving server disk space. Check out our Cloud Object Storage for cost-effective storage solution that is accessible via API call. If you are having trouble with your cron check out our other helpful article, Troubleshooting: Cron Jobs.

 

How to Use Ansible

Ansible symbolAnsible is an easy to use automation software that can update a server, configure tasks, manage daily server functions and deploys jobs as needed on a schedule of your choosing. It is usually administered from a single location or control server and uses SSH to connect to the remote servers. Because it employs SSH to connect, it is very secure and, there is no software to install on the servers being managed. It can be run from your desktop, laptop or other platforms to assist with automating the tedious tasks which every server owner faces.

Once it is configured, Ansible performs tasks based on an ordered list of assignments in what is called a Playbook. The Playbook outlines what tasks need to be run on the remote server and in what order. Once this is configured, Ansible acts like a bash for-loop command that allows a section of code to be repeated over and over again. The difference between using a bash command and Ansible is that Ansible is idempotent. The term Idempotent sounds a little scary, but it merely means that you can make the same type of request over and over again and unless something has changed, you will get the same result.

Pre-flight: Server Requirements

Source Server Requirements

Ansible requires the installation of Python 2.7 or Python 3.5 on the source server. The source server is where you will be running the tasks in the playbook for the remote servers. The remote servers receive commands defined in the playbook.  A playbook is a file which defines the scripts that will be run on the remote servers.

Note:
Unfortunately, Windows is unsupported as a source server. Certain Ansible plugins and/or modules will have other needs or requirements. Usually, these plugins or modules are installed on the same server of the Ansible installation.

Let’s start by installing Python on the source server.
root@test:~# apt-get install python

 

Target Server Requirements

The only requirement from the target server is an open SSH port. Access can also be granted for scp (secure copy) and/or SFTP connections if configured in the /etc/ansible/ansible.cfg file.

Install Ansible On Ubuntu 16.04

To install Ansible on a source Ubuntu server, let’s follow these steps:

Note:
The PPA for Ansible is here: https://launchpad.net/~ansible/+archive/ubuntu/ansible if you would like to review the versions available for your variant of Ubuntu.

root@test:~# apt-get update
root@test:~# apt-get install software-properties-common
root@test:~# apt-add-repository ppa:ansible/ansible
root@test:~# apt-get update
root@test:~# apt-get install ansible
(install text)After this operation, 42.0 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Answer “Y” to the prompt. The install will complete and take you back to the command prompt. Now, let’s check the version of Ansible installed.

Check Ansible Version

root@test:~# ansible --version
ansible 2.7.0
 config file = /etc/ansible/ansible.cfg
 configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
 ansible python module location = /usr/lib/python2.7/dist-packages/ansible
 executable location = /usr/bin/ansible
 python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

As an alternative, you can also install Ansible on your CentOS 7 server.
Ansible also can be installed on RedHat, Debian, MacOS, and any of the BSD flavors!

Install Ansible on CentOS 7

In order to install Ansible on a source CentOS 7 server, follow these steps.
First, we need to make sure that the CentOS 7 EPEL repository is added:

[root@test ~]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)

[root@test ~]# yum install epel-release
Loaded plugins: fastestmirror, priorities, universal-hooks
Loading mirror speeds from cached hostfile

...
Resolving Dependencies
--> Running transaction check
---> Package epel-release.noarch 0:7-11 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================================================================
Package Arch Version Repository Size
==========================================================================================================
Installing:
epel-release noarch 7-11 system-extras 15 k
Transaction Summary
==========================================================================================================
Install 1 Package
Total download size: 15 k
Installed size: 24 k
Is this ok [y/d/N]: y
Downloading packages:
epel-release-7-11.noarch.rpm | 15 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : epel-release-7-11.noarch 1/1
Verifying : epel-release-7-11.noarch 1/1
Installed:
epel-release.noarch 0:7-11
Complete!

Select “y”. The EPEL repo will then be added. Once the repository is enabled, we can install Ansible with yum:

root@test:~# yum install ansible
Loaded plugins: fastestmirror, priorities, universal-hooks
Loading mirror speeds from cached hostfile
epel/x86_64/metalink                                                               | 18 kB 00:00:00
* EA4: 208.100.0.204
* cpanel-addons-production-feed: 208.100.0.204
* epel: mirrors.liquidweb.com
epel                                                                               | 3.2 kB 00:00:00
(1/3): epel/x86_64/group_gz                                                        | 88 kB 00:00:00
(2/3): epel/x86_64/updateinfo
(3/3): epel/x86_64/primary                                                         | 3.6 MB 00:00:00
epel                                                                                          12756/12756
Resolving Dependencies
… (dependencies check)
Dependencies Resolved
==========================================================================================================
Package                        Arch Version            Repository Size
==========================================================================================================
Installing:
ansible                        noarch 2.4.2.0-2.el7            system-extras 7.6 M
Installing for dependencies:
21 k
Transaction Summary
==========================================================================================================
Install  1 Package (+17 Dependent packages)
Total download size: 12 M
Installed size: 58 M
Is this ok [y/d/N]:


Select “y” to start the Ansible install:

Is this ok [y/d/N]: y
… Downloading 18 packages:
----------------------------------------------------------------------------------------------------------
Total                                                                      30 MB/s | 12 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
… (installing 18 python related software)
...
Installed:
ansible.noarch 0:2.4.2.0-2.el7
Dependency Installed:
... (dependencies verified)
Complete!

Check Ansible Version on CentOS 7

Now, let’s verify the version installed:

root@test ~]# ansible --version
ansible 2.4.2.0
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]

 

Setting Up Ansible Connections

Initially, we will be adding server names or IP’s to the /etc/ansible/hosts file to identify which “ungrouped” servers and “groups” of servers we are going to be connecting to. We mention ungrouped and grouped in this specific order because this is the way the Ansible hosts file is usually arranged.

We can use any name we like for the hosts file itself but typically it is just called hosts. Ansible also states that the hosts file can also be identified as an inventory file and, you can have multiple inventory files.

Let’s start by opening the hosts file with vim and inserting some entries into the file.

root@test:~vim /etc/ansible/hosts
Here is what the default hosts file will look like:

# This is the default ansible 'hosts' file.
#
# It should live in /etc/ansible/hosts
#
# - Comments begin with the '#' character
# - Blank lines are ignored
# - Groups of hosts are delimited by [header] elements
# - You can enter hostnames or ip addresses
# - A hostname/ip can be a member of multiple groups
# Example 1: Ungrouped hosts, specify before any group headers.
#green.example.com
#blue.example.com
#192.168.100.1
#192.168.100.10
# Example 2: A collection of hosts belonging to the 'webservers' group
#[webservers] #alpha.example.org
#beta.example.org
#192.168.1.100
#192.168.1.110
# If you have multiple hosts following a pattern you can specify
# them like this:
#www[001:006].example.com
# Example 3: A collection of database servers in the 'dbservers' group
#[dbservers] #
#db01.intranet.mydomain.net
#db02.intranet.mydomain.net
#10.25.1.56
#10.25.1.57
# Here's another example of host ranges, this time there are no
# leading 0s:
#db-[99:101]-node.example.com


As you can see, there are individual servers “#
green.example.com”, and groups like #[webservers] which have multiple servers under the group and, another section with multiple servers listed like #db-[99:101]-node.example.com which identifies all of the individual servers from 99-101; eg.

  • db-99-node.example.com
  • db-100-node.example.com
  • db-101-node.example.com

So, let’s quickly add another server to the #[webservers] group:

#[webserver1]
#alpha.example.org
#beta.example.org
#192.168.1.100
#192.168.1.110gamma.example.com

Now, simply save the file using :wq

Note:
Make sure you uncomment any ‘#’ entries you place in the file otherwise, the entry is excluded!

 

SSH Keys

You can set up public SSH keys from the control server to log in to those remote servers noted in the hosts file. In this case, you simply want to make sure your local SSH keys are located in the /root/.ssh/authorized_keys file on the remote systems. (Depending on your setup, you may wish to use Ansible’s –private-key option to specify a .pem file instead)

 

Verify Ansible Connections

The ansible inventory file (/etc/ansible/hosts) contains the server names you will have control over and can run tasks against. To verify Ansible’s connectivity, run:

ansible remote -m ping

 

Ansible Playbooks

Ansible playbooks (also called inventory files) define the tasks ran on the remote servers. You can have one playbook or multiple playbooks to accomplish different tasks on different servers.  To easily apply a task to all of the servers in a pool, use the ‘group’ name to apply the task for that group (using the example above, you would use the [webserver1] in the command.

 

Create a Playbook

Step 1: In order to create a playbook, let’s create a new file in the /etc/ansible/playbooks/ folder:

mkdir -p /etc/ansible/playbooks/ && touch /etc/ansible/playbooks/playbook.yml && vim /etc/ansible/playbooks/playbook.yml

 

Step 2: Let’s add a server and file entry into the playbook filer:

(Click the insert key to open VIM’s edit access on the file)

- hosts: gamma.example.com
 tasks:
     - name: Create file
       file:
           path: /tmp/test.txt
           state: touch


Once we have added the entry, let’s save the file using

:wq

Step 3: Now, to set up 0644 permissions on that file, we can reopen it and add another line defining the permission set:

- hosts: gamma.example.com
 tasks:
     - name: Create file
       file:
           path: /tmp/test.txt
           state: touch            mode: "u=rw,g=r,o=r"

Again, let’s save the file using

:wq

Step 4: Next, let’s create a folder and then place a text file in it using Ansible. We will add another section defining the element needed:

- hosts: gamma.example.com
 Tasks:       - name: Create folder
       path: /home/tmp/
           state: directory
           mode: 0755
     - name: Create file
       file:
           path: /home/tmp/test.txt
           state: touch            mode: "u=rw,g=r,o=r"

Once we have added this entry, save the file using

:wq

 

Running a Playbook

To start a playbook, simply run:

ansible-playbook /etc/ansible/playbooks/playbook.yml

or if you have multiple playbooks in a folder, can run a specific playbook using the -i <path> option from the command line:

ansible-playbook -i /etc/ansible/playbooks/playbook1.yml
In addition to .yaml files, Ansible can use .json files as well to control the playbook. It is also very easy to convert 
bash or shell scripts into playbooks as well!

Schedule a Playbook Using Cron

As an additional option, you can schedule a playbook to run at a specific time using your servers cron command! To accomplish this, log in to your server as root and run the following command:

crontab -e
This command opens a temporary cron file in your system’s 
default text editor and then simply add a line like so:

0 4 * * * /usr/bin/ansible-playbook /etc/ansible/playbooks/playbook.yml
this will run the
/etc/ansible/playbooks/playbook.yml file at 0400 a.m. using the ansible-playbook command.

 

Troubleshooting A Playbook

Sometimes, a set of commands in the playbook file may fail. If it does, you have several options to address this. Generally, playbooks will simply stop completing the commands in the playbook. If this occurs, you can define a follow-up task in the playbook to overlook the error by adding another section like so:

- name: ignore this error
 command: /bin/false
 ignore_errors: yes


Unreachable Hosts

this command will only work when the task is run but returns a “failed” value.

If Ansible fails to connect to a server, it will set the host as being ‘UNREACHABLE’. This effectively removes the server temporarily from the list of active hosts for that task. To correct this, we can use an entry to reactivate them and have all current hosts previously indicated as being unreachable cleared, so subsequent tasks can use the playbook again.

meta: clear_host_errors


Handlers and Failure

A handler is simply a specially named task that runs when told to by another task. Handlers are executed at the end of the playbook by default as opposed to other tasks, which are executed immediately when defined within the playbook. This behavior can be modified by using the

--force-handlerscommand-line option, or by including

force_handlers: Truein a playbook, or addingforce_handlers = Truein the ansible.cfg file.


If you want to force a handler to run in the middle of two separate tasks instead of at the end of the playbook, you will need to add this entry between the two tasks:

- meta: flush_handlers

When handlers are “forced” like this, they will run when notified even if a task fails on that host.

Note:
Certain errors can still prevent the handler from running, such as a host becoming unreachable.
Handlers will only be visible in the output if they have actually been executed. Also, handlers are only fired when there are changes made by a task. For example, a task may update a specific configuration file and then notify a handler to restart a service. If a task in the same playbook fails later on, the service will not be restarted despite the previous configuration change.

Overall, Ansible is an indispensable tool for managing and administrating a single server or an entire group of geographically diverse servers.

 

Troubleshooting: Cron Jobs

Cron is a service for Linux servers that automatically executes scheduled commands. A cron job can be a series of shell commands, scripts, or other programs. Cron tasks or jobs can perform a variety of functions and once ran can send out an e-mail message to inform you of its completion or errors. If you receive an error, there are many ways to troubleshoot the cron task.  Use this article for troubleshooting assistance or a tutorial on the basics of cron jobs. If you would like to learn more about creating a cron job check out our Knowledge Base tutorials on the subject.

Checking Configurations with Crontab

From the command line, you can review the scheduled cron jobs by listing the crontab for the user. This command outputs the contents of the user’s crontab to the terminal.

As the user you can run:
crontab -l

As root, you can see any user’s crontab, by specifying the username.
crontab -l -u username

You can find some detailed information about how to format the cron jobs in the /etc/crontab file.  Below is the example within that file.  Each asterisk can be replaced with a number or to its corresponding field.  Or you can leave the asterisk in place to represent all possible numbers for that position.  For example, if left with all asterisk this means that the cron job will run every minute, all the time.

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# For details see man 4 crontabs
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed

 

Altering the E-mail Address of a Cron

Once initiated, a cron sends a notification to an email address, set within the MAILTO line of the crontab.

MAILTO="user@domain.com"

To edit the crontab, you can run the following commands as the user:

crontab -e

Or if logged in as root, you can type in the username for any of your users to see a scheduled task that they have created.

crontab -e -u usernameThese open the crontab of the user in the default editor. Typically the vim or nano command will open the file. Be aware that this similar to opening any other text file where you’ll save before closing.

The MAILTO line indicates where the execution status of a cron should be sent. The sending address will typically be the cron task creator’s username along with the server’s hostname. So an email’s sender address would follow this syntax, unixuser@serverhostname.com. If you don’t see an email right away, it may be a good idea to check your spam box.

Silenced Crons

Sometimes cron jobs are configured to either produce no output or have its output silenced, even if set with a MAILTO address. If you see a cron job listed with any of the following on end, it is a sign that the cron has output has been silenced. These send any output to the null device (the black hole on a Linux server). In cases like this, you’ll need to remove the line from the cron job script to generate an output.

&> /dev/null

2>&1 /dev/nullSome cron jobs are disabled entirely. These will have a “#” in front of the command, resulting ignored lines when executed. Remove the “#” to re-activate the cron job.

Verifying the Crond Service

Once you’ve confirmed the correct settings, it’s time to verify that the cron system is enabled and running. The three following commands can each be used to verify if the crond (the cron service) is running.

/etc/init.d/crond status

service crond status

systemctl status crond

After running any of the above commands, if you find the crond service is not running, you can start it with one of the following.

/etc/init.d/crond start

service crond start

systemctl start crond

/var/log/cron

Once you know that the cron is enabled, not silenced, and crond is running, then it’s time to check the cron log, located in the path of /var/log/cron.

cat /var/log/cron

Example Output:

Oct 2 23:45:01 host CROND[3957]: (root) CMD (/usr/local/lp/apps/kernelupdate/lp-kernelupdate.pl > /dev/null 2>&1)
Oct 2 23:50:01 host CROND[4143]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Oct 2 23:50:01 host CROND[4144]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
In the log, you will see if, when and what user ran the cron. If initiated, you’ll see the date and time of execution followed by brackets of the individual cron number. This timestamp does not confirm that the script ran normally or at all, it only signifies when the cron system last ran the task. Beyond that, you may need to investigate the cron script itself or application-level configurations and their respective logs to ensure the code is executing correctly.

Other Cron Services

This article is merely an overview of the main crond service as there many other cron tasks services. The anacron system is a commonly used cron service that configures daily or hourly jobs, and can even be set to run at reboot. The logs for these kinds of tasks are within /var/log/cron, and are not executed by crond.

Other scheduled tasks, although also referred to as cron jobs, are not executed from the crond system. These cron jobs are often configured within the code or configuration of a website. To determine if executed, you’ll need to investigate other configurations and logs that the cron script interacts with.

As with all cron services, automated jobs can be manipulated to execute numerous daily tasks, so you don’t have to. Cron tasks can occasionally go awry even without altering them or years, but knowing where to look is half the battle in troubleshooting a cron job.

 

Configuring and Troubleshooting WHMCS Crons

Over the years WHMCS has made some changes to where it stores certain directories, specifically directories outside of public_html. The goal of this is to increase overall security by moving sensitive files to a more protected location. While this change does help to improve WHMCS security, it also adds a few steps of complexity.

This article is meant to help simplify this complexity, or at least provide a reference configuration that you can use to troubleshoot cron issues, or gain a better understanding of WHMCS crons in general. I used WHMCS 7.3 for this article, but the general concept and instructions should apply for any 7.0 version of WHMCS.
Continue reading “Configuring and Troubleshooting WHMCS Crons”

How To Set up a Cron Job in cPanel

  1. This tutorial assumes you’ve already logged in to cPanel, and are starting on the home screen.cpanel-paperlantern-28-cron--01
  2. Now let’s learn how to setup a cron job.cpanel-paperlantern-28-cron--02
  3. Click the “Cron Jobs” icon.cpanel-paperlantern-28-cron--03
  4. Enter the email address where you want the cron job results sent after each time it runs.cpanel-paperlantern-28-cron--04
  5. Now you have to define exactly when and how often you want the cron job to run.cpanel-paperlantern-28-cron--05
  6. This is made easier by selecting one of the pre-defined common settings.cpanel-paperlantern-28-cron--06
  7. Notice that by choosing a common setting, all fields are filled in automatically. This also helps you understand what each field means.cpanel-paperlantern-28-cron--07
  8. Next, enter the command of the script you want to run, including the path (from root).cpanel-paperlantern-28-cron--08
  9. Then click “Add New Cron Job”.cpanel-paperlantern-28-cron--09
  10. That’s it! The cron job has been set, and will be listed at the bottom of the screen.

 

How to Display (List) All Jobs in Cron / Crontab

Servers can automatically perform tasks that you would otherwise have to perform yourself, such as running scripts. On Linux servers, the cron utility is the preferred way to automate the running of scripts. In this article we’ll cover how to view the jobs scheduled in the crontab list. For an introduction to Cron check-out our KB: How To: Automate Server Scripts With Cron. Knowing how to setup crontab is an important skill, but even if you’re not editing these knowing how to view them is important as well. Continue reading “How to Display (List) All Jobs in Cron / Crontab”