How to Add a User and Grant Root Privileges on Ubuntu 16.04

Ubuntu 16.04 LTS provides you the ability to add a user for anyone who plans on accessing your server.  Creating a user is a basic setup but an important and critical one for your server security. In this tutorial, we will create a user and grant administrative access, known as root, to your trusted user.

 

Pre-Flight Check

  1. Open a terminal and log in as root.  
  2. Work on a Linux Ubuntu 16.04 server

Step 1:  Add The User

Create a username for your new user, in my example my new user is Tom:

adduser tom

You’ll then be prompted to enter a password for this user.   We recommend using a strong password because malicious bots are programmed to guess simple passwords. If you need a secure password, this third party password generator can assist with creating one.

Output:

~# adduser tom
Adding user `tom' ...
Adding new group `tom' (1002) ...
Adding new user `tom' (1002) with group `tom' ...
Creating home directory `/home/tom' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Note
Usernames should be lowercase and avoid special characters. If you receive the error below, alter the username. ~# adduser Tom
adduser: Please enter a username matching the regular expression configured via the NAME_REGEX[_SYSTEM] configuration variable.  Use the `--force-badname' option to relax this check or reconfigure NAME_REGEX.

 

Prompts will appear to enter in information on your new user.  Entering this information is not required and can be skipped by pressing enter in each field.

Enter the new value or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:

 

Lastly, the system will ask you to review the information for accuracy.  Enter Y to continue to our next step.

Is the information correct? [Y/n]

 

Step 2: Grant Root Privileges

Assigning a user root access is to grant a user the highest power.  My user, tom, can then make changes to the system as a whole, so it’s critical to allow this access only to users who need it. Afterward, this user will be able to use sudo before commands that are usually designed to be used by the root user.

usermod -aG sudo tom

 

Step 3: Verify New User

As root, you can switch to your new user with the su – command and then test to see if your new user has root privileges.

su - tom

If the user has properly been granted root access the command below will show tom in the list.

grep '^sudo' /etc/group

Output:

sudo:x:27:tom

 

How To Install Oracle Java 8 in Ubuntu 16.04

Pre-Flight Check

  1. Open the terminal and log in as root.  If you are logged in as another user, you will need to add sudo before each command.
  2. Working on a Linux Ubuntu 16.04 server
  3. No installations of previous Java versions

Step 1:  Update & Upgrade

It is advised to update your system by copy and pasting the command below.  Be sure to accept the update by typing Y when asked to continue:

apt-get update && apt-get upgrade

 

Step 2: Install the Repository

WebUpd8 Team Personal Package Archive (PPA), a third party repository,  allows us to download the package necessary for Java 8 installation.  Press Enter to continue the installation.

add-apt-repository ppa:webupd8team/java

Once again, update your package list.

apt-get update

 

Step 3: Install Java 8

Use the apt-get command to install Oracle’s Java 8 via their installer:

apt-get install oracle-java8-installer

 

Click Y to continue and press Enter to agree to the licensing agreement.

 

Select Yes and hit the Enter key.

 

Step 4: Verify Java 8 is Installed

java -version

Output:

java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

 

It’s essential to know the path of our Java installation for our applications to function. Where is Java installed? Run this command to find its path:update-alternatives --config java

Output:

~# update-alternatives --config java
There is 1 choice for the alternative java (providing /usr/bin/java).
Selection Path Priority Status
------------------------------------------------------------
0 /usr/lib/jvm/java-8-oracle/jre/bin/java 1081 auto mode
* 1 /usr/lib/jvm/java-8-oracle/jre/bin/java 1081 manual mode

 

Copy the highlighted path from the second row: /usr/lib/jvm/java-8-oracle/jre/bin/java/.  After copying, open the file /etc/environment and add in the path of your Java installation to the end of your file.

vim /etc/environment

JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre/bin/java"

 

Save the file by hitting ESC button and type :wq to execute the command below to recognize the changes to the file:

source /etc/environment

 

You should now see the path of installation when using the $Java_Home variable:

echo $JAVA_HOME

Output:

~# echo $JAVA_HOME
/usr/lib/jvm/java-8-oracle/jre/bin/java

 

MySQL Performance: MyISAM vs InnoDB

 

A major factor in database performance is the storage engine used by the database, and more specifically, its tables. Different storage engines provide better performance in one situation over another. For general use, there are two contenders to be considered. These are MyISAM, which is the default MySQL storage engine, or InnoDB, which is an alternative engine built-in to MySQL intended for high-performance databases. Before we can understand the difference between the two storage engines, we need to understand the term “locking.”

To protect the integrity of the data stored within databases, MySQL employs locking. Locking, simply put, means protecting data from being accessed. When a lock is applied, the data cannot be modified except by the query that initiated the lock. Locking is a necessary component to ensure the accuracy of the stored information.  Each storage engine has a different method of locking used. Depending on your data and query practices, one engine can outperform another. In this series, we will look at the two most common types of locking employed by our two storage engines.

 

Table locking:  The technique of locking an entire table when one or more cells within the table need to be updated or deleted. Table locking is the default method employed by the default storage engine, MyISAM.

Example: MyISAM Table LockingColumn AColumn BColumn C
Query 1 UPDATERow 1Writingdatadata
Query 2 SELECT (Wait)Row 2datadatadata
Query 3 UPDATE (Wait)Row 3datadatadata
Query 4 SELECT (Wait)Row 4datadatadata
Query 5 SELECT (Wait)Row 5datadatadata
The example illustrates how a single write operation locks the entire table causing other queries to wait for the UPDATE query finish.

 

Row-level locking: The act of locking an effective range of rows in a table while one or more cells within the range are modified or deleted. Row-level locking is the method used by the InnoDB storage engine and is intended for high-performance databases.

Example: InnoDB Row-Level LockingColumn AColumn AColumn A
Query 1 UPDATERow 1Writingdatadata
Query 2 SELECTRow 2Readingdatadata
Query 3 UPDATERow 3dataWritingdata
Query 4 SELECTRow 4ReadingReadingReading
Query 5 SELECTRow 5ReadingdataReading
The example shows how using row-level locking allows for multiple queries to run on individual rows by locking only the rows being updated instead of the entire table.

 

By comparing the two storage engines, we get to the crux of the argument between using InnoDB over MyISAM. An application or website that has a frequently used table works exceptionally well using the InnoDB storage engine by resolving table-locking bottlenecks. However, the question of using one over the other is a subjective as neither of them is perfect in all situations. There are strengths and limitations to both storage engines. Intimate knowledge of the database structure and query practices is critical for selecting the best storage engine for your tables.

MyISAM will out-perform InnoDB on large tables that require vastly more read activity versus write activity. MyISAM’s readabilities outshine InnoDB because locking the entire table is quicker than figuring out which rows are locked in the table. The more information in the table, the more time it takes InnoDB to figure out which ones are not accessible. If your application relies on huge tables that do not change data frequently, then MyISAM will out-perform InnoDB.  Conversely, InnoDB outperforms MyISAM when data within the table changes frequently. Table changes write data more than reading data per second. In these situations, InnoDB can keep up with large amounts of requests easier than locking the entire table for each one.

 

Should I use InnoDB with WordPress, Magento or Joomla Sites?

The short answer here is yes, in most cases. Liquid Web’s Most Helpful Humans in Hosting Support Teams have encountered several table-locking bottlenecks when clients are using some of the standard web applications of today. Most users of popular third-party applications like WordPress, Magento, and Joomla have limited knowledge of the underlying database components or code involved to make an informed decision on storage engines. Most table-locking bottlenecks from these content management systems (CMS) are generally resolved by changing all the tables for the site over to  InnoDB instead of the default MyISAM.  If you are hosting many of these types of CMS on your server, it would be beneficial to change the default storage engine in MySQL to use InnoDB for all new tables so that any new table installations start off with InnoDB.

 

Set your default storage engine to InnoDB by adding default_storage_engine=InnoDB to the [mysqld] section of the system config file located at:  /etc/my.cnf . Restarting the MySQL service is necessary for the server to detect changes to the file.

~ $ cat /etc/my.cnf
[mysqld]
log-error=/var/lib/mysql/mysql.err
innodb_file_per_table=1
default-storage-engine=innodb
innodb_buffer_pool_size=128M

 

Unfortunately, MySQL does not inherently have an option to convert tables, leaving each table to be changed individually. Liquid Web’s support team has put together an easy to follow maintenance plan for this process. The script, which you can run on the necessary server via shell access (SSH) will convert all tables between storage engines.

Note
Plan accordingly when performing batch operations of this nature just in case downtime occurs. Best practice is to backup all your MySQL Databases before implementing a change of this magnitude, doing so provides an easy recovery point to prevent any data loss.

Step 1: Prep

Plan to start at a time of day where downtime would have minimal consequences. This process itself does not require any downtime, however, downtime may be necessary to recover from unforeseen circumstances.  

 

Step 2: Backup All Databases To A File

The command below creates a single file backup of all databases named all-databases-backup.sqld and can be deleted once the conversion has succeeded and there are no apparent problems.
mysqldump --all-databases > all-databases-backup.sql

 

Step 3: Record Existing Table Engines To A File

Run the following script to record the existing table engines to a file named table-engine-backup.sql. You can then “import” or “run” this file later to convert back to their original engines if necessary.

mysql -Bse 'SELECT CONCAT("ALTER TABLE ",table_schema,".",table_name," ENGINE=",Engine,";") FROM information_schema.tables WHERE table_schema NOT IN("mysql","information_schema","performance_schema");' | tee table-engine-backup.sql

If you need to revert the table engines back for any reason, run:
mysql < table-engine-backup.sql

 

Step 4a: Convert MyISAM Tables To InnoDB

The below command will proceed even if a table fails and lets you know which tables failed to convert. The output is saved to the file named convert-to-innodb.log for later review.
mysql -Bse 'SELECT CONCAT("ALTER TABLE ",table_schema,".",table_name," ENGINE=InnoDB;") FROM information_schema.tables WHERE table_schema NOT IN ("mysql","information_schema","performance_schema") AND Engine = "MyISAM";' | while read -r i; do echo $i; mysql -e "$i"; done | tee convert-to-innodb.log

 

Step 4b: Convert All InnoDB Tables To MyISAM

This command will proceed even if a table fails and lets you know which tables failed to convert. The output is also saved to the file named convert-to-myisam.log for later review.

mysql -Bse 'SELECT CONCAT("ALTER TABLE ",table_schema,".",table_name," ENGINE=MyISAM;") FROM information_schema.tables WHERE table_schema NOT IN ("mysql","information_schema","performance_schema") AND Engine = "InnoDB";' | while read -r i; do echo $i; mysql -e "$i"; done | tee convert-to-myisam.log

 

The following commands illustrate how converting a single table is accomplished.

Note
Replace database_name with the proper database name and table_name with the correct table name. Make sure you have a valid backup of the table in question before proceeding. 

Backup A Single Table To A File
mysqldump database_name table_name > backup-table_name.sql

 

Convert A Single Table To InnoDB

mysql -Bse ‘ALTER TABLE database_name.table_name ENGINE=InnoDB;’

 

Convert A Single Table To MyISAM:

mysql -Bse ‘ALTER TABLE database_name.table_name ENGINE=MyISAM;’

 

Check out our other articles in this series, MySQL Performance: Identifying Long Queries, to pinpoint slow queries within your database.  Stay tuned for our next article where we will cover caching and optimization.

VPS Server Space/Disk Quota

The term “server space” refers to the amount of disk space that is available on your server’s hard disk drive. This space varies according to server type, hosting plan and possibly by additional services that are set up and available on your Liquid Web account.

Some of the largest hard disk drives on the market now can hold up to 100TB of data. To better visualize this, 100 terabytes of data is approximately equivalent to:

  • 42,000,000,000 trillion single-spaced typewritten pages
  • 8,000,000 phone books
  • 160,000 regular compact discs
  • 20,000 DVDs
  • 200 average-sized hard disks (500GB)
  • 80 human brains (the capacity of a human being’s functional memory is estimated to be 1.25 terabytes by futurist Raymond Kurzweil in The Singularity Is Near)

Your disk space can hold many types of data including file types like HTML, TXT, CSS, PHP, PDF, JPG, MP3, MP4, compressed (tar.gz) backups, SQL databases and more. These files are in specific folders which are defined by the applications configuration files or locations you determine.

How do I locate the folders containing a particular set of data?

The location of a file depends primarily on the type of file. On a Linux server, your typical cPanel account is set up under the /home/username folder, and your cPanel account username specifies the username folder. This folder is sometimes called the top-level or root folder of your cPanel account. This root folder is not publicly accessible on the web but, contains folders which are accessible via a web browser. The root folder holds other cPanel specific system folders that use a variety of functions.

As you can see, when uploading files to your account, you’ll likely want them to be in public_html to be accessible on the web. Uploading an image.jpg file to the public_html folder makes it available at domain.com/image.jpg. Additionally, if you create a folder inside of the public_html directory and add the same image there, it would be accessible at yourdomain.com/foldername/image.jpg.

To see the location of a file, you have several options;

  1. Log into your cPanel and open the File Manager under the Files section
    cPanel >> Home >> Files >> File Manager’ here you can view all of the files and folders in your account’s root directory.cPanel File Manager
  2. When a cPanel account is initially set up, it also creates the main FTP user. You can use the servers FTP functionality to access folders from a remote location to view the file listings. Several software titles like Filezilla, Cyberduck, and WinSCP are available for this type of connection.
  3. Lastly, you can connect to the server via SSH and get access to folders/files on the server.

How do I see how much space I’m using?

Disk Usage Graph

Let’s start by reviewing a few command line examples; mainly the “du” and “df” commands.

Note
The ‘du’ command sums up the total space of files that exist on the filesystem, while the ‘df’ command shows blocks available in the file system.

The ‘df’ command (abbreviation for disk free) simply lists the space used per partition:

df Command Output

The ‘du’ command (abbreviation for disk usage) reports the sizes of directories including all of their contents and the sizes of individual files:

du Command Output

Note
There are times when the ‘du’ and ‘df’ commands show different usage amounts. Previously removed files can cause this discrepancy from a running process holding open that file. Open processes cause the ‘df’ command to report that space as still being used. The solution to this is to restart the service to close any open process.

You can also use cPanel to determine the amount of space used and where its located. If you log in to cPanel, you would need to go to cPanel >> Home >> Files >> Disk Usage to get graphical of your disk usage.

cPanel Disk Usage

Lastly, to view your server’s disk usage in your Manage account server resource graph

  1. Log into your Manage portal.
  2. Navigate to the Servers section and then click on the Plus sign (+) next to the server of focus.
  3. Click on the Dashboard button and click the link next to the Disk Usage text as seen below

graphical statistics

This view provides a graphical representation of your disk space and the used amount.

How do I prevent disk space overages?

Disk space overages can result in lost emails, backups or even websites or the server going down! Just like your car, your server requires regular server maintenance. Attention to server maintenance reduces lost data. One way to prevent disk space overages is to use cPanel’s built-in tools.

cPanel possess the ability to send “Disk Quota Warning”  emails that denote when your server is using too much space. They contain specific locations to check, and the space used. The settings for these emails notifications are in WHM (Web Hosting Manager) under the Home »Server Configuration »Tweak Settings .

Email Notifications AreaOther areas of server maintenance to check on regularly include:

  • Pruning backups
  • Logs are rotating correctly (including Domlogs, Apache2, MySql, and Chkservd)
  • Regularly archiving email
  • Using the /home directory for large user accounts

What are the dangers of being too close to Disk Quota?

When a server gets close to or is at its max disk space capacity, strange errors and problems can manifest themselves in many ways including:

  • Services (like MySQL or Apache) can error out or stop
  • Websites can become very sluggish
  • The servers overall responsiveness can become slower
  • The server may exhibit a high load
  • You may see degraded disk performance
  • The server may display an increase in I/O wait
  • The server may demonstrate an increase in CPU usage
  • The file system can go into “read-only” mode
  • The server can run out of inodes
  • Files can become corrupted
  • Decreased swap space may occur causing issues

So what do I do if I’m running out of space?

As Benjamin Franklin stated, “By failing to prepare, you are preparing to fail.” In light of this knowledge, taking steps in advance to prevent these issues is always the best course of action. Directly monitoring your server disk space on a weekly or monthly basis prevents most space issues from turning into actual problems.

If you have already reached the point where immediate action needs to be taken to bring a server back in line with normal space expectations, you have several options. Using the “du” and “df” commands are your primary weapons in tracking down used server space.

The primary steps needed are:

    1. Log into your server
    2. Run a df -h command to locate which partitions are using the most spacedf Command Output
    3. Change directories into the affected folders using the most space.
    4. Run the following command:

du-sk Command Output(This is an advanced du command that sorts the contents of a directory by size. Use this to drill down into a folder to see used space.)

  1. Move files (to a backup drive or folder) or, remove the files that are no longer needed using the ‘rm’ command.
  2. Repeat steps 2 through 6 as needed until reaching desired space level.

Final Thoughts

Over time, any operating system can become overcrowded with addition and removal of programs or accounts. Actively monitoring your servers disk space is the most effective method to prevent server space issues. If you do run into issues, using the du, df command line tools or, using the graphical interface in your account allows you to view files as needed. As always, if you have further thoughts or questions about this topic, please contact our Linux Support department for more information.

 

How To Install Apache Tomcat 8 on Ubuntu 16.04

Apache Tomcat is used to deploy and serve JavaServer Pages and Java servlets. It is an open source technology based off Apache.

Pre-Flight Check

  • This document assumes you are installing Apache Tomcat on Ubuntu 16.04.
  • Be sure you are logged in as root user.

Installing Apache Tomcat 8

Step 1: Create the Tomcat Folder

Logged in as root, within the opt folder make a directory called tomcat and cd into that folder after completion.

mkdir /opt/tomcat

cd /opt/tomcat

 

Step 2: Install Tomcat Through Wget

Click this link to the Apache Tomcat 8 Download site. Place you cursor under 8.5.32  Binary Distributions, right click on the tar file and select copy link address (as shown in the picture below). At the time of this article Tomcat 8 is the newest version but feel free to pick whatever version is more up-to-date.

Tomcat 8's Download Page

Next from your server, use wget command to download the tar to  the tomcat folder from the URL you copied in the previous step:

wget http://apache.spinellicreations.com/tomcat/tomcat-8/v8.5.32/bin/apache-tomcat-8.5.32.tar.gz

Note
You can down the file to your local desktop, but you’ll then want to transfer the file to your Liquid Web server. If assistance is needed, check out this article: Using SFTP and SCP Instead of FTP

After the download completes, decompress the file in your tomcat folder:

tar xvzf apache-tomcat-8.5.32.tar.gz

 

Step 3: Install Java

Before you can use Tomcat you’ll have to install the Java Development Kit (JDK). Beforehand, check to see if Java is installed:

java -version

If that command returns the following message then Java has yet to be installed:
The program 'java' can be found in the following packages:

To install Java, simply run the following command (and at the prompt enter Y to continue):
apt-get install default-jdk

 

Step 4: Configure .bashrc file

Set the environment variables in .bashrc with the following command:

vim ~/.bashrc

Add this information to the end of the file:
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
export CATALINA_HOME=/opt/tomcat/apache-tomcat-8.5.32

Note
Verify your file paths! If you downloaded a different version or already installed Java, you may have to edit the file path or name. Older versions of Java may say java-7-openjdk-amd64 instead of java-1.8.0-openjdk-amd64 . Likewise, if you installed Tomcat in a different folder other then /opt/tomcat (as suggested) you’ll indicate the path in your bash file and edit the lines above.

Save your edits and exit from the .bashrc file, then run the following command to register the changes:

. ~/.bashrc

 

Step 5: Test Run

Tomcat and Java should now be installed and configured on your server. To activate Tomcat, run the following script:

$CATALINA_HOME/bin/startup.sh

You should get a result similar to:

Using CATALINA_BASE: /opt/tomcat
Using CATALINA_HOME: /opt/tomcat
Using CATALINA_TMPDIR: /opt/tomcat/temp
Using JRE_HOME: /usr/lib/jvm/java-7-openjdk-amd64/
Using CLASSPATH: /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar
Tomcat started

 

To verify that Tomcat is working visit the ip address of you server:8080 in a web browser. For example http://127.0.0.1:8080.

Apache Tomcat 8 Verification Page

 

How To Install Apache Tomcat 7 on Ubuntu 16.04

Apache Tomcat is used to deploy and serve JavaServer Pages and Java servlets. It is an open source technology based off Apache.
Pre-Flight Check

  • This document assumes you are installing Apache Tomcat on Ubuntu 16.04.
  • Be sure you are logged in as root user.

Installing Tomcat 7

Step 1: Create the Tomcat Folder

Logged in as root, within the opt folder make a directory called tomcat and cd into that folder after completion.

mkdir /opt/tomcat
cd /opt/tomcat

 

Step 2: Install Tomcat Through Wget

Click this link to the Apache Tomcat 7 Download site. Place your cursor under 7.0.90 Binary Distributions, right click on the tar.gz file and select Copy Link Address (as shown in the picture below).  At the time of this article Tomcat 7 is the newest version but feel free to pick whatever version is more up-to-date.

Tomcat Version 7.0.90

Next, from your server, use wget command to download the tar to  the tomcat folder from the URL you copied in the previous step:

wget http://www.trieuvan.com/apache/tomcat/tomcat-7/v7.0.90/bin/apache-tomcat-7.0.90.tar.gz

Note
You can down the file to your local desktop, but you’ll then want to transfer the file to your Liquid Web server. If assistance is needed, check out this article: Using SFTP and SCP Instead of FTP

After the download completes, decompress the file in your Tomcat folder:

tar xvzf apache-tomcat-7.0.90.tar.gz

You will end up with a file called apache-tomcat-7.0.90.

 

Step 3: Install Java

Before you can use Tomcat, you’ll have to install the Java Development Kit (JDK). Beforehand, check to see if Java is installed:

java -version
If that command returns the following message then Java has yet to be installed:
The program 'java' can be found in the following packages:
To install Java, simply run the following command (and at the prompt enter Y to continue:
apt-get install default-jdk

 

Step 4: Configure .bashrc file

Set the environment variables in .bashrc with the following command:

vim ~/.bashrc
Add this information to the end of the file:
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
export CATALINA_HOME=/opt/tomcat/apache-tomcat-7.0.90

Note
Verify your file paths! If you downloaded a different version or already installed Java, you may have to edit the file path or name. Older versions of Java may say java-7-openjdk-amd64 instead of java-1.8.0-openjdk-amd64 . Likewise, if you installed Tomcat in a different folder other then /opt/tomcat (as suggested) you’ll indicate the path in your bash file and edit the lines above.

Save your edits and exit from the .bashrc file, then run the following command to register the changes:

. ~/.bashrc

 

Step 5: Test Run

Tomcat and Java should now be installed and configured on your server. To activate Tomcat, run the following script:

$CATALINA_HOME/bin/startup.sh

You should get a result similar to:
Using CATALINA_BASE: /opt/tomcat
Using CATALINA_HOME: /opt/tomcat
Using CATALINA_TMPDIR: /opt/tomcat/temp
Using JRE_HOME: /usr/lib/jvm/java-7-openjdk-amd64/
Using CLASSPATH: /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar
Tomcat started.

 

To verify that Tomcat is working by visiting the IP address of your server:8080 in a web browser. For example http://127.0.0.1:8080.

Tomcat 7.0.90 Test Page

 

How to Install Pip on Ubuntu 16.04 LTS

Arguably one of the easiest tools to use for installing and managing Python packages, Pip has earned is notoriety by the number of applications utilizing this tool. Fancied for its capabilities in handling binary packages over the easy_installed packaged manager, pip enables 3rd party package installations. Though Python does sometimes come with pip as a default, this tutorial will show how to install, check its version as well as some basic commands for using pip on Ubuntu 16.04.

 

Pre-Flight Check

  • These instructions are intended for an Ubuntu 16.04 LTS server, and we are logged in as root.
  • If you are using a different operating system, check out our other pip installation guides.

Step 1: 

Ensure that all packages are up-to-date. After running the command below, you’ll get an output of any packages getting their update.

apt-get update

Step 2:

Install pip with cURL and Python. Downloading using the cURL command ensures the latest version of pip.curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
python get-pip.py

Step 3: 

Verifying the installation of pip:

pip --version

Output:
pip --version
pip 18.0 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)

Installing Libraries

Pip can install 3rd party packages like Django, Tensorflow, Numpy, Pandas and many more with the following command.

pip install <library_name>

 

Searching for Libraries

You can also search for other libraries in Python’s repository via command line. For our example let’s look for Django packages. The search command shows us an extensive list similar to the one below.

pip search django
django-bagou (0.1.0) - Django Websocket for Django
django-maro (0.0.2) - `django-maro` is utility for django.
django-hooked (0.1.7) - WebHooks for Django and Django Rest Framework.
django-ide (0.0.5) - A Django app to develop Django apps
django-mailwhimp (0.1) - django-mailwhimp integrates mailchimp into Django
django-six (1.0.4) - Django-six —— Django Compatibility Library
django-umanage (1.1.1) - Django user management app for django
django-nadmin (0.1.0) - django nadmin support django version 1.8 based on django-xadmin
diy-django (1.3.1) - diy-django

 

Uninstalling a Library

If you don’t need the library and your scripts use them you can uninstall easily with this command:

pip uninstall

 

Installing Python Resources

Many times Python packages have a requirements.txt file, if you see this file, you can run this command to install all libraries in that package

pip install -r requirements.txt

 

Malware – How to Detect and Remove

Maldet, a free popular malware scanning software for Linux servers, can be used to scan an entire server for potentially malicious files. Properly configured and monitored, it can even be used to disable or fully remove malware when it is detected. However, the removal of files should only be configured once you are certain no false positives will be picked up in the scans.

 

How to Install Maldet

To Install Maldet on your linux server copy and paste the following into the command lines. Maldet will then be pre-scheduled to run daily.

pushd /usr/local/src/
rm -vrf /usr/local/src/maldetect-*
rm -vrf /usr/local/src/linux-malware-detect*
wget http://www.rfxn.com/downloads/maldetect-current.tar.gz
tar -zxvf maldetect-current.tar.gz
cd maldetect-*
sh ./install.sh
maldet --update-ver
#sed patch - commands added to address current problem with maldet overriding values in the conf file
sed -i 's/quarantine_hits=\"1\"/quarantine_hits=\"0\"/' /usr/local/maldetect/conf.maldet
sed -i 's/quarantine_clean=\"1\"/quarantine_clean=\"0\"/' /usr/local/maldetect/conf.maldet
sed -i 's/email_alert=\"1\"/email_alert=\"0\"/' /usr/local/maldetect/conf.maldet
sed -i 's/email_addr=\"you@domain.com\"/email_addr=\"\"/' /usr/local/maldetect/conf.maldet
#end sed patch
maldet --update
if [ -e /usr/local/cpanel/3rdparty/bin/clamscan ] then
ln -s /usr/local/cpanel/3rdparty/bin/clamscan /usr/bin/clamscan
ln -s /usr/local/cpanel/3rdparty/bin/freshclam /usr/bin/freshclam
if [ ! -d /var/lib/clamav ] then mkdir /var/lib/clamav
fi
ln -s /usr/local/cpanel/3rdparty/share/clamav/main.cld /var/lib/clamav/main.cld
ln -s /usr/local/cpanel/3rdparty/share/clamav/daily.cld /var/lib/clamav/daily.cld
ln -s /usr/local/cpanel/3rdparty/share/clamav/bytecode.cld /var/lib/clamav/bytecode.cld
else
echo -e "\n\e[31mClamAV does not appear to be installed through cPanel.\nThe ClamAV definitions will not be used.\e[39m\n"
fi
Popd

Scanning for Malware

Once you have completed the installation you will want to configure the scanning process. The configuration for maldet is located /usr/local/maldetect/conf.maldet. You will want to open the file with your favorite text editor such as vim or nano:

vim /usr/local/maldetect/conf.maldet
Once you are editing the file you will want to add your email address between the “ “ on the line email_addr=,  like so email_addr=“myemail@mydomain.tld”

You can also set up the scan to quarantine the malicious files it finds by changing the line quarantine_hits= from “0” to “1”, it should look like quarantine_hits=“1”. I would advise against this option as it can pick up legitimate code mistakenly. If the scan does mistakenly place a legitimate file into quarantine, you will need to move it back into place by using the following command template, replacing SCANID with the proper scan ID reported by maldet:

Maldet --restore {SCANID}
Once you have run the scan with quarantines for some time and you are confident that no safe files are being picked up, you may want to turn on removal of quarantined files in the same configuration /usr/local/maldetect/conf.maldet at the line quarantine_clean= from “0” to “1” , it should look like quarantine_clean=”1”. I would personally avoid this configuration option as it can always pick up new edits mistakenly and destroy your hard work.

Looking for pre-configured protection for servers and websites? Check out our wide security offerings that are sure to fit any of your security concerns!

MySQL Performance: Identifying Long Queries

Every MySQL backed application can benefit from a finely tuned database server. The Liquid Web Heroic Support team has encountered numerous situations over the years where some minor adjustments have made a world of difference in website and application performance. In this series of articles, we have outlined some of the more common recommendations that have had the largest impact on performance.

Preflight Check

This article applies to most Linux based MySQL servers. This includes, but is not limited to, both Traditional Dedicated and Cloud VPS servers running a variety of common Linux distributions. The article can be used with the following Liquid Web system types:

  • Core-managed CentOS 6x/7x
  • Core-managed Ubuntu 14.04/16.04
  • Fully-managed CentOS 6/7 cPanel
  • Fully-managed CentOS 7 Plesk Onyx 17
  • Self-managed Linux servers
Note
Self-managed systems, which have opted out of direct support can take advantage of the techniques discussed here, however, the Liquid Web Heroic Support Team cannot offer direct aid on these server types.

This series of articles assumes familiarity with the following basic system administration concepts:

 

What is MySQL Optimization?

There is no clearly defined definition for the term MySQL Optimization. It can mean something different depending on the person,  administrator, group or company. For the sake of this series of articles on MySQL Optimization, we will define MySQL Optimization as:  The configuration of a MySQL or MariaDB server which has been configured to avoid commonly encountered bottlenecks discussed in this series of articles.

What is a bottleneck?

Very similar to the neck on a soda bottle, a bottleneck as a technical term is a point in an application or server configuration where a small amount of traffic or data can pass through without issue. However, a larger volume of the same type of traffic or data is hindered or blocked and cannot operate successfully as-is. See the following example of a configuration bottleneck:

Visual Difference between Optimized and Non-Optimized DatabaseIn this example, the server is capable of handling 10 connections simultaneously. However, the configuration only accepts 5 connections. This issue would not manifest so long as there were 5 or less connections at one time. However, when traffic ramps up to 10 connections, half of them start to fail due to unused resources in the server configuration. The above examples illustrates the bottleneck shape where it derives its name versus an optimized configuration which corrects the bottleneck.

When Should I Optimize My MySQL database?

Ideally, database performance tuning should occur regularly and before productivity is affected. It is best practice behavior to conduct weekly or monthly audits of database performance to prevent issues from adversely affecting  applications. The most obvious symptoms of performance problems are:

  • Queries stack up and never completing in the MySQL process table.
  • Applications or websites using the database become sluggish.
  • Connection timeouts errors, especially during peak hours.

While it is normal for there to be several concurrent queries running at one time on a busy system, it becomes a problem when these queries are taking too long to finish on a regular basis. Although the specific threshold varies per system and per application, average query times exceeding several seconds will manifest as a slowdown within attached websites and applications. These slowdowns can sometimes start out small and go unnoticed until a large traffic surge hits a particular bottleneck.

Identifying Performance Issues

Knowing how to examine the MySQL process table is vital for diagnosing the specific bottleneck being encountered. There is a number of ways to view the process table depending on your particular server and preference. For the sake of brevity this series will focus on the most common methods used via Secure Shell (SSH) access:

 

Using The MySQL Process Table: Method 1

Use the ‘mysqladmin’ command line tool with the flag ‘processlist’ or ‘proc’ for short. (Adding the flag ‘statistics’ or ‘stat’ for short will show running statistics for queries since MySQL’s last restart.)

Command:

mysqladmin proc stat

Output:

 +-------+------+-----------+-----------+---------+------+-------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-------+------+-----------+-----------+---------+------+-------+--------------------+----------+
| 77255 | root | localhost | employees | Query | 150 | | call While_Loop2() | 0.000 |
| 77285 | root | localhost | | Query | 0 | init | show processlist | 0.000 |
+-------+------+-----------+-----------+---------+------+-------+--------------------+----------+
Uptime: 861755 Threads: 2 Questions: 20961045 Slow queries: 0 Opens: 2976 Flush tables: 1 Open tables: 1011 Queries per second avg: 24.323

Pro: Used on the shell interface, this makes piping output to other scripts and tools very easy.
Con: The process table’s info column is always truncated so does not provide the full query on longer queries.

Using The MySQL Process Table: Method 2

Run the ‘show processlist;’ query from within MySQL interactive mode prompt. (Adding the ‘full’  modifier to the command disables truncation of the Info column. This is necessary when viewing long queries.)

 

Command:

show processlist;

Output:
MariaDB [(none)]> show full processlist;
+-------+------+-----------+-----------+---------+------+-------+-----------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-------+------+-----------+-----------+---------+------+-------+-----------------------+----------+
| 77006 | root | localhost | employees | Query | 151 | NULL | call While_Loop2() | 0.000 |
| 77021 | root | localhost | NULL | Query | 0 | init | show full processlist | 0.000 |
+-------+------+-----------+-----------+---------+------+-------+-----------------------+----------+

Pro: Using the full modifier allows for seeing the full query on longer queries.
Con: MySQL Interactive mode cannot access scripts and tools available in the shell interface.

Using The slow query log

Another valuable tool in  MySQL is the included slow query logging feature. This feature is the preferred method for finding long running queries on a regular basis. There are several directives available to adjust this feature. However, the most commonly needed settings are:

 

slow_query_logenable/disable the slow query log
slow_query_log_filename and path of the slow query log file
long_query_timetime in seconds/microseconds defining a slow query

These directives are set within the [mysqld] section of the MySQL configuration file located at /etc/my.cnf and will require a MySQL service restart before they will take affect. See the example below for formatting:

Caution
There is a large disk space concern with the slow query log file, which needs to be attended to continually until the slow query log feature is disabled. Keep in mind, the lower your long_query_time directive the faster the slow query log fills up a disk partition.
[mysqld]
log-error=/var/lib/mysql/mysql.err
innodb_file_per_table=1
default-storage-engine=innodb
innodb_buffer_pool_size=128M
innodb_log_file_size=128M
max_connections=300
key_buffer_size = 8M
slow_query_log=1
slow_query_log_file=/var/lib/mysql/slowquery.log
long_query_time=5

Once the slow query log is enabled you will need to periodically follow-up with it to review unruly queries that need to be adjusted for better performance. To analyze the slow query log file, you can parse it directly to review its contents. The following example shows the statistics for the sample query which ran longer that the configured 5 seconds:

Caution
There is a performance hit taken by enabling the slow query log feature. This is due to the additional routines needed to analyze each query as well as the I/O needed to write the necessary queries to the log file. Because of this, it is considered best practice on production systems to disable the slow query log. The slow query log should only remain enabled for a specific duration when actively looking for troublesome queries that may be impacting the application or website.
# Time: 180717 0:23:28
# User@Host: root[root] @ localhost [] # Thread_id: 32 Schema: employees QC_hit: No
# Query_time: 627.163085 Lock_time: 0.000021 Rows_sent: 0 Rows_examined: 0
# Rows_affected: 0
use employees;
SET timestamp=1531801408;
call While_Loop2();

Optionally, you can use the mysqldumpslow command line tool, which parses the slow query log file and groups like queries together except values of number and string data:
~ $ mysqldumpslow -a /var/lib/mysql/slowquery.log
Reading mysql slow query log from /var/lib/mysql/slowquery.log
Count: 2 Time=316.67s (633s) Lock=0.00s (0s) Rows_sent=0.5 (1), Rows_examined=0.0 (0), Rows_affected=0.0 (0), root[root]@localhost
call While_Loop2()
(For usage information visit MySQL documentation here: mysqldumpslow – Summarize Slow Query Log Files)

So concludes the first part of our Database Optimization series and gives us a solid basis to refer back to for benchmark purposes. Though database issues can be complicated, our series will break down these concepts to provide means to optimize your database through database conversion, table conversion, and indexing.

 

Upgrade PHP 5.6 to 7


PHP is a programming language that can run with Apache or Microsoft IIS and works with your server to execute the requests that make up your website. 88% of online sites run on, soon to be vulnerable PHP 5.X technology. At the close of this year, scheduled by Dec 31, 2018 security support will end for our dear old friend PHP 5.6, meaning bugs and security fixes will not be tended to and could lead to security vulnerabilities. 
Each PHP version gets supported actively for two years while the third year only get critical security updates. Luckily, the PHP gods had smiled upon us and extended the life for just a year longer than the typical PHP version before giving us the new year deadline. For all of you developers out there wanting to know exactly what is changing, here’s a helpful migration guide from PHP 5.6 to PHP 7.X.

While the last of PHP 5 closes out with PHP 5.6, this will inevitably leave websites utilizing PHP 5 vulnerable to attacks as well as poor performance. It has substantially reached its infamous End of Life (EOL) title. Switching to the newer PHP 7 versions is not only good for the security, but updating can ultimately save you money. Reducing the cost of doing business by avoiding software incompatibility and compliance issues. If an emotional headache isn’t enough to persuade developers to switch, the benefits will. Benchmarks show PHP 7.x has been tested to run three times faster than PHP5.6!

Let’s see:

  • Faster performance resulting in less memory usage
  • Three times faster page loads*
  • Better for heavy traffic sites
*Performance increase as benchmarked in a testing environment. Other developer’s website performance changes between PHP 5 and PHP 7 may vary.

If you are in a shared environment that manages the OS and framework, then your hosting provider should be sending out notifications of the upcoming change, their plan of action, and cut off dates. Our managed hosting products, such as Storm VPS, Cloud Sites or Managed WordPress, have support teams that can help you switch from PHP 5.X to PHP 7.X easily. Our Managed WordPress product has a compatibility checker built in & one click button to upgrade, yet another reason to love it!


While using WordPress to power your site you can check some vital aspects by going to the
WordPress plugin page and searching for the plugins that you use. Once you find the plugin or themes that you utilize, their spec pages will usually say what PHP version they employ. Also, check out the review tab for comments from users as this section gives useful information. This review tab is helpful for seeing if others have had issues with the plugin or theme and newer PHP versions. It is good practice to look up reviews and see what people have been saying about said plugin. If you don’t see any responses or it hasn’t rated well, then you will want to stray away from it. If you use custom plugins, check with your developer to see how they operate in new PHP versions. The WordPress Compatibility Plugin check will give you a list of plugins and themes that may not mesh well with PHP 7.X.

If you run a mission-critical site its best to do a compatibility checker because blindly upgrading could result in some parts of your page to not function. Checking PHP compatibility, as you would imagine, is a little more in depth but from research online, there is a compatibility checker for VPS servers that you can utilize by downloading the repo from GitHub.

It is worthwhile to note that some plugins may need a PHP module to be installed for the plugin to work. When upgrading the PHP version, you may also need to re-install the PHP module. Fortunately, our support team can assist with installing any PHP module you may need or give the best course of action if the PHP module is not available for your PHP version.

If you are using a Linux VPS the easiest way to check is to ssh into your server and run the following command via your terminal:

php --versionOutput: PHP 7.0.30 (cli) (built: Jun 26 2018 20:34:16)

cPanel:

Note
It’s important to make a backup of your site before migrating to PHP 7.X

Search php, select Multi PHP Manager, will show this screen to show which php version you are using. While on the PHP Version screen you can update the PHP version here by clicking on the check mark next to the domain and selecting the desired PHP version on the right drop down and click Apply.

Search For PHP and Click MulitPHP Manager Icon