Apache Performance Tuning: Swap Memory

Before we get into the nitty-gritty of Apache tuning, we need to understand what happens when servers go unresponsive due to a poorly optimized configuration. An over-tuned server is one that is configured to allow more simultaneous requests (ServerLimit) than the server’s hardware can manage. Servers set in this manner have a tipping point, once reached, the server will become stuck in a perpetual swapping scenario. Meaning the Kernel is stuck rapidly reading and writing data to and from the system swap file. Swap files have read/write access speeds vastly slower than standard memory space. The swap files’ latency causes a bottleneck on the server as the Kernel attempts to read and write data faster than is physically possible or more commonly known as thrashing. If not caught immediately, thrashing spirals the system out of control leading to a system crash. If trashing is left running for too long, it has the potential of physically harming the hard drive itself by simulating decades of read/write activity over a short period. When optimizing Apache, we must be cautious not to create a thrashing scenario. We can accomplish this by calculating the thrashing point of the server based on several factors.

 

Pre-Flight Check

This article covers all Apache-based servers including but is not limited to, both traditional Dedicated servers and Cloud VPS servers running a variety of Linux distributions. We will include the primary locations where stored Apache configurations on the following Liquid Web system types:

Liquid Web Server Types

Calculating the estimated thrashing point or ServerLimit of a server uses a simple equation:

Caculating the Thrashing Point

  • buff/cache: The total memory used by the Kernel for buffers and cache.
  • Reserved: The amount of memory reserved for non-Apache processes.
  • Available: The difference between buff/cache and Reserved memory.
  • Avg.Apache: The average of all running Apache children during peak operational hours.

 

Important
Calculating the thrashing point/ServerLimit should be done during peak operational hours and periodically reassessed for optimal performance.

The thrashing point value is equal to the number Apache children the server can run; this applies to either threaded or non-thread children. When the number of children running in memory meets the calculated thrashing point, the server will begin to topple. The following example walks through a standard Liquid Web Fully Managed cPanel server to illustrate gathering the necessary details to calculate a system’s estimated thrashing point.

 

Buff/Cache Memory

On modern Linux systems, the buffer/cache can be derived using the /proc/meminfo file by adding the Buffers, Cached and Slab statistics. Using the free command, we can quickly grab this information, as in the example below:

free

Output:

Use the free command to get the buff/cache info

Don’t be fooled by the column labeled “available.” We are solely looking at the memory we can reappropriate, which is the buff/cache column (708436).

 

Reserved Memory

Reserved memory is a portion of memory held for other services aside from Apache. Some of the biggest contenders for additional memory outside of Apache are MySQL, Tomcat, Memcache, Varnish, and Nginx. It is necessary to examine these services configs to determine a valid reserved memory. These configs are outside the purview of this article. However, MySQL is the most commonly encountered service running with Apache. You can find tools online to help analyze and configure MySQL separate from this article.

 

Rule-of-Thumb:
Save 25% of the total buff/cache memory for any extra services ran on the server. Examples:

  • A standard cPanel server runs several services along with Apache and MySQL. A server with these services runs on the heavier side, needing 25% reserved for non-Apache services.
  • A pure Apache web node in a high capacity load balanced configuration does not need to reserve any additional ram for other services.

 

Average Apache Memory

Finding the average size of Apache processes is relatively simple using the ps command to list the RSS (Resident Set Size) of all running httpd processes. Note: some distributions use “apache” instead of “httpd” for process name.

This example uses a short awk script to print out the average instead of listing the sizes.

ps o rss= -C httpd|awk '{n+=$1}END{print n/NR}'

Output:

22200.6

 

This example is easy to average by hand, but larger servers will require more calculation.

ps o rss -C httpd

Output:

RSS
5092
27940
28196
27572

 

Calculate the Thrashing Point

Once collected divided the Available memory by Avg. Apache, rounding down to the nearest whole number. Available memory is the buff/cache memory minus the Reserved memory. Below is a summary of the calculation process in table form.

Thrashing point calculations

Conservative estimates are provided below for various memory configurations. These estimates can be used as a starting configuration but will require additional follow up performance assessments during peak hours to adjust directives by the servers.

General Thrashing Estimates

Install Multiple PHP Versions on Ubuntu 16.04

As a default, Ubuntu 16.04 LTS servers assign the PHP 7.0 version. Though PHP 5.6 is coming to the end of life in December of 2018, some applications may not be compatible with PHP 7.0. For this tutorial, we instruct on how to switch between PHP 7.0 and PHP 5.6 for Apache and the overall default PHP version for Ubuntu.

Step 1: Update Apt-Get

As always, we update and upgrade our package manager before beginning an installation. If you are currently running PHP 7.X, after updating apt-get, continue to step 2 to downgrade to PHP 5.6.

apt-get update && apt-get upgrade

Step 2: Install PHP 5.6
Install the PHP5.6 repository with these two commands.

apt-get install -y software-properties-common
add-apt-repository ppa:ondrej/php
apt-get update
apt-get install -y php5.6

Step 3: Switch PHP 7.0 to PHP 5.6
Switch from PHP 7.0 to PHP 5.6 while restarting Apache to recognize the change:

a2dismod php7.0 ; a2enmod php5.6 ; service apache2 restart

Note
Optionally you can switch back to PHP 7.0 with the following command: a2dismod php5.6 ; a2enmod php7.0 ; service apache2 restart

Verify that PHP 5.6 is running on Apache by putting up a PHP info page. To do so, use the code below in a file named as infopage.php and upload it to the /var/www/html directory.

<? phpinfo(); ?>

By visiting http://xxx.xxx.xxx.xxx/infopage.php (replacing the x’s with your server’s IP address), you’ll see a PHP info banner similar to this one, confirming the PHP Version for Apache:

Example of PHP Info page

Continue onto the section PHP Version for Ubuntu to edit the PHP binary from the command line.

Step 4: Edit PHP Binary

Maintenance of symbolic links or the /etc/alternatives path through the update-alternatives command.

update-alternatives --config php

Output:
There are 2 choices for the alternative php (providing /usr/bin/php).
Selection Path Priority Status
------------------------------------------------------------
* 0 /usr/bin/php7.0 70 auto mode
1 /usr/bin/php5.6 56 manual mode
2 /usr/bin/php7.0 70 manual mode
Press to keep the current choice[*], or type selection number:

Select php5.6 version to be set as default, in this case, its the number one option.

You can now verify that PHP 5.6 is the default by running:
php -v

Output:
PHP 5.6.37-1+ubuntu16.04.1+deb.sury.org+1 (cli)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

Install Nginx on Ubuntu 16.04

Nginx is an open source Linux web server that accelerates content while utilizing low resources. Known for its performance and stability Nginx has many other uses such as load balancing, reverse proxy, mail proxy and HTTP cache. With all these qualities it makes a definite competitor for Apache. To install Nginx follow our straightforward tutorial.

Pre-Flight Check

  • Logged into an as root and are working on an Ubuntu 16.04 LTS server powered by Liquid Web! If using a different user with admin privileges use sudo before each command.

Step 1: Update Apt-Get

As always, we update and upgrade our package manager.

apt-get update && apt-get upgrade

Step 2: Install Nginx

One simple command to install Nginx is all that is needed:

apt-get -y install nginx

Step 3: Verify Nginx Installation

When correctly installed Nginx’s default file will appear in /var/www/html as index.nginx-debian.html . If you see the Apache default page rename index.html file. Much like Apache, by default, the port for Nginx is port 80, which means that if you already have your A record set for your server’s hostname you can visit the IP to verify the installation of Nginx. Run the following command to get the IP of your server if you don’t have it at hand.

ip addr show eth0 | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'

Take the IP given by the previous command and visit via HTTP. (http://xxx.xxx.xxx.xxx) You will be greeted with a similar screen, verifying the installation of Nginx!

Nginx Default Page

Note
Nginx, by default, does not execute PHP scripts and must be configured to do so.

If you already have Apache established to port 80, you may find the Apache default page when visiting your host IP, but you can change this port to make way for Nginx to take over port 80. Change Apache’s port by visiting the Apache port configuration file:

vim /etc/apache2/ports.conf

Change “Listen 80” to any other open port number, for our example we will use port 8090.

Listen 8090

Restart Apache for the changes to be recognized:

service apache2 restart

All things Apache can now be seen using your IP in the replacement of the x’s. For example http://xxx.xxx.xxx.xxx:8090

Migration to Managed WooCommerce

Liquid Web is here to support your migration needs into our Managed WooCommerce Hosting platform. Whether you are migrating from an external or internal source, our in-house team of migration experts transforms the data migration process into a simple task. To ensure the smoothest and best possible data transfer, we have a quick overview and a few points for your consideration.

 

Our first step includes taking a copy of your live site (known as the origin site) and migrating it over to our Managed WooCommerce Hosting platform. Rest assured, when performing the migration, the only changes made to the site will be to assist in the movement. Within this timeframe, it is advised to avoid making changes or updates to the site as it will extend the migration timeline and could result in data loss. Changes and updates are included but not limited to themes, designs, contents, products, blog posts or WordPress versions. The initial sync process should result in no downtime for your live site.

Once the initial sync is complete, our Migration Specialists perform a series of basic tests to the site. During this time, our team will send information on ways to test out your new site to ensure that all aspects have carried over correctly and are in working order. Before going live, it is essential to take the time to thoroughly review your site and if at any point you do find a discrepancy our specialist is there to assist.

The third and most exciting step is the push to go live. We will coordinate the best date and time for the final sync of your site. This last sync will ensure the latest data on orders, products, and customers transfers to your new server. Upon completion of the final sync, you will be asked to update the staging domain’s name and DNS record. With a little DNS propagation time, you will begin to see the new site populate!

With the updating of DNS and the site name, you are now entirely done with the migration process. In subsequent steps, we will create a ticket with our Product Team to connect your store to our partnered applications, Glew and Jilt. Credentials to these valued applications will be sent out in an email, after which, our product team can suggest performance optimization methods to get the most out of your eCommerce store.

 

Knowing the details behind the migration process aligns us with our next step in creating a migration request from your Liquid Web control panel! Once completed, our Migration Specialists will be in touch to schedule the migration and answer any questions you may have.

 

Install XFCE Desktop Environment on Ubuntu 16.04

Since 1996, XFCE Desktop gives users the ability to have a graphical user interface (GUI) environment, visually turning your Linux server into an environment more like your desktop computer. With its no-frills look, XFCE does not weigh heavy on the server’s hardware and is faster than GNOME and KDE to boot. Once completed with this small tutorial, you’ll be able to share and connect to the XFCE GUI by continuing to the next tutorial on How To Install VNC.

Pre-flight

  • These instructions are intended for installing Xfce Desktop Environment on an Ubuntu 16.04 LTS server.
  • Logged in as a root user, but for non-root users precede all commands with the word sudo.

Step 1: Update apt-get
With best practices in mind, we will update before proceeding to install XFCE 4

apt-get update

Step 2: Install XFCE4 Desktop Environment
With one command we can install Xfce itself and some useful utilities that come with Xfce:
apt-get install -y xfce4 xfce4-goodies
 

Step 1:
Run each of these commands so that apt-get can utilize them during the purge of Xfce.
apt-get -f install
apt-get clean
apt-get autoclean
apt-get update

Step 2:
Purge Xfce from your Ubuntu server:
apt-get purge xfce4

As mentioned in our opening paragraph the next step is to configure VNC (virtual network computing) Installing VNC is necessary to open the recently installed Xfce interface. It’s optional but advisable to set up an SSH tunnel that connects to VNC for a secure connection.  Check out our Knowledge Base on the subject of VNC to find your choice of articles.

 

Install Memcached on Ubuntu 16.04

Memcached works to enhance performance by keeping a copy of commonly used script elements within the server’s memory in a form that is more easily read by the server thus reducing time. A bonus feature of this object cacher is its ability to decrease the number of connections to your database. In this tutorial, we instruct how to install Memcached, but it’s important to note that when using Memcache in an application, the application must be specially coded or configured to store and retrieve data this cached data.

Learn more about caching from our dedicated article or visit our series for database optimization.

Pre-flight

  • We are logged in as root on an Ubuntu 16.04 VPS powered by Liquid Web!
  • Installed and running Apache and PHP 7.

Installation of Memcached

Step 1:
Following best practices, we will do a quick package update by using the following command:
apt-get update
Step 2:
Install the Memcached daemon using
apt-get install memcached -y
Step 3:
Install the Memcache module for PHP fuctionality:
apt-get install php-memcached -y

Verify installation of Memcached

Use the php -m flag to show compiled modules while sorting through specifically looking for memcached.

php -m | grep memcached
memcached

Optional Configurations

At some point, you may find that you need to change the default settings of Memcached. These include adjusting the port number, memory for your cache, and the listening IP address.
vim /etc/memcached.conf

Adjust these configurations by keeping the same flags (-m, -p, -u, -l), adjusting the letter or number after the flag and save the file by typing :wq .
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m 64
 
# Default connection port is 11211
-p 11211
 
# Run the daemon as root. The start-memcached will default to running as root if no
# -u command is present in this config file
-u memcache
 
# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
# it's listening on a firewalled interface.
-l 127.0.0.1

 

Restart your Memcached service to recognize the changes to this file:
systemctl restart memcached

How to Remove (Delete) a User on Ubuntu 16.04

User management includes removing users who no longer need access, removing their username and any associate root privileges are necessary for securing your server. Deleting a user’s access to your Linux server is a typical operation which can easily be performed using a few commands.  

Pre-flight Check

  • We are logged in as root on an Ubuntu 16.04 VPS powered by Liquid Web!

Step 1: Remove the User

Insert the username you want to delete by placing it after the userdel command. In our example, I’ll be deleting our user, Tom.

userdel tom

Simultaneous you can delete the user and the files owned by this user with the -r flag.  Be careful these files are not needed to run any application within your server.

userdel -r tom

If the above code produces the message below, don’t be alarmed, it is not an error, but rather /home/tom existed but /var/mail/tom did not.

userdel: tom mail spool (/var/mail/tom) not found

 

Step 2: Remove Root Privileges

By removing Tom’s username from our Linux system we are halfway complete, but we still need to remove their root privileges.

visudo

Navigate to the following section:

## Allow root to run any commands anywhere
root ALL=(ALL:ALL) ALL
tom ALL=(ALL:ALL) ALL

Or:

## User privilege specification
root ALL=(ALL:ALL) ALL
tom ALL=(ALL:ALL) ALL

With either result, remove access for your user by deleting the corresponding entry:

tom ALL=(ALL:ALL) ALL

Save and exit this file by typing :wq and press the enter key.

To add a user, see our frequently used article, How to Add a User and Grant Root Privileges on Ubuntu 16.04. Are you using a different Ubuntu version? We’ve got you covered, check out our Knowledge Base to find your version.

Create a Robots.txt File

A web robot’s primary job is to scan websites and pages for information; they work tirelessly to collect data on behalf of search engines and other applications. For some, there is good reason to keep pages away from search engines.  Whether you want to fine-tune access to your site or want to work on a development site without showing up Google results, once implemented the robots.txt file lets web crawlers know which parts they can collect information.

Create a Robots.txt File

As being one of the first aspects analyzed by crawlers, the robots.txt file can be implemented on a page(s) or an entire site to discourage search engines from showing details about your site. Through this article, we will be providing insight into how to use the robots.txt file as well as syntax needed to keep these bots at bay.

User-agent: *
Disallow: /

 

Let’s break down the code below “user-agent” pertains to the web crawlers and the * sign means all web crawlers. Consequently, the first line grabs attention by saying “Listen up all web crawlers!” We move onto our second line which lets the web crawler know its direction. The forward slash (/) stops the bots from searching all the pages on your site. You can also discourage information collected for one specific page, in this case, it is a map of our building layout. Since the design of our building does not need to searchable, with the command below, I can tell all bots to leave out the index of the buildinglayout.png photo, while keeping it viewable to any guest that want to view.

User-agent: *
Disallow: /buildinglayout.png

 

Contrary, if you would like for all search engines to collect information on all the pages in your site you can leave the Disallow section blank.

User-agent: *
Disallow:

 

There are many types of web crawlers (aka user-agents) that can be specified. Below is a chart of the most popular web crawlers followed by as their associations. Furthermore, you can also instruct these bots to index a certain page by using Allow, as shown in the example below. You can implement these web crawlers within your robots.txt file like so:

User-agent:Googlebot
Allow: /parkinglotmap.png
Disallow: /buildinglayout.png

User-agent and their Association
Mostly, sites don’t automatically come with a robots.txt file (and isn’t required) so you can create one using a text editor and upload the file to your root directory or any other directory.  Luckily, if you use the popular CMS, WordPress and its helpful SEO plugin Yoast, you’ll see a section within the admin window to create a robots.txt file.

Robots.txt File In WordPress

Yoeast SEO Tool Section

 

After logging into your WordPress backend (yourdomain.com/wp-login.php) locate the SEO section and select Tools. After clicking on the file editor link, you see a page that looks similar to the code used in the first of our article.

Wordpress Robots.txt File

 

Our example keeps web bots from WordPress login page, including wp-includes directory while still allowing users and bots to see other pages of our site. Take note of the necessary ending slashes after the directory (but not needed when disallowing pages). After editing select the “save changes to robots.txt” button to activate the robots.txt file.

 

 

How to Add a User and Grant Root Privileges on Ubuntu 16.04

Ubuntu 16.04 LTS provides you the ability to add a user for anyone who plans on accessing your server.  Creating a user is a basic setup but an important and critical one for your server security. In this tutorial, we will create a user and grant administrative access, known as root, to your trusted user.

 

Pre-Flight Check

  1. Open a terminal and log in as root.  
  2. Work on a Linux Ubuntu 16.04 server

Step 1:  Add The User

Create a username for your new user, in my example my new user is Tom:

adduser tom

You’ll then be prompted to enter a password for this user.   We recommend using a strong password because malicious bots are programmed to guess simple passwords. If you need a secure password, this third party password generator can assist with creating one.

Output:

~# adduser tom
Adding user `tom' ...
Adding new group `tom' (1002) ...
Adding new user `tom' (1002) with group `tom' ...
Creating home directory `/home/tom' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Note
Usernames should be lowercase and avoid special characters. If you receive the error below, alter the username. ~# adduser Tom
adduser: Please enter a username matching the regular expression configured via the NAME_REGEX[_SYSTEM] configuration variable.  Use the `--force-badname' option to relax this check or reconfigure NAME_REGEX.

 

Prompts will appear to enter in information on your new user.  Entering this information is not required and can be skipped by pressing enter in each field.

Enter the new value or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:

 

Lastly, the system will ask you to review the information for accuracy.  Enter Y to continue to our next step.

Is the information correct? [Y/n]

 

Step 2: Grant Root Privileges

Assigning a user root access is to grant a user the highest power.  My user, Tom, can then make changes to the system as a whole, so it’s critical to allow this access only to users who need it. Afterward, Tom will be able to use sudo before commands that are usually designed to be used by the root user.

usermod -aG sudo tom

 

Step 3: Verify New User

As root, you can switch to your new user with the su – command and then test to see if your new user has root privileges.

su - tom

If the user has properly been granted root access the command below will show tom in the list.

grep '^sudo' /etc/group

Output:

sudo:x:27:tom

 

How To Install Oracle Java 8 in Ubuntu 16.04

Pre-Flight Check

  1. Open the terminal and log in as root.  If you are logged in as another user, you will need to add sudo before each command.
  2. Working on a Linux Ubuntu 16.04 server
  3. No installations of previous Java versions

Step 1:  Update & Upgrade

It is advised to update your system by copy and pasting the command below.  Be sure to accept the update by typing Y when asked to continue:

apt-get update && apt-get upgrade

 

Step 2: Install the Repository

WebUpd8 Team Personal Package Archive (PPA), a third party repository,  allows us to download the package necessary for Java 8 installation.  Press Enter to continue the installation.

add-apt-repository ppa:webupd8team/java

Once again, update your package list.

apt-get update

 

Step 3: Install Java 8

Use the apt-get command to install Oracle’s Java 8 via their installer:

apt-get install oracle-java8-installer

 

Click Y to continue and press Enter to agree to the licensing agreement.

 

Select Yes and hit the Enter key.

 

Step 4: Verify Java 8 is Installed

java -version

Output:

java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

 

It’s essential to know the path of our Java installation for our applications to function. Where is Java installed? Run this command to find its path:update-alternatives --config java

Output:

~# update-alternatives --config java
There is 1 choice for the alternative java (providing /usr/bin/java).
Selection Path Priority Status
------------------------------------------------------------
0 /usr/lib/jvm/java-8-oracle/jre/bin/java 1081 auto mode
* 1 /usr/lib/jvm/java-8-oracle/jre/bin/java 1081 manual mode

 

Copy the highlighted path from the second row: /usr/lib/jvm/java-8-oracle/jre/bin/java/.  After copying, open the file /etc/environment and add in the path of your Java installation to the end of your file.

vim /etc/environment

JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre/bin/java"

 

Save the file by hitting ESC button and type :wq to execute the command below to recognize the changes to the file:

source /etc/environment

 

You should now see the path of installation when using the $Java_Home variable:

echo $JAVA_HOME

Output:

~# echo $JAVA_HOME
/usr/lib/jvm/java-8-oracle/jre/bin/java