How to Check Server Load on a Windows Server

Reading Time: 6 minutes

What Does Server Load Mean?

Checking a server’s load allows us to evaluate server resources and confirm they are sufficient for any running application. It enables us to troubleshoot slow performance and reliably pinpoint any server resource that may need attention.

While there are many tools and options available, today let’s focus on Windows Task Manager as a way to help us quickly see what is going on, and interact with applications, processes, and services to identify the load. This article will also include an introduction to Resource Monitor as it can be opened from Task Manager to provide more detail.

Ways to start Task Manager

  • Click the Start menu and type task… then choose Task Manager
  • Right-click the Taskbar area and choose Task Manager from the menu
  • Press Ctrl+Alt+End keys on the keyboard when in a Remote Desktop session
  • Run the command taskmgr

Let’s bring up the Task Manager and take a look at what it has to offer.

On systems where it has never been used, you may find Task Manager offering this very uninteresting display. Click More details to discover the treasure trove of information it is hiding.

Task Manager provides quick access to Processes, Performance, Users, Details, and Services. We’ll go through each tab to see what they have to offer and discover what to look for when checking server load.

Processes

The Processes tab shows us everything that is running in the system and the amount of CPU and memory resources it is using. At the top, we can see the total CPU and memory utilization.

By clicking the CPU or Memory column headings, we can sort the processes list by that criteria, and use the sort arrow to determine whether to sort from highest to lowest usage or the opposite. You can click on any individual process and end the task, see resource usage, and more.

Troubleshooting Tip:
If we see a particular application is using a high amount of the CPU or memory, it may be a potential source of performance issues. In the example above, we can see this server is using 78% of memory and only a very small amount of the CPU.

Performance

Performance tab has the most visual display of information and allows us to select from CPU, Memory, and Ethernet views to show activity over a 60 second period. With this view, we can identify spikes or see the trend over time to determine if a condition is temporary or sustained.

 

CPU Performance

CPU performance information shows us the type of CPU and speed, the number of processes, threads, and handles in use, as well as the number of virtual CPUs, in most cases. We can also see how long the system has been up (up time). This last bit of information can tell us how long the server has been running, confirm if it successfully completed a restart, or if it rebooted unexpectedly due to running out of resources.

Troubleshooting Tip:
In this example, we see the CPU is at 94%. If this level or higher is sustained over a long period of time, server performance will be sluggish, and it could affect the stability of the system. Sustained high CPU use is an indicator the system is struggling. We need to look at other systems to determine whether it is due to applications or insufficient physical memory that pushes the system to use virtual memory. Doing this will cause the CPU and disk resources to spike and remain high.

 

Memory Performance

Memory Performance information shows us the total amount of memory in the system as well as what is in use and available. Committed represents virtual memory and the pagefile (an extension of RAM) on disk. Cached represents memory used by Windows, and the Paged pool represents memory used by Windows that can be paged out to the pagefile on disk if memory starts running low. Non-paged cannot be paged to the pagefile.

Troubleshooting Tip:
n this example, we see the CPU is at 94%, Memory is at 90%, and we are using virtual memory. When looking at the Committed Memory, we can see that virtual memory is 2.7 GB while the pagefile is 4.9 GB. In this example, we have not maxed out the pagefile. If we find the system is continuously running with the CPU and Memory at or above 90%, it is a strong indicator to add physical memory to the system to reduce the use of virtual memory.

 

Ethernet Performance

Ethernet performance information shows us the type of network adapter and the amount of resources it is using with a graphed line for both send and receive as well as numeric values for data being sent. We can also see the Adapter name, Connection type, and the IP address(es) assigned. Right-clicking on the graph will allow us to see network details including network utilization, link speed and state, bytes send and received, etc. On the Performance tab, we also have the option to launch Resource Monitor to see even more detail.

 

Users

The Users tab shows us a list of all the users connected to the server and how much CPU and memory resources the user is utilizing. We can click on a specific user to Disconnect them, send them a message, or take over their session if we have Administrator rights. In the context of checking for load, we can determine if a specific user is consuming too many resources or has disconnected from a session, leaving it running in memory, and choose whether to log the user out to free up resources.

Details

The Details tab shows us a list of all the running programs and processes along with their PID (Process ID) number, whether the program is running or suspended, the user name it is running under, the amount of CPU and memory it is using, and a description of the process. You can click any of the column names to sort by that column in highest to lowest or the opposite order. The PID number can be very helpful to track down a specific process that is referenced in event logs. Right-clicking an item allows us to choose options including:

  • ending a process or process tree
  • set a priority for the running process
  • establish affinity to a specific processor or all processors
  • additional options

 

Services

The Services tab shows us a list of service names, their PID (Process ID) numbers, a description of the service, the status as either stopped or running, and the Group the service is running under. Right-clicking on a service allows us to start, stop, restart, and access additional options. We should be careful not to change the status of some services as they depend on others, and stopping the wrong one could have unintended consequences on the system or devices. To learn more about a service, we can right-click it and choose Search Online.

How Do I Check My Resouce Monitor?

Ways to start Resource Monitor

  • Click the Start menu and type resource… then choose Resource Monitor
  • Right-click the Taskbar area and choose Task Manager from the menu, then from Performance tab choose Open Resource Monitor
  • Run the command resmon

Let’s bring up Resource Monitor and take a look at what it has to offer. You’ll find this has more depth but is very similar to the information available from Task Manager. For this reason, we’ll only cover the overview and a brief description of each tab in this article.

Overview provides us with data on CPU, Memory, Disk, and Network options and graphs all on one page with the option to expand or collapse each section. It will also show current usage of a resource as well as the highest active time. Clicking individual sections provides more detail.

CPU shows processes, services, associated handles, and modules, and will show individual CPUs and their load in addition to total CPU.

Memory shows processes in addition to a breakdown of the physical memory and graphs to show commit charge which relates to use of the pagefile and the number of hard faults per second which can be an indicator of how many times Windows has to access the swap file. If your system is showing hundreds of hard faults per second, this indicates a need more physical memory.

Disk shows the processes in addition to a breakdown of how much each task is reading and writing to disk. The graphs show total disk activity in addition to Queue Length. Disk Queue length indicates how many disk I/O operations are queued up waiting for their turn to be processed by the disk. If we find that the highest active time is above 80% and the disk queue length is 2 or higher, it means processes are waiting, and the performance of the disk is affecting the overall performance of the system. In many cases, this number will be high due to a system that lacks sufficient physical memory and is constantly paging information to disk or relying too heavily on virtual memory. It will often be accompanied by a CPU running above 90% for sustained periods.

Network shows the processes with network activity, in addition to TCP connections and listening ports, and graphs to show network transfer and TCP connections. Sustained high network utilization can indicate congestion issues and a need for more capacity.

Still having trouble determining what is bogging down your server?  With Liquid Web’s servers, you can talk to a experienced support tech night or day.  Our techs have the expertise needed to help determine bottlenecks in your system. Switch to Liquid Web today and get the support you’ve been looking for!

MySQL Performance: InnoDB Buffers & Directives

Reading Time: 6 minutes

As discussed earlier in our MySQL Performance series, the InnoDB storage engine is designed to be a high-performance database for very large datasets. The row-locking technique it uses allows for many read and write requests to occur on a single table concurrently. This is a vast improvement in speed over traditional Continue reading “MySQL Performance: InnoDB Buffers & Directives”

MySQL Performance: System Configuration File & Routine Maintenance

Reading Time: 3 minutes

The majority of work needed when adjusting the MySQL server is editing the applicable directives within a MySQL configuration file. There are multiple, optional configuration files that MySQL looks for when starting up. They are read in the following order: Continue reading “MySQL Performance: System Configuration File & Routine Maintenance”

Apache Performance Tuning: MPM Directives

Reading Time: 12 minutes

How directives behave and which directives are mainly available hinges on the loaded MPM. As discussed in our previous series, MPM is short for MultiProcess Modules, and they determine the basis for how Apache addresses multiprocessing. Using our last article on Apache MPM Modules as a springboard, we will use this section to cover the following subsections:

Each part will focus on how the directives affect performance for their respective MPM and some common considerations that should be assessed when optimizing Apache with those specific MPMs.

Note:
Be sure to review this article in its entirety as the universal directives operate in the same manner regardless of the MPM chosen.

 

General Optimization

IfModule

An important directive to learn when working with Apache servers is the IfModule conditional statement. There are two parts to the IfModule statement. A beginning, which also accepts a module name or module source file name, as well as a closing statement. When the provided module is loaded into Apache, then all directives between the beginning IfModule statement and the closing IfModule statement are also read into the Apache running configuration. Please review the provided example below for further clarification:

 

<ifModule mpm_prefork_module>
MaxSpareServers 16
</ifModule>
Timeout 60
The above example defines the MaxSpareServers directive only when loaded by mpm_prefork_module. The Timeout directive is applied every time due to it being outside of the IfModule closing statement.

 

IfModule statements are used to maintain compatibility within Apache configuration between module changes. Maintaining compatibility is done by grouping directives into IfModule statements, so they are only used when the required module is loaded. Ensuring a syntactically correct configuration file even when swapping modules.

Rule of Thumb:
Appropriately wrapping everything in an IfModule statement is a best practice standard with Apache and should be adhered to for superior compatibility in config files.

Timeout

The numerical value of seconds Apache waits for all common I/O events. Apache will abandon requests fail to complete before the provided Timeout value.

Determining the right Timeout depends on both traffic habits and hosted applications. Ideally, Timeout should be as low as possible while still allowing the vast majority of regular traffic to operate without issue. Large timeouts, those above 1 minute, open the server to SlowLoris style DOS attacks and foster a long wait in the browser when it encounters a problem. Lower timeouts allow Apache to recover from errant stuck connections quickly. It becomes necessary to strike a balance between the two extremes.

Tip:
Avoid increasing the global Timeout when addressing issues with a single script, or user, that requires a long Timeout. Problems can usually be resolved by a .htaccess file or include file to increase the Timeout directive for that specific script.

KeepAlive

KeepAliveA simple on|off toggle enables the KeepAlive protocols with supported browsers. The KeepAlive feature can provide as much as a 50% reductions in latency, significantly boosting the performance of Apache. KeepAlive accomplishes this by reusing the same initial connections a browser creates when connecting to Apache for all follow-up requests which occur within a short period.

KeepAlive is a powerful feature and in general, should be enabled in most situations. It works great for reducing some of the CPU and Network overhead with modern element heavy websites. For example, an easy way to visualize KeepAlive is with the “hold the door” phrase. Imagine a queue of people entering a building through a single doorway. Each person is required to open the door, walk through it, then close the door before the next person does the same process. Mostly, that’s how Apache works without KeepAlive. When enabled, the door stays open until all the people in line are through the door before it closes again.

Two additional related directives also govern KeepAlive. MaxKeepAliveRequests and KeepAliveTimeout. Discussed in the next section, each one plays a vital role in fine-tuning of the KeepAlive directive.

 

MaxKeepAliveRequests

Sets a limit on the number of requests an individual KeepAlive connection is permitted to handle. Once reached, Apache forces the connection to terminate, and creates a new one for any additional requests.

Determining an ideal setting here is open to interpretation. Generally, you want this value to be at least as high as the largest count of elements (HTML, Text, CSS, Images, Etc..) served by the most heavily trafficked pages on the server.

Rule-of-Thumb:
MaxKeepAliveRequestsSet MaxKeepAliveRequests to double that of the largest count of elements on common pages. (Services like webpagetest.org or gtmetrix.com can count elements on a page).

KeepAliveTimeout

This directive is measured in seconds and will remain idle waiting for additional requests from its initiator. Since these types of connections are only accessible to their initiator, we want to keep KeepAliveTimeout very low. A low value prevents too many KeepAlive connections from locking out new visitors due to connection priority.

Tip:
KeepAliveTimoutA large MaxKeepAliveRequests directive with a very low KeepAliveTimeout allows active visitors to reuse connections while also quickly recovering threads from idle visitors.
Configuration: Set MaxKeepAliveRequests to 500+, Set KeepAliveTimeout to 2
Requirements: Works best on MPM Event.

MPM Event/Worker Optimization

This section details the use and performance considerations that are essential when running Worker based MPMs, including both MPM Event and MPM Worker. These MPMs are considered multi-threaded solutions and some directives behave differently based on the loaded MPM. The information provided in this section is only a portion about Worker based MPMs.

Note:
In Worker based MPMs: ServerLimit, ThreadsPerChild, and MaxRequestWorkers are intrinsically linked with each other. It is essential to understand the role of each one and how changing one affects the others. The following directives govern the fine-tuning of the thread handling capabilities of Apache web servers.
MPM Worker and MPM Event

The two modules, MPM Event, and MPM Worker for most intents and purposes operate identically. The difference is apparent in the way each handles KeepAlive requests. The MPM Worker locks threads for the duration of the KeepAlive process and directly affects the number of available threads able to handle new requests. The MPM Event uses a Listener thread for each child. These Listener threads handle standard requests, and KeepAlive requests alike meaning thread locking will not reduce the capacity of the server. Without thread locking, MPM Event is the superior choice but only in Apache 2.4. Before Apache 2.4 the MPM Event was unstable and prone to problems.

ServerLimit

ServerLimit represents the upper limit of children Apache is allowed. The practical usage for ServerLimit is creating a hard ceiling in Apache to protect against input errors with MaxRequestWorkers. The cap prevents spawning vastly more children than a system can handle, resulting in downtime, revenue loss, reputation loss or even data loss.ServerLimit

ServerLimit ties in directly with the thrashing point discussed earlier in this article. The thrashing point is the maximum number of children Apache can run before memory usage tips the scale into perpetual swap. Match the ServerLimit to the calculated thrashing point to safeguard the server.

 

ThreadsPerChild

Used to define the limit of threads that each Apache child can manage. Every thread running can handle a single request. The default of 25 works well for most cases and is a fair balance between children and threads.ThreadsPerChild

There is an upper limit on this directive as well, and the limit is controlled by the ThreadLimit directive, which defaults to 64 threads. The adjustments to increase ThreadsPerChild past 64 threads also need to be made to ThreadLimit.

Increasing this value allows each child to handle more requests keeping memory consumption down while allowing a larger MaxRequestWorkers directive. A key benefit of running more threads within each child is shared memory cache access. Threads from one child cannot access caches from another child. Boosting the number of threads per child squeezes out more performance due to this sharing of cache data. The major downside for increased threads per child occurs during child recycling. The capacity of the server is diminished by the number of threads configured for each child when that child process is eventually recycled (graceful restart).

MPM Event/Worker

Inversely the opposite reaction is achieved by lowering ThreadsPerChild. Fewer threads per child require more children to run an equal amount of MaxRequestWorkers. Since children are full copies of Apache, this increases Apache’s overall memory footprint but reduces the impact when recycling children. Fewer threads mean fewer potential “stuck” threads during the recycle procedure, keeping the higher capacity of requests available overall children. Having fewer threads per child provides increased shared memory isolation. For instance, dropping ThreadsPerChild to 1 gives the same request isolation of MPM Prefork but also inherits its massive performance tax as well, requiring one child per one request.

Tip:
When setting ThreadsPerChild always consider the server environment and hardware.

  • A memory-heavy shared server hosting numerous independent accounts might opt for a lower ThreadsPerChild, reducing the potential impact of one user affecting another.
  • A dedicated Apache server in a high capacity load balanced configuration can choose to increase ThreadsPerChild significantly for a better overall performance of each thread.

ThreadLimit

Used to set the maximum value of ThreadsPerChild. This directive is a hard ceiling for ThreadsPerChild. It helps protect against typographical errors with the ThreadLimit
ThreadsPerChild directive which could quickly spin a server out of control if too many threads are allowed due to an input error. This setting need to be adjusted in some high-end servers when the system needs more than the default of 64 threads per child.

MaxRequestWorkers / MaxClients

The directive sets the limit for active worker threads across all running children and acts as a soft ceiling with ServerLimit taking control as the hard limit. When the number of total running threads has reached or exceeded MaxRequestWorkers, Apache no longer spawns new children.MaxRequestWorkers/MaxClients

Determining the MaxRequestWorkers is a critical part of server optimization. An optimal setting is based on several changing variables. This means its configuration needs to be reevaluated and tailored periodically over time, changed by watching traffic habits and system resource usage. The Apache status Scoreboard is an effective tool for analysis of Apache performance.

 

It is typical of Worker based MPM systems to run an isolated third-party PHP handler like Mod_fcgid, PHP-FPM, and mod_lsapi. These modules are responsibleMPM Event/Worker2

for processing PHP code outside of Apache and frees up Apache to handle all other non-PHP requests such as HTML, TEXT, CSS, Images, etc… These requests are far less taxing on server resources which allows Apache to handle larger volumes of requests, such as those beyond 400 MaxRequestWorkers.

MinSpareThreads

The least number of Threads that should remain open, waiting for new requests. MinSpareThreads is a multiple of ThreadsPerChild and cannot exceed MaxSpareThreads, though it can match it.

Rule-of-Thumb:
Set MinSpareThreads to equal 50% of MaxRequestWorkers.

Spare threads are idle workers threads. These threads are merely waiting for new incoming requests and are governed by the Apache child process that spawned them. If there are less available threads than MinSpareThreads, The Apache parent will generate a new child with another ThreadsPerChild worth of threads.

MinSpareThreads

MaxSpareThreads

This directive governs the total number of idle threads allowed on the server across all children. Any threads above this limit direct their parent to shut down to reduce memory consumption during off-peak hours.MaxSpareThreads

Having a limit to the number of idle open threads is excellent for smaller servers with hardware constraints. However, it mostly unneeded on today’s modernizing hardware.

Tip:
Configuring Apache as an open throttle is a high-performance configuration for servers with significant RAM and multiple CPU cores. When running the open throttle configuration, all available threads become available at all time. Apache’s memory usage will stay near its peak at all times, a side effect due to running all the configured children into memory preemptively. This configuration will produce the best possible response times from Apache by maintaining persistent open connections ready to do work and removing the overhead of spawning new processes in response to traffic surges.

Configuration: Match both MinSpareThreads and MaxSpareThreads to MaxRequestWorkers.

Requirements: Make sure there is enough server RAM to run all MaxRequestWorkers at once.

StartServers

This directive governs the initial amount of children the Apache Parent process spawns when the Apache service is started or restarted. This is commonly left unchanged since Apache continuously checks the current running children in conjunction with ThreadsPerChild and compare it to MinSpareThreads to determine if more children get forked. This process is repeated perpetually, with a doubling of new children on each iteration, until MinSpareThreads is satisfied.

Rule-of-Thumb:
StartServerManually calculating StartServers is done by dividing MaxRequestWorkers by ThreadsPerChild, rounding down to the nearest whole number. This process forces all children to be created without delay at startup and begins handling requests immediately. This aspect is especially useful in modern Apache servers which require periodic restarts to load in directive changes.

MaxConnectionsPerChild / MaxRequestsPerChild

The number of requests a single Apache child process can handle equals a cumulative total on the child server across all threads it controls. Each request handled by a thread counts toward this limit to its parent. Once the child server has reached its limit, the child is then recycled.
This directive is a stop-gap for accidental memory leaks. Some code executed through Apache threads may contain memory leaks. Leaked memory are portions of memory that subprocess failed to release properly, so they are inaccessible to any outside processes. The longer a leaking program is left running, the more memory it will leak. Setting a MaxConnectionsPerChild limit is a specific method for assuring Apache is periodically recycling programs to reduce the impact of leaked memory on the system. When using external code handlers like Mod_fcgid, PHP-FPM or mod_lsapi, it becomes necessary to set MaxConnectionsPerChild to 0 (unlimited), doing so prevents periodic error pages caused by Apache terminating threads prematurely.

Rule-of-Thumb:
MaxConnectionsPerChild/MaxRequestsPerChild If the server encounters a memory leak never set the MaxConnectionsPerChild / MaxRequestsPerChild too low, instead start with 10,000 and reduce it incrementally.

 

MPM Prefork Optimization

This MPM Prefork section details the use and performance considerations for various directives when running this module. This MPM is a non-threaded multi-processor designed for compatibility. It consists of a single Apache parent process, which is used to govern all new Apache processes also known as children. The following directives show how Apache is capable of performance tuning when using MPM Prefork. Unlike Worker based MPMs, optimizing MPM Prefork is generally simple and straightforward. There is a 1:1 ratio of Apache processes to incoming requests. However, MPM Prefork does not scale well with hardware and the more traffic it encounters, the more hardware it will need to keep up with the pace. It should be noted that some directives behave differently based on which MPM is loaded. The information provided in this section is only the portion about MPM Prefork.

MaxRequestWorkers / MaxClients

Used to control the upper limit of children that the Apache parent server is allowed to have in memory at one time. These children (also called workers) handle requests on a 1:1 ratio. This translates into the maximum number of simultaneous requests the server can handle.

MaxRequestWorkers / MaxClientsIf this directive is too low, Apache under-utilizes the available hardware which translates to wasted money and long delays in page load times during peak hours. Alternatively, if this directive is too high, Apache outpaces the underlying hardware sending the system into thrashing (link to thrashing article) scenario which can lead to server crashes and potential data loss.

 

MinSpareServers

This directive defines a minimum number of spare children the Apache parent process can maintain in its memory. An additional server is a preforked idle Apache child that is ready to respond to a new incoming request. Having idle children waiting for new requests is essential for providing the fastest server response times. When the total idle children on the server drop below this value, a new child is preforked at the rate of one per second until this directive is satisfied. The “one per second” rule is in place to prevent surges of the creation process that overload the server, however, this failsafe comes at a cost. The one per second spawn rate is particularly slow when it comes to handling page requests. So it’s highly beneficial to make sure enough children are preforked and ready to handle incoming requests.

 

Rule of Thumb:
MinSpareServers Default settingNever set this to zero. Setting this to 25% of MaxRequestWorkers ensures plenty resources are ready and waiting for requests.

MaxSpareServers

MasSpareServers controls the maximum number of idle Apache child servers running at one time. An idle child is one which is not currently handling a request but waiting for a new request. When there are more than MaxSpareServers idle children, Apache kills off the excess.

If the MaxSpareServers value is less than MinSpareServers, Apache will automatically adjust MaxSpareServers to equal MinSpareServers plus one.

Like with MinSpareServers, this value should always be altered with available server resources in mind.

Rule of Thumb:
MaxSpareServersSet this to double the value of MinSpareServers.
Tip:
Configuring Apache as an open throttle is a high-performance configuration for servers with significant RAM and multiple CPU cores. When running the open throttle configuration, all available Apache children become available at all times. As a side effect of running open throttle, the Apache memory usage will stay near its peak at all times, due to running all the configured children into memory preemptively. This configuration will produce the best possible response times by maintaining persistent open connections. Furthermore, in response to traffic surges, it removes the overhead that comes from spawning new processes.
Configuration: Match both MinSpareServers and MaxSpareServers to MaxRequestWorkers.
Requirements: Make sure there is enough server RAM to run all MaxRequestWorkers at once.

StartServers

Created at startup, are the initial amount of Apache child servers.

This seldom changed directive only impacts Apache startup and restart processes. Generally not altered because Apache uses internal logic to work out how many child servers should be running.

Many modern servers periodically restart Apache to address configuration changes, rotate log files or other internal processes. When this occurs during a high load traffic surge, every bit of downtime matters. You can manually set the StartServers directive to mirror that of your MinSpareServers to shave off time from the Apache startup.

Rule of Thumb:
StartServersThe StartServers directive should mirror that of MinSpareServers.

 

ServerLimit

The ServerLimit directive represents the upper limit of MaxRequestWorkers. This setting is generally used as a safeguard or ceiling against input errors when modifying MaxRequestWorkers.ServerLimit Default Setting

It becomes necessary to adjusted ServerLimit when the server is expected to handle more than the default of 256 requests simultaneously.

ServerLimit ties in directly with the thrashing point. The thrashing point is the maximum number of children Apache can run before memory usage tips the scale into perpetual swap. Match the ServerLimit to the calculated thrashing point to safeguard the server.

Note:
Increasing ServerLimit is not recommended with MPM Prefork. Running more than 256 simultaneous requests is hardware intensive when using the MPM Prefork module.

ThreadsPerChild Default Settings

MaxConnectionsPerChild / MaxRequestsPerChild

This directive equals the number of requests a single Apache child server can handle.

This directive is a stop-gap for accidental memory leaks. Code executed through Apache may contain faults which leak memory. These leaks add up over time making less and less of the shared memory pool of the child usable. The way to recover from leaked memory is to recycle the affected Apache child process. Setting a MaxConnectionsPerChild limit will protect from this type of memory leakage.

Note:
MaxConnectionsPerChild/MaxRequestsPerChild Rule-of-Thumb: Never set this too low. If the server encounters memory leak issues start with 10,000 and reduce incrementally.

 

Apache Performance Tuning: Swap Memory

Reading Time: 3 minutes

Before we get into the nitty-gritty of Apache tuning, we need to understand what happens when servers go unresponsive due to a poorly optimized configuration. An over-tuned server is one that is configured to allow Continue reading “Apache Performance Tuning: Swap Memory”

MySQL Performance: MyISAM vs InnoDB

Reading Time: 5 minutes

 

A major factor in database performance is the storage engine used by the database, and more specifically, its tables. Different storage engines provide better performance in one situation over another. For general use, there are two contenders to be considered. These are MyISAM, which is the default MySQL storage engine, or InnoDB, which is an alternative engine built-in to MySQL intended for high-performance databases. Before we can understand the difference between the two storage engines, we need to understand the term “locking.”

To protect the integrity of the data stored within databases, MySQL employs locking. Locking, simply put, means protecting data from being accessed. When a lock is applied, the data cannot be modified except by the query that initiated the lock. Locking is a necessary component to ensure the accuracy of the stored information.  Each storage engine has a different method of locking used. Depending on your data and query practices, one engine can outperform another. In this series, we will look at the two most common types of locking employed by our two storage engines.

 

Table locking:  The technique of locking an entire table when one or more cells within the table need to be updated or deleted. Table locking is the default method employed by the default storage engine, MyISAM.

Example: MyISAM Table LockingColumn AColumn BColumn C
Query 1 UPDATERow 1Writingdatadata
Query 2 SELECT (Wait)Row 2datadatadata
Query 3 UPDATE (Wait)Row 3datadatadata
Query 4 SELECT (Wait)Row 4datadatadata
Query 5 SELECT (Wait)Row 5datadatadata
The example illustrates how a single write operation locks the entire table causing other queries to wait for the UPDATE query finish.

 

Row-level locking: The act of locking an effective range of rows in a table while one or more cells within the range are modified or deleted. Row-level locking is the method used by the InnoDB storage engine and is intended for high-performance databases.

Example: InnoDB Row-Level LockingColumn AColumn AColumn A
Query 1 UPDATERow 1Writingdatadata
Query 2 SELECTRow 2Readingdatadata
Query 3 UPDATERow 3dataWritingdata
Query 4 SELECTRow 4ReadingReadingReading
Query 5 SELECTRow 5ReadingdataReading
The example shows how using row-level locking allows for multiple queries to run on individual rows by locking only the rows being updated instead of the entire table.

 

By comparing the two storage engines, we get to the crux of the argument between using InnoDB over MyISAM. An application or website that has a frequently used table works exceptionally well using the InnoDB storage engine by resolving table-locking bottlenecks. However, the question of using one over the other is a subjective as neither of them is perfect in all situations. There are strengths and limitations to both storage engines. Intimate knowledge of the database structure and query practices is critical for selecting the best storage engine for your tables.

MyISAM will out-perform InnoDB on large tables that require vastly more read activity versus write activity. MyISAM’s readabilities outshine InnoDB because locking the entire table is quicker than figuring out which rows are locked in the table. The more information in the table, the more time it takes InnoDB to figure out which ones are not accessible. If your application relies on huge tables that do not change data frequently, then MyISAM will out-perform InnoDB.  Conversely, InnoDB outperforms MyISAM when data within the table changes frequently. Table changes write data more than reading data per second. In these situations, InnoDB can keep up with large amounts of requests easier than locking the entire table for each one.

 

Should I use InnoDB with WordPress, Magento or Joomla Sites?

The short answer here is yes, in most cases. Liquid Web’s Most Helpful Humans in Hosting Support Teams have encountered several table-locking bottlenecks when clients are using some of the standard web applications of today. Most users of popular third-party applications like WordPress, Magento, and Joomla have limited knowledge of the underlying database components or code involved to make an informed decision on storage engines. Most table-locking bottlenecks from these content management systems (CMS) are generally resolved by changing all the tables for the site over to  InnoDB instead of the default MyISAM.  If you are hosting many of these types of CMS on your server, it would be beneficial to change the default storage engine in MySQL to use InnoDB for all new tables so that any new table installations start off with InnoDB.

 

Set your default storage engine to InnoDB by adding default_storage_engine=InnoDB to the [mysqld] section of the system config file located at:  /etc/my.cnf . Restarting the MySQL service is necessary for the server to detect changes to the file.

~ $ cat /etc/my.cnf
[mysqld]
log-error=/var/lib/mysql/mysql.err
innodb_file_per_table=1
default-storage-engine=innodb
innodb_buffer_pool_size=128M

 

Unfortunately, MySQL does not inherently have an option to convert tables, leaving each table to be changed individually. Liquid Web’s support team has put together an easy to follow maintenance plan for this process. The script, which you can run on the necessary server via shell access (SSH) will convert all tables between storage engines.

Note
Plan accordingly when performing batch operations of this nature just in case downtime occurs. Best practice is to backup all your MySQL Databases before implementing a change of this magnitude, doing so provides an easy recovery point to prevent any data loss.

Step 1: Prep

Plan to start at a time of day where downtime would have minimal consequences. This process itself does not require any downtime, however, downtime may be necessary to recover from unforeseen circumstances.  

 

Step 2: Backup All Databases To A File

The command below creates a single file backup of all databases named all-databases-backup.sqld and can be deleted once the conversion has succeeded and there are no apparent problems.
mysqldump --all-databases > all-databases-backup.sql

 

Step 3: Record Existing Table Engines To A File

Run the following script to record the existing table engines to a file named table-engine-backup.sql. You can then “import” or “run” this file later to convert back to their original engines if necessary.

mysql -Bse 'SELECT CONCAT("ALTER TABLE ",table_schema,".",table_name," ENGINE=",Engine,";") FROM information_schema.tables WHERE table_schema NOT IN("mysql","information_schema","performance_schema");' | tee table-engine-backup.sql

If you need to revert the table engines back for any reason, run:
mysql < table-engine-backup.sql

 

Step 4a: Convert MyISAM Tables To InnoDB

The below command will proceed even if a table fails and lets you know which tables failed to convert. The output is saved to the file named convert-to-innodb.log for later review.
mysql -Bse 'SELECT CONCAT("ALTER TABLE ",table_schema,".",table_name," ENGINE=InnoDB;") FROM information_schema.tables WHERE table_schema NOT IN ("mysql","information_schema","performance_schema") AND Engine = "MyISAM";' | while read -r i; do echo $i; mysql -e "$i"; done | tee convert-to-innodb.log

 

Step 4b: Convert All InnoDB Tables To MyISAM

This command will proceed even if a table fails and lets you know which tables failed to convert. The output is also saved to the file named convert-to-myisam.log for later review.

mysql -Bse 'SELECT CONCAT("ALTER TABLE ",table_schema,".",table_name," ENGINE=MyISAM;") FROM information_schema.tables WHERE table_schema NOT IN ("mysql","information_schema","performance_schema") AND Engine = "InnoDB";' | while read -r i; do echo $i; mysql -e "$i"; done | tee convert-to-myisam.log

 

The following commands illustrate how converting a single table is accomplished.

Note
Replace database_name with the proper database name and table_name with the correct table name. Make sure you have a valid backup of the table in question before proceeding. 

Backup A Single Table To A File
mysqldump database_name table_name > backup-table_name.sql

 

Convert A Single Table To InnoDB

mysql -Bse ‘ALTER TABLE database_name.table_name ENGINE=InnoDB;’

 

Convert A Single Table To MyISAM:

mysql -Bse ‘ALTER TABLE database_name.table_name ENGINE=MyISAM;’

 

Check out our other articles in this series, MySQL Performance: Identifying Long Queries, to pinpoint slow queries within your database.  Stay tuned for our next article where we will cover caching and optimization.

MySQL Performance: Identifying Long Queries

Reading Time: 5 minutes

Every MySQL backed application can benefit from a finely tuned database server. The Liquid Web Heroic Support team has encountered numerous situations over the years where some minor adjustments have made a world of difference in website and application performance. In this series of articles, we have outlined some of the more common recommendations that have had the largest impact on performance. Continue reading “MySQL Performance: Identifying Long Queries”

Optimizing Your Website in Cloud Sites

Reading Time: 2 minutes

optimizing-your-site-cloud-sties-header

To ensure that your site performs at its best, there are a few things you can do to optimize it when using Liquid Web Cloud Sites technology. This article will take you through the top five best practices to optimize your website.

Continue reading “Optimizing Your Website in Cloud Sites”