Skip to content


Surviving WordPress Part 2 – Performance Tuning

By Strictly Software
www.strictly-software.com

Surviving WordPress – Performance and Site Optimization

This is the second part of my guide to surviving wordpress and as promised it looks at performance tweaks and tips which I have gathered on my way. It has been quite a while since the first installment and the main reason for this was that I was suffering my own performance killer which I wanted to solve first before writing this article. Luckily this has now been solved with the help of Robert from Tiger Tech blog who helped me get to the bottom of the issue so here it is.

My own personal journey into WordPress performance tuning started off when I started to experience out of PHP memory errors when manually rebuilding my Google sitemap. I started to play around with different plugins and then delve into the code which is when I started to realise the danger that WordPress plugins can carry out to a site when the user doesn’t realise what’s going on behind the scenes. You can check out a detailed examination here but in my case it was using a Google Sitemap plugin and setting it to rebuild when a new post is saved and combining that with WP-O-Matic which imports articles at scheduled intervals and a TwitterBot such as my own which can send Tweets to multiple accounts whenever new content is added.

If you have a similar setup it’s worth running TOP, MyTOP and checking your access logs to see how it affects your own system but what was happening on my own setup was:

  • WP-O-Matic starts to import a feeds worth of articles (max of 10) for each article that is saved.
  • Numerous procedures hooked into the SavePost or PublishPost action run. In my case:
    1. My Strictly AutoTags plugin runs which analyses the article and adds relevant tags
    2. The Google Sitemap plugin runs which runs lots of SQL queries and creates a new file as well as pinging multiple SERPs with HTTP requests.
    3. My Strictly Tweetbot Plugin runs which posts a tweet to multiple accounts.
    4. Any other plugin using the Save hooks runs such as caching tools which create static files.
  • As soon as the Tweets arrive on Twitter a multitude of Bots (19 on my last test) will visit the site to index the link that has just been posted.
  • If the link was posted to multiple accounts you will find that the same bots will visit for each account you posted to. Some bots like Yahoo seem to be particularly bad and visit the article multiple times anyway.
  • All these visits create new Apache processes and depending on the amount of memory that each process uses you could find that your server starts swapping to disk to handle the increase and in my case my server load would quickly jump from 0.15 to 50+.

The more articles you import the more iterations of this chain of performance killing events occurs. I found that these events would sometimes pass off without any noticeable problems but other times the server load would get so high (highest value recorded was 174 on a 1GB RAM Linux server) that I would have to reboot my machine. In fact on some days I would have to reboot 3-5 times which is not good.

Getting to the bottom of the problem

A common solution to any performance related problem is to throw more resources at it. Many message boards recommended increasing the maximum memory limit to get round the Out of Memory errors the Google Sitemap was throwing up but that just masks the issue and doesn’t actually solve it.

As a by product of my system tuning I ended up creating my own Google Sitemap Plugin to overcome limitations of the others. Not only could it be easily set to rebuild at scheduled intervals instead of only when new posts were added which helps reduce unnecessary rebuilds but it used far less memory and made a tiny number of database queries in comparison to the other market leaders. I also created a System Reporting plugin so that I could be kept informed when my site was playing up and I found this invaluable in keeping my site running during this performance nightmare.

One of the first ideas to reduce the amount of times I was rebooting was to try and prevent any performance intensive tasks from running if the server load was already high. I did this by adding in some checks to all my major plugins that made a call to the following function before running anything. If the load was above 1.0 I just exited immediately.

function GetServerLoad(){
// this only works for non windows systems e.g LINUX
$os = strtoupper(PHP_OS);
if(substr($os, 0, 3) !== 'WIN'){
 if(file_exists("/proc/loadavg")) {
  $load = file_get_contents("/proc/loadavg");
  $load = explode(' ', $load);
  return $load[0];
 }elseif(function_exists("shell_exec")) {
  $load = @shell_exec("uptime");
  $load = explode(' ', $load);
  return $load[count($load)-3];
 }
}
return false;
}

Apache Configuration

I finally got to the bottom of the problem I was suffering with the help of Tiger Tech after examining the output of ps auxwwH –sort rss during a period of high load. This listed all the currently running processes ordered by the amount of memory they were consuming. At the time of running this my average load was 50 which meant there was a big queue of processes waiting to be run which included over 70 Apache processes each using between 8MB and 30MB and this alone was easily using up my 1GB of RAM.

This high number of Apache processes meant that my server was busily swapping from real memory to disk based virtual memory which was causing high I/O (clearly seen from the output of iostat) and slowing down the response times of each Apache process. As each process got slower to respond new processes were spawned using up even more virtual memory adding to the problem. This spiral of death was only resolved if for some reason the traffic suddenly screeched to a halt (not likely during an article import that delivers hundreds of bots from Twitter on top of normal traffic) OR I killed Apache or the server.

The solution to this problem was to reduce the number of simultaneous Apache processes that could be run at one time by reducing the MaxClients setting in the Apache config file. My existing setting of 256 was far too high for my 1GB RAM server. The way to calculate a more appropriate setting is to take the average size of an Apache process and then divide the total available memory by that number leaving room for other processes such as MySQL. In my case I was advised to set MaxClients to a value of 20 which seems small in comparison to the original value but makes more sense when you do the maths.

Reducing my MaxClients setting to a much smaller value meant that the memory allocation for my system would never reach such unmanageable amounts again. If my server is swamped by traffic then instead of 256 Apache processes being spawned all trying to claim 20MB or more for themselves they will be queued up in an orderly fashion. It might slow down some requests as they wait to be dealt with but that is far better than the whole server freezing up which was occurring regularly.

Two other settings I changed in the Apache conf file was the Timeout value down from 300 to 30 and HostnameLookups was turned off. You can read more about these settings at the Apache performance tuning site.

Common WordPress Performance Tuning Tips

There are a number of common tips for performance tuning WordPress which you can read about in detail at other sites but I will quickly cover them here:

1. Install APC or another PHP caching system such as XCache or eAccelerator as these Opcode systems improve performance by saving and re-using compiled PHP which speeds up the execution of server side code.

2. Install a WordPress caching plugin such as WP Super Cache or W3 Total Cache. There is a debate over which one is best and whilst W3 Total Cache does offer more features such as Minification and Browser cache options the main issue that you want to resolve with WordPress is reducing the huge amount of database queries and code that is run on each page load. The aim is to do expensive tasks once and then re-use the results as many times as possible. Caching the results of database queries so that they don’t have to be run every time the page loads is a great idea especially if the results hardly change and whilst W3 offers database query result caching as well as caching the output of the generated HTML Super Cache will only cache the generated output.

What is the difference? Well if you cached database query results then during the building of cached files the results of queries that are used to create category lists or tag clouds can be shared across builds rather than being recalculated for every page being cached that uses them. How much difference this makes when you take all MySQL’s own internal query caching into consideration is debatable. However both plugins offer the major way to improve fast page loads which is disk based caching of the generated output incorporating GZIP compression.

If you do install W3 Total Cache and you have APC or another PHP Accelerator installed make sure that you enable the Disk Based Cache option for Page Caching and not Opcode which will be default selected if APC or XCache is installed.

3. If bandwidth is a problem then serving up minified and compressed HTML, CSS and Javascript will help but you don’t want to be repeatedly compressing files as they load. Some cache plugins will do this minification on the fly which hurts CPU whereas you really want it done once. There is nothing stopping you combining, compressing and minifying your files by hand. Then you will benefit from small files, fewer HTTP requests and less bandwidth whether or not you make use of a cache plugin.

4. Reduce 404 errors and ensure WordPress doesn’t handle them as it will cane performance unnecessarily. Create a static 404 error page or ensure your cache system is setup to handle 404’s. Also make sure that common files that cause 404’s such as IPhone icons, Crossdomain.xml and favicons exist even if they are empty files.

5. If you’re not planning on using a caching system then you should ensure that you tune your htaccess file manually to ensure that browsers cache your files for specified periods of time rather than downloading them each time they visit your site. You can do this by setting the future expire headers on your static content such as JS, CSS, images and so on like so:

<FilesMatch "(?i)^.*\.(ico|flv|ogg|swf|jpg|jpeg|png|gif|js|css)$">
ExpiresActive On
ExpiresDefault "access plus 1 weeks"
Header unset Last-Modified
Header set Cache-Control "public, no-transform"
</FilesMatch>

6. Tune your MySQL database by ensuring that your database is set to cache query results and has enough space to do so wisely. Ensure options you don’t use or require are disabled and make sure you regularly maintain your tables and indexes by keeping fragmentation to a minimum. There are a couple of well known tuning scripts which can be used to aid in the setting of your MySQL configuration settings and which use your current database load and settings as a guide to offer recommendations.

http://github.com/rackerhacker/MySQLTuner-perl

http://hackmysql.com/mysqlreport

Uninstall Performance Hogging Plugins

There are lots of plugins available for WordPress and it can be like a case of a kid let lose in a candy shop as there seems to be at least 10 plugins for everything. However having too many plugins installed is definitely a bad thing in terms of performance and unless you know what the code is doing you could be shooting yourself in the foot by installing the next greatest plugin onto your site without thoroughly checking the source code out for yourself first.

The problem is that literally anyone can write and then publish a plugin on WordPress and many of these authors are not programmers by trade or have performance in the forefront of their minds as they develop the code that you might use.

Even plugins that are targeted as performance saving tools are not always beneficial and I have seen plugins that are designed to reduce bandwidth by returning 304 Not Modified headers or 403 Forbidden status codes but have to make numerous database queries, DNS lookups and carry out multiple regular expressions to do so. If Bandwidth is a problem then this might be worth the extra load but if it isn’t then you are just swapping a small gain in one area for extra work somewhere else.

If you are going to use a plugin then take a look over the source code to see if you can help improve the performance by adding any missing indexes to any new tables the plugin might have added to your WordPress database. Many plugins do add tables especially if they need to store lots of data and many authors don’t include the SQL statements to add appropriate indexes which could end up slowing down lookups down the road as the amount of data within the tables grows.

The following list are extra indexes I have added to tables within the WordPress database for both Plugins I installed and core WordPress tables that were missing indexes for certain queries. Remember WordPress is mainly a READ based system so the extra expense of adding indexes when data is inserted is usually worth it.

PluginTableIndexNameColumnsIndexType
wp_postsstatus_password_idpost_status, post_password, IDNormal
wp_postspost_datepost_Date, IDUnique
fuzzySEOBoosterwp_seoqueries_termsterm_value_stidterm_value, stidunique
fuzzySEOBoosterwp_seoqueries_datastid_pageid_pagetype_foundedstid,page_id, page_type,foundedunique
WP-O-Maticwp_wpo_campaign_postcampaignid_feedid_hash`campaign_id, feed_id, hashNormal
Yet Another
Relatd Post
wp_yarpp_related_cachereference_idreference_ID, IDNormal

Ensure that you reguarly check the MySQL slow query log especially if you have just installed a new plugin as this will help you find queries that need optimising and potential bottlenecks caused by poorly thought out SQL. On my own site I started off using a well known Related Posts plugin but I found out from the Slow log that the queries it ran to create the lists were killing performance due to their design. They were taking 9-12 seconds to run and were scanning up to 25 million records at a time as well as carrying out unnecessary UNION statements which doubled the records it needed to look at. I ended up replacing it with a different plugin called LinkWithin which not only looked great due to the images it used but was perfect for performance because it was a Javascript widget and all the work was carried out on their own server rather than mine.

This might not be the solution for you as obviously Javascript is disabled by 10% of all visitors and bots won’t be able to see the links. If SEO is a concern, and it should be then you need to make sure that SERP crawlers find all your content easily and having a server side created list of related articles is a good idea for this reason alone. Therefore you can always create your own Related Posts section very easily with a function placed at the bottom of your articles that uses the categories assigned to the post to find other posts with the same category.

The following example shows one way in which this can be done and it makes use of a nice ORDER BY RAND() trick to ensure different articles and categories appear each time the SQL is run. It also uses WordPresses inbuilt cache to store the results to prevent the query being executed too many times.

<?php
function get_my_related_posts($id, $limit){
// enable access to the WordPress DB object
global $wpdb;
// define SQL
$sql = "SELECT  CONCAT('http://www.mysite.com/',year(p.post_date),'/',RIGHT(concat('0' ,month(p.post_date)),2),'/',post_name,'/') as permalink,
    p.post_title as title
  FROM (
   SELECT p.ID, p.post_name, p.post_title, p.post_date, terms.slug as category
   FROM  wp_posts p,  wp_term_relationships tr,  wp_term_taxonomy tt,  wp_terms as terms
   WHERE p.ID               != $id                 AND
     p.post_type         = 'post'              AND
     p.post_status       = 'publish'           AND
     p.ID                = tr.object_id        AND
     tr.term_taxonomy_id = tt.term_taxonomy_id AND
     tt.taxonomy         in ( 'category')      AND
     tt.term_id          = terms.term_id
   GROUP BY  p.ID, p.post_title, p.post_name, p.post_date
   ORDER BY terms.term_id
  ) as p,
  (
   SELECT distinct terms.slug
   FROM wp_term_relationships tr, wp_term_taxonomy tt, wp_terms as terms
   WHERE tr.object_id        = $id     AND
     tr.term_taxonomy_id = tt.term_taxonomy_id AND
     tt.taxonomy in ( 'category')    AND
     tt.term_id          = terms.term_id
   ORDER BY RAND() LIMIT 1
  ) as t
  WHERE p.category = t.slug
  ORDER BY  RAND()
  LIMIT $limit";
 // see if we have a cached recordset
 $cache_name = "get_my_related_posts_" . $id;
 $result = wp_cache_get( $cache_name );
 if ( false == $result ) {
  // get results and then cache for later use
  $result = $wpdb->get_results( $sql );
  wp_cache_set( $cache_name, $result );
 }
  // return result set as object
  return $result;
}
?>
<div id="StrictlyRelatedPosts">
<h3>Related posts</h3>
<ul>
  <?php
 // fetch 5 related posts
 $related_posts = get_related_posts($post->ID, 5);
 // open loop
 foreach ($related_posts as $related_post) {
   $permalink = $related_post->permalink;
   $title     = $related_post->title;
   print "<li><a title=\"$title\" href=\"$permalink\">$title</a></li>\n";
  } ?>
</ul>
</div>

Identifying Bottlenecks in WordPress

One good plugin which I use for identifying potential problematic queries is the Debug Queries plugin which allows administrators to see all the queries that have run on each page. One extra tweak you should add is to put the following line in at the bottom of the get_fbDebugQueries function (around line 98)

$debugQueries .= '<li class="none_list">' . sprintf(__('» Memory Used %s'),$this->ConvertFromBytes($this->GetMemoryUsage(true))) . '</li>' . "\n";

and then add these two functions underneath that function (around line 106) which get the memory usage and format the value nicely.

// format size from bytes
function ConvertFromBytes($size){
$unit=array('B','KB','MB','GB','TB','PB');
return @round($size/pow(1024,($i=floor(log($size,1024)))),2).$unit[$i];
}
// get PHP memory usage
function GetMemoryUsage(){
if(function_exists("memory_get_peak_usage")) {
 return memory_get_peak_usage(true);
}elseif(function_exists("memory_get_usage")) {
 return  memory_get_usage(true);
}
}

This will help you see just how many database queries a standard WordPress page makes (88 on my homepage!) and if you haven’t done any performance tuning then you may suddenly feel the urge before you suffer similar problems to those I experienced.

Remember a high performing site is one which attracts visitors and one which SERP bots are now paying more attention to when indexing. Therefore you should always aim to get the best performance out your system as is feasibly possible and as I have shown that doesn’t mean spending a fortune on hardware.

WordPress Performance Summary

  • Ensure Apache is configured correctly and don’t leave the default values as they are. Make sure MaxClients is set correctly by dividing your RAM by the average Apache process size leaving room for MySQL and anything else you might be running.
  • Tune your MySQL database by configuring correctly and maintaining regularly. Use one of the many free tuning scripts to help set your configuration up correctly but ensure you read up about the various settings and what they do first.
  • Install a Caching plugin that creates hard copies of commonly requested files. Static HTML is fast to load. PHP is costly to compile. Use a PHP accelerator and ensure database query results are cached.
  • Reduce bandwidth by combining, compressing and minifying your CSS, JS and HTML. If your caching plugin doesn’t do it once rather than on the fly do it by hand. Remember the key is to do expensive operations once and then re-use the results as many times as possible.
  • Set your htaccess file up correctly. Ban bad bots to reduce traffic, set far future expiry headers on your static files and use static files to handle 404, 403, 503 errors etc.
  • Reduce the number of plugins and ensure any that you use are not hurting performance. Make sure any tables they use are covered by indexes and use the slow query log to identify problems.

Further Reading:

WordPress’s own tips for performance and high traffic sites

Tiger Tech discusses WP Super Cache versus W3 Total Cache

Super Cache versus W3 Total Cache – Benchmark Tests

Speed up Apache with Mod Deflate

Tips and Tricks for WordPress Optimization

MySQL’s Query Cache explained

View the original article at Strictly Software

Related Posts with Thumbnails

Posted in Computing, Dark Politricks Articles, Technology.

Tagged with , , .

Support #altnews & keep Dark Politricks alive

Remember I told you over 5 years ago that they would be trying to shut down sites and YouTube channels that are not promoting the "Official" view. Well it's happening big time. Peoples Channels get no money from YouTube any more and Google is being fishy with their AdSense giving money for some clicks but not others. The time is here, it's not "Obama's Internet Cut Off Switch" it's "Trumps Sell Everyones Internet Dirty Laundry Garage Sale".

It's not just Google/YouTube defunding altenative chanels (mine was shut), but Facebook is also removing content, shutting pages, profiles and groups and removing funds from #altnews that way as well. I was recently kicked off FB and had a page "unpublished" with no reason given. If you don't know already all Facebooks Private Messages and Secret Groups are still analysed and checked for words related to drugs, sex, war etc against their own TOS. Personally IU know there are undercover Irish police moving from group to group cloning peoples accounts and getting people booted. Worse than that I know people in court at the moment for the content they had on their secret private group. Use Telegrams secret chat mode to chat on, or if you prefer if you need to or buy a dumb phone with nothing for the NSA to hack into if you are that paranoid.

So if your not supporting this site already which brings you news from the Left to the Right (really the same war mongering bollox) then I could do with some. Even if it's just £5 or tick the monthly subscription box it will be much appreciated. Read on to find out why/

Why?

Any support to keep this site would be appreciated. You could set up a monthly subscription for £2 like some people do or you could pay a one off donation as a gift.
I am not asking you to pay me for other people's articles, this is a clearing house as well as place to put my own views out into the world. I am asking for help to write more articles like my recent
false flag gas attack to get WWIII started in Syria, and Trump away from Putin. Hopefully a few missiles won't mean a WikiLeaks release of that infamous video Trump apparently made in a Russian bedroom with Prostitutes. Also please note that this article was written just an hour after the papers came out, and I always come back and update them.

If you want to read JUST my own articles then use the top menu I have written hundreds of articles for this site and I host numerous amounts of material that has seen me the victim of hacks, DOS plus I have been kicked off multiple hosting companies, free blogging sites, and I have even had threats to cease and desist from the US armed forces. Therefore I have to pay for my own server which is NOT cheap. The more people who read these article on this site the more it costs me so some support would be much appreciated.

I have backups of removed reports shown, then taken down after pressure, that show collusion between nations and the media. I have the full redacted 28/29 pages from the 9.11 commission on the site which seems to have been forgotten about as we help Saudi Arabia bomb Yemeni kids hiding in the rubble with white phosphorus, an illegal weaapon. One that the Israeli's even used when they bombed the UN compound in Gaza during Operation Cast Lead. We complain about Syrian troops (US Controlled ISIS) using chemical weapons to kill "beautiful babies". I suppose all those babies we kill in Iraq, Yemen, Somalia and Syria are just not beautiful enough for Trumps beautiful baby ratio. Plus we kill about 100 times as many as ISIS or the Syrian army have managed by a factor of about 1000 to 1.

I also have a backup of the FOX News series that looked into Israeli connections to 9.11. Obviously FOX removed that as soon as AIPAC, ADL and the rest of the Hasbra brigade protested.

I also have a copy of the the original Liberal Democrats Freedom Bill which was quickly and quietly removed from their site once they enacted and replaced with some watered down rubbish instead once they got into power. No change to police tactics, protesting or our unfair extradition treaty with the USA but we did get a stop to being clamped on private land instead of the mny great ideas in the original.

So ANY support to keep this site running would be much appreciated! I don't have much money after leaving my job and it is a choice between shutting the server or selling the domain or paying a lot of money just so I can show this material. Material like the FSB Bombings that put Putin in power or the Google no 1 spot when you search for protecting yourself from UK Police with "how to give a no comment interview". If you see any adverts that interest you then please visit them as it helps me without you even needing to give me any money. A few clicks per visit is all it takes to help keep the servers running and #altnews alive!

However if you don't want to use the very obvious and cost free ways (to you) to help the site and keep me writing for it then please consider making a small donation. Especially if you have a few quid sitting in your PayPal account doing nothing useful. Why not do a monthly subscription for less money instead. Will you really notice £5 a month?


One Response

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

Continuing the Discussion

  1. Tweets that mention Surviving Wordpress Part 2 – Performance Tuning | Dark Politricks -- Topsy.com linked to this post on October 3, 2010

    […] This post was mentioned on Twitter by DarkPolitricks, Dark Politricks RT. Dark Politricks RT said: New article on darkpolitricks: Surviving WordPress Part 2 – Performance Tuning http://bit.ly/bTHE8a #Google #ID #Performance #PHP #RAM #SQL […]



Some HTML is OK

or, reply to this post via trackback.



css.php