404 Errors: Report, monetize and analyse

Posted by Olaf
June 14, 2010

After your websites getting more pages and links, the chance that a visitor will follow a dead link to your site exists. If a visitor is trying to access a page on your site, the server will report (normally) a 404 error. The response is by default some unfriendly page with some spare information about the error which let most visitors stop visiting your site. But using the 404 error the right way, you the site owner can collect important information like:

Using the right tools you’re able to turn 404 errors into a powerful resource:

In this tutorial we will show you how-to:

Error reporting page

With the Apache webserver it’s possible to use custom directives for your error script, place this code into your .htaccess file (place the file into the site root):

ErrorDocument 400 /error.php?err=400
ErrorDocument 401 /error.php?err=401
ErrorDocument 403 /error.php?err=403
ErrorDocument 404 /error.php?err=404
ErrorDocument 500 /error.php?err=500

We use for the custom error script the most common HTTP errors.
Next we need to create a PHP script called error.php which can handle the different errors:

<?php
$errorNum = (int)$_GET['err'];
$err_str = array(404=>'Not Found', 400=>'Bad Request', 401=>'Unauthorized', 403=>'Forbidden', 500=>'Internal Server Error');
echo '
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>'.$err_str[$errorNum].'</title>
</head>
<body>
<h1>An error occured: '.$err_str[$errorNum].'</h1>
 
<!-- place here your advertisement -->
 
<!-- place here your Google analytics code -->
</body>
</html>';
?>

This script will show the different errors and also some advertisement if you add the ad code. Don’t forget to add the GA code snippet.

Track dead links in Google Analytics

In case of a 404 error the page title on this custom error page will be “Not Found”. We use the page title as a filter in Google Analytics to track the page views. Create a new profile for the site you’re working on and add this filter:

Adding Google Site Search to your 404 error page

If you haven’t done yet, create a Google site search for your website. Add only your own website to the list of searched sites and don’t search the entire web. Add your Google Adsense ID (section “Make Money”) and head to the section “Look and feel” and select the option Iframe. Choose a style for the search form / result and maybe you like to customize the style. Push now the button “Get code” (or enter the section “Get code” from the sidebar) and enter there the URL from your error page.
Copy / paste the code for the search form and the results into the body section from your error page. Your completed page will look like:

<?php
$errorNum = (int)$_GET['err'];
$err_str = array(404=>'Not Found', 400=>'Bad Request', 401=>'Unauthorized', 403=>'Forbidden', 500=>'Internal Server Error');
echo '
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>'.$err_str[$errorNum].'</title>
</head>
<body>
<h1>An error occured: '.$err_str[$errorNum].'</h1>
 
<form action="http://www.yourdomain.com/error.php" id="cse-search-box">
  <div>
    <input type="hidden" name="cx" value="thisCodeIsProvidedByGoogleCSE" />
    <input type="hidden" name="cof" value="FORID:10" />
    <input type="hidden" name="ie" value="UTF-8" />
    <input type="text" name="q" size="31" />
    <input type="submit" name="sa" value="Zoeken" />
  </div>
</form>
<script type="text/javascript" src="http://www.google.com/cse/brand?form=cse-search-box&lang=en"></script>
 
<div id="cse-search-results"></div>
<script type="text/javascript">
  var googleSearchIframeName = "cse-search-results";
  var googleSearchFormName = "cse-search-box";
  var googleSearchFrameWidth = 600;
  var googleSearchDomain = "www.google.com";
  var googleSearchPath = "/cse";
</script>
<script type="text/javascript" src="http://www.google.com/afsonline/show_afs_search.js"></script>';
 
if (empty($_GET['q'])) { // show the ad only if there is no search
	echo '
<!-- place here your advertisement -->';
} 
echo '
<!-- place here your Google analytics code -->
</body>
</html>';
?>

We placed the add code also into some IF clause, because there should not be another Google Adsense advertisement beside the Google ads from the result page.

Enable site search tracking in Google Analytics

The code for the error page is complete and we move to the last step: Tracking the site search queries from the error page. To do this we need to go in Google Analytics to the profile we created for the error page and click Edit (twice), check the setting Do Track Site Search, enter a “q” as the “Query Parameter” and click Safe Changes.

This error page is very basic and you need add your sites web template to make it complete. If you like this tutorial and you have used the code on your own site please share the URL to your new or updated error page. Even if you don’t like to use the code from this page, we advice to track the errors and also the site search queries from your visitors. If you have questions or comments please post them below.

Delicious
, , , , ,
Google services, PHP scripts, Tutorials

Related posts

Did you enjoy this post? Why not leave a comment and continue the conversation.

Comments

Trackback URL for this post: http://www.web-development-blog.com/archives/404-errors-report-monetize-and-analyse/trackback/

very helpful. Thanks.
Do you recommend a good tool to track all the dead links of the website ?

Comment by skillguru on June 14, 2010 @ 5:11 pm

The strategy to detect links is not about finding dead links on your own site (I hope you don’t have some).
This is about dead links from external sites. Check also the information about your site provided by the google webmaster tools

Comment by Olaf on June 14, 2010 @ 6:12 pm

Years ago in the early days of the web there have been buisness models in making money of 404 traffic,
or free hosting services taking only your 404 traffic. Maybe something of that still exists.

Comment by John PHP on June 14, 2010 @ 6:41 pm

Hi John,
right th idea from this article is based on this concept. Accept one thing, the strategy is to serve your visitors better content ;)

Comment by Olaf on June 14, 2010 @ 6:45 pm

You forgot something important:

Setting the header() via PHP to the appropriate error code! Without that, the 404 error pages would return an 200/ok header and counted as duplicated content…

With the correct headers, track and analysis is easier through tools like Google Webmaster tools.

Comment by HitMan on June 15, 2010 @ 12:22 pm

Hi,
You don’t need to set a header because Apache will send the header to the browser.
Of course you have to instruct the search engines to NOT index the error reporting script.

I using a similar script for finalwebsites.com and never found a file that doesn’t exists in the index.

Comment by Olaf on June 15, 2010 @ 1:21 pm

Hello, Olaf.

Thanks for good article for novices.

I translated this article on Russian language and posted it in my blog (http://drupal.qpart.ru/blog/404-oshibka-otchety-monetizatsiya-i-analitika). Of course, with link to this page. You do not mind?

Thanks, Dmitry

Comment by Crazy Developer on June 17, 2010 @ 10:15 pm

Great that you like (and translated) the tutorial.

Comment by Olaf on June 20, 2010 @ 9:18 pm

I found this modified Google Analytics code snippet, which is able to track information about the HTTP_REFERER:

<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? " https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + " google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
try{
var pageTracker = _gat._getTracker("UA-xxxxx-x");
pageTracker._trackPageview("/404.html?page=" + document.location.pathname + document.location.search + "&from=" + document.referrer);
} catch(err) {}
</script>

More information about the snippet:
http://www.google.com/support/googleanalytics/bin/answer.py?hl=en&answer=86927

Comment by Olaf on June 25, 2010 @ 8:51 am
Leave a comment

(required)

(required)