Web Development Blog » google

404 Errors: Report, monetize and analyse

Olaf — Mon, 14 Jun 2010 04:57:00 +0000

After your websites getting more pages and links, the chance that a visitor will follow a dead link to your site exists. If a visitor is trying to access a page on your site, the server will report (normally) a 404 error. The response is by default some unfriendly page with some spare information about the error which let most visitors stop visiting your site. But using the 404 error the right way, you the site owner can collect important information like:

Of course the broken link or URL
The HTTP_REFERER information where the dead link is available
How often people try to access the bad URL

Using the right tools you’re able to turn 404 errors into a powerful resource:

Provide a site search feature and let people search what they are looking
Add advertisements to your error page and start earning money
Learn about what people like to see on your site

In this tutorial we will show you how-to:

Create a dynamic error page using the Google Site search and Adsense content ads
Setup Google analytics to track 404 errors using a filter
Set the site search feature with Google Analytics to collect the search queries from your visitor

Error reporting page

With the Apache webserver it’s possible to use custom directives for your error script, place this code into your .htaccess file (place the file into the site root):
ErrorDocument 400 /error.php?err=400 ErrorDocument 401 /error.php?err=401 ErrorDocument 403 /error.php?err=403 ErrorDocument 404 /error.php?err=404 ErrorDocument 500 /error.php?err=500
We use for the custom error script the most common HTTP errors.
Next we need to create a PHP script called error.php which can handle the different errors:


$errorNum = (int)$_GET['err'];
$err_str = array(404=>'Not Found', 400=>'Bad Request', 401=>'Unauthorized', 403=>'Forbidden', 500=>'Internal Server Error');
echo '



'</span><span style="color: #339933;">.</span><span style="color: #000088;">$err_str</span><span style="color: #009900;">[</span><span style="color: #000088;">$errorNum</span><span style="color: #009900;">]</span><span style="color: #339933;">.</span><span style="color: #0000ff;">'


An error occured: '.$err_str[$errorNum].'
 

 


';
?>

This script will show the different errors and also some advertisement if you add the ad code. Don’t forget to add the GA code snippet.

Track dead links in Google Analytics

In case of a 404 error the page title on this custom error page will be “Not Found”. We use the page title as a filter in Google Analytics to track the page views. Create a new profile for the site you’re working on and add this filter:

Adding Google Site Search to your 404 error page

If you haven’t done yet, create a Google site search for your website. Add only your own website to the list of searched sites and don’t search the entire web. Add your Google Adsense ID (section “Make Money”) and head to the section “Look and feel” and select the option Iframe. Choose a style for the search form / result and maybe you like to customize the style. Push now the button “Get code” (or enter the section “Get code” from the sidebar) and enter there the URL from your error page.
Copy / paste the code for the search form and the results into the body section from your error page. Your completed page will look like:


$errorNum = (int)$_GET['err'];
$err_str = array(404=>'Not Found', 400=>'Bad Request', 401=>'Unauthorized', 403=>'Forbidden', 500=>'Internal Server Error');
echo '



'</span><span style="color: #339933;">.</span><span style="color: #000088;">$err_str</span><span style="color: #009900;">[</span><span style="color: #000088;">$errorNum</span><span style="color: #009900;">]</span><span style="color: #339933;">.</span><span style="color: #0000ff;">'


An error occured: '.$err_str[$errorNum].'
 

  
    
    
    
    
    
  


 


';
 
if (empty($_GET['q'])) { // show the ad only if there is no search
	echo '
';
} 
echo '


';
?>

We placed the add code also into some IF clause, because there should not be another Google Adsense advertisement beside the Google ads from the result page.

Enable site search tracking in Google Analytics

The code for the error page is complete and we move to the last step: Tracking the site search queries from the error page. To do this we need to go in Google Analytics to the profile we created for the error page and click Edit (twice), check the setting Do Track Site Search, enter a “q” as the “Query Parameter” and click Safe Changes.

This error page is very basic and you need add your sites web template to make it complete. If you like this tutorial and you have used the code on your own site please share the URL to your new or updated error page. Even if you don’t like to use the code from this page, we advice to track the errors and also the site search queries from your visitors. If you have questions or comments please post them below.
Similar Posts:

Create surveys with Google Forms

Olaf — Tue, 06 Apr 2010 08:52:41 +0000

These days I needed an online tool to create a customer survey. Google Docs has a form feature which I tried this time and I was very surprised how easy and powerful this function is. Using Google forms, you don’t need any knowledge about how a web based form works. In this article we will show-case the following functionality:

Create a form with the WYSIWYG form editor
How your customers getting access to the survey
Analyze and and work on the results
Optional: How-to insert your survey form in your website using a lightbox

About Google Forms and Google Docs

Google Forms is part of Google Docs or better some kind of extension for Google Spreadsheets a web based spreadsheet program. To create a form using Google Docs, you need to have a Google account or a Google Apps account. Both account types are free available. Google offers also premium services for Google applications.

Creating web based forms without being a webmaster

If you start using Google Docs the first time, you get the idea that you’re using a similar application from the Microsoft Office suite. The menu and a lot of functions are very similar. Creating a form is very easy, you don’t need to write any code, just click some buttons and enter your text information:

Using Google Docs, you can build forms with all form elements you need: text fields, text areas, check boxes, radio buttons, select lists. Create forms with multiple pages and describe each page and/or form element with the information you like. There are also basic validations for required fields.
Every element has clear and easy to use dialogs. Based on the language you’re using in Google Docs, you can create forms in most common languages. If your form is ready, you can choose from many fancy themes or if you don’t like them use plain theme type.

How-to send the survey to your customers?

Each Google Form is connected to the contact list from your Gmail account, you can use this list or just post the link to your survey using your mailing list hosted by Mailchimp or any other provider you like. If you use the Google contact list you should create a contact group to save time. Google Apps users are able to collect the user name and/or restrict the form only for users having an Google account from the same domain. If you use a regular Google account, these functions are not available. Don’t forget to post the survey to your company’s facebook fan page or your twitter feed if you have one or both of them.

Easy data access and statistics

Every form submission is stored in a spreadsheet at the moment the user has clicked the submit button. All submitted form values are accessible and stored together with the timestamp from the moment that the form was submitted. Beside the spreadsheet there is a data summary using graphics which will give you some quick information about how often the data was submitted.

Optional: Stylish integrations for your website or blog

Since Google Forms are hosted off-site, you need some way to embed the code in your website. Google is offering some code snippet which you can use as an Iframe but you can’t customize the style to match your own site’s design. Because you can’t change the style or code, we need an easy way “make it a part” from your website:

Create your form and use the plain theme

Add the following code to your websites header to include jQuery and the thickbox plugin:

<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="thickbox.js"></script>
<link rel="stylesheet" type="text/css" href="thickbox.css" media="screen" />

You need to download upload the files if they don’t exist on your site.

Create a link to the form URL (copy paste from the footer section on the page where you can edit the form)
```
<a href="http://spreadsheets.google.com/viewform?hl=en&formkey=dDZjNGROdVllS2JYMGdSZnVSSWF4YkE6MA&TB_iframe=true&height=400&width=500" class="thickbox">Survey Form</a>
```
Don’t mix the query parameters, any parameter after “TB_iframe” is not recognized on the target page. Change also the values for the width and height to match your forms dimensions. The pseudo class name “thickbox” is used to enable the tickbox feature.
After submission Google will show a confirmation screen, don’t forget to edit this message (from the edit screen: More actions -> Edit confirmation

We created some example forms using multiple pages and most of the form elements.

Similar Posts:

8 Link building ideas for corporate websites

Olaf — Wed, 10 Mar 2010 23:07:44 +0000

If you need to promote your corporate website, you will find out that this is not an easy job. Self-promotion via forums or blog comments is often eliminated as SPAM and many website owners doesn’t like to link to your site because you offer a product or service. We all know that inbound links are important to get high-listed in search engines like Google. So what do you need to do to get links to your site? In this article we will suggest things which should work for most of the websites on the Internet. But first we need to change our mind:

To get inbound links you need content which is not just your product information or some sales page. Start creating content which is interesting for your target group AND related to your products.

What do you need?

Beside your corporate website you need a blog. This blog might be a part of your website or if possible hosted on a second domain and IP address.
You need to create unique and fresh content, related to your product and services. Hire a content writer if you can’t write the articles by yourself.
You need time and maybe some money (more about the costs later)

Okay here are the natural link building ideas:

Write your blogs frequently
If you have already a blog, I guess most of your articles are about your news, products and services. Keep writing them, but write also information which is “not about you”. For example if your product is Web Hosting, provide valuable information about how-to secure websites or give some other advice related to hosting or websites. If your articles are interesting even for non-customers, those people might write about on their own blog and will link to the source.
Write content for 3rd party websites
Write articles on blogs, websites or networks like HubPages.com, EzineArticles.com or blogs which allow other authors to write for them. In these articles it’s possible to add also links to your site.
Promote all articles via social networks
This one important, if you post your article-links to your twitter or facebook profile, more people will know about your blog and the chance is bigger that other people will blog about your article. Try also sites like Digg.com or delicious.
Get links from customers or partners
Ask your customers and partners for a link from their website. Start with the partners where you buy products and services, they might give some link back because you’re a customer. For your own customers it’s more effective to offer something for free.
Offer freebies
Provide in some of your own blog posts a free gadget, e-book or service. Freebies are always a welcome subject to write about. Be creative and you will get many free links!
Submit your press releases
Write press releases about your most important news and submit them to press release submission services. There are many of them (check Google), try also paid services like PRWeb.com, this way you’re able to reach many people which might pick up your news and will write about (incl. link). If you can’t write a press release article by yourself, hire a copy-writer.
Ask for a product review
Search related websites or blogs and ask the owner to review your product or service in exchange for a free product. Get this way a link and also some publicity (works great if your product is a service).
Create a second blog
If you’re able to produce enough unique content or articles, start a second website or blog. Host this website on by different web-host and don’t link this site from your other sites. If you keep this site “product neutral” you can even write about your competitors products and services. Build links from different sections to your site.

We didn’t mention here the reciprocal link exchange and paid links because those are against the Google webmaster guidelines. Exchanging links is not forbidden, but don’t have too much of them.

Do you know some nice link building techniques? Please share them below as a comment, thanks!Similar Posts:

Google Custom Search in a Lightbox!

Olaf — Fri, 05 Feb 2010 08:42:33 +0000

There are several tutorials on the Internet on how-to integrate the Google Custom search in your website, but the results are not very attractive. The older iframe version doesn’t fit perfectly in most of our templates and the style for ads and results doesn’t look good in the past. A few month Google has released some nice new features:

Custom search element placements
Custom styles and themes using CSS
Promotions, set your own premium results for specific search queries
A new AJAX powered CSE using the Google AJAX API loader

With all these new features the CSE (Custom Search Engine) becomes very attractive to use it on your own site as site search. In this tutorial we show you how-to add the custom search feature to you site using jQuery, the Thickbox plugin and the AJAX API provided by Google. To do this a result page is not necessary anymore, we show only a search box somewhere on a particular website.

Including the Google API, jQuery and the Thickbox plugin

First include the following rows to your page HEAD, you need to add them on every page where you like to show the search box.

<script type="text/javascript" src="jquery-1.3.2.min.js"></script>
<link rel="stylesheet" href="thickbox.css" type="text/css" />
<script type="text/javascript" src="thickbox.js"></script>
<script type="text/javascript" src="http://www.google.com/jsapi?key=YOURAPYKEY"></script>

To use the Google API you need to register an API key for your website, request a key here.

HTML code for the search box

Below the simple HTML code for the search form and the container for the results. Place both somewhere on your webpage.

<div id="searchcont">
	<label for="query_input">Search Query: </label>
	<input type="text" id="query_input" />
	<input type="button" value="ok" id="searchbtn" />
</div>
<div id="resultcont">
	<div id="title" style="visibility:hidden;"><a href="#">Close</a></div>
	<div id="results"></div>
</div>

jQuery code snippet

The following JS code will load the Google API and if the user has clicked the search button the customSearchControl is initiated, the Thickbox is called, the title inside the Thickbox becomes visible, the result element is prepared and at last the input element value is executed for the search query. Add this code in your page’s HEAD surrounded by some script tags.

google.load('search', '1');
$(document).ready(function() {
	$("#searchbtn").click(function () {
		var customSearchControl = new google.search.CustomSearchControl('UNIQUE_CSE_ID');
		tb_show("", "#TB_inline?height=480&width=800&inlineId=resultcont&modal=true");
		$('#title').css("visibility", "visible");   
		var drawOptions = new google.search.DrawOptions();
		customSearchControl.draw(document.getElementById("results"), drawOptions);
		customSearchControl.execute(document.getElementById("query_input").value);
	});
	$("#title a").click(function () {
		tb_remove();
		$('#title').css("visibility", "hidden");   
		$("#results").hide();
	});
});

The other click event is used to hide the Thickbox, the event will hide the title and hides also the results. The last action is needed because we show an inline element, if we don’t hide the content the results are still visible after the Thickbox is closed.

That’s all, simple or not? The code above needs of course some styling. I used a slightly modified version for my network site search on this blog. If you search for “wordpress” you see the “promotion” feature I mentioned above.

Similar Posts:

Top your sales with Google Commerce Search

Olaf — Sun, 29 Nov 2009 18:39:07 +0000

As a web shop owner you like to make your shop as popular and successful as possible. If you like to take your business to the next level, you should take a look at the Google Commerce Search. This hosted e-commerce search solution will allow your customers to have a much more pleasant and better shopping experience. The easier your e-commerce website for your customers, the better the chances are that they will buy your product and maybe will become a loyal customer.

Why Google Commerce Search?

Google Commerce Search is a scalable hosted e-commerce search solution. Using this Google service it’s possible to present your products online with a “special” type of Google site search.
But there is more:

Don’t worry about pikes of high traffic, Google Commerce Search will serve results for all your customers.
Your shop data become available in Google Merchant Center and inside the Google Product Search.
Synchronize or upload your data using the Google XML API.
Customize your on-site search engine so that it has the appearance of your website.
Manage all your products via the easy to use administrative console.
Analyze your statistics and use Google Analytics for details you might have never seen before; Follow reports on such items as top queries, special features usage, and much more.
Google Commerce Search makes the product search user-friendly. Provide for your products alternative spelling options or common synonyms. Your customer will see how easy it is to find a product, with the result of a higher conversion rate and a big chance that the customer will buy again.
Promote certain products and add them to related search results. This way you’re give more attention to specific products and maybe you can raise the purchase amount as well.
There is no waiting time, after your products are available in the Google Merchant center, they are available inside the search results.
Integrate Google Checkout and create a standalone shopping cart system (option)

Is my old shop or website not needed anymore?

Sure of course not! The checkout process and all detailed product information stays on your current website. Even if you use Google Checkout as payment solution you should use your own website for detailed product information and reviews. Your main website is the key to get products into the results of the “regular” Google Search engine. Add the search engine form to all pages of your own site for your customers convenience.

Is this a great solution or not?

There is one point which makes this service only attractive for established online shops: The Google Commerce Search is not free, but it is well worth the cost.

Google Commerce Search will help your customers to find what product it is they are looking for quick and easy. They will be able to filter the product results they are looking for according to the price of the product, the brand, the category, or many other ways. This helps to cut down on the chances of your potential customers locating the products on another page first. When people can’t find exactly what they are looking for on one page, they will pull up a second page and begin looking at different options. With the help of Google Commerce Search you can avoid this happening to you.

Google Commerce Search is and easy and effective way to handle your online sale. The product search makes everything simple and you don’t need to use your own shopping cart software anymore

Promote your products on your website, they way you did before and make more sales using Google Commerce Search!

Add a Google Sitemap for your Posterous Blog

finalwebsites.com — Wed, 18 Nov 2009 19:53:30 +0000

Posterous is a great hosted Blog system that we talked about in this blog about social media services. The current version has already a lot of features like custom themes and functions for services like Twitter and Facebook.

The best of all their service is still free!

I noticed the last weeks that Google doesn’t index all my Posterous pages and that while I point my blog Prime Blogger to a .com domain name. My first idea was a missing Google Sitemap. I tried before to add my Posterous site to my Google Webmaster Tools account, but this requires a custom meta tag on your homepage or you need to upload a unique file to your site to proof the ownership. I suggested a function for a Google Sitemap to Brett from Posterous and he gave me the hint that I’m able to change the HTML using a custom theme and yes this is key to get your site accepted in your Google Webmaster Tools account.

Just add your site to your account and choose the meta tag to proof the sites ownership:

Next surf to your Posterous site and in the control panel choose your blog -> Settings -> Theme and customize my site -> Advanced.

Paste the meta tag code into the code box at the top. Save the settings and confirm to copy your custom theme settings.

Now it’s possible to confirm the website ownership within the Google Webmaster Tools. Next we like to add a sitemap, click Site Configuration -> Sitemaps -> Submit a sitemap

Posterous doesn’t offer a Google sitemap format but you can use the XML feed instead. This gives you not a complete sitemap but at least you can submit the latest 20 blogs to Google. I hope it helps…Similar Posts:

New Search Options but unrelated results in Google Blog Search

Olaf — Thu, 01 Oct 2009 18:40:01 +0000

Today I read about the new search options on this Google Blog and thought great more control. I didn’t understood why people should use the Blog Search feature from Google because I was never that satisfied as the regular search.

Using the new search options I tried some searches and selected the Blog filter as well. I was very surprised to find in the Blog search result a few duplicates from my Blog post two weeks ago.

First I thought maybe my blog was not so relevant at the moment, but after I switched off the Blog filter I found my Blog post at the top of the list.

I know that the Blog search has some great real time results but this make no sense. Google shows some excepts or duplicates instead of the original content. Similar Posts:

Sending e-mails via SMTP with PHPmailer and Gmail

Olaf — Tue, 01 Sep 2009 19:40:25 +0000

These days I tried some plugin to send e-mail message within bbpress via SMTP. Since my domains email is hosted with Google applications I decided to send my messages via the SMTP server from GMail. I found several articles and tutorials, but a lot of them didn’t worked for me.

Why using GMail for sending mail messages?

First of all it’s FREE! Sure most website owners can use their own SMTP server for sending email messages from their website, but it makes sense even than to use Gmail for sending mail. The chance is big that your websites IP address is on a blacklist if your site is on hosted by a shared web hosting provider. If not or you host your site on your own server, there is always a risk that your IP address get blacklisted. Because of some limitations, the SMTP server from Google is a good choice applications with less than 500 recipients a day, check this information from the Google help pages.

Requirements

You need for this code example a PHP5 enabled web host (I tested only on Linux), the port 465 need to be open and of course you need a GMail or Google Apps account.

SMTP for Gmail tutorial

If you don’t have one, register a GMail account or setup your domain for Google applications.
Download a recent version of PHPMailer (I used the version 5.02)
Check with your web hosting provider if port 465 (TCP out) is open, if not ask him to open that port
Include the PHPMailer class file: require_once('phpmailer/class.phpmailer.php');
Create those two constant variable to store your GMail login and password. Use the login for your Google application mail if you have one.
```
define('GUSER', 'you@gmail.com'); // Gmail username
define('GPWD', 'password'); // Gmail password
```

Use the following function to send mail messages (add the function in one of your included files):

function smtpmailer($to, $from, $from_name, $subject, $body) { 
	global $error;
	$mail = new PHPMailer();  // create a new object
	$mail->IsSMTP(); // enable SMTP
	$mail->SMTPDebug = 0;  // debugging: 1 = errors and messages, 2 = messages only
	$mail->SMTPAuth = true;  // authentication enabled
	$mail->SMTPSecure = 'ssl'; // secure transfer enabled REQUIRED for Gmail
	$mail->Host = 'smtp.gmail.com';
	$mail->Port = 465; 
	$mail->Username = GUSER;  
	$mail->Password = GPWD;           
	$mail->SetFrom($from, $from_name);
	$mail->Subject = $subject;
	$mail->Body = $body;
	$mail->AddAddress($to);
	if(!$mail->Send()) {
		$error = 'Mail error: '.$mail->ErrorInfo; 
		return false;
	} else {
		$error = 'Message sent!';
		return true;
	}
}

Most of the setting inside the function are required by GMail. While searching for tutorials I found articles with different settings for the port and security. My advice use them as in this tutorial.

Call the function within your code:

smtpmailer('to@mail.com', '', 'from@mail.com', 'yourName', 'test mail message', 'Hello World!');

Use this more “advanced” usage inside your application:

if (smtpmailer('to@mail.com', 'from@mail.com', 'yourName', 'test mail message', 'Hello World!')) {
	// do something
}
if (!empty($error)) echo $error;

Advanced setup with fall-back SMTP server

Because of the limit it might be useful to use a secondary SMTP server if the Gmail option didn’t send the message. For this functionality we need to replace the part with the SMTP settings a little bit. First create login/server variables for the 2nd SMTP server:

define('SMTPUSER', 'you@yoursmtp.com'); // sec. smtp username
define('SMTPPWD', 'password'); // sec. password
define('SMTPSERVER', 'smtp.yoursmtp.com'); // sec. smtp server

Next we need to create a if/else statement using the variables for the second server (replace).

function smtpmailer($to, $from, $from_name, $subject, $body, $is_gmail = true) { 
	global $error;
	$mail = new PHPMailer();
	$mail->IsSMTP();
	$mail->SMTPAuth = true; 
	if ($is_gmail) {
		$mail->SMTPSecure = 'ssl'; 
		$mail->Host = 'smtp.gmail.com';
		$mail->Port = 465;  
		$mail->Username = GUSER;  
		$mail->Password = GPWD;   
	} else {
		$mail->Host = SMTPSERVER;
		$mail->Username = SMTPUSER;  
		$mail->Password = SMTPPWD;
	}        
	$mail->SetFrom($from, $from_name);
	$mail->Subject = $subject;
	$mail->Body = $body;
	$mail->AddAddress($to);
	if(!$mail->Send()) {
		$error = 'Mail error: '.$mail->ErrorInfo;
		return false;
	} else {
		$error = 'Message sent!';
		return true;
	}
}

And use the function now as followed:

$msg = 'Hello World';
$subj = 'test mail message';
$to = 'to@mail.com';
$from = 'from@mail.com';
$name = 'yourName';
 
if (smtpmailer($to, $from, $name, $subj, $msg)) {
	echo 'Yippie, message send via Gmail';
} else {
	if (!smtpmailer($to, $from, $name, $subj, $msg, false)) {
		if (!empty($error)) echo $error;
	} else {
		echo 'Yep, the message is send (after hard working)';
	}
}

Both examples are very simple and demonstrate only how-to send mail messages via the SMTP server from Gmail. Of course the user can extend the code to handle HTML messages and attachments etc. If you have any issues with these examples, just let me know. Post a comment here or post to our partner PHP script forum.

Similar Posts:

Impressions from the Google I/0 conference

Olaf — Thu, 29 May 2008 20:52:23 +0000

or Google the Open Source Company presents their new technology!

The Google I/O conference in San Francisco is the first of his kind, during two days the conference provides the attendee 90 sessions grouped by the following categories:

Ajax and JavaScript
APIs and Tools
Maps and Geo
Mobile
Social

Furthermore they held sessions like Code labs, Tech Talks and the Fireside Chat. So maybe you’re looking for the “Search” category? Yes there are a few lessons about search optimization and the Ajax Search API, but most sessions are about new modern technologies like Gears and the Google App Engine.

Those people who knows Google as the search engine provider and advertising company would be very surprised about the strong open source character of this conference. With the vision

Create a better, full featured Internet using modern technologies

Google has invited any developer to use Google technologies in their own applications.

While listening to several sessions, I get an good idea about Gears, OpenSocial, Google APIs like Authentication API, content API’s and the Google Checkout API. Before joining this conference I accepted to learn something I can use within my next projects. OpenSocial is something I like, a kind of standard how to create your own Social Community.

My first impression about Gears is not so positive; the idea is really good, but it looks like a browser add-on aka Adobe flash (not the function but the idea how to provide the technology to the client). All presentations are available to the public next week on the Google I/0 website.Similar Posts:

Optimizing your WordPress Blog for Google: Part 1

Olaf — Sun, 02 Mar 2008 09:34:33 +0000

How to avoid the duplicate content penalty when using a WordPress blog

There’s no denying it, Google is the powerhouse of the internet; more than capable of showering targeted visitors on your website like a torrential downpour backed by gale-force winds. That is, if you happen to be in Google’s good graces and rank highly for popular keywords.

Bloggers often times find themselves in the position to receive great traffic from Google simply because of the way that blogs work; namely, they allow for quick inclusion into major search engines even when using the default setup. Being the most popular blogging software on the planet, WordPress blogs are the content management system (CMS) of choice for bloggers due to ease of use and the large community of supporters that help to develop plugins for this open source blogging software.

However, just because WordPress blogs can help websites to get noticed by the major search engines easily, this doesn’t mean that they are inherently search engine friendly; in fact, the default settings for WordPress blogs almost guarantee that, if left untouched, your blog will end up suffering from the duplicate content penalty.

The duplicate content penalty and WordPress blogs

The duplicate content penalty is a term that is used to describe what happens when a web page is removed from the primary search results for a certain keyword phrase due to identical content elsewhere on the internet. In Google, the lower-ranking websites and individual pages that contain the duplicate content are hidden, and this phrase is displayed instead:

In order to show you the most relevant results, we have omitted some entries very similar to the X (number) already displayed.

This happens primarily in two situations; with article marketing, A.K.A. article directory submissions, and when you post content to an unmodified blog. Since this article is focused on WordPress blog optimization, we’ll focus on how you can avoid the duplicate content penalty for the latter.

Unmodified WordPress blogs are search engine “unfriendly”

By using a WordPress blog with its default setup, you are creating an atmosphere where your blogs’ content is almost guaranteed to suffer from the duplicate content penalty. The reason for this is simple: By default, your WordPress blog will have the exact same content, word for word:

On the blogs main page
Within the blogs RSS feed
On the category page
On the monthly archive page
On the posts unique page

Clearly, having 5 different instances of the exact same content within a single website will undoubtedly lead to your content being seen as “duplicate” content by Google, and other major search engines for that matter.

Avoiding the duplicate content penalty with a WordPress blog

There are a couple of things that can be done with your WordPress blog that will help you to keep your blog in the primary search results:

Limit the text shown on your blogs pages

The duplicate content penalty comes about due to large amounts of significantly identical text being shown on numerous pages throughout the internet, not a few characters or even a couple of sentences. WordPress blogs allow bloggers the option to use what is called the “more” tag, where they can limit the amount of text that is displayed on the blogs main page, the category pages and the archive pages. By utilizing this tag, WordPress users are limiting the content for a post to be displayed, in full, only on the posting page itself; thus removing four of the five aforementioned instances of locations of duplicate content when adding a post to their blog, which can help to decrease the chances of being affected by the duplicate content penalty.

However, if you have an established blog – it would be a cumbersome chore to edit every existing post, not to mention it’s another step you would have to endure to use this option for all future posts. Fortunately, there is a WordPress plugin that will replicate the effects of the “more” tag, yet not requiring you to manually add it to your blog posts.

Evermore WordPress Plugin

The evermore WordPress Plugin will replicate the effects of the “more” tag, effectively limiting the amount of text shown on your blogs main page, category page and archive pages. You have the ability to specify the character limit, so you can have more or less text displayed in these other areas depending on your needs. Installing this plugin is as easy as uploading a file via FTP to your WordPress plugin directory, and the effects can be reversed completely by deactivating it.

Choose “Summary” for your blog’s RSS feed file

Another way to help stop the chances of becoming the next victim of the duplicate content penalty is to simply change the settings from within your WordPress blogs administration area.

Once you’re logged into your blogs administration area, go to Options>>Reading. Under the sub-heading “Syndication Feeds”, select the “summary” option. This will effectively cut down your chances of suffering from the duplicate content penalty.

Using the robots.txt file to keep Google (and other search engines) away

Now, of course you want search engines to crawl your blog, but in some cases it’s not in your best interest to have all of your pages crawled and indexed by all search engine robots. Another trick to helping you avoid the duplicate content penalty is to tell the Googlebot to stay away from your RSS feed. Some bloggers even go so far as to disallow the Googlebot from crawling their blog archives and category pages, but that is a personal preference; in all honesty we have yet to see conclusive evidence as to which approach is best. Here is an example of a set of commands that will help you to keep the Googlebot from accessing your blogs feeds.

Sample robots.txt file to disallow the Googlebot from your RSS feed

(if your WordPress blog is at the root of your domain)

User-agent: Googlebot Disallow: /feed/$ Disallow: /feed/rss/$ Disallow: /trackback/$

You can check out some advanced commands for your WordPress blogs robots.txt file on the Ask Apache website. As with any modifications of this type, you should implement changes with extreme caution as improperly forming commands to your robots.txt file may cause it to be excluded from search engines all together!

Using the tips and tricks outlined here, you can help keep your blog in Google’s primary index and avoid the duplicate content penalty that many other bloggers suffer from. It goes without saying that one of the major contributing factors to the duplicate content penalty is syndicating articles from article directories; and for best results it’s always suggested that your blog is comprised of unique content rather than syndicated content.

This article is a 2 part series on how to optimize your WordPress blog for Google. In our next article, we will show you how to optimize your WordPress blog for ultimate search engine friendliness and also how you can pull your blog out of Google’s supplemental index as well as covering ways to stay out of it.Similar Posts:

Web Development Blog » google

404 Errors: Report, monetize and analyse

Error reporting page

An error occured: '.$err_str[$errorNum].'

Track dead links in Google Analytics

Adding Google Site Search to your 404 error page

An error occured: '.$err_str[$errorNum].'

Enable site search tracking in Google Analytics

Create surveys with Google Forms

About Google Forms and Google Docs

Creating web based forms without being a webmaster

How-to send the survey to your customers?

Easy data access and statistics

Optional: Stylish integrations for your website or blog

8 Link building ideas for corporate websites

What do you need?

Google Custom Search in a Lightbox!

Including the Google API, jQuery and the Thickbox plugin

HTML code for the search box

jQuery code snippet

Top your sales with Google Commerce Search

Why Google Commerce Search?

Is my old shop or website not needed anymore?

Is this a great solution or not?

Related products, services and resources:

Add a Google Sitemap for your Posterous Blog

New Search Options but unrelated results in Google Blog Search

Sending e-mails via SMTP with PHPmailer and Gmail

Why using GMail for sending mail messages?

Requirements

SMTP for Gmail tutorial

Advanced setup with fall-back SMTP server

Impressions from the Google I/0 conference

Optimizing your WordPress Blog for Google: Part 1

How to avoid the duplicate content penalty when using a WordPress blog

The duplicate content penalty and WordPress blogs

Unmodified WordPress blogs are search engine “unfriendly”

Avoiding the duplicate content penalty with a WordPress blog

Limit the text shown on your blogs pages

Choose “Summary” for your blog’s RSS feed file

Using the robots.txt file to keep Google (and other search engines) away