Main Content

What search terms FAIL to bring visitors to our site, when they should?

Archive - Originally posted on "The Horse's Mouth" - 2009-08-05 18:02:06 - Graham Ellis

My web log files, with careful analysis, tell me what search terms brought people to our pages. But what I would really like is to find commonly searched terms which should bring people to us, but do not ... so that I can optimise the site for them. And of course there's nothing in the log files to tell you what these underrated/unrated terms are - or so you might have thought.

With the combined log file format, the Referer string includes the query that the user made (encoded) and also a parameter showing the start point in the results set. Without that parameter inthe referer, you are loking at the start of the result set ... with that parameter, you have a user who has moved on from the first page to a later page of results.

Here are a couple of analyses from yesterday:

4 arrival(s) to /resources/ex.php4?item=h110/mcheck.php
  1 - php select radio
  1 - php radio exemple
  1 - php template for radio
  1 - php select from (page starts 10)

That's three users who came to the page in question because their search presented our site on the first page, and one who searched for "php selelct from" who wasn't satisfied with what (s)he found on the first page, and moved on to the second page.

6 arrival(s) to /resources/ex.php4?item=j907/index.jsp
  2 - shopping cart jsp
  1 - jsp source code for online shop (page starts 30)
  1 - shopping cart example jsp
  1 - download sample code for shopping cart in jsp
  1 - shopping cart jsp example

And that's a page which our patient user, searching for "jsp source code for online shop", didn't reach until the fourth page.

This page start information is a huge clue to lost traffic. With only 4% of our visits being as result of our links being found on 2nd or subsequent pages (see here), we're been shown the tip of an iceberg - and we could do well to trawl through all our second (and later) page arrivals and see if we should be making more effective use of the words that were used in them.

If you're interested in what subjects users have found recently that are not on the first page, look here for Python - Perl - PHP - MySQL - Tcl - Ruby - Lua - Java - Tomcat - Apache - Linux - Shell - Expect - Database - Unix and other topics