x-Factor web pages are "advanced" says Google's reading level

Google has rolled out a new search option that assigns a reading level to the pages in your results list. Don't be surprised if you haven't spotted it yet; it is hidden on the advanced search screen. Under the "Need more tools?" section you can choose from the drop down menu to see all of the results with reading level annotations, basic results, intermediate results or advanced results.

Google does not give much away as to how it calculates the reading level and it has nothing to do with the reading age that publishers assign to books. It could involve sentence structure, grammar, the length of sentences on a web page, the length of the document, the terminology used and doubtless many other criteria. But Google isn't saying.

If you have opted to see the annotations, at the top of your results page you will see a graphic showing the percentages for each of the categories. Under the title of each entry in your results list is the reading level.

Click on the Basic, Intermediate or Advanced links next to the bar chart to see pages for that reading level. The eagle-eyed amongst you will have spotted that Google appears to be mathematically challenged because the numbers do not add up to 100%. In all of the searches I have done so far 1 or 2% are missing from the statistics. Looking through the lists of results some pages have no reading level assigned to them and they seem to be documents that contain very little information, have more numbers than text, and some are formatted files. Note, though, that most file formats do have a reading level so why some are not picked up remains a mystery to me. Some Daily Mail articles do not have a reading level either but many would argue that they fall into the 'very little information' category!

Once you have used the Reading Level in the advanced search screen you can change your search on the results page and it remains as part of your search strategy until you close down your browser or tab.

You can also check out an entire web by using the site command, for example site:rba.co.uk

And this is where you can start to have some fun comparing sites (WARNING - this is addictive!). Phil Bradley has done some in his blog posting Google adds reading level
(http://philbradley.typepad.com/phil_bradleys_weblog/2010/12/google-adds-reading-level.html). He also highlights some potential problems with labelling pages in this way. For example 'basic' does not necessarily mean stupid, but some people may be deterred from selecting basic pages because of the tag.

Most of my pages are classed as intermediate and I am happy with that. Many of them are listings and analyses of business information sources. My husband's blog on the other hand is 71% advanced and 27% intermediate. This comes as no surprise to me as he has a habit of littering his postings with complex calculations on topics such as wind turbine energy generation and the EROEI of tar sands oil production. (Just the sort of thing not to read before you have had your second cup of coffee of the day.) That plus the industry specific jargon that he uses makes an advanced tag inevitable.

The evidence so far seems to be suggesting that using terms or jargon that are relatively uncommon in the whole of the Google database is a heavy factor in determining the reading level. Let's look at what one might consider to be an intellectually challenging topic: the use of zeolites in environmental remediation.

That seems to confirm it.

As a final test and for a bit of fun let's look at what Google makes of a search on the recent x Factor final.

Noooooo! Surely some mistake? The X factor home page is rated as basic but 93% of the results are advanced. There is indeed a mistake but it was my sloppy search strategy. Changing the x factor part of the search to a phrase gives what I would expect and a switch to 53% basic, 40% intermediate and 6% advanced.

Out of curiosity, I looked at the content of the advanced pages and am now totally bemused. I cannot see how they could ever have been classified as such, but then this is Google we're talking about. Perhaps Google cannot comprehend the scoring system, why so many people watch it or why the programme exists at all?

I have experimented with several other searches. Some came up with results as bizarre as those for the x Factor search but it is interesting how the breakdown can be changed by slightly modifying your search strategy, for example by using phrases when appropriate or a plus sign before a term to force an exact match search. Google's Reading Level could be useful as a training tool to show how small alterations to a search strategy can radically change the results. But as with all things Google, we do not know how it works and the results can sometimes be very strange. Use with caution.