RSS feedsMeasured in terms of click-through numbers, Google dominates search in the UK, according to market research company NetRatings, with a 68% share. Yahoo trails in far behind in second place with 9.1%, followed by Ask (7.8%), Microsoft (5.8%), AOL (4.1%) and Orange (1.8%). In an endeavour to find out which is best, IWR put the UK’s six most popular web search engines head to head in the labs.
Speed of search, size of database, total number of hits returned – these are nothing but eye-candy in the world of serious web search. Who cares if a search generates 21 million hits? No-one is ever going to sort through that lot. All that really matters in search are the first few pages an engine finds.
The awful truth is that the vast majority of people will never venture further than the first page of results. Usually, the list contains 10 hits by default, although that number can rise considerably once the sponsored hits have been added top and tail. But whether the number of first-page results is 10 or 50, the crucial thing is that your relevancy ranking has done its stuff and moved the site you want to find near to the top – and that is what we have looked for in these tests.
We counted the number of relevant sites returned within the first 10 non-sponsored results for each of the 10 tests and simply added up the scores for a final relevancy percentage. This figure enabled us to come to a meaningful and very real-world conclusion as to who should be crowned king of web search.
The terms we searched for were deliberately chosen to put pressure on specific aspects of web search engines. They were as follows:
• “Apple Pie” –“Apple Computers” tests simple Boolean filtering, including within sponsored results (“Microsoft Office Online” OR “MS Office Online”) NEAR (Outlook –“Outlook Express”) tests complex Boolean constructs
• Plumbers in New Cross tests localisation functionality
• Raleigh tests ambiguous term refinement – we are interested in bicycles and nothing else
• What time is it in Bangalore? tests question interpretation
• iPAQ hw6900 tests product information location ability
• 08700 100 222 tests number handling; we were looking for links directly to the BBC as this is the complaints desk number
• 22/11/1963 tests date format handling; we wanted to find details of the JFK assassination
• pas de bourre tests foreign and niche handling (it’s a ballet term)
• Information World Review tests the ability to interpret keyword phrase grouping, ignoring common words to find the phrase without the use of quote marks
AOL
http://search.aol.com
Relevancy score 64%
Although AOL is probably not on the top of your list of trusted information
research sources, it is powered by Google and the fifth most popular search
engine in the UK.
We were disappointed with the fairly limited Advanced Search options that let you search within title/URL/text/links, specify language and document format, and filter by date, but only on a rudimentary three months, six months or year basis.
By default, Safesearch adult filtering is switched on, and there is no way to change the number of hits shown per page from the default 10. Sponsored links are topped and tailed to the results, but not obviously so because the word “sponsored” is greyed out and on the far right of the screen, well away from the results themselves.
We liked the FullView feature which integrates image, video, audio and shopping search results – or an overview of them at least – when a search triggers relevant hits. The FullView sidebar is informative and easy to navigate, but, along with a Smartbox keyword suggestion tool and the ability to rewind through your recent search history, only available if you use the US service and strangely absent from the UK portal.
Interestingly, our apple pie search produced no sponsored links, while the Microsoft query produced none for Outlook Express, unlike some of the other engines on test. The plumbers in New Cross search unfortunately had a hit at the top of the list which simply read “We have no plumbers in New Cross,” which was about as helpful as the second hit listing plumbers in Brent Cross.
The Raleigh test was always going to be a tough one – after all, how could the search engine know we were interested in bikes when we hadn’t actually said so? We thought the Smartbox keyword suggestion tool – which gives possible refinements in a dropdown list as you type your keyword – might help, but it didn’t mention bicycles at all.
What time is it in Bangalore? This straightforward question found four answers in the first four hits, after which things fell apart, with results pertaining to part-time jobs in India and publications such as the Times of India and Time Asia magazine.
But the product search for the iPAQ hw6900 was spectacularly impressive. Not only were all 10 hits information about the device in question rather than links to shops, but the FullView function came alive to display related items for sale – albeit at just one US-based store.
PROS: Multimedia search integration with FullView, Smartbox keyword suggestion, search history browsing
CONS: All the pros not available in the UK version, poor customisation, limited advanced functionality
Ask
www.ask.co.uk
Relevancy score 41%
Ask, formerly Ask Jeeves and also responsible for the popular MyWay service, is
powered by Teoma, which it owns. It uses a proprietary ExpertRank link
popularity algorithm that analyses links in context in a subject-specific manner
(so that a page about football, say, is ranked higher if other pages about
football link to it).
We liked some of the little touches such as the discreet binoculars icon next to many search results, which, when hovered over, pops up a preview thumbnail of that site. But the sponsored results display is disappointingly very similar to AOL, with the forced hits almost blending in with the real ones.
Ask fares better when it comes to site customisation. It lets you specify the number of results shown (to a maximum of 100) and choose to open results in a new window, although that’s about it. Advanced Search also mirrors AOL, albeit with finer date filtering granularity.
The adult filter is switched off by default – an odd decision we thought. But Ask has always wanted to be judged on its ability to find what you are looking for, without fuss, so how did it perform?
Well, the apple pie search returned not a single apple pie recipe or direct reference outside of the sponsored hits on the first page. The location-based plumber search found only two relevant hits among the “none found in SE14” and pubs, hospitals and builders merchants listed.
Although the Raleigh search had the same end result, the official Raleigh Bicycles site was listed first and the Narrow Your Search refinement tool had links for bikes, cycles, mountain bikes and even the much loved Chopper.
Considering Ask has always been a plain English, ask-a-question-get-an-answer-driven service, it was a surprise to find our one direct question – “What time is it in Bangalore?” – returning only one correct link. The saving grace was that the answer was also given as a “The time in Bangalore is…” statement at the top of the page, along with a box for checking the time in other locations.
The reviewer’s wife used to be a professional ballet dancer, so she supplied an obscure technical term in French: pas de bourre. Ask returned six hits, four of which were in English.
And our JFK date search revealed that 22/11/1963 equals 0.0010188487 but little else, although a “What happened on 22/11/1963” refinement link produced all the JFK information any conspiracy theorist could want. While the relevancy rating for Ask is low, its search refinement tools work well and often generate a single answer that meets your needs towards the top of the list.
PROS: Excellent search refinement tools
CONS: Intrusive sponsored hits, poor results relevancy overall, no adult filtering by default
Google
www.google.co.uk
Relevancy score 70%
Far and away the market leader for search both nationally and globally, Google
also has the biggest database (20 billion pages or more), although the precise
size is down to guesswork since the company stopped sharing such details with
the press some years ago. The market dominance and ensuing advertising clout has
sent the financial value of the company soaring, enabling it to invest heavily
in new areas of search, which shows in the sheer variety of niche search
services on offer.
However, we were concentrating purely on web search for this test, which is where Google’s legendary PageRank algorithm comes to the fore, using a combination of links and authority to sort relevancy.
Customisation is good, from the number of hits per page to a three-step adult filter set to moderate by default that removes explicit images but not text. When it comes to language settings, Google rules supreme. It can change the interface language itself and restrict searches by language or country of origin. There’s a free onsite text translation service and native Googles in myriad locations. Sponsored links are easier to spot than some, being in a shaded area heading the results, as well as in a separate sidebar.
In our tests we were expecting Google to perform well, but just how well was the big question. The simple apple Boolean construct generated nine hits with just the one, for a band called Hot Apple Pie, to spoil things.
The more complex nested search for Microsoft Office Online also did remarkably well with nine hits. Unsurprisingly, our plumber search suffered the same “not in New Cross” fate as the Google-powered AOL, but performed slightly better with three hits for Raleigh bicycles. The refinement box appeared with this test, but, unlike Ask, was restricted to stuff in and around Raleigh in North Carolina.
The right time in Bangalore was displayed at the top of the page, as well as five relevant hits, and a decent eight for that BBC complaints number. There were only six for the JFK date, though, which was disappointing, and no easy Ask-like refinement to come to the rescue – the similar Pages link is somewhat hit and miss.
The iPAQ product search threw up the Froogle box to take us to shopping comparisons if we wanted. We didn’t but were given eight relevant sites to ponder anyway. The final Information World Review search was impressive in not only generating a full 10 relevant hits, but expanding the first to include links to our blog, features category and so on.
Google turned in a good – if slightly inconsistent in places – performance and just manages to pip Yahoo to the winning post with a relevancy rating of 70%.
PROS: Database size, PageRank algorithm, niche search offerings
CONS: Disappointing localisation search
Microsoft
www.live.com
Relevancy score 52%
The successor to the popular but rather crowded and portal-driven MSN Search,
Microsoft Windows Live Search has gone for the polar opposite of its predecessor
and adopted an even less cluttered approach than Google, if such a thing is
possible.
Look beneath the minimalist exterior and you discover equally minimalist options. You are restricted to just changing your default language until you perform your first search, when the Options button opens up a much wider choice of customisation. As well as offering a good choice of number of hits returned (a maximum of 50) and language options, there are useful settings such as limiting the number of results (from one to three) from the same site that are grouped and displayed.
Adult filtering is via SafeSearch and defaults as Google does. But it is when you hit the Advanced search link that Windows Live Search really comes into its own. The link pops up a box that brings truly easy-to-use refinement to search, including a set of innovative results ranking sliders that let you quickly alter the ranking weighting between static and recently updated sites, popular and less popular sites, and exact and approximate match with the search term. Each slider has a 100-point scale, and defaults to 50 for each weighting category.
Sponsored search display is similar to Google and fairly unobtrusive. The inclusion of data from regional newspapers brings a local feel to news searches, and the large home-grown database powering the entire search is pleasingly effective.
Unfortunately, Microsoft got off to a very bad start with our simple Boolean test, returning just a single relevant apple pie result among the teddy bear shops and marketing companies. The complex Boolean construct did surprisingly well considering that the Near operator was treated as a search term – more luck than judgement we suspect.
Luck, however, was missing from the equation with our localisation search for plumbers. It turned up only two in New Cross – the same number as in Wales.
It was a much better performance all round with the Raleigh search. It returned six bicycle-related links as well as a news search box containing rather too much information about shooting in Raleigh, North Carolina.
Unfortunately, there was no nice “The time in Bangalore is…” response at the top of the page when we asked for one. In fact, a Bangalore escort agency placed higher than the first of the three correct responses.
Our JFK date search prompted the usual maths lesson at the top of the page, but only two relevant results topped and tailed the list, and there was no obvious way to refine the search.
All of which leaves us rather underwhelmed. If this is the best Microsoft can do, then Google and Yahoo have little to worry about.
PROS: Clean interface, innovative weighting controls
CONS: Poor results, lack of search refinement
Orange
http://search.orange.co.uk
Relevancy score 28%
Orange has a huge problem with the Overture network-driven sponsored links in
its results. Despite these having an easy-to-miss, very pale grey “sponsored”
tag, the same cannot be said of the results themselves, with sponsored links
dominating the first page of most searches.
The trouble is that Orange doesn’t show you the sponsored links and then the default 12 “real” results; instead, the sponsored links are included in the default 12. Thus, the search for apple pie but not Apple Computers threw up six sponsored links, all for the computer company, with just six “real” results, and only three of those were relevant.
Nor is there any control over the number of hits displayed, which just compounds the problem. When it comes to configuration options, Orange allows search term highlighting, which is most welcome, although we would have preferred full adult filtering rather than the default “display consent warning before displaying content” default.
And speaking of odd defaults, PDF, PowerPoint and Excel files are excluded from searches unless you specify otherwise. The Advanced Search page is very similar to the Options page, with language, country, file format and the most basic of matching choice (all, any or exact).
Complex Boolean searching is not something that appeals to Orange. Our test returned only one relevant hit, although thankfully it also successfully confused the sponsored links engine, so we saw none of those at all.
There was precious little sign of plumbers in New Cross either, with the first hit confusingly offering “All you need to know about Banbury” and others including a no doubt helpful in the right circumstances “EMERGENCY LOCKSMITH.
EMERGENCY LOCKSMITH.
EMERGENCY LOCKSMITH” but no New Cross plumbers at all.
The Raleigh search did at least stimulate the sidebar into action with further links for shopping, eBay and Thomson Local results (where was this when we needed the plumbers?) Of the eight non-sponsored links, four were bang on target.
Our quest for the time in Bangalore flummoxed Orange, generating zero results. Phone numbers fared little better: just four results including one duplicate – the only duplicate for any of the search engines on test.
Similarly, although JFK came second in the list when we looked for 22/11/1963, the only other reference was for another date and pointed to the Warren Commission findings.
We were not too hopeful for our technical French ballet term by this point, and Orange didn’t surprise, throwing up no directly relevant hits at all.
Even the seven hits for IWR couldn’t save Orange from what were, frankly, appalling results. The engine’s Search Made Simple motto seems sadly inappropriate.
PROS: Search highlighting
CONS: Overbearing sponsored hits, very poor results, lack of configurability
Yahoo
http://uk.search.yahoo.com
Relevancy score 68%
Yahoo started life as a directory of websites and has always had more than a
touch of the portal about it, rather than being a serious search engine in its
own right. In 2004 it reverted to using its own search engine database to power
its results, with full support for Boolean searching and the use of wildcard
words in phrases among the features, driving good search performance.
Like Google, Yahoo has introduced a number of niche searches as well as an onsite language translation service in Babel Fish.
Sponsored results are flagged by a combination of a shaded portion topping and tailing the main results and a dedicated side panel.
Unlike Google, Yahoo switches off SafeSearch adult filtering off by default, despite the preferences saying it is on – you need to sign in to your Yahoo account to lock the settings on which it would appear.
One unique and very appealing feature is the ability to include Factiva, LexisNexis and other databases within your search if you have a valid subscription.
Yahoo is at its most flexible and advanced when it comes to configuring the services we tested, as exemplified by the Shortcuts feature, which lets you search directly for time, weather, measurements, currency exchange rates and so on.
But the proof of this particular pudding is in the searching, and if Yahoo is to mount a serious challenge to Google it will need to better it in our rigorous testing procedure.
The simple Boolean apple pie test was passed with flying colours and eight relevant hits, the complex construct doing equally as well.
Localisation proved something of a success as well, with Yahoo popping up a Local Results entry atop the page showing three plumbers within a mile of New Cross together with a map link. We say “something” of a success because the actual hits revealed only one local plumber, although the combined hit rate of four matches Google.
The time in Bangalore was displayed via a Yahoo shortcut link prominently at the head of the page, followed by five relevant hits.
Our iPAQ product search turned up nine relevant information links with just a solitary shopping link to spoil things, although we were pleasantly surprised by the total lack of any advertising pushing us towards shopping comparison engines, which was a pleasant relief.
Yahoo has no problem handling numbers. The phone query pointed us towards the BBC in seven of the 10 results, and the 22/11/1963 format test produced six dead Kennedys.
Things went a little haywire with the French ballet terminology; despite an impressive seven relevant hits, only three were in English.
The magazine search with nine hits was no surprise, but Yahoo’s failure to beat Google overall did given that Yahoo felt like the more consistent search tool of the two.
PROS: Flexibility, database subscription option, full Boolean support
CONS: Not quite as accurate as Google
All Tags: Group Tests
del.icio.us
Digg this
reddit!
