A reader on my mailing list recently asked why search results are different between the different search engines. The short answer is that each search provider has independently developed software that converts its view of the content on the web into a list of results when you enter a search phrase. There’s no “unified search algorithm” for search so there can’t be “unified search results.”
How do search engines work?
The process is pretty straightforward: A search engine starts out with a collection of web pages from the Internet in a big database. It gets these page through direct submissions and through automated web browsers called spiders.
The search engine also has software to analyze and index that collection of data. It’s like the Dewey Decimal System on crystal meth.
When you enter a search phrase the search engine organizes its collection of web pages into the most relevant results. Those are the search results.
Because the search engines don’t know what you’re going to search for, the system has to be very adaptive and fast. When you search for “best BBQ grills to buy for my mom,” the search engine has to figure it out.
The computer logic that creates that list of search results is worth a lot of money to the search provider because the more reliable you perceive the results to be, the more you’ll trust that provider in the future.
Pay Per Click ads and not search results
By the way, most search engines also display a series of paid advertisements that relate to your search phrase. These ARE NOT search results. They are paid ads. Paying for an add is not SEO. More on that in another post.
The evolution of search
The complexity of the search algorithm has evolved a lot over the years. Once upon a time, you put keywords into the META KEYWORDS field of each HTML page on your web site. If you wanted to beat someone else in the search listings, you would repeat your main keyword more than the other guys. Obviously, this got abused and now the META tags are completely ignored for search results.
Some people think the META tags are what SEO is still about…usually when I’m talking to a new client about SEO, it’ll come up. Someone in the room will say “Oh, we’re already doing SEO. We’ve filled in the META tags.”
The results from even one search engine change over time because the search algorithm is in a constant state of tweaking and development. Simple frequency of keywords in your content used to be enough for a good ranking. Other factors like “back links,” and words related to the keyword being searched for also oscillate in importance. Google and the other major search engines—basically Bing—invest a lot of time and effort into their search technology.
Well someone’s gotta pay for this. If you go to Google for your search, and assume the results to be the best results, why would you go to another search engine and look up the same thing? The more a search engine keeps you on their site, the more valuable their ad space.
What if you’re not Google and want people to use your search engine? You need to convince people that your results are either better or that you provide some additional service making you more valuable.
The dark side of search
Search engines also have to deal with spammers and “black hat” SEO techniques. There have been lots of attempts to manipulate the search algorithms and provide junk pages. One method involved something called cloaking. Special web software knew the difference between a search engine spider and a human being—in theory anyway—when the spider would visit, it would be shown a web page highly optimized for a specific keyword. When a human visited, they were shown a sales page of some sort.
This makes the search engine look bad because you blame them for sending you to a spam page. As a result, some of the search technology is designed to keep web pages out of the listings. This also varies from search engine to search engine. It’s a constant cat and mouse game.
What’s on the horizon for search?
Competition for search is heating up. Microsoft executive Yusuf Mehdi recently told the LA Times that Bing has “caught up to Google” with their search results. That sounds like a dare to me, but we only benefit from two sluggers like Google and Microsoft working harder to provide us with better search capabilities.
Times change and so do search results
New content is showing up in search engines as well. Twitter is now indexed by Google in “real time,” and YouTube videos often get top search rankings these days. This is a reflection of what people consider to be relevant results. If you Google for the latest buzz on a new movie, Twitter traffic is likely to be where the conversations are taking place. Some crusty movie review section of a major newspaper is less relevant because it’s not where the people are.
Ultimately, search is a reflection of the people behind the content. If the Internet was only about connecting computers, it would be an uninteresting place indeed. Likewise, if search results were nothing more than a dry listing of data and web pages, it wouldn’t be all that interesting either. As search evolves, pulling in geolocation as part of the relevance factor as well, the results will become an almost group Rorschach Ink Blot test.
The results of a search are a reflection of us.
SEO and you
When it comes to your own web presence, SEO is not something you can set and forget. It requires constant effort. Not daily effort but if your search results are important to your business, you monitor the results you’re getting for keywords you’re interested in and keep an eye out for competition.
Is search and SEO important to your business? What questions do you have about it? Sound off and leave a comment below.