It all began with Yahoo, the largest entity in search following the invention of the World Wide Web in the modern sense. The year was 1994; people at Yahoo used to edit search results manually and then form the search listings.
The Birth of the Crawler
It all began when budding tech enthusiast and coder, Brian Pinkerton, created the first automated crawler that could display a list of top 25 websites from the available categories. Webcrawler was quickly gobbled up by Excite, a much larger business that used the crawler’s capabilities to generate their own search results. By late 1994, Lycos had implemented automated crawlers to generate all of their search listings, reaching a total of 60 million indexed documents during the next few years. Yahoo, on the other hand, refused to follow suit and continued to manually index websites, only using crawler-based results if there no human-powered search results available for a specific search term.
Google Page Rank
At this, the threat of SEO spam had pretty much taken over and was destroying the credibility of search engine results. Google Search, in 1998, came into being with their native Page Rank system which began ranking pages based on the number and quality of external links that lead back to the page. This effectively laid waste to the content farms that had become a popular way to increase search engine visibility, and instead contributed in the rise of link farms. People began bombing the search results with links, and managed to get completely irrelevant pages on top of the ranks. Entire business were being set up that provided paid services to site owners to increase search rank by creating link farms. By 2002, the issue had escalated to such a degree that it made its way to BBC news!
Smack Down!
Thus began the Google smack down! The search engine giant initiated a severe crackdown on sites that tried to manipulate search algorithms and blacklisted hundreds of thousands of websites during the next few years. The base algorithm was completely rewritten several times in the coming years to counter SEO spam and further streamline the relevance of the results that returned upon a search. An interesting trend remained common each time Google updated their algorithms though; a sudden influx of posts and books that proclaimed SEO was dead! Search engine optimization remains strong to this day though.
Analyzing Campaign Performance
A critical point in the history of SEO was the introduction of the Google Analytics tool in 2005. The tool enabled site owners to track and manage their SEO efforts through provision of detailed statistics regarding incoming traffic on their web pages. This changed the way people did SEO drastically, for the good.
During the last 10 years, numerous updates like Caffeine, Vince, Panda, Penguin, and the latest, Hummingbird have been implemented by Google to improve the relevance of search results and provide real-time search listings through collaboration with major social networks like Facebook and Twitter.