Last updated on December 22nd, 2015
Welcome to SEO Advantage’s Trivia feature, where we discuss, dissect and comment on the internet and marketing, and how the two intertwine.
When was WebCrawler created?
WebCrawler, the internet’s first crawler used to index full webpages, was the brainchild of Brian Pinkerton. Pinkerton created WebCrawler in 1994 and it originally worked to generate a listing of the top websites. This ranking of 25 websites was created by blending search results from both Google and Yahoo! (now includes Bing! and Ask).
WebCrawler was the first search engine to allow users to search for more than just text. Images, audio, video, news; as well as, the yellow and white pages could also be searched in addition to text like other search engines. WebCrawler also was the first to bring us a search engine for full text searching and originally had its own dedicated database of information.
Unlike many of the search engines today, WebCrawler features a dedicated, separate section to display advertisements to searchers. WebCrawler is still active today; however, it has become a metasearch engine. A metasearch is a search engine that uses data from other search engines to deliver information to internet users rather than a traditional search engine.
While WebCrawler may not be a popular way to perform a web search, it does rank 1,633rd and 3,258th (globally and in the U.S. respectively) according to Alexa traffic ranks in terms of most popular website. There are roughly 644 million active websites according to Netcraft.