How search engines work

Hi there. Let's discover more about how search engines work. We'll give you the basics on how search engines find web pages, what they do with the pages they find and how they decide what results to show.

When you're using a search engine to find the closest coffee shop, you're probably not thinking about the technology behind it all. But later you might wonder how did that search engine do that? How did it sort through the entire Internet so quickly and choose the result you saw on the page?

Each search engine uses its own software program, but they all work in a similar way. They all perform three basic tasks. First, they examine the content they learn about and have permission to see; that's called crawling.

Second, they categorize each piece of content; that's called indexing. And, third, they decide which content is most useful to the searchers; that's called ranking. Let's take a closer look at how these work.

Search engines crawl the internet to discover content, like web pages, images and videos. Each search engine uses computer programs to make their way through the pages. These are known as crawlers, spiders or bots, which is short for robot.

The bots hop from page to page by following links to other pages. These bots never stop. Their sole purpose is to visit and revisit pages, looking for new links and new content to include in the index.

Indexing is the second part of the process. The index is a gigantic list of all the web pages and content found by the bots. The search engine uses this index as a source of information displayed on the search results pages, but not everything the bot finds makes it onto a search engine's index.

Search engines may find multiple copies of the exact same piece of content located on different websites. Is that even possible? Well, here's an example.

Imagine you're not searching for a coffee shop but a coffee maker. You might notice that the top-of-the-line Coffee King 2000 has the same word-for-word description on the websites of many major retailers. The description might have been provided by the manufacturer, but now the search engine has a decision to make as to which version to keep in the index.

There's no need for hundreds of duplicates so it's unlikely that every page will be added. So what if you own a website that's selling coffee makers? You're likely better off writing your own description of the Coffee King 2000.

Makes sense? Right, that covers crawling and indexing, which just leaves us with ranking. Think about what happens after you type in a search.

The search engine compares the words and phrases to its index and looks for matching results. But what if it brings up hundreds of millions of matching results? This is where its next important task kicks in: ranking.

The way search engines rank pages is top secret. It's kind of their special source. There are literally hundreds of ways search engines determine rank, including things like words on the page, the number of other websites linking to it and the freshness of the content.

But no matter what formula they use to determine rank, the goal remains the same to try and connect the searcher with what they are looking for. Say you've read about an Australian style cappuccino called a flat white, and you want to try it. If you search for "flat white coffee near me," the search engine will show you nearby shops selling the drink because your search indicated your location.

You might even see a map to help you find them. So just to remind you, search engines are constantly working to scour the web for content, organize it and then display the most relevant results to searchers. Understanding how this process works can come in really useful to your business.

as you try to climb higher in those important search results.