So, let’s talk about search indexes and how we create them, so split-second searches are possible.

What is a Search Index?

A search index is a regularly, and automatically, updated database of all your products and related information in a simple form. It’s something that enables users on a site to perform fast and accurate searches.

It makes it possible for the search engine to perform any search in microseconds, because it doesn’t have to crawl hundreds or thousands of pages every time, it only has to search in the index.

It’s basically like an index at the end of the book where you can find the most important terms and phrases with the indicated pages where you can learn more about them.

What is a Search Algorithm?

A search algorithm is used in searching for information on the Internet. There are two types of it: deterministic (or exact) and probabilistic (or approximate). Deterministic algorithms return an answer immediately; they do not require any probability calculations. Probabilistic algorithms use some form of calculation to determine how likely a particular result is correct.

Why Search is important in eCommerce – and why is it hard?

In short: because it’s very complex. You have to pay attention to a lot of details: things like providing the right attributes and tags, the natural language your customers use, seasonality, and the design of your product pages. There are many eCommerce search best practices you can and should try out, and fine-tuning is a never ending task.

There is a massive thought process behind every single purchase decision, even if most of it is subconscious, and there are so many little details on your site that can alienate a possible customer.

And this isn’t possible without a proper search index.

Good to know: Searching for business data within a company is very similar to eCommerce search processes, however it requires a completely different approach and solution, so it’s worth getting a line in advance on how to implement effective enterprise search in practice.”

Creation of a Search Index

In the case of books, indexes are created manually, usually by the authors and editors, which makes sense: the information included on a couple hundred pages is comprehensible for a single human, especially if they compiled it in the first place.

Now, if we are talking about websites, especially eCommerce websites, where it is not uncommon that thousands of pages are created for different products, attributes, content and so on – it’s essential that this indexing work is automated.

So, online search indexes are created using algorithms. Crawlers (or web spiders, as they are also known), which are basically automated algorithms, visit every page of the website regularly, scanning and collecting the information on every single one and assigning those to a simple database.

How Indexes Return Search Results

This is (or seems to be) a fairly easy process, in its essence, it isn’t very different from when you hit Ctrl+F in a document or on a website to find a certain word.

Of course, there is more to it, as modern site search engines have to account for typos, misspellings, relevant keywords based on semantics and user behavior, natural language and so on.

But these algorithms sort themselves out, and essentially as a user, you only see the results after you hit search on your query. These results usually include product names or content titles, some attributes or an excerpt, a picture and a price.

The Benefits of a Search Index for Your eCommerce Business

It will improve the user experience and click-through rates of your site search.

First of all, the faster you return relevant results derived from a proper search index, the more satisfied your customers will be.  Today, people expect instant results when typing a search query. A search index allows the search query to be processed without using resources from the server and therefore produces faster results.

And secondly, you can promote search results based on certain factors like relevancy, freshness, ongoing promotions and so on. If you want to learn more about this, have a look at our article on Searchandising Strategies.

Let’s have a look at some basic methods on…

Managing and Adjusting Site Search Results and Ranking

This enables you to define certain parameters, which influences the results shown to your customers when they search.

Utilize Searchandising to Show what is Important

Searchandising in itself is a vast topic, so you should really give a look at our article on it (link in the previous paragraph).

To summarize and simplify, you can boost certain products to appear at the top of the SERP based on how recently they were uploaded, how many are in stock, if there are any promotions connected to them, if there are available bundle offers that are relevant to the query, or any other criteria you want.

Guide Customers with Pinned Search Results

You can even temporarily, or permanently, pin certain results on the top of result pages – like offers in your holiday campaigns or evergreen content

Customers won’t notice the pin as it appears as normal results, however it may become apparent if they run more searches.

Promote Certain Site Areas

You can also direct your customers to certain areas of your site, for example if you want to highlight a certain type of content like holiday gifts or products from a certain category before others.

Personalized Search Results

A big part of enhancing your user experience is giving everyone what they personally need the most. Since we can’t read their minds, we have to prioritize search results for them based on their previous behavior on our site: the products they looked at, the searches they performed, and the purchases they made.

If you provide user-specific results, you can improve overall click-through and conversion rates together with overall user experience.

Analytics: Track Customers and Learn From Them

If you track the data about how people use your site search engine, and fine-tune results accordingly, you have a better chance of giving them what they actually want.

The main metrics to track are:

• how often they use your search,
• what queries they execute,
• and their click-through rate (meaning they actually find something relevant).
• Implementing comprehensive search analytics is incredibly important for optimizing your site.

Search UI and API

A search index, as we clarified, is the memory bank behind the site search function, which improves your user experience on its own.

But you can, in many ways, design your own search UI. For example, you can make sure that your 0 result pages are built in a way that encourages users to keep shopping, or you can use badges on certain results to highlight discounts.

It’s important that your search index is updated regularly and indicates the most recent results. You can do this by including an indexing API in the crawler.

Building a Product Database for a Well-functioning Search Index

The crawlers need data to work from, so it’s your (and your developers’) responsibility to organize your existing data in a way that the bots will understand and be able to translate to results as fast as possible.

This is why you should pay attention to…

Cleaning Your Data

Creating clean data for the search index (data cleansing) is the process of making sure that your data is actually valid. This can become especially problematic if you have a great number of products and the data that you get from the manufacturer is not compatible with your database.

Use Convertible Units of Measurement

Your data has to be uniform and in convertible measures, like from centimeters to millimeters, or centimeters to inches.

Imagine that you get screen sizes for TVs or smartphones from the manufacturer in millimeters, but on your product page you have to display those values in inches, because your customers are used to that measurement with these products. By having uniform and easily convertible units of measurement, you save yourself a lot of trouble.

Include Inventory Stock Data

Your stock should be continuously updated in your online store for multiple reasons. This helps you avoid situations in which you’re actively selling a product you don’t have in stock, which means your customers would have to cancel their orders. Or the flip side, your inventory could be piling up, but no orders are coming through.

Stock data can also indicate whether you should give more prominence to a product in the search results via Searchandising or push it back when stocks are low.

Consider the Product Variants

Your search index must be capable of handling different variants.. There are many instances in which you’re selling products that are similar, but have slight variations – shirts with different size and color, phone cases with different art, smartphones with different memory sizes, etc.

These variants have a tendency to appear very quickly – let’s stay with shirts for a moment and say that you have a T-shirt in 4 colors and 4 sizes. That is already 16 variants for the same product. So make sure that everything is in order in your database – and test it in your search solution, to ensure shoppers can find the exact product they want as easily, and quickly, as possible.

Deduplicate Products

If there are hundreds, or thousands, of products in your store, which is common, especially if you take product variations into account (e.g. different sizes, colors), it is very likely that duplications will occur. These sometimes will go unnoticed, for example if the same product is uploaded into different categories.

These duplications need to be found and eliminated. This is most easily done by running an automated algorithm to check photos, product descriptions and attributes.

Support Multi-language eCommerce

Finally, if your target market is wider than one country, you should have dedicated fields in your product database for every language and pay close attention to filling them in correctly. This enables each of your customers to use their own language and find the products they’re looking for.

Also look for an enterprise ecommerce search provider that has a built-in language identification feature.

What to Pay Attention When Building Your On-Site Search

Search Bar and What You Indicate With it

When considering your search bar, the thing you want to pay attention to is visibility: users should be able find it right away and understand how to use it.

This is why it can be beneficial to put text in the bar itself. For example, some stores allow people to search by article number, but in others this returns no results. This is why you have to let shoppers know how they can use your search bar.

Learn more about the most important search autocomplete features to decrease zero-result pages by 31% and increase cart value by 19%.

The Redirect Function

Your search solution could include a redirect function where certain queries take the user to a landing page or category page directly.

Without getting into the technical details, this is something your developers would have to look into – but determining the pages where you want to redirect users is yours.

Search-as-you-type Function

If your index is built the right way, the site search solution should be able to suggest queries in real-time – you can see this every time you use Google. Their solution is a mix of frequently used terms (user behavior) and terms that show up frequently in the index. You should follow a similar path.

Title of your sub-chapter

Based on user behavior and some custom product tags, you can provide cross/upsell opportunities on your result pages.

You could order results in a way that only a certain number of results are directly related to the query itself, and the rest are related products or bundled offers.

Faceted Search in Your Store

In order for faceted search to work properly, this detailed product database is essential, because results are edited in real-time based on the filters selected (which also need to update based on available products). These search filter options are based on the various detailed attributes in your database.

We could go on for an entire article about this topic, and we already have: if you are interested about it, check out our guide on faceted search!

Your Product Pages are Important

The way you structure your product pages greatly impacts how effectively the crawler is able to gather information. To get the best results, your product pages should be uniform and fairly simple when displaying details and data like sizes, colors, etc.

Use Analytics to Learn About Your Search Index Performance

Analytics can say a lot about the health of your search index. Some of the basic metrics that can indicate a problem are:

• Loading time of your result pages
• The percentage of no result pages
• Conversion rates of your result pages

Conclusion

As you can see, it’s important that your search index stays up-to-date and is full of well-structured data in order to work properly. When you have a high-quality search index, you can provide the best possible search and customer experience via providing fast results.

A lot of what goes into a great search engine is very, very technical and complex – but some decisions cannot be made by developers alone. This is why it is important to always work together across units – and for you to pay close attention to the details.

• Make sure your databases are compatible with each other
• Make sure all information and data is up-to-date
• Create pages where a crawler can easily find information
• Design your result pages so they don’t only load faster, but also provide more opportunities for conversion

And you won’t have any problems!