Ecommerce Search Best Practices Using Learn-to-Rank Technology

Your website’s search bar is one of your most powerful revenue generators. McKinsey estimates that organizations that excel at search personalization generate 40% more revenue than their average competitors. We, alongside our partners at Infinite Lambda, also witnessed how one of our customers achieved an incremental £2m in annual profit directly from search personalization.

How did they do this? Through a Learn-to-Rank (LTR) framework—a process that reorders search results based on individual preferences, significantly increasing engagement and conversion likelihood.

In this blog, we’ll share proven ecommerce search best practices using the LTR methodology. The insights shared are based on real-world experiences working with leading travel and online retail companies.

Exploring the Business Case for Advanced Search

Social proof company WiserNotify estimates that personalized search can increase conversion rates by up to 50%. Businesses that have implemented advanced personalization techniques are seeing returns as high as $20 for every $1 invested (Bloomreach).

Contrast this with traditional search functionality, and you see a different picture. When a user visits your site and uses your search feature, they’re telling you what they want. As a result, these users are extremely valuable. According to Forrester, 43% of website visitors go straight to the internal search bar when they land on a site—and these search users convert at almost double the rate of those who don’t use search.

The problem though is that your search engines are only as good as the results they spit out. If we look at standard faceted search, users are only given part of what they’re looking for. They then have to manually filter through the options to find what they were after. 

LTR, on the other hand, modernizes this experience. It removes the need for users to filter through search results and provides them with the most relevant products based on their individual behavior and preferences. 

In effect, LTR does not just improve your search engine. It provides you with a sophisticated sales tool that truly understands your customers and points them towards the products or services they’re most likely to purchase. As a result, organizations that have implemented LTR have driven significant improvements in conversion rates and revenue. 

Understanding the Search Problem

We all know the scenario. You’re on an ecommerce site. You search for what you’re looking for and you’re then presented with a vast array of products. This makes it difficult to find the option that best suits your needs. If users can’t find what they’re looking for, this can obviously have a negative impact on the bottom line for ecommerce companies. 

Let’s run through a more detailed practical example: you visit a travel site. You search for accommodations in London and you’re presented with hundreds of options. If the site uses traditional faceted search, you’ll be able to filter by price range, amenities, or proximity to landmarks. But the issue here is that this type of search fails to learn from collective user behavior to surface the most relevant options. 

This abundance of choice leads to several negative issues for ecommerce companies. In the first instance, you’re likely to become overwhelmed with the sheer volume of options. You’ll grow frustrated when you can’t quickly identify what you’re looking for. These factors combined then result in a lower engagement and reduced conversion rates for the ecommerce vendor. 

So, ecommerce companies face a fundamental problem here around relevance and personalization. Each of your visitors have unique wants and needs, but standard search functionality treats all your users the same. Without personalized search, the most relevant items to a specific user may appear on page five of the results—where they’re unlikely to ever be seen. 

Therefore, the ambition for ecommerce companies has to be around reordering search results to match individual user preferences based on their behavior and context. Organizations that have achieved this have skyrocketed their engagement and conversion rates. In industries like the travel sector where there are numerous similar options with subtle differences, this type of search is revolutionary in influencing purchasing decisions. 

The Learn-to-Rank Solution Overview 

This is where Learn-to-Rank (LTR) search comes in. LTR is a machine learning approach that transforms how your search results are ranked, creating personalized experiences that boost conversion rates. Unlike basic keyword matching, LTR models learn from user behavior to determine what makes a search result relevant to individual users. 

This is exactly what Infinite Lambda built for one of our joint travel customers. Their model firstly ranks hotels based on keyword searches. It then uses Snowplow data to personalize the results. And to increase shared learning and effectiveness, the model works across multiple geographies. Here are the six components that make up the travel company’s LTR solution: 

1. Data Collection: The travel company had implemented Snowplow Customer Data Infrastructure for data collection. This means they can collect detailed behavioral data and events including search queries, impressions, clicks, dwell time, conversions, and user context. Snowplow also delivers this data in real time to the customer’s data warehouse, giving them complete data ownership and the foundations for AI-driven personalization. 
2. Data Processing: Next up, the Infinite Lambda team helped the customer to prepare their data into two lists: Judgment Lists and Feature Sets. The Judgment lists help the model to understand how good different search results are for specific search terms. While the Feature Sets provide the model with insights into which features matter most to different users. 
3. Feature Engineering: To help the model identify the most relevant hotels to each visitor, Infinite Lambda transformed the raw Snowplow data into specific patterns, including:  

• Content-based features (keyword matches, semantic similarities)
• Collaborative features (user-item interaction histories)
• Contextual features (time of day, user location, device type)
• Behavioral features (engagement metrics, recency of interactions)

4. Model Training: To train the LTR model, Infinite Lambda used the XGBoost algorithm, which is useful for structured data and doesn’t require extensive pre-processing.  
5. Deployment: Once the model had enough intelligence, the team connected it to  Elasticsearch with an LTR plugin. This integration enables real-time personalized results.
6. Evaluation and Optimization: Finally, the customer uses metrics like Normalized Discounted Cumulative Gain (NDCG) to assess performance. In addition, they conduct A/B testing to understand the model’s impact on users and revenue. 

This may seem like a lot of work, but it’s absolutely necessary to deliver a true personalized search engine. For the travel company, this investment has dramatically improved its users’ search experience and this is now translating into revenue gains. 

Essential Ecommerce Search Best Practices 

When implementing an effective LTR solution, it’s important to consider several critical best practices. The guidelines we outline below are drawn from our experience working with a leading travel company and are designed to help you maximize the effectiveness of your ecommerce search personalization. Let’s dive in. 

1. Collect Granular User Interaction Data

As mentioned in the example of the travel company, all successful LTR models must be fed with high quality behavioral data. We’re all familiar with the “garbage in, garbage out” adage now. The same applies here. You can use tools like Snowplow to collect the necessary granular events for your model such as: 

• Search Queries: The specific terms users enter into the search bar
• Impressions: The list of results displayed to users after a query
• Clicks: Which results users interact with by clicking
• Dwell Time: How long users spend engaging with content
• Conversions: Actions indicating success (purchases, bookings)
• User Context: Metadata like user ID, session ID, and device type

The richer and more granular your behavioral data is, the better your personalized search experiences will become. That’s why a tool like Snowplow is so valuable compared to other solutions like Google Analytics and packaged customer data platforms.  

2. Create Your Judgment Lists

Once you have your data collection foundation in place, it's time to construct your Judgement Lists. You can build these by: 

• Extracting search sessions from your data warehouse
• Assigning relevance grades based on user interactions (e.g., Grade 4 for conversions, Grade 3 for high intent, Grade 2 for clicks)
• Automating updates to keep Judgment Lists current

‍

3. Develop Comprehensive Feature Sets

Feature Sets are collections of attributes that serve as input to your LTR model. These include:

• Content-Based Features: Keyword matches in titles and descriptions
• Collaborative Features: User-item interaction histories
• Contextual Features: Time of day, user location, device type
• Behavioral Features: Engagement metrics, recency of interactions

For optimal results, you want to categorize features and evaluate their impact on performance through careful testing.

4. Choose the Right Machine Learning Algorithm

Next up, you’ll need to select algorithms that efficiently handle structured data with minimal pre-processing Here are some options:

• XGBoost: Efficient for gradient-boosted decision trees and widely effective in LTR scenarios
• LightGBM: Optimized for speed and performance
• TensorFlow Ranking: For deep learning approaches

We found XGBoost is particularly effective as it can handle a wide range of features with different scales and ranges while running efficiently on standard hardware.

5. Implement Effective Error Tolerance and Synonym Management

Users often misspell queries or use different terminology than your catalog. Therefore, you need to ensure your search solution:

• Handles typos and misspellings
• Recognizes synonyms (e.g., "t-shirt" vs. "tee shirt")
• Supports natural language processing to understand query intent

6. Optimize Training and Evaluation Processes

When you’re ready to develop your model, we found these steps useful to follow: 

• Split your data into training, validation, and test sets (80/10/10 split)
• Use Normalized Discounted Cumulative Gain (NDCG) to assess ranking quality
• Implement regular retraining to keep your model current with changing user preferences

7. Deploy with Real-Time Feature Access

Certain feature values must be available at inference time for personalization:

• Use feature stores (like Redis) for low-latency access
• Ensure user behavior data can be accessed in real time
• Store necessary features within the search index

8. Implement Continuous Monitoring and A/B Testing

Finally, you must validate your model's effectiveness through:

• A/B testing with clearly defined success metrics
• Continuous monitoring of key performance indicators
• Guardrail metrics to ensure changes don't negatively impact user experience

‍

‍

We cover these best practices in more detail in our technical guide. When followed properly, these steps can allow you to build a search personalization system that improves your user experience, increase conversions, and deliver an uptick in ROI for your business. 

Technical Implementation Insights 

Now, what’s the best architecture for your Learn-to-Rank implementation to succeed? Below, we outline the exact architecture the travel company uses for its LTR model. 

Whatever architecture you opt for, you need to ensure it can support a scalable, data-driven system that optimizes search results based on user behavior. 

In our example, the customer uses Snowplow for event tracking, AWS S3 and Snowflake for scalable data storage, and transformation tools like dbt or Snowpark. This makes it easy for them to gain access to high-quality data for model training. 

If you prefer SQL, dbt offers a great framework for data transformation. If you’re more Python-focused, you’ll probably find Snowpark offers more flexibility. 

Going back to our example, the travel company uses Apache Airflow for workflow orchestration. It also uses AWS SageMaker to manage its LTR model training. 

Using this example, we can summarize that the best technical stack for Learn-to-Rank implementation is as follows: 

• Event Tracking: Snowplow for collecting user interactions
• Data Storage: AWS S3 and Snowflake for scalable storage
• Data Transformation: dbt and/or Snowpark for data modeling
• Orchestration: Apache Airflow for scheduling workflows
• Model Training: AWS SageMaker or local environments
• Search Platform: Elasticsearch/OpenSearch with LTR plugin
• CI/CD Pipeline: Jenkins or GitHub Actions for deployment

‍

‍

It’s important to note here that your trained models must be integrated into your search platform using appropriate plugins. If we take OpenSearch’s LTR plugin, for example, you can use this to upload models and define scoring features that determine how search results are ranked. 

With this tech stack, you can ensure you have high-quality data flowing through your system. It allows for continuous model improvement and is proven to deliver personalized search experiences.

Conclusion and Next Steps 

By following the steps outlined above and within our technical guide, you can deliver Learn-to-Rank search for your business. Not only will this enhance your user experience, but it will boost conversions and drive up revenue. 

This article gives you the high level view to help you get started. If you want the complete implementation guide along with code examples, detailed architecture, and step-by-step implementation strategies, download our technical guide

Or, if you’d like to discuss this approach with one of our team, contact us here. Good luck!