Algolia launches hyper-scalable vector and keyword search API

Bernadette Nixon, Chief Executive Officer, Algolia
Bernadette Nixon, Chief Executive Officer at Algolia

Algolia launched Algolia NeuralSearch™, a next-generation vector and keyword search in a single API with powerful, end-to-end AI processing every query. Algolia NeuralSearch understands natural language and delivers highly accurate and relevant results in milliseconds.

What does Algolia NeuralSearch bring to the market?

This technology is a breakthrough in search and discovery that promises to revolutionize the way individuals engage with content online or in apps. Algolia NeuralSearch delivers superior conversions and increased revenue at enterprise scale for huge production workloads.

It uses advanced Large Language Models (LLM) – the same technology underpinning ChatGPT and generative AI – and goes a step further with Algolia’s Neural Hashing™ for hyper-scale and constantly learns from user interactions for better results.

Bernadette Nixon, Chief Executive Officer, Algolia said: “Algolia is committed to advancing AI-powered Search, and we believe Algolia NeuralSearch does just that. It’s a first-of-its-kind hybrid search product, provides users with a smarter and more intuitive way to discover the most relevant content, when they need it, irrespective of the type of query presented.”

“Importantly, we make it easy to achieve live production quickly—specifically, we provide the set-up, scaling, and management of all search capabilities and services—all of which helps accelerate and power discovery. Moreover, Algolia NeuralSearch is backward compatible, which means there is zero engineering required for customers to become AI-enabled.”

“Industry-wide, retailers are leaving a significant potential revenue on the table because it’s challenging to capture revenue from ‘long tail’ search queries (like ‘stunning fall outfit for mother of the bride’), which could represent up to 55% of all search queries,” added Nixon.

“These low volume searches could collectively amount to millions of queries corresponding to billions of dollars in unfulfilled sales of less popular or searched for products. Algolia NeuralSearch optimizes for all queries, popular or infrequently searched for, while using specific keywords or natural free-form expression—truly putting search on autopilot at a price point that is 90% less than other vector-based search options,” Nixon further commented.

How does Algolia NeuralSearch work?

Algolia NeuralSearch analyzes the relationships between words and concepts, generating vector representations that capture their meaning in an abstract and contextual manner.

Because vector-based understanding and retrieval is combined with Algolia’s innovative and award-winning full-text keyword engine, it works for exact matching too. Algolia NeuralSearch uniquely addresses an industry-wide problem with vector search: the inherent limitation to scale and the high burden of costs associated with using specialized computers.

To solve this problem, Algolia pioneered Neural Hashing, which compresses these search vectors from 2,000 decimal long numbers into static length expressions making computing them very fast and significantly more economical. Prior to Algolia’s proprietary breakthrough, vector-based search has been too computationally expensive to run in production.

Algolia is the only company in the world that incorporates artificial intelligence (AI) across three primary functions—query understanding, query retrieval, and ranking of results.

  • Query understanding – Algolia’s advanced natural language understanding (NLU) and AI-driven vector search provide free-form natural language expression understanding and AI-powered query categorization that prepares and structures a query for analysis. Moreover, Adaptive Learning based on user feedback fine-tunes intent understanding.
  • Retrieval – The most relevant results are then retrieved and ranked from most to least relevant. The retrieval process merges the Neural Hashing results in parallel with keywords using the same index for easy retrieval and ranking. This approach solves the ‘null results’ problem and significantly improves click positions and click-through rates. No other search platform in the search and discovery space offers this powerful capability.
  • Ranking – Finally, the best results are pushed to the top by Algolia’s AI-powered Re-ranking, which takes into account the many signals attached to the search query, (including the exact keyword matching score, the contextual personalization profile, the observed popularity of items, the semantic matching score, etc.) and learns to reach maximum relevance.

Most organizations only have the resources to optimize their search for a few popular queries. However, this leaves a significant amount of potential revenue on the table.

As the index changes, new products are added, new content is uploaded, or as terms take on new meaning, the Algolia NeuralSearch will adjust automatically. It doesn’t require any additional headcount or manual operations. It will match keywords or concepts—possibly a mix of both—depending on the query or search phrase. This truly puts search on autopilot.

What does Algolia NeuralSearch mean for retailers?

Commenting on the solution, Guillermo Romero, Director Enterprise Architecture at Best Buy Canada said: “Algolia’s NeuralSearch technology will help us better understand customer intent to improve our search relevance across our extensive, rich product catalog. We’re excited to partner with Algolia to integrate this next generation vector and keyword search technology and create a better search and discovery experience for our customers.”

Frasers Group, which encompasses an array of fashion apparel brands serving unique audiences, was among the first Algolia customers to use Algolia NeuralSearch in a real world environment. Kyle Sanders, Head of Digital Optimisation, Frasers Group, said: “We tested Algolia NeuralSearch with two of our brands (Missguided and Isawitfirst), and were thrilled to see above ~65% drop in zero search results and up to 17% uplift in conversion rates.”

“These results, despite only sending a portion of our query traffic to Algolia NeuralSearch over a four-week period, exceeded our expectations. Notably, our existing search implementation seamlessly evolved to further improve our customers’ discovery journey and improve their experience on our website—all without having to make any changes to a single line of code. We are excited to see what the future holds with Algolia NeuralSearch,” Kyle Sanders said.

Hayley Sutherland, Research Manager, IDC said: “By adding Neural Hashing of vectors to its keyword-based search within a single index, leveraging a single API, Algolia has the potential to disrupt AI-powered search with better precision and recall, in a manner that requires less manual work to set up and update, while incurring fewer storage and processing costs.”

Sutherland said: “This evolution from search to discovery through vector search is significant for ecommerce and retail due to implications for product discovery use cases. In retail, long-tail searches—that is, less commonly used search terms that may not find exact keyword matches and return null results when queried — represent lost revenue when they return null results, forcing potential clients to abandon searches and take their business elsewhere.”

“Vector search has become popular due to its ability to provide clients with similar or related products when an exact match is not found, allowing customers to find relevant results using free-form natural language and helping to ensure their revenue does not go to competitors.”

Rachel Maxwell, Digital Merchandising Manager at Everlane said: “When we implemented the AI-powered Algolia NeuralSearch solution, the overall results have been amazing and have included a 9% increase in clickthrough rates and a 9% increase in the conversion rate. We are also finding our merchandisers are spending less time on manual tasks such as creating synonyms to optimize search results, and more time on more strategic work.”

Algolia NeuralSearch is available now; more information here.