<a href="https://www.doofinder.com/en/">Doofinder </a>is a powerful, quick, and intuitive search engine that will help you to improve your online store’s sales. Magently is responsible for developing the Magento 2 module (and our sister brand <a href="https://chop-chop.org/">Chop-Chop</a> developed the WooCommerce plugin). It communicates between the Doofinder API and Magento. 

The biggest challenge about this project was to make the Doofinder module a fully-fledged Magento Search Engine and this will be the focus of this case study. 

Magento Search Engine is a complex logic responsible for providing search results on category and search pages. Whenever you visit a category page and filter the results, the Search Engine works in the background and decides what products should be displayed. 

In earlier versions, the Magento Search Engine was MySQL. It browsed products in the database. Magento 2.3 introduced ElasticSearch that is substantially quicker than MySQL. The solution was so efficient that Magento proceeded to deprecate MySQL leaving ElasticSearch as the only natively available Search Engine in Magento 2.4.

The Doofinder module adds Doofinder as an alternative Search Engine to be responsible for the search results. 

As mentioned before, Magently is responsible for building the module from scratch. Initially, it was a hybrid based on MySQL and only worked on the search results page. 

Along with the release of Magento 2.3, the Doofinder team decided to extend the functionality to work on the category pages, too. It required rewriting the module logic almost from scratch to integrate it with Magento Search Engine logic. 

At Magently, we had no prior experience creating new Magento 2 Search Engines. Before we started coding, we performed thorough research to fully understand the way it works. While we studied existing Search Engines, we experimented and outlined the steps to move forward. 

A Search Engine consists of two major functionalities:

Indexing products takes all products available in the store and sends them to the proper place - in our case, it’s the Doofinder API. Before refactoring the module, the indexer had already existed but it was only responsible for the data required by Doofinder. The new indexer had to index all attributes that Magento would use to filter the results. 

The module fetches products on entering the category page or searching for products. Initially, Doofinder would provide product IDs based on a searched phrase and MySQL would build a proper set of products. After our refactoring, the module returns all required data and bypasses MySQL when it passes it to build a category or a search results page. 

The biggest challenge that we faced were frequent Magento updates, especially from version 2.3.2. up. Magento had gone through many changes at that point and some of the features we implemented stopped working at some point. What’s worse, fixing them caused issues in older versions. Once Magento 2.3.5 was released, together with Doofinder we decided to drop support for versions older than 2.2.6.

As a solution to go forwards, we created a compatibility module that took over some of the functions. It is supplied in a different version depending on the Magento 2 version. From then on we decided that the core module contains functionalities available for all versions and the compatibility module only supplies features required for it to work with a specific version. 

The next two problems were related to the indexer. 

In Magento 2, the indexer works in two modes: on Schedule and on Save. “On Schedule” is based on cron that periodically runs the indexer. In our case, it deletes the old products’ index and creates it from scratch. In the original implementation, this caused downtime in loading category pages because the old products were gone by the time the new ones were fetched. It was solved by a new Doofinder API version that provides a temporary index. It is created first and then it’s replaced by the main index, hence avoiding the downtime. 

The second problem occurred whenever a store indexed products on save. For whatever reason, Magento calls for products’ index update three or four times on that occasion. If working with multiple store views, it can result in dozens of such requests at a time. And if we add bulk products’ updates, then API can receive large amounts of requests that not only slow it down but also slow down Magento. We solved this problem with Delayed Updates. The module keeps track of products that are updated and saves them to the database. Next, using cron, it collects the information and updates the Doofinder products only once, limiting the number of requests to just a few - only when they are actually needed. 

the module is complex but not over-complicated. We know how it works and every feature has its purpose. 

we became proficient with Magento Search Engines. 

The ever-changing Magento environment is a real challenge for module creators. Along with new Doofinder updates, Magently regularly faces even more challenges as we thrive to keep the module bug-free and fully functional.

You can also read more about this Case Study <a href="https://magently.com/case-study/doofinder-search/">here</a>. 