Goal: to develop an algorithm which would automatically evaluate the relevancy of search results to a search query entered by a user.Source data: 74 000 search queries on ~55 000 products.Solution: We used all varieties of Text Mining techniques to identify semantic similarity of words and texts and semantic clusterization. The resulting model was built as an ensemble of decision trees.Result: the suggested model was 15% more accurate than the standard method for estimating the relevancy of search results to a search query which was based on keyword comparison.