You are here:
Influence Hybrid Search Relevance Ranking
Increase the accuracy of search index query results and improve the quality of RAG applications by sending relevance-specific parameters. Using SQL, you can send relevance configuration settings for supported fusion ranker models and improve the result relevance for your specific use case.
Hybrid search supports three fusion ranking models: the industry standard Reciprocal Rank Fusion (RRF) model, a customizable Linear Fusion Ranker (LFR) model, and a more robust Deep Fusion Ranker (DFR) model.
A hybrid search query accepts a JSON string that represents the relevance configuration setting.
select * from hybrid_search(
table(<Search_index_DMO>),
‘<Search_string>’,
'<PreFilteringColumn><Operator><Value>',
'<Limit_results>',
‘<”Relevance Ranking Configuration” (JSON)>’
);
Reciprocal Rank Fusion Model
The Reciprocal Rank Fusion (RRF) model uses a reciprocal ranking fusion formula that considers the weight for each search type to get the most relevant results.
The reciprocal rank fusion formula is the sum of both the keyword search (sparse) and vector search (dense) reciprocal rank scores with a default decay of k=60. The formula also provides an alpha parameter, which controls the sparse weight in a convex combination of the two scores. The search results are ranked using the sum derived from the RRF rank fusion formula.
For example, this hybrid search query expression passes the keyword search (sparse) weight parameter for the RRF model.
select * from hybrid_search(
table(<Search_index_DMO>),
‘<Search String>’,
'<PreFilteringColumn><Operator><Value>',
'<Limit Results>',
‘{ "rrf": { "keyword_weight": 0.4 } }’
);
Input parameters for relevance configuration:
- rrf: Enable Reciprocal Ranking Fusion model
- keyword_weight: The importance or weight of the keyword search component in hybrid search. The range is between zero and one. The vector search importance is automatically calculated by deducting the keyword weight from 1.0.
The RRF model also supports an advanced formula that can affect the ranking factors you configure when you create the search index. For more information on ranking factors, see Hybrid Search Fusion Ranking.
Input parameters for relevance configuration in the advanced RRF formula:
- Alpha (a): represents the importance given to keyword search. Default is 0.5f
- Beta (B): represents the importance given to vector search and is calculated based on the importance given to keyword search (1-alpha).
- Gamma (Y):represents the importance of ranking factor popularity. Default is 0.00074478f. Maximum is 1.
- Delta (D):represents the importance given to ranking factor recency. Default is 0.00028472f. Maximum is 1.
Specify the importance of weights in your query. Tweak and refine the values in low increments to make sure you achieve the right relevance for your results.
Linear Fusion Ranking Model
The Linear Fusion Ranking (LFR) model is a customizable form of the default fusion ranker model, which calculates a weighted sum of the transformed scores from both vector search (dense) and keyword search (sparse). The search results are ranked using the sum derived from the LFR formula.
By default, hybrid search provides the best linear fusion ranker configuration using transformations and weights based on internal experiments and benchmarks. With LFR, ranking is further configurable using the transformation functions for the keyword search (sparse) score and vector search (dense) score, and their weights.
The LFR model can also use the recency and popularity ranking factors which can be configured while setting up hybrid search for a DMO. The additional ranking factors are configurable using transformation functions and weights.
For example, this hybrid search query expression passes relevance parameters for the LFR model.
select * from hybrid_search (
table(<Search_index_DMO>),
‘<Search String>’,
'<PreFilteringColumn><Operator><Value>',
'<Limit Results>',
‘{
"lfr": {
"vector": {
"score_transform": {
"type": "reciprocal_rank",
"decay": 60
},
"weight": 0.66913
},
"keyword": {
"score_transform": {
"type": "query_min_max",
"theoretical_min": 0.1
},
"weight": 4.0343633
}
}
}’);Input parameters for relevance configuration:
- lfr - Enable Linear Fusion Ranking model
- vector: Configures ranking for vector search
- keyword: Configures ranking for keyword search
Each of the ranking configurations for vector and keyword search can take these parameters:
- score_transform: The configuration for score transformation for the search component
- type: The type of score transformation
- reciprocal_rank
- query_min_max
- decay: Supported only for Reciprocal Rank Transform, the decay parameter controls
the gradient between result ranks. For example, the reciprocal rank score between results
ranked 1 and 2 are farther apart with a decay of 10 versus a decay of 20.
- If the query includes a decay parameter, then score = 1/(rank+decay).
- theoretical_min: Supported only for Theoretical Min/Max Transform, represents the theoretical minimum score for the underlying search feature. Use this parameter to scale the feature distribution up or down in a range between [0,1]. So choose a value that is the lowest value for the feature to get the best outcome. For the keyword search score, 0 is the appropriate value. Do not tweak this value unless you have data indicating a higher value provides better relevance.
- weight: The importance to give to this component of hybrid search
Deep Fusion Ranker Model
The Deep Fusion Ranker (DFR) model enhances search relevance by integrating traditional signals such as vector similarity, keyword matches, popularity, and recency with a deeper understanding of the query text. By leveraging deep learning techniques and transformer architectures, the model captures complex feature interactions and jointly scores records. The search results are accurate, refined, and contextually relevant. This model effectively handles missing and noisy data, which ensures reliable performance across diverse search scenarios.
Enable the Deep Fusion Ranker model using the dfr relevance configuration setting. There are no other customizations available for this model.
For example, this hybrid search query expression passes the relevance parameter for the Deep Fusion Ranker model.
select * from hybrid_search (
table(<Search_index_DMO>),
‘<Search String>’,
'<PreFilteringColumn><Operator><Value>',
'<Limit Results>)',
‘{ "dfr": {}}’
);
Input parameters for relevance configuration:
- dfr: Enable Deep Fusion Ranker Model
