Search pipeline
Note
By default, Configured Commerce uses Elasticsearch version 5.5 (Nest5) for search functionality on customer storefronts. The following search pipelines are applicable only for Elasticsearch version 5.5 (Nest5), if you are upgrading to or on Elasticsearch version 7.10 (Nest7), refer Third party search pipelines.
CategoryIndexerPipeline
CreateElasticsearchCategory
| Order | Pipe | Description |
|---|---|---|
| 100 | CreateElasticsearchCategory | Create a new ElasticsearchCategory object using the parameter object's fields and assigns it to result.ElasticsearchCategory. |
GetIndexableCategories
| Order | Pipe | Description |
|---|---|---|
| 100 | GetIndexableCategoriesQuery | Creates an Entity Framework Category query using the parameter object's fields and assigns it to result.IndexableCategories. |
MapElasticSearchCategoryProperty
| Order | Pipe | Description |
|---|---|---|
| 100 | MapElasticSearchCategoryCustomProperties | If the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty. |
CategorySearchPipeline
RunCategorySearch
| Order | Pipe | Description |
|---|---|---|
| 100 | InitializeSearchProperties | Retrieves the current SiteContext, ElasticsearchIndex and ElasticsearchQueryBuilder and assigns them to the result object. |
| 200 | GetSettings | Retrieves SearchGeneralSettings and FuzzySearchSettings and assigns them to the result object. |
| 300 | GetQueryFields | Assigns the result.CategorySearchFields object with the fields that are queryable. |
| 400 | FormPaging | Assigns result.StartRow and result.PageSize from the parameter object. |
| 500 | FormCategoryExcludeList | Finds the disallowed categories for the current user (using the FormProductFilter pipeline) and caches this value. The value is then assigned to result.DisallowedCategories. |
| 600 | FormCategoryExcludeFilter | If there are any disallowedCategories adds a new negated Terms query to result.AllFilters using result.DisallowedCategories on the categoryId field. |
| 700 | FormLanguageFilter | Adds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters. |
| 800 | FormWebsiteFilter | Adds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters. |
| 900 | CombineFilters | Creates a BooleanQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter. |
| 1000 | FormMainSearchQuery | Creates a MultiMatchQuery of type CrossFields using parameter.Criteria and result.CategorySearchFields and apply boosts. Creates a MultimatchQuery of type PhrasePrefix using parameter.Criteria and result.CategorySearchFields and apply boosts. Adds both of these queries to result.AllQueries. |
| 1100 | FormFuzzySearchQuerry | If fuzzy search is enabled, creates a MultiMatchQuery with fuzziness set to FuzzySearchSettings.MaxEdits using parameter.Criteria and result.CategorySearchFields and apply boosts. Adds this query to result.AllQueries. |
| 1200 | CombineQueries | Creates a BooleanQuery (OR) of all queries in result.AllQueries and assigns it to result.CombinedQuery. |
| 1300 | ApplyQueryAndFilter | Creates a SearchDescriptor with a FunctionScore query using result.CombinedQuery and result.CombinedFilter, with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor. |
| 1400 | ApplyPaging | Sets result.SearchDescriptor's From and Size variables using result.StartRow and result.PageSize. |
| 1500 | ApplySortOrder | Sets result.SearchDescriptor's sort to sort on the sortOrder field in descending order then on _score in descending order. |
| 1600 | PerformSearch | Uses result.ElasticsearchIndex.Client to call Elasticsearch using result.SearchDescriptor and assigns the result to result.SearchResponse. |
| 1700 | ProcessSearchResponse | If result.SearchResponse is not null and is valid, creates a new CategorySearchResult with the search result count and an IEnumerable of CategorySearchResultDto objects using result.SearchResponse.Documents then assigns this to result.CategorySearchResult. |
ContentIndexerPipeline
CreateElasticsearchContent
| Order | Pipe | Description |
|---|---|---|
| 100 | CreateElasticsearchContent | Creates a new ElasticsearchContent using the parameter object's fields and assigns it to result.ElasticsearchContent. Fields that vary based on language, persona are populated by filtering parameter.IndexableContentItem.Fields for the field with the appropriate context. |
GetIndexableContentItems
| Order | Pipe | Description |
|---|---|---|
| 100 | GetIndexableContentItemQuery | Creates an Entity Framework IndexableContentItem query using the parameter object's fields and assigns it to result.IndexableContentItems. |
ContentSearchPipeline
RunContentSearch
| Order | Pipe | Description |
|---|---|---|
| 100 | ValidatePageSize | Sets result.PageSize based on parameter.PageSize, if it is less than or equal to zero set to 10, if it is greater than 1000 set to 1000 or else set to parameter.PageSize. |
| 200 | FormQuery | Creates a BooleanQuery (OR) of two MultiMatchQuery queries for the content fields (title, body, metaDescription, and metaKeywords) one of type CrossFields and one of type PhrasePrefix and assigns it to result.Query. |
| 300 | FormFilter | Creates a BooleanQuery (AND) of TermQueries to filter on website, language, deviceType, persona and not restricted pages and assigns it to result.Filter. |
| 400 | PerformSearch | Creates a new SearchRequest using result.Query, result.Filter and result.PageSize, then performs the search and assigns the response to result.SearchResponse. |
| 500 | ProcessSearchResponse | Creates an IEnumerable of ContentSearchResultsDto from the result.SearchResponse.Document fields (title, pageType, and URL) and assigns it to result.ContentSearchResults. |
ProductIndexerPipeline
CreateElasticsearchProduct
| Order | Pipe | Description |
|---|---|---|
| 100 | CreateElasticsearchProduct | Create a new ElasticsearchProduct object using the parameter object's fields and assigns it to result.ElasticsearchProduct. |
| 200 | SetSpellingCorrection | Joins ElasticseachProduct fields (ShortDescription, Content, ManufacturuerItem, Name, Specifications, ErpNumber, CategoryNames, and FilterNames) into a string and assigns it to result.ElasticsearchProduct.SpellingCorrection. |
GetIndexableProducts
| Order | Pipe | Description |
|---|---|---|
| 100 | GetIndexableProductsSqlStatementParts | Creates a string with the Optimizely default SQL query to get the correct products and fields from the database and assigns it to result.SqlStatement.Assigns result.IncremenalFilter with the proper SQL snippet if parameter.IsIncremental is true. |
| 200 | CombineIndexableProductsSqlStatementParts | Combines result.SqlStatement, result.CustomFields, and result.IncrementalFilter and assigns it to result.FormattedSqlStatement. |
| 300 | PerformIndexableProductsSqlQuery | Gets an IEnumerable using the SQL query from result.FormattedSqlStatement and assigns it to result.IndexableProducts. |
MapElasticSearchProductProperty
| Order | Pipe | Description |
|---|---|---|
| 100 | MapElasticSearchProductCustomProperties | If the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty. |
ProductSearchPipeline
FormProductFilter
| Order | Pipe | Description |
|---|---|---|
| 100 | GetSettings | Retrieves SearchGeneralSettings and ProductRestrictionsSettings and assigns them to the result object. |
| 200 | FormWebsiteFilter | Adds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters and assigns it to result.WebsiteFilter. |
| 300 | FormLanguageFilter | Adds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters and assigns it to result.LanguageFilter. |
| 400 | FormCategoryFilter | Creates a new PrefixQuery to search a single category or the whole category tree based on parameter.ProductSearchParameter fields (SearchCriteria and IncludeSubcategories) then adds it to result.AllFilters and assigns it to result.CategoryFilter. |
| 500 | FormRestrictionGroupFilter | If Restriction Groups are enabled, uses the ProductSearchPipeline's FormRestrictionGroupFilter pipe to generate a new filter then adds the filter to result.AllFilters and assigns it to result.RestrictionGroupFilter. |
| 600 | FormAttributeValueFilter | Creates PrefixQueries for attributes based on parameter.ProductSearchParameter.AttributeValueIds and combines them into a BooleanQuery. That BooleanQuery is added to result.AllFilters and assigned to result.AttributeValueFilter. |
| 700 | FormAllowedProductsFilter | Creates MatchQuery objects for each product in parameter.ProductSearchParameter.AllowedProductIds and combines them into a BooleanQuery. That BooleanQuery is add to result.AllFilters and assigned to result.AllowedProductsFilter. |
| 800 | FormPriceRangeFilter | If parameter.ProductSearchParameter.PriceFilters has any values a new MatchQuery is created for each filter and combined into a BooleanQuery (OR) then assigned to result.PriceRangeFilter and added to result.AllFilters. If parameter.ProductSearchParameter.PriceFilters has no values and parameter.ProductSearchParameter.MinimumPrice or parameter.ProductSearchParameter.MaximumPrice have been supplied a new Range query is created using the supplied min/max values with defaults of 0 and 1000000 if either was omitted. That Range query is then added to result.AllFilters and assigned to result.PriceRangeFilter. If no filters were supplied and neither the minimumPrice or maximumPrice were supplied nothing is done and the pipe returns. |
| 900 | CombineFilters | Creates a BoolQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter. |
FormRestrictionGroupFilter
| Order | Pipe | Description |
|---|---|---|
| 100 | GetRestrictionGroupFilterFromCache | Attempts to retrieve the restriction group filter from cache based on the current BillTo/ShipTo and assigns it to result.RestrictionGroupFilter. If there was a cached filter the pipeline exits. if there was no cached filter the pipeline continues to the next pipe. |
| 200 | GetDefaultVisibilityFilters | Creates a new MatchQuery on the restrictionGroups field to match products with no restriction groups and adds it to result.DefaultVisibilityFilters. Creates a Terms query on the defaultVisibility field with values of empty and show then adds it to result.DefaultVisibilityFilters. |
| 300 | GetWebsiteRestrictionGroupQuery | Creates an EF query to get the active restriction groups based on the current website and assigns it to result.WebsiteRestrictionGroupQuery. |
| 400 | GetWebsiteRestrictionGroups | Uses result.WebsiteRestrictionGroupQuery to get active restriction groups then assigns them to result.WebsiteRestrictionGroups. If there are any active restriction groups a BoolQuery (AND) is created from result.DefaultVisibilityFilters and assigned to result.RestrictionGroupFilter then the pipeline exits. If there are no active restriction groups the pipeline continues to the next pipe. |
| 500 | GetCustomerRestrictionGroupIds | If there is no BillTo the pipeline does nothing and continues to the next pipe. If BillTo or ShipTo are populated result.CustomerRestrictionGroupIds is set to the applicable restriction group IDs from result.WebsiteRestrictionGroupQuery. |
| 600 | GetShowOnlyRestrictionGroupIds | Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "ShowOnly" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowOnlyRestrictionGroupIds. |
| 700 | GetShowRestrictionGroupIds | Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Show" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowRestrictionGroupIds. |
| 800 | GetHideRestrictionGroupIds | Adds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Hide" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.HideRestrictionGroupIds. |
| 900 | FormShowOnlySubFilter | Creates a Terms query on the field restrictionGroups with the values from result.ShowOnlyRestrictionGroupIds then adds it to result.SubFilters and assigns it to result.ShowOnlySubFilter. |
| 1000 | FormShowSubFilter | If there are no filters in result.DefaultVisibilityFilters and no filters in result.ShowRestrictionGroupIds continue to next pipe. Creates BoolQuery (AND) from the filters in result.DefaultVisibilityFilters.Creates MatchQuery objects on the field restrictionGroups for all restriction groups in result.ShowRestrictionGroupIds. Combines the previously created queries into a BoolQuery (OR) then adds it to result.SubFilters and assigns it to result.ShowSubFilter. |
| 1100 | FormHideSubFilter | If there are no filters in result.HideRestrictionGroupIds continue to next pipe. Creates MatchQuery (NOT) object on the field restrictionGroups for all restrcition groups in result.HideRestrictionGroupIds. Combines the previously created queries into a BoolQuery (AND) then adds it to result.SubFilters and assigns it to result.HideSubFilter. |
| 1200 | CombineFilters | If result.ShowOnlySubFilter is empty and result.SubFilters is empty continue to next pipe. Create a BoolQuery (AND) fromt he filters in result.SubFilters. Combine BoolQuery just created and result.ShowOnlySubFilter into a new BoolQuery (OR) and assign it to result.RestrictionGroupFilter. |
| 1300 | AddRestrictionGroupFilterToCache | Adds result.RestrictionGroupFilter to the cache based on the current BillTo/ShipTo. |
RunProductSearch
| Order | Pipe | Description |
|---|---|---|
| 100 | GetSettings | Retrieves SearchGeneralSettings, CatalogGeneralSettings, FuzzySearchSettings, SponsoredSearchSettings and SearchSuggestionsSettings then assign them to the result object. Set result.UsingListPrice, result.CanSortPrice, and result.PriceFacetingEnabled from various settings objects. |
| 200 | InitializeSearchProperties | Retrieve and assign values for result.BoostHelper, result.ElasticsearchIndex, result.ElasticsearchQueryBuilder, result.PhraseSuggestConfiguration, and result.ProductSearchFacetProcessor. |
| 300 | FormSortOrder | Populate result.SortOrderFields based on parameter.ProductSearchParameter fields. |
| 400 | ApplySortOrder | Create a new Nest SortDescriptor and set the sort based on result.SortOrderFields. |
| 500 | GetQueryFields | Adds the fields that are queryable using data from the search.SearchBoosts table to result.ExactMatchFields, result.PrefixMatchFields, and result.PhraseMatchFields. Adds the fields that are part of fuzzy match (categoryNames and shortDescription) to result.FuzzyMatchFields. |
| 600 | FormSearchCriteriaQuery | If parameter.ProductSearchParameter.SearchCriteria is blank continue on to next pipe without doing anything else. Creates and assigns queries for result.ExactMatchQuery, result.PrefixMatchQuery, result.PhraseMatchQuery and result.FuzzyMatchQuery (if enabled) based on the fields assigned in GetQueryFields pipe. All new queries added to result.AllQueries. |
| 700 | FormCustomerNameQuery |
|
| 800 | FormSearchWithinQuery | If parameter.ProductSearchParameter.SearchWithin is blank continue on to the next pipe without doing anything else. Create a MultiMatchQuery of type CrossFields for Product on the result.ExactMatch fields querying on parameter.ProductSearchParameter.SearchWithin and assign it to result.SearchWithinQuery. |
| 900 | CombineQueries | Create a BoolQuery (OR) from result.AllQueries. If result.SearchWithinQuery is not null create a BoolQuery (AND) with it and the previously created BoolQuery. Assign the resulting BoolQuery to result.CombinedQuery (either just the first BoolQuery or the combined BoolQuery). |
| 1000 | FormProductFilter | Use the ProductSearchPipeline's FormProductFilter pipeline to get the product filter and assign it to result.FormProductFilterResult. |
| 1100 | FormCategoryAggregation |
|
| 1200 | FormAttributeValueAggregation | If parameter.ProductSearchParameter.DoFacetedSearches is false or result.SearchGeneralSettings.EnableAttributeFilters is false continue on to the next pipe without doing anything else. If result.SearchGeneralSettings.FilterOnSearch if false and parameter.ProductSearchParameter.SearchCriteria is not blank continue on to the next pipe without doing anything else. Using result.FormProductFilterResult.AllFilters create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AttributeValueAggregation. If parameter.ProductSearchParameter.GetAllAttributeFacets is false continue on to the next pipe without doing anything else. Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.AttribueValueFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AllAttributeValueAggregation. |
| 1300 | FormPriceRangeAggregation | If parameter.ProductSearchParameter.DoFacetedSearches is false or result.PriceFacetingEnabled is false continue on to the next pipe without doing anything else. Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.PriceRangeFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the priceFacet field and add it to result.AllAggregations and assign it to result.PriceRangeAggregation. |
| 1350 | FormPersonaScoreFunctionList | If parameter.SiteContext.PersonaDtos contains personas with EnableSearchBoost true, the result ScoreFunctions collection will be populated with one WeightFunction for each persona with a TermQuery on the Persona.Id and boost value as a Weight. The ScoreFunctions collection will be added to the query by the ApplyQueryAndFilter pipe. |
| 1400 | CombineAggregations | If parameter.ProductSearchParameter.DoFacetedSearches is false continue on to the next pipe without doing anything else. Call result.CombinedAggregation.Filter for each aggregation in result.AllAggregations and assign the result to result.CombinedAggregation. |
| 1500 | FormDidYouMeanSuggestion | If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.DidYouMeanThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions. |
| 1600 | FormAutoCorrectSuggestion | If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.AutoCorrectThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions. |
| 1700 | CombineSuggestions | If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.CombinedSuggestion.Phrase for each suggestion in result.AllSuggestions and assign the result to result.CombinedSuggestion. |
| 1800 | ApplySuggestion | If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Call result.SearchDescriptor.Suggest with result.CombinedSuggestion and assign the result to result.SearchDescriptor. |
| 1900 | PerformAggregationSearch | If parameter.ProductSearchParameter.DoFacetedSearches is false or result.AllAggregations is empty continue on to the next pipe without doing anything else. Create an AggregationSearchDescriptor from result.CombinedQuery and result.FormProductFilterResult.CombinedFilter then use that to query Elasticsearch. Assign result from Elasticsearch to result.AggregationSearchResponse. |
| 2000 | ProcessCategoryAggregation | Use result.ProductSearchFacetProcessor.ProcessCategoryFacets to process the facets and assign the return value to result.ProductSearchResult.CategoryDtos. |
| 2100 | ProcessAttributeValueAggregation | Use ProductSearchFacetProcessor.ProcessAttributeTypeFacets to process the facets and assign the return value to result.ProductSearchResult.AttributeTypeDtos. |
| 2200 | ProcessPriceRangeAggregation | Use ProductSearchFacetProcessor.ProcessPriceRangeFacets to process the facets and assign the return value to result.ProductSearchResult.PriceRangeDto. |
| 2300 | PerformSponsoredSearch | If result.SponsoredSearchSettings.Enabled is false or parameter.IsAutocomplete is true or this is a filtered search or if this is not sorted by relevance or parameter.ProductSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else. Create a new TermQuery on the isSponsored field. That is added to a new query along with result.CombinedQuery and result.FormProductFilterResult.CombinedFilter with a max result length of result.SponsoredSearchSettings.Limit or 5. This is used to query Elasticsearch. The results are assigned to result.SponsoredSearchResponse. |
| 2400 | ProcessSponsoredSearchResponse | If result.SponsoredSearchResponse is null continue on to the next pipe without doing anything else. Create a new QueryContainerDescriptor to exclude all sponsored result Ids and assign it to result.ExcludeSponsoredProductsFilter. Set result.ProductSearchResult.Count to the sponsored search's total results. If this is the first page of results (result.StartRow equals 0), add the returned products to result.ProductSearchResult.Products and set result.PageSize to result.PageSize minus the number of sponsored results returned. If this is not the first page of results (result.StartRow does not equal 0), set result.StartRow to result.StartRow minus the number of sponsored results returned. |
| 2500 | ApplyQueryAndFilter | If result.ExcludeSponsoredProductsFilter is not null make a new new boolean query with result.FormProductFilterResult.CombinedFilter and result.ExcludeSponsoredProductsFilter. If result.CombinedQuery is null create a SearchDescriptor using only the above created filter or result.FormProductFilterResult.CombinedFilter. If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is true, create a SearchDescriptor with a FunctionScore query using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor. If result.ScoreFunctions collection is not null or empty, these functions will be added to the FunctionScore query. This will provide boosting of products with certain personas. If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is false, createa SearchDescriptor using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter. |
| 2600 | ApplyPaging | Set from and size on result.SearchDescriptor using result.StartRow and result.PageSize. |
| 2700 | PerformSearch | Set from and size on result.SearchDescriptor using result.StartRow and result.PageSize. |
| 2800 | ProcessSearchResponse | If result.SearchResponse is null or is not valid return a SearchFailure code with relevant message. Add returned products from SearchResponse to result.ProductSearchResult.Products. Add the total count of returned products to result.ProductSearchResult.Count. If result.CanSortByPrice is false or result.UsingListPrice is false and result.ProductSearchResult.Count is greater than result.CatalogGeneralSettings.PriceSortMaximum, then set result.CanSortByPrice to false, set result.ProductSearchResult.SortOrder to result.SortBy and set result.ProductSearchResult.SortOptions to parameter.SortOptions where PriceLowToHigh and PriceHighToLow are removed. If result.CanSortByPrice is true or result.UsingListPrice is true and result.ProductSearchResult.Count is less than or equal to result.CatalogGeneralSettings.PriceSortMaximum, then set result.ProductSearchResult.SortOptions to parameter.SortOptions. |
| 2900 | ProcessDidYouMeanSuggestion | If result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "didyoumean" or didYouMeanSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else. Populate result.ProductSearchResult.DidYouMeanSuggestions from the result.SearchResponse.Suggest entry for "didyoumean". |
| 3000 | ProcessAutoCorrectSuggestion | If result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "correction" or autoCorrectSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else. Populate result.ProductSearchResult.AutoCorrectSuggestion from the result.SearchResponse.Suggest entry for "correction". |
Updated about 2 years ago