HomeGuides
Submit Documentation FeedbackJoin Developer CommunityLog In

Search

This topic describes Search pipelines.

CategoryIndexerPipeline

CreateElasticsearchCategory

OrderPipeDescription
100CreateElasticsearchCategoryCreate a new ElasticsearchCategory object using the parameter object's fields and assigns it to result.ElasticsearchCategory.

GetIndexableCategories

OrderPipeDescription
100GetIndexableCategoriesQueryCreates an Entity Framework Category query using the parameter object's fields and assigns it to result.IndexableCategories.

MapElasticSearchCategoryProperty

OrderPipeDescription
100MapElasticSearchCategoryCustomPropertiesIf the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty.

CategorySearchPipeline

RunCategorySearch

OrderPipeDescription
100InitializeSearchPropertiesRetrieves the current SiteContext, ElasticsearchIndex and ElasticsearchQueryBuilder and assigns them to the result object.
200GetSettingsRetrieves SearchGeneralSettings and FuzzySearchSettings and assigns them to the result object.
300GetQueryFieldsAssigns the result.CategorySearchFields object with the fields that are queryable.
400FormPagingAssigns result.StartRow and result.PageSize from the parameter object.
500FormCategoryExcludeListFinds the disallowed categories for the current user (using the FormProductFilter pipeline) and caches this value. The value is then assigned to result.DisallowedCategories.
600FormCategoryExcludeFilterIf there are any disallowedCategories adds a new negated Terms query to result.AllFilters using result.DisallowedCategories on the categoryId field.
700FormLanguageFilterAdds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters.
800FormWebsiteFilterAdds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters.
900CombineFiltersCreates a BooleanQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter.
1000FormMainSearchQuery

Creates a MultiMatchQuery of type CrossFields using parameter.Criteria and result.CategorySearchFields and apply boosts.

Creates a MultimatchQuery of type PhrasePrefix using parameter.Criteria and result.CategorySearchFields and apply boosts.

Adds both of these queries to result.AllQueries.

1100FormFuzzySearchQuerry

If fuzzy search is enabled, creates a MultiMatchQuery with fuzziness set to FuzzySearchSettings.MaxEdits using parameter.Criteria and result.CategorySearchFields and apply boosts.

Adds this query to result.AllQueries.

1200

CombineQueriesCreates a BooleanQuery (OR) of all queries in result.AllQueries and assigns it to result.CombinedQuery.
1300ApplyQueryAndFilterCreates a SearchDescriptor with a FunctionScore query using result.CombinedQuery and result.CombinedFilter, with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor.
1400ApplyPagingSets result.SearchDescriptor's From and Size variables using result.StartRow and result.PageSize.
1500ApplySortOrderSets result.SearchDescriptor's sort to sort on the sortOrder field in descending order then on _score in descending order.
1600PerformSearchUses result.ElasticsearchIndex.Client to call Elasticsearch using result.SearchDescriptor and assigns the result to result.SearchResponse.
1700ProcessSearchResponseIf result.SearchResponse is not null and is valid, creates a new CategorySearchResult with the search result count and an IEnumerable of CategorySearchResultDto objects using result.SearchResponse.Documents then assigns this to result.CategorySearchResult.

ContentIndexerPipeline

CreateElasticsearchContent

OrderPipeDescription
100CreateElasticsearchContent

Creates a new ElasticsearchContent using the parameter object's fields and assigns it to result.ElasticsearchContent.

Fields that vary based on language, persona are populated by filtering parameter.IndexableContentItem.Fields for the field with the appropriate context.

GetIndexableContentItems

OrderPipeDescription
100GetIndexableContentItemQueryCreates an Entity Framework IndexableContentItem query using the parameter object's fields and assigns it to result.IndexableContentItems.

ContentSearchPipeline

RunContentSearch

OrderPipeDescription
100ValidatePageSizeSets result.PageSize based on parameter.PageSize, if it is less than or equal to zero set to 10, if it is greater than 1000 set to 1000 or else set to parameter.PageSize.
200FormQueryCreates a BooleanQuery (OR) of two MultiMatchQuery queries for the content fields (title, body, metaDescription, and metaKeywords) one of type CrossFields and one of type PhrasePrefix and assigns it to result.Query.
300FormFilterCreates a BooleanQuery (AND) of TermQueries to filter on website, language, deviceType, persona and not restricted pages and assigns it to result.Filter.
400PerformSearchCreates a new SearchRequest using result.Query, result.Filter and result.PageSize, then performs the search and assigns the response to result.SearchResponse.
500ProcessSearchResponseCreates an IEnumerable of ContentSearchResultsDto from the result.SearchResponse.Document fields (title, pageType, and URL) and assigns it to result.ContentSearchResults.

ProductIndexerPipeline

CreateElasticsearchProduct

OrderPipeDescription
100CreateElasticsearchProductCreate a new ElasticsearchProduct object using the parameter object's fields and assigns it to result.ElasticsearchProduct.
200SetSpellingCorrectionJoins ElasticseachProduct fields (ShortDescription, Content, ManufacturuerItem, Name, Specifications, ErpNumber, CategoryNames, and FilterNames) into a string and assigns it to result.ElasticsearchProduct.SpellingCorrection.

GetIndexableProducts

OrderPipeDescription
100GetIndexableProductsSqlStatementPartsCreates a string with the Insite default SQL query to get the correct products and fields from the database and assigns it to result.SqlStatement.Assigns result.IncremenalFilter with the proper SQL snippet if parameter.IsIncremental is true.
200CombineIndexableProductsSqlStatementPartsCombines result.SqlStatement, result.CustomFields, and result.IncrementalFilter and assigns it to result.FormattedSqlStatement.
300PerformIndexableProductsSqlQueryGets an IEnumerable using the SQL query from result.FormattedSqlStatement and assigns it to result.IndexableProducts.

MapElasticSearchProductProperty

OrderPipeDescription
100MapElasticSearchProductCustomPropertiesIf the parameter.SearchBoost.IsQueryable is true and parameter.PropertyDefinition.IsCustomProperty is true sets result.Property to a new Nest TextProperty.

ProductSearchPipeline

FormProductFilter

OrderPipeDescription

100

GetSettingsRetrieves SearchGeneralSettings and ProductRestrictionsSettings and assigns them to the result object.
200FormWebsiteFilterAdds a new MatchQuery using the result.SiteContext.WebsiteDto.Id on the websiteId field to result.AllFilters and assigns it to result.WebsiteFilter.
300FormLanguageFilterAdds a new MatchQuery using the result.SiteContext.LanguageDto.LanguageCode on the languageCode field to result.AllFilters and assigns it to result.LanguageFilter.
400FormCategoryFilterCreates a new PrefixQuery to search a single category or the whole category tree based on parameter.ProductSearchParameter fields (SearchCriteria and IncludeSubcategories) then adds it to result.AllFilters and assigns it to result.CategoryFilter.
500FormRestrictionGroupFilterIf Restriction Groups are enabled, uses the ProductSearchPipeline's FormRestrictionGroupFilter pipe to generate a new filter then adds the filter to result.AllFilters and assigns it to result.RestrictionGroupFilter.
600FormAttributeValueFilterCreates PrefixQueries for attributes based on parameter.ProductSearchParameter.AttributeValueIds and combines them into a BooleanQuery. That BooleanQuery is added to result.AllFilters and assigned to result.AttributeValueFilter.
700FormAllowedProductsFilterCreates MatchQuery objects for each product in parameter.ProductSearchParameter.AllowedProductIds and combines them into a BooleanQuery. That BooleanQuery is add to result.AllFilters and assigned to result.AllowedProductsFilter.
800FormPriceRangeFilter

If parameter.ProductSearchParameter.PriceFilters has any values a new MatchQuery is created for each filter and combined into a BooleanQuery (OR) then assigned to result.PriceRangeFilter and added to result.AllFilters.

If parameter.ProductSearchParameter.PriceFilters has no values and parameter.ProductSearchParameter.MinimumPrice or parameter.ProductSearchParameter.MaximumPrice have been supplied a new Range query is created using the supplied min/max values with defaults of 0 and 1000000 if either was omitted. That Range query is then added to result.AllFilters and assigned to result.PriceRangeFilter.

If no filters were supplied and neither the minimumPrice or maximumPrice were supplied nothing is done and the pipe returns.

900CombineFiltersCreates a BoolQuery (AND) of all filters in result.AllFilters and assigns it to result.CombinedFilter.

FormRestrictionGroupFilter

OrderPipeDescription
100GetRestrictionGroupFilterFromCache

Attempts to retrieve the restriction group filter from cache based on the current BillTo/ShipTo and assigns it to result.RestrictionGroupFilter.

If there was a cached filter the pipeline exits.

if there was no cached filter the pipeline continues to the next pipe.

200GetDefaultVisibilityFilters

Creates a new MatchQuery on the restrictionGroups field to match products with no restriction groups and adds it to result.DefaultVisibilityFilters.

Creates a Terms query on the defaultVisibility field with values of empty and show then adds it to result.DefaultVisibilityFilters.

300GetWebsiteRestrictionGroupQueryCreates an EF query to get the active restriction groups based on the current website and assigns it to result.WebsiteRestrictionGroupQuery.
400GetWebsiteRestrictionGroups

Uses result.WebsiteRestrictionGroupQuery to get active restriction groups then assigns them to result.WebsiteRestrictionGroups.

If there are any active restriction groups a BoolQuery (AND) is created from result.DefaultVisibilityFilters and assigned to result.RestrictionGroupFilter then the pipeline exits.

If there are no active restriction groups the pipeline continues to the next pipe.

500GetCustomerRestrictionGroupIds

If there is no BillTo the pipeline does nothing and continues to the next pipe.

If BillTo or ShipTo are populated result.CustomerRestrictionGroupIds is set to the applicable restriction group IDs from result.WebsiteRestrictionGroupQuery.

600GetShowOnlyRestrictionGroupIdsAdds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "ShowOnly" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowOnlyRestrictionGroupIds.
700GetShowRestrictionGroupIdsAdds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Show" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.ShowRestrictionGroupIds.
800GetHideRestrictionGroupIdsAdds all restriction group IDs from result.WebsiteRestrictionGroups where the restrictiongroup type is "Hide" and is an applicable restriction group (result.CustomerRestrictionGroupIds) to result.HideRestrictionGroupIds.
900FormShowOnlySubFilterCreates a Terms query on the field restrictionGroups with the values from result.ShowOnlyRestrictionGroupIds then adds it to result.SubFilters and assigns it to result.ShowOnlySubFilter.
1000FormShowSubFilter

If there are no filters in result.DefaultVisibilityFilters and no filters in result.ShowRestrictionGroupIds continue to next pipe.

Creates BoolQuery (AND) from the filters in result.DefaultVisibilityFilters.Creates MatchQuery objects on the field restrictionGroups for all restriction groups in result.ShowRestrictionGroupIds.

Combines the previously created queries into a BoolQuery (OR) then adds it to result.SubFilters and assigns it to result.ShowSubFilter.

1100FormHideSubFilter

If there are no filters in result.HideRestrictionGroupIds continue to next pipe.

Creates MatchQuery (NOT) object on the field restrictionGroups for all restrcition groups in result.HideRestrictionGroupIds.

Combines the previously created queries into a BoolQuery (AND) then adds it to result.SubFilters and assigns it to result.HideSubFilter.

1200CombineFilters

If result.ShowOnlySubFilter is empty and result.SubFilters is empty continue to next pipe.

Create a BoolQuery (AND) fromt he filters in result.SubFilters.

Combine BoolQuery just created and result.ShowOnlySubFilter into a new BoolQuery (OR) and assign it to result.RestrictionGroupFilter.

1300AddRestrictionGroupFilterToCacheAdds result.RestrictionGroupFilter to the cache based on the current BillTo/ShipTo.

RunProductSearch

OrderPipeDescription
100GetSettings

Retrieves SearchGeneralSettings, CatalogGeneralSettings, FuzzySearchSettings, SponsoredSearchSettings and SearchSuggestionsSettings then assign them to the result object.

Set result.UsingListPrice, result.CanSortPrice, and result.PriceFacetingEnabled from various settings objects.

200InitializeSearchPropertiesRetrieve and assign values for result.BoostHelper, result.ElasticsearchIndex, result.ElasticsearchQueryBuilder, result.PhraseSuggestConfiguration, and result.ProductSearchFacetProcessor.
300FormSortOrderPopulate result.SortOrderFields based on parameter.ProductSearchParameter fields.
400ApplySortOrderCreate a new Nest SortDescriptor<ElasticsearchProduct> and set the sort based on result.SortOrderFields.
500GetQueryFields

Adds the fields that are queryable using data from the search.SearchBoosts table to result.ExactMatchFields, result.PrefixMatchFields, and result.PhraseMatchFields.

Adds the fields that are part of fuzzy match (categoryNames and shortDescription) to result.FuzzyMatchFields.

600FormSearchCriteriaQuery

If parameter.ProductSearchParameter.SearchCriteria is blank continue on to next pipe without doing anything else.

Creates and assigns queries for result.ExactMatchQuery, result.PrefixMatchQuery, result.PhraseMatchQuery and result.FuzzyMatchQuery (if enabled) based on the fields assigned in GetQueryFields pipe.

All new queries added to result.AllQueries.

700FormCustomerNameQuery
  • If parameter.ProductSearchParameter.SearchCriteria is blank for BillTo and ShipTo are null continue on to next pipe without doing anything else.
  • If BillTo is not null create a PrefixQuery on the customerNames field and match on BillTo.Id + searchCriteria.
  • If ShipTo is not null create a PrefixQuery on the customerNames field and match on ShipTo.Id + searchCriteria.
  • Create a BoolQuery (OR) from the previously created queries then add it to result.AllQueries and assign it to result.CustomerNameQuery.
800FormSearchWithinQuery

If parameter.ProductSearchParameter.SearchWithin is blank continue on to the next pipe without doing anything else.

Create a MultiMatchQuery of type CrossFields for Product on the result.ExactMatch fields querying on parameter.ProductSearchParameter.SearchWithin and assign it to result.SearchWithinQuery.

900CombineQueries

Create a BoolQuery (OR) from result.AllQueries.

If result.SearchWithinQuery is not null create a BoolQuery (AND) with it and the previously created BoolQuery.

Assign the resulting BoolQuery to result.CombinedQuery (either just the first BoolQuery or the combined BoolQuery).

1000FormProductFilterUse the ProductSearchPipeline's FormProductFilter pipeline to get the product filter and assign it to result.FormProductFilterResult.
1100FormCategoryAggregation
  • If parameter.ProductSearchParameter.DoFacetedSearches is false continue on to the next pipe without doing anything else.
  • If parameter.ProductSearchParameter.SearchCriteria is blank and parameter.ProductSearchParameter.IncludeSubcategories is false continue on to the next pipe without doing anything else.
  • Using result.FormProductFilterResult.AllFilters create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the categories field and add it to result.AllAggregations and assign it to result.CategoryAggregation.
1200FormAttributeValueAggregation

If parameter.ProductSearchParameter.DoFacetedSearches is false or result.SearchGeneralSettings.EnableAttributeFilters is false continue on to the next pipe without doing anything else.

If result.SearchGeneralSettings.FilterOnSearch if false and parameter.ProductSearchParameter.SearchCriteria is not blank continue on to the next pipe without doing anything else.

Using result.FormProductFilterResult.AllFilters create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AttributeValueAggregation.

If parameter.ProductSearchParameter.GetAllAttributeFacets is false continue on to the next pipe without doing anything else.

Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.AttribueValueFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the filters field and add it to result.AllAggregations and assign it to result.AllAttributeValueAggregation.

1300FormPriceRangeAggregation

If parameter.ProductSearchParameter.DoFacetedSearches is false or result.PriceFacetingEnabled is false continue on to the next pipe without doing anything else.

Using result.FormProductFilterResult.AllFilters where the filters are not equal to result.FormProductFilterResult.PriceRangeFilter create a BoolQuery (AND). With that query create FilterAggregationDescriptor func for the priceFacet field and add it to result.AllAggregations and assign it to result.PriceRangeAggregation.

1350FormPersonaScoreFunctionListIf parameter.SiteContext.PersonaDtos contains personas with EnableSearchBoost true, the result ScoreFunctions collection will be populated with one WeightFunction for each persona with a TermQuery on the Persona.Id and boost value as a Weight. The ScoreFunctions collection will be added to the query by the ApplyQueryAndFilter pipe.
1400CombineAggregations

If parameter.ProductSearchParameter.DoFacetedSearches is false continue on to the next pipe without doing anything else.

Call result.CombinedAggregation.Filter for each aggregation in result.AllAggregations and assign the result to result.CombinedAggregation.

1500FormDidYouMeanSuggestion

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.DidYouMeanThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions.

1600FormAutoCorrectSuggestion

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.PhraseSuggestConfiguration.Configure with productSearchParameter.SearchCriteria and result.SearchSuggestions.AutoCorrectThreshold (limited to 0 to 5) and assign the value to result.DidYouMeanSuggestion and add it to result.AllSuggestions.

1700CombineSuggestions

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.CombinedSuggestion.Phrase for each suggestion in result.AllSuggestions and assign the result to result.CombinedSuggestion.

1800ApplySuggestion

If result.PhraseSuggestConfiguration.Enabled is false or productSearchParameter.IncludeSuggestions is false or productSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Call result.SearchDescriptor.Suggest with result.CombinedSuggestion and assign the result to result.SearchDescriptor.

1900PerformAggregationSearch

If parameter.ProductSearchParameter.DoFacetedSearches is false or result.AllAggregations is empty continue on to the next pipe without doing anything else.

Create an AggregationSearchDescriptor from result.CombinedQuery and result.FormProductFilterResult.CombinedFilter then use that to query Elasticsearch. Assign result from Elasticsearch to result.AggregationSearchResponse.

2000ProcessCategoryAggregationUse result.ProductSearchFacetProcessor.ProcessCategoryFacets to process the facets and assign the return value to result.ProductSearchResult.CategoryDtos.
2100ProcessAttributeValueAggregation Use ProductSearchFacetProcessor.ProcessAttributeTypeFacets to process the facets and assign the return value to result.ProductSearchResult.AttributeTypeDtos.
2200ProcessPriceRangeAggregationUse ProductSearchFacetProcessor.ProcessPriceRangeFacets to process the facets and assign the return value to result.ProductSearchResult.PriceRangeDto.
2300PerformSponsoredSearch

If result.SponsoredSearchSettings.Enabled is false or parameter.IsAutocomplete is true or this is a filtered search or if this is not sorted by relevance or parameter.ProductSearchParameter.SearchCriteria is blank continue on to the next pipe without doing anything else.

Create a new TermQuery on the isSponsored field. That is added to a new query along with result.CombinedQuery and result.FormProductFilterResult.CombinedFilter with a max result lenght of result.SponsoredSearchSettings.Limit or 5. This is used to query Elasticsearch. The results are assigned to result.SponsoredSearchResponse.
2400ProcessSponsoredSearchResponseIf result.SponsoredSearchResponse is null continue on to the next pipe without doing anything else.

Create a new QueryContainerDescriptor<ElasticsearchProduct> to exclude all sponsored result Ids and assign it to result.ExcludeSponsoredProductsFilter.

Set result.ProductSearchResult.Count to the sponsored search's total results.

If this is the first page of results (result.StartRow equals 0), add the returned products to result.ProductSearchResult.Products and set result.PageSize to result.PageSize minus the number of sponsored results returned.

If this is not the first page of results (result.StartRow does not equal 0), set result.StartRow to result.StartRow minus the number of sponsored results returned.
2500ApplyQueryAndFilter

If result.ExcludeSponsoredProductsFilter is not null make a new new boolean query with result.FormProductFilterResult.CombinedFilter and result.ExcludeSponsoredProductsFilter.

If result.CombinedQuery is null create a SearchDescriptor using only the above created filter or result.FormProductFilterResult.CombinedFilter.

If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is true, create a SearchDescriptor with a FunctionScore query using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter with a FieldValueFactor function using the boost field and assign it to result.SearchDescriptor.

If result.ScoreFunctions collection is not null or empty, these functions will be added to the FunctionScore query. This will provide boosting of products with certain personas.

If result.CombinedQuery is not null and result.SearchGeneralSettings.EnableProductBoost is false, createa  SearchDescriptor using result.CombinedQuery and the above created filter or result.FormProductFilterResult.CombinedFilter. 
2600ApplyPagingSet from and size on result.SearchDescriptor using result.StartRow and result.PageSize.
2700PerformSearchSet from and size on result.SearchDescriptor using result.StartRow and result.PageSize.
2800ProcessSearchResponse

If result.SearchResponse is null or is not valid return a SearchFailure code with relevant message.

Add returned products from SearchResponse to result.ProductSearchResult.Products.

Add the total count of returned products to result.ProductSearchResult.Count.

If result.CanSortByPrice is false or result.UsingListPrice is false and result.ProductSearchResult.Count is greater than result.CatalogGeneralSettings.PriceSortMaximum, then set result.CanSortByPrice to false, set result.ProductSearchResult.SortOrder to result.SortBy and set result.ProductSearchResult.SortOptions to parameter.SortOptions where PriceLowToHigh and PriceHighToLow are removed.

If result.CanSortByPrice is true or result.UsingListPrice is true and result.ProductSearchResult.Count is less than or equal to result.CatalogGeneralSettings.PriceSortMaximum, then set result.ProductSearchResult.SortOptions to parameter.SortOptions.
2900ProcessDidYouMeanSuggestionIf result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "didyoumean" or didYouMeanSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else.
Populate result.ProductSearchResult.DidYouMeanSuggestions from the result.SearchResponse.Suggest entry for "didyoumean".
3000ProcessAutoCorrectSuggestion

If result.SearchResponse.Suggest is null or result.SearchResponse.Suggest does not contain the key "correction" or autoCorrectSuggestions.Length is not equal to 1 continue on to the next pipe without doing anything else.

Populate result.ProductSearchResult.AutoCorrectSuggestion from the result.SearchResponse.Suggest entry for "correction".

Did this page help you?