inRiver (IR) and Configured Commerce integration strategy
Describes the XML Schema exports provided by IR to create a data feed for Optimizely Configured Commerce to pick up and transcribe.
IR and Optimizely Configured Commerce have worked together to understand the data structures within IR and have arrived at the strategy for integration which will leverage the out of the box XML Schema exports provided by IR to create a data feed for Configured Commerce to pick up and transcribe.
Both IR and Configured Commerce have ETL mapping tools and the ability to write code specific to a connector (IR) or plug-in. Optimizely has taken on the responsibility to write IR-specific plug-ins and leveraging the standard schema export from IR.
Configured Commerce will pull Products (including Items and SKUs discussed later), Channel Nodes (category hierarchy in Configured Commerce parlance), and Resources (digital assets) as part of the initial, standard integration.
This document will address some schema standards we wish to impose for any IR/Configured Commerce customers so that we can use a standardized strategy to map IR data to Configured Commerce structures.
IR entity setup
We request that the following entities be required and statically named.
Channel
This is the typical publication channel that would be established for inRiver. It can represent print or other recipient. Each targeted website that would have a catalog hierarchy should be set up as a separate channel.
Channel Node
This entity represents a hierarchical representation of products within categories for a given channel/web site. This will be used to create the category hierarchy in Configured Commerce.
Products
We will expect a Product entity to be established that can act as the end item to be created or as a placeholder for a product that is merchandised but has variants. If there are no variants, the product record itself can be set up with the SKU number or a singular item can be set up that holds the SKU number.
Items
These are variants for the product. It will hold information specific to the variant (that is color images or specific attributes). Attribution will mostly come from the Product entity. Within Items, we will have an optional XML field called SKUs which can hold additional information about sub-level skus with their variant-specific data.
General Reasoning
inRiver will want entities that represent saleable items that capture relevant merchandising information. For example, if we have a specific shirt - we merchandise all the information about fabric, care instructions, and so on at the Product entity. We create Items for the different colors but these are not specific SKUs. At the SKU level, we have the actual sku and the size so that, together, the shirt is ultimately defined by the Product (shirt), Item (color) and SKU (size).
Configured Commerce entities
Product
Configured Commerce has a product table that can hold generic products (the same as the IR product), specific SKU's tied to a product (Items), services, add-ons, and so on. This is the master holding entity for anything that is sold or merchandised on the platform. Products that have children are termed "Styled Products".
Style Class
This entity represents a type of product with a predefined set of attributes that define it for purchasing purposes (not merchandising purposes). For example, a T-Shirt may always have a size and color but nothing else. Any number of styled products can use the same style class. This is not unlike the fieldset construct in IR.
Style Trait
This entity holds the specific traits within a style class.
Style Trait Value
These are the individual values available for the trait associated to the class.
Attribute Type
This is a more generic set of attributes and is used for product comparisons and faceting. Attribute Types are global in nature and can be assigned by category. Products then are assigned specific values for any attribute type that is associated with all of the categories the product is assigned to. CVLs in IR will be mapped to attributes in Configured Commerce.
Attribute Value
This is the actual value of the attributes. Attributes in Configured Commerce look a lot like CVLs in IR.
General Note
Currently, there is a distinction between Style Trait Values and Attribute Values even though they could be repeated within a single product. Style Traits are used to narrow down the selections of a product to the orderable SKU, while attributes are used for merchandising, searching, and comparing.
Field mapping
Rather than impose any specific naming conventions (other than entity naming conventions), there are some fields that Configured Commerce will require in order for the integration to be effective. The tables below will describe these requirements. The assumption is that these data will be provided in the schema and it will be up to the Configured Commerce or IR implementer to create the mappings during implementation.
Required fields
Target Table | Target Field | Usage/Notes |
---|---|---|
ChannelName | Name | We need to ensure that the channel name in IR matches the website name in <> for things to match up - there is a workaround to this shown below by using a website configuration. |
Product | ERPNumber | <> requires a unique field to identify each product and, if this is a purchasable product, then it must also conform to the product identifier within the ERP system - this is often thought of as the SKU. For a product that is not directly orderable (styled product), there still needs to be some unique identifier. |
Product | URLSegment/Name | These can be the same as the ERP Number or different but are required for all products in <> |
Item | ERPNumber | We will still require a unique field here since items and products will both have the same target table. If the final ERP numbers are represented within the SKUs field, then that will become the ERP number and nothing more specific is required on the item itself. |
Item | URLSegment/Name | These can be the same as the ERP Number or different but are required for all products in <> |
StyleClass | Name | We will need something to anchor to in order to set up a style class. We will used a mapped field for this instead of the fieldset. |
AttributeType |
| CVLs must be named either the same in the entity or prefixed with the entity name in order to be resolved. For example, if we have a CVL called Color, the field mapped to it in IR must be either Color or ProductColor (or ItemColor) or it cannot be mapped. The XML export does not provide enough detail to map it back to the source data element so this convention-based approach must be used. |
How it is put together
We have established a series of predefined integration jobs and plug-ins to handle the integration. Each of these will be explained below along with supporting technical information regarding settings.
Because we wanted to keep this integration as standard as possible, we are using the out of the box inRiver.Connectors.Schema.Output.Files connector with a single file per entity export from inRiver. Developers can further enhance or extend this as desired - the provided code will definitely allow for a solid and deep integration but may not accommodate every circumstance given the highly configurable nature of a product information management solution.
Keep in mind that in inRiver, there are Product and Item entities where in Configured Commerce they are all considered products and if there is an item related to a product, those are considered styled children. The steps involved in the integration are laid out intentionally to optimally fill out all of these relationships in a particular order to complete the structures.
Warnings/Known Issues
- The system does not handle multi-value fields out of the box
- The system does not handle locale strings - most data fields in Configured Commerce are not enabled for multi-language support. The system will pull out the entry that matches the default language identified for the site. This means that we could import multi-language to language-specific sites but not to multiple languages on the same site.
- While the system will create the entire category structure from the channel node feed from IR, the attributes associated with those channels is not mapped by the integration. In order for attributes to be assigned to products, the product must be assigned to a category that has those attributes assigned to it. Nothing in the integration will create that particular relationship. We suggest that the users set up the taxonomy in IR, import just the taxonomy and CVLs and then manually map the attributes to the categories before importing products and items - that way the attributes for the products will have the ability to be mapped in.
- As data is changed in IR, it automatically creates XML files in the export directory. Oftentimes it will create multiple versions of the same data. The system is built to pick up these files in time-sequenced order so that the most current version will be transmitted. If an entire set of files is missing, however, the integration job will register that job as CompletedWithWarnings and you will notice in the job log that there will be warnings such as "No files found matching 'xxxxxxx' - this is a standard feature of flat file imports.
- The FileUpload job is an Execution job but it still must have at least 1 step as a dummy step in order to trigger the integration processor.
- If an item is modified after initial load, it generates its XML file as 'Updated' vs. 'New' and does not include the links to the parent - this breaks the integration for that item. We are working with inriver to have them fix that behavior.
Technical extensions
Other than the integration plug-ins defined below, the following are some special constructs used to support the integration.
- WebsiteConfiguration. There is only one special configuration entry that is generated automatically provided that the channel name in IR matches the website name in Configured Commerce. The system will create an entry called inRiver Channel EntityId which contains the IR internal entity id # which is a simple identity/integer field. This is used in different places in the system to tie data together from the
- InRiverCommonVocabulary table. Since CVL-linked data in products and items don't resolve to their value but only send up their entity id, we are persisting that data directly on the server in a special table built by the custom post processor.
Integration jobs and plugIns
Things to understand
- In general, there are a number of fields that come back from the integration processors that are "free". There are other fields, primary the ones in the Product and Item entities that only come back for mapping if they are included in the select statement. All fields within the XML structure "ProductFields" or "ItemFields" are available in this manner.
- The UniqueId field is a special field and is assigned and should be mapped to the ERPNumber field (natural key in Configured Commerce)
PlugIns
Name | Type | Usage |
---|---|---|
InRiverLookupInformation | Preprocessor | This is used to send the contents of the CVL table back into the integration processors to resolve for data values |
InRiverCommonVocabulary | IntegrationProcessor | Gathers all of the information for CVLs and places them into a predefined dataset formal for consumption on the server |
InRiverCommonVocabulary | Postprocessor | Consumes the predefined CVL format and creates Attribute Types, Attribute Values and the translations for all of the values |
InRiverProduct | IntegrationProcessor | Gathers and formats the data from the Product entity including the relationships, key information, and specified fields from the selection statement |
InRiverStyleClass | IntegrationProcessor | Gathers and formats the data for Product to Item relationships and formats in a way that can be consumed for populating Style Classes, Style Traits and Style Trait Values |
InRiverItem | IntegrationProcessor | Similar to the product version, this gathers information about the item entity |
InRiverChannel | IntegrationProcessor | Gathers and formats the data for the channel in order to create the category hierarchy |
InRiverChannelNode | IntegrationProcessor | Gathers and formats the data for the channel node to create the category hierarchy |
InRiverResource | IntegrationProcessor | Gathers the information so that the standard images can be mapped into the product table |
Integration jobs
Below are the details of the various jobs that we created to help enable and streamline a given integration from InRiver to Configured Commerce. These are starting points for the most part and will need to be adjusted to meet any client's specific needs.
We build these jobs where possible to implement the standard field mapper so that the implementation consultant would be able to add new fields and put them where they way when integration with IR.
Some of the integrations have some fields available as "free fields" to use that do not need to be included in the selection column list. These are as follows:
- Channel
- UniqueIdentifier
- ChannelNode
- UniqueIdentifier
- CategoryName
- ProductEntityId
- Item
- UniqueIdentifier
- Active
- ParentProductId
- Product
- UniqueIdentifier
- Active
- Resource
- UniqueIdentifier
- SmallImagePath
- MediumImagePath
- LargeImagePath
- StyleClass
- StyleClass
- StyleTrait
- StyleTraitValue
Job definitions
Name/Step | Processors (prefixed with 'InRiver') or Object | What it does/Notes |
---|---|---|
Common Vocabulary Refresh | Pre: LookupInformation WIS:CommonVocabulary Post:CommonVocabulary | Reads all CVLs and creates the internal table, creates/updates AttributeTypes, Attribute Values and all translations of them This pulls from files that are masked with *CVLS/*.xml. |
Step 1 - Attribute Types | AttributeType | This job should not be altered. |
Step 2 - Attribute Values | AttributeValue | This job should not be altered |
Step 3 - Translation Dictionary | TranslationDictionary | This job should not be altered |
Product Refresh | Pre: LookupInformation WIS: Product Post: FieldMap | This job will populate the style classes, traits, values and create/update the product record for product entities This pulls from files that are masked with *_Product_*.xml |
Style Class | StyleClass | Pick a product field to use as the name of the style class - if you don't have one, you will likely need to add one. The field map should only need to list the fields that define the style class code/name and description (which can be the same as the name field). The sample set uses a field called ProductGroup which may or may not exist in your dataset. The field map is to the object StyleClass and is used to create the actual classes |
Product Initializer | StyleProductInfo | The intent of this step is simply to establish the existence of the parent/styled product in the database and then relate it to the style class it belongs to. The rest of the relationships happen in the Item refresh. Note: You must populate the ERPNumber, Name and URLSegment from the data. ProductGroup and ProductNumber are proxies for the fields you would use in your specific implementation of inRiver. |
Product Finalizer | Product | This is the step that will fill out all of your information for a product including the basic content. We recommend that you have a short description field specifically available. Only the fields listed in the Columns tab will be pulled over and they must exist in the ProductFields section of the XML. Then you can map them to product content, product custom properties or directly to product fields. |
Product Attributes | Product | While this does not need to be a separate step, it is separated out for clarity only. For this to work, you select the product fields you want mapped as attributes - these are limited to CVL fields in IR. For each value selected, you set up a ChildCollection where the From Property is the field from IR and the ToProperty is AttributeValues. Note:These will only map if the attribute types are named specifically the same as the field name or the field name with the entity prepended. The attribute types must already be mapped to one of the categories associated with the product or it also cannot be successfully mapped. |
Item Refresh | Pre: LookupInformation WIS: StyleClass Post: FieldMap | This job will create the products for the child products and then update the style traits and values. This pulls from files that are masked with *_Item_*.xml |
Style Trait Refresh | StyleTrait | The values you must pull out of the Item entity need to be prefixed with 'Variant' to identify the actual style trait being used. For example, VariantColor will map to style trait Color. Note that in <>, style traits are not the same as attributes - they are similar and may actually be the same values but are used in different ways. The column list only needs to be a list of the variant values to map. StyleClass is a "free" column that will map the item to the parent item's style class. The mapping should not be altered - we need the StyleTrait Name and Class only - description is optional and is typically set to the same as the name. This step will create all of the unique combinations of style traits associated with the style class based on the items sent from IR. |
Style Trait Value Refresh | StyleTraitValue | The column list here should be the same as for the style trait step. This steps will create the actual valid values within each style trait based on the items sent from IR. The mapping should not be altered. |
Item Refresh | StyleProductInfo | This is a bit more constrained object - not all properties are available to it. This step will create/update the products that are styled children in B2B and set up the associated style trait values to that product. This step must set the ERPNumber, Name and URLSegment fields. The query must contain all of the variant fields. Additional fields like short description or any other ItemField would be mapped into this step as well. |
Category Refresh | Pre: LookupInformation WIS: Channel Post: FieldMap | This pulls from files that are masked with *_Channel_*.xml |
Channel Refresh | WebSiteConfiguration | All this step does is establish the WebsiteConfiguration entry "inRiver Channel Entity" to implant the unique identity in order to tie the website to the channel. This mapping should not be altered. |
Channel Node Refresh | Category | This step creates/updates categories and their hierarchy and assigns products to the categories. It is unlikely this will require much change in the mapping. |
Resource Refresh | Pre: LookupInformation WIS: Resource Post: FieldMap | This pulls from files that are masked with *_Resource_*.xml |
Resource Refresh | Product | This step will embed the path of the resources (images) into the product for SmallImagePath, MediumImagePath, and LargeImagePath. The mapping should not be changed. There is a step parameter called inRiverResourceLocation that is used to identify the prefix to the actual name of the file. The default in the job is /UserFiles/Images/inRiver. The original images are of any type and the full name is provided in the interface from inRiver, however the smaller images are automatically resized and renamed using configuration within inRiver. To accommodate this we have the PreviewExtension and ThumbnailExtension parameters that will define the extension with a default of jpg. The plug-in will map the Thumbnail image to SmallImagePath, Preview image to MediumImagePath and Original to LargeImagePath. |
FileTransfer | Pre: None WIS: FileUpload Post: None | This is a special purpose job that is used to physically copy the files from the source location into the target location. The WIS uses a web service to move the images. This is an execution job which uses no steps but has to have at least 1 step in order for the WIS processor to pick it up. The presumption is that the publishing process in inRiver will populate the resource files with the images based on changes made to the entities. The following parameters control the process: SourceDirectory - the location on the inRiver server where the resources are located SourceMask - the mask used to define the files to select and send and can include multiple masks separated by a pipe (|) Destination Directory - the relative location on the B2B server to place the images - default value is images/inriver. DeleteSourceFilesOnCopy - true or false - once copied up, indicates if the files should be deleted on the IR server |
Updated over 1 year ago