Troubleshoot delta datasets

This article describes a unique situation where, if certain steps are taken, records in the dataset may not accurately update.

How delta datasets work

Basically, when delta datasets is enabled, the system takes a snapshot and saves it in the DllsFolder (as defined in the site connections file) under a folder called DeltaDataSets. It creates an xml file of the snapshot and temporarily names the file "processing" while the job is running. Upon notification from the server that the job was successful, it renames the file to "success" (name = connection_jobname_hashcode_success.xml) - which is how the system "finds" the 'last successful snapshot' for the delta or change.

Then Optimizely Configured Commerce uses a hash (previously from step parameters, changed to the select string in Version 4.1) to determine the basic format of the dataset. It generates a new dataset and compares it against the last 'successful' dataset and persists the differences to the Configured Commerce database.

If deltas are enabled and the format is changed, the next snapshot will then become the new baseline since there is nothing for comparison.

Issue

When a step parameter is used and it is changed at runtime, a new hash value is created. This results in certain records being missed and ultimately not being deleted.

If, for example, the data changes (mostly affects deletions) between the last refresh and the current changed delta, then these deletes will be skipped and will forever remain active in the database.

Workaround

If delta datasets are established AND the selection is changed:

1. Turn OFF delta data.
2. Refresh one time as a full refresh to synchronize the database.
3. Turn on deltas and run it again, as this ensures a current and synchronized dataset is created for future comparisons.