Index
Video Courses
Watch structured video courses
Knowledge Base
Visit our User Forum for discussions & solutions
Data Preservation on a Data source
Data preservation is needed when a template retrieves data from a database (such as Salesforce or any other external data source) and that data consists of a main collection together with one or more sub-collections. This applies to the standard way of working with collections and sub-collections in XpressDox and extends to SOQL or SQL queries as well.
In addition to data retrieved from a database, it is also possible for the user to add non-data source data elements into repeaters. In other words, the dataset consists of data source data elements and user-captured data elements at the same levels.
The Problem
When a dataset of this kind is saved and later reused, via “Use Previous Data” or “Load”, together with a Choose/Include/Link data source command that has the Refresh parameter on it, the system faces a matching problem. The Refresh code had no reliable way of knowing which item in the data source repeater corresponded to which item in the dataset repeater. This is particularly problematic when records have been reordered, updated or otherwise changed in the database since the data was originally captured. Without a reliable match, the user-captured data could end up paired with the wrong database record or even lost entirely.
Example scenario:A template retrieves a family record. The data source provides the father’s name, mother’s name and a list of children. Each child also has a list of pets. The user then adds extra information for each child (e.g. favourite colour, school name, etc.) and for each pet (e.g. colour) that does not come from the database. If the child records are later reordered in the database and the template is refreshed, the system needs to know that child one’s favourite colour belongs with child one and not with a sibling whose record now appears in child one’s former position. |
The Underlying Database
The following table illustrates some of the sample data used throughout this article. Note that the children are not stored in a predictable order. Their IDs do not follow the sequence in which they appear in the table. This is exactly the kind of situation that makes data preservation necessary.
The Solution: Data Preservation IDs
To solve this, the concept of the Data Preservation ID was introduced. A special field, named XD_DataPreservationId, is added to each record in each sub-collection. This field holds the unique database ID of that record and is stored within the document’s XML alongside the user-captured data. When a Refresh is performed, XpressDox uses these IDs as anchors to correctly match each database record to its corresponding row in the document. User-captured data is then merged into the right position, regardless of how records may have moved or changed in the database.
Configuration
To enable data preservation, the Data Preservation ID must be configured for the ID column of each sub-collection in the Data Element Name Mapping dialog. This is done at each level of the data hierarchy.
Step 1: Configure the Child sub-collection ID
In the Data Element Name Mapping Dialog, select the ID field under the Child node. The Name in Data Source will show Child/ID. Click the Use XD_DataPreservation_Id button to automatically populate the mapped name.
After clicking the button, the Map to this name field is populated with XD_Preservation_Id. Click Accept to confirm.
Step 2: Configure the Pet sub-collection ID
Repeat the same process for the ID field under the Pet node. The Name in Data Source will show Child/Pet/ID, and it maps to the same XD_DataPreservation_Id name (This is perfectly fine because the two fields exist at different levels of hierarchy).
Note: Multiple sub-collections can map their ID to the same field name because they are at different levels of the data structure. XpressDox resolves them correctly based on context.
XML Structure
The following sample XML shows what is stored in the dataset. The key element is the XD_DataPreservation_Id fields. These are the critical anchors that make data preservation work. Note the XML attributes you will encounter on these elements: xddsn identifies the data source name, xdid marks the primary ID field, and xdDataId holds the internal row identifier used by XpressDox during matching.
The following shows what this looks like in a real XpressDox template. Notice how the ChooseFromDataSource command includes the Refresh parameter, and how the Heading commands separate data source fields from manually captured fields at each level:
«ChooseFromDataSource(Family,Choose a family,Refresh)»
«FatherName»
«MotherName»
«ForEach(Child)»
«Heading(ChildFirstName,These data elements come from the data source)»
«ChildFirstName» «ChildLastName» («ChildSSN»)
«Heading(ChildFavColor,These data elements are captured manually)»
«ChildFavColor» «ChildSchool»
«ForEach(Pet)»
«Heading(PetName,These data elements come from the data source)»
«PetName» «PetType»
«Heading(PetColour,These data elements do not come from the data source)»
«PetColour»
«End(pet)»
«End(child)»
When records are moved, reordered, or updated in the database and the template is run again with the Refresh parameter active, XpressDox will use the Data Preservation IDs to ensure that each updated database record is correctly matched to its corresponding user-captured data. The manually entered information such as a child’s favourite colour or a pet’s colour will always appear in the right position, regardless of how the underlying data may have changed.