Blog Series: Unlocking xDB – Vol: #3
Referenced Sitecore Version: Sitecore 8.1U3 – 8.2
In this edition of Unlocking xDB, we’ll expand from the last blog and demonstrate how we can map CSV data uploaded to List Manager into complex facet elements. This blog will begin as if you have read and understood the previous blogs in Unlocking xDB, so if you are just joining, it is recommended that you read over those first before continuing on.
Let’s start off with understanding our data xDB Facet Data model. Previously we created a simple Facet addition to our Contact record. Now we want to complicate it a bit more by adding an Element Dictionary properties to our Facet that can collect an array of information. In this example, our Company, which has added Marketing Data to their Contact records, now want to include historical information about products each Contact has purchased and the price they paid. This information was exported from their ecommerce system and is being uploaded to the Contact record via the CSV uploader.
This fake scenario doesn’t make any sense, and I realize that they could just connect their ecommerce API to the application. I also realize that a flat CSV file might not be the best way to import multiples of information. However, bare with me. This is just setting up an example to demonstrate a use case.
Extending the Custom Contact Facet Data Model
Step 1: Create The Element Contract
We start off by creating an interface for our new Element. Using this example, we create the “IProduct” interface with two property attributes: Product Name, and Product Price.
If you recall from previous posts, Elements plug into the Facet, and a Facet is and of itself an Element as well as a collection of Elements.
using Sitecore.Analytics.Model.Framework;
namespace SitecoreHacker.Sandbox.Facets
{
public interface IProduct : IElement
{
string ProductName { get; set; }
string ProductPrice { get; set; }
}
}
Simple interface deriving from the IElement interface. Now we need to create our concrete class that will support this interface. We will call this the “Product” class.
Step 2: Create The Element Concrete Class
using System;
using Sitecore.Analytics.Model.Framework;
namespace SitecoreHacker.Sandbox.Facets
{
[Serializable]
public class Product : Element, IProduct
{
private const string PRODUCT_NAME = "ProductName";
private const string PRODUCT_PRICE = "ProductPrice";
public string ProductName
{
get { return GetAttribute<string>(PRODUCT_NAME); }
set { SetAttribute(PRODUCT_NAME, value); }
}
public string ProductPrice
{
get { return GetAttribute<string>(PRODUCT_PRICE); }
set { SetAttribute(PRODUCT_PRICE, value); }
}
public Product()
{
EnsureAttribute<string>(PRODUCT_NAME);
EnsureAttribute<string>(PRODUCT_PRICE);
}
}
}
You’ll note that this is a pretty straight forward class. If something about this class looks foreign to you, refer to the previous blog posts in this series to get an understanding.
Here we have setup the ProductName and ProductPrice property elements for a single Product Element. This is the single object representation for the collection objects that our Facet (“Marketing Data”) is going to reference.
Step 3: Add Element Dictionary to our Parent Element Interface
In this case, our parent element is our class based off of the IFacet contract. I call this out because you can nest elements within elements, but that’s outside the scope of this blog. We’ll reopen our IMarketingData contract and add the Element Dictionary Property.
using Sitecore.Analytics.Model.Framework;
namespace SitecoreHacker.Sandbox.Facets
{
public interface IMarketingData : IFacet
{
string CustomerId { get; set; }
string Segment { get; set; }
IElementDictionary<IProduct> Products { get; }
}
}
Note here that IElementDictionary<IProduct> Products {get;} is the property attribute that we added to the IMarketingDara Facet. IElementDictionary is the class that Sitecore uses to refer to a collection of like Elements. In this case, our like Element is the IProduct type.
Step 4: Add Element Dictionary to our Parent Element Class
We’ve finished updating our interface, now we need to update the concrete class representing our parent element/facet and implement the IElementDictionary property.
using System;
using Sitecore.Analytics.Model.Framework;
namespace SitecoreHacker.Sandbox.Facets
{
[Serializable]
public class MarketingData: Facet, IMarketingData
{
private const string CUSTOMER_ID = "CustomerId";
private const string SEGEMENT = "Segment";
private const string PRODUCTS = "Products";
#region Properties
public string CustomerId
{
get { return GetAttribute<string>(CUSTOMER_ID); }
set { SetAttribute(CUSTOMER_ID, value); }
}
public string Segment
{
get { return GetAttribute<string>(SEGEMENT); }
set { SetAttribute(SEGEMENT, value); }
}
public IElementDictionary<IProduct> Products
{
get { return GetDictionary<IProduct>(PRODUCTS); }
}
#endregion
public MarketingData()
{
EnsureAttribute<string>(CUSTOMER_ID);
EnsureAttribute<string>(SEGEMENT);
EnsureDictionary<IProduct>(PRODUCTS);
}
}
}
Stepping through this we do the same three steps for every property in an element:
- We’ve added our constant string variable which contacts the text name of our dictionary property.
- Implemented the Property based on the contract.
- In the constructor, we’ve Ensured that the Dictionary is loaded.
Step 5: Patch New Element to Model Configuration
Finally, we just need to add our Element to the Element collection for the Contact model in the Sitecore configuration.
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
<sitecore>
<model>
<elements>
<element interface="SitecoreHacker.Sandbox.Facets.IMarketingData, SitecoreHacker.Sandbox"
implementation="SitecoreHacker.Sandbox.Facets.MarketingData, SitecoreHacker.Sandbox"/>
<element interface="SitecoreHacker.Sandbox.Facets.IProduct, SitecoreHacker.Sandbox"
implementation="SitecoreHacker.Sandbox.Facets.Product, SitecoreHacker.Sandbox"/>
</elements>
<entities>
<contact>
<facets>
<facet name="Marketing Data" contract="SitecoreHacker.Sandbox.Facets.IMarketingData, SitecoreHacker.Sandbox"/>
</facets>
</contact>
</entities>
</model>
</sitecore>
</configuration>
We have now extended our already custom Facet, and added an additional Element collection (ElementDictionary) to hold additional information. Now time to switch gears
Adding Custom Import Field Item
We’ll start by loading up the Sitecore Desktop and switching to the Core database and then fire up Content Editor. Head to: /sitecore/client/Applications/List Manager/Dialogs/ImportWizardDialog/PageSettings/TabControl Parameters/Map/ImportModel
From the previous blogs, you should see this in the listing:
Let’s take a closer look at the DataField. It’s a JSON string with two properties in it: facet and property.
For a Mississippi minute I thought that this was a pretty fixed way of doing the CSV import but that it didn’t matter, it was fulfilling the need I had in the previous blog. But then I realized that the Email field was in child element dictionaries called “Entries”. How was it populating from the CSV using the same JSON field?
So this just got way more interesting. The JSON field is actually different. There’s a preferred property that appears to be setting a boolean, and a new property called entryProperty. So, seems to be I can have different JSON structures. This made me realize that Sitecore probably has a data model or pipeline associated with this field. So I decided that I would create my own JSON object.
A couple notes that I derived from just looking at these fields:
- Facet needs to reference the Facet that my data needs to go into. This is consistent with each type of field.
- I can probably create whatever JSON property field names that I want, but I want to make sure that I stay away from field names representing something different.
Given those notes that I took, i came up with this JSON structure:
{facet:“<facet name>”, innerDictionary:“<property name of element dictionary>”, key:“<unique key of element in dictionary>”, propertyName:“<property name in inner Element>”}
I then added my items to the Core database using that structure.
The facet I want to modify is Marketing Data.
The innerDictionary I want to modify is Products
The key that I want to assign can be anything but needs to be unique. For this example, this is Product_1.
And the propertyName that I want to populate is called ProductName.
I did the same thing for Product Price 1 in the import model mapping, making sure that I use Product_1 as the key since they are related to the same Element. Lather, rinse, repeat this step for each CSV field that you want to add into this Element dictionary.
But how do I tie this all together?
Introducing the Sitecore Contact Mapper
It took me an embarrassing amount of time to figure out Sitecore was populating from the CSV field into the Email element. I didn’t know what I was searching for. Using a combination of dotPeek searches and different combinations, I finally figured out that Sitecore has an import configuration node that has a contactMapper child. This looks interesting. You can find the OOTB config for contactMapper in the Sitecore.ListManagement.config.
<!-- CONTACT MAPPERS -->
<contactMapper type="Sitecore.ListManagement.Import.CompositeContactTemplateMapper, Sitecore.ListManagement">
<contactMappers hint="list:AddContactMapper">
<contactMapper type="Sitecore.ListManagement.Import.InstantPropertyMapper, Sitecore.ListManagement" />
<contactMapper type="Sitecore.ListManagement.Import.FacetedMapper, Sitecore.ListManagement" />
<contactMapper type="Sitecore.ListManagement.Import.PreferredEntryFacetMapper, Sitecore.ListManagement" />
<contactMapper type="Sitecore.ListManagement.Import.PreferredEntryDictionaryMapper, Sitecore.ListManagement" />
<contactMapper type="Sitecore.ListManagement.Import.MapperNotFoundLogger, Sitecore.ListManagement" />
</contactMappers>
</contactMapper>
</import>
I recommend that you decompile these classes to gain an understanding of exactly what they are doing. However, in a nut shell, Sitecore is calling the contactMapper class “ContactTemplateMapper” that has a .Resolve() method. This method takes the collection of Mappers associated in the contactMappers list node, and processes them top down. It works like a pipeline, however, each mapper can recursively call .Resolve() again to drill down deeper.
Heavily utilizing the PreferredEntryDictionaryMapper, I created my own mapper. There are two types of Mappers in this list so I found out:
- Facet Resolving Mappers
- Property Name/Value Setting Mappers
At the end of the day, we want to identify an element object out of the Contact Facet that allows us to set a Property Name to a Property Value. So the way this works is once we are able to resolve down to the Element that we are looking for, we Resolve() again to initiate the Mapper that sets a Property Value to the respective Property Name in the Element.
Awesome, clear as mud right? Let’s see the mapper I created.
using Newtonsoft.Json;
using Sitecore.Analytics.Model.Framework;
using Sitecore.Diagnostics;
using Sitecore.ListManagement.Import;
using Sitecore.Reflection;
namespace SitecoreHacker.Sandbox.Facets.Mappers
{
public class InnerElementDictionaryMapper : IContactTemplateMapper
{
public bool Map(MappingInfo dataField, object destination, MappingContext context)
{
Assert.ArgumentNotNull(dataField, "dataField");
Assert.ArgumentNotNull(destination, "destination");
Assert.ArgumentNotNull(context, "context");
var contactTemplate = destination as IFaceted;
if (contactTemplate == null)
return false;
var settings = new JsonSerializerSettings()
{
Error = (sender, args) => args.ErrorContext.Handled = true
};
var contactMappingInfo = JsonConvert.DeserializeObject<InnerDictionaryMappingInfo>(dataField.PropertyName, settings);
var facetName = contactMappingInfo?.Facet;
if (string.IsNullOrWhiteSpace(facetName))
return false;
var facet = contactTemplate.Facets[facetName];
if (facet == null ||
string.IsNullOrWhiteSpace(contactMappingInfo.InnerDictionary) ||
string.IsNullOrWhiteSpace(contactMappingInfo.Key) ||
string.IsNullOrWhiteSpace(contactMappingInfo.PropertyName))
return false;
var innerDictionary = ReflectionUtil.GetProperty(facet, contactMappingInfo.InnerDictionary) as IElementDictionary<IElement>;
if (innerDictionary == null)
return false;
var dictionaryKey = contactMappingInfo.Key;
var element = innerDictionary.Contains(dictionaryKey)
? innerDictionary[dictionaryKey]
: innerDictionary.Create(dictionaryKey);
return context.Resolve().Map(new MappingInfo(contactMappingInfo.PropertyName, dataField.PropertyValue), element, context);
}
}
}
Alright, this is a complex class. Let’s step through this:
Step 1: Understanding the Map Method
public bool Map(MappingInfo dataField, object destination, MappingContext context)
The Map method is what Resolve() calls when it processes each Mapper.
MappingInfo is the raw construct of the dataField representation coming out of the Import Field in the Core Database that we created. It has two properties: PropertyName and PropertyValue. Name contains the string JSON object. Value contains the value of the field coming out of the CSV file. This is the eventual value that we want to set our Element Property to.
Object destination is the incoming start element that Resolve() provides the Map() method. This is listed as object because this can be of any type: IElement, IFacet, IFaceted, etc.
MappingContext is last, but not least. This provides the context wrapper that is created from our ContactTemplateMapper. The actual MappingContext object contains the Resolve() method.
Step 2: Ensure we have the correct starting Destination Object
var contactTemplate = destination as IFaceted;
if (contactTemplate == null)
return false;
As mentioned earlier, the contactMappers can be called recursively and perform the same action over a number of objects coming into the destination argument. We want to start at the very beginning, in order to capture the Facet name coming out of our Json. The very first time that Resolve() is called out of the MappingContext, the destination is the set to ContactTemplate item, which is the base object representing an xDB contact. This is the ONLY class in the entire solution that is based off of the IFaceted interface. This is so that we can recognize that we are at the start.
Step 3: Deserialize JSON Field
var settings = new JsonSerializerSettings()
{
Error = (sender, args) => args.ErrorContext.Handled = true
};var contactMappingInfo = JsonConvert.DeserializeObject<InnerDictionaryMappingInfo>(dataField.PropertyName, settings);
The first thing to note here is that we are deserializing the JSON to an object called InnerDictionaryMappingInfo. This is a simple class that represents our JSON object.
namespace SitecoreHacker.Sandbox.Facets.Mappers
{
class InnerDictionaryMappingInfo
{
public string Facet { get; set; }
public string InnerDictionary { get; set; }
public string Key { get; set; }
public string PropertyName { get; set; }
}
}
Now that we’ve defined our JSON object definition class, it will be deserialized into the contactMappingInfo variable.
Step 4: Get Facet and Validate our JSON Field
var facetName = contactMappingInfo?.Facet;
if (string.IsNullOrWhiteSpace(facetName))
return false;var facet = contactTemplate.Facets[facetName];
if (facet == null ||
string.IsNullOrWhiteSpace(contactMappingInfo.InnerDictionary) ||
string.IsNullOrWhiteSpace(contactMappingInfo.Key) ||
string.IsNullOrWhiteSpace(contactMappingInfo.PropertyName))
return false;
First I validate that we have a facetName in our JSON. If I don’t, I can’t go any further, so I will return out of the Map() method with false. But if I do, then I want to get the Facet from the contactTemplate and set the variable facet.
The JSON.Deserialize will generally attempt to create the class regardless of the JSON presented, so if our fields don’t match up with the fields coming out of the Import Field core db item, then our object property values will be null. Therefore, to ensure we have the right JSON field, we want to validate ALL of the properties of our InnerDictionaryMappingInfo.
Step 5: Get the Inner Dictionary from our Facet.
var innerDictionary = ReflectionUtil.GetProperty(facet, contactMappingInfo.InnerDictionary) as IElementDictionary<IElement>;
if (innerDictionary == null)
return false;
Using reflection, we are going to get the IElementDictionary property from our facet that has the name populated in our contactMappingInfo.InnerDictionary. If you recall from our example Data Field above the following values are:
facet = “Marketing Data”
innerDictionary = “Products”
If the resultant innerDictionary variable is null, then the facet isn’t the right type meaning we want to exit out of the Map() method.
Step 6: Identify the Dictionary Key
var dictionaryKey = contactMappingInfo.Key;
So this really isn’t a step, but I’m calling it out to make a point. In .NET, the Dictionary class requires a “key” and a “value”. The ElementDictionary class is no different. So we need to specify a key to represent the dictionary entry that our Product Element is going to be associated with. We defined this key in our JSON, thus I pull it from the contactMappingInfo object and set it to a variable called dictionaryKey. This is a verbose step, and is not needed. You could just use contactMappingInfo.Key where needed.
Step 7: Get or Create the Element entry In Dictionary
var element = innerDictionary.Contains(dictionaryKey)
? innerDictionary[dictionaryKey]
: innerDictionary.Create(dictionaryKey);
If our innerDictionary contains our key, then return the Element (Product) object. If not, then create an entry with that key based on the Element Dictionary class type, in this case it will create a Product object.
We then want to set it to the element variable. Hint: This will become a destination in a future Resolve() call.
Step 8: Resolve!
return context.Resolve().Map(new MappingInfo(contactMappingInfo.PropertyName, dataField.PropertyValue), element, context);
The stage is set! We’ve successfully figured out where we want to store our CSV values (element), and we know the property name (contactMappingInfo.PropertyName) and we have the value from the CSV file (dataField.PropertyValue). There’s already a mapper that’s been created that maps property field to property value in a given element structure, instead of writing our own, we just consume what’s already there, and call Resolve().Map().
Putting the Puzzle Together
The last step is patching our new Contact Mapper to the Sitecore configuration. Pretty basic here as shown below.
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
<sitecore>
<import>
<contactMapper>
<contactMappers hint="list:AddContactMapper">
<contactMapper type="SitecoreHacker.Sandbox.Facets.Mappers.InnerElementDictionaryMapper, SitecoreHacker.Sandbox"
patch:after="contactMapper[@type='Sitecore.ListManagement.Import.FacetedMapper, Sitecore.ListManagement']"/>
</contactMappers>
</contactMapper>
</import>
</sitecore>
</configuration>
Note that I’m putting this before the FacetedMapper. I’m doing that because I just want to make sure that this is near the top.
Summary
Once it’s all said and done, after running through the CSV import process, your xDB contact model will look like this!
Sitecore Hacker 
Leave a Reply