The Origin

Mike Etheridge, SharePoint Enterprise Architect and fellow co-pilot, came up with this design after we were swimming in a sea of Visual Web Parts with an offshore development firm. For years we had been trying to create a templated, dynamic, and in the truest since of the term "reusable" web part. Not just a web part single instance in a web part zone. Rather, a web part that points to a datasource and a template that could be used on any page. Once the vision became clear, we embarked on creating this web part in our pursuit of creating "SharePoint Architectural Harmony". In this post, we will explain how to build a truly reusable web part that decreased the need for customizations without compromising upgrade capability.

Query Display Template Web Part - Overview

Odds are that your current employer or client has an instance of SharePoint 2010 hanging around. According to a recent survey from Forrester, nearly 65% of companies have yet to upgrade to SharePoint 2013 on premise or online. Although in the same survey, 55% plan on upgrading in the next 18-24 months. With 70% of Fortune 500 companies engaged in SharePoint on some level, maintaining a compliant upgrade path is not only fortuitous, but mission critical. Enough percentages for you? Me too. My current client falls into this category. While we have test driven SharePoint Online and Office 365 for functional or divisional collaboration purposes, SharePoint 2010 dominates the enterprise. There are over 14 applications ranging from an enterprise intranet, branded public facing responsive websites, line of business applications, business intelligence, and application integration.

While each has been architected, coded, and deployed (automated end-to-end through all environments) to our governed development standards, they all face the daunting challenge of a SharePoint upgrade path. This situation is not unlike many other companies. As a matter of fact all companies must endure the pain, analysis, and practice of executing an upgrade at some point. I have met many clients very nervous about the upgrade to SharePoint 2013 becoming a complete rewrite. In order to reduce this pain and adhere to the new development paradigm of SharePoint 2013, many of our custom applications utilized an extensible model. In this model we either provided an out of the box solution to meet ~60% of the requirements (there I go with percentages again) or an extension of SharePoint that leveraged best practices to ease the pain of future upgrades. Of course it should be stated that SharePoint 2010 has some unavoidable farm solution customizations. You know the type when you need to deploy and configure custom branding in the form of feature receivers, list or library event receivers, web services in the vti_bin, and the smattering of timer jobs. However, when it comes to SharePoint 2010 web parts, is it true that one in particular could solve the majority of your requirements? Let's check in and see.

Extensible Model

SharePoint 2010 allowed for more sub classing of assemblies throughout the stack than previous versions. Many decompilers are used today (my favorite is dotPeek) allow for a developer to quickly review whether a class is sealed or not. In the case that they are not sealed, a developer can then extend the web part's class and enjoy all of the out of the box functionality while peppering some goodies for themselves. We all know that most data in a SharePoint application resides in lists or libraries across multiple sites. Lists and libraries are nothing more than abstracted database tables available for server side or client side consumption. The most powerful web part to consume list or library data has always been the Content Query Web Part. Branded as the "SharePoint Swiss Army Knife", we decided to start there with our vision. With lists or libraries as the datasource, we needed a way to generate a strongly typed generic assembly on the fly. In addition, we also needed to couple the extension with the ability to allow front end developers a way to bind templated data to the web part easily without server side customizations but still controlling logic and formatting. The developers could interact with list data through the client side object model or ListData.svc (among others). Putting the two together, the Query Display Template Web Part (QDTWP) was born.

DotLiquid Templates

Our first quest was a templating engine. After researching a host of options it was decided that Dot Liquid Templates was the proper engine that allows developers to bind server side data programmatically to client side objects. We combined the extended QDWP with the Liquid Template Engine in order to provide developers with the ability to work with client side libraries efficiently without needed server side customizations. The LiquidTemplate engine allows us to achieve the following:

  • Iterate over list or library datasources
  • Bind row and column (field) information client side
  • Inject any client side plug in required for functionality (scrolling, animation etc.)
  • Reduce server side customizations
  • Full control over HTML and Styling for Responsive Web Design principles
  • When editing the QDTWP properties, you simply point the "Template" URL to the template you have created
  • While developers were removed from server side customizations, they still had control over logic and formatting of data and content
  • Templates were stored in a Library at the root site called "LiquidTemplates"

Here is a snippet of how binding would be used in Liquid Templates (HTML may be injected within the templating language):

  • Output is surrounded by two curly brackets
  • Tags are surrounded by curly bracket and percent sign
  • Full control over looping and conditional statements on datasource
 {% for result in SP.QueryResults %}
 {% if result.SlideLearnMoreURL != null %}
 {% assign learnMoreUrl = {{result.SlideLearnMoreURL}} | Split: ', ' | First %}
 {{result.TeaserTitle }}
 {% else %}
 {% assign learnMoreUrl = '' %}
 {% endif %}
 {% endfor %}
Adding the LiquidTemplates Engine into our Solution

Dot Liquid Templates were added to our SharePoint solution in the following format and the assembly deployed through a package:

  • Note: The DotLiquid.dll assembly is located at the http://dotliquidmarkup.org/ site and must be added to your solution's package as either a Bin addition using Code Access Security or GAC'd. Many of our extensions in the classes below subclass interfaces and methods in this assembly.
Class Instantiation and Properties

The QDTWP is composed of the web part sub class as well as an editor class. This blog post will focus on the former with a following post to show how we achieved the editor portion. The contents of the web part file in source control is comprised of the following components:

The following code snippet shows the subclasssing of the Content Query Web Part and the properties to be set.

  • TemplateUrl - When in an editable state, the user can select which template runs the client side functionality for the QDTWP instance
  • ExtendedProperties - When in an editable state, the user can enter key/value pairs that are essentially a dictionary of serialized custom properties they can "inject" into the Web Part Properties
[ToolboxItemAttribute(false)]
 public class QueryDisplayTemplateWebPart : ContentByQueryWebPart
 {
 private LiteralControl _html = new LiteralControl();

 [Personalizable(PersonalizationScope.Shared),
 WebBrowsable(false)]
 public string TemplateUrl { get; set; }

 [Personalizable(PersonalizationScope.Shared),
 WebBrowsable(false)]
 public string ExtendedProperties { get; set; }

OnPreRender Method

This method establishes the main list, property bag, and filter values for the web part instance prior to rendering. Enables use in public facing anonymous or secured extranet or intranet zones.

 protected override void OnPreRender(EventArgs e)
 {
 if (!string.IsNullOrEmpty(TemplateUrl))
 {
 var tm = new TemplateManager();
 tm.RegisterTag("splists");
 tm.RegisterTag("sppropertybagvalue");
 tm.RegisterFilter(typeof(SPFilters));

 var templateUrl = SPContext.Current.Site.MakeFullUrl(TemplateUrl);
 var template = "";
 var httpContext = Context;
 var principal = Thread.CurrentPrincipal;
 try
 {
 SPSecurity.RunWithElevatedPrivileges(() =>
 {
 var identity = WindowsIdentity.GetCurrent(System.Security.Principal.TokenAccessLevels.MaximumAllowed);
 using (var context = WindowsIdentity.Impersonate(identity.Token))
 {
 Thread.CurrentPrincipal = HttpContext.Current.User =
 new WindowsPrincipal(WindowsIdentity.GetCurrent(System.Security.Principal.TokenAccessLevels.MaximumAllowed));
 using (var wc = new WebClient())
 {
 if (!SPContext.Current.Site.IISAllowsAnonymous)
 {
 wc.Credentials = CredentialCache.DefaultNetworkCredentials;
 }
 wc.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko");
 wc.Headers.Add(System.Net.HttpRequestHeader.Cookie, CredentialUtility.GetAkamaiAuthCookie());
 wc.Encoding = System.Text.Encoding.UTF8;
 template = wc.DownloadString(templateUrl);
 }
 }
 });
 var modBuilder = CreateModuleBuilder();
 _html.Text = tm.Render(template.Trim().Trim(new char[]{'\uFEFF','\u200B'}), new Dictionary 
 { 
 { "QueryResults", RetrieveData(modBuilder) },
 { "ExtendedProperties", RetrieveExtendedProperties(modBuilder) }
 });
 }
 finally
 {
 HttpContext.Current = httpContext;
 Thread.CurrentPrincipal = HttpContext.Current.User = principal;
 }
 }
 }
,>
Query Display Template Strongly Typed Assembly

The web part needed to dynamically bind any type of column in any list on the fly upon page rendering regardless if this was a custom list, out of the box, or derived from any content type. In order to achieve this, we created a strongly typed assembly through a standard DataTable on the fly using the following code through using CodeDom. CodeDom can be used to create a dynamic object graph an in turn change that object graph to an in memory namespaced assembly dynamically. The less than pleasing part is iterating over a DataTable to create the dynamic object's assembly on the fly. However, this also meant it worked across any column type including Lookups.

 private object RetrieveData(ModuleBuilder modBuilder)
 {
 var dynamicTypeBuilder = CreateTypeBuilder(modBuilder, "DynamicType");
 if (Data.Rows.Count > 0)
 {
 foreach (var col in Data.Columns.OfType())
 {
 CreateProperty(dynamicTypeBuilder, col.ColumnName, col.DataType);
 }
 }
 var dynamicType = dynamicTypeBuilder.CreateType();
 var ctor = dynamicType.GetConstructor(Type.EmptyTypes);
 var objArray = new List();
 foreach (var row in Data.Rows.OfType())
 {
 var obj = ctor.Invoke(null);
 foreach (var col in Data.Columns.OfType())
 {
 var item = row[col];
 if (!(item is DBNull))
 {
 var prop = dynamicType.GetProperty(col.ColumnName);
 prop.GetSetMethod().Invoke(obj, new[] { item });
 }
 }
 objArray.Add(obj);
 }
 return objArray;
 }
Strongly Typed Properties for our new Strongly Typed Object Method

In order to bind DotLiquid Templates client side using the proprietary binding semantics (double curly brackets etc.), we had to create the properties and methods that would be injected into our strongly typed object created in the following method. With a property created for the dynamic strongly typed object, a developer could then bind the data elements wherever they needed to in their HTML template using DotLiquid's syntax.

 private static void CreateProperty(TypeBuilder dynamicTypeBuilder, string propertyName, Type propertyType) 
 {
 // Our intermediate language generator
 ILGenerator ilgen;
 FieldBuilder fBuilderDataColumn = dynamicTypeBuilder.DefineField(string.Format("m_{0}", propertyName), propertyType, FieldAttributes.Private);

 PropertyBuilder pBuilderDataColumn = dynamicTypeBuilder.DefineProperty(propertyName, System.Reflection.PropertyAttributes.HasDefault, propertyType, null);
 // Custom attributes for get, set accessors
 MethodAttributes getSetAttr = MethodAttributes.Public | MethodAttributes.HideBySig | MethodAttributes.SpecialName;

 // get,set accessors for DataColumn
 MethodBuilder mGetDataColumnBuilder = dynamicTypeBuilder.DefineMethod(string.Format("get_{0}", propertyName), getSetAttr, propertyType, Type.EmptyTypes);

 // Code generation
 ilgen = mGetDataColumnBuilder.GetILGenerator();
 ilgen.Emit(OpCodes.Ldarg_0);
 ilgen.Emit(OpCodes.Ldfld, fBuilderDataColumn); // returning the DataColumn
 ilgen.Emit(OpCodes.Ret);

 MethodBuilder mSetDataColumnBuilder = dynamicTypeBuilder.DefineMethod(string.Format("set_{0}", propertyName), getSetAttr, null, new Type[] { propertyType });

 // Code generation
 ilgen = mSetDataColumnBuilder.GetILGenerator();
 ilgen.Emit(OpCodes.Ldarg_0);
 ilgen.Emit(OpCodes.Ldarg_1);
 ilgen.Emit(OpCodes.Stfld, fBuilderDataColumn); // setting the DataColumn from the first argument (1)
 ilgen.Emit(OpCodes.Ret);

 // Assigning get/set accessors
 pBuilderDataColumn.SetGetMethod(mGetDataColumnBuilder);
 pBuilderDataColumn.SetSetMethod(mSetDataColumnBuilder);
 }
Private Builder Methods

Below are two private methods that support building the dynamic object for reference.

 private static TypeBuilder CreateTypeBuilder(ModuleBuilder modBuilder, string typeName)
 {
 // The dynamicType class builder
 TypeBuilder dynamicTypeBuilder = modBuilder.DefineType(string.Format("DynamicTypeBuilder.{0}", typeName), TypeAttributes.Class | TypeAttributes.Public);
 dynamicTypeBuilder.SetParent(typeof(Drop));

 // The default constructor
 dynamicTypeBuilder.DefineDefaultConstructor(MethodAttributes.Public);
 return dynamicTypeBuilder;
 }

  private static ModuleBuilder CreateModuleBuilder()
 {
 // The assembly builder
 AssemblyBuilder asmBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(new AssemblyName("DynamicTypeBuilder"), AssemblyBuilderAccess.RunAndSave);

 // The module builder
 ModuleBuilder modBuilder = asmBuilder.DefineDynamicModule("DynamicTypeBuilder", "DynamicTypeBuilder.dll");
 return modBuilder;
 }

Closing

Once this web part was introduced to the enterprise, customizations were decreased dramatically. Brace yourself, more percentages coming...well over 80% of our requirements for a public facing responsive web design project were satisfied with this web part. Essentially in our agile approach, stories were satisified one after the other using this web part. All we had to do was deploy the .liquid files and edit our web part instances accordingly to point to the particular liquid file instance. Every functional component that read from a SharePoint list could operate through these client side templates with associated stylesheets, liquid file templates with binding, and client side code. Furthermore, if need be, we could bypass a deployment by simply updating the liquid file templates and reloading them to the Style Library (not recommended unless it's an emergency). This was an enormous advantage should a bug or large issue be found as we could correct them on the fly. Of course, the update would make its way back into source control but at least the web applications did not have to be recycled for an emergency update. In the end, for a SharePoint 2013 upgrade, we would have to ensure the content existed and update the client side logic to use the new _api functionality (if need be) and we were all set. Well ok, not all set as branding is another discussion for any upgrade but we bypass a rewrite and a major refactoring nevertheless.

The image below depicts some of the DotLiquid files that satisfied requirements:

In our next post, I will highlight the editor portion of this web part. In addition, we created another extended web part we dubbed the "ScriptManagerWebPart".