Calling WCF using Content Enrichment in Content Processing Pipeline in SharePoint 2013 Search

Introduction

SharePoint 2013 (RTM) search has revamped a lot from its predecessor SharePoint 2010 search. In SharePoint 2013 we can find "Fast Search" is integrated with the "SharePoint 2010" search platform to provide an excellent search experience with less hazards. It has the back bone of SharePoint 2010 Search along with the glamour of Fast Search.

Background

Some days ago I was given a situation where the requirement is to manipulate crawled data before indexing happens. In SharePoint 2013 search we found a "Content Processing" pipeline, where there is a provision where you can invoke an external SOAP service (in my case I used WCF) to manipulate your raw crawled data and put it back to the content processing pipeline to be processed as per the defined steps.

Using the code

I have a requirement to consider those records for which the "Location" property is marked as "Calcutta" and let them make searchable when someone puts a query against "Kolkata". Therefore my target was to add a value of "Kolkata" against the "Location" filed for those records (items) where I found the "Location" value is set as "Calcutta". I am giving below the core requirements for calling an external WCF from the Content Processing pipeline. Please follow the following figure to get an idea of the location where WCF (SOAP Web Service) can be called.

The development steps are depicted below:

1. Create a WCF application from VS 2012 and add a reference to Microsoft.office.server.search.contentprocessingenrichment.dll which you can find in C:\\program files\Microsoft office servers\15.\Search\Application\External.
2. Delete the default interface (e.g., IService1).
3. Add the following references to the Service1.svc.cs file
• Microsoft.office.server.search.contentprocessingenrichment
• Microsoft.office.server.search.contentprocessingenrichment.PropertyTypes
4. Inherit Icontentporcessingenrichmentservice in the Service1.svc.cs file
5. Implement the method ProcessItem. This is the method where you get the required properties for each item.
6. Following is the sample code for implementing ProcessItem.
C#

private const string LocationProperty = "Location";
       
// Defines the error code for managed properties with an unexpected type.
private const int UnexpectedType = 1;
// Defines the error code for encountering unexpected exceptions.
private const int UnexpectedError = 2;
private readonly ProcessedItem processedItemHolder = new ProcessedItem
{
    ItemProperties = new List<AbstractProperty>()
};  
 public ProcessedItem ProcessItem(Item item)
{
   processedItemHolder.ErrorCode = 0;
   processedItemHolder.ItemProperties.Clear();
   var LocationProperty = item.ItemProperties.Where(p => p.Name == "Location").FirstOrDefault();
   Property<List<string>> LocProp = LocationProperty as Property<List<string>>;
   // WriteLog("previous step");
   if (LocProp != null && LocProp.Value.Count > 0)
   {
        //WriteLog("second step");
        string[] propValues = LocProp.Value.First().Split(';');
        if (propValues.Length > 0)
        {

            strLocation = propValues[0];
        }
        // WriteLog("zipcode is " + strzipcode);
        string  locname = strLocation.Trim();
        if(locname.ToUpper() == "CALCUTTA")
        {
            LocProp.Value.Add("KOLKATA");
            processedItemHolder.ItemProperties.Add(LocProp);
        }
   } 
}

7. Add following in <system.servicemodel> in the web.config file:
XML

<bindings>
   <basicHttpBinding>
   <!-- The service will accept a maximum blob of 8 MB. -->
      <binding maxReceivedMessageSize = "8388608"> 
         <readerQuotas maxDepth="32"
          maxStringContentLength="2147483647"
          maxArrayLength="2147483647"   
          maxBytesPerRead="2147483647"   
          maxNameTableCharCount="2147483647" />  
             <security mode="None" />
      </binding>
   </basicHttpBinding>
</bindings>

8. Host this ECF to IIS (Create a virtual directory. Map this to the physical path of the WCF application. Right click on the Virtual Directory and click on “Convert to Application”).
9. Browse and get the URL for the hosted .svc file.
10. Execute the following PowerShell script to map “Content Enrichment” to the hosted custom WCF.

$ssa = Get-SPEnterpriseSearchServiceApplication
$config = New-SPEnterpriseSearchContentEnrichmentConfiguration
$config.Endpoint = http://Site_URL/<service name>.svc
$config.InputProperties = "Location"
$config.OutputProperties = "Location"
$config.SendRawData = $True
$config.MaxRawDataSize = 8192
Set-SPEnterpriseSearchContentEnrichmentConfiguration –SearchApplication
$ssa –ContentEnrichmentConfiguration $config

11. Run a full crawl on the content source.

Points of Interest

I have considered the Location property here which is a default managed property in SharePoint Search but if you want to process a very unique property name then you need to crawl your content source first, find out your property name in the crawled properties section, which might be of type "ows_<your custom field name>", and then create a managed property of your field name and map this managed property to the "ows_<your custom field name>" property to make it available in the content processing pipeline.

This is the first part of the content enrichment call out, I will post more advanced aspects in my later articles.