Enhancing Sitecore Search Results with GeoJson and Solr

Recenty I had a requirement to perform a region location search to get all relevent content within boundries of given region polygon. Few quick google searches resulted in Solr having built-in functionality for GeoSpatial search based on JTS library (jts-core-1.15.0.jar). However, JTS library needs to be copied to each Solr node "SOLR_INSTALL/server/solr-webapp/webapp/WEB-INF/lib/" folder (if running SolrCloud). Solr server needs to be restarted to properly load this library. More about Solr JTS library Solr.

I have also found helpfull implementation for Sitecore 9 on Polygon Search Sitecore 9 so need to give kudos to author of this blog post too. However, as I am using Sitecore 10.2 some of the Content Search constructors are different than in Sitecore 9 version and I found it too cumbersome to write Linq to Sitecore extensions for my implementation. Therefore, instead of using queryable extensions I did direct call to Solr endpoint to get required data. Furthermore, my implementation uses Sitecore to deploy new Solr field, instead of upading Solr schema manually.

Jumping into implementation. Once we have Solr configured we can add new custom field to Solr Schema. To do this we need to extend SchemaPopulateHelper class. When deployed we need to go to Sitecore CMS and trigger Populate Solr Schema. To confirm all went well and new field is in schema we can access schema from Solr app or directly check schema.xml file on Solr server.


using System.Collections.Generic;
using System.Linq;
using System.Xml.Linq;
using global::Sitecore.ContentSearch.SolrProvider.Pipelines.PopulateSolrSchema;
using SolrNet.Schema;
public class CustomPopulateHelper : SchemaPopulateHelper
{
  public CustomPopulateHelper(SolrSchema schema) : base(schema)
  {
  }
  public override IEnumerable<XElement> GetAllFields()
  {
      return base.GetAllFields().Union(GetAddCustomFields());
  }
  public override IEnumerable<XElement> GetAllFieldTypes()
  {
      return base.GetAllFieldTypes().Union(GetAddCustomFieldTypes());
  }
  private IEnumerable<XElement> GetAddCustomFields()
  {
      yield return CreateField("*_grpt",
          "geojson_rpt",
          isDynamic: true,
          required: false,
          indexed: true,
          stored: true,
          multiValued: false,
          omitNorms: false,
          termOffsets: false,
          termPositions: false,
          termVectors: false);
  }
  private IEnumerable<XElement> GetAddCustomFieldTypes()
  {
      yield return CreateFieldType("geojson_rpt", "solr.SpatialRecursivePrefixTreeFieldType",
          new Dictionary<string, string>
          {
              { "indexed", "true" },
              { "spatialContextFactory", "JTS" },
              { "geo", "true" },
              { "format", "GeoJSON" },
              { "autoIndex", "true" },
              { "validationRule", "repairBuffer0" },
              { "distErrPct", "0.025" },
              { "maxDistErr", "0.001" },
              { "distanceUnits", "kilometers" }
          });
  }
}

Custom field added to index configuration with new Solr field added to schema as returnType.


  <field fieldName="polygon" returnType="geojsonrptgeom">Custom.Indexes.Polygon, Custom.Assembly</field>


using Sitecore.ContentSearch;
using Sitecore.ContentSearch.ComputedFields;
using Sitecore.Diagnostics;
using System;
  public class Polygon : IComputedIndexField
  {
      public string Parameters { get; set; }
      public string FieldName { get; set; }
      public string ReturnType { get; set; }
      public object ComputeFieldValue(IIndexable indexable)
      {
          Assert.ArgumentNotNull(indexable, "indexable");
          try
          {
              if (!(indexable is SitecoreIndexableItem indexableItem))
              {
                  Log.Warn(string.Format("{0} : unsupported IIndexable type : {1}", this, indexable.GetType()), this);
                  return null;
              }
              if (indexableItem.Item?.TemplateID != Templates.Templates.Global.RegionPolygonData)
              {
                  return null;
              }
              return indexableItem.Item?.Fields["polygongeom"]?.Value;
          }
          catch (Exception ex)
          {
              Log.Error("Error while indexing Polygon", ex, this);
              return null;
          }
      }
  }

Solr connection string is taken from Sitecore connection strings config and if Basic Authentication is enabled on Solr it will have to be added in header request. Worth to note to select only necessary fields in Solr query as this will speed things up significantly. In my case I only needed location name, item Id and item name.


  public async Task<IEnumerable<LocationSearchResultName>> GetRegionNames(double lat, double lng)
  {
      var regionNames = new List<LocationSearchResultName>();
      var geoFieldName = "polygon_grpt";
      using (var client = new HttpClient())
        {
            var query = $"{YourIndexName}/select?fl=location_t%20and%20_group%20and%20_name&q={geoFieldName}%3A\"Intersects({lng}%20{lat})\"&wt=json";
            //Set basic auth authorization header if user info exists in uri
            var uri = new Uri(Sitecore.Configuration.Settings.GetConnectionString("solr.search"));
            if (!string.IsNullOrEmpty(uri.UserInfo))
            {
                var byteArray = Encoding.ASCII.GetBytes(uri.UserInfo);
                client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", System.Convert.ToBase64String(byteArray));
            }
            var response = await client.GetAsync($"{Sitecore.Configuration.Settings.GetConnectionString("solr.search")}/{query}");
            var responseContent = await response.Content.ReadAsStringAsync();
            if (!response.IsSuccessStatusCode)
            {
                MyLogger.Log.Warn($"{responseContent}");
            }
            var docs = JObject.Parse(responseContent);
            regionNames = docs["response"]["docs"].ToObject<List<LocationSearchResultName>>();
        }
      return regionNames;
  }

Solr response is deserialized into model.


using Newtonsoft.Json;
public class LocationSearchResultName
{
  [JsonProperty("location_t")]
  public string LocationName { get; set; }
  [JsonProperty("_name")]
  public string Name { get; set; }
  [JsonProperty("_group")]
  public string ItemId { get; set; }
}

Polygon GeoJson data field (MultiLine text) is quite large and it might take some time to load content within region boundries dynamically on page load. Therefore, we have custom field on content item where on scheduled content import we try to associate it with region. Additionally for manual step to associate already existing content items I have created Sitecore command and Schedule that can be triggered only manually by right clicking context item or listing item and selecting Sitecore Powershell Extensions script to trigger scheduled task. Additional, computed index field created for regions field helps to get data quickly to front end.

GeoData Polygon Sitecore Field

Content item regions field. Item also have Sitecore's built-in Coordinate field, Latitude and Longitude are passed into Polygon search method. Each content item can have multiple regions assigned. There is additional field not shown in screenshot that represents which region type it is (Country, County etc).

Regions Multilist Field with Search

Troubleshooting and Hints

Direct query in Solr app to find location, can be done by using:
```
  Intersects(11.111111 11.11111) 
```
If there is error when indexing item containing GeoJson polygon, then it probably has wrong format. There should be no Properties object in Json and should look as one shown in screenshot above. Easiest way to determine faulty item is to try to index it from Sitecore Developer's tab. It will show error message if field is wrong. Solr and Sitecore error logs will not tell us exact field name/id that have bad formatting and in my case it just stopped indexing completely, so might be worth to have this data in seperate index.
GeoJson data can be taken from GeoPortal for UK and will have to be slightly formatted to remove Properties object and match Solr requirements.