How to use Solr with Sitecore 8 – the easy way

A default installation of Sitecore uses Lucene and not Solr, although using Solr is the preferred way to go when running Sitecore 8. The advantages are obviously:

So, how easy is it to switch to Solr instead of Lucene?

Luckily enough there’s help to get in the Sitecore community. I find this post really good:

https://sitecore-community.github.io/docs/search/solr/Configuring-Solr-for-use-with-Sitecore-8/

I have been using that post a lot of times and that has helped a lot, thanks! Now, at Alpha Solutions, we always use Solr instead of Lucene. Besides the obvious reasons listed above, we often use an awesome Search Solution (http://www.sitecoresearchsolutions.com) that’s also build on top of Solr. That makes the setup easier, and architecture simpler, only to use the same core Search Engine technology to the same: Apache Solr.

Can we patch the switch to Solr?

Since we always use Solr instead of Lucene, I set out to establish the easiest was to do that using patching of the configuration files. That could simplify some of the steps to do the switch. That rest of this blog are exactly about that: How to switch all your Sitecore Indexes from Lucene to Solr as easy as possible.

Prerequisites

Step 1: Install Java, Solr and the Solr up running

  1. Install Java SDK using the install file: jdk-8u77-windows-x64.exe. I usually deselect the Java Source Code:
  2. java install settingsExtract the solr-5.5.0.zip to the root of one of you drives. I simply put it on the C drive (c:\solr-5.5.0).
  3. Start Solr, and check that it’s running:

Open a Command line in the Solr directory and execute the following command:

Start Solr Command

Now, open a browser to verify that Solr Admin console is running:

Solr Admin Console

 

Step 2: Generate the basic Solr schema file from your Sitecore installation

(Note: I have created a zip file you can use – so you don’t have to do all these steps really – unless you use another version of Sitecore)

Follow the steps in this KB article: https://kb.sitecore.net/articles/227897 (as also described in https://sitecore-community.github.io/docs/search/solr/Configuring-Solr-for-use-with-Sitecore-8/) or you can follow these steps (Should be pretty much the same):

  1. Make changes to the basic schema file that ships with Solr (C:\solr-5.5.0\server\solr\configsets\basic_configs\conf\managed-schema):
    • enclose all <field> and <dynamicField> elements in the <fields> tag.
    • enclose all <fieldType> elements in the <types> tag.
  2. From the Sitecore Control Panel: Start the “Generate Solr Schema xml file”. Change the paths to the source schema so it fits your installation. For the output I created a new folder “Sitecore”:

Generate The Solr Schema File

Generate The Solr Schema File Result

Disable Solr Shema less feature

Solr now ships with the schema less feature as default. We want to disable this and use the schemas the old-fashioned way. To do that – you can follow the steps from this link: http://stackoverflow.com/questions/29819854/how-does-solrs-schema-less-feature-work-how-to-revert-it-to-classic-schema – in short:

  1. Edit solrconfig.xml (Copy the this file from C:\solr-5.5.0\server\solr\configsets\basic_configs\conf to our Sitecore conf folder: C:\solr-5.5.0\server\solr\configsets\sitecore
  2. Search for <schemaFactory class=”ManagedIndexSchemaFactory”> and comment the whole element
  3. search for <schemaFactory class=”ClassicIndexSchemaFactory”/> and uncomment it
  4. search for the <initParams> element that refers to add-unknown-fields-to-the-schema and comment out the whole <initParams>…</initParams> if you get a hit.
  5. While we are at it – also add <fieldType name=”pint” class=”solr.TrieIntField”/> so it looks like this:

Solr Schema Add FieldType pint

(Note again: I have created the files for you – so unless you use another version of Sitecore and Solr – you don’t have to do this work).

Step 3: Prepare and organize your Solr core structure

I like to have a separation between configuration and data. Default Solr cores has a folder structure like:

[core name]

[conf]

[data]

Instead I like to have the structure like:

[cores_conf]

[core name]

[conf]

[cores_data]

[core data]

So for each index we need to create a folder for each of our Sitecore indexes:

Solr Core File Structure

To create the first core follow these steps:

  1. Copy all files from C:\solr-5.5.0\server\solr\configsets\basic_configs\conf to the same folder:
    • C:\solr-5.5.0\server\solr\cores_conf\sitecore_analytics_index\conf
  2. Delete the “managed-schema” file in the sitecore_analytics_index\conf folder.
  3. Copy the schema.xml and the solrconfig.xml we generated previously (C:\solr-5.5.0\server\solr\configsets\sitecore) to the C:\solr-5.5.0\server\solr\cores_conf\sitecore_analytics_index\conf folder
  4. Create a property file that tells Solr where the configuration files are, and where to place the data files:
    • Create a text file in the sitecore_analytics_index folder and name it: core.properties
    • Make sure to create the content of the file has the following line: dataDir=../../cores_data/sitecore_analytics_index
  5. Check that Solr locates the core on startup:
  6. Restart Solr by using the two command lines below, or restart the service if you installed NSSM.Solr Command Restart
  7. Open Solr admin interface on http://localhost:8983 and check if you see the core. You should see “sitecore_analytics_index” in the core selector:Sitecore Analytics Index Core Selector
  8. From here on, it’s just a matter of copying and renaming the core we just created. For each core do the following:
    1. Duplicate the sitecore_analytics_index folder. Rename it to [new-core-name]
    2. Edit the core.properties in the [new-core-name] folder, change the only line to:dataDir=../../cores_data/[new-core-name]
    3. Repeat the process until you have following cores created:
      • sitecore_analytics_index
      • sitecore_core_index
      • sitecore_fxm_master_index
      • sitecore_fxm_web_index
      • sitecore_list_index
      • sitecore_marketing_asset_index_master
      • sitecore_marketingdefinitions_master
      • sitecore_marketingdefinitions_web
      • sitecore_master_index
      • sitecore_suggested_test_index
      • sitecore_testing_index
      • sitecore_web_index
      • social_messages_master
      • social_messages_web
  9. Restart Solr and verify that you have all the cores up running in Solr.

Step 4: Patch the configuration to use Solr for all the indexes instead of Lucene:

  1. I created a folder “zzzSolrIndexes” and placed this in App_Config\Include. In this folder I have copied all the Solr configuration files that ships with Sitecore as .disabled or .example. So that’s a pretty long list of files:Sitecore Configuration Files List
  2. Remove the .example and .disabled from the above files.
  3. Disable all the currently enabled Lucene indexes by a single patch file:

Create a file in our zzzSolrIndexes folder, and ensure that this patch fill will be executed before all your Solr configurations. I named the patch file: aaaSitecore.ContentSearch.DisableAllLuceneIndexes.config

The content of that file is where we simply remove the Lucene indexes – one by one:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/">
   <sitecore>
      <contentSearch>
         <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
            <indexes hint="list:AddIndex">
               <index id="sitecore_analytics_index" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
                  <patch:delete />
               </index>
            <index id="sitecore_master_index" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
               <patch:delete />
            </index>
……

Repeat the lines for all the index ids. You can use the lists of the cores we created in the beginning of this blog. You should end up with disabling 15 indexes (Sitecore 8.1 update 2).

Note: you can download a copy of these files instead – see download and useful links at the end of this post.

Step 5: Install the Solr support packages from Sitecore

  • Download the Sitecore support package from Sitecore that matches the Sitecore version.

There’s support for a number of number of IoC containers. Choose the one that you are comfortable in using, or already uses as part of you project.

  • The packages uses the global.ascx for initiating the Solr provider. We generally recommend to do this initialization in a Pipeline instead – leaving the global.ascx as-is.

Step 6: Check that it works!

  • Re-index it all
    • Go to the Control Panel
    • Choose Indexing Manager
    • Select All
    • Click “Start”
  • Switching to the Solr interface you should be able to see the indexes getting content:

Note: I actually got an error in Solr telling that field “__display_name_t_da” was missing. Sitecore uses Solr Dynamic Fields. If you experience these error you have to add the missing dynamic field configuration. For the above, this will be in the schema.xml file in all the cores:

  • Do some testing and watch the Solr logs for errors – check both Solr and Sitecore logs.

Step 7: A couple of hints

  • SolrNet – You now have the option to use SolrNet for querying your content. SolrNet makes it possible for you to use a lot of Solr feature that is not possible to utilize from the Sitecore’s ContentSearch API.
  • There has been some annoying bugs in the ContentSearch implementation for Solr during the 8.0 versions. Watch out for those as well – you will most like be able to see those on the Solr logs. For Sitecore 8.0 update 2 the Content Search API queried Solr using the __templatename instead of _templatename.

Useful links and downloads:

How to use Solr with Sitecore 8 – the easy way

2 thoughts on “How to use Solr with Sitecore 8 – the easy way

  • September 6, 2016 at 12:05 am
    Permalink

    Great article. I have followed all these steps and was able to rebuild all indexes without any errors (after several trials). In your blog Step-6 Check that it works!=> Can you upload some sample screen shots how to test real search engine? How to test if Solr is working on Sitecore? This must be silly question for you? But I am stuck in final part.

    Reply
  • October 5, 2016 at 2:20 am
    Permalink

    Hi Srinivas.
    Sorry – i missed you comment here. So – when you rebuild you indexes from the Sitecore Control Panel you should look at the Solr Adming console – e.g. look at the master index – http://localhost:8983/solr/#/sitecore_master_index
    You should see from that page how many documents that has been indexed – e.g. “Num Docs:17465”.

    When you work in Sitecore you should not see any difference – it should just work! And if you get a lot of items in Sitecore you will see the difference in speed.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *