A default installation of Sitecore uses Lucene and not Solr, although using Solr is the preferred way to go when running Sitecore 8. The advantages are obviously:
- Indexing of the search indexes happens on a single server, instead of indexing the same content on all Sitecore instances.
- Sitecore recommends Solr if the number of items exceeds a little as 50,000 items (https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/search_and_indexing/indexing/using_solr_or_lucene)
- Solr offer a lot of additional functionality that can be utilized if needed (E.g. “More Like this”, Advanced Ranking options, Indexing documents etc.).
- The Community around Apache Solr are huge! There are always help to get (I recommend using the email lists, if googling doesn’t solve you problems, see resources: http://lucene.apache.org/solr/resources.html)
So, how easy is it to switch to Solr instead of Lucene?
Luckily enough there’s help to get in the Sitecore community. I find this post really good:
I have been using that post a lot of times and that has helped a lot, thanks! Now, at Alpha Solutions, we always use Solr instead of Lucene. Besides the obvious reasons listed above, we often use an awesome Search Solution (http://www.sitecoresearchsolutions.com) that’s also build on top of Solr. That makes the setup easier, and architecture simpler, only to use the same core Search Engine technology to the same: Apache Solr.
Can we patch the switch to Solr?
Since we always use Solr instead of Lucene, I set out to establish the easiest was to do that using patching of the configuration files. That could simplify some of the steps to do the switch. That rest of this blog are exactly about that: How to switch all your Sitecore Indexes from Lucene to Solr as easy as possible.
- Java SDK Java SE Development Kit 8u77 (jdk-8u77-windows-x64.exe). Downloaded from Oracle: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
- Solr: Download of latest version 5.5.0 (solr-5.5.0.zip). Downloaded from mirror at http://www.apache.org/dyn/closer.lua/lucene/solr/5.5.0
- Sitecore 8.1 update 2 (rev. 160302) and whatever needed, MS SQL, MongoDB etc.
- If you want Solr to run as a Windows Service. Get NSSM and follow this brilliant guide: http://www.norconex.com/how-to-run-solr5-as-a-service-on-windows/
Step 1: Install Java, Solr and the Solr up running
- Install Java SDK using the install file: jdk-8u77-windows-x64.exe. I usually deselect the Java Source Code:
- Extract the solr-5.5.0.zip to the root of one of you drives. I simply put it on the C drive (c:\solr-5.5.0).
- Start Solr, and check that it’s running:
Open a Command line in the Solr directory and execute the following command:
Now, open a browser to verify that Solr Admin console is running:
Step 2: Generate the basic Solr schema file from your Sitecore installation
(Note: I have created a zip file you can use – so you don’t have to do all these steps really – unless you use another version of Sitecore)
Follow the steps in this KB article: https://kb.sitecore.net/articles/227897 (as also described in https://sitecore-community.github.io/docs/search/solr/Configuring-Solr-for-use-with-Sitecore-8/) or you can follow these steps (Should be pretty much the same):
- Make changes to the basic schema file that ships with Solr (C:\solr-5.5.0\server\solr\configsets\basic_configs\conf\managed-schema):
- enclose all <field> and <dynamicField> elements in the <fields> tag.
- enclose all <fieldType> elements in the <types> tag.
- From the Sitecore Control Panel: Start the “Generate Solr Schema xml file”. Change the paths to the source schema so it fits your installation. For the output I created a new folder “Sitecore”:
Disable Solr Shema less feature
Solr now ships with the schema less feature as default. We want to disable this and use the schemas the old-fashioned way. To do that – you can follow the steps from this link: http://stackoverflow.com/questions/29819854/how-does-solrs-schema-less-feature-work-how-to-revert-it-to-classic-schema – in short:
- Edit solrconfig.xml (Copy the this file from C:\solr-5.5.0\server\solr\configsets\basic_configs\conf to our Sitecore conf folder: C:\solr-5.5.0\server\solr\configsets\sitecore
- Search for <schemaFactory class=”ManagedIndexSchemaFactory”> and comment the whole element
- search for <schemaFactory class=”ClassicIndexSchemaFactory”/> and uncomment it
- search for the <initParams> element that refers to add-unknown-fields-to-the-schema and comment out the whole <initParams>…</initParams> if you get a hit.
- While we are at it – also add <fieldType name=”pint” class=”solr.TrieIntField”/> so it looks like this:
(Note again: I have created the files for you – so unless you use another version of Sitecore and Solr – you don’t have to do this work).
Step 3: Prepare and organize your Solr core structure
I like to have a separation between configuration and data. Default Solr cores has a folder structure like:
Instead I like to have the structure like:
So for each index we need to create a folder for each of our Sitecore indexes:
To create the first core follow these steps:
- Copy all files from C:\solr-5.5.0\server\solr\configsets\basic_configs\conf to the same folder:
- Delete the “managed-schema” file in the sitecore_analytics_index\conf folder.
- Copy the schema.xml and the solrconfig.xml we generated previously (C:\solr-5.5.0\server\solr\configsets\sitecore) to the C:\solr-5.5.0\server\solr\cores_conf\sitecore_analytics_index\conf folder
- Create a property file that tells Solr where the configuration files are, and where to place the data files:
- Create a text file in the sitecore_analytics_index folder and name it: core.properties
- Make sure to create the content of the file has the following line: dataDir=../../cores_data/sitecore_analytics_index
- Check that Solr locates the core on startup:
- Restart Solr by using the two command lines below, or restart the service if you installed NSSM.
- Open Solr admin interface on http://localhost:8983 and check if you see the core. You should see “sitecore_analytics_index” in the core selector:
- From here on, it’s just a matter of copying and renaming the core we just created. For each core do the following:
- Duplicate the sitecore_analytics_index folder. Rename it to [new-core-name]
- Edit the core.properties in the [new-core-name] folder, change the only line to:dataDir=../../cores_data/[new-core-name]
- Repeat the process until you have following cores created:
- Restart Solr and verify that you have all the cores up running in Solr.
Step 4: Patch the configuration to use Solr for all the indexes instead of Lucene:
- I created a folder “zzzSolrIndexes” and placed this in App_Config\Include. In this folder I have copied all the Solr configuration files that ships with Sitecore as .disabled or .example. So that’s a pretty long list of files:
- Remove the .example and .disabled from the above files.
- Disable all the currently enabled Lucene indexes by a single patch file:
Create a file in our zzzSolrIndexes folder, and ensure that this patch fill will be executed before all your Solr configurations. I named the patch file: aaaSitecore.ContentSearch.DisableAllLuceneIndexes.config
The content of that file is where we simply remove the Lucene indexes – one by one:
<?xml version="1.0" encoding="utf-8" ?> <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/"> <sitecore> <contentSearch> <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch"> <indexes hint="list:AddIndex"> <index id="sitecore_analytics_index" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider"> <patch:delete /> </index> <index id="sitecore_master_index" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider"> <patch:delete /> </index> ……
Repeat the lines for all the index ids. You can use the lists of the cores we created in the beginning of this blog. You should end up with disabling 15 indexes (Sitecore 8.1 update 2).
Note: you can download a copy of these files instead – see download and useful links at the end of this post.
Step 5: Install the Solr support packages from Sitecore
- Download the Sitecore support package from Sitecore that matches the Sitecore version.
There’s support for a number of number of IoC containers. Choose the one that you are comfortable in using, or already uses as part of you project.
- The packages uses the global.ascx for initiating the Solr provider. We generally recommend to do this initialization in a Pipeline instead – leaving the global.ascx as-is.
Step 6: Check that it works!
- Re-index it all
- Go to the Control Panel
- Choose Indexing Manager
- Select All
- Click “Start”
- Switching to the Solr interface you should be able to see the indexes getting content:
Note: I actually got an error in Solr telling that field “__display_name_t_da” was missing. Sitecore uses Solr Dynamic Fields. If you experience these error you have to add the missing dynamic field configuration. For the above, this will be in the schema.xml file in all the cores:
- Do some testing and watch the Solr logs for errors – check both Solr and Sitecore logs.
Step 7: A couple of hints
- SolrNet – You now have the option to use SolrNet for querying your content. SolrNet makes it possible for you to use a lot of Solr feature that is not possible to utilize from the Sitecore’s ContentSearch API.
- There has been some annoying bugs in the ContentSearch implementation for Solr during the 8.0 versions. Watch out for those as well – you will most like be able to see those on the Solr logs. For Sitecore 8.0 update 2 the Content Search API queried Solr using the __templatename instead of _templatename.
Useful links and downloads:
- Download the Solr files I created for this to work here – must be placed in C:\solr-5.5.0\server\solr folder
- Download the Sitecore config files for Sitecore 8.1 rev 160302 here – must be placed in C:\inetpub\sc81rev160302\Website\App_Config\Include