Solr deployment

OUTDATED, see Bug 12538

Using Koha with Solr

• This page summerize things about solr.
• Work at kohacon11

Solr setup

This page is a Work in progress, if you have feedback, ask @clrh

Setup java

$ apt-get install openjdk-jre

On some systems (debian unstable) you need to

$ sudo apt-get install openjdk-6-jre

Validation step

$ java -version
java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8.13) (6b18-1.8.13-0+squeeze1)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

Setup Solr

Todo

<PATH> can be something like /home/you/dev/apps

git clone git://github.com/clrh/solrinstall.git <PATH>/solr

# Move into
cd <PATH>/solr
# Create the logs directory
mkdir logs

# Create the webapps directory
mkdir webapps

Feel free to get another Solr version and test it (the current is 3.6 solr and lucene versions) - get the solr archive (download link and choose a mirror) http://lucene.apache.org/solr/mirrors-solr-latest-redir.html - unzip or untar it - find the war in <solr_archive>/dist/apache-solr-x.x.x.war

# Example
koha@xxx-dev:/tmp$ wget http://www.eu.apache.org/dist/lucene/solr/3.6.2/apache-solr-3.6.2.tgz
koha@xxx-dev:/tmp$ tar -xvzf apache-solr-3.6.2.tgz
koha@xxx-dev:~/solr/webapps$ cp /tmp/apache-solr-3.6.2/dist/apache-solr-3.6.2.war solr.war

• The war must be names "solr.war"

git clone git://github.com/clrh/solrconf.git <PATH>/solrconf
mkdir <PATH>/solrconf/multicore/koha/data
chmod 777  <PATH>/solrconf/multicore/koha/data

• Change <PATH> in <PATH>/solrconf/multicore/solr.xml
• Change SOLR_HOME and SOLR_CONF_HOME in <PATH>/solrconf/launch_solr.sh file
• Launch it: ./launch_solr.sh

Validation step

• open a browser and go to http://localhost:8983/solr/koha/admin/
• you will see the solr administration page
• you SolrAPI will be http://localhost:8983/solr/koha (nothing to see)

Install perl libs

This part needs to be reviewed...

• During kohacon12, Robin spent time to package some libraries needed, you can download them: https://depot.biblibre.com/chernandez/lib/all/ (Carefull XML::Easy is a core lib, try to not using it)

• You need to install some libraries

# I used cpanm but you can do with cpan
cpanm Data::SearchEngine::Solr
cpanm Data::Pagination
cpanm Moose
cpanm MooseX::Types

• At the end you should have (install pmtools package):

# cat test_vers.sh 
#!/bin/bash

echo "Data::Paginator"
pmvers Data::Paginator
echo "Data::Pagination"
pmvers Data::Pagination
echo "MooseX::Storage"
pmvers MooseX::Storage
echo "MooseX::Type"
pmvers MooseX::Types
echo "Data::SearchEngine"
pmvers Data::SearchEngine
echo "Data::SearchEngine::Solr"
pmvers Data::SearchEngine::Solr
echo "WebService::Solr"
pmvers WebService::Solr

# ./test_vers.sh 
Data::Paginator
0.05
Data::Pagination
0.44
MooseX::Storage
0.32
MooseX::Type
0.35
Data::SearchEngine
0.33
Data::SearchEngine::Solr
0.18
WebService::Solr
0.21

• You need will their dependencies:
  • Data::SearchEngine
  • Data::SearchEngine::Solr
  • WebService::Solr

Setup Koha

Source code

bug 8233 has been pushed.

Updatedatabase

2/ Launch updatedatabase (SearchEngine syspref) - Change it in Solr for example ;)

mysql> UPDATE systempreferences SET value='Solr' where variable='SearchEngine';
# or via syspref UI

Config file

Be carefull, your koha-conf.xml file must contains a line with "installdir" like that (same level than opacdir directive): You might need to add this line.

# Example
<installdir>/home/you/sites/<INSTALL_DIR></installdir>

It should point to Koha source installation directory, in which you have etc/searchengine directory describe below.

3/ Setup config file (solr api etc.)

$ cd <PATH_KOHA_SRC>
$ mkdir -p ~/<PATH_KOHA_INSTALL>/etc/searchengine/solr
$ cp etc/searchengine/solr/config.yaml ~/<PATH_KOHA_INSTALL>/etc/searchengine/solr/config.yaml
$ cp etc/searchengine/solr/indexes.yaml ~/<PATH_KOHA_INSTALL>/etc/searchengine/solr/indexes.yaml
$ chown you:www-data ~/<PATH_KOHA_INSTALL>/etc/searchengine/solr/indexes.yaml
$ chmod g+w  ~/<PATH_KOHA_INSTALL>/etc/searchengine/solr/indexes.yaml

Change SolrAPI in config.yaml if you setup the solr core like describe before

SolrAPI: 'http://localhost:8983/solr/koha'

The indexes setup in indexes.yaml will me modifiables by web interface, you don't need to edit the file :)

Validation step

• If you add a mapping and you save it, it will be recorded in indexes.yaml file:

http://<pro.community.local>/cgi-bin/koha/admin/searchengine/solr/indexes.pl

• Another validation step:

"-i" option gives you information about your setup, very usefull to check the good SolrAPI ;)

$ ./misc/migration_tools/rebuild_solr.pl -i
SolrAPI = http://localhost:8983/solr/koha
How many indexed documents = 25001;

4/ Index you first biblio and authorities

You can now record your first 10 biblio and 10 authority::

$ ./misc/migration_tools/rebuild_solr.pl -t biblio -n 10 ; ./misc/migration_tools/rebuild_solr.pl -t authority -n 10

Please be carefull of the reset option ;) use it only if you do what your are doing where...

5/ Try opac (simple) search and give us feedback :)