Z3950 Enhancements

Status:	unknown
Sponsored by:	Rijksmuseum
Developed by:	marcelr
Expected for:	2011-06-27
Bug number:	Bug 6536
Work in progress repository:
Description:	Z3950 Enhancements: SRU search targets, MARC conversion and additional XSLT processing

Include SRU search targets

This makes it possible to include SRU targets (supporting MARC21 or UNIMARC schema) in the Z3950 search. The results are merged as always. To make this work, add specific server-dependent SRU options on the host line of the search target between colons (LIKE: sru=get,sru_version=1.1,hostname-bla-bla) and provide the field mapping in the sru_fields column.

The first sru option in hostname (i.e. the starting chars "sru=") makes Koha recognize that this search target is not a Z3950 server, but a SRU server. The option sru= should be followed by get (probably in most cases), post or soap. Other options could be sru_version, schema, etc. Schema is relevant if the SRU server by default returns another schema such as Dublin Core.

The field mapping in sru_fields tells how to translate the field names for title, author, isbn, etc. for this particular SRU server. If a fieldname is not supported, it will be replaced by the mapping for any[where]. The following fields are used: title=,author=,isbn=,issn=,lccn=,any=,dewey=,subject=,controlnumber=,stdid= (Add the appropriate server index after the equal sign; if no translation follows the equal sign or the field is removed, any[where] will be used instead.)

Include MARC21/UNIMARC conversion

When a record is imported into the reservoir, it is converted to the system MARC flavour if needed. This allows users to search on MARC21 as well as UNIMARC targets and import records from these targets. Two system xslt files are added. The conversion from MARC21 to UNIMARC is the most complete one and has been created by BibLibre. The conversion from UNIMARC to MARC21 currently only supports the most important MARC fields and should gradually be extended to cover remaining fields.

Include additional XSLT processing (BibLibre enhancement)

In addition to the eventual MARC conversion, it is also possible to run a user xslt stylesheet over the external search results. This should be indicated on Z3950 target [server] level. This step is run after the MARC conversion; so it assumes that the record is in the system MARC flavour already. This could be used to correct certain characters, fields or subfields from a particular Z3950 server. Another application could be to convert e.g. Dublin Core to your system MARC flavour, it a SRU target would not support a MARC schema.

Code refactoring

Merge common code of cataloguing and acquisition z3950 script into Breeding module. A few small optimizations have been made in the code. Mainly, the code has been rearranged into some more subroutines.

Test plan

To test the patches, follow these steps. (These 12 steps will not cost you so much time.)

1) Apply both patches. See bugzilla.

2) Run the db revision with step 3 of the webinstaller: enter your staff client's URL followed by cgi-bin/koha/installer/install.pl?step=3 resulting in something like "http://yourlibrary.com:8080/cgi-bin/koha/installer/install.pl?step=3". Note: If you would run this step more than once, the second time you should get two error messages warning you that the columns srufields and add_xslt are already present. This is not a problem. This patch will not update your koha version number while testing; after it gets pushed, it will only update once and increase your version number.

3) Go to Administration/Z3950 Servers. Add one or more SRU targets.

Example: host sru=get,sru_version=1.1,library.rijksmuseum.nl, port 9998, database biblios (MARC21) srufields: title=title,author=author,isbn=isbn,issn=,lccn=,any=any,dewey=,subject=subject,controlnumber=,stdid=

Example: host sru=get,sru_version=1.1,schema=marcxml,z3950.loc.gov, port 7090, database Voyager (MARC21) srufields: title=title,author=creator,isbn=bath.isbn,issn=bath.issn,lccn=,any=,dewey=,subject=subject,controlnumber=,stdid=

4) Start a Z3950 search in Cataloging on the target of the previous step. Look for a search term with <20 results and another with >20 results. Check the outcome. Use a few of the supported indexes (title, author, subject, etc.) Check the MARC record by clicking on the link in the Preview column.

5) Start a Z3950 search in Acquisitions on the same target. (Open a basket, add order from external source.) [See comments under point 4.]

6) If you use MARC21, add a UNIMARC Z3950 target to test MARC conversion.

Example: host carmin.sudoc.abes.fr, port 210, database abes-z39-public (UNIMARC, ISO_5426)

7) If you use UNIMARC, add a MARC21 Z3950 target.

Example: host catalog.nypl.org, port 210, database innopac (MARC21)

8) Start a Z3950 search (from Cataloging or Acquisition) on the target you just added. Check the MARC record.

9) Test additional XSLT processing. Create a XSLT file that transforms MARC records in your MARC flavour. (Or use an existing one; if you need a 'dumb' example, copy the following lines.) Save the file and update the add_xslt column of one search target with the path to that file. (A relative path means that the file is found in koha-tmpl/intranet-tmpl/prog/en/xslt)

Example XSLT (for MARC21; just switching chars in the title):

 <xsl:stylesheet version="1.0"
       xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
       xmlns:marc="http://www.loc.gov/MARC21/slim">
 <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
 <xsl:template match="/">
         <xsl:apply-templates/>
 </xsl:template>
 <xsl:template match="marc:datafield[@tag='245']/marc:subfield[@code='a']">
   <marc:subfield code="a">
       <xsl:value-of select="translate(current(),'ae', 'ea')"/>
   </marc:subfield>
 </xsl:template>
 <xsl:template match="node()">
   <xsl:copy select=".">
     <xsl:copy-of select="@*"/>
     <xsl:apply-templates/>
   </xsl:copy>
 </xsl:template>
 </xsl:stylesheet>

10) Start a Z3950 search (from Cataloging or Acquisition) on the target you just edited. Check if the MARC record shows the transformation you expected.

11) Start a Z3950 search on multiple targets. Include a SRU target, Z3950 target, MARC21 target and UNIMARC target.

12) Edit and delete some targets in Administration. Check the list of targets each time.

Future work

Extending the UNIMARC to MARC21 conversion.
Default value for srufields if a new server is a SRU target. When adding a target, test with javascript if hostname starts with string "sru=" and update srufields with title=,author=,isbn=,issn=,lccn=,any=,dewey=,subject=,controlnumber=,stdid=
Do you have another suggestion?

Z3950 RFC

Contents