Abstract

Ever since upgrading to JackRabbit 2.x and relying on Tika parsers for full text indexing, people have struggled with configuring those indexes. This concept is about making full text parsing configuration explicit and thus allowing Magnolia users to easily customize it to tune performance of their instances.

 

Status

(tick) ready for implementation

Implementation

  • Modify jackrabbit-xxx-search.xml
    • add <param name="tikaConfigPath" value="/info/magnolia/jackrabbit/tika-config.xml"/>

  • And have tika-config.xml by default contain following:

    <properties>
  <detectors>
    <detector class="org.apache.tika.detect.DefaultDetector"/>
  </detectors>
  <parsers>
    <parser class="org.apache.tika.parser.DefaultParser"/>
    <parser class="org.apache.tika.parser.EmptyParser">
      <!-- Disable package extraction as it's too resource-intensive - if you need archives indexed, enable index extraction by removing this section and recreating the index -->
      <mime>application/x-archive</mime>
      <mime>application/x-bzip</mime>
      <mime>application/x-bzip2</mime>
      <mime>application/x-cpio</mime>
      <mime>application/x-gtar</mime>
      <mime>application/x-gzip</mime>
      <mime>application/x-tar</mime>
      <mime>application/zip</mime>
      <!-- Disable image extraction as there's no text to be found - if you need to extract meta tags, enable tag extraction by removing this section and recreating the index -->
      <mime>image/bmp</mime>
      <mime>image/gif</mime>
      <mime>image/jpeg</mime>
      <mime>image/png</mime>
      <mime>image/vnd.wap.wbmp</mime>
      <mime>image/x-icon</mime>
      <mime>image/x-psd</mime>
      <mime>image/x-xcf</mime>
      <!-- Disable video extraction as there's no text to be found - if you need video meta tags, enable extraction by removing this section and recreating the index -->
      <mime>video/mpeg</mime>
      <mime>video/mp4</mime>
      <mime>video/ogg</mime>
      <mime>video/quicktime</mime>
      <mime>video/webm</mime>
      <mime>video/x-matroska</mime>
      <mime>video/x-ms-wmv</mime>
      <mime>video/x-flv</mime>
      <!-- If you are having issues w/ indexing PDF uncomment following: -->
      <!--
      <mime>application/pdf</mime>
      -->
    </parser>
  </parsers>
</properties>

The alternative to the above would be to have such config file located directly in the workspace and configure full text parsing per workspace.

The above configuration is proposed as default since it removes most of resource intensive indexing while having minimum impact on available data for full text search. For DAM resources we still rely on extracted metadata to locate such resource, we are not using full text search index to locate them.

 

Default configuration is placed in org/apache/jackrabbit/core/query/lucene/tika-config.xml and contains following (JR 2.6.2):

<?xml version="1.0" encoding="UTF-8"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at       http://www.apache.org/licenses/LICENSE-2.0
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<properties>
  <detectors>
    <detector class="org.apache.tika.detect.DefaultDetector"/>
  </detectors>
  <parsers>
    <parser class="org.apache.tika.parser.DefaultParser"/>
 
    <parser class="org.apache.tika.parser.EmptyParser">
      <!-- Disable package extraction as it's too resource-intensive -->
      <mime>application/x-archive</mime>
      <mime>application/x-bzip</mime>
      <mime>application/x-bzip2</mime>
      <mime>application/x-cpio</mime>
      <mime>application/x-gtar</mime>
      <mime>application/x-gzip</mime>
      <mime>application/x-tar</mime>
      <mime>application/zip</mime>
      <!-- Disable image extraction as there's no text to be found -->
      <mime>image/bmp</mime>
      <mime>image/gif</mime>
      <mime>image/jpeg</mime>
      <mime>image/png</mime>
      <mime>image/vnd.wap.wbmp</mime>
      <mime>image/x-icon</mime>
      <mime>image/x-psd</mime>
      <mime>image/x-xcf</mime>
    </parser>
  </parsers>
</properties>

 

 

  • No labels