Concept - Explicit Full Text Parsing Configuration

Abstract

Ever since upgrading to JackRabbit 2.x and relying on Tika parsers for full text indexing, people have struggled with configuring those indexes. This concept is about making full text parsing configuration explicit and thus allowing Magnolia users to easily customize it to tune performance of their instances.

Implementation

• Modify jackrabbit-xxx-search.xml

  • add <param name="tikaConfigPath" value="${rep.home}/../../tika-config.xml"/>

• Place following file to ${rep.home} upon installation:

  <properties>
    <detectors>
      <detector class="org.apache.tika.detect.DefaultDetector"/>
    </detectors>
    <parsers>
      <parser class="org.apache.tika.parser.DefaultParser"/>
      <parser class="org.apache.tika.parser.EmptyParser">
        <!-- Disable package extraction as it's too resource-intensive -->
        <mime>application/x-archive</mime>
        <mime>application/x-bzip</mime>
        <mime>application/x-bzip2</mime>
        <mime>application/x-cpio</mime>
        <mime>application/x-gtar</mime>
        <mime>application/x-gzip</mime>
        <mime>application/x-tar</mime>
        <mime>application/zip</mime>
        <!-- Disable image extraction as there's no text to be found -->
        <mime>image/bmp</mime>
        <mime>image/gif</mime>
        <mime>image/jpeg</mime>
        <mime>image/png</mime>
        <mime>image/vnd.wap.wbmp</mime>
        <mime>image/x-icon</mime>
        <mime>image/x-psd</mime>
        <mime>image/x-xcf</mime>
        <!-- Disable video extraction as there's no text to be found -->
        <mime>video/mpeg</mime>
        <mime>video/mp4</mime>
        <mime>video/ogg</mime>
        <mime>video/quicktime</mime>
        <mime>video/webm</mime>
        <mime>video/x-matroska</mime>
        <mime>video/x-ms-wmv</mime>
        <mime>video/x-flv</mime>
        <!-- If you are having issues w/ indexing PDF uncomment following: -->
        <!--mime>application/pdf</mime-->
      </parser>
    </parsers>
  </properties>
  
  

The alternative to the above would be to have such config file located directly in the workspace and configure full text parsing per workspace.

The above configuration is proposed as default since it removes most of resource intensive indexing while having minimum impact on available data for full text search. For DAM resources we still rely on extracted metadata to locate such resource, we are not using full text search index to locate them.