Importing and exporting JCR data for bootstrapping

This page deals with importing and exporting data to and from the JCR. Magnolia provides functions to export data from a JCR workspace into a file, and to import data by file into a JCR workspace. This process is known as Bootstrapping. 

Importing and exporting JCR data is useful when migrating content from one system to another, to provide configuration data or when making a backup. You can choose between XML and YAML formats.

Bootstrap files

File formats

Magnolia supports two formats for file-based JCR data.

• JCR System View XML format - Subsequently referred to as XML format.
• YAML format

Data can be exported and imported using either format. 

All features and functions work for both formats.

File names match the path of the data in the repository, such as website.my-page.xml or website.my-page.yaml. When you export a node, its children are also exported.

XML

Magnolia can export JCR data to XML format. This format is specified by the JCR specification (see JCR System View XML). Magnolia uses this format for importing and exporting JCR data.

The exported XML reflects the hierarchy of the data in the repository. 

<sv:node xmlns:sv="http://www.jcp.org/jcr/sv/1.0" xmlns:rep="internal" xmlns:mix="http://www.jcp.org/jcr/mix/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:jcrfn="http://www.jcp.org/jcr/xpath-functions/1.0" xmlns:fn_old="http://www.w3.org/2004/10/xpath-functions" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:mgnl="http://www.magnolia.info/jcr/mgnl" xmlns:nt="http://www.jcp.org/jcr/nt/1.0" sv:name="my-page">
  <sv:property sv:name="jcr:primaryType" sv:type="Name">
    <sv:value>mgnl:page</sv:value>
  </sv:property>
  <sv:property sv:name="jcr:uuid" sv:type="String">
    <sv:value>701efb1f-c53b-4830-87b5-873776798d80</sv:value>
  </sv:property>
  <sv:property sv:name="hideInNav" sv:type="Boolean">
    <sv:value>false</sv:value>
  </sv:property>
  <sv:property sv:name="mgnl:created" sv:type="Date">
    <sv:value>2017-05-10T11:40:24.968+07:00</sv:value>
  </sv:property>
  <sv:property sv:name="mgnl:createdBy" sv:type="String">
    <sv:value>superuser</sv:value>
  </sv:property>
  <sv:property sv:name="mgnl:lastModified" sv:type="Date">
    <sv:value>2017-05-10T11:40:34.569+07:00</sv:value>
  </sv:property>
  <sv:property sv:name="mgnl:lastModifiedBy" sv:type="String">
    <sv:value>superuser</sv:value>
  </sv:property>
  <sv:property sv:name="mgnl:template" sv:type="String">
    <sv:value>mtk:pages/basic</sv:value>
  </sv:property>
  <sv:property sv:name="title" sv:type="String">
    <sv:value>This is my page</sv:value>
  </sv:property>
</sv:node>

YAML

The YAML-based JCR data file is Magnolia specific and not part of the official JCR specification.

The structure of the data in a YAML export is comparable to the XML-based JCR data file because it also reflects the hierarchy of the data in the repository. However, YAML files have a simple format. This makes them smaller, easier for humans to read and simpler to diff, merge and edit. These characteristics make YAML versions of JCR data particularly useful, for example, in the case of configuration data that needs to be edited by several team members during the development process.

'my-page': 
  'hideInNav': false
  'title': 'This is my page'
  jcr:primaryType: 'mgnl:page'
  jcr:uuid: '701efb1f-c53b-4830-87b5-873776798d80'
  mgnl:created: 2017-05-10T11:40:24.968+07:00
  mgnl:createdBy: 'superuser'
  mgnl:lastModified: 2017-05-10T11:40:34.569+07:00
  mgnl:lastModifiedBy: 'superuser'
  mgnl:template: 'mtk:pages/basic'

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

Importing and exporting JCR data with apps from the Admin UI

Many Magnolia content apps that store their data in JCR workspaces provide actions to import and export JCR data. Some examples: Pages app, Assets app, Contacts app, Categories app.

Additionally, the JCR Tools app allow you to operate on data in all Magnolia workspaces, including those where the Export and Import actions are not available in the workspace-specific app, for example in the Security app.  

Note that all the apps mentioned, including the JCR Tools app, rely on the default import and export actions defined in the module magnolia-ui-admincentral.

Default import and export actions

The default import and export actions are used in content apps and the JCR Tools app.

The Import and Export actions use the JCRImportCommand and JCRExportCommand commands. The commands are defined in the magnolia-ui-admincentral module and implemented in JcrImportCommand and JcrExportCommand classes.

The export action can also contain filters.

Exporting binary data (such as images or videos) from the JCR to YAML throws an exception. Magnolia does not allow you to export binary data to YAML files in order to prevent excessive memory consumption.

Export filters

The default export action provides the ability to apply filters to improve readability and omit data you do not need outside of Magnolia. Filtered properties are not exported to the XML or YAML file. However, when you import the file into the JCR repository again, the JCR node created contains all properties according to its node type definition.

The default export action defines a filter for the configuration workspace and excludes the following properties:

• jcr:created, jcr:createdBy, jcr:uuid
• mgnl:activationStatus, mgnl:created, mgnl:createdBy, mgnl:lastActivated, mgnl:lastActivatedBy, mgnl:lastModified, mgnl:lastModifiedBy

You can adapt the filter or apply other filters for other workspaces according to your requirements.

Exporting and importing from content apps

The screenshot below is based on the Pages app. However, importing and exporting JCR data works in the same way on every content app which provides these actions. Note that you can add these actions to your custom content apps too.

Export and Import actions are available in the action bar and in the context menu (right-click).

When you import a file, the imported nodes become children of the selected parent node.

To export:

1. Select a node to export the node and its children.
2. Click Export.
3. Choose a format: XML or YAML.

To import:

1. Select a parent node under which you want to import the nodes.
2. Click Import.
3. Choose a format: XML or YAML.

Exporting and importing from the JCR Tools app

The import and export tools in the JCR Tools app allow you to operate on data in all Magnolia workspaces, including those where the Export and Import actions are not available in the workspace-specific app, for example in the Security app.  

To export:

1. Select the Workspace where the content resides.
2. In Base path, type the path to the node to export.
3. Select Format: XML or YAML.
4. Select the type of Compression: None, ZIP or GZIP.
5. Click Execute.

To import:

1. Select the Workspace into which the content should be imported.
2. In Base path, type the path into which content should be imported.
3. Browse to the file to import.
4. Select how to handle conflicting UUIDs. These options only apply when the file to import contains a UUID - remember that the UUID can be omitted dring export - and when an identical UUID already exists in the repository.
   
   • Only import if no existing node only import and generate a new UUID if the node does not already exist.
   • Generate a new id for imported nodes will result in a new UUID being generated for nodes being imported.
   • Remove existing nodes with the same id will result in nodes with the same UUID as those imported being deleted before the import.
   • Replace existing nodes with the same id  will result in nodes with the same UUID as those imported being replaced with the imported nodes.
5. Click Execute.
   

Importing bootstrap files from modules

As a developer, you typically adapt configuration data and content on your development environment. You then need to make it accessible on the production environment without manually importing the bootstrap files. Magnolia provides mechanisms to automatically import bootstrap files, for example when a module is installed for the first time.

The bootstrapping mechanism, directory location and management for Maven modules and light modules is different. See:

• Bootstrapping in Maven modules.
• Bootstrapping in light modules. 

Importing and exporting JCR data with Groovy scripts

You can execute Groovy scripts on a running instance to both export and import JCR data. This type of export or import only supports XML file format. See the Groovy module and Groovy app for information about how to execute Groovy scripts on a running Magnolia instance.

Sample Groovy script to export the node /my-page from the website workspace:

import info.magnolia.importexport.DataTransporter
session = ctx.getJCRSession("website")
xmlFileOutput = new FileOutputStream("/Users/jdoe/tmp/test/website.my-page.xml")
DataTransporter.executeExport(xmlFileOutput, false, false, session, "/my-page", "website", DataTransporter.XML)
xmlFileOutput.close()

import info.magnolia.importexport.DataTransporter
import javax.jcr.ImportUUIDBehavior
xmlFile = new File("/Users/jdoe/tmp/test/website.my-page.xml")
DataTransporter.importFile(xmlFile, "website", "/", false, ImportUUIDBehavior.IMPORT_UUID_CREATE_NEW, true, true)

Importing and exporting JCR data programmatically with Java

The need to export and import JCR data using custom Java classes is very rare. Generally, using Groovy scripts is sufficient.

If you need to export JCR data, use the class DataTransporter.

try {
    File exportFolder = new File("/User/jdoe/tmp/test");
    String xmlFileName = "website.my-page.xml";
    String workspaceName = "website";
    Session session = MgnlContext.getJCRSession(workspaceName);
    String nodePath = "/my-page";

    File xmlFile = new File(exportFolder.getAbsoluteFile(), xmlFileName);
    FileOutputStream fos = new FileOutputStream(xmlFile);

    try {
        DataTransporter.executeExport(fos, false, false, session, nodePath, workspaceName, DataTransporter.XML);
    }
    finally { IOUtils.closeQuietly(fos);}

} catch (Exception e) {
    /* do something clever here */
}

When importing bootstrap files, we recommend you use task classes, which can handle bootstrap files called from your module version handler. For an example about how to use the module version handler, see create version handler. Magnolia provides tasks to handle bootstrap files out of the box, see ModuleBootstrapTask, ModuleDependencyBootstrapTask and SamplesBootstrapTask; you also can write a custom task for instance by extending AbstractTask.

Alternatively, you can use the class BootstrapUtil.