CAUTION: this page simply describes the ideas and discussion of the first iteration of the DAM 2.0 & CMIS implementation effort. Some things are still up to date - others have been changed completely.

 

Introduction

The objective is to have a CMIS integration for our DAM module, both for exposing the DAM over CMIS and for connecting to a CMIS server.


Goals

As a user I can browse assets on a CMIS server in the Assets app.

As an editor I can use assets from a CMIS server in my pages by selecting them in a dialog.

As an administrator I can configure one or more CMIS providers in the DAM.

 

UPDATE

As of January 22nd were shifting focus from the assets app onto the choose dialog. This is driven by the realization that browsing a read only external server is of questionable use. It's more relevant to have different sources to pick from in the choose dialog.

It will still be necessary to build this UI, at least for providers with a folders and files layout, using our standard components that we use for content apps.

Also, in order for renditions and linking to use properly we will need to have proper providers. They will just not be accessible from the Assets app.

 

Scope

Write operations on a remote CMIS server is out of scope. Note that this contradicts using the Assets app on top of the DAM API only and tying the CMIS server to the DAM API rather than the DAM workspace. (UPDATE: both of these are out scope now).

 

Infrastructure setup

We have a Sharepoint installation to test with. https://www.evernote.com/shard/s249/sh/4401996f-4fc4-4748-ba1f-ff52b590b980/f5e00b6dff287197c2fe0a2a67721bcd

Alfresco has a test server that we can use for testing. http://www.alfresco.com/cmis

 

Implementation plan

Imaging module that can work on images from the dam.

Updated DAM API with new support for:

  • folders
  • provider registry
  • provider capabilities (implementation option TBD)

CMIS provider that supports listing folders and reading assets.

Out of scope is:

    • Versioning
    • Activation models
    • User mapping

A mechanism for serving assets from any provider, similar to what the URI2Repository mapping for DAM does now.

Extension to the Link api that supports linking to assets.

Updated RichTextField that can use and save references to assets from any asset provider.

Updated STKAssetRenderer that doesn't rely on JCR and uses new imaging module (see above).

CMIS server module that uses the DAM workspace.

For the App we see a plan in two steps:

  1. We develop a prototype, a separate app that uses only the DAM API. It will use a custom provider and have custom actions all based on the DAM API.
  2. We evaluate the prototype and decide to
    1. either merge the prototype into the Assets app and use the DAM API also for everything that is currently done directly on the JCR.
    2. or we merge in the prototype as a provider specific view, enhancing the assets app with this functionality.

 

Topics

 

DAM API improvements

We need a number of improvements to the DAM API to make it capable of supporting the needs of the Assets app and the CMIS server.

Folder support

The DAM API needs to have a concept for folders to make the assets navigable. Folders do not have metadata and cannot be linked to (href links, referencing them is supported) nor do they have renditions.

We will introduce a new interface in the API called AssetFolder and a number of additional methods related to folders, primarily on the AssetProvider interface.

AssetElement

  • AssetElement.getName()
  • AssetElement.getIdentifier()
  • AssetElement.getPath()
  • AssetElement.getAssetProviderIdentifier()
  • AssetElement.getParent()
  • AssetElement.getDepth()
  • AssetElement.isFolder()
  • AssetElement.isAsset()
  • AssetElement.getLastModified() (might not make sense on folders)

AssetFolder

  • AssetFolder.getChildren()

AssetProvider

  • AssetProvider.getRootFolder()
  • AssetProvider.getFolderForPath(String absPath)
  • AssetProvider.getFolderForId(String absPath)
  • AssetProvider.getAssetElementForPath(String absPath)
  • AssetProvider.getAssetElementForId(String id)

 

Modification operations

Currently we only have support for reading assets. We need to expand the API to allow for:

  • creating a folder
  • deleting a folder
  • renaming folder
  • moving a folder
  • creating an asset
  • deleting an asset
  • duplicating an asset
  • renaming an asset
  • setting metadata on an asset

Should we have operations for copying and moving between providers?

Capabilities discovery

Not all providers will have the same feature set. We need a mechanism in which users of the API can test for certain capabilities.

Option 1

An enum of capabilities were a provider either supports or doesn't support a certain one. The provider has two methods, one for getting all the supported capabilities and one for testing if it has a certain one.

Example: provider.hasCapability(Capabilities.VERSIONING)

Option 2

A dedicated type returned by the provider with query methods for each capability. The advantage being that it can have methods return boolean, integers or enums where a set of responses make sense. Such as NONE, SOME, ALL.

Example: provider.getCapabilities().isVersioningSupported()

Option 3

Use a key-value based approach that is extendable to allow for querying for properties not explicitly defined in the api. i.e new providers exposing capabilities that have not been defined before-hand. This should not use enums.

Example: Boolean.valueOf(provider.hasCapability(Capabilities.VERSIONING))

Querying

We currently have an AssetFilter class with basic properties for searching for specific assets. It contains a string that's used as an addition to the where clause. This will obviously not work for other providers,

Maybe we could use a criteria like api.

We could allow for passing the extra where clause in multiple languages, where JCR-SQL2 would be one and CMIS would be another.

Other topics

Paths must be strictly validated to not allow bypassing security by adding in . or .. references.

Deleted nodes need special attention, we might need to make this part of the api.


Provider registry

There is currently no way to configure providers. The registry is hard coded to have only the JCR provider.

Option 1 - Observed Manager

An observed manager that looks for asset providers under each module node.

Option 2 - DAM module config node (DECIDED)

We use the config node in the dam module. This has the benefit of making ordering easier.

Additionally

We should make it possible to configure which media types are contained on a certain provider.

 

Renditions

Renditions are provided by implementations of the AssetRenderer registry. Every media type is tied to a particular renderer. There is currently two renderers, JcrAssetRenderer that simply generates a link starting with /dam, no resizing or other image operations are supported. The other is STKAssetRenderer which is described in the section on STK improvements.

We need to have renderers that are provider independent.

 

JCR provider improvements

JcrAsset opens an input stream always, whether its used or not. It is likely not closed properly.

JcrAsset should not act like a DTO, it should delegate back operations to the provider for operations such as opening the input stream.

 

Assets app

The assets app works directly on the JCR never going through the DAM API. As we add more providers this has to change to make it operate on remote data.

Option 1 - Using DAM API only

Expand the DAM API enough to make it support the entire feature set of the Assets app.

This includes activation and dealing with deleted nodes. We would basically need to write new actions that call into the jcr asset provider where it does the same thing as the current actions.

Option 2 - Provider specific views (DECIDED)

Allow the app to have custom views per provider allowing for an optimized user interface.

The JCR view would be identical to what we have now, accessing the JCR directly. For CMIS we would have a new view that would use the DAM API only. It would be usable against all providers with similar functionality as CMIS.

Providers such as Flickr and youtube which don't have traditional files-and-folder would have very customized views.

We must also update the choose dialog and give it custom views.

 

Optional

  • Introduce uses/subtabs in Asset detail view/dialog (UI Concept)
  • Introduce metadate in Asset detail view/dialog (UI Concept)
    • Add metadata extractor logic based on the uploaded MediaType
  • Add MediaType previews 
    • Video
    • Sound
    • Documents
  • Add MediaType Editor (like image editor)
    • Video --> select and create cover/thumbnail image
  • DamUploadFile
    • Use Asset
    • Use transformer
  • Create a vaadinImage component that perform the bridge between Asset's and the Imaging module.

This mockup shows what we're aiming for. Note the tabs for each media type and the source selectors beneath within each tab.

CMIS provider

The CMIS provider should be able to communicate with a CMIS server using both the web service and the atom binding. It should be possible to specify a path within the remote repository to use as root path.

There's a number of things we need to decide on here:

Versioning

If we decide to support versioning we need to define this in the DAM API and add the appropriate methods of

  • list versions
  • get version
  • create version
  • restore version

Activation

Should we support a model where a customer has a public and an author cmis repository and wants to use activation to transfer from one to the other?

We would want to prevent activation of a page if its using an asset in CMIS that hasn't been activated.

The activation status must be a available on the asset.

We've identified four models where activation comes in:

  • same server for authoring and public
  • same server, flag that makes public, magnolia should flag (flag is just an example, it could be a complicated mechanism)
  • different servers, magnolia should copy
  • different servers, servers copy, magnolia invokes the server to copy (basically same as model 2)

Authentication and authorization

There's two options here, we could decide to support both or support just option 1 to begin with.

Option 1 - Use a single user on the CMIS server for all Magnolia users (DECIDED)

 

Option 2 - Mapping users in Magnolia to CMIS server users

 

  • mapping users requires some kind of sso otherwise we'd need to store passwords for the other end in Magnolia

  • configured per cmis provider
  • anonymous must have access

Renditions

  • commonly used for thumbnails
  • we could take advantage of this for thumbnails
  • however it should not be used as our renditions that need to work independently of provider
    • we could though use our renditions for as cmis renditions returned with the cmis server module

Mapping metadata

CMIS supports properties on documents and custom properties.

TODO can we support the full dublin core this way?

Querying

CMIS has its own query syntax. How we can map onto this in the DAM API?

Session management

  • session reuse, can it be reused between threads?
  • use same session for request, i.e. thread local, is there an advantage to this?
  • do we need to limit the number of connections executing at the same time? globally for all users
  • must investigate the parallelism and thread safety of the api

Caching

  • key folder listings per path and user (see above)

Requirements

M5 CMIS client module may access a CMIS server in order to read documents.

The server has to be easily configurable

CMIS Client module has to implement the AssetProvider interface.

 

CMIS server module

Two options here, either we expose the DAM JCR as we did for the DMS in 4.5, or we expose the DAM API and every provider.

Option 1 - Expose DAM workspace (DECIDED)

Essentially the same as the 4.5 compatible module but fitted for the DAM workspace instead.

Option 2 - Expose arbitrary DAM provider

Every provider in DAM is exposed as a CMIS repository. All functionality that we want support needs to be available on the DAM API.

 

Out of scope

  • Versioning
  • Mapping our renditions to CMIS renditions

 

Current state

The current CMIS module is now able to access DAM repository.

  • Create/Read Folders
  • Create/Read Assets

A direct access to the DAM repository is currently performed. New Implementation step will involved the usage of Assets Provider to expose Assets.

Implementation is pushed under: https://git.magnolia-cms.com/gitweb/?p=modules/cmis.git;a=shortlog;h=refs/heads/magnolia-module-cmis-2.0

Workbench CMIS client url:

http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws/cmis?wsdl

DAM module reorganization

Proposal:

  • dam-api
  • dam-jcr
  • dam-app
  • dam-templating

 

DAM API refactoring proposal

TODO greg

 

UI framework improvements

The rich text field stores references to assets it links to. It must be able to support linking to asset from any provider. It is currently hard wired to work only for JCR assets.

TODO research this

 

Link API improvements

Update / Review the core Link API. This API has to be able to handle Asset's composite keys.

See the section on UI improvements.


STK improvements

STKAssetRenderer uses imaging to resize, crop and transform images. Imaging is tied to the JCR and can't be used for assets from other providers.

We need to redesign this.


Additional tasks

 

  • Upgrade to latest Apache Chemistry
  • Test with Adobe Drive

 

Notes

Links

M5 DAM API

4.5 CMIS Server Module

CMIS Worbench

Logging configuration

org.apache.chemistry.opencmis.binding.spi.type=webservices
org.apache.chemistry.opencmis.binding.webservices.RepositoryService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/RepositoryService?wsdl
org.apache.chemistry.opencmis.binding.webservices.NavigationService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/NavigationService?wsdl
org.apache.chemistry.opencmis.binding.webservices.ObjectService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/ObjectService?wsdl
org.apache.chemistry.opencmis.binding.webservices.VersioningService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/VersioningService?wsdl
org.apache.chemistry.opencmis.binding.webservices.DiscoveryService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/DiscoveryService?wsdl
org.apache.chemistry.opencmis.binding.webservices.MultiFilingService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/MultiFilingService?wsdl
org.apache.chemistry.opencmis.binding.webservices.RelationshipService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/RelationshipService?wsdl
org.apache.chemistry.opencmis.binding.webservices.ACLService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/ACLService?wsdl
org.apache.chemistry.opencmis.binding.webservices.PolicyService=http://localhost:8080/magnolia-bundled-webapp/.magnolia/cmisws11/PolicyService?wsdl
org.apache.chemistry.opencmis.user=superuser
org.apache.chemistry.opencmis.password=superuser
org.apache.chemistry.opencmis.binding.compression=true

Appendix - Proposed API

Appendix - Current API and implementation

 

 

  • No labels