Main features

• Images will be generated on demand (not on save as in magnolia-module-image-filtering-1.x)
• Generated images can be reused (i.e do not store 2907 copies of the same menu item image)
• Avoid passing parameters in url to avoid hacks
• URLs should be kept relatively short and easy to generate
• Store/cache rendered content in a separate workspace. Solve issues related to cache, etc. (generated image needs to be in synch with their "source")
• Performance and scalability are issues we've solved with a few tricks in a few different places in customer projects; consolidate these solutions in this module

Target features

Features using this module could be:

• Image galleries (thumbnail creation) : resize/crop
• Banner generator (2.24)
• Teasers, ...

Usage scenarios

• Designer configures filters or dialogs to generate desired images (grayscale, resized, added border and text overlay) so that authors only have the minimal amount of options. (upload or select a background image, enter the text to overlay)
• ...

Sample use cases

"N days left to register to Magnolia conference"

Approach A

• Specific Text ImageOperation: can calculate N based on a configured date, prints it. (extend AbstractTextOverlay)
• Doesn't need a specific ParameterProvider
• Background can be either image specified through Parameter or configured
• Concerns:
  • generated-images-cache: generated image node could be removed through a scheduled task (or cache could be disabled, relying solely on page-level cache)
  • page-level cache: scheduled task could also flush the specific url every day.

Approach B

• Specific ParameterProvider (or something akin to the StringParameterProvider) - N is basically passed through the url
• Doesn't need a specific ImageOperation (or a simple PassedText if I don't commit it - it's currently not in the codebase)
• Background image can be either configured or specified through Parameter if further specializing the ParameterProvider
• No concerns about generated-images-cache nor page-level cache.

Concepts

ImageOperation

The info.magnolia.module.imagefiltering.operations.ImageOperation interface is a simple interface to manipulate images. We provide a wrapper around java.awt.image.BufferedImageOp, so any existing implementation of this interface should be reusable. (see link to the pixels project)

ImageOperationChain is an implementation of ImageFilter that delegates to a list of other ImageOperation s. By making it the default implementation for c2b and using the CollectionPropertyHidingTransformer, we can achieve a readable configuration of chained filters.

Working "on" an image (i.e cropping an existing image) is also be an ImageOperation, i.e the first in the chain, which will load the image.

ImageGenerator

The info.magnolia.imaging.ImageGenerator interface is the entry point to image generation. The default implementation is also ImageOperationChain.

Image generation parameters can come from various places:

1. in the ImageOperation configuration itself
2. in the content node we're generating an image for (example: crop coordinates, piece of text, ...)
3. in another node ? (example: navigation menu item text)
4. in the URL ? (path or parameters) (if the template displaying the image determines some of the parameters/variations to use)

ParameterProvider and ParameterProviderFactory

ParameterProvider encapsulates a parameter. One single ParameterProvider is passed to an ImageGenerator. That doesn't mean there is only a single parameterizable option or item for the whole generated image, because the Parameter itself can be any type of object (from a String to a Content node to an HashMap).

ParameterProvider implementations are typed, bound to the type of parameter they encapsulate. (ie ParameterProvider<Content> wraps a Content instance). ImageOperation implementations can be written such that they ignore the Parameter type, or if they don't, they need to be written against a specific type of ParameterProvider (using generics). This makes them "stronger" (less guess-work when it comes to getting parameters). If operations are well separated (i.e do one single thing), it is fairly easy to come up with multiple implementation of a similar operation that uses different types or Parameters (see info.magnolia.imaging.operations.text or info.magnolia.imaging.operations.load)

The info.magnolia.imaging.ParameterProvider and info.magnolia.imaging.ParameterProviderFactory couple is meant to abstract away where and how the parameters are passed to an ImageGenerator.

ParameterProviderFactory implementation are written such that they can instanciate a specific type of ParameterProvider (existing implementations of ParameterProvider are only typed implementation of the interface) for a given "environment" (a servlet request, a workitem, ...)

Parameter vs Configuration

It is important to differentiate parameter and configuration. While the whole image generation is dynamic, the parameter is the only "attribute" that will make different requests generate different images. I think as little as possible should be parameterized, and as much as possible should be configured.
Take for example the info.magnolia.imaging.operations.text.TextFromNode operation: the text that is added to the image is not configured nor is it a parameter. The parameter is the "location" of the text (the node), and the configuration determines which property of the given node to use for the text.
The same could be applied for a cropping operation: the source would also be a node, and the configuration would tell us which property(ies) of that node store the CropInfo.

The configuration might seem a little redundant with, for instance, dialogs configuration (because the dialog itself will also have info on how/where to store the CropInfo), but it also helps having simpler URLs (see above); we also stay relatively strongly-typed (by avoiding a "big pot" of parameters in the form of a HashMap).

Status

• Lots of TODOs, see code.
• Absolutely no caching, threading, etc. This should be hidden/abstracted away from the components that exists at the moment. (separation of concerns)
• Http headers need to be set appropriately.

Next steps

Additional filters to provide as examples:

• "Default" cropping, i.e cropping that works without a UI, cropping properly (centered) to the configured ratio (i.e of the target size)
• Border (adds configurable borders around pictures)
• Watermark (adds a static piece of text in a corner of the picture)
• AuthorInfo (adds a dynamic piece of text in a corner of the picture)
• TextForNavigation ? (i.e uses navTitle as dynamic text..?)

Having a repeatable control (upload n images, upload+crop n images, ...) would be helpful but not something that should implemented in this specific module. MME-29@jira

Try it out!

• Build and deploy the magnolia-module-imaging module. Other modules not needed at the moment. (crop-ui does nothing afaik, imaging-tools just adds the page that displays the available ImageIO providers)
• Bootstrap config.modules.imaging.config.generators.xml from the temp-bootstrap folder in the appropriate location.
  (you'll probably need to restart since the imaging module had no config node when you started up)

The current setup/examples all use the one servlet mapped to /.imaging/*, the first pathInfo element determines the ImageGenerator name, the second the workspace and the rest is the path of the node to load. This is handled by ImageServlet (generator name) and ContentParameterProviderFactory (a ParameterProviderFactory which loads a node based on the request uri).

Example #1

• Create a page called "test" in the website repo
• Go to http://localhost:8080/magnolia-empty-webapp-4.1-SNAPSHOT/.imaging/sample-chain1/website/test.jpg
• This uses an image background loaded off the web (which is why it's slow). ("website/test" must exist because that's how the current ProviderParameterFactory behaves)
• It also resizes the image and adds a fixed text to it.

Example #2

• With STK, create a dummy "Section" page under "test" called "hello", and in there an "Article" page called "world".
• In this new page, upload a teaser image. Also type in something for the "abstract".
• Go to http://localhost:8080/magnolia-empty-webapp-4.1-SNAPSHOT/.imaging/sample-chain2/website/test/hello/world.jpg
• This now uses an image loaded off the path "website/test"; the propertyName of the binary is configured in the ImageOperation (/generators/sample-chain2/operations/img)
• It also resizes the image and adds to texts 1) a fixed text 2) the "abstract" text from the "test/hello" node (this property name is also configured in the ImageOperation)

Example #3

• Create a folder in dms called "myFolder" (in the URL column, i.e the node name). Create a document in there ("myDoc" - in the URL column, i.e the node name) and upload an image to it.
• Go to the generator configuration (/generators/sample-chain2/operations/) - change the propertyName of the img operation to document (instead of teaserImgBinary)
• Change the propertyName of the text1 operation from abstract to subject
• I suppose that by you've guessed that you should go to http://localhost:8080/magnolia-empty-webapp-4.1-SNAPSHOT/.imaging/sample-chain2/dms/myFolder/myDoc.jpg, and that what this does is fairly obvious

Wood !

Try the wood chain too

Background information

Complex problems

We had to solve a bunch of quite complex problems in the past, specifically with the Seat project. Have a look at http://seat.de/seat_/magazin.html or see the attached screenshot. There are a bunch of generated images on that page:

1. top image - left (given background photo, text, curve shape "cutting" in the original photo)
2. menu items on the top right (text, generated background from #1, which can include part of the photo)
3. menu items on the bottom right (similar, usually no photo)
4. menu items on the bottom left (simple text, plain background; colors depend on the "context", depending on the page, the background might be black, white, ..)

Issues to solve:

• Avoid generating N copies of the same image.
• Get various parameters, encode/decode URLs, ...
• Performance: with so many images to generate, it was necessary to queue and pool the image generation jobs.
• Dependencies/locking: since #2 and #3 depend on #1, we needed some way to ensure that #1 was generated first, no matter what order the http requests came in
• Instance synchronization: TODO.

Discussion about parameters

URL schema

Considering most images are generated "for" a given node, or based on a given node, (which can be an image, a piece of text...), we could assume the following URL schema:

http://localhost:8080/magnoliaPublic/image-generator/<ImageGenerator name>/<Parameters>
                      \_>contextPath \_>Servlet                             \_> workspace and path to node

Where Parameters could be further decomposed into:

  /<workspaceName>/<home/foo/bar/myPic>.<extension>

This is assuming that ImageOperation implementations know how to get the parameters they need, either through the given node (i.e they now that the cropper configuration is stored as the property or subnode "mgnl:cropperInfo"), or through their own configuration.

Other Parameters decoding can be implemented, for instance where the path elements would be, respectively, the locale, the bundle name (or path) and key to a text message: /de_CH/info/magnolia/module/templatingkit/messages/link.readon.gif (or rather, with a configured filter, de_CH/link.readon.gif, where the bundle name would be configured at ImageOperation level.

If we have such strict URL schemes:

1. it is fairly easy to build urls
2. it is fairly easy understand the mechanisms involved, while still protecting against malicious parameters (although the weak point would be the node path)
3. we can store generated images under the same path in the generated-images workspace. Path to the generated image node would simply be <ImageGenerator name>/<Parameters>.

Problems / other approaches

Using AggregationState causes issues because the AggregationFilter checks if the path is a page node or a binary (dms nodes are neither, for instance)

See Proposal - merging of URI2RepositoryMapping and VirtualURIMappings
See Proposal - overhaul and refactoring of AggregationState

Using UUIDs

We could have a system that generates a unique ID for image/set of parameters. The parameters are then stored in the generated-images workspace. The generated uuid is not the jcr:uuid.

• this ensure uniqueness: one ID = one image. If the parameters/sources change, the ID will be different. This is good for caches, proxies, ...
• complete safety (nothing can be tempered with in the uri)
• mechanisms involved are a little more obscure to understand: URLs are more complicated, a helper class HAS TO be used to generate image links (whereas with the proposed mechanism, templaters could write their links "manually"), ...
• templates (through the use the link building class) are more tightly coupled with the ImageFilters (even though there are already IS coupling since the template has to know the filter name and what the <ImageFilterParameters> URL portion should be (but this should usually be simple and straighforward)

In any case, there needs to be some guarantee that the content upon which the image generation is based has not changed; if a new background image has been uploaded, or text changed, the generated images needs to be regenerated.

Generating links and images

If we have a helper class that generates uuid-based links (see above), then why not trigger the image generation already at that moment? This was probably the first thing that was removed from the Seat implementation, but considering the following, wouldn't it be an option now ?

• Image generation is threaded; happens in separate thread(s) (or a pool of)
• Image generation is blocking: if an http request comes in, requesting image #152, it is blocked until the image is available (alternatively, it could be sent with a "404" or other temporary image - possibly after a timeout)
• We use java.util.concurrency for managing threads and tasks, so it could be fairly clean too.

Conclusion

I have the feeling none of these approaches is essentially contradicting.

• Strict URL schemes: the most straightforward. Image is generated upon request. (if needed)
• Generating links and images: to be validated. Image generation task is launched upon link generation.
• UUIDs with deferred generation: Link generation stores the image "job" (definition of the job, parameters), but the image is still generated upon request (if needed)

Links

• Generic filters: http://www.jhlabs.com/ip/filters/index.htmlprovides a nice api which we could reuse for all "effects" type of filters (grayscale, colors, blur, etc) - pretty trivial as all filters have an empty constructor and setters for their specific parameters. (this is what I used for my grayscale p.o.c)
  • jhlabs/filters has been in the process of moving to https://pixels.dev.java.net/ since 2007; I had an email from the author regarding this at the time, not sure what the current status is, as his own webpage has not been updated to reflect this yet.
  • I will actually help directly on this project and have a complete new release (the current one in the maven central repo is outdated, and doesn't correspond to the sources delivered on the homepage)
• There is still a lot to learn: Using Java 2D's Image-Processing Model: http://www.informit.com/articles/article.aspx?p=1013851