This document relates to the "Lower the Entry Barrier" initiative, codenamed "Ramp".

Here I will go into more detail about some of the concepts. These will probably be broken into individual documents.

Concepts to cover:

• Template & Resource Loading Cascade
• Inplace editing of Templates & Resources: Access filesystem
• Config By File: Configuration & Bootstrap improvements
  
  • Work with config more like templates & resources.
  • JCR Document View or JSON. Autoloading. Autoexporting
  • Maybe it works like code in that it creates a dynamic configuration every time - not a permanent change to the configuration tree.
• Better Javascript handling.
• ConfigTree improvements. Extends insight. Template Extender tool.
• Enable frontend developers to create apps & fields with templates.

References

STK 2.0 Workshop

http://wiki.magnolia-cms.com/download/attachments/42270723/stk-remarks-vpro.pdf

Template & Resource loading cascade & Filesystem loading

Templates & Resources (and any other file assets) should be treated equivalently where possible.

Current problems

• Resources in repo are used by default (While templates in classpath are used by default)
• Its hard to get Resources into the repo. (reinstall). Adding new resources must be easy.
• Developers prefer to work on resources in the file-system - they are using IDE's.
• Storing templates in modules requires a java development environment.
  
  

Proposal

In order to stay flexible and support different use cases - system should implement a "loading cascade": loading things in a specified way from a specified series of locations
By default, Templates/Resources should be loaded in the following order. Items loaded later override the previous ones. The order goes from least flexible to most flexible.

1. Modules (classpath).
2. Directory on the filesystem.
3. Repository that have "override" box checked.
   
   1. A master configuration item causes all items from repository to override.

(Alternatively, perhaps the mechanism only loads templates/resources that are not yet defined, instead of overwriting as defined above, in which case the default order would be reversed.) 

Filesystem loading

• System should watch all files in configured directory and when changed
  
  • reload them in the system
  • add them to the repository, reload them to the repository

Questions:

• How to handle theme images?
• Would GIT be a fourth option for a loading cascade? Could be -  but people could just use normal git to sync things on the filesystem. So maybe only the filesystem option is necessary.
  
  

To consider -

• maybe things should actually not be loaded to repository. This means you cannot do a clean uninstall.
• Should templates and resources really be loaded to separate locations in separate workspaces? What is the advantage of having these things scattered, instead of together?

Config by File

Meta: Break this topic into independent sub-topics:

• Readable bootstrap files.
• Config By File - No writing to repository
• Config By File - Writing to repository

What if configuration for templates and dialogs could be done in a file instead of in the adminCentral configuration tree?
This comes out of thinking of ways to treat resources and templates more equivelently, ie in a standard way. And then realizing that developers like working with files and have a lot of familiarity with tools and workflows there.

Why does a developer prefer to work on resources and templates as files (in an ide or nice text editor) rather then in adminCentral? 

Key points

• Files have clean syntax (Not JCR “system view”) (xml like "document view", json, ...)
• Files are applied dynamically. - Maybe at system start and everytime a file changes.
• AdminCentral can export this file format.
• Working on configtree in AdminCentral, Either:
  
  • Auto export config files on change.
  • You are seeing what is in files - and your edits are immediately written to the files.

Proposals for Consideration

1. The format of the bootstrap file is "clean", BUT the bootstrapping behaves the same. Loaded once at install, version updates.
2. The files are loaded every time magnolia starts, and the repository nodes are overwritten.
3. The files are loaded at mag start and used for item definitions, but the repository is not written to or used at all by them.
4. The files are loaded at mag start and used for item definitions, repo is not written to...
   
   1. and adminCentral ConfigTree shows a view of the files.
   2. Edits in adminCentral are saved to the files immediately.
      
      

Expanding on Proposal 2:

Any config in a file is loaded to repository at mag startup - completely overwriting the root node of the file and all children.
We need to ensure that these are loaded in a consistant, predictable order - as parts of files could be overwriting each other. 

Potential problems:

Deleting a file.

Because they are loaded everytime - whatever is in the file is in the configuration. But if one deletes a file - that configuration probably stays the same. Unless we build a mechanism that if a configfile does not exist for a node, it automatically deletes. But that strategy assumes everything is configured by file, which is unlikely.

What if I change a node in the configTree. Will this get overwritten the next time I change the file?

Yes. Could we handle this like the templates? An extra property on a node "preserve=true", that prevents the ConfigByFile from overwriting that node.
Alternatively, if writes to the configTree were automatically written back to the ConfigFile - that would solve that. 

How will this interact with bootstrap loaded configurations?

ConfigFiles are loaded every start, so when we change file - we know the repository will be in proper state. But bootstraps are only loaded once.

• What if Bootstrap & Configfile define the same node?
  
  • Bootstrap is applied first, Configfile overwrites.
• So what if I have a bootstrap - then I add a config file, but then later I take some nodes out of the configfile. This causes a problem because changes made by the original config file are now persisted in the repo - even though we want and expect them to be dynamic
  
  • If I re-install the module, the bootstraps run again - so that would fix that problem.
  • Possible solution: Bootstraps & MVH write to the "stored configuration". ON startup, "stored configuration" is copied to "active configuration" and all of the ConfigFiles are applied to it. This ensures that changes in ConfigFiles are never persisted.. they are totally volatile.
    
    • Problem with this is manual changes in adminCentral ConfigTree. How can they then be persisted?
      • What if all manual changes were actually written to yet another level: "manual configuration"? Maybe manual configuration is just written to another ConfigFile which is always applied last. This would solve the overwrite problem - and it would be easier to see what manual changes should be merged back to files in source control.
      • A variation is that many different "manual configuration" files would be written - cooresponding to the same ConfigFile structure - or at the very least one "manual configuration" file per module.

How do people override things specified in module ConfigFiles?

• Do they still use MVH?
• I think they just write their own ConfigFile fragments

How would ConfigByFile work with MVH that people have written?

• I guess it continues to work the same. 

Analyzing implications of ConfigByFile

ConfigByFile vs ConfigByCode

Thought experiment: what if all bootstraps were installed everytime you restarted admin central?

So you would treat hand edits to the configtree as only experimental and temporary - like using firebug or browser dev-tools. You would know you have to "persist them" officially somehow. Hotfixes no longer possible.
Additionally - imagine that anytime the bootrap was changed it is reloaded. (problem with ordering, what if two bootraps write the same node, 2nd one overwrites. Then you edit the 1st file. Now repo is incorrect.) 
So you could do your "hotfixes" by editing the files themselves.
Maybe a solution could be that if you edit in adminCentral - your changes are saved to a "manual change" layer - which is actually stored in another repository or file. This can be viewed independently and is therefore easy to re-integrate.

Thought experiment: could the repository know from which file all config comes from? What if whenever a bootstrap is loaded, an additional propertly is written to the root node in the config tree - the filepath of the bootstrap. So from any node you can walk up the tree to determine which bootstrap actually wrote that value. This would kind of be like "source maps" in web resources. What about MVH, would they also write a note to root of every node they change? Or would this eliminate need for MVH?

Editing files vs Editing Configuration - the two seem antagonistic.

Inplace editing of Templates & Resources: Access filesystem

What if nothing was stored in repository?: Not templates or resources?

• Templates and resources do not get stored in repository.
  • Admincentral apps load the files from filesystem for display.
  • inplace editing - edits the files directly. (In exploded War)

That does not work for JARS, the files cannot be written to.

ConfigByFile - Autowrite to File on ConfigTree edits.

Proposals for Consideration: (How to find file)

• Filename is written as a property to the root node of the configuration it represents. So on change, it can walk down the tree till it finds the filename (or flag).
• Unlike bootstraps, ConfigFiles always start at a specific level. There is a separate file for each app, each dialog, and the basic "config" node for a module. As this is set - its easy for the system to generate the filepath of the file and check if it exists.
• All files are scanned to find matching file.

Questions

If a user makes a ConfigTree change to a node that is not created by ConfigByFile, does system create a file for this manual change?

If a user makes a ConfigTree change to a node that is created by ConfigByFile, but by another module - it's not correct that this change be written to the file of another module. If anything it should just be written to the "project module". As the project module has super dependencies, its always loaded last and so its ConfigFiles would be applied last, mimicking the manual change.

HTML Modules

This means a few things.

• In AdminCentral, you can create a module. Why? - a place to store your configuration of new apps etc.
• On the filesystem, you can populate a directory - and all contents of that directory are loaded and treated similar to a module - dependencies, etc (what else explicitly?)

How could that work? Maybe there is actually one java module which is a container for all of the "dynamic" "javafree modules".

What does the filesystem structure look like? It should probably match existing configuration and in-module file structures as closely as possible.

• There is one root directory where the running system looks for these filesystem based modules. The filepath is configurable.
• Each subdirectory is considered as a module. Each subdirectory contains an module.xml file which acts like a pom file, declaring dependencies and version. It or another file also act as module descriptor.
• ....

Javafree Template

What would the file structure be for a component template stored in a directory. 
templateName.xml could be the template definition that ties everything together.

But if everything follows conventions, is this file actually necessary? If the directory holds a template script, a dialog configuration, a model, resources - then we dont need an xml file to redundantly point to all of these things again. Less boilerplate. Less chance to write something wrong.

• Dir: myYoutubePlayer
  
  • myYoutubePlayer.ftl
  • myYoutubePlayerDialog.xml
  • myYoutubePlayerModel.js
  • resources
    
    • whatever.css
    • whatever.png
    • whatever.svg
    • whatever.js
    • whatever_en.properties
    • libs
      
      • superyoutube
      • parquetteLayout