Page History
...
MessagesManager implementation
info.magnolia.cms.i18n.MessagesManager
is the component that has been used to handled i18n up until Magnolia 4.5. It has its flaws, and could be rewritten.
- it tries to cover too many use-cases (either enforce passing a Locale, or never pass one)
- its responsibilities are not clear - it observes and holds some i18n config (but not all?) and at the same time provides translation support
- DefaultMessagesManager is still very tied to using property files.
- DefaultMessagesManager isn't cleanly decoupled from the system - it's still using content2bean "manually", etc; it's not a real "component".
- rething package name and/or class name (package mixes i18n for UI and for content, to start with...)
- It's not easy to test
- Some logic is burried in the
MessagesChain
class - which is where, for instance, we've been appending the ??? when a key wasn't found so far. Except that we've seen workarounds popup here and there to remove those, etc..grep -r '???' ./magnolia-core/src/main/java/info/magnolia/cms/i18n/
info.magnolia.cms.i18n.MessagesUtil
proposes too many methods; as a result, we're completely inconsistently use those in many places in our code. Get rid of this.- http://jira.magnolia-cms.com/issues/?jql=text%20~%20%22messagesmanager%22%20and%20resolution%3DUnresolved
Existing uses and inconsistencies
See #status.
Proposal
This concept and proposal focuses on in-configuration translations. Hopefully, the concepts can be applied to in-code translations. In-template translations will be taken into account (i.e facilitate the maintenance of the corresponding message files), but changing the mechanism they use might be considered a "next step".
...
- The
dialogs
prefix is not relevant and noisy. It was historically introduced to separate those labels from the page templates and page components names and description. Indeed, we're likely to have astkFAQHeader.name
somewhere. Currently leaning towards using separate message bundles. Or have an non-mandatory prefix/suffix (i.e there's a chance the component's title/name needs the same label as its first tab ?) - The
pages.faq
statement is arbitrary and is derived from the fact that the dialog in question happens to be configured under an arbitrary folder structure (pages/faq/
)
Message bundles
i18nBasename
is the property we use to look up a message bundle. This tells the system where the translation files are. It is called "basename" because i18n mechanisms typically append the locale to this and use the result as a path to a file (eg lookup <mybasename>_de_CH.properties
, then <mybasename>_de.properties
, then <mybasename>.properties
- some even go as far as going up the path hierarchy (parent folders) of the basename until they find what they're looking for)
Up to Magnolia 4.5, the i18nBasename
property was defined in a dialog definition (or further down). With 5.0, this exists in DialogDefinition
, but also in one level below (in FormDefinition
), and still exists in all elements below (TabDefinition
, FieldDefinition
, ...). The property is also present in ActionDefinition
(members of DialogDefinition
).
In 99% of the cases, the i18nBasename
property set at dialog level should be enough. It is useful to keep the possibility to redefine it in actions, forms, tabs, and fields, but it should not be necessary. Defining i18nBasename
at module level would be ideal - in terms of minimalizing redundancy anyway - but I'm not sure we'd have support for that right now. It'd be interesting to have i18nBasename in a module descriptor though. It would still be possible for individual components to override it if needed. We could also create a naming convention for i18nBasename as well (see below).
- We don't need to specifiy
i18nBasename
anymore for translatable items. (but we can, at the very least to maintain backwards compatibility) - Every module will still have their own message bundle file(s); the system will chain and look for messages in all of these
- We could imagine having a check that would warn, or even fail, when several bundles contains the same key(s).
- However we still need to be able to override messages (for projects).
- Global chains of message bundles - look into all known bundles
- Basename helps grouping translation work - "I am now translating module X" - but that doesn't mean the basename has to be specified necessarily
- Order of message bundles chain would need to be consistent and predictable
Date formats and other localized items
Message-bundles-naming-convention
Instead of having a huge fat bundle within ui-admincentral, every app should have its own bundle. The naming of an app-specific bundle should have the following pattern:
app-<app-name>-messages_<locale>.properties (e.g. app-security-messages_en.properties, app-contacts-messages_en.properties, etc. ).
It should be located directly under <module>/src/main/resources/mgnl-i18n/ (e.g. security-app/src/main/resources/mgnl-i18n/app-security-messages_en.properties)
Date formats and other localized items
We should make sure things like ColumnFormatter not only use the current user's locale, but also that this is indeed an "enabled" locale. A UI entirely in english but with a date formatted in french would be silly.
Exceptions
Exceptions are exceptions. They should not be translated. The message
of an exception is targeted at developers and admins, and it is expected that they understand english. (if only to understand the meaning of the exception class name!)
If/when exceptions are currently reported to the user, we need to actually treat the exception. The below shows how not to do this.
Code Block | ||||
---|---|---|---|---|
| ||||
try {
... something that fails ...
} catch (SomeException e) {
uiComponent.alert(e.getMessage());
} |
The below is what we should be aiming for (complete solution pending)
Code Block | ||
---|---|---|
| ||
try {
... something that fails ...
} catch (SomeException e) {
// if there is added value: log.error("sumfin happened: {}", e.getMessage(), e);
// don't feel forced to bloat the logs, though.
// if this is something the *user* should be informed about, and *can* do something about:
uiComponent.alert(i18n.getMessage(e));
// but really, is it ? Can a user do anything about a JCR Exception ?
throw new RuntimeException(e); // did you need to catch it in the first place ?
} |
Of course, the question is - what IS i18n.getMessage(e)
?
This is TBD. An idea is to have an extra method to decorate exceptions, like we suggest for non-configured texts in
Jira | ||||
---|---|---|---|---|
|
IOException
as opposed to java.io.IOException
). I don't think we should try and use the message
of the exception, because there is absolutely nothing that prevents anyone from injecting arbitrary text in there. However, some exceptions do have some sort of convention - a PathNotFoundException
's message is usage the path that was indeed not found. If possible, I'd favor using explicit properties of the exception rather than the message (getPath()
if it existed, in this case).But: this would spread the use of I18nizer
much further than it should (remember that one of the initial goals was to completely hide from the developers and have the decoration done by node2bean or Guice). Perhaps it could be "hidden" in uiComponent.handleException(e)
, and/or in a specific service. Need to think about the context, too (which would be lost if we use this last suggestion)We should make sure things like ColumnFormatter not only use the current user's locale, but also that this is indeed an "enabled" locale. A UI entirely in english but with a date formatted in french would be silly.
Implementation
Introducing a couple of concepts. The basic API will be in its own module in magnolia_main; it doesn't need to be in core, and will maybe not even depend on it. It could even be outside of magnolia_main if there is no dependency to core, but we currently lack a good location for such modules
...
magnolia-i18n
in magnolia_main
would contain contains the API and some implementationthe underlying. Key generators etc would live near their counterparts (i.e in _ui mostly)
I would simply use info.magnolia.i18n
as a package name. However, if this ends up only being usable within the context of magnolia_ui
, I would move it there instead of _main, and reflect that in the module and package names.
API
@I18nable
annotation. "Internationalizable": marks any object as a candidate for translations. Used on interfaces/classes such asFieldDefinition
. Is inherited.@I18nText
annotation. Marks a String as to be translated.I18nKeyGenerator<T>
interface. Implementations generate translation keys for<T>
.
Implementation "details"
. (i.e in _ui mostly)
The main package is info.magnolia.i18nsystem
as a package name.
API
@I18nable
annotation. "Internationalizable": marks any object as a candidate for translations. Used on interfaces/classes such asFieldDefinition
. Is inherited.@I18nText
annotation. Marks a String as to be translated.I18nKeyGenerator<T>
interface. Implementations generate translation keys for<T>
.TranslationService
I18nizer
. Transforms/decorate a given object and internationalizes it.I18nParentable
. This is an interface that allows setting a "parent" on an object. This interface should not be used by client code.
Implementation "details"
- Tried to implement a Guice module that would do this transparently; also tried to implement this at Node2Bean level. Neither worked - didn't find a way to "swap" instances in Guice (i.e replace bean created by n2b by a proxy)
- I18nizer thus needs to be invoked explicitely. We have one implementation based on ProxyToys which creates a proxy. It intercepts methods that return other
@I18nable
objects (by returning one instance, a collection, or a map where value are of an annotated type) and decorate those. Those proxies in turn intercept - A Guice module which intercepts the creation of objects annotated with
@I18nable
and proxies them - Said proxy intercepts method calls annotated with
@I18nText
and returns translated values by delegating toi18nKeyGenerator
andTranslatorService
. I18nizer
also "injects" theI18nParentable
interface and sets the parent object while intercepting calls.
Update tools and tasks
To migrate our own modules, we can write a tool which:
- Starts up a repo and a specific (or several) component managers
- Load up a translation file we want to migrate (or all its language counter parts)
- Imports a bootstrap file we want to migrate
- This should instantiate a whole bunch of forms etc
- Go through these one by one
- Go through each
@I18nText
property of the object- Does it correspond to a key currently existing in the translation file ?
- yes: replace key in translation file by deduced key, remove property from jcr
- no: add key in translation file
- If it's a key but it doesn't have an existing translation
- add deduced key to translation file (instead of configured key), remove property from jcr
- If it's not a key, i.e current configuration has an "hardcoded" text
- warn, blow up, panic, ...?
- add deduced key to translation file (instead of configured key)
- yes: replace key in translation file by deduced key, remove property from jcr
- no: add key in translation file
- If it's a key but it doesn't have an existing translation
- add deduced key to translation file (instead of configured key), remove property from jcr
- If it's not a key, i.e current configuration has an "hardcoded" text
- warn, blow up, panic, ...?
- add deduced key to translation file (instead of configured key), remove property from jcr
- Does it correspond to a key currently existing in the translation file ?
- Track unused keys in translation files
- Track possible duplicates
- Go through each
- Re-export file to replace bootstrap file
- Re-export translation file(s)
The same tool could maybe be used to generate update tasks, or generate some sort of config/mapping file passed to a specific update task: it will only need a list of properties to be removed from nodes, with their original value so that we don't remove a property that's been modified by a user.
Message bundles for non-english texts
Bundle languages in separate jars
- Track unused keys in translation files
- Track possible duplicates
- Re-export file to replace bootstrap file
- Re-export translation file(s)
The same tool could maybe be used to generate update tasks, or generate some sort of config/mapping file passed to a specific update task: it will only need a list of properties to be removed from nodes, with their original value so that we don't remove a property that's been modified by a user.
Message bundles for non-english texts
Bundle languages in separate jars
- english/master language still bundled with each module, other languages bundled in 1 jar (1 jar per language, containing translations for many modules)
- on git, this would be, for example
languages/french.git
- which would contain N files. - These language bundles could be a module. They could thus be versioned, and have dependencies. Dependencies to module they translate would be marked as optional, but we would be able to maintain some sort of version-compatilbility between modules and translations - granted, that wouldn't be super simple to manage - where, for example, we'd update a dependency when keys have been added/removed from that dependency.
- on git, this would be, for example
- maintenance is somewhat easier
- but at the same time we might get "dependency" issues when modules add/remove keys
- if we have tools for migration/validation of existing translations, the same tools could be used, perhaps as a maven plugin or sthg.
- such a tool could potentially help enforcing compatibility between versions (i.e keep a key that was removed in version X+1 of module M)
- Need some sort of version handling - keep keys for older versions, add keys for newer ones ...
- Chain (overlay) current translation file with older ones ?
...
- Enable inline translations within a dialog
- I'm not sure the current proposal would work to enable inline translations. It'd be nice for translators to have at least a hint of what the key used for a specific item is. And if we somehow have elements explicitly use the KeyGenerator and other API methods for this, we might as well get rid of the proxy magic and use consistently...
- Review process for in-house maintained translations as well as for contributed ones (currently relying on Google spreadsheet)
- Have a Magnolia-hosted (or SaaS) tool to replace the google spreadsheet
- some rules like "a translation needs to be validated by 2 other persons to be applied", "once applied it can't be changed directly - only via a 'request'", ...
- could have a "MessagesManager" impl that fetches translations from this service
- candidates so far: Transifex (am in contact with CEO Dimitris Glezos, who seems eager to help), Crowdin, WebTranslateIt, ... probably others.
- Have a Magnolia-hosted (or SaaS) tool to replace the google spreadsheet
Roadmap
- 5.1 : system in place, translations migrated for AdminCentral and most modules (DAM, STK, ...)
- Module updates can be postponed
- ? : extract languages other than english into language-based bundles - needs a separate concept.
- ? : review processes for translating Magnolia, both internally and externally. Get contributions. With language files extracted from their modules, it might be easier. - needs a separate concept.
Status
Main issue: MGNLUI-1826@Jira
Validate Concepts
Validate Roadmap
API
Finalize module name, location, and package name.
Implementation details (proxy, Guice module, ...)
Parental relationships
How does this work with Multi/composite fields
Check things like info.magnolia.ui.form.field.definition.DateFieldDefinition#getDateFormat
- might actually work with @I18nText
or another annotation
Is there a conflict with merged objects ? (i.e template definitions merged with site def prototype - but also look at other merge cases!!)
An i18Basename could also be defined in site definitions (for in-template translations) - does this work ?
Clarify what to be with i18nBasename properties - the current tendency is to get rid of them. Adapt update tools as needed.
Replace MessagesManager with a cleaner impl
Migration and updates
Topics to validate or research
...
Status
Main issues:
Jira server Magnolia - Issue tracker key MAGNOLIA-5268 Jira server Magnolia - Issue tracker key MGNLUI-1826
Validate Concepts
Validate Roadmap
API
Finalize module name, location, and package name.
Implementation details (proxy, Guice module, ...)
Parental relationships
How does this work with Multi/composite fields
Check things like info.magnolia.ui.form.field.definition.DateFieldDefinition#getDateFormat
- might actually work with @I18nText
or another annotation
Is there a conflict with merged objects ? (i.e template definitions merged with site def prototype - but also look at other merge cases!!)
An i18Basename could also be defined in site definitions (for in-template translations) - does this work ?
Clarify what to be with i18nBasename properties - the current tendency is to get rid of them. Adapt update tools as needed.
Replace MessagesManager with a cleaner impl
Migration and updates
While this implementation has focused on definition objects, (some) "live" objects already know their parent (see for example info.magnolia.ui.form.FormItem
) - validate and justify the choice to decorate definition objects.
Known bugs or problems:
Jira server Magnolia - Issue tracker key MAGNOLIA-5315 Jira server Magnolia - Issue tracker key MAGNOLIA-5317 - we have no proper way, yet to i18n-ize GWT- / Vaadin-Client-Classes; neither info.magnolia.cms.i18n.MessagesUtil nore the i18n-toolz can be applied on those client-classes
Topics to validate or research
...
- A form can be used in different contexts. It should be translatable according to context.
- TBD: details, examples
should be translatable according to context.- TBD: details, examples
- Can we set it at injection time ?
- Do we add an interface for retrieving the locale-to-use ? (Which
UiContext
could implement for example)
Existing uses and inconsistencies to fix for 5.1
info.magnolia.ui.form.AbstractFormItem#getMessages
- should not be public- Is broken: doesn't use the user locale afaict.
info.magnolia.ui.form.AbstractFormItem#getMessage
- should not be publicAbstractFormItem
defines a semi-arbitrary message bundles chain (seeAbstractFormItem#UI_BASENAMES
)- Definition objects(
TabDefinition
, etc) have ani18nBasename
property, which is very redundant with that of the "runtimes" objects (FormTab
, ...). Usage seems consistent (return definition.getI18nBasename();
) but I don't know why this isn't implemented ininfo.magnolia.ui.form.AbstractFormItem
. - Definition objects don't have a common interface. If they did, we could move i18nBasename and label in there. OTOH, some of these objects have more than 1 item to translate (label and description, for example).
view.addFormSection(tab.getMessage(tabDefinition.getLabel()), tab.getContainer());
...translates the message from the tab and passes it translated before it's actually "displayed". (while the method argument is calledtabName
nottabLabel
- but that passed object becomes an argument calledcaption
later down the stack) - the below would make this sort of code much more explicit. You pass an object meant to be a label. You translate it explicitly - most likely at the last possible moment. Or we even extend Vaadin component so that they know aboutI18nItem
.getI18nBasename
is defined in too many places. It's inconsistent and unintuitive. Why the redundancy betweeninfo.magnolia.ui.dialog.Dialog#getI18nBasename
andinfo.magnolia.ui.dialog.definition.DialogDefinition#getI18nBasename
for example ?.DialogDefinition#getI18nBasename
for example ?-
info.magnolia.ui.form.FormBuilder#buildForm
still does "StringUtils.isNotBlank(description)", but since this is now decorated, it will never be blank or null. See
though.Jira server Magnolia - Issue tracker key MAGNOLIA-5315 info.magnolia.ui.form.FormBuilder#buildForm
Suggested key patterns
The below is a rough outline. The 3 goals below are somewhat hard to reach all at once.
- avoid redundancy or noise (no
dialog.
prefix, ...) - avoid conflicts (but allow them - on purpose - for labels that are actually meant to be the same in most situations)
- be consistent (this part is hard - sometimes we prefix with module name, sometimes with app, sometimes with nothing)sometimes with nothing)
Note: as the dialog names usually follow the pattern moduleName:dialogNameWithinTheModule
, the <dialog-name>
part of the keys mentioned bellow is in fact <module-name>.<dialog-name-within-the-module>
(as the ':
' character cannot be part of a key).
Field labels: – optional fallback to a key without a .label
suffix to make things less verbose
...
<dialog-name>.<tab-name>.label (or .tablabel for explicitness?)
Dialog (Form) labels:
<dialog-name>.label
<dialog-name>
Action labels:
99% of our dialogs have the same save/cancel actions. Those should be defaults. Labels should still be overridable on a dialog-per-dialog basis.
...
<dialog-name>.actions.<action-name>
I introduced the .actions
portion here to avoid confusion with fields; consistency would dictate having a .fields
or .tabs
portion for field and tabs labels too, but that would downplay the conciseness.
for all dialog-related items, we could also use <dialog-id>
and fallback to dialog-name
, for further specializing. Dialog ID is possibly currently not available; if it is, it's a single string concatenating module name and dialog name, which isn't ideal. It'd be sweet to be able to get back to module id (and app id) from a dialog.
Apps:
<app-id>.label
<app-id>.icon # because the icon might have to be localized
<app-id>.description # because a mouse-over title of the app might be interesting ?
<app-id>
could be <module-id>.<app-name>
or just app-name
(same as for dialogs)
App launcher groups:
app-launcher.<app-group-name>.label # we use the app-launcher prefix, as if app-launcher was an app (which we should consider considering, I suppose)
Templates:
<module-name>.<template-name>.title # I think "title" is what we've been using in 4.x - we could use label for consistency, or simply name
<module-name>.<template-name> # same as above
<module-name>.<template-name>.desc # Useful in template selector template selector
<module-name>.<template-name>
is essentially the component ID.
Page components:
<module-name>.<component-name>.title # see remark above
<module-name>.<component-name>
<module-name>.<component-name>.desc
<module-name>.<template<component-name>
is essentially the component ID.
Workbench columns:
<app-id>.<sub-app-name>.views.<view-name>.
<column-name>
configuration.browser.views.list.name
for /modules/ui-admincentral/apps/configuration/subApps/browser/workbench/contentViews/list/columns/name/label
<app-name>.<sub-app-name>.actions.<action-name>.<name of getter or field annotated withinfo.magnolia.i18n.I18nText>
contacts.browser.actions.addContact[.label]
for /modules/contacts/apps/contacts/subApps/browser/actions/addContact/label
<app-name>.chooseDialog.actions.<action-name>.<name of getter or field annotated withinfo.magnolia.i18n.I18nText>
assets.chooseDialog.actions.upload[.label]
for /modules/dam/apps/assets/chooseDialog/actions/upload
Page components:
<module-name>.<component-name>.title # see remark above
<module-name>.<component-name>
<module-name>.<component-name>.desc
<module-name>.<component-name>
is essentially the component ID.
Workbench columns:
<app-id>.<sub-app-name>.views.<view-name>.<column-name>
configuration.browser.views.list.name
for /modules/ui-admincentral/apps/configuration/subApps/browser/workbench/contentViews/list/columns/name/label
- actions in dialog
actionbar in subapps- workbench/<view>/columns in content apps: column names
- in workbench/view/columns, we also have
formatterClass
which should be locale-sensitive - app (label in app launcher, tab)
- page templates
- page components
...