...

These tests should be executed at least once per release. The procedure to execute them is described at
Release QA Plan#Manualtestsusingtheintegrationtestswebapp Manual tests using the integration-tests webapp [Internal, Outdated].

To add such manual tests:

• checkout httpclone https://svngit.magnolia-cms.com/svnscm/community/bundle/trunk/platform/ce.pub.git
• depending on what needs to be tested, you can do a combination of:
  • add templates in magnolia-integration-tests-fixture-module/src/main/resources/mgnl-files/templates/test/manual
  • declare those templates in magnolia-integration-tests-fixture-module/src/main/resources/info/magnolia/test/config.properties or via info.magnolia.test.SetupStuffForTests.
  • in the template itself, describe what actions need to be performed for the test to be considered successful
  • add a page with this template to the website, under /manual-tests, via magnolia-integration-tests-fixture-module/src/main/resources/info/magnolia/test/website.properties.

please take this into consideration if you find yourself creating many templates that have no purpose other than holding the test description: in the future, the current website.properties file might be specialized (i.e use a custom, simpler, format, instead of the properties import format; additionally, the test description might figure in there instead of the template, especially for tests where templating isn't relevant); if you're in such case, feel free to do so yourself or give us a shout at Greg !!

We have also provided "integration tests" for REST.

• Check out - https://git.magnolia-cms.com/scm/modules/rest.git
• We use Rest Assured to easy to test our API. See more syntax in its documentation https://github.com/rest-assured/rest-assured
• Add new integration tests under the path  /magnolia-rest-integration-tests/tests/src/test/java
• Launch the container by using "manual-test" profile: mvn clean verify -Pjetty9-standalone,manual-tests
• Want to put some bootstrap dummy data? Put them under the folder  /magnolia-rest-integration-tests/magnolia-rest-test-webapp/src/main/webapp/WEB-INF/bootstrap/common
• In order to add more endpoints or override endpoint configuration. Find the configuration in the folder /magnolia-rest-integration-tests/magnolia-rest-test-webapp/src/main/webapp/modules/rest-content-delivery/restEndpoints

Automated tests

We also have a suite of automated tests. They rely on the same process, but are executed differently:

• since magnolia-integration-tests lies inside the CE bundle project, they are executed every time to bundle is built. This means they're executed on a regular basis by Hudson Jenkins (httphttps://hudsonjenkins.magnolia-cms.com/job/magnolia-bundle_trunk/) and every time we do a release.

...

There are also groovy scripts in magnolia-integration-tests/tests/src/main/:

• Testing of activation [Internal] that activates content and checks for results
• Crawling test [Internal] that requests all the pages and checks for rendering exceptions
• Log crawling test that reads logs and checks for rendering exceptions.

REST integration tests will be executed everytime the REST module is built - https://jenkins.magnolia-cms.com/job/m_rest/configure

Concurrent builds for bundles on Jenkins

Integration tests run concurrently on Jenkins for CE and EE bundles, and for our various branches (5.0.x up to master). Therefore each branch/bundle must use its own ports for starting up Jetty and e.g. sending shutdown signals through the Cargo wrapper.
Here are the ports in use for our current builds:    

Configuration for these ports is located in:

...

 When branching a bundle's master to the next major version, one should update these ports according to the pattern in the table above. 

Since 2.0, REST integration tests also run on the cargo/test-webapp setup, therefore they also have to use unique ports:

These ports are configured in: /rest/magnolia-rest-integration-tests/tests/pom.xml

Debugging

To debug these tests, one way to go is to start the container the same way as for the manual tests, with mvn clean verify -Pmanual-tests, and run the tests from the IDE in debug mode.
Somehow, memory issues seem to arise, at least with EE, in the manual-tests mode. This seemed to have helped, not sure which statement exactly:

...

All these tests are executed within the context of a single Jetty instance, which is started by the Cargo plugin of Maven.

The current setup deploys two Magnolia instances (: magnolia-test-webapp and magnolia-test-webapp-public, under the /magnoliaTest and /magnoliaTestPublic contexts, respectively) which currently both .
Both are an extension of magnolia-empty-webapp, additionally installing the magnolia-moduletemplating-samples and magnolia-integration-tests-fixture-module, which is containing contains the pages, templates etc created above, which are used for use while executing the tests.

...