Adobe Business Catalyst

Business Catalyst is a web application that lets you create, manage and maintain your own website. Key features of this product are that there is no server side coding needed and hosting is provided with a very high level of security.

CUSTOMER

Adobe is the global leader in digital marketing and digital media solutions. At Vanity Fair, Martha Stewart Living, and other leading publications, for example, our creative software is used to develop compelling, interactive content delivered in print, online, and on tablets. At the same time, industry leaders such as Sony, U.S. Bank, and Caesar’s Entertainment are turning to Adobe Digital Marketing solutions to maximize their sales in a multichannel, multiscreen world.

CHALLENGES

Tests needed to cover both backend (admin interface) and frontend testing of the application. During test execution some services needed testing (SOAP and emails). In some cases for services had to be tested asynchronously. A very large number of tests had to be implemented Database needed to be used to store test data. Comprehensive reports, easy integration and distribution with other tools such as CI, parallel running of test capability.

SOLUTION

A Java application was implemented, with maven builder and Thucydides framework. The runner was custom made to support data reading and writing via IIS to a SQL database. Each test would check in the setup if there is any data stored in the database; if no data was found the test would create a new site and all the needed data to run the validations. Email verifications were done via POP or SMTP, depending on the email server, with email body validation of data. For SOAP testing, custom methods for each call were implemented, these being called in a test or run totally separately of the functional test suite.

More than 400 tests were implemented with test execution time ranging between 5 to 20 minutes per test. These tests covered an estimated 30% of the application (all modules and major flows) and had a success rate over 90% at delivery. The suite, without creating the setup data, would run in about 9 hours, generating a 4 GB report with print screens at each step, in a website manner. The reports were published on a Tomcat server for later review.

This is a summary of the test results. It contains a graphical representation of the overall coverage, the tests are categorized by features and the entire report is actually a fully navigable website starting from the summary to individual tests with steps and screenshot for each step if needed.