Who is the Client

The company is a fast-growing US-based brand that produces fashion apparel for women.

The Challenge

The company was developing a new mobile app for its business. They had a partially automated web testing framework with a manual mobile testing component that consumed costly internal resource bandwidth. They needed to automate over 500 test cases altogether, requiring a dedicated 43 hours for four quality engineers to undertake a manual execution process. The company required:

  • An integrated testing framework: A framework to properly test apps not only on browsers, such as Chrome, Safari, and Firefox, but also on all supported mobile platforms, including iOS and Android.
  • A comprehensive yet cost-effective solution: Instead of writing a custom automated framework, using an existing framework would save time in delivering the solution and save money for the company. The company also needed a cost-effective way of testing a series of new mobile devices. The solution needed to address the company’s issues with device procurement cost, time, and effort.
  • Timely delivery of new apps and features: The company reduced the overall project completion timeline for rolling out new apps and features by three months, providing additional incentive to quickly implement an automated testing solution.

Due to the reduced project completion timeline, our QE team started receiving app builds at a much more rapid pace. For each build, the team had to test new functionality, test bug fixes, and perform regression testing to ensure the new features and fixes did not impact existing functionality. To further compound the problem, test suites had to be performed on all supported mobile platforms.

The Solution

After carefully reviewing the company’s requirements, our QE team proposed an integrated automated testing framework to drastically reduce expenses and effort. Our team used the Page Object Model (POM) design pattern and followed a Test-Driven Development (TDD) approach for project implementation. This resulted in high-quality code that significantly reduced long-term project costs and increased development flexibility. Highlights of the automated QA test framework project include:

  • Created a single script for multiple platforms: A single script runs all native mobile, web, and desktop application tests on all supported platforms, including Chrome, Firefox, iOS, and Android.
  • Gained flexibility: Our QE team implemented the solution using a loosely-coupled pattern where individual components operate in isolation from each other but work together since the configuration is maintained externally. With a few easy configuration changes, execution is possible in different environments.
  • Provided extensibility: Our team leveraged the Fillo API, allowing the company to customize configuration using simple Excel spreadsheets and Java Faker to generate an unlimited amount of real-world data. The flexible design allowed easy integration with TestRail, Sauce Labs, and Jenkins.
  • Reduced test execution time: In addition to automation, our implementation introduced further improvements in the time required to conduct comprehensive testing by leveraging Apache HttpClient. When creating test data activity, this well-proven technology handles the pre- and post-conditions surrounding each test.
  • Introduced industry best practices: The solution followed industry best practice design patterns, producing standard code for both Android and iOS applications which fostered reusability and maintainability, and also facilitated training on the framework.

A further advantage of our implementation was mapping test cases down to the assertion level. This implementation allows the company to execute multiple test cases simultaneously in a single test method, resulting in the same execution time for multiple scenarios.

Business Impact

  • Integrated framework results in 50% maintenance cost reduction: The new integrated framework can support both existing web application automation and new mobile app automation. It reduced the company’s total cost of ownership by 50% since they don’t need to maintain two frameworks anymore.
  • Test runtime drops from days to hours: Earlier, the mobile app team required two or more days of manual testing to validate the back-end APIs accessed by the mobile apps. The new system executes entire test suites on multiple platforms in less than two hours. This reduced testing time enabled new releases to be released at an accelerated pace.
  • Reduced resource requirements: The new automated testing framework requires fewer staff resources since it no longer needs to run a grueling set of manual tests. Additional tests can easily be added without allocating additional staff.
  • Loosely-coupled components improve scalability: By loosely coupling the various components, upscaling the framework has less impact on code changes.
  • Reduced device procurement costs: Automation is triggered for every build and executed in Sauce Labs. Adhering to the device policy used for testing, the client doesn’t need to procure all devices, irrespective of how many new models are subsequently added to the device policy. This policy significantly reduces device procurement costs, making this approach cost-effective and efficient.
  • Faster feedback on builds: The automated testing framework vastly reduces build’s release cycle time. The QE team is now able to immediately incorporate build feedback into a release instead of waiting to identify it at later stages.
  • Time and money saved by leveraging an existing testing framework: Rather than developing a custom solution, our QE team decided to extend Sauce Labs, an existing and proven web framework, adding extra functionality to support the mobile platforms. This approach saved the client time and money, with proven reliability as an additional benefit.

Technologies Used

Appium. Test automation framework for native, hybrid, and mobile web apps
Fillo. Allows the user to use simple Excel files as a source of test configuration and test data
HttpClient. Standards-based HTTP client written in Java from Apache Software Foundation
Java. A widely-used programming language
JavaFaker. Generates fake real-world data used for real-time testing
Jenkins. Continuous integration and continuous delivery
Sauce Labs. Cloud-based integrated application testing framework
TestRail. Test case management

Related Capabilities

Manage the Creation of Personalized and Relevant Content Across Platforms

Content drives engagement. We simplify information management for you to accelerate content productivity of uncompromised quality. Our competencies in CMS platforms help you optimize content management and stay relevant in the ever-changing digital landscape.

Do you have a similar project in mind?

Enter your email address to start the conversation