Integrated Automated Testing Platform with TestComplete

An automated solution easily integrates with well-known existing CI/CD technologies and provides the client with a robust foundation for future expansion with improved system transparency.

The Client

An American corporation engaged in designing, manufacturing, marketing, and servicing semiconductor processing equipment used to fabricate integrated circuits.

The Challenge

The client had multiple tools and tech stacks, and there was no automated way to validate and verify Message Queuing Telemetry Transport (MQTT) messages propagated through Kafka. Spark jobs had to be run manually, and Spark log messages had to be independently verified by the Quality Engineering (QE) team. Additionally, while Jira and Zephyr were being used, there was no end-to-end traceability.

The client needed a unified automation solution that integrated and managed all test cases across a variety of tools. The overall testing framework needed to follow behavior-driven design (BDD) principles, provide accurate test results, and tie testing closely to business outcomes. In short, the customer was looking for:

A single tool to seamlessly manage test cases: Required one tool to handle all types of test cases, including unit tests, and tests to verify and validate Kafka messages, and tests to verify Spark log files and data from a Hadoop Distributed File System (HDFS) and Apache Solr.

End-to-end traceability using Jira and Zephyr: Any test automation needed to monitor test status through the existing Jira/Zephyr infrastructure, which was not possible in the current environment.

Automation of Regression tests: When new features were introduced, the client's Smart Data Analysis (SDA) team needed the ability to automate regression test execution when code was checked in, allowing them to catch bugs early.

Testing integrated with CI/CD: The system needed to be integrated with the existing Jenkins CI/CD infrastructure to ensure continuous testing.

Another consideration was to select an automation framework that supported multiple languages, including Python, C#, and Java, and ensure data integrity as data made its way through the newly adopted Kafka infrastructure. Finally, the testing framework needed to be easy for the client to define tests and reuse modules to simplify maintenance and development and ensure quality engineering.

Share this Document

Quality Engineering

Manufacturing

B2B

Automation

Related Resources

AI-Powered ContentHubGPT Boosts Business Growth and Engagement

Reflecting Excellence: Make Product Descriptions Better with ContentHubGPT

Reduce Product Time to Market with Salsify PXM and ContentHubGPT

The Solution

After carefully analyzing the current client environment, GSPANN's QA SMEs proposed using one tool to accomplish the overall goal. Three tools were evaluated – TestComplete, Tricentis Tosca, and Katalon. We did a proof-of-concept (POC) with each of the three tools in the customer’s environment. Use cases were selected which covered UI validation, accurate reading of console output, database connectivity, graph validation, upload/download of test files, and SSH connectivity to remote servers.

TestComplete exhibited an advantage over the other tools due to its scalability, API testing features, support for multiple programming languages, and ability to integrate with the client’s current set of tools. Since TestComplete also stood out in its product support, maintainability, and lower cost, the client chose TestComplete to automate its desktop, web, and backend test cases.

Here are some highlights of the newly automated quality engineering framework:

Created custom utilities: Custom modules read Kafka MQTT messages, upload, download, and rename files via SFTP, read Spark job logs, and start Spark jobs running via SSH. The client can now design new test cases by simply uploading XML, JSON, or Excel test files.

Automated data verification and validation: Other custom modules developed by Team GSPANN perform automated verification of HDFS, Apache Solr, and JSON data.

Implemented key integrations: Configured TestComplete to interact with the existing Jira/Zephyr platform to publish execution results in a Jira test plan. Another integration allows a Jenkins DevOps pipeline to execute automated test scripts, and yet another takes the form of web application tests using Selenium Grid operating remotely as a headless browser.

Created template project: The generic Template project incorporates the custom modules and the client can easily clone this template to build additional extended testing capabilities in the future.

The following image gives a clear picture of the new implementation.

Image: TestComplete integration with Jira, Zephyr, and Jenkins

As you can see from the image, TestComplete nicely integrates with Jira, Zephyr, and Jenkins, providing the glue that makes test automation possible. It manages BDD business scenarios and includes UI, backend, and API testing.

Business Impact

Foundation for future expansion: The generic Template project is available in a Bitbucket code repository, allowing the client to rapidly automate testing of any new projects.

Increased system transparency: Test suites are processed on HDFS, allowing rapid validation and verification of Solr search results, Spark logs, and Kafka MQTT messages. The net result enables the client to have a much clearer picture of how well their system is performing.

Efficient maintenance: Continuous quality engineering testing identifies problems quickly, making application maintenance easier and more effective.

Highly scalable with limited resources: Since the testing framework is now almost entirely automated, further expansion and addition of new test cases are possible using far fewer resources than before.

Availability of critical system insights: Execution reports are now published regularly through Jenkins, Jira, and Zephyr. Over time, the client's DevOps team can compare system insights, identify problem areas, and improve system performance.

Technologies Used

TestComplete: It automates UI testing and integrates with Jira, Zephyr, and JenkinsPython: A popular scripted programming languageJira Jenkins: Platform that provides CI/CD pipeline supportZephyr: A test management and reporting system that provides native Jira integrationReadyAPI: A REST API test case automation toolTestExecute: It automates tests suites on a Jenkins agentSelenium Grid: A Selenium feature that runs test cases in different machines across different platformsApache Spark: A highly scalable data analytics support engine

Related Capabilities

Utilize Actionable Insights from Multiple Data Hubs to Gain More Customers and Boost Sales

Unlock the power of the data insights buried deep within your diverse systems across the organization. We empower businesses to effectively collect, beautifully visualize, critically analyze, and intelligently interpret data to support organizational goals. Our team ensures good returns on the big data technology investments with the effective use of the latest data and analytics tools.