<p>The client sells millions of lifestyle products to customers through its e-commerce website, contributing to their way of life. The website data related to sales, such as product name, payment method, and more, are stored in MySQL database, from where the client’s analytics team access it to measure and analyze sales performance and understand customers’ choices. These insights help the client plan new business initiatives such as discounts and promotions, to push sales.</p> <p>However, the challenge was that the reports were generated only once a day using an internal system based on Hadoop that fetched data in batches rather than real-time. This led to gaps in data reporting that reflected in MySQL (source of truth) and the on-premises system. This lack of real-time reporting and data mismatches led to inept decision-making, hampering the client’s progression.</p>