The main goal of our Analytics team was to develop a few additional application layers that integrated purchase order, delivery, and sales order data to produce consolidated inventory transfer and sales information. Technologies used in the development of the application layers included PySpark, Databricks, Snowflake, and Hive.
The team leveraged Microsoft Azure Data Lake Storage (ADLS) and Azure Copy to retain historical information. Historical data allows the company to make more accurate sales and inventory predictions from o9. The Analytics team created a final layer to consolidate data into customizable re-usable tables that allow business users, such as store managers, to conduct day-to-day sales services.
The GSPANN Information Analytics team also incorporated its in-house innovation - BEAT, a data quality framework, which performed data quality checks as data moved from one layer to the next within the data storage and processing infrastructure. The team created dashboards for both technical and business users. Technical users can view details on data flow within the pipeline and completion status. Business users can access information such as inventory order status, including shipped, received, or still in transit.
For more information on BEAT, read this article:
Automating Quality of Data Transformations in the Big Data World.The solution helps the company gain:
- Comprehensive end-to-end data access: Previously, business users only had access to physical data such as direct in-store purchases. Business and technical teams have complete data stream transparency with the new system and easily comprehensible dashboards. It includes data like shipping information, purchase orders, sales orders, and online purchases from all stores across all regions, enabling them to make informed inventory and supply chain decisions.
- A data quality check framework: The implemented quality framework monitors every data stream at each layer, ensuring strict data accuracy.
