Who is the Client

A US-based manufacturer of skincare products with $1.5B annual sales. Millions of customers across United States, Canada, and Australia exclusively depend on this brand for their skincare and makeup routine.

The Challenge

The client distributes its products through a multi-level marketing model. They hire independent sales consultants that connect with potential preferred customers (PCs) via social media or in-person meetings. The PCs eventually buy and resell the products to the end customers. They place a certain amount of order every 60 days on the client’s e-commerce portal, which forms a major part of the client’s sales.

The client uses Splunk to monitor the health of various internal e-commerce apps. These applications have more than 400 internal users and over 100,000 end-users, which results in daily data ingestion of 1TB. However, due to inefficient Splunk architecture implementation and manual efforts involved, the client spent a considerable amount of funds on Splunk professional support and long troubleshooting hours. Unfiltered data storage further added to the cost involved.

The Solution

Team GSPANN streamlined the Splunk framework by performing architectural change, migration, and upgrade of Splunk components, and standardizing the process of deployment and data onboarding.

We established Splunk as the primary source for monitoring and analyzing e-commerce applications. Our team developed a new Splunk architecture to improve the ROI of Splunk services, reduce the cost of data licensing, and enhance the availability of Splunk as a central monitoring service.

The solution proposed by team GSPANN consist of the following elements:

  • Architecture: We used the load-balanced Heavy Forwarder cluster for data routing, implemented syslog-ng for firewall and security data, and dedicated data route for Payment Card Industry Data Security Standard (PCI-DSS) and internal compliance data.
  • Migration: Migrated all services from the on-premise Splunk Enterprise instance.
    • All Splunk data ingestion apps, DB Connect, and scripted inputs moved to Heavy Forwarder cluster with a backup option.
    • Apps, alerts, and dashboards moved to Splunk cloud cluster.
  • Upgrade: All Splunk components – Heavy Forwarder, Universal Forwarder, Splunk Cloud, DB Connect, along with other Splunk applications and add-ons were moved to the latest supported and compatible versions.
  • Standardize Deployment: Implemented deployment server and updated docker images used on Kubernetes clusters.
  • Data Onboarding: Created modular components/apps, which can be reused for Scripted, REST, HEC (HTTP Event Collector), and other custom inputs.
  • User Management: Okta SSO (Single Sign-On) used for secure authentication. Restructured LDAP (Lightweight Directory Access Protocol) groups for valid authorization.
  • Audits: Query optimization for dashboards and alerts. Cleaned obsolete items and frequently reviewed scheduled jobs and reports.

Business Impact

  • The new architecture has improved the Splunk performance by 25% and increased the system availability up to 100%.
  • Data filtering helps the client in cutting down the cost of data storage licensing up to 30%.
  • The solution has reduced the manual intervention needed for managing Splunk environment by 80%.
  • Better compliance, no issues reported after four months of solution implementation.
  • After migrating apps into Splunk, the client has better tracking capabilities and improved security posture.

Technologies Used

Splunk Version: 7.1.3. A horizontal technology used for application management, security, compliance, as well as business and web analytics
Ansible 2.7. An open-source software provisioning, configuration management, and application-deployment tool enabling infrastructure as code
Jenkins. An open-source automation server that helps in building, testing, deploying, and facilitating continuous integration and continuous delivery
PagerDuty. An American cloud computing company specializing in a SaaS incident response platform for IT departments
Platform. AWS, Rackspace, Equinix, Linux, UNIX, and Windows
PuTTY. A free and open-source terminal emulator, serial console, and network file transfer application
Shell Scripts, PowerShell. A cross-platform task automation and configuration management framework, consisting of a command-line shell and scripting language

Related Capabilities

Reduce Downtime by Identifying Improvement Areas with a Proactive Production Support

We have expertise in implementing a preventative approach during production support. Our network operations center (NOC) provides deep application and system monitoring to ensure that you don’t face any surprises. Our production support team can help run your application uninterrupted to keep your customers happy and satisfied.

Do you have a similar project in mind?

Enter your email address to start the conversation