When you develop a non-trivial application, you often split it into multiple components or services. There are many instances where you may need to access a database, reach some external services, or maybe access some cloud-based services to store files, such as simple storage services (s3). In such a scenario, you are faced with 2 challenges: How to access those services? This may be possible in a scenario where you don’t know the host and port of a MySQL database. Where to store your credentials? In MySQL, where do you store your username and password? A common way to solve both problems is to store the information in a configuration file. But the solution itself has the following limitations: It’s not entirely safe. We all have heard horror stories where passwords were stored heedlessly in a GitHub Repo. It’s a challenge to change the URL services in order to set up different environments, such as development, QA, and production. HashiCorp offers two open-sourced tools to address these problems: Vault: A tool to manage secrets. Consul: To ease the service discovery and configuration. Core Principles of the Vault: 1. Forbids sprawling of secrets: Vault can help you in managing usernames, passwords, database credentials, API tokens (or access tokens), Transport Layer Security (TLS) certificates, etc. These secrets are easily sprawl-able all over the infrastructure, such as source code, configuration management, or version control system, and we cannot really control these secrets appropriately. In this case, the Vault can provide you with a centralized management system, wherein all the secrets are encrypted at REST and ACL. 2. Ephemeral and unique secrets: Imagine a scenario where the Vault is set up and the application is leveraging its capabilities to the fullest. What if the application leaks the credentials through logs, diagnostics, or monitoring system? In this case, the Vault can offer a second level of security through ephemeral secrets, which means short-lived and dynamically generated secrets. Herein, all your clients will be given a unique secret that is generated dynamically and can be revoked at any time. As an example, if you have 50 web servers, you can now generate 50 unique secrets. 3. Key Lifecycle: The Vault provides you full lifecycle management, which includes key versioning, rotation, decommissioning, and revoking of a key. 4. Encrypt as a service: Instead of giving an application to encrypt the secret, the Vault will provide you with the Application Programming Interface (API) to do the cryptography chores, such as encrypt or decrypt, sign, verify, and a keyed hash message authentication code (HMAC). Vault Architecture: One of the major advantages of the Vault is that it has a highly pluggable architecture, i.e., it is made of different components or plugins. Some of the aspects covered by the plugins are: Core: As an authentication plugin, the core is responsible for serving the requests and key lifecycle management. It can provide you with the Elastic Block Store (EBS) encryption key of the boot volume to the Elastic Compute Cloud (EC2) instance while it is booting up. Audit Back-end: Responsible for auditing the secret access, request or response trail, and the information regarding who-has-done-what with the secrets. You can store these generated logs in Splunk or s3. Storage Back-end: The Vault stores its own data in storage back-end at rest. It gives you the leverage of choosing the transmission system. You can choose from the Routing Database System (RDBS) or a Consul, depending upon their durability and availability. Secret Back-end: Here, your application’s dynamic and static secrets are safely stored and encrypted. It supports multiple types of plugins, such as the key-value pair, Amazon Web Services (AWS), database, Public Key Infrastructure (PKI), etc. You can also generate dynamic API keys to give read / write permission to S3 for a limited period of time. Key Concepts of the Consul: 1. Service discovery: As a tool, the Consul can enable you to discover and configure a variety of different services in your infrastructure. If you have an e-commerce application, some of the services that the Consul would consider include account services, cart and checkout services, billing services, etc. The Consul enables various services to communicate and maintain the load balancing tasks among the different instances of running service by registering all service instances in a consul registry. 2. Key-value store: To have a consistent view of the configurations across the distributed environment, the Consul captures different configurations dynamically in a key-value store. For instance, we use log location or log level as a kill switch. Herein, all your clients will be given a unique secret that is generated dynamically and can be revoked at any time. As an example, if you have 50 web servers, you can now generate 50 unique secrets. 3. Segmentation: You can configure multiple web servers through the Consul to communicate with multiple database servers for client communications, just like a firewall or security group. 4. Health checks: Stores an account of health checks of a registered service. For example, you can configure the health of a web server as healthy when it is returning the response code 200. It can be concluded that the Vault coupled with the Consul backend can help you big to secure your key or value configurations. While the Vault can help in storing the “secrets”, it requires authentication to access all the stored secret goodies.