infra-automation/docs/security/vault-management.md

Ansible Vault Management Guide

This document describes how to manage encrypted secrets using Ansible Vault in this infrastructure.

Overview

Ansible Vault is used to encrypt sensitive data such as passwords, API tokens, and private keys. All vault files are stored in inventories/<environment>/group_vars/all/vault.yml.

Table of Contents

• Quick Start
• Vault File Locations
• Creating Vault Files
• Encrypting and Decrypting
• Editing Vault Files
• Using Vault Variables
• Vault Password Management
• Best Practices
• Troubleshooting

Quick Start

# 1. Create vault file from example
cp inventories/production/group_vars/all/vault.yml.example \
   inventories/production/group_vars/all/vault.yml

# 2. Edit and fill in secrets
vi inventories/production/group_vars/all/vault.yml

# 3. Encrypt the vault file
ansible-vault encrypt inventories/production/group_vars/all/vault.yml

# 4. Run playbook with vault password
ansible-playbook site.yml --ask-vault-pass

Vault File Locations

Vault files are organized by environment:

inventories/
├── production/
│   └── group_vars/
│       └── all/
│           ├── vault.yml.example  # Template
│           └── vault.yml          # Encrypted (gitignored)
├── staging/
│   └── group_vars/
│       └── all/
│           ├── vault.yml.example
│           └── vault.yml
└── development/
    └── group_vars/
        └── all/
            ├── vault.yml.example
            └── vault.yml

Important: vault.yml files should be added to .gitignore to prevent accidental commits.

Creating Vault Files

From Example Template

# Copy example template
cp inventories/production/group_vars/all/vault.yml.example \
   inventories/production/group_vars/all/vault.yml

# Edit and replace CHANGEME placeholders
vi inventories/production/group_vars/all/vault.yml

# Encrypt the file
ansible-vault encrypt inventories/production/group_vars/all/vault.yml

Create New Vault File

# Create and encrypt in one step
ansible-vault create inventories/production/group_vars/all/vault.yml

This opens your editor to add vault contents, then automatically encrypts on save.

Encrypting and Decrypting

Encrypt a File

ansible-vault encrypt inventories/production/group_vars/all/vault.yml

You'll be prompted to create a vault password.

Decrypt a File

# Decrypt to view/edit (dangerous - creates plaintext file)
ansible-vault decrypt inventories/production/group_vars/all/vault.yml

# View without decrypting
ansible-vault view inventories/production/group_vars/all/vault.yml

Warning: Decrypting a file leaves it in plaintext. Always re-encrypt after editing.

Encrypt Multiple Files

ansible-vault encrypt inventories/*/group_vars/all/vault.yml

Editing Vault Files

Edit Encrypted File

# Edit encrypted file directly (recommended)
ansible-vault edit inventories/production/group_vars/all/vault.yml

This decrypts the file in memory, opens your editor, and re-encrypts on save.

Change Vault Password

ansible-vault rekey inventories/production/group_vars/all/vault.yml

You'll be prompted for the old password, then the new password.

Using Vault Variables

In Playbooks

Reference vault variables like normal variables:

---
- name: Configure database
  hosts: databases
  tasks:
    - name: Set MySQL root password
      mysql_user:
        name: root
        password: "{{ vault_mysql_root_password }}"
        host: localhost

In Templates

# /etc/my.cnf
[client]
user = root
password = {{ vault_mysql_root_password }}

In Role Defaults

# roles/mysql/defaults/main.yml
---
mysql_root_password: "{{ vault_mysql_root_password }}"

Vault Password Management

Option 1: Interactive Password Prompt (Most Secure)

ansible-playbook site.yml --ask-vault-pass

Option 2: Password File

Create a password file:

# Create password file (gitignored)
echo "YourVaultPassword123!" > .vault_pass
chmod 600 .vault_pass

Add to .gitignore:

.vault_pass

Update ansible.cfg:

[defaults]
vault_password_file = .vault_pass

Run playbooks without prompt:

ansible-playbook site.yml

Option 3: Environment Variable

export ANSIBLE_VAULT_PASSWORD_FILE=.vault_pass
ansible-playbook site.yml

Option 4: Script-Based Password (Advanced)

Create a script that retrieves the password from a secure source:

#!/bin/bash
# vault-password.sh
# Retrieve password from AWS Secrets Manager, HashiCorp Vault, etc.
aws secretsmanager get-secret-value \
  --secret-id ansible-vault-password \
  --query SecretString \
  --output text

Make it executable:

chmod +x vault-password.sh

Use in ansible.cfg:

[defaults]
vault_password_file = ./vault-password.sh

Best Practices

Security

1. Never commit unencrypted vault files to version control
2. Use different vault passwords for each environment
3. Rotate vault passwords every 90 days
4. Restrict access to vault password files (chmod 600)
5. Use strong passwords (minimum 20 characters, mixed case, numbers, symbols)
6. Store production passwords in a secure password manager (1Password, LastPass, etc.)

Organization

1. Prefix vault variables with vault_ for clarity:

   vault_mysql_root_password: "secret123"
   vault_api_token: "token456"
   

2. Use vault variables in role defaults:

   # roles/mysql/defaults/main.yml
   mysql_root_password: "{{ vault_mysql_root_password }}"
   

3. Document all vault variables in vault.yml.example

4. One vault file per environment for easier management

Git Management

Add to .gitignore:

# Vault passwords
.vault_pass
vault-password.sh

# Unencrypted vault files
**/vault.yml
!**/vault.yml.example

Verify vault files are encrypted before committing:

# Check if file is encrypted
head -1 inventories/production/group_vars/all/vault.yml
# Should output: $ANSIBLE_VAULT;1.1;AES256

Multiple Vault Passwords

For environments with different vault passwords:

Using Vault IDs

# Encrypt with vault ID
ansible-vault encrypt \
  --vault-id production@prompt \
  inventories/production/group_vars/all/vault.yml

ansible-vault encrypt \
  --vault-id staging@prompt \
  inventories/staging/group_vars/all/vault.yml

# Run playbook with multiple vault IDs
ansible-playbook site.yml \
  --vault-id production@.vault_pass_production \
  --vault-id staging@.vault_pass_staging

Common Vault Variables

User Credentials

vault_ansible_user_ssh_key: "ssh-rsa AAAA..."
vault_root_password: "password"
vault_ansible_become_password: "password"

API Tokens

vault_aws_access_key_id: "AKIA..."
vault_aws_secret_access_key: "secret"
vault_netbox_api_token: "token"

Database Credentials

vault_mysql_root_password: "password"
vault_postgresql_postgres_password: "password"

Application Secrets

vault_app_secret_key: "secret_key"
vault_app_api_key: "api_key"

Troubleshooting

Wrong Vault Password

Error: Decryption failed (no vault secrets were found that could decrypt)

Solution: Verify you're using the correct vault password for that environment.

Vault File Not Found

Error: ERROR! Attempting to decrypt but no vault secrets found

Solution: Create the vault file or check the path is correct.

Permission Denied

Error: Permission denied: 'vault.yml'

Solution: Check file permissions:

ls -la inventories/production/group_vars/all/vault.yml
chmod 600 inventories/production/group_vars/all/vault.yml

Forgot Vault Password

Solution: Unfortunately, there's no way to recover a forgotten vault password. You'll need to:

1. Re-create the vault file from scratch
2. Re-enter all secrets
3. Re-encrypt with a new password

Prevention: Store vault passwords in a secure password manager.

Check Vault File Integrity

# Verify file can be decrypted
ansible-vault view inventories/production/group_vars/all/vault.yml

# Check encryption format
file inventories/production/group_vars/all/vault.yml
# Should output: ASCII text (encrypted vault file)

Emergency Procedures

Compromised Vault Password

1. Immediately change the vault password:

   ansible-vault rekey inventories/production/group_vars/all/vault.yml
   

2. Rotate all secrets stored in the vault

3. Audit access logs to determine scope of compromise

4. Update vault password in all secure storage locations

Lost Access to Production Vault

1. Use backup vault password from secure password manager
2. If no backup exists, rotate all production credentials
3. Create new vault file with new credentials
4. Update all systems with new credentials

References

• Ansible Vault Documentation
• Ansible Best Practices - Vault
• Internal: CLAUDE.md - Secrets Management

---

Document Version: 1.0 Last Updated: 2025-11-11 Maintainer: Infrastructure Team