diff --git a/README.md b/README.md index 6208fba..75f52fa 100644 --- a/README.md +++ b/README.md @@ -2,20 +2,42 @@ Enterprise-grade Ansible infrastructure with security-first principles, modularity, and scalability. +## Repository Structure + +This repository uses **git submodules** for proper separation of concerns: + +- **Main Repository** (PUBLIC): Playbooks, roles, and infrastructure code +- **Inventories Submodule** (PRIVATE): Dynamic inventories and host configurations +- **Secrets Submodule** (PRIVATE): SSH keys, vault files, and sensitive data + ## Quick Start +### Initial Setup + ```bash -# Test connectivity with SSH config inventory -ansible all -i plugins/inventory/ssh_config_inventory.py -m ping +# Clone with submodules (recommended) +git clone --recurse-submodules ssh://git@git.mymx.me:2222/ansible/infra-automation.git +cd infra-automation -# Test connectivity with Libvirt dynamic inventory -ansible running_vms -i plugins/inventory/libvirt_kvm.py -m ping +# Or initialize submodules after clone +git submodule init +git submodule update -# Use static development inventory -ansible all -i inventories/development/hosts.yml -m ping +# Set up SSH agent for git operations +source .ssh-agent-init +``` + +### Basic Usage + +```bash +# Test connectivity with dynamic inventory +ansible all -i inventories/production/libvirt.yml -m ping + +# List inventory +ansible-inventory -i inventories/production/libvirt.yml --list # Run a playbook -ansible-playbook -i inventories/development/hosts.yml site.yml +ansible-playbook -i inventories/production/libvirt.yml playbooks/gather_system_info.yml ``` ## Project Structure @@ -26,28 +48,33 @@ ansible-playbook -i inventories/development/hosts.yml site.yml ├── CLAUDE.md # Development guidelines and standards ├── ansible.cfg # Ansible configuration ├── site.yml # Master playbook +├── .ssh-agent-init # SSH agent auto-initialization │ -├── inventories/ # Inventory configurations -│ ├── production/ # Production (dynamic only) -│ ├── staging/ # Staging (dynamic only) -│ └── development/ # Development environment -│ ├── hosts.yml # Static inventory -│ ├── libvirt_kvm.yml # Libvirt config -│ └── group_vars/ # Group variables -│ ├── all.yml -│ ├── kvm_guests.yml -│ └── hypervisors.yml +├── inventories/ # → Git submodule (PRIVATE) +│ ├── production/ # Dynamic libvirt inventory +│ ├── staging/ # Staging environment +│ └── development/ # Development environment │ -├── plugins/ # Custom plugins -│ └── inventory/ # Dynamic inventory scripts -│ ├── ssh_config_inventory.py # SSH config parser -│ └── libvirt_kvm.py # Libvirt/KVM discovery +├── secrets/ # → Git submodule (PRIVATE) +│ ├── ssh/ # SSH keys for automation +│ ├── machines/ # Machine-specific secrets +│ └── vaults/ # Ansible vault files +│ +├── playbooks/ # Playbooks +│ ├── gather_system_info.yml # System information collection +│ ├── configure_swap.yml # Swap configuration +│ ├── install_qemu_agent.yml # QEMU guest agent +│ └── audit_docker.yml # Docker security audit │ ├── roles/ # Ansible roles -├── playbooks/ # Playbooks -├── collections/ # Ansible collections +│ ├── system_info/ # Production-ready +│ └── deploy_linux_vm/ # Production-ready │ +├── collections/ # Ansible collections ├── docs/ # Documentation +│ ├── submodule-workflow.md # Submodule usage guide +│ ├── git-ssh-setup.md # Git SSH configuration +│ └── security/ # Security documentation │ ├── inventory.md # Inventory documentation │ └── [other docs] │ diff --git a/docs/submodule-workflow.md b/docs/submodule-workflow.md new file mode 100644 index 0000000..6f5e3e8 --- /dev/null +++ b/docs/submodule-workflow.md @@ -0,0 +1,400 @@ +# Git Submodule Workflow + +This repository uses git submodules to separate concerns and follow CLAUDE.md guidelines. + +## Repository Structure + +``` +infra-automation/ # Main repository (PUBLIC) +├── inventories/ # → ansible-inventories submodule (PUBLIC) +├── secrets/ # → secrets submodule (PRIVATE) +├── playbooks/ +├── roles/ +└── docs/ +``` + +## Submodules + +### 1. inventories (PRIVATE) +- **URL:** `ssh://git@git.mymx.me:2222/ansible/ansible-inventories.git` +- **Type:** PRIVATE repository +- **Contents:** Dynamic inventories, host/group variables (may contain internal IPs/hostnames) +- **Purpose:** Separate inventory management from infrastructure code, protect internal network topology + +### 2. secrets (PRIVATE) +- **URL:** `ssh://git@git.mymx.me:2222/ansible/secrets.git` +- **Type:** PRIVATE repository +- **Contents:** SSH keys, vault files, sensitive data +- **Purpose:** Security isolation, separate access control + +## Initial Clone + +### Option 1: Clone with submodules (recommended) + +```bash +git clone --recurse-submodules ssh://git@git.mymx.me:2222/ansible/infra-automation.git +cd infra-automation +``` + +### Option 2: Clone then initialize submodules + +```bash +git clone ssh://git@git.mymx.me:2222/ansible/infra-automation.git +cd infra-automation +git submodule init +git submodule update +``` + +## Working with Submodules + +### Update All Submodules + +```bash +# Update to latest commits from remote +git submodule update --remote + +# Commit submodule updates in main repository +git add inventories secrets +git commit -m "Update submodule references" +git push +``` + +### Update Specific Submodule + +```bash +# Update only inventories +git submodule update --remote inventories + +# Update only secrets +git submodule update --remote secrets +``` + +### Making Changes in Submodules + +#### Method 1: Work inside submodule + +```bash +# Navigate to submodule +cd inventories + +# Ensure on correct branch +git checkout master + +# Make changes +vim production/libvirt.yml + +# Commit and push from submodule +git add production/libvirt.yml +git commit -m "Update libvirt inventory configuration" +git push origin master + +# Update parent repository reference +cd .. +git add inventories +git commit -m "Update inventories submodule" +git push +``` + +#### Method 2: Direct submodule update + +```bash +# Make changes, commit, and push in one workflow +cd inventories +git pull origin master +# Make changes +git add . +git commit -m "Changes" +git push origin master +cd .. +git add inventories +git commit -m "Update inventories" +git push +``` + +### Checking Submodule Status + +```bash +# View submodule status +git submodule status + +# View detailed info +git submodule + +# Check for uncommitted changes in submodules +git submodule foreach git status +``` + +## Common Workflows + +### Workflow 1: Update Inventory Configuration + +```bash +# 1. Navigate to inventories +cd inventories + +# 2. Pull latest changes +git pull origin master + +# 3. Make changes +vim production/group_vars/all.yml + +# 4. Commit and push +git add production/group_vars/all.yml +git commit -m "Update production variables" +git push origin master + +# 5. Update parent repository +cd .. +git add inventories +git commit -m "Update inventories submodule reference" +git push origin master +``` + +### Workflow 2: Add New SSH Key to Secrets + +```bash +# 1. Navigate to secrets +cd secrets + +# 2. Pull latest (important for private repo) +git pull origin master + +# 3. Add new key +ssh-keygen -t ed25519 -f ssh/newkey -C "description" + +# 4. Document in README +vim ssh/README.md + +# 5. Commit and push +git add ssh/ +git commit -m "Add new SSH key: newkey" +git push origin master + +# 6. Update parent +cd .. +git add secrets +git commit -m "Update secrets submodule" +git push origin master +``` + +### Workflow 3: Clone Project on New Machine + +```bash +# 1. Clone with submodules +git clone --recurse-submodules ssh://git@git.mymx.me:2222/ansible/infra-automation.git +cd infra-automation + +# 2. Verify submodules initialized +git submodule status +# Should show: ebe29b6... inventories (heads/master) +# 8def011... secrets (heads/master) + +# 3. Set up SSH agent for git operations +source .ssh-agent-init + +# 4. Verify inventory works +ansible-inventory -i inventories/production/libvirt.yml --list + +# 5. Ready to use! +``` + +## Troubleshooting + +### Submodule Not Initialized + +**Problem:** Submodule directory exists but is empty + +```bash +# Solution +git submodule init +git submodule update +``` + +### Submodule Detached HEAD + +**Problem:** Submodule is in detached HEAD state + +```bash +# Solution: checkout master branch +cd submodule_name +git checkout master +git pull origin master +cd .. +git add submodule_name +git commit -m "Update submodule to track master" +``` + +### Submodule Changes Not Showing + +**Problem:** Made changes in submodule but parent doesn't show update + +```bash +# Solution: Stage submodule in parent +git add submodule_name +git commit -m "Update submodule reference" +``` + +### Permission Denied on Private Submodule + +**Problem:** Cannot clone/update secrets submodule + +```bash +# Solution: Ensure SSH key is loaded +source /opt/ansible/.ssh-agent-init +ssh-add -l + +# Verify access +ssh -T git@git.mymx.me -p 2222 + +# Try update again +git submodule update --init +``` + +### Accidentally Committed Changes to Detached HEAD + +**Problem:** Made commits in submodule while in detached HEAD + +```bash +# Solution: Create branch from detached commits +cd submodule_name +git branch temp-branch +git checkout master +git merge temp-branch +git push origin master +git branch -d temp-branch +``` + +## Best Practices + +### 1. Always Work on Branches +```bash +cd inventories +git checkout master +# Never work in detached HEAD +``` + +### 2. Pull Before Push +```bash +cd secrets +git pull origin master +# Make changes +git push origin master +``` + +### 3. Update Parent After Submodule Changes +```bash +# After pushing submodule changes +cd .. +git add submodule_name +git commit -m "Update submodule" +git push +``` + +### 4. Regular Submodule Updates +```bash +# Weekly: update all submodules +git submodule update --remote +git add inventories secrets +git commit -m "Update submodules to latest" +git push +``` + +### 5. Document Submodule Changes +```bash +# Use descriptive commit messages +git commit -m "Update inventories: Add staging environment configuration" +``` + +## Security Considerations + +### Secrets Submodule (PRIVATE) +- ⚠️ Never make secrets repository public +- ⚠️ Verify .gitignore before committing +- ⚠️ Use SSH key authentication only +- ⚠️ Regular access audits +- ⚠️ Rotate keys according to schedule + +### Inventories Submodule (PRIVATE) +- ⚠️ Private repository - protects network topology +- ⚠️ Contains internal IPs, hostnames, network structure +- ✅ Use vault references for passwords/secrets +- ✅ Document all group/host variables +- ✅ Controlled team access only + +## Advanced Operations + +### Reset Submodule to Specific Commit + +```bash +cd inventories +git checkout +cd .. +git add inventories +git commit -m "Pin inventories to specific version" +``` + +### Remove Submodule + +```bash +# 1. Deinitialize +git submodule deinit -f inventories + +# 2. Remove from git +git rm -f inventories + +# 3. Remove module directory +rm -rf .git/modules/inventories + +# 4. Commit +git commit -m "Remove inventories submodule" +``` + +### Change Submodule URL + +```bash +# Update .gitmodules +git config --file=.gitmodules submodule.inventories.url + +# Sync +git submodule sync +git submodule update --init --recursive + +# Commit +git add .gitmodules +git commit -m "Update submodule URL" +``` + +## Quick Reference + +| Action | Command | +|--------|---------| +| Clone with submodules | `git clone --recurse-submodules ` | +| Init submodules | `git submodule init` | +| Update submodules | `git submodule update` | +| Update to latest | `git submodule update --remote` | +| Check status | `git submodule status` | +| Foreach command | `git submodule foreach ` | +| Work in submodule | `cd submodule && git checkout master` | +| Update parent reference | `git add submodule && git commit` | + +## Related Documentation + +- [Git SSH Setup](git-ssh-setup.md) - SSH key configuration +- [CLAUDE.md](../CLAUDE.md) - Repository structure guidelines +- [ansible-inventories README](https://git.mymx.me/ansible/ansible-inventories) - Inventory documentation +- [secrets README](https://git.mymx.me/ansible/secrets) - Secrets management (PRIVATE) + +## Compliance + +This submodule structure follows CLAUDE.md requirements with security enhancements: +- ✅ `./inventories` in PRIVATE repository (protects network topology) +- ✅ `./secrets` in PRIVATE repository (protects sensitive data) +- ✅ Separation of concerns +- ✅ Independent version control +- ✅ Proper access controls +- ✅ Network topology protection + +--- + +**Last Updated:** 2025-11-11 +**Workflow Version:** 1.0 diff --git a/inventories b/inventories index ebe29b6..dba3d7b 160000 --- a/inventories +++ b/inventories @@ -1 +1 @@ -Subproject commit ebe29b698f35d0fd896ae7ffb4b48ce21dcb8c4a +Subproject commit dba3d7b92217fe936b809414c2e2c9531a120215