infra-automation/cheatsheets/test-deploy-linux-vm-role.md

# Test Deploy Linux VM Role - Quick Reference

## Playbook
`plays/test-deploy-linux-vm-role.yml`

## Description
Test playbook for the `deploy_linux_vm` role. Deploys a Debian 12 test VM with full LVM configuration and SSH hardening (GSSAPI disabled) to validate role functionality.

## Quick Test

### Run Full Test
```bash
ansible-playbook plays/test-deploy-linux-vm-role.yml
```

### Test with Different Distribution
```bash
# Test with Ubuntu
ansible-playbook plays/test-deploy-linux-vm-role.yml \
  -e "deploy_linux_vm_os_distribution=ubuntu-22.04"

# Test with AlmaLinux
ansible-playbook plays/test-deploy-linux-vm-role.yml \
  -e "deploy_linux_vm_os_distribution=almalinux-9"
```

### Test with Custom Resources
```bash
ansible-playbook plays/test-deploy-linux-vm-role.yml \
  -e "deploy_linux_vm_vcpus=4" \
  -e "deploy_linux_vm_memory_mb=4096"
```

## Test Configuration

### Default Test Settings
```yaml
VM Name: test-lvm-vm
Hostname: test-lvm
Distribution: debian-12
vCPUs: 2
Memory: 2048 MB
Disk: 20 GB
LVM: Enabled (30GB on /dev/vdb)
```

### Features Being Tested
✅ LVM configuration (CLAUDE.md compliant)
✅ SSH hardening (GSSAPI disabled)
✅ Multi-distribution support
✅ Security features (Firewall, SELinux/AppArmor, Audit)
✅ Automatic updates
✅ Cloud-init provisioning

## Test Execution Flow

### 1. Pre-flight Validation
- Checks if test VM already exists
- Validates distribution support
- Verifies virtualization capabilities

### 2. VM Deployment (~2-3 minutes)
- Downloads cloud image (if not cached)
- Creates primary disk (20GB)
- Creates LVM disk (30GB on /dev/vdb)
- Generates cloud-init with security hardening
- Deploys VM with both disks

### 3. LVM Configuration (~3-5 minutes)
- Installs LVM packages
- Creates vg_system volume group
- Creates 8 logical volumes
- Formats filesystems
- Migrates existing data
- Updates /etc/fstab

### 4. Validation
- Verifies VM is running
- Checks SSH connectivity
- Validates LVM configuration
- Confirms security features

### Total Test Time: ~5-8 minutes

## Post-Test Verification

### Get Test VM Information
```bash
# Get IP address
ssh grokbox "virsh domifaddr test-lvm-vm"

# SSH to test VM
ssh -J grokbox ansible@<VM_IP>
```

### Verify LVM Configuration
```bash
# Check LVM status
ssh -J grokbox ansible@<VM_IP> "sudo pvs && sudo vgs && sudo lvs"

# Expected output:
# VG: vg_system
# PV: /dev/vdb (30.00g)
# LVs: lv_opt, lv_tmp, lv_home, lv_var, lv_var_log, lv_var_tmp, lv_var_audit, lv_swap
```

### Verify SSH Hardening
```bash
# Check GSSAPI is disabled
ssh -J grokbox ansible@<VM_IP> "sudo sshd -T | grep -i gssapi"

# Expected output:
# gssapiauthentication no
# gssapicleanupcredentials no
```

### Verify Security Features
```bash
# Check firewall (Debian)
ssh -J grokbox ansible@<VM_IP> "sudo ufw status"

# Check SELinux (RHEL - if testing RHEL family)
ssh -J grokbox ansible@<VM_IP> "getenforce"
# Expected: Enforcing
```

### Reboot and Verify LVM Mounts
```bash
# Reboot test VM
ssh -J grokbox ansible@<VM_IP> "sudo reboot"

# Wait ~1 minute, then verify all mounts
ssh -J grokbox ansible@<VM_IP> "df -h && lsblk"

# Check all LVM volumes are mounted
ssh -J grokbox ansible@<VM_IP> "mount | grep vg_system"
```

## Tag-Based Testing

### Test Specific Components
```bash
# Test pre-flight validation only
ansible-playbook plays/test-deploy-linux-vm-role.yml --tags validate,preflight

# Test VM deployment only (skip LVM)
ansible-playbook plays/test-deploy-linux-vm-role.yml --skip-tags lvm

# Test LVM configuration only (if VM exists)
ansible-playbook plays/test-deploy-linux-vm-role.yml --tags lvm,post-deploy

# Test cloud-init generation only
ansible-playbook plays/test-deploy-linux-vm-role.yml --tags cloud-init
```

## Test Validation Checklist

Complete this checklist to validate test success:

### Basic Functionality
- [ ] Playbook completes without errors
- [ ] VM created: `virsh list | grep test-lvm-vm`
- [ ] IP assigned: `virsh domifaddr test-lvm-vm`
- [ ] SSH accessible: `ssh -J grokbox ansible@<VM_IP>`

### LVM Configuration
- [ ] Physical volume exists: `sudo pvs` shows /dev/vdb
- [ ] Volume group created: `sudo vgs` shows vg_system
- [ ] All 8 LVs created: `sudo lvs` shows all volumes
- [ ] Filesystems formatted: `lsblk -f` shows ext4/swap
- [ ] Fstab updated: `cat /etc/fstab | grep vg_system`
- [ ] After reboot, all mounted: `df -h | grep vg_system`

### SSH Security
- [ ] GSSAPI disabled: `sudo sshd -T | grep gssapiauthentication` shows "no"
- [ ] GSSAPI cleanup disabled: `sudo sshd -T | grep gssapicleanupcredentials` shows "no"
- [ ] Root login disabled: `sudo sshd -T | grep permitrootlogin` shows "no"
- [ ] Password auth disabled: `sudo sshd -T | grep passwordauthentication` shows "no"
- [ ] Key-based auth works: SSH connection successful

### Security Features
- [ ] Firewall enabled: `sudo ufw status` (Debian) or `sudo firewall-cmd --state` (RHEL)
- [ ] Audit daemon running: `systemctl status auditd`
- [ ] Time sync active: `chronyc tracking`
- [ ] SELinux enforcing (RHEL): `getenforce` shows "Enforcing"

### System Health
- [ ] Cloud-init complete: `cloud-init status` shows "done"
- [ ] System updated: Package updates applied during cloud-init
- [ ] No errors in logs: Check `/var/log/cloud-init-output.log`
- [ ] Swap active: `free -h | grep Swap` shows non-zero

## Cleanup After Testing

### Delete Test VM
```bash
# Stop and remove test VM
ssh grokbox "virsh destroy test-lvm-vm"
ssh grokbox "virsh undefine test-lvm-vm --remove-all-storage"

# Verify removal
ssh grokbox "virsh list --all | grep test-lvm-vm"
# Should return nothing
```

### Clean Test Files
```bash
# On hypervisor, remove any temporary files
ssh grokbox "rm -f /tmp/cloud-init-test-lvm-vm/*"
ssh grokbox "rm -f /tmp/*-CHECKSUM"
```

## Testing Different Distributions

### Test Debian Family
```bash
# Debian 12
ansible-playbook plays/test-deploy-linux-vm-role.yml \
  -e "deploy_linux_vm_os_distribution=debian-12" \
  -e "deploy_linux_vm_name=test-debian12"

# Ubuntu 22.04
ansible-playbook plays/test-deploy-linux-vm-role.yml \
  -e "deploy_linux_vm_os_distribution=ubuntu-22.04" \
  -e "deploy_linux_vm_name=test-ubuntu22"
```

### Test RHEL Family
```bash
# AlmaLinux 9
ansible-playbook plays/test-deploy-linux-vm-role.yml \
  -e "deploy_linux_vm_os_distribution=almalinux-9" \
  -e "deploy_linux_vm_name=test-alma9"

# Rocky Linux 9
ansible-playbook plays/test-deploy-linux-vm-role.yml \
  -e "deploy_linux_vm_os_distribution=rocky-9" \
  -e "deploy_linux_vm_name=test-rocky9"
```

## Troubleshooting Tests

### Test Fails at Pre-flight
```bash
# Check if test VM already exists
ssh grokbox "virsh list --all | grep test-lvm-vm"

# If exists, delete it
ssh grokbox "virsh destroy test-lvm-vm"
ssh grokbox "virsh undefine test-lvm-vm --remove-all-storage"

# Re-run test
ansible-playbook plays/test-deploy-linux-vm-role.yml
```

### LVM Configuration Fails
```bash
# Check if second disk is attached
ssh grokbox "virsh domblklist test-lvm-vm"

# Should show both:
# - test-lvm-vm.qcow2 (primary)
# - test-lvm-vm-lvm.qcow2 (LVM disk)

# Verify disk visibility on VM
ssh -J grokbox ansible@<VM_IP> "lsblk"
# Should show vda (20G) and vdb (30G)
```

### SSH Connection Issues
```bash
# Check VM is running
ssh grokbox "virsh list | grep test-lvm-vm"

# Get IP again
ssh grokbox "virsh domifaddr test-lvm-vm"

# Test with verbose SSH
ssh -vvv -J grokbox ansible@<VM_IP>

# Check SSH service on VM (via console)
ssh grokbox "virsh console test-lvm-vm"
```

### GSSAPI Still Enabled
```bash
# Check SSH config file was created
ssh -J grokbox ansible@<VM_IP> "ls -la /etc/ssh/sshd_config.d/"

# View security config
ssh -J grokbox ansible@<VM_IP> "cat /etc/ssh/sshd_config.d/99-security.conf"

# Restart SSH service
ssh -J grokbox ansible@<VM_IP> "sudo systemctl restart sshd"

# Test again
ssh -J grokbox ansible@<VM_IP> "sudo sshd -T | grep gssapi"
```

## Continuous Testing

### Automated Test Suite
Create a test script for continuous validation:

```bash
#!/bin/bash
# test-role.sh

DISTRIBUTIONS=(
  "debian-12"
  "ubuntu-22.04"
  "almalinux-9"
  "rocky-9"
)

for distro in "${DISTRIBUTIONS[@]}"; do
  echo "Testing $distro..."

  ansible-playbook plays/test-deploy-linux-vm-role.yml \
    -e "deploy_linux_vm_os_distribution=$distro" \
    -e "deploy_linux_vm_name=test-$distro"

  if [ $? -eq 0 ]; then
    echo "✅ $distro test PASSED"
    # Cleanup
    ssh grokbox "virsh destroy test-$distro && virsh undefine test-$distro --remove-all-storage"
  else
    echo "❌ $distro test FAILED"
    exit 1
  fi
done

echo "All tests completed successfully!"
```

## Expected Test Output

Successful test execution should show:

```
PLAY [Test Deploy Linux VM Role] ***********************************

TASK [Gathering Facts] *********************************************
ok: [grokbox]

...

TASK [deploy_linux_vm : Display LVM configuration summary] ********
ok: [grokbox] => {
    "msg": [
        "=== LVM Configuration Complete ===",
        "Volume Group: vg_system",
        "Physical Volume: /dev/vdb (30GB)",
        "Logical Volumes: 8",
        "",
        "⚠️  IMPORTANT: VM needs reboot to use new mounts",
        "After reboot, LVM volumes will be mounted automatically"
    ]
}

TASK [Display test completion message] *****************************
ok: [grokbox] => {
    "msg": [
        "╔════════════════════════════════════════════════════════════════╗",
        "║              Role Test Completed Successfully                  ║",
        "╚════════════════════════════════════════════════════════════════╝",
        "",
        "VM deployed with:",
        "  ✓ LVM Configuration (CLAUDE.md compliant)",
        "  ✓ SSH Hardening (GSSAPI disabled)",
        "  ✓ Security Features (Firewall, Audit, Auto-updates)",
        "  ✓ Multi-distribution support",
        "",
        "Next steps:",
        "  1. SSH to VM: ssh -J grokbox ansible@192.168.122.X",
        "  2. Verify GSSAPI: sudo sshd -T | grep -i gssapi",
        "  3. Check LVM: sudo vgs && sudo lvs",
        "  4. Reboot for LVM: sudo reboot",
        "  5. After reboot verify: df -h && lsblk"
    ]
}

PLAY RECAP *********************************************************
grokbox: ok=X changed=Y unreachable=0 failed=0 skipped=Z
```

## Test Documentation

This test validates:

1. **Role Structure**: Proper task organization and variable handling
2. **Multi-Distribution**: Works across Debian, Ubuntu, RHEL families
3. **LVM Implementation**: Creates CLAUDE.md compliant LVM layout
4. **SSH Security**: GSSAPI disabled, key-only authentication
5. **Security Hardening**: Firewall, SELinux/AppArmor, audit daemon
6. **Cloud-Init**: Proper provisioning and package installation
7. **Idempotency**: Can be re-run without errors

## Related Documentation

- Role README: `roles/deploy_linux_vm/README.md`
- Role cheatsheet: `cheatsheets/deploy-linux-vm-role.md`
- CLAUDE.md: Infrastructure requirements
- Test playbook: `plays/test-deploy-linux-vm-role.yml`

## Support

For test failures:
- Check playbook output for specific error messages
- Review role tasks: `roles/deploy_linux_vm/tasks/`
- Check VM logs: `/var/log/cloud-init-output.log`
- Verify hypervisor resources: disk space, memory
- Consult role README for detailed troubleshooting