432 lines
9.8 KiB
Markdown
432 lines
9.8 KiB
Markdown
# Harbor Registry Deployment
|
|
|
|
## Overview
|
|
|
|
This guide covers building and pushing FlaskPaste container images to a Harbor
|
|
registry, and deploying from Harbor to Kubernetes.
|
|
|
|
Harbor is an open-source container registry with security features including
|
|
vulnerability scanning, image signing, and role-based access control.
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
- Harbor registry deployed and accessible
|
|
- Container tools: `podman`, `docker`, or `ctr` (containerd)
|
|
- Network access to Harbor (direct or via SSH tunnel)
|
|
|
|
---
|
|
|
|
## Registry Configuration
|
|
|
|
### Harbor Details
|
|
|
|
```
|
|
Internal: 192.168.122.154:30443 (K8s cluster access)
|
|
External: harbor.mymx.me (CI/CD, remote access)
|
|
Project: library (public)
|
|
Repository: library/flaskpaste
|
|
```
|
|
|
|
### Authentication
|
|
|
|
```bash
|
|
Username: <harbor-username>
|
|
Password: <harbor-password>
|
|
```
|
|
|
|
For CI/CD automation, configure Gitea Actions secrets:
|
|
- `HARBOR_USER` - Harbor username (e.g., `ansible` automation account)
|
|
- `HARBOR_PASS` - Harbor password
|
|
|
|
---
|
|
|
|
## Building Images
|
|
|
|
### Local Build
|
|
|
|
```bash
|
|
# Build with Podman
|
|
podman build -t flaskpaste:latest .
|
|
|
|
# Build with Docker
|
|
docker build -t flaskpaste:latest .
|
|
|
|
# Verify the image
|
|
podman images | grep flaskpaste
|
|
```
|
|
|
|
### Multi-architecture Build
|
|
|
|
```bash
|
|
# Build for multiple architectures
|
|
podman manifest create flaskpaste:latest
|
|
podman build --platform linux/amd64 -t flaskpaste:amd64 .
|
|
podman build --platform linux/arm64 -t flaskpaste:arm64 .
|
|
podman manifest add flaskpaste:latest flaskpaste:amd64
|
|
podman manifest add flaskpaste:latest flaskpaste:arm64
|
|
```
|
|
|
|
---
|
|
|
|
## Pushing to Harbor
|
|
|
|
### Direct Push (Network Access)
|
|
|
|
```bash
|
|
# Login to Harbor (skip TLS verify for self-signed certs)
|
|
podman login 192.168.122.154:30443 \
|
|
-u "$HARBOR_USER" -p "$HARBOR_PASS" \
|
|
--tls-verify=false
|
|
|
|
# Tag for Harbor
|
|
podman tag localhost/flaskpaste:latest \
|
|
192.168.122.154:30443/library/flaskpaste:latest
|
|
|
|
# Push
|
|
podman push 192.168.122.154:30443/library/flaskpaste:latest \
|
|
--tls-verify=false
|
|
```
|
|
|
|
### Push via SSH Tunnel
|
|
|
|
When Harbor is on an internal network:
|
|
|
|
```bash
|
|
# Set up SSH port forward
|
|
ssh -f -N -L 30443:192.168.122.154:30443 jumphost
|
|
|
|
# Login and push via localhost
|
|
podman login localhost:30443 -u "$HARBOR_USER" -p "$HARBOR_PASS" --tls-verify=false
|
|
podman tag localhost/flaskpaste:latest localhost:30443/library/flaskpaste:latest
|
|
podman push localhost:30443/library/flaskpaste:latest --tls-verify=false
|
|
```
|
|
|
|
### Push via Image Transfer
|
|
|
|
When direct push isn't possible:
|
|
|
|
```bash
|
|
# Save image to tar
|
|
podman save localhost/flaskpaste:latest -o flaskpaste.tar
|
|
|
|
# Copy to machine with Harbor access
|
|
scp flaskpaste.tar user@k8s-master:/tmp/
|
|
|
|
# On k8s-master: Import and push via containerd
|
|
ssh user@k8s-master '
|
|
sudo ctr -n k8s.io images import /tmp/flaskpaste.tar
|
|
sudo ctr -n k8s.io images tag \
|
|
localhost/flaskpaste:latest \
|
|
192.168.122.154:30443/library/flaskpaste:latest
|
|
sudo ctr -n k8s.io images push --skip-verify \
|
|
--user <harbor-username>:<harbor-password> \
|
|
192.168.122.154:30443/library/flaskpaste:latest
|
|
'
|
|
```
|
|
|
|
---
|
|
|
|
## Pulling from Harbor
|
|
|
|
### Podman/Docker
|
|
|
|
```bash
|
|
podman pull 192.168.122.154:30443/library/flaskpaste:latest \
|
|
--tls-verify=false
|
|
```
|
|
|
|
### Containerd (Kubernetes nodes)
|
|
|
|
```bash
|
|
sudo ctr -n k8s.io images pull --skip-verify \
|
|
--user <harbor-username>:<harbor-password> \
|
|
192.168.122.154:30443/library/flaskpaste:latest
|
|
```
|
|
|
|
---
|
|
|
|
## Kubernetes Deployment
|
|
|
|
### Configure Registry Trust
|
|
|
|
For nodes to pull from Harbor with self-signed certificates:
|
|
|
|
**containerd** (`/etc/containerd/config.toml`):
|
|
|
|
```toml
|
|
[plugins."io.containerd.grpc.v1.cri".registry.configs]
|
|
[plugins."io.containerd.grpc.v1.cri".registry.configs."192.168.122.154:30443"]
|
|
[plugins."io.containerd.grpc.v1.cri".registry.configs."192.168.122.154:30443".tls]
|
|
insecure_skip_verify = true
|
|
[plugins."io.containerd.grpc.v1.cri".registry.configs."192.168.122.154:30443".auth]
|
|
username = "<harbor-username>"
|
|
password = "<harbor-password>"
|
|
```
|
|
|
|
Restart containerd after changes:
|
|
```bash
|
|
sudo systemctl restart containerd
|
|
```
|
|
|
|
### Create Image Pull Secret
|
|
|
|
```bash
|
|
kubectl create secret docker-registry harbor-creds \
|
|
--docker-server=192.168.122.154:30443 \
|
|
--docker-username=<harbor-username> \
|
|
--docker-password=<harbor-password> \
|
|
--docker-email=<your-email>
|
|
```
|
|
|
|
### Deployment Manifest
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: flaskpaste
|
|
labels:
|
|
app: flaskpaste
|
|
spec:
|
|
replicas: 2
|
|
selector:
|
|
matchLabels:
|
|
app: flaskpaste
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: flaskpaste
|
|
spec:
|
|
imagePullSecrets:
|
|
- name: harbor-creds
|
|
containers:
|
|
- name: flaskpaste
|
|
image: harbor.mymx.me/library/flaskpaste:latest
|
|
ports:
|
|
- containerPort: 5000
|
|
env:
|
|
- name: FLASK_ENV
|
|
value: "production"
|
|
- name: FLASKPASTE_DB
|
|
value: "/app/data/pastes.db"
|
|
volumeMounts:
|
|
- name: data
|
|
mountPath: /app/data
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /health
|
|
port: 5000
|
|
initialDelaySeconds: 10
|
|
periodSeconds: 30
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /health
|
|
port: 5000
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 10
|
|
resources:
|
|
requests:
|
|
memory: "128Mi"
|
|
cpu: "100m"
|
|
limits:
|
|
memory: "256Mi"
|
|
cpu: "500m"
|
|
volumes:
|
|
- name: data
|
|
persistentVolumeClaim:
|
|
claimName: flaskpaste-data
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: flaskpaste
|
|
spec:
|
|
selector:
|
|
app: flaskpaste
|
|
ports:
|
|
- port: 80
|
|
targetPort: 5000
|
|
type: ClusterIP
|
|
---
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: flaskpaste-data
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
resources:
|
|
requests:
|
|
storage: 1Gi
|
|
```
|
|
|
|
### Deploy
|
|
|
|
```bash
|
|
kubectl apply -f flaskpaste-k8s.yaml
|
|
|
|
# Verify deployment
|
|
kubectl get pods -l app=flaskpaste
|
|
kubectl logs -l app=flaskpaste
|
|
```
|
|
|
|
---
|
|
|
|
## CI/CD Integration
|
|
|
|
### Gitea Actions Example
|
|
|
|
```yaml
|
|
name: Build and Push to Harbor
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
tags: ['v*']
|
|
|
|
jobs:
|
|
build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
git clone --depth 1 --branch "${GITHUB_REF_NAME}" \
|
|
"https://oauth2:${{ github.token }}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git" .
|
|
|
|
- name: Build image
|
|
run: |
|
|
podman build -t flaskpaste:${{ github.sha }} .
|
|
podman tag flaskpaste:${{ github.sha }} flaskpaste:latest
|
|
|
|
- name: Push to Harbor
|
|
env:
|
|
HARBOR_USER: ${{ secrets.HARBOR_USER }}
|
|
HARBOR_PASS: ${{ secrets.HARBOR_PASS }}
|
|
run: |
|
|
podman login 192.168.122.154:30443 \
|
|
-u "$HARBOR_USER" -p "$HARBOR_PASS" --tls-verify=false
|
|
|
|
podman tag flaskpaste:${{ github.sha }} \
|
|
192.168.122.154:30443/library/flaskpaste:${{ github.sha }}
|
|
podman tag flaskpaste:latest \
|
|
192.168.122.154:30443/library/flaskpaste:latest
|
|
|
|
podman push 192.168.122.154:30443/library/flaskpaste:${{ github.sha }} \
|
|
--tls-verify=false
|
|
podman push 192.168.122.154:30443/library/flaskpaste:latest \
|
|
--tls-verify=false
|
|
```
|
|
|
|
---
|
|
|
|
## Harbor API
|
|
|
|
### Check Image Exists
|
|
|
|
```bash
|
|
curl -k -s -u <harbor-username>:<harbor-password> \
|
|
"https://192.168.122.154:30443/api/v2.0/projects/library/repositories/flaskpaste/artifacts" \
|
|
| jq '.[] | {digest: .digest, tags: [.tags[].name], size: .size}'
|
|
```
|
|
|
|
### List Tags
|
|
|
|
```bash
|
|
curl -k -s -u <harbor-username>:<harbor-password> \
|
|
"https://192.168.122.154:30443/api/v2.0/projects/library/repositories/flaskpaste/artifacts" \
|
|
| jq -r '.[].tags[].name'
|
|
```
|
|
|
|
### Delete Old Tags
|
|
|
|
```bash
|
|
# Delete specific tag
|
|
curl -k -X DELETE -u <harbor-username>:<harbor-password> \
|
|
"https://192.168.122.154:30443/api/v2.0/projects/library/repositories/flaskpaste/artifacts/v1.0.0"
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Connection Refused
|
|
|
|
```bash
|
|
# Check Harbor is running
|
|
kubectl get pods -n harbor
|
|
|
|
# Check service is exposed
|
|
kubectl get svc -n harbor
|
|
|
|
# Test connectivity
|
|
curl -k https://192.168.122.154:30443/api/v2.0/health
|
|
```
|
|
|
|
### Authentication Failed
|
|
|
|
```bash
|
|
# Verify credentials via API
|
|
curl -k -u <harbor-username>:<harbor-password> \
|
|
https://192.168.122.154:30443/api/v2.0/users/current
|
|
|
|
# Check if project exists
|
|
curl -k -u <harbor-username>:<harbor-password> \
|
|
https://192.168.122.154:30443/api/v2.0/projects
|
|
```
|
|
|
|
### TLS Certificate Errors
|
|
|
|
```bash
|
|
# For podman/docker: use --tls-verify=false
|
|
# For ctr: use --skip-verify
|
|
# For curl: use -k
|
|
|
|
# To properly trust the cert, export Harbor's CA:
|
|
kubectl get secret -n harbor harbor-nginx \
|
|
-o jsonpath='{.data.ca\.crt}' | base64 -d > harbor-ca.crt
|
|
sudo cp harbor-ca.crt /etc/pki/ca-trust/source/anchors/
|
|
sudo update-ca-trust
|
|
```
|
|
|
|
### Image Pull Failures in Kubernetes
|
|
|
|
```bash
|
|
# Check secret exists
|
|
kubectl get secret harbor-creds -o yaml
|
|
|
|
# Verify pod can reach Harbor
|
|
kubectl run test --rm -it --image=alpine -- \
|
|
wget -qO- --no-check-certificate https://192.168.122.154:30443/api/v2.0/health
|
|
|
|
# Check events
|
|
kubectl describe pod <pod-name> | grep -A10 Events
|
|
```
|
|
|
|
---
|
|
|
|
## Version Tagging Strategy
|
|
|
|
```
|
|
latest - Most recent build from main branch
|
|
v1.2.3 - Semantic version releases
|
|
sha-abc123 - Git commit SHA (for traceability)
|
|
main-20260117 - Branch + date (for nightly builds)
|
|
```
|
|
|
|
Example workflow:
|
|
```bash
|
|
VERSION="1.0.0"
|
|
COMMIT=$(git rev-parse --short HEAD)
|
|
|
|
podman build -t flaskpaste .
|
|
podman tag flaskpaste 192.168.122.154:30443/library/flaskpaste:v${VERSION}
|
|
podman tag flaskpaste 192.168.122.154:30443/library/flaskpaste:sha-${COMMIT}
|
|
podman tag flaskpaste 192.168.122.154:30443/library/flaskpaste:latest
|
|
|
|
for tag in v${VERSION} sha-${COMMIT} latest; do
|
|
podman push 192.168.122.154:30443/library/flaskpaste:${tag} --tls-verify=false
|
|
done
|
|
```
|