initial runbook

2026-04-30 14:06:21 -05:00
commit c8598f8985
2 changed files with 840 additions and 0 deletions
--- a/Runbook.md
+++ b/Runbook.md
@@ -0,0 +1,800 @@
+# Runbook: Self-hosted Pilot
+
+## Gaussian Wellworks
+
+## 1. Runbook Objective
+
+## 2. Critical Dependencies
+
+### 2.1 Business Domain Ownership
+### 2.2 Administrative Access
+### 2.3 Decision Checklist
+
+## 3. Detailed Runbook
+
+### 3.1 Create Digital Ocean Account
+
+### 3.2 Create SSH Key Pair (Windows 11)
+
+**Generate key**
+```Powershell
+</> Powershell
+
+ssh-keygen -t ed25519 -C "admin@company.com"
+```
+
+Accept default path:
+```
+C:\Users\<user>\.ssh\id_ed25519
+```
+
+**Copy public key**
+```Powershell
+</> Powershell
+
+Get-Content $env:USERPROFILE\.ssh\id_ed25519.pub | Set-Clipboard
+```
+
+**Upload to DigitalOcean** 
+
+- Settings → Security → SSH Keys → Add
+- Name: admin-windows11
+
+**Validation**
+
+[✔] Key visible in DigitalOcean
+
+
+### 3.3 Provision Droplet
+
+**Configuration**
+```
+Image: Ubuntu 22.04 LTS
+Plan: Premium Intel
+Size: 4 vCPU / 8GB RAM
+Region: closest to users
+Auth: SSH key
+Backups: Enabled
+Hostname: gw-drive-01
+```
+
+**Test SSH**
+```Powershell
+ssh root@<IP>
+```
+
+**Validation**
+
+[✔] SSH login works
+[✔] IP recorded
+
+### 3.4 Initial Server Configuration
+
+**Create admin user**
+
+```Bash
+</> Bash
+
+adduser adminuser
+usermod -aG sudo adminuser
+```
+
+**Copy SSH key**
+
+```Bash
+</> Bash
+
+rsync --archive --chown=adminuser:adminuser ~/.ssh /home/adminuser
+```
+
+**Disable root login**
+
+```Bash
+</> Bash
+
+sudo tee /etc/ssh/sshd_config > /dev/null << 'EOF'
+PermitRootLogin no
+PasswordAuthentication no
+EOF
+```
+
+```Bash
+</> Bash
+
+sudo systemctl restart ssh
+```
+
+**Test login**
+
+```Powershell
+</> Powershell
+
+ssh adminuser@<IP>
+```
+
+**Update system**
+
+```Bash
+</> Bash
+
+sudo apt update
+sudo apt upgrade -y
+sudo apt autoremove -y
+```
+
+**Validation**
+[✔] adminuser login works
+[✔] root login blocked
+[✔] system updated
+
+### 3.5 Configure Firewall
+
+**DigitalOcean Firewall**
+
+Allow:
+```
+22 (SSH) → your IP
+80 (HTTP) → all
+443 (HTTPS) → all
+```
+
+Attach to droplet.
+
+**UFW**
+
+```Bash
+</> Bash
+
+sudo apt install ufw -y
+sudo ufw default deny incoming
+sudo ufw default allow outgoing
+
+sudo ufw allow 22/tcp
+sudo ufw allow 80/tcp
+sudo ufw allow 443/tcp
+
+sudo ufw enable
+```
+
+**Validation**
+```Bash
+</> Bash 
+
+sudo ufw status
+```
+
+### 3.6 Configure DNS
+
+**Create A record**
+```
+Host: drive
+Value: <DROPLET_IP>
+TTL: 300
+```
+
+**Validate**
+```Powershell
+</> Powershell
+
+nslookup drive.company.com
+```
+
+*Note*
+⚠ DNS propagation required before SSL
+
+
+### 3.7 Install Docker
+
+```Bash
+</> Bash
+sudo apt install ca-certificates curl gnupg -y
+sudo mkdir -p /etc/apt/keyrings
+
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
+sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
+```
+
+```Bash
+</> Bash
+echo \
+"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
+https://download.docker.com/linux/ubuntu \
+$(lsb_release -cs) stable" | \
+sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+```
+
+```Bash
+</> Bash
+sudo apt update
+sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y
+```
+
+**Enable Docker**
+
+```Bash
+</> Bash
+sudo systemctl enable docker
+sudo systemctl start docker
+sudo usermod -aG docker adminuser
+```
+
+Reconnect SSH.
+
+**Test**
+
+```Bash
+</> Bash
+
+docker run hello-world
+```
+
+### 3.10 Deploy Nextcloud
+
+**Create directory**
+
+```Bash
+</> Bash
+
+mkdir -p ~/apps/nextcloud
+cd ~/apps/nextcloud
+```
+
+**Create .env**
+
+```Bash
+</> Bash
+sudo tee .env > /dev/null << 'EOF'
+MYSQL_ROOT_PASSWORD=StrongRootPassword
+MYSQL_PASSWORD=StrongDBPassword
+MYSQL_DATABASE=nextcloud
+MYSQL_USER=nextcloud
+
+NEXTCLOUD_ADMIN_USER=admin
+NEXTCLOUD_ADMIN_PASSWORD=StrongAdminPassword
+
+MYSQL_HOST=db
+REDIS_HOST=redis
+NEXTCLOUD_TRUSTED_DOMAINS=<IP>
+EOF
+```
+
+**Create docker-compose**
+
+```Bash
+</> Bash
+sudo tee docker-compose.yml > /dev/null << 'EOF'
+version: '3.9'
+
+services:
+  db:
+    image: mariadb:10.6
+    restart: always
+    env_file:
+      - .env
+    volumes:
+      - db_data:/var/lib/mysql
+
+  redis:
+    image: redis:7-alpine
+    restart: always
+
+  app:
+    image: nextcloud:apache
+    restart: always
+    ports:
+      - "8080:80"
+    env_file:
+      - .env
+    depends_on:
+      - db
+      - redis
+    volumes:
+      - nextcloud_data:/var/www/html
+
+volumes:
+  db_data:
+  nextcloud_data:
+EOF
+```
+
+**Start**
+
+```Bash
+</> Bash
+
+docker compose up -d
+```
+
+**Validate**
+```
+http://<IP>:8080
+```
+
+### 3.11 Validate Internal Access
+
+**Test:**
+
+- [ ] Login
+- [ ] Upload file
+- [ ] Create folder
+- [ ] Restart containers
+- [ ] Data persists
+
+### 3.12 Install Nginx
+
+```Bash
+</> Bash
+sudo apt install nginx -y
+sudo systemctl enable nginx
+sudo systemctl start nginx
+```
+
+**Test:**
+```
+http://<IP> → nginx welcome page
+```
+
+### 3.13 Configure HTTP Reverse Proxy (Staging Only)
+
+```Bash
+</> Bash
+
+sudo tee /etc/nginx/sites-available/nextcloud > /dev/null << 'EOF'
+server {
+    listen 80;
+    server_name drive.company.com;
+
+    location / {
+        proxy_pass http://127.0.0.1:8080;
+
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto http;
+    }
+}
+EOF
+```
+
+**Enable:**
+
+```Bash
+</> Bash
+
+sudo ln -s /etc/nginx/sites-available/nextcloud /etc/nginx/sites-enabled/
+sudo nginx -t
+sudo systemctl reload nginx
+```
+
+*Important*
+- ⚠ This step is NOT fully usable yet
+- ⚠ HTTPS may redirect and show 404 (expected)
+
+### 3.14 Install SSL (Certbot)
+
+```Bash
+</> Bash
+
+sudo apt install certbot python3-certbot-nginx -y
+sudo certbot --nginx -d drive.company.com
+```
+
+**Validate**
+- ✔ Certificate issued
+- ✔ HTTPS reachable
+- ⚠ May still show 404 (expected)
+
+### 3.15 Finalize HTTPS Reverse Proxy
+
+**Replace config**
+
+```Bash
+</> Bash
+
+sudo tee /etc/nginx/sites-available/nextcloud > /dev/null << 'EOF'
+server {
+    listen 80;
+    server_name drive.company.com;
+
+    return 301 https://$host$request_uri;
+}
+
+server {
+    listen 443 ssl http2;
+    server_name drive.company.com;
+
+    ssl_certificate /etc/letsencrypt/live/drive.company.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/drive.company.com/privkey.pem;
+
+    client_max_body_size 10G;
+
+    location / {
+        proxy_pass http://127.0.0.1:8080;
+
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+    }
+}
+EOF
+```
+
+**Reload**
+
+```Bash
+</> Bash
+
+sudo nginx -t
+sudo systemctl reload nginx
+```
+
+**Fix Nextcloud**
+
+```Bash
+</> Bash
+
+docker exec -u www-data nextcloud-app php occ config:system:set trusted_domains 1 --value="drive.company.com"
+docker exec -u www-data nextcloud-app php occ config:system:set overwrite.cli.url --value="https://drive.company.com"
+docker exec -u www-data nextcloud-app php occ config:system:set overwriteprotocol --value="https"
+```
+
+**Final Validation**
+
+https://drive.company.com
+
+- ✔ Login works
+- ✔ Upload works
+- ✔ Download works
+- ✔ No warnings
+- ✔ HTTPS padlock present
+
+**Cleanup**
+
+```Bash
+</> Bash
+
+sudo ufw delete allow 8080/tcp
+```
+
+✅ Final Outcome
+- ✔ Secure Nextcloud deployment
+- ✔ HTTPS reverse proxy functional
+- ✔ Docker-based architecture
+- ✔ Ready for users and workflows
+
+### 3.16 Create Email Provider Account (Fastmail)
+
+**Objective**
+
+Establish a business-grade email platform using Fastmail so that:
+
+- domain-based email (user@company.com)
+- reliable outbound mail (for Nextcloud notifications later)
+- foundation for SPF/DKIM/DMARC
+
+**Architectural Context**
+
+Email is not hosted on your droplet. Your infrastructure will later integrate with it for:
+
+- Nextcloud notifications
+- User accounts
+- Client communication
+
+**Prerequisites**
+- Domain ownership confirmed
+- Access to DNS registrar
+- Decision: Fastmail selected (per runbook)
+
+**Step 1 - Create Fastmail Account**
+
+Go to: https://www.fastmail.com
+
+1.1 Sign up
+- Click Sign Up
+- Choose a plan (Standard is sufficient for pilot)
+- Enter:
+    ```
+    Name
+    Email (temporary Fastmail domain)
+    Password
+    ```
+
+1.2 Complete account creation
+
+You will land in the Fastmail admin interface.
+
+**Step 2 - Add Your Domain**
+
+2.1 Navigate to Domains
+
+In Fastmail:
+```
+Settings → Domains → Add Domain
+```
+
+2.2 Enter your domain
+```
+company.com
+```
+Click:
+```
+Add Domain
+```
+
+**Step 3 - Verify Domain Ownership**
+
+Fastmail will present a TXT verification record.
+
+3.1 Copy TXT record
+
+Example:
+```
+Type: TXT
+Name: @
+Value: fm-domain-verification=abc123xyz
+```
+
+3.2 Add to DNS (Registrar)
+
+In your DNS provider (e.g., GoDaddy or Cloudflare):
+
+Add:
+```
+Type: TXT
+Host: @
+Value: fm-domain-verification=abc123xyz
+TTL: 300
+```
+
+3.3 Wait for propagation
+```
+~5–15 minutes typical, but may be as much as 24 hours
+```
+
+3.4 Verify in Fastmail
+
+Click:
+```
+Verify Domain
+```
+
+Expected Result
+- Domain verified successfully
+
+**Step 4 - Set Domain as Default**
+
+In Fastmail:
+```
+Settings → Domains → company.com → Set as default
+```
+
+This ensures:
+
+- new users → user@company.com
+
+
+**Step 5 - Create Initial Admin Mailbox**
+
+5.1 Navigate to Users
+```
+Settings → Users → Add User
+```
+
+5.2 Create admin user
+
+Example:
+```
+Username: admin
+Email: admin@company.com
+Password: (strong password)
+```
+
+5.3 Save
+
+Expected Result
+
+- Mailbox created
+- Can send/receive (after DNS setup in 3.8)
+
+
+**Step 6 - Basic Access Test (Pre-DNS)**
+
+Login via:
+```
+https://app.fastmail.com
+```
+
+Expected
+- Inbox accessible
+- Account functional (even before MX setup)
+- Email is NOT functional yet
+
+**Validation Checklist**
+- [✔] Fastmail account created
+- [✔] Domain added to Fastmail
+- [✔] TXT verification record created
+- [✔] Domain verified
+- [✔] Default domain set
+- [✔] Admin mailbox created
+- [✔] Able to login to Fastmail
+
+### 3.17 Configure Email DNS
+
+**Objective**
+
+Make your domain fully capable of sending and receiving email by configuring DNS records required by Fastmail:
+
+- Incoming mail routing (MX)
+- Authorized senders (SPF)
+- Cryptographic signing (DKIM)
+- Policy & reporting (DMARC)
+
+*Changing MX records WILL impact live email.*
+
+If the client currently uses another provider:
+
+- Schedule a cutover window
+- Lower TTL beforehand (if possible)
+- Expect brief delivery delays during propagation
+
+**Prerequisites**
+- Domain added & verified in Fastmail (Task 3.7)
+- Access to DNS registrar
+- Admin mailbox created
+
+**Step 1 - Gather Fastmail DNS Records**
+
+In Fastmail:
+```
+Settings → Domains → company.com → DNS Settings
+```
+
+Fastmail will provide all required records. Use those exact values if they differ from below.
+
+**Step 2 - Configure MX Records (Mail Routing)**
+
+2.1 Remove Existing MX Records
+
+⚠️ Only do this at cutover time.
+
+2.2 Add Fastmail MX Records
+```
+Type: MX
+Host: @
+Priority: 10
+Value: in1-smtp.messagingengine.com
+
+Type: MX
+Host: @
+Priority: 20
+Value: in2-smtp.messagingengine.com
+```
+
+**Step 3 - Configure SPF (Sender Authorization)**
+
+3.1 Add SPF record
+```
+Type: TXT
+Host: @
+Value: v=spf1 include:spf.messagingengine.com ~all
+TTL: 300
+```
+
+Notes
+- Only ONE SPF record per domain
+- Merge with existing SPF if present
+
+**Step 4 - Configure DKIM (Email Signing)**
+
+4.1 Get DKIM from Fastmail
+
+In Fastmail DNS settings, copy the DKIM record.
+
+Example:
+```
+Type: CNAME
+Host: fm1._domainkey
+Value: fm1.company.com.dkim.fmhosted.com
+
+Type: CNAME
+Host: fm2._domainkey
+Value: fm2.company.com.dkim.fmhosted.com
+
+Type: CNAME
+Host: fm3._domainkey
+Value: fm3.company.com.dkim.fmhosted.com
+4.2 Add all DKIM records
+```
+
+These enable:
+- Email authenticity verification
+- Better deliverability
+
+**Step 5 - Configure DMARC (Policy & Reporting)**
+
+5.1 Add DMARC record
+```
+Type: TXT
+Host: _dmarc
+Value: v=DMARC1; p=none; rua=mailto:admin@company.com
+TTL: 300
+```
+
+Explanation
+- p=none   → monitor only (safe for pilot)
+- rua=     → aggregate reports sent to admin mailbox
+
+Future Hardening
+Later you can move to:
+- p=quarantine → soft enforcement
+- p=reject     → full enforcement
+
+**Step 6 - Verify DNS Propagation**
+
+6.1 Check MX
+nslookup -type=MX company.com
+
+Expected:
+
+in1-smtp.messagingengine.com
+in2-smtp.messagingengine.com
+6.2 Check SPF
+nslookup -type=TXT company.com
+6.3 Check DKIM
+nslookup -type=CNAME fm1._domainkey.company.com
+6.4 Check DMARC
+nslookup -type=TXT _dmarc.company.com
+
+**Step 7 - Functional Email Test**
+7.1 Send inbound test
+
+From Gmail/Outlook:
+```
+Send email to: admin@company.com
+```
+Expected:
+- Email received in Fastmail inbox
+
+7.2 Send outbound test
+
+From Fastmail:
+```
+Send email to external address
+```
+Expected:
+- Email delivered
+- Not marked as spam
+
+**Step 8 - Validate Deliverability (Recommended)**
+
+Use:
+
+- mail-tester.com
+- mxtoolbox.com
+
+Expected
+- SPF pass
+- DKIM pass
+- DMARC pass
+- Low spam score
+
+**Step 9 - Propagation Expectations**
+- MX:     5–30 minutes typical
+- SPF:    immediate to 1 hour
+- DKIM:   15–60 minutes
+- DMARC:  15–60 minutes
+
+Worst case:
+Up to 24 hours
+
+**Validation Checklist**
+- [✔] MX records updated
+- [✔] SPF record present
+- [✔] DKIM records configured (all 3)
+- [✔] DMARC record created
+- [✔] DNS resolves correctly
+- [✔] Email received externally
+- [✔] Email sent externally
+- [✔] Deliverability validated
+