Files
gaussian-wellworks/Runbook.md
2026-04-30 14:06:21 -05:00

801 lines
13 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Runbook: Self-hosted Pilot
## Gaussian Wellworks
## 1. Runbook Objective
## 2. Critical Dependencies
### 2.1 Business Domain Ownership
### 2.2 Administrative Access
### 2.3 Decision Checklist
## 3. Detailed Runbook
### 3.1 Create Digital Ocean Account
### 3.2 Create SSH Key Pair (Windows 11)
**Generate key**
```Powershell
</> Powershell
ssh-keygen -t ed25519 -C "admin@company.com"
```
Accept default path:
```
C:\Users\<user>\.ssh\id_ed25519
```
**Copy public key**
```Powershell
</> Powershell
Get-Content $env:USERPROFILE\.ssh\id_ed25519.pub | Set-Clipboard
```
**Upload to DigitalOcean**
- Settings → Security → SSH Keys → Add
- Name: admin-windows11
**Validation**
[✔] Key visible in DigitalOcean
### 3.3 Provision Droplet
**Configuration**
```
Image: Ubuntu 22.04 LTS
Plan: Premium Intel
Size: 4 vCPU / 8GB RAM
Region: closest to users
Auth: SSH key
Backups: Enabled
Hostname: gw-drive-01
```
**Test SSH**
```Powershell
ssh root@<IP>
```
**Validation**
[✔] SSH login works
[✔] IP recorded
### 3.4 Initial Server Configuration
**Create admin user**
```Bash
</> Bash
adduser adminuser
usermod -aG sudo adminuser
```
**Copy SSH key**
```Bash
</> Bash
rsync --archive --chown=adminuser:adminuser ~/.ssh /home/adminuser
```
**Disable root login**
```Bash
</> Bash
sudo tee /etc/ssh/sshd_config > /dev/null << 'EOF'
PermitRootLogin no
PasswordAuthentication no
EOF
```
```Bash
</> Bash
sudo systemctl restart ssh
```
**Test login**
```Powershell
</> Powershell
ssh adminuser@<IP>
```
**Update system**
```Bash
</> Bash
sudo apt update
sudo apt upgrade -y
sudo apt autoremove -y
```
**Validation**
[✔] adminuser login works
[✔] root login blocked
[✔] system updated
### 3.5 Configure Firewall
**DigitalOcean Firewall**
Allow:
```
22 (SSH) → your IP
80 (HTTP) → all
443 (HTTPS) → all
```
Attach to droplet.
**UFW**
```Bash
</> Bash
sudo apt install ufw -y
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
```
**Validation**
```Bash
</> Bash
sudo ufw status
```
### 3.6 Configure DNS
**Create A record**
```
Host: drive
Value: <DROPLET_IP>
TTL: 300
```
**Validate**
```Powershell
</> Powershell
nslookup drive.company.com
```
*Note*
⚠ DNS propagation required before SSL
### 3.7 Install Docker
```Bash
</> Bash
sudo apt install ca-certificates curl gnupg -y
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
```
```Bash
</> Bash
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
```
```Bash
</> Bash
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y
```
**Enable Docker**
```Bash
</> Bash
sudo systemctl enable docker
sudo systemctl start docker
sudo usermod -aG docker adminuser
```
Reconnect SSH.
**Test**
```Bash
</> Bash
docker run hello-world
```
### 3.10 Deploy Nextcloud
**Create directory**
```Bash
</> Bash
mkdir -p ~/apps/nextcloud
cd ~/apps/nextcloud
```
**Create .env**
```Bash
</> Bash
sudo tee .env > /dev/null << 'EOF'
MYSQL_ROOT_PASSWORD=StrongRootPassword
MYSQL_PASSWORD=StrongDBPassword
MYSQL_DATABASE=nextcloud
MYSQL_USER=nextcloud
NEXTCLOUD_ADMIN_USER=admin
NEXTCLOUD_ADMIN_PASSWORD=StrongAdminPassword
MYSQL_HOST=db
REDIS_HOST=redis
NEXTCLOUD_TRUSTED_DOMAINS=<IP>
EOF
```
**Create docker-compose**
```Bash
</> Bash
sudo tee docker-compose.yml > /dev/null << 'EOF'
version: '3.9'
services:
db:
image: mariadb:10.6
restart: always
env_file:
- .env
volumes:
- db_data:/var/lib/mysql
redis:
image: redis:7-alpine
restart: always
app:
image: nextcloud:apache
restart: always
ports:
- "8080:80"
env_file:
- .env
depends_on:
- db
- redis
volumes:
- nextcloud_data:/var/www/html
volumes:
db_data:
nextcloud_data:
EOF
```
**Start**
```Bash
</> Bash
docker compose up -d
```
**Validate**
```
http://<IP>:8080
```
### 3.11 Validate Internal Access
**Test:**
- [ ] Login
- [ ] Upload file
- [ ] Create folder
- [ ] Restart containers
- [ ] Data persists
### 3.12 Install Nginx
```Bash
</> Bash
sudo apt install nginx -y
sudo systemctl enable nginx
sudo systemctl start nginx
```
**Test:**
```
http://<IP> → nginx welcome page
```
### 3.13 Configure HTTP Reverse Proxy (Staging Only)
```Bash
</> Bash
sudo tee /etc/nginx/sites-available/nextcloud > /dev/null << 'EOF'
server {
listen 80;
server_name drive.company.com;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
}
}
EOF
```
**Enable:**
```Bash
</> Bash
sudo ln -s /etc/nginx/sites-available/nextcloud /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx
```
*Important*
- ⚠ This step is NOT fully usable yet
- ⚠ HTTPS may redirect and show 404 (expected)
### 3.14 Install SSL (Certbot)
```Bash
</> Bash
sudo apt install certbot python3-certbot-nginx -y
sudo certbot --nginx -d drive.company.com
```
**Validate**
- ✔ Certificate issued
- ✔ HTTPS reachable
- ⚠ May still show 404 (expected)
### 3.15 Finalize HTTPS Reverse Proxy
**Replace config**
```Bash
</> Bash
sudo tee /etc/nginx/sites-available/nextcloud > /dev/null << 'EOF'
server {
listen 80;
server_name drive.company.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl http2;
server_name drive.company.com;
ssl_certificate /etc/letsencrypt/live/drive.company.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/drive.company.com/privkey.pem;
client_max_body_size 10G;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
}
EOF
```
**Reload**
```Bash
</> Bash
sudo nginx -t
sudo systemctl reload nginx
```
**Fix Nextcloud**
```Bash
</> Bash
docker exec -u www-data nextcloud-app php occ config:system:set trusted_domains 1 --value="drive.company.com"
docker exec -u www-data nextcloud-app php occ config:system:set overwrite.cli.url --value="https://drive.company.com"
docker exec -u www-data nextcloud-app php occ config:system:set overwriteprotocol --value="https"
```
**Final Validation**
https://drive.company.com
- ✔ Login works
- ✔ Upload works
- ✔ Download works
- ✔ No warnings
- ✔ HTTPS padlock present
**Cleanup**
```Bash
</> Bash
sudo ufw delete allow 8080/tcp
```
✅ Final Outcome
- ✔ Secure Nextcloud deployment
- ✔ HTTPS reverse proxy functional
- ✔ Docker-based architecture
- ✔ Ready for users and workflows
### 3.16 Create Email Provider Account (Fastmail)
**Objective**
Establish a business-grade email platform using Fastmail so that:
- domain-based email (user@company.com)
- reliable outbound mail (for Nextcloud notifications later)
- foundation for SPF/DKIM/DMARC
**Architectural Context**
Email is not hosted on your droplet. Your infrastructure will later integrate with it for:
- Nextcloud notifications
- User accounts
- Client communication
**Prerequisites**
- Domain ownership confirmed
- Access to DNS registrar
- Decision: Fastmail selected (per runbook)
**Step 1 - Create Fastmail Account**
Go to: https://www.fastmail.com
1.1 Sign up
- Click Sign Up
- Choose a plan (Standard is sufficient for pilot)
- Enter:
```
Name
Email (temporary Fastmail domain)
Password
```
1.2 Complete account creation
You will land in the Fastmail admin interface.
**Step 2 - Add Your Domain**
2.1 Navigate to Domains
In Fastmail:
```
Settings → Domains → Add Domain
```
2.2 Enter your domain
```
company.com
```
Click:
```
Add Domain
```
**Step 3 - Verify Domain Ownership**
Fastmail will present a TXT verification record.
3.1 Copy TXT record
Example:
```
Type: TXT
Name: @
Value: fm-domain-verification=abc123xyz
```
3.2 Add to DNS (Registrar)
In your DNS provider (e.g., GoDaddy or Cloudflare):
Add:
```
Type: TXT
Host: @
Value: fm-domain-verification=abc123xyz
TTL: 300
```
3.3 Wait for propagation
```
~515 minutes typical, but may be as much as 24 hours
```
3.4 Verify in Fastmail
Click:
```
Verify Domain
```
Expected Result
- Domain verified successfully
**Step 4 - Set Domain as Default**
In Fastmail:
```
Settings → Domains → company.com → Set as default
```
This ensures:
- new users → user@company.com
**Step 5 - Create Initial Admin Mailbox**
5.1 Navigate to Users
```
Settings → Users → Add User
```
5.2 Create admin user
Example:
```
Username: admin
Email: admin@company.com
Password: (strong password)
```
5.3 Save
Expected Result
- Mailbox created
- Can send/receive (after DNS setup in 3.8)
**Step 6 - Basic Access Test (Pre-DNS)**
Login via:
```
https://app.fastmail.com
```
Expected
- Inbox accessible
- Account functional (even before MX setup)
- Email is NOT functional yet
**Validation Checklist**
- [✔] Fastmail account created
- [✔] Domain added to Fastmail
- [✔] TXT verification record created
- [✔] Domain verified
- [✔] Default domain set
- [✔] Admin mailbox created
- [✔] Able to login to Fastmail
### 3.17 Configure Email DNS
**Objective**
Make your domain fully capable of sending and receiving email by configuring DNS records required by Fastmail:
- Incoming mail routing (MX)
- Authorized senders (SPF)
- Cryptographic signing (DKIM)
- Policy & reporting (DMARC)
*Changing MX records WILL impact live email.*
If the client currently uses another provider:
- Schedule a cutover window
- Lower TTL beforehand (if possible)
- Expect brief delivery delays during propagation
**Prerequisites**
- Domain added & verified in Fastmail (Task 3.7)
- Access to DNS registrar
- Admin mailbox created
**Step 1 - Gather Fastmail DNS Records**
In Fastmail:
```
Settings → Domains → company.com → DNS Settings
```
Fastmail will provide all required records. Use those exact values if they differ from below.
**Step 2 - Configure MX Records (Mail Routing)**
2.1 Remove Existing MX Records
⚠️ Only do this at cutover time.
2.2 Add Fastmail MX Records
```
Type: MX
Host: @
Priority: 10
Value: in1-smtp.messagingengine.com
Type: MX
Host: @
Priority: 20
Value: in2-smtp.messagingengine.com
```
**Step 3 - Configure SPF (Sender Authorization)**
3.1 Add SPF record
```
Type: TXT
Host: @
Value: v=spf1 include:spf.messagingengine.com ~all
TTL: 300
```
Notes
- Only ONE SPF record per domain
- Merge with existing SPF if present
**Step 4 - Configure DKIM (Email Signing)**
4.1 Get DKIM from Fastmail
In Fastmail DNS settings, copy the DKIM record.
Example:
```
Type: CNAME
Host: fm1._domainkey
Value: fm1.company.com.dkim.fmhosted.com
Type: CNAME
Host: fm2._domainkey
Value: fm2.company.com.dkim.fmhosted.com
Type: CNAME
Host: fm3._domainkey
Value: fm3.company.com.dkim.fmhosted.com
4.2 Add all DKIM records
```
These enable:
- Email authenticity verification
- Better deliverability
**Step 5 - Configure DMARC (Policy & Reporting)**
5.1 Add DMARC record
```
Type: TXT
Host: _dmarc
Value: v=DMARC1; p=none; rua=mailto:admin@company.com
TTL: 300
```
Explanation
- p=none → monitor only (safe for pilot)
- rua= → aggregate reports sent to admin mailbox
Future Hardening
Later you can move to:
- p=quarantine → soft enforcement
- p=reject → full enforcement
**Step 6 - Verify DNS Propagation**
6.1 Check MX
nslookup -type=MX company.com
Expected:
in1-smtp.messagingengine.com
in2-smtp.messagingengine.com
6.2 Check SPF
nslookup -type=TXT company.com
6.3 Check DKIM
nslookup -type=CNAME fm1._domainkey.company.com
6.4 Check DMARC
nslookup -type=TXT _dmarc.company.com
**Step 7 - Functional Email Test**
7.1 Send inbound test
From Gmail/Outlook:
```
Send email to: admin@company.com
```
Expected:
- Email received in Fastmail inbox
7.2 Send outbound test
From Fastmail:
```
Send email to external address
```
Expected:
- Email delivered
- Not marked as spam
**Step 8 - Validate Deliverability (Recommended)**
Use:
- mail-tester.com
- mxtoolbox.com
Expected
- SPF pass
- DKIM pass
- DMARC pass
- Low spam score
**Step 9 - Propagation Expectations**
- MX: 530 minutes typical
- SPF: immediate to 1 hour
- DKIM: 1560 minutes
- DMARC: 1560 minutes
Worst case:
Up to 24 hours
**Validation Checklist**
- [✔] MX records updated
- [✔] SPF record present
- [✔] DKIM records configured (all 3)
- [✔] DMARC record created
- [✔] DNS resolves correctly
- [✔] Email received externally
- [✔] Email sent externally
- [✔] Deliverability validated