📈 Hybrid Clouds
12 Aug, 2020
Since I haven’t posted about servers in a while, I thought I’d take a look at how I’m utilizing a hybrid cloud for my infrastructure.
Visuals are fun, so lets starts there.
I tried to make the most complete view as I could:
graph LR
inet(Internet)
prem[On-Prem Servers]
gh(GitHub)
gl(GitLab)
ms(Microsoft 365)
vali(Valimail)
inet-->gh
inet-->gl
gh-->gl
inet-->ms
subgraph cf [Cloudflare Edge Network]
edge(Edge Proxy)
faas[Serverless/FaaS Workers]
dns[DNS]
edge-->faas
end
inet-->edge
dns-->inet
edge-->gh
edge-->gl
edge-.->ms
subgraph vps [Virtual Server/VPS]
vpn[VPN]
SSH
Nginx
subgraph Docker
builder[/Builder/]
app1[App 1]
app2[App 2]
db[(Database)]
builder-.->app1
builder-.->app2
app2-->db
end
SSH-->builder
Nginx-->app1
Nginx-->app2
Nginx-->Nginx
misc[Apache/Misc]
Nginx-->misc
end
edge-->vpn
edge-->SSH
edge-->Nginx
vpn-->prem
inet-.->vali
Here’s the buzzword bingo version of what I’m using:
graph LR
inet(Internet)
vps[Virtual Server/Cloud Hosting]
prem[On-Prem Servers]
saas(SaaS/Managed Service)
subgraph cf [Edge Network]
edge(Edge Proxy)
faas[Serverless/FaaS]
edge-->faas
end
inet-->edge
edge-->vps
vps-->prem
edge-->saas
From the top
graph TD
user[User]
cf(Cloudflare)
static[Static Hosting]
cfworkers[Cloudflare Workers]
vps[VPS]
ms[Microsoft 365]
prem[On-Prem Servers]
user-->cf
cf-->static
cf-->cfworkers
cf-->vps
cf-->ms
vps-->prem
Cloudflare acts are the edge of my websites.
It handles DNS, security, proxying/caching, and general routing.
sequenceDiagram
participant User
participant Cloudflare
participant Origin
User->>+Cloudflare : Requests Page
opt security upgrade
Cloudflare->>User : Redirect/Upgrade
end
alt in cache
Cloudflare->>User : Serves from Cache
else not in cache
Cloudflare->>Origin : Requests Page
Origin-->>Cloudflare : (Page)
opt cacheable
Cloudflare->>Cloudflare : Puts in Cache
end
Cloudflare->>-User : Serves Response
end
I use static hosting for most of my sites.
Specifically, I tend to use GitLab Pages because of convenience, custom TLS certs, and support for a variety of frameworks.
My static sites mainly get generated with GitLab CI, Hugo, and Nuxt, although I also have some other things too.
These static sites can still callback to other sites for dynamic behavior.
A good example of this is Find, a search engine for my sites:
sequenceDiagram
participant User
participant Search
participant Cloudflare
participant APIs
User->>+Search : Loads Page
Search->>Cloudflare : Requests Content
alt in cache
Cloudflare->>Search : Serves from Cache
else not in cache
Cloudflare->>APIs : Requests Page
APIs-->>Cloudflare : (Page)
opt cacheable
Cloudflare->>Cloudflare : Puts in Cache
end
Cloudflare->>Search : Serves Response
end
Search->>Search : Parses Data
Search->>-User : Presents Experience
Cloudflare Workers handle small serverless functions.
Currently, the main use is to be an oEmbed Provider by scraping my site for metadata.
The logic flow for this is very similar to my static sites, but happens behind Cloudflare instead of in front of it.
sequenceDiagram
participant User
participant Cloudflare
participant worker as oEmbed Worker
participant Origin
User->>Cloudflare : Requests oEmbed
Cloudflare->>+worker : Runs Worker
worker->>Cloudflare : Requests Page
alt in cache
Cloudflare->>worker : Serves from Cache
else not in cache
Cloudflare->>Origin : Requests Page
Origin-->>Cloudflare : (Page)
opt cacheable
Cloudflare->>Cloudflare : Puts in Cache
end
Cloudflare->>worker : Serves Response
end
worker->>worker : Scrapes Data
worker->>-User : Presents Experience
I have a VPS hosted on DigitalOcean, which has a VPN “roundabout” (as I’ve taken to calling it) and can host backend apps.
graph LR
Me
inet([Internet])
Me-->inet
subgraph VPS
ssh[SSH]
vpn[Roundabout]
nginx[Nginx]
end
inet-->ssh
inet-->vpn
inet-->nginx
The roundabout is called as such because it simply forwards the VPN clients to each other over the bridge; it doesn’t provide full client-to-site (another tunnel does that), but it does allow me to do firewall punching when I can’t port foward.
graph BT
inet([Internet])
vps[VPS]
me[Me]
prem[On-Prem]
me-->inet
inet-->vps
prem-->inet
subgraph Roundabout
subgraph tun2 [Client-to-Site]
me-. via Roundabout .->prem
end
me-.->vps
prem-.->vps
end
The VPN, however, does allow me to host apps from my server.
I’ve used this for a variety of things, including private DNS, internal (dev/testing) websites, data storage, Jupyter Lab, and hosting apps like Nextcloud or Gitea.
graph LR
inet([Internet])
subgraph VPS
nginx[Nginx]
nginx-- TLS Termination -->nginx
subgraph dokku [Dokku/Docker]
app1[App 1]
app2[App 2]
db[(Database)]
app2-->db
end
nginx-->app1
nginx-->app2
misc[Apache/Misc]
nginx-->misc
end
inet-->nginx
There’s a second part to this though, which is that Dokku provides me git hooks to run Heroku buildpacks on push.
graph LR
CI
subgraph VPS
SSH
subgraph dokku [Dokku/Docker]
Builder[/Builder/]
app1[App 1]
app2[App 2]
db[(Database)]
Builder-.->app1
Builder-.->app2
app2-->db
end
SSH-->Builder
end
CI-->SSH
Finally, Microsoft 365 is used for email, although it also has Teams (and related) functionality enabled.
Valimail is also used for DMARC monitoring.
graph TD
Sender
Recipient
subgraph srv [Managed Services]
subgraph ms [Microsoft 365]
Inbox[My Inbox]
end
subgraph DNS
cf[Cloudflare]
end
end
Sender-- Email -->Inbox
Inbox-- Email -->Recipient
cf-. MX .->Sender
cf-. SPF,DKIM,DMARC .->Recipient
Recipient-- DMARC Report -->Valimail
Deploying
Some things, like my VPS and on-prem servers, are managed somewhat manually.
Others, like Cloudflare and Microsoft 365, are unmanaged.
However, I have a variety of CI/CD pipelines set up to manage most of my sites.
All pipelines are designed to have minimal intervention; a push to GitHub should be all that’s needed.
Cloudflare Workers are deployed via GitHub Actions:
sequenceDiagram
participant Me
participant GitHub
participant actions as GitHub Actions
participant Cloudflare
Me->>GitHub : Push Commit
GitHub->>actions : Runs Workflow
actions->>Cloudflare : Deploys
Static Pages are usually generated using GitLab CI and then hosted by GitLab Pages:
sequenceDiagram
participant Me
participant GitHub
participant GitLab
participant pages as GitLab Pages
participant Cloudflare
Me->>GitHub : Push Commit
GitHub->>+GitLab : Webhook
GitLab->>GitHub : Fetch Repo
GitHub-->>GitLab : (Repo)
GitLab->>GitLab : CI Build
GitLab->>pages : Deploy
GitLab->>Cloudflare : Purge Cache
GitLab->>-GitHub : Reports Pass/Fail
Dynamic apps, like rimg, may have GitHub Actions testing for commits and PRs, as well as GitLab CI deploying to my VPS:
sequenceDiagram
participant Me
participant Contributor
participant GitHub
participant actions as GitHub Actions
participant GitLab
participant VPS
loop PR or Push
Contributor->>GitHub : Push Commit
GitHub->>+actions : Runs Tests
actions->>-GitHub : Reports Pass/Fail
end
loop Merge to Main Branch
Me->>GitHub : Push Merge
GitHub-->>+actions : Runs Tests
actions-->>-GitHub : Reports Pass/Fail
GitHub->>+GitLab : Webhook
GitLab->>GitHub : Fetch Repo
GitHub-->>GitLab : (Repo)
GitLab->>GitLab : Run Tests
GitLab->>+VPS : Deploy
VPS->>VPS : Compiles
VPS->>VPS : Runs via Docker
VPS->>VPS : Health Check
VPS-->>-GitLab : (Pass/Fail)
GitLab->>-GitHub : Reports Pass/Fail
end
Optimizing for Cost
As a college student, you don’t have much of time or budget to work with.
However, there are a ton of free and cheap options available if you look around enough.
Everything I have in the cloud currently costs about $12.75/month.
This includes domain names, edge computing, VPS, email, and dynamic apps (LAMP, PaaS, SaaS).
pie
title Cost Breakdown/Month
"Domains ($33/yr)" : 2.75
"VPS ($5/mon)" : 5
"Email ($5/mon)" : 5
Over time, this has lead to some design choices that are intentionally lightweight, particularly a focus on static sites and managed services.
Here’s a short list of some of the cost-saving measures I’ve done:
- VPS also taking on the roles of Shared (LAMP) Hosting + Heroku/PaaS/Kubernetes
- If you use lightweight tools for management, you don’t need much
- Public GitHub & GitLab instead of self-hosted
- Both are free for Open Source, which pretty much everything I do is
- GitHub Pages & GitLab Pages instead of Shared Hosting
- Static sites instead of WordPress/PHP-based sites
- Cloudflare to protect VPS & unify experience
- Cloudflare Workers to move load off VPS
Well, that was fun but that’s all I have for now.
Ciao 👋