Not your keys, not your coins is an expression frequently used in the crypto world to indicate how relying on centralized exchange results in not having control over your money: if the exchange goes down your coins go down.
Similarly, this may apply to every other data we want to store safely on the cloud: we like having files synchronized, collaborating on the same documents at the same time, and sharing photos with our family. But what happens if, for any reason, our provider decides to shut down our account? It happens. A lot.
Moreover, privacy is a big topic: everyone should be concerned about it and I don’t believe in the idea “I have nothing to hide!”. I won’t discuss it further in this post, DYOR.
This is why I decided over time to move from centralized and third-party services, embrace the self-hosting world, and have control over my data.
Components Overview
What I’m going to describe is my first attempt at self-hosting services. I’ve started from scratch and I’ve selected the architecture and the components that I think work well for me. There are certainly better ways (Kubernetes?) but for now, I’m happy with what I got.
I’ve started by getting a public IP for my network, dealing with DDNS and Port Forwarding, and finally learning how to use Docker Compose in a structured and reproducible environment.
Let’s dig a bit more into every component.
DDNS and Port Forwarding
To access your network from the outside you first need a public IP: many ISPs offer a bunch of public IPs, usually with a price, but mine doesn’t. The solution is using Dynamic DNS or DDNS! I’ve selected a DDNS provider, picked one of their free domains, configured it on my router, and finally got my public IP, dynamically updated by itself every time the ISP updates it. Some examples of free DDNS providers are dynu.net, dyndns.com, no-ip.com and, cloudflare.com.
The IP you get from either a static public IP or a dynamic public IP points to the gates of your network, usually your router. But I’m not planning to host any service on my router, but on a local device instead. Hence, I need to re-direct the traffic on a certain port from WAN to the local device: this idea translates to the concept of Port Forwarding.
This first bit of configuration gives you a way of accessing your local device from the public internet! But at this point, any communication will probably fail, why? Because nothing is listening on the device side. Time to host something!
UPDATE: I ended up switching to Cloudflare as DNS provider, check the post here.
Caddy as a Reverse Proxy
In the previous step, I decided to forward only one port. One with some knowledge of networking would think that one port = one service and this is generally true. The first idea would then be just forward multiple ports!, one per hosted service. This is generally a bad and less maintainable idea in terms of configuration, certificates, and security.
The solution is using a Reverse Proxy, which is software that redirects the incoming traffic to the configured port and service depending on (in my case) the requested subdomain.
The idea is that accessing the URL https://service1.mydomain.example.com would be forwarded to the local device, where a service is listening on port 443 (HTTPS) and finally redirected to the configured service1 port.
There are many different pieces of software that can be configured to perform Reverse Proxy, I picked Caddy because it feels modern, easy to configure, and it can handle SSL certificates from Let’s Encrypt by itself.
Since I don’t want to mess with the filesystem of my local device and I want a reproducible environment for my services, I opted for the Docker version of Caddy. In particular, the Docker Compose version.
I’ll link my entire setup at the end but the tree for the Caddy directory is something like this:
caddy/
├── Caddyfile
├── README.md
├── config
├── data
└── docker-compose.yml
The Caddyfile is where the magic is configured. Let’s get back to the example above, proxying service1. The minimal Caddyfile section would be:
service1.mydomain.example.com:443 {
reverse_proxy service1:5006
}
Time to host some service now.
Dockerized Services
This bit is very personal, we don’t share the same needs and everyone is free to try new applications, and experiments and host whatever they need. What’s still important to me is not messing with the filesystem and having a reproducible environment: Docker!
These are my currently hosted services:
- Actual Budget: a personal finance application
- Jellyfin: a video streaming application
- Immich: a replacement for Google Photos, with automatic backup, face recognition and shared albums
- Nextcloud: a Google Suite replacement, Drive, Office, Notes, …
- Vaultwarden: a Rust implementation of the Bitwarden server application
Following the directory organization described for Caddy, every service has its own subdirectory and Docker file.
AdGuard Home
One last service I’m hosting is AdGuard Home: if configured as your network DNS it can block ads, trackers, and similar crap the internet is filled with nowadays.
I’ve decided to not expose this service over Caddy because it doesn’t make sense to me. The DNS is only used in my LAN and it doesn’t make sense to me to have the Admin Panel exposed on the public internet.
Still using Docker compose for it though, for the same reasons as the other applications.
Conclusion
Is this the final setup? Probably not. I’m actually afraid I’ll spend so much time on it, hopefully learning every time something new (looking at you k8s).
When I started I didn’t know much about DDNS, and Reverse Proxy and I definitely wasn’t comfortable with Docker Compose. I now feel a bit more expert in the networking area and my data is also safe at home!
My server can go down? Yes sure, it can happen: I’ll buy a UPS. My storage can die? Yes, it will. You better have a daily (if not hourly) backup running.
Can it be done better? I’m perfectly sure about that, but I think I shouldn’t forget the original intent of this whole work and enjoy what I got.
Additional tips
You can find my setup here: feel free to clone it, configure it, and give it a try. As I said, my network, my choices: you may need different services, you may want to set it up differently, or whatever. Just fork it or keep it as a reference.
And please, think about backups before actually using it (Borg is what I currently suggest, I may write something about it in the future).