The dragonhive/dergnz web stack
Setting up this website reminded me how happy I am with my current automation and web stack.
It just works, itâs not flawless but I barely ever have to touch it, and if I want to add something new, I just copy a folder with a template docker-compose file, customize it to what I need it to do, and voila, a new service is up and running, automatically updated on a daily basis, and backed up to an external server with some extra scripts (not in this tutorial).
Setting up something like that takes a little effort, especially the figuring out part, but once you have it going, it is absolutely the most blissful and managable way to deal with more than a few services, even though this is obviously a little bit much to set up if you only care about a single web app.
But how does it work?
The core idea
I like Containers. Some people may prefer Virtual Machines, and thatâs fine too, you can even combine the two concepts if you like. But this current concept is very easy to deploy, and Iâve been pondering turning it into a quick deploy stack with some basic web interface to add more containers and services to run, as it is very robust and very easy to use so far.
The core concept, is to have a few simple containers that together form the base for a very easy to use webservices host, while also being very easy to update, back up and manage. It contains the following base containers:
- An
nginx
frontend webserver that sits on both 80 and 443, completely vanilla nginx, with its configuration folders exported out of the container. - JWildersâ nginx-proxy/
docker-gen
container will look at a read-only copy of the docker socket to query what containers are running, and what their desired outside hostnames would be, to generate configuration files for our frontend nginx webserver to use acme-companion
, which will request letsencrypt certificates and cooperate with thedocker-gen
container to set up HTTPS- Any number of other services, with at very least an environment variable for the
VIRTUAL_HOST
name (domain/hostname) and one for theLETSENCRYPT_HOST
name (the https domain/hostname). In this case I will demonstrate it with Jekyll, which is what this website runs on :)
A basic example
For our core we need to set up a container host program.
There are some options you could pick, but for simplicity sake and most widespread usage, Iâll focus on Docker for now. One could also pick for example Podman
, which should in theory also have an API socket that docker-gen could use, but I have not gotten this to work myself so YMMV.
One of the advantages (and disadvantages) of Docker is that itâs centrally managed by a daemon, which you can query and command over a socket to do your bidding automatically, and weâre going to use this socket. This has some downsides in terms of security of course, if a hacker ever manages to compromise a container that has access to the docker socket, they can read everything thatâs running on the docker hostâ as long as the docker socket is mounted read only, otherwise itâs much more dangerous and will allow the attacker to spawn new containers that could do anything. But it works exceptionally well for automation purposes, if you secure the machines that have access to it well, and realize that these are one of your weak points.
JWildersâ nginx-proxy/docker-gen
A tiny but very useful little tool I found, which uses the docker socket to see what containers there are running, checks the environment variables, and uses these to automatically generate a nginx configuration. Optionally, one can add letsencrypt-companion to also generate matching LetsEncrypt certificates, to make everything nice and HTTPS-y.
- More info about nginx-proxy can be found here
- More info about LetsEncrypt-Companion can be found here
Setting up a full server and Jekyll
The best way to see how this works, is by just diving right in and trying to set up the stack, with some test application like Jekyll.
- Install your favorite Linux distro. It doesnât matter much if youâre using Fedora, Ubuntu, Debian, OpenSuse or something else, as long as you have Docker available. Note that some distros (Like Fedora) may have SELinux enabled, which can be painful to configure correctly during setup. If you have SELinux, I recommend using
setenforce 0
to temporarily set it to permissive mode. You can always generate an SELinux policy based on the logs itâll generate at a later stage - Install both
docker
anddocker-compose
- Set up your files the way you like; Be it some externally mounted disk on its own
/serverfiles
mounting point, or somewhere in the existing filesystem, it doesnât matter too much what you pick. For the sake of this tutorial Iâm going to use/serverfiles
as the top level directory for our stored files. You can search/replace this string with whatever you end up using in practice. - Create a folder for your docker-compose files, and your containerâs working files. You can put those in the same directory, but I recommend splitting them out for easier backups and management, although in the end youâll have to backup both if you want to keep your data, so itâs up to you. Iâm going to assume using
/serverfiles/compose-files
for the compose files, and/serverfiles/data
for the data thatâs mounted inside the container during runtime. Docker also supports a virtual file system stored elsewhere, which is more compatible with Swarm, but for this tutorial Iâll focus on the old folder mounting system. - Letâs get the actual webserver running first, which wonât do anything yet, but weâll need in order to make Jekyll show up later on.
- Make sure docker is running (
systemctl enable --now docker
or something alike for your distro) - Create the folder named something like
/serverfiles/compose-files/nginx-frontweb
and open it, then create a new docker-compose.yml file with the following contents:
- Make sure docker is running (
version: '2'
services:
# First we set up an nginx container as our frontend proxy. We grab and export the config folders so we can control its behavior from jwilder's docker-gen container.
nginx-proxy:
image: nginx:alpine
container_name: nginx-proxy
environment:
- DHPARAM_GENERATION=false
ports:
- "80:80"
- "443:443"
volumes:
- /serverfiles/srv/nginx-frontweb/conf:/etc/nginx/conf.d
- /serverfiles/srv/nginx-frontweb/vhost:/etc/nginx/vhost.d
- /serverfiles/srv/nginx-frontweb/html:/usr/share/nginx/html
- /serverfiles/srv/nginx-frontweb/certs:/etc/nginx/certs:ro
privileged: true
# This container will automatically generate nginx configs based on the environment variable `VIRTUAL_HOST` of running containers. This allows you to add any number of services very easily without every having to touch nginx configs.
docker-gen:
image: nginxproxy/docker-gen
container_name: nginx-proxy-gen
command: -notify-sighup nginx-proxy -watch /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
environment:
- DHPARAM_GENERATION=false
volumes_from:
- nginx-proxy
volumes:
- /serverfiles/srv/nginx-frontweb/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro
- /run/docker.sock:/tmp/docker.sock:ro # We map it read-only so that if our container gets hacked, we cannot write to the docker socket.
- /serverfiles/srv/nginx-frontweb/conf:/etc/nginx/conf.d
- /serverfiles/srv/nginx-frontweb/vhost:/etc/nginx/vhost.d
- /serverfiles/srv/nginx-frontweb/html:/usr/share/nginx/html
- /serverfiles/srv/nginx-frontweb/certs:/etc/nginx/certs:ro
labels:
- "com.github.jrcs.letsencrypt_nginx_proxy_companion.docker_gen"
privileged: true
# And finally the Acme-companion requests certificates from LetsEncrypt for every machine that has the `LETSENCRYPT_HOST` variable defined
acme-companion:
image: nginxproxy/acme-companion
container_name: nginx-proxy-acme
environment:
- DHPARAM_GENERATION=false
volumes_from:
- nginx-proxy
volumes:
- /serverfiles/srv/nginx-frontweb/certs:/etc/nginx/certs:rw
- /serverfiles/srv/nginx-frontweb/acme:/etc/acme.sh
- /run/docker.sock:/var/run/docker.sock:ro
- /serverfiles/srv/nginx-frontweb/conf:/etc/nginx/conf.d
- /serverfiles/srv/nginx-frontweb/vhost:/etc/nginx/vhost.d
- /serverfiles/srv/nginx-frontweb/html:/usr/share/nginx/html
privileged: true
# Here we're defining an external network for other containers to join, which is required for nginx to reach those containers internally.
networks:
default:
external:
name: frontendweb
This will serve as the base for all your webservices, as it will take care of all the subdomain routing, certificate requesting, et cetera, for you. All you have to do after this, is set up extra containers with whatever services you might like, and theyâll be usable and have HTTPS certificates as soon as you start them!
- Now that we have that, we can set up Jekyll and have it be forwarded to the internet.
- Go back up one folder, and create a new folder named something like
mywebsite-jekyll
, open it, and create anotherdocker-compose.yml
file. - Hereâs the
docker-compose.yml
file that Iâm using for this very website (paths changed): ``` version: â3â services: jekyll: image: jekyll/jekyll environment:- VIRTUAL_HOST=derg.nz,dragonhive.net
- VIRTUAL_PROTO=http
- VIRTUAL_PORT=4000
- LETSENCRYPT_HOST=derg.nz,dragonhive.net
command: jekyll serve âwatch âtrace âincremental
expose: - 4000 volumes: - /serverfiles/srv/mywebsite-jekyll:/srv/jekyll - /serverfiles/srv/mywebsite-jekyll/vendor/bundle:/usr/local/bundle:cached networks: default: external: name: frontendweb ``` Notice theVIRTUAL_HOST
,VIRTUAL_PROTO
, andVIRTUAL_PORT
environment variables? Thatâs the magic. Thatâs whatâs going to instruct docker-gen to create a nginx config appropriate for that (sub)domain name, and also request a HTTPS certificate while at it.
- Go back up one folder, and create a new folder named something like
- Jekyll requires some manual set up before it can be used. To prepare Jekyll to be used the first time, execute the following:
docker-compose run jekyll jekyll new .
# docker-compose run (container name) (the jekyll command) (new subcommand) (current folder)- you should now have the Jekyll files in your
/serverfiles/srv/mywebsite-jekyll
folder docker-compose run jekyll jekyll build
# will build the website and make it ready for serving (seems optional as it does it automatically on container startup though)
- Start your engines!
- Please note, I did not explain any DNS aspects in this tutorial, I will assume you have a wildcard subdomain pointing to your server before you configure any subdomains or domains in the config files. If a DNS tutorial is desired, let me know! Iâll be happy to write one or add it here if needed.
- Neither did I explain port forwarding here. Iâll assume you have both 80 and 443 forwarded to your webserver, as this is required for HTTP and HTTPS traffic. Youâll still need port 80 even if you only intend to use 443, as itâs a hard requirement for the LetsEncrypt certificates to be renewed.
cd
into yourfrontweb-nginx
folder and run:docker-compose up -d
. This will bring up the frontend webserver stack, and if all goes well it should stay running (you can check withdocker-compose ps
)cd
into your../mywebsite-jekyll
folder and executedocker-compose up -d
again.- If all went well, you should be see the Jekyll default page now, and you can start customizing it by editing the markdown files!
F.A.Q, common issues, and fixes
- If the container doesnât start, try running it without
-d
(daemon), itâll spit out all the logs and stay attached to the console output of the containers while they run. You can also usedocker-compose logs
to get the last logs. - as of 22-8-2022 itâs still an early version of this large writeup, I hope I got everything right, if not, please feel free to poke me.