Folks, this has taken me ages (ok to be honest – three weeks, which is still a lot ;-)) to get up and running. But finally, I have made it and learned a lot about Docker, Docker-Compose, Nginx and Kubernetes of course.
In this post I am showing how to set up a Nextcloud on Kubernetes + Ubuntu 18.04. I am running a Nextcloud-instance on a public VM to sync stuff with my phone. Another one (with more data) is only accessible via my LAN/VPN. The first one
I am encrypting via Let’s Encrypt (open todo!) and the 2nd one via self-signed certificates.
The stacks are set up via Kubernetes to be able to scale horizontally in the future, since I had to live with some performance-issues in the past. GitHub – andremotz – nextcloud-kubernetes
For more than two years I was running my own Nextcloud on a Linux-VM, which was accessible via my LAN & OpenVPN. Therefore, I was using one of the thousands of Nextcloud-Docker Images, that are available on Docker Hub. However, the solution did not meet my requirements anymore:
- The Image’s author recommended not update the Docker-container, which is an anti-pattern for Container-driven software.
- I was not able to create my own certificates or use Let’s Encrypt.
- I had some serious performance-issues that I could not really explain to myself.
Most stuff could have been done using Docker-Compose but in this case, I would have to upgrade to Docker-Swarm, to be able to scale horizontally.
My own tutorial is marvellous – since I’ve created the post/GitHub-content I had to re-setup my own NC-cluster at least four times. I always used my own tutorial to prove if everything is still working 🙂
- 2019-12-02: Updated Deployments: added selector/matchLabels, changed apiVersion: apps/v1
What I will cover
- Setting up Kubernetes on Ubuntu 18.04 (although this task can be easily ported to other distros as well). I did not use any special/latest Docker-repos to be compatible with Kubernetes’ latest release.
- Initializing and configuring all necessary Pods for Nextcloud to run a secure home to sync data. Therefore, I am using the default Nextcloud-Docker image, a Nginx-Proxy + OMGWTFSSL-Docker Image for self-signed certificates and MariaDB.
What I want to cover in the future
- Putting the certs into an emptyDir-Volume
- Running the cluster on multiple machines
- So far, I have issues with the Nextcloud-iOS app. I could connect to my old Nextcloud-setup, so I assume it is some issue with the latest version.
- Let’s Encrypt offers the user’s browser some public authority. The bad side is that the encrypted server needs to be public available on the Internet. Self-signed certificates can be created pretty easy but show the user an ugly “not secure!!11” message in the browser.
What I do not cover
Kubernetes is a big universe, but not all features were necessary to get my Nextcloud up and running.
- It is good to have the option to scale in the future. But so far, I am running the Kubernetes-cluster on one machine without any scaling. As of now, this is some overkill, but this allows me in the future to extend my setup.
- I am not using any fancy PersistetVolumes. Instead I am using HostPaths. In the future I might update this post to be able to scale better.
- Using Ingress-proxies was just not necessary to me. All I need was a port 80 & 443-routing to my Kubernetes-cluster. Hence, a Nginx reverse proxy-Pod does the job pretty well, too.
- Declarative vs Imparative: I put everything into YAML -files. However, I stopped using kubectl apply since kubectl create did a better & faster job when changing config-files. Maybe here I will learn something else in the future, but to make my set-up, I had less headache with using kubectl create + YAML-files.
Let’s jump in!
Download and set up Ubuntu 18.04 LTS. I won’t cover any further steps here since there are a lot of information out there.
Setting up Docker & Kubernetes on Ubuntu
Please check the source for further explanation. I have done the following steps on three different machines without problems.
$ sudo apt update && sudo apt upgrade -y $ sudo apt install docker.io $ sudo systemctl enable docker $ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add $ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main" $ sudo apt install kubeadm $ sudo swapoff -a $ sudo kubeadm init --pod-network-cidr=10.244.0.0/16 $ mkdir -p $HOME/.kube $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config $ sudo chown $(id -u):$(id -g) $HOME/.kube/config $ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml $ kubectl taint nodes --all node-role.kubernetes.io/master-
At this place you should note down the shown kubeadm join-message in your console to be able to connect further Kubernetes-nodes in the future.
Setting up Nextcloud on Kubernetes
Extra-hint: Use the following in an extra-terminal to be able to see what the Kubernetes-cluster is doing
$ watch -n 10 kubectl get deployment,svc,pods,pvc,pv,ing
Deployment + Service: MariaDB
As a user (not root) create a folder nc-deployment, download pre-defined MariaDB-descriptions, adjust it to your needs and deploy:
$ mkdir nc-deployment $ cd nc-deployment $ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-deployment.yaml $ nano db-deployment.yaml --> change MYSQL_PASSWORD here --> change MYSQL_ROOT_PASSWORD here --> change db's HostPath here, which should be the absolute location of 'nc-deployment'/db-pv (eg /home/andremotz/nc-deployment/db-pv) $ kubectl create -f db-deployment.yaml $ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-svc.yaml $ kubectl create -f db-svc.yaml
Deployment + Service: Nextcloud:
Next, download Nextcloud-descriptions, adjust them and deploy:
$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-deployment.yaml $ nano nc-deployment.yaml --> change NEXTCLOUD_URL --> change NEXTCLOUD_ADMIN_PASSWORD --> change MYSQL_PASSWORD (the value you've entered before) --> change html's hostPath (eg. to /home/andremotz/nc-deployment/nc-pv) $ kubectl create -f nc-deployment.yaml $ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-svc.yaml $ kubectl create -f nc-svc.yaml
Create self-signed certificates
The OMGWTFSSL-Docker image offers easy-to-use certificate-creation. Here we are using only a Pod, not a Deployment. Once the certificates are created, the Pod will stop.
$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/omgwtfssl-pod.yaml $ nano omgwtfssl-pod.yaml --> change SSL_SUBJECT to your server's name --> change CA_SUBJECT to your mail-adress --> change SSL_KEY to a proper filename --> change SSL_CSR to a proper filename --> change SSL_CERT to a proper filename --> change cert's hostPath (eg. to /home/andremotz/nc-deployment/certs-pv) $ kubectl create -f omgwtfssl-pod.yaml
Deployment + Service: Nginx reverse Proxy
One could already easily adjust the Nextcloud-service to publish HTTP-driven service. However we want to use a Nginx-instance in front of our Nextcloud to be able to use HTTPS-encryption. For the proxy we are not using a Deployment but a Pod, to be able to make use of standard HTTP/HTTPS-ports 80 & 443
$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nginx.conf $ nano nginx.conf --> change server_name (two locations in the file!) to the server name you've provided before for SSL_SUBJECT --> change ssl_certificate to the filename you've provide before for SSL_CERT --> change ssl_certificate_key to the filename you've provide before for SSL_KEY $ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/proxy-pod.yaml $ nano proxy-pod.yaml --> change cert's hostPath to the location you have provided before---> change nginx-config's hostpath to the location where you've stored nginx.conf before (eg. /home/andremotz/nc-deployment/nginx.conf) --> change nginx-logs' hostpath to a proper location $ kubectl create -f proxy-pod.yaml
Now you should be able to point your browser to https://<yourserver> and see a new Nextcloud-instance, running on a super-hyper nextlevel-Kubernetes cluster, that you could use for further cool stuff 😉
Disable swap permanently – add a comment to the /etc/fstab – swap entry like this:
#/swap.img none swap sw 0 0
That’s it!! 😉