Folks, this has taken me ages (ok to be honest – three weeks, which is still a lot ;-)) to get up and running. But finally, I have made it and learned a lot about Docker, Docker-Compose, Nginx and Kubernetes of course.

---

Summary

In this post I am showing how to set up a Nextcloud on Kubernetes + Ubuntu 18.04. I am running a Nextcloud-instance on a public VM to sync stuff with my phone. Another one (with more data) is only accessible via my LAN/VPN. The first one I am encrypting via Let’s Encrypt (open todo!) and the 2nd one via self-signed certificates.

The stacks are set up via Kubernetes to be able to scale horizontally in the future, since I had to live with some performance-issues in the past. GitHub – andremotz – nextcloud-kubernetes

Introduction

For more than two years I was running my own Nextcloud on a Linux-VM, which was accessible via my LAN & OpenVPN. Therefore, I was using one of the thousands of Nextcloud-Docker Images, that are available on Docker Hub. However, the solution did not meet my requirements anymore:

1. The Image’s author recommended not update the Docker-container, which is an anti-pattern for Container-driven software. 
2. I was not able to create my own certificates or use Let’s Encrypt.
3. I had some serious performance-issues that I could not really explain to myself.

Most stuff could have been done using Docker-Compose but in this case, I would have to upgrade to Docker-Swarm, to be able to scale horizontally.

Updates:

My own tutorial is marvellous – since I’ve created the post/GitHub-content I had to re-setup my own NC-cluster at least four times. I always used my own tutorial to prove if everything is still working 🙂

• 2019-12-02: Updated Deployments: added selector/matchLabels, changed apiVersion: apps/v1

What I will cover

• Setting up Kubernetes on Ubuntu 18.04 (although this task can be easily ported to other distros as well). I did not use any special/latest Docker-repos to be compatible with Kubernetes’ latest release.
• Initializing and configuring all necessary Pods for Nextcloud to run a secure home to sync data. Therefore, I am using the default Nextcloud-Docker image, a Nginx-Proxy + OMGWTFSSL-Docker Image for self-signed certificates and MariaDB.
  

  What I want to cover in the future

  • Putting the certs into an emptyDir-Volume
  • Running the cluster on multiple machines
  • So far, I have issues with the Nextcloud-iOS app. I could connect to my old Nextcloud-setup, so I assume it is some issue with the latest version.
  • Let’s Encrypt offers the user’s browser some public authority. The bad side is that the encrypted server needs to be public available on the Internet. Self-signed certificates can be created pretty easy but show the user an ugly “not secure!!11” message in the browser.

What I do not cover

Kubernetes is a big universe, but not all features were necessary to get my Nextcloud up and running.

• It is good to have the option to scale in the future. But so far, I am running the Kubernetes-cluster on one machine without any scaling. As of now, this is some overkill, but this allows me in the future to extend my setup.
• I am not using any fancy PersistetVolumes. Instead I am using HostPaths. In the future I might update this post to be able to scale better.
• Using Ingress-proxies was just not necessary to me. All I need was a port 80 & 443-routing to my Kubernetes-cluster. Hence, a Nginx reverse proxy-Pod does the job pretty well, too.
• Declarative vs Imparative: I put everything into YAML -files. However, I stopped using kubectl apply since kubectl create did a better & faster job when changing config-files. Maybe here I will learn something else in the future, but to make my set-up, I had less headache with using kubectl create + YAML-files.

Let’s jump in!

Download and set up Ubuntu 18.04 LTS. I won’t cover any further steps here since there are a lot of information out there.

Setting up Docker & Kubernetes on Ubuntu

Source: https://linuxconfig.org/how-to-install-kubernetes-on-ubuntu-18-04-bionic-beaver-linux

Please check the source for further explanation. I have done the following steps on three different machines without problems.

$ sudo apt update && sudo apt upgrade -y
$ sudo apt install docker.io
$ sudo systemctl enable docker
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
$ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
$ sudo apt install kubeadm
$ sudo swapoff -a
$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
$ kubectl taint nodes --all node-role.kubernetes.io/master-

At this place you should note down the shown kubeadm join-message in your console to be able to connect further Kubernetes-nodes in the future.

Setting up Nextcloud on Kubernetes

Extra-hint: Use the following in an extra-terminal to be able to see what the Kubernetes-cluster is doing

$ watch -n 10 kubectl get deployment,svc,pods,pvc,pv,ing

Deployment + Service: MariaDB

As a user (not root) create a folder nc-deployment, download pre-defined MariaDB-descriptions, adjust it to your needs and deploy:

$ mkdir nc-deployment
$ cd nc-deployment
$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-deployment.yaml

$ nano db-deployment.yaml
--> change MYSQL_PASSWORD here
--> change MYSQL_ROOT_PASSWORD here
--> change db's HostPath here, which should be the absolute location of 'nc-deployment'/db-pv (eg /home/andremotz/nc-deployment/db-pv)

$ kubectl create -f db-deployment.yaml

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-svc.yaml
$ kubectl create -f db-svc.yaml

Deployment + Service: Nextcloud:

Next, download Nextcloud-descriptions, adjust them and deploy:

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-deployment.yaml

$ nano nc-deployment.yaml
--> change NEXTCLOUD_URL
--> change NEXTCLOUD_ADMIN_PASSWORD
--> change MYSQL_PASSWORD (the value you've entered before)
--> change html's hostPath (eg. to /home/andremotz/nc-deployment/nc-pv)

$ kubectl create -f nc-deployment.yaml

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-svc.yaml
$ kubectl create -f nc-svc.yaml

Create self-signed certificates

The OMGWTFSSL-Docker image offers easy-to-use certificate-creation. Here we are using only a Pod, not a Deployment. Once the certificates are created, the Pod will stop.

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/omgwtfssl-pod.yaml

$ nano omgwtfssl-pod.yaml
--> change SSL_SUBJECT to your server's name
--> change CA_SUBJECT to your mail-adress
--> change SSL_KEY to a proper filename
--> change SSL_CSR to a proper filename
--> change SSL_CERT to a proper filename
--> change cert's hostPath (eg. to /home/andremotz/nc-deployment/certs-pv)

$ kubectl create -f omgwtfssl-pod.yaml

Deployment + Service: Nginx reverse Proxy

One could already easily adjust the Nextcloud-service to publish HTTP-driven service. However we want to use a Nginx-instance in front of our Nextcloud to be able to use HTTPS-encryption. For the proxy we are not using a Deployment but a Pod, to be able to make use of standard HTTP/HTTPS-ports 80 & 443

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nginx.conf

$ nano nginx.conf
--> change server_name (two locations in the file!) to the server name you've provided before for SSL_SUBJECT
--> change ssl_certificate to the filename you've provide before for SSL_CERT
--> change ssl_certificate_key to the filename you've provide before for SSL_KEY

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/proxy-pod.yaml

$ nano proxy-pod.yaml
--> change cert's hostPath to the location you have provided before---> change nginx-config's hostpath to the location where you've stored nginx.conf before (eg. /home/andremotz/nc-deployment/nginx.conf)
--> change nginx-logs' hostpath to a proper location

$ kubectl create -f proxy-pod.yaml

Now you should be able to point your browser to https://<yourserver> and see a new Nextcloud-instance, running on a super-hyper nextlevel-Kubernetes cluster, that you could use for further cool stuff 😉

Final hint:
Disable swap permanently – add a comment to the /etc/fstab – swap entry like this:

#/swap.img none swap sw 0 0

That’s it!! 😉