featured-image

Nextcloud-Docker on Kubernetes-cluster + SSL certificates

Folks, this has taken me ages (ok to be honest – three weeks, which is still a lot ;-)) to get up and running. But finally, I have made it and learned a lot about Docker, Docker-Compose, Nginx and Kubernetes of course.


Summary

In this post I am showing how to set up a Nextcloud on Kubernetes + Ubuntu 18.04. I am running a Nextcloud-instance on a public VM to sync stuff with my phone. Another one (with more data) is only accessible via my LAN/VPN. The first one I am encrypting via Let’s Encrypt (open todo!) and the 2nd one via self-signed certificates.

The stacks are set up via Kubernetes to be able to scale horizontally in the future, since I had to live with some performance-issues in the past. GitHub – andremotz – nextcloud-kubernetes

Introduction

For more than two years I was running my own Nextcloud on a Linux-VM, which was accessible via my LAN & OpenVPN. Therefore, I was using one of the thousands of Nextcloud-Docker Images, that are available on Docker Hub. However, the solution did not meet my requirements anymore:

  1. The Image’s author recommended not update the Docker-container, which is an anti-pattern for Container-driven software. 
  2. I was not able to create my own certificates or use Let’s Encrypt.
  3. I had some serious performance-issues that I could not really explain to myself.

Most stuff could have been done using Docker-Compose but in this case, I would have to upgrade to Docker-Swarm, to be able to scale horizontally.

Updates:

My own tutorial is marvellous – since I’ve created the post/GitHub-content I had to re-setup my own NC-cluster at least four times. I always used my own tutorial to prove if everything is still working 🙂

  • 2019-12-02: Updated Deployments: added selector/matchLabels, changed apiVersion: apps/v1

What I will cover

  • Setting up Kubernetes on Ubuntu 18.04 (although this task can be easily ported to other distros as well). I did not use any special/latest Docker-repos to be compatible with Kubernetes’ latest release.
  • Initializing and configuring all necessary Pods for Nextcloud to run a secure home to sync data. Therefore, I am using the default Nextcloud-Docker image, a Nginx-Proxy + OMGWTFSSL-Docker Image for self-signed certificates and MariaDB.

    What I want to cover in the future

    • Putting the certs into an emptyDir-Volume
    • Running the cluster on multiple machines
    • So far, I have issues with the Nextcloud-iOS app. I could connect to my old Nextcloud-setup, so I assume it is some issue with the latest version.
    • Let’s Encrypt offers the user’s browser some public authority. The bad side is that the encrypted server needs to be public available on the Internet. Self-signed certificates can be created pretty easy but show the user an ugly “not secure!!11” message in the browser.

What I do not cover

Kubernetes is a big universe, but not all features were necessary to get my Nextcloud up and running.

  • It is good to have the option to scale in the future. But so far, I am running the Kubernetes-cluster on one machine without any scaling. As of now, this is some overkill, but this allows me in the future to extend my setup.
  • I am not using any fancy PersistetVolumes. Instead I am using HostPaths. In the future I might update this post to be able to scale better.
  • Using Ingress-proxies was just not necessary to me. All I need was a port 80 & 443-routing to my Kubernetes-cluster. Hence, a Nginx reverse proxy-Pod does the job pretty well, too.
  • Declarative vs Imparative: I put everything into YAML -files. However, I stopped using kubectl apply since kubectl create did a better & faster job when changing config-files. Maybe here I will learn something else in the future, but to make my set-up, I had less headache with using kubectl create + YAML-files.

Let’s jump in!

Download and set up Ubuntu 18.04 LTS. I won’t cover any further steps here since there are a lot of information out there.

Setting up Docker & Kubernetes on Ubuntu

Source: https://linuxconfig.org/how-to-install-kubernetes-on-ubuntu-18-04-bionic-beaver-linux

Please check the source for further explanation. I have done the following steps on three different machines without problems.

$ sudo apt update && sudo apt upgrade -y
$ sudo apt install docker.io
$ sudo systemctl enable docker
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
$ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
$ sudo apt install kubeadm
$ sudo swapoff -a
$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
$ kubectl taint nodes --all node-role.kubernetes.io/master-

At this place you should note down the shown kubeadm join-message in your console to be able to connect further Kubernetes-nodes in the future.

Setting up Nextcloud on Kubernetes

Extra-hint: Use the following in an extra-terminal to be able to see what the Kubernetes-cluster is doing

$ watch -n 10 kubectl get deployment,svc,pods,pvc,pv,ing

Deployment + Service: MariaDB

As a user (not root) create a folder nc-deployment, download pre-defined MariaDB-descriptions, adjust it to your needs and deploy:

$ mkdir nc-deployment
$ cd nc-deployment
$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-deployment.yaml

$ nano db-deployment.yaml
--> change MYSQL_PASSWORD here
--> change MYSQL_ROOT_PASSWORD here
--> change db's HostPath here, which should be the absolute location of 'nc-deployment'/db-pv (eg /home/andremotz/nc-deployment/db-pv)

$ kubectl create -f db-deployment.yaml

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/db-svc.yaml
$ kubectl create -f db-svc.yaml

Deployment + Service: Nextcloud:

Next, download Nextcloud-descriptions, adjust them and deploy:

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-deployment.yaml

$ nano nc-deployment.yaml
--> change NEXTCLOUD_URL
--> change NEXTCLOUD_ADMIN_PASSWORD
--> change MYSQL_PASSWORD (the value you've entered before)
--> change html's hostPath (eg. to /home/andremotz/nc-deployment/nc-pv)

$ kubectl create -f nc-deployment.yaml

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nc-svc.yaml
$ kubectl create -f nc-svc.yaml

Create self-signed certificates

The OMGWTFSSL-Docker image offers easy-to-use certificate-creation. Here we are using only a Pod, not a Deployment. Once the certificates are created, the Pod will stop.

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/omgwtfssl-pod.yaml

$ nano omgwtfssl-pod.yaml
--> change SSL_SUBJECT to your server's name
--> change CA_SUBJECT to your mail-adress
--> change SSL_KEY to a proper filename
--> change SSL_CSR to a proper filename
--> change SSL_CERT to a proper filename
--> change cert's hostPath (eg. to /home/andremotz/nc-deployment/certs-pv)

$ kubectl create -f omgwtfssl-pod.yaml

Deployment + Service: Nginx reverse Proxy

One could already easily adjust the Nextcloud-service to publish HTTP-driven service. However we want to use a Nginx-instance in front of our Nextcloud to be able to use HTTPS-encryption. For the proxy we are not using a Deployment but a Pod, to be able to make use of standard HTTP/HTTPS-ports 80 & 443

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/nginx.conf

$ nano nginx.conf
--> change server_name (two locations in the file!) to the server name you've provided before for SSL_SUBJECT
--> change ssl_certificate to the filename you've provide before for SSL_CERT
--> change ssl_certificate_key to the filename you've provide before for SSL_KEY

$ wget https://raw.githubusercontent.com/andremotz/nextcloud-kubernetes/master/kubernetes-yaml/proxy-pod.yaml

$ nano proxy-pod.yaml
--> change cert's hostPath to the location you have provided before---> change nginx-config's hostpath to the location where you've stored nginx.conf before (eg. /home/andremotz/nc-deployment/nginx.conf)
--> change nginx-logs' hostpath to a proper location

$ kubectl create -f proxy-pod.yaml

Now you should be able to point your browser to https://<yourserver> and see a new Nextcloud-instance, running on a super-hyper nextlevel-Kubernetes cluster, that you could use for further cool stuff 😉

Final hint:
Disable swap permanently – add a comment to the /etc/fstab – swap entry like this:

#/swap.img none swap sw 0 0

That’s it!! 😉

 

6 thoughts on “Nextcloud-Docker on Kubernetes-cluster + SSL certificates”

  1. –> change NEXTCLOUD_URL
    –> change NEXTCLOUD_ADMIN_PASSWORD
    –> change MYSQL_PASSWORD (the value you’ve entered before)
    –> change html’s hostPath (eg. to /home/andremotz/nc-deployment/nc-pv)

    what is right formate to added, I’m trying but got validation an error

  2. Hey mhd,

    Certain things might be wrong in your case. However here some additional hints:
    1) for NEXTCLOUD_URL I am using my internal’s host-name:
    – name: NEXTCLOUD_URL
    value: https://ubuntu-server-2:8001
    ubuntu-server-2 is the internal Host-name. However I do not enter my NC via ubuntu-server-2 but having an additional DNS entry in my home-net: nextcloud.nestfunk.lan which I use to access my NC.

    2) NEXTCLOUD_ADMIN_PASSWORD
    Make sure having the following format/syntax:
    – name: NEXTCLOUD_ADMIN_PASSWORD
    value: yourpAssw0rd

    In general, with YAML-files keep track of correct indention, it’s like coding with Python.

    3) MYSQL_PASSWORD -> Like said in #2, make sure to check the syntax. Also double-check to have here the same entry as in db-deployment.yaml.

    Hope that helps! Keep us informed 🙂

  3. A quick search suggested me:
    sudo hostname -> allows you to change your hostname. I do not use Google-cloud services, hence, I can’t provide you support there.

  4. Hi Alok, definitiely a good choice. However when starting new skills (in this case Kubernetes) I find it very useful to start with basic things first and try to set it up on your own. Hence, one understands better what is happening under the hood.

    Some weeks ago my Kubernetes cluster was not working anymore, after doing a system-wide update of software. Turned out that my Kubernetes-internal network was not working anymore. As a result I have rebuilt everything based on my configuration files and this blog-entry.

    Now I have my data back online plus an updated Kubernetes-cluster 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *