WordPress on Kubernetes

The Definitive Guide for WordPress on k8s

Just add another pod, no?

We hear this a lot in conversations about scaling. In this section we’re going to literally “just add another pod” and see what happens with our WordPress application in the Kubernetes cluster.

Note: all code samples from this section are available on GitHub.

Preparing MariaDB

From our previous sections you should be left off with a clean default namespace, no pods or services. OpenEBS should be installed and running with a working DiskPool and the Storage Classes for replicated and Hostpath volumes.

$ kubectl -n openebs get diskpool
NAME      NODE   STATE     POOL_STATUS   CAPACITY      USED   AVAILABLE
pool-k0   k0     Created   Online        10724835328   0      10724835328
pool-k1   k1     Created   Online        10724835328   0      10724835328
pool-k2   k2     Created   Online        10724835328   0      10724835328

$ kubectl get sc                 
NAME                     PROVISIONER               RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
mayastor-etcd-localpv    openebs.io/local          Delete          WaitForFirstConsumer   false                  5d1h
mayastor-loki-localpv    openebs.io/local          Delete          WaitForFirstConsumer   false                  5d1h
openebs-hostpath         openebs.io/local          Delete          WaitForFirstConsumer   false                  5d1h
openebs-replicated       io.openebs.csi-mayastor   Delete          Immediate              true                   4d23h
openebs-single-replica   io.openebs.csi-mayastor   Delete          Immediate              true                   5d1h

We’re going to focus on “just adding another pod” to our WordPress application in this section, so let’s first get a working MariaDB service with a single pod and a replicated persistent volume.

We’re going to introduce a new concept here, a volumeClaimTemplates attribute in our StatefulSet spec. This attribute dynamically creates persistent volumes for pods managed by our StatefulSet. This allows us to ditch the previously used volume-clamis.yml manifest altogether, and have our mariadb.statefulset.yml file define its volume:

  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
      - ReadWriteOnce
      storageClassName: openebs-replicated
      resources:
        requests:
          storage: 1Gi
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Delete
    whenScaled: Delete

This is similar to a regular persistent volume claim. We have a retention policy here as well, set to delete the volume claim when the StatefulSet is deleted or scaled. These are set to Retain by default to prevent any unintentional data loss, but Delete works best when you’re mostly testing around.

Also note that we’ve changed the name to data here, and the final PVC name will be prefixed with the name of the StatefulSet, resulting in claims named mariadb-data-0. This also requires a slight change in how we mount the volumes: the volumes section in the template spec is gone, the volumeMounts section in the container definition can use the “locally” defined claim name:

    spec:
      containers:
      - name: mariadb
        image: mariadb:10.11
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql

Let’s apply the MariaDB manifests and observe the cluster:

$ kubectl apply \
  -f mariadb.service.yml \
  -f mariadb.configmap.yml \
  -f mariadb.statefulset.yml
service/mariadb created
configmap/mariadb unchanged
statefulset.apps/mariadb created

$ kubectl get pods
NAME        READY   STATUS    RESTARTS   AGE
mariadb-0   1/1     Running   0          29s

$ kubectl get pvc 
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         VOLUMEATTRIBUTESCLASS   AGE
data-mariadb-0   Bound    pvc-b424e36a-8ecd-46ec-97d8-255cd0ab62b4   1Gi        RWO            openebs-replicated   <unset>                 35s

Let’s move on to our WordPress service.

Preparing WordPress

Just like with a MySQL or MariaDB service, a WordPress application will have some state: the WordPress core files, any plugins or themes we install, any media that we upload.

These are stored on the filesystem, and we’ll want to retain them in case of a pod crash or restart. We want to be able to run our pods anywhere in the cluster, so just like with MariaDB, we’ll use a replicated persistent volume for every WordPress pod.

For simplicity, let’s also disregard our previous Nginx deployment and service for now and switch back to using the Apache-based WordPress container image. This means our WordPress service will be the one exposed through a NodePort mapped to port 80 (rather than a FastCGI service on port 9000).

Let’s update our StatefulSet for WordPress wordpress.statefulset.yml to reflect all that:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
  template:
    metadata:
      labels:
        app: wordpress

Nothing new here so far.

    spec:
      containers:
      - name: wordpress
        image: wordpress:6.5
        ports:
        - containerPort: 80
        volumeMounts:
        - name: data
          mountPath: /var/www/html

We changed the image here to the Apache-based container image on DockerHub. We also updated the volumeMounts name to reference a volume named data which we’ll use in our claim templates.

        env:
          - name: WORDPRESS_DB_HOST
            value: mariadb
          - name: WORDPRESS_DB_USER
            valueFrom:
              configMapKeyRef:
                name: mariadb
                key: username
          - name: WORDPRESS_DB_NAME
            valueFrom:
              configMapKeyRef:
                name: mariadb
                key: database
          - name: WORDPRESS_DB_PASSWORD
            valueFrom:
              configMapKeyRef:
                name: mariadb
                key: password

This section didn’t change much, we’re still reading some ConfigMaps, but do note that we removed a volumes section since claims are now going to be defined directly in the StatefulSet:

  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
      - ReadWriteOnce
      storageClassName: openebs-replicated
      resources:
        requests:
          storage: 1Gi

  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Delete
    whenScaled: Delete

Similar to the MariaDB specification, a single volume claim named data which will be prefixed with our StatefulSet name and form claims named wordpress-data-0, etc.

Our wordpress.service.yml file will also need a slight update for the new port:

apiVersion: v1
kind: Service
metadata:
  name: wordpress
spec:
  type: NodePort
  ports:
  - port: 80
    nodePort: 30007
  selector:
    app: wordpress

Let’s create the Service and StatefulSet with kubectl and observe the running pods:

$ kubectl apply \
  -f wordpress.service.yml \
  -f wordpress.statefulset.yml 
service/wordpress created
statefulset.apps/wordpress created

$ kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
mariadb-0     1/1     Running   0          17m
wordpress-0   1/1     Running   0          39s

So far so good, we have a single-pod MariaDB database, and a single-pod WordPress application, which we can reach on any of our Node’s IP addresses or hostname on the port 30007:

Run through the installer and make sure you can login to wp-admin. Now comes the fun part.

Scaling WordPress

Scaling things up and down is incredibly easy in Kubernetes. Watch!

$ kubectl scale statefulset/wordpress --replicas=2
statefulset.apps/wordpress scaled

$ kubectl get pods
NAME          READY   STATUS    RESTARTS   AGE
mariadb-0     1/1     Running   0          24m
wordpress-0   1/1     Running   0          7m59s
wordpress-1   1/1     Running   0          32s

There we have it! You’ve “successfully” scaled your WordPress application to two pods in Kubernetes. Don’t get too excited just yet though!

If you try and visit the site now, you’ll find that it almost works. However, if you login, you’ll find yourself logged out after a couple of refreshes. Why is that?

We won’t be diving too deep into how authentication in WordPress works, but worth knowing that the “logged in” state is shared between the MariaDB database and some cookies in your browser. Some of this data also happens to be hashed and salted.

These salts and some authentication related keys are usually defined in the wp-config.php file, but since we haven’t defined any, the WordPress container will generate them in the entrypoint script. This is good for security, but not good in our case, since the two pods are now operating with different sets of keys and salts.

Luckily for us, we can override these using environment variables passed to the container. Let’s create a new wordpress.secrets.yaml file and define a new Secret for our Kubernetes cluster. Secrets are very similar to ConfigMaps but treated slightly differently:

apiVersion: v1
kind: Secret
metadata:
  name: wordpress-secrets
stringData:
  WORDPRESS_AUTH_KEY: 7052c0863ea6269f44f0de17dc5c5879fdc02fd4
  WORDPRESS_SECURE_AUTH_KEY: e769128cb8376581e309a002466b1346910e8bd3
  WORDPRESS_LOGGED_IN_KEY: 9d3a12e80b9aa24c5a045f1f092e6ffb93f9823a
  WORDPRESS_NONCE_KEY: 2b4d5a460c2dde65380a47188b4b5ebbe723ab18
  WORDPRESS_AUTH_SALT: ba846d9aeefe6c706e5c93a750c1e4cd601e4408
  WORDPRESS_SECURE_AUTH_SALT: 667895c8d69089620e44a7f1a97b1ac3efc6e386
  WORDPRESS_LOGGED_IN_SALT: fcf21a031d5a54eab2c4749257829b0d2830ea7f
  WORDPRESS_NONCE_SALT: 6f3a1db3c6bd211d404e071a0ad1801c53e117dd

These are just a few random hashes we generated:

$ head -n100 /dev/urandom | sha1sum

Note that Kubernetes Secrets like to accept base64 encoded data in a data attribute by default, but since we’re using unencoded strings, we use the stringData attribute instead.

Let’s also update our wordpress.statefulset.yml manifest to include these new keys in our environment variables, passed to the WordPress container:

    spec:
      containers:
      - name: wordpress
        image: wordpress:6.5-apache
        ports:
        - containerPort: 80
        volumeMounts:
        - name: data
          mountPath: /var/www/html
        envFrom:
        - secretRef:
            name: wordpress-secrets
        env:
          - name: WORDPRESS_DB_HOST
            value: mariadb

The secretRef attribute is a very convenient of adding sensitive data to a container’s environment variables. Similarly, you can use configMapRef to add keys from a ConfigMap, we’ll leave that up to you for now.

Let’s add our Secret and updated StatefulSet to Kubernetes:

$ kubectl apply \
  -f wordpress.secrets.yml \
  -f wordpress.statefulset.yml
secret/wordpress-secrets created
statefulset.apps/wordpress configured

Our WordPress pods will be recreated after applying these manifests, and we’ll finally be able to login to the wp-admin Dashboard once, and have it work across all pods, since they are now all running with the same set of salts and keys.

Problem solved? Well not quite…

The Filesystem

Surely clicking around the admin, adding posts, pages, users and whatnot, will all work like a charm. This is because all that data is stored in the database, which is shared between our two pods. The filesystem, however, is a different story.

We have two pods, each with its own individual PersistentVolumeClaim and PersistentVolume:

$ kubectl get pvc
NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         VOLUMEATTRIBUTESCLASS   AGE
data-mariadb-0     Bound    pvc-b424e36a-8ecd-46ec-97d8-255cd0ab62b4   1Gi        RWO            openebs-replicated   <unset>                 145m
data-wordpress-0   Bound    pvc-8322e7ff-0cb1-4fc2-a696-806514709128   1Gi        RWO            openebs-replicated   <unset>                 129m
data-wordpress-1   Bound    pvc-9359cb0d-2e17-488f-8e99-280b97b315a3   1Gi        RWO            openebs-replicated   <unset>                 121m

What will happen, if we add a new plugin to our WordPress install? Let’s try that out. Navigate to Plugins – Add New Plugin. Find one called Query Monitor (quite a useful one!) and install it, don’t activate it just yet though!

After a successful installation, browse to the plugins list, and hit the Refresh button in your browser a few times. You’ll see that it’s inconsistent – on some requests, the new Query Monitor plugin will be in the list, but on some requests it won’t be there.

What’s going on?

Plugins (and themes) in WordPress consist of one or more files in the wp-content/plugins (or themes) directory, so when you install a new on in the Dashboard, WordPress will download an archive and unpack it there. The problem though that this happens on just one lucky pod.

Let’s inspect the filesystem on each of the running pods to find out more:

$ kubectl exec -it wordpress-0 -- ls /var/www/html/wp-content/plugins
akismet  hello.php  index.php  query-monitor

$ kubectl exec -it wordpress-1 -- ls /var/www/html/wp-content/plugins
akismet  hello.php  index.php

Okay, so in our case, the query-monitor plugin exists on the wordpress-0 pod, but missing on the wordpress-1 pod, which is why when our requests hits our second pod (remember our Service load balances between the two pods) the plugin is not there.

You can try activating the plugin and you might be lucky to get it working for one or two requests, but the moment you hit the pod where the plugin files do not exist, WordPress will automatically deactivate the plugin (assuming you’re still on the plugins list page).

We can certainly try and fix this problem by hand – i.e. by copying the plugin files from one pod to another – and that will work, right until we add a third pod and then a fourth pod. The same is true for themes, and even media uploads!

Going Stateless

State is what usually breaks horizontal scaling in web applications, and many modern apps are stateless by design, relying on databases, cache servers, shared filesystems or other remote storage services for state.

WordPress however is over twenty years old and very, very stateful: plugins, themes, media, translations, upgrades, caches and more! For all this to work, we’ll need to patch a few things up, and there will be a couple of tradeoffs along the way.

In this section, we’ve successfully just added another pod to our Kubernetes cluster, and it also worked to some extent. In the next section we’ll be looking at some options to remove state from a WordPress application to make sure we can scale out.

Feel free to remove all the services, stateful sets, secrets and config maps along with any persistent volumes and claims:

$ kubectl delete -f .
configmap "mariadb" deleted
service "mariadb" deleted
statefulset.apps "mariadb" deleted
secret "wordpress-secrets" deleted
service "wordpress" deleted
statefulset.apps "wordpress" deleted

Head over to the next section when ready!