Raspberry Pi 4B — Kubernetes Cluster Temperature Monitoring with DaemonSets — Slow Guide

Easy way to monitor temperature of your Kubernetes Cluster

Raspberry Pi 4B — Kubernetes Cluster Temperature Monitoring with DaemonSets
Raspberry Pi 4B — Kubernetes Cluster Temperature Monitoring with DaemonSets
Photo by Kenny Luo on Unsplash

There has been a lot of talk about Raspberry Pi 4B overheating issues. New firmware helped out with that, but wouldn’t it be nice to monitor temperature of all the nodes in our Raspberry Pi Kubernetes cluster? In this article I will describe how we can do that.

I wrote an article before about my cluster setup withRaspberry Pi 4B and if interested, you can find more details in post below.

I thought that best way to do this might be to create and deploy agent application that will be running on all nodes, and have it deployed via DaemonSet. Agent is written in Node.js as I thought that would be simplest for our purposes. On top our pods we are going to put Headless Service, which will enable pods that share same label, to know about each others via DNS. We will also create regular service with load balancer which give us external IP address that we will use to get temperature of our cluster.

Our agent is going to expose two end points api/get-node-temp and api/get-cluster-temp. First one will be used to get temperature for individual node, and second one to get temperature of entire cluster.

Once request comes to api/get-cluster-temp to one of our agents, plan is to have that particular agent find all the other agents, and contact them via api/get-node-temp. It will then aggregate, and send back the results.

To get temperature we will be using

cat /sys/class/thermal/thermal_zone0/temp

As temperature is provided in millidegrees Celsius, we would need to divide it by 1000 to get more meaningful numbers.

In short DaemonSets ensure that pod we choose is running on all nodes in the cluster. This is particularly useful when you need to deploy something that provides some basic services to all your other pods and needs to be present on every node. During scaling if you let’s say add new nodes, DaemonSet will ensure that they have our pod up and running.

First we would need to download latest LTS Node.js from here, and install it to development computer.

Node.js installation on MacBook
Node.js installation on MacBook
Node.js installation on MacBook

We will create a new folder that will hold our agent

mkdir pi-temp-agent
cd pi-temp-agent

Now we will run npm init to create new project and accept all the defaults. After that is done, we will add few libraries to it

npm install --save express
npm install --save express-prettify
npm install --save moment
npm install --save request
npm install --save request-promise

We should create new file with a name TemperatureAgent.js and open it. I’m using Visual Studio Code which you can download from here, but you can use whatever editor you prefer.

RaspberryPi Temperature Monitoring Agent source code
RaspberryPi Temperature Monitoring Agent source code
RaspberryPi Temperature Monitoring Agent

Copy following code inside the file

// Node Temperature Monitoring agent// Declarations
const express = require('express');
const pretty = require('express-prettify');
const exec = require('child_process').exec;
const rp = require('request-promise');
const moment = require('moment');
const app = express();
const port = 5000;
const route_404 = '404 - Page not found';
app.use(pretty({ query: 'pretty' }));
// Execute shell command
function sh_exec(command) {
return new Promise((resolve, reject) => {
exec(command, (error, stdout, stderr) => {
if (error) console.warn(error);
if (stderr) {
console.warn(stderr);
resolve("");
}
else resolve(stdout);
});
});
}
// API route for getting temperature of individual node
app.get('/api/get-node-temp', async (_req, resp) => {
var podName = (await sh_exec('hostname')).trim();
var nodeName = (process.env.MY_NODE_NAME);
var temperature = (await sh_exec('cat /sys/class/thermal/thermal_zone0/temp')).trim() / 1000;
var dateUpdated = moment().format("DD-MM-YYYY, HH:mm:ss");
resp.json({nodeName, podName, temperature, dateUpdated});
});
// API route for getting temperature of the cluster
app.get('/api/get-cluster-temp', async (req, resp) => {
var podIpList = (await sh_exec("nslookup pi-temp-headless 2>/dev/null | grep Address | awk '{print $3}'")).split("\n");
var podIpList = podIpList.filter((item) => { return item.trim() != '' });
var result = [];
for (let i = 0; i < podIpList.length; i++){
await rp(`http://${podIpList[i]}:5000/api/get-node-temp`).then(body => {
result.push(JSON.parse(body));
}).catch(err => {
console.log(err);
});
}
resp.json(result);
});
// All other routes return 404
app.get('*', function(_req, res){
res.status(404).send(route_404);
});
// Start listening
app.listen(port, () => console.log(`TemperatureAgent listening on port ${port}!`));

With agent ready, we can now create docker container that will hold it. Create new file named Dockerfile and copy following content inside it.

FROM arm64v8/node:current-alpineWORKDIR /appCOPY package*.json ./
COPY *.js .
RUN npm install --productionEXPOSE 5000CMD ["node", "TemperatureAgent.js"]

Let’s describe what’s going on here:

  • As a base we are using Apline Linux, which is advertised as small, resource efficient and secure distribution.
  • We are setting our app to be running out of /app folder
  • We copy our application to /app folder
  • We run npm install which will install packages we are using
  • We are exposing port 5000 which will be used by our agent
  • And finally we are executing our application

With that prepared, we are going to build and push container to our repository

docker build -t [your-username]/pi-temp-agent .
docker push [your-username]/pi-temp-agent

It is time to create our Kubernetes DaemonSet. Create file named pi-temp-agent.yaml and copy following content to it. Don’t forget to replace [your-username] with your docker username:

apiVersion: v1
kind: Namespace
metadata:
name: pi-temp
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: pi-temp-agent
namespace: pi-temp
labels:
app: pi-temp-agent
spec:
selector:
matchLabels:
app: pi-temp-agent
template:
metadata:
labels:
app: pi-temp-agent
spec:
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: pi-temp-agent
image: [your-username]/pi-temp-agent:latest
env:
- name: MY_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
ports:
- containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
name: pi-temp-headless
namespace: pi-temp
spec:
clusterIP: None
selector:
app: pi-temp-agent
---
apiVersion: v1
kind: Service
metadata:
name: pi-temp-public
namespace: pi-temp
spec:
selector:
app: pi-temp-agent
ports:
- protocol: TCP
port: 80
targetPort: 5000
type: LoadBalancer

In first section we are defining namespace pi-temp that we will be using. After that DaemonSet will be created which will run our pod on all the nodes. It’s important to note, that by default, DaemonSet will only use worker nodes. To have it running also running on master node, we have to add

tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule

To know on what node is each individual container exactly running, we will be setting env variable MY_NODE_NAME on container creation and assign name of the node to it with following lines:

env:
- name: MY_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName

Third section will create headless service pi-temp-headless, by setting service’s clusterIP: None. This enables pods to learn about each other by running nslookup pi-temp-headless

List of all pods created with DaemonSet
List of all pods created with DaemonSet
List of all pods created with DaemonSet

And finally we are creating a service pi-temp-public with load balancer which will give us external IP address to use to access our agent application. It will expose port 80 to us and forward our request to pods on port 5000.

With everything in place, we can deploy our agent by running following command

kubectl apply -f pi-temp-agent.yaml

Let’s now watch as our cluster is doing it’s magic and deploying containers to nodes.

watch -n 1 kubectl get all -n pi-temp -o wide

Once everything is in running state, our agents are ready to be queried.

RaspberryPi Temperature Monitoring Agent is now running in our cluster
RaspberryPi Temperature Monitoring Agent is now running in our cluster
RaspberryPi Temperature Monitoring Agent is now running in our cluster

Please write down external IP address for pi-temp-public service. We will be using it to retrieve temperature information from our nodes.

Simplest way to test the the application would be by running

curl 192.168.200.1/api/get-cluster-temp?pretty

We are using pretty parameter to format our JSON output.

Getting RaspberryPi Kubernetes Cluster temperature from command line
Getting RaspberryPi Kubernetes Cluster temperature from command line
Getting RaspberryPi Kubernetes Cluster temperature from command line

And once more from the browser

Getting RaspberryPi Kubernetes Cluster temperature from command line
Getting RaspberryPi Kubernetes Cluster temperature from command line
Getting RaspberryPi Kubernetes Cluster temperature from browser

That’s it for now. We are ready for summer! Our agents are running and providing us with temperature information of our cluster.

IT professional with over 20 years of experience in the field with great passion for technology, space exploration and science fiction.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store