📅 Fri, Dec 31, 2021 ⏲️ ~ 6 minutes

How to communicate an ESP32 with a web page via MQTT

New entry about communication with the ESP8266 and ESP32. This time we will see how to communicate a web page served from the ESP32 via MQTT.

We have already seen How to use asynchronous MQTT and How to send text in Json format via MQTT. These communications were between devices, between backends. Now we want to communicate with a web served to the client, with the frontend.

This is not common (or shouldn’t be) because we have many ways to communicate an ESP8266/ESP32 acting as a backend with the frontend. So we’ve seen Ajax calls and communication with Websockets.

But, if our system is using MQTT, it can be useful to use the same system to inform the frontend. So, in short, we want to create a web client for the MQTT protocol, and serve this client from the ESP32.

Here is where we encounter the first problem, how can we communicate a web page, which lives in its “web world” of HTTP calls, with a communication via MQTT.

Communicating a web via MQTT

A web page communicates via HTTP, which is a protocol that operates on the Application layer (layer 7 of the OSI model). This, in turn, operates over TCP, which operates on the transport layer (layer 4). That is to say, except for certain exceptions or changes in the future (HTTP 3?), a web page can only communicate through HTTP requests, websockets, or WebRTC.

And how does MQTT fit into this? MQTT is also a protocol of the Application layer (layer 7) that generally operates over TCP/IP. In summary, MQTT goes directly against a socket and will not enter an HTTP request.

So we can’t connect them? Well, here come the Websockets to the rescue. We can configure MQTT to work over Websockets, which as we said, we can manipulate from a web page.

But first we need to configure our broker to accept Websockets. If this is possible, or how to configure it, will depend on the broker we are using. In the rest of the entry, we will see it as if we had a local installation of Mosquitto as we saw in this post.

And, on the other hand, we will need a JavaScript library that allows us to read this MQTT communication via Websockets from the served web page.

Configure Mosquitto to use Websockets

If we are using Mosquitto as an MQTT broker, communication via Websockets is disabled by default. So we must enable it in the configuration in order to communicate from a web page.

mqtt-mosquitto

To do this, we edit the ‘mosquitto.conf’ file, which you will have in the folder where you have installed Mosquitto. Here look for the “Extra listeners” section and uncomment the Websockets lines, removing the ’#’ symbol from the beginning of the line.

## =================================================================

## Extra listeners

## =================================================================
...
listener 9001
protocol websockets
...

Thus we have added an extra listener for Mosquitto, through Websockets acting on port 9001.

Paho JavaScript Client Library

The other component we need for our communication is a JavaScript library for MQTT. There are several libraries, but the most used is the Paho JavaScript Client library from Eclipse.

It is a library available in several languages, including Java, JavaScript, Python, C/C++, Rust, and Golang. More information on the project page https://www.eclipse.org/paho

pago_logo

Code Example

Now that we have configured our broker to accept communication via Websockets, and with our Paho JS Client library, it’s time to put it all together to see an example.

The program we will upload to the ESP32, responsible for the MQTT communication, is basically the same as the previous entry. Therefore, we will only comment on the differences, and in case of doubts you can consult the previous tutorial.

main.ino

The code of the main program loop is as follows. The difference is that we have added a call to ‘InitServer()’ to serve the web page from the ESP32. In addition, we have added the necessary includes.

#include <WiFi.h>
#include <SPIFFS.h>
#include <ESPAsyncWebServer.h>
#include <AsyncMqttClient.h>
#include <ArduinoJson.h>

#include "config.h"  // Replace with your network data
#include "Server.hpp"
#include "MQTT.hpp"
#include "ESP32_Utils.hpp"
#include "ESP32_Utils_MQTT_Async.hpp"

void setup(void)
{
  Serial.begin(115200);
  SPIFFS.begin();

  delay(500);

  WiFi.onEvent(WiFiEvent);
  InitMqtt();

  ConnectWiFi_STA();
  InitServer();
}

void loop()
{
  PublishMqtt();

  delay(1000);
}

Server.hpp

The other difference is the ‘Server.hpp’ file, where we have put in the logic associated with serving the web page. We are simply serving a web page from SPIFFS, something we have already seen in the rest of the entries in the series.

AsyncWebServer server(80);

void InitServer()
{ 
  server.serveStatic("/", SPIFFS, "/").setDefaultFile("index.html");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(400, "text/plain", "Not found");
  });

  server.begin();
  Serial.println("HTTP server started");
}

Now we go to what’s really new, which is the page we are serving, and which must act as a MQTT communication client.

index.html

In the ‘index.html’ file, it would look like this.

<!doctype html>
<html lang="">

<head>
  <title>ESP32 MQTT</title>
  <meta charset="utf-8">
  <meta http-equiv="x-ua-compatible" content="ie=edge">
  <title></title>
  <meta name="description" content="">
  <meta name="viewport" content="width=device-width, initial-scale=1">
</head>

<body>
  <div id="app">
  </div>

  <script type="text/javascript" src="./vendor/nativeWs.min.js"></script>
  <script type="text/javascript" src="./vendor/mqttws31.min.js" ></script>

  <script type="text/javascript" src="./js/API.js"></script>
  <script type="text/javascript" src="./js/app.js"></script>
</body>

</html>

API.js

In the ‘API.js’ file, we have encapsulated the functions associated with the workflow of our application. It is worth noting that in the onConnect() method we subscribe to the ‘hello/world’ topic.

On the other hand, the onMessageArrived method receives text formatted as JSON, parses it, and uses a little Vanilla JavaScript to add the message content to the web page. The rest of the functions in this example only display the data for debugging in the console.

function onConnect() {

  var options = {
    qos: 0,
    onSuccess: onSubSuccess,
    onFailure: onSubFailure
  };
  client.subscribe('hello/world', options);
}

function onFailure(message) {
  console.log(message)
}

function onConnectionLost(responseObject) {
  if (responseObject.errorCode !== 0) {
    console.log("onConnectionLost:" + responseObject.errorMessage);
  }
}

function onMessageArrived(message) {
  console.log(message)
  var topic = message.destinationName;
  var payload = message.payloadString;

  let json = JSON.parse(payload);

  var element = document.getElementById('app');
  var newElement = document.createElement('div');
  newElement.appendChild(document.createTextNode(json.data));
  element.appendChild(newElement);
}

function onSubFailure(message) {
  console.log(message)
}

function onSubSuccess(message) {
  console.log(message)
}

app.js

Finally, the logic of our page is in the ‘app.js’ file. We define an MQTT client with the Paho library, where we specify the address of our broker, the listening port for Websockets, and a unique identifier for the client.

For the identifier, we have used a function that generates a random GUID, although we could have used any other, as long as we don’t repeat the client identifier. Finally, we have associated each of the client’s events to our functions in the ‘API.js’ file.

function createGuid() {
  return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {
    var r = Math.random() * 16 | 0,
      v = c === 'x' ? r : (r & 0x3 | 0x8);
    return v.toString(16);
  });
}

client = new Paho.MQTT.Client("192.168.1.150", 9001, createGuid())

var options = {
  onSuccess: onConnect,
  onFailure: onFailure
};

client.onConnectionLost = onConnectionLost;
client.onMessageArrived = onMessageArrived;

client.connect(options);

Result

With this, we upload both the program and the web to the ESP32, run everything, and access the web page served from the ESP32. If everything has gone well, we will see the received messages appear as they are added to the served web page.

esp32-mqtt-web-resultado

In this example, it simply contains the value of Millis(). Obviously, in a real project, the JSON-formatted data would contain more information. We will see this in the last entry of the series. But first, in the next one, we will see how to do this using VueJS. Until next time!

Communicating a web via MQTT

Configure Mosquitto to use Websockets

Paho JavaScript Client Library

Code Example

Result

Download the code