Device Dashboard

Overview

This tutorial shows an example of how to build a device dashboard, what can be very useful for headless devices.

• This Web dashboard provides:
  • User Authentication: login protection with multiple permission levels
  • The web UI is optimised for size and for TLS usage
  • Logged users can view/change device settings and communicate to the MQTT server
  • The web UI is fully embedded into the firmware binary, and does not need a filesystem to serve it, making it resilient

The concept is to decouple the user interface (UI) from the supporting API. The UI runs on a web browser and interacts with the device via a RESTful API. Changes on the device are propagated to the UI via notifications over a WebSocket connection, so the UI can request the necessary data when needed, instead of periodically poll the device.

The UI is developed using the Preact framework, in order to fit on very small devices.

The device employs a hybrid server architecture which serves both static and dynamic content. Static files, like CSS/JS/HTML or images, are packed and compiled to an embedded filesystem that is linked into the server binary, in order to not rely on nor require that a filesystem be present on the device.

To make this device more practical, we include an MQTT client and a repeating timer.

The timer simulates periodic measurements of some sensor, these measurements are sent as notifications over the WebSocket connection, so the UI can graph their values.

The MQTT client subscribes to a preconfigured topic, all messages received are sent as notifications over the WebSocket connection, so the UI can show them. The REST API has a send endpoint, so the device can be asked to send a message to a preconfigured topic.

REST API

The RESTful server API provides the following authenticated endpoints:

• /api/hi - returns "hi", does not require authentication and serves as a test point
• /api/config/get - returns the device configuration as a JSON object
• /api/config/set - sets a configuration variable. A variable is urlencoded text in the POST body
• /api/login - returns user name and user token as a JSON object
• /api/message/send - tells device to send an MQTT message. A message is urlencoded text in the POST body

WebSocket API

The WebSocket server API provides the following authenticated WebSocket endpoint:

• /api/watch - receive notifications from a device

Build and try

• Follow the Build Tools tutorial to setup your development environment.
• Start a terminal in the project directory; if you've not already done so, clone the Mongoose Library repo

  git clone https://github.com/cesanta/mongoose
  

• Build the example, this will also start Mongoose:

  cd mongoose/examples/device-dashboard
  make clean all
  

• Start your browser on http://localhost:8000, the main page is loaded but authorization will fail, so you should see a simple login page:

• Login, the JavaScript code on the web page renders the UI and connects to the WebSocket server at /api/watch

• It also GETs the configuration from /api/config/get and shows it, we can simulate that manually by connecting to the REST API with curl, for example:

  curl -u admin:pass0 localhost:8000/api/config/get
  {"url":"mqtt://broker.hivemq.com:1883","pub":"mg/my_device","sub":"mg/#","connected":true}
  

• Now try manually changing the configuration, let's say we want to change the publish topic to "my_mg_device". We can do that using the UI, but we will manually call the REST API, as the UI would do:

  curl -u admin:pass0 localhost:8000/api/config/set -d 'pub=mg/my_mg_device'
  

• The REST API processes the change request, then sends a notification over the WebSocket connection, the JavaScript code on the web page processes this notification, requests the current configuration using the REST API, and updates the UI.

• Now let's change it back to "my_device" using the UI. The UI POSTs to the REST API and the whole process is the same as before. If we want, we can see the notifications by connecting to /api/watch with a WebSocket client like for example wscat:

  wscat --auth user2:pass2 --connect ws://localhost:8000/api/watch
  Connected (press CTRL+C to quit)
  < {"name":"metrics","data":[1655755419, 11]}
  < {"name":"config","data":null}
  

• Now let's send a message to the device via MQTT. We can do that using any MQTT client, for example mosquitto_pub, part of the Mosquitto broker package, by publishing to any topic starting with mg/:

  mosquitto_pub -h broker.hivemq.com -t "mg/to_my_device" -m "hello"
  

  If for some reason you don't have an MQTT client at hand, you can use HiveMQ's WebSocket client

• Since the device is subscribed to any topic starting with mg/, the broker will publish our message to the device, that will send a notification over the WebSocket connection. The JavaScript code on the web page processes this notification, gets the message, and shows it in the UI. We can also see that in wscat, as before:

  wscat --auth user2:pass2 --connect ws://localhost:8000/api/watch
  Connected (press CTRL+C to quit)
  < {"name":"metrics","data":[1655755913, 16]}
  < {"name":"message","data":{"topic": "mg/to_my_device", "data": "hello", "qos": 0}}
  

• Now let's publish a message from the device. We can do that using the UI, but we will manually call the REST API, as the UI would do:

  curl -u user1:pass1 localhost:8000/api/message/send -d 'message=test msg'
  

• The REST API processes the request, then publishes the message to the broker using the configured mg/my_device topic. Since the device is subscribed to any topic starting with mg/, the broker will publish that message back to the device, that will send a notification over the WebSocket connection. Again, the JavaScript code on the web page processes this notification, gets the message from the notification, and shows it in the UI.

• Now send another message using the UI. The UI POSTs to the REST API and the whole process is the same as before.

NOTE: As we are using a public broker, and Mongoose is a widely tried product, you may see messages from other users and demos while trying these.

Build with TLS support

Check the "How to build" section of the TLS tutorial for specific information on building options for your OS

Backend implementation

Let's now dive into the internals of how to build a device dashboard like this one. We'll start from the server backend, running on the device itself.

UI files and embedded filesystem

When you connect your browser to http://localhost:8000, it will ask to GET the index file, which contains references to JavaScript files, including the Preact code, and images. All these are static files and are served by the static server section of our hybrid server architecture. The event handler function calls the mg_http_serve_dir() function when it receives an MG_EV_HTTP_MSG event for a non-API URI:

void device_dashboard_fn(struct mg_connection *c, int ev, void *ev_data,
                         void *fn_data) {
  ...                         
  if (ev == MG_EV_HTTP_MSG) {
    if (mg_http_match_uri(hm, "/api/#")) {
    ...
    } else {
      struct mg_http_serve_opts opts = {...};
      mg_http_serve_dir(c, ev_data, &opts);
    }
  }
}

For this example we've chosen to embed all the web files in a packed filesystem, more information on the embedded filesystem tutorial

Auth and login

The JavaScript code will try to GET /api/login, which requires authorization. If it fails, it will present the login screen. Once you enter your credentials, it will GET /api/login again but this time using the authorization headers. If user and password are valid, the server will return a token that can be used for further accesses.

The hybrid server handles authorization by checking for a valid user and denying all /api/ URIs in case there is not.

User credentials are extracted from the headers using the mg_http_creds() function. This code snippet shows a simple way of doing user authentication, valid for the purpose of this example:

RESTful API

The RESTful server makes use of mg_http_match_uri() for URI matching, and mg_http_reply() to generate responses. Here, the %Q extension simplifies printing JSON strings. Updating the configuration requires the admin user.

Then, the desired values are extracted from the POST body using the mg_http_get_var() function:

If receiving a JSON object is preferred, take a look at the JSON-RPC tutorial for details on how to use Mongoose's built-in JSON and RPC APIs.

The message API endpoint also uses mg_http_get_var() to extract the message from the POST body, and publishes to the MQTT broker using mg_mqttp_pub()

For more information on how to implement an MQTT client, check its tutorial.

Notification API

Requests to /api/watch are upgraded to a WebSocket connection using the mg_ws_upgrade() function. For more information on WebSocket clients, check the corresponding tutorial.

When an event requires a notification to be sent, the following function is called:

It traverses all marked valid WebSocket connections sending a JSON message constructed using the mg_ws_vprintf() function; very similar to a regular printf() function but able to print to a WebSocket connection and handling a variable argument list.

Notification messages have the following format: {"name": "..", "data": ...}, where: possible name values are:

• metrics for (simulated) sensor data, data will contain an array [timestamp, sample value]
  • E.g.: {"name":"metrics","data":[1655755419, 11]}
• message for received messages, data will contain the topic and message strings, plus the message QoS level
  • E.g.: {"name":"message","data":{"topic": "mg/my_device", "data": "a message", "qos": 0}}
• config for config change notifications, data will be null
  • E.g.: {"name":"config","data":null}

Each section calling this function, though, will be able to shape its own messages by using variable arguments, as we'll see.

MQTT client

The MQTT client subscribes to the desired topic using the mg_mqtt_sub() function and sends a notification when it receives a message, using the send_notification function we've seen above.

It also uses a timer to keep the connection alive. The timer event handler function creates an MQTT client connection s_mqtt if it is NULL; by calling the mg_mqtt_connect() function:

Check the MQTT client tutorial for a more in deep description on how to implement these functionalities.

main() and init

The main() function maintains a structure similar to that of an HTTP server, in which we just initialize an event manager and start the event loop, as usual.

The timer that will start and maintain the MQTT client connection is initialized by the server event handler at creation time, that is, when the HTTP server is initialized the handler function will receive an MG_EV_OPEN event:

  if (ev == MG_EV_OPEN && c->is_listening) {
    mg_timer_add(c->mgr, 1000, MG_TIMER_REPEAT, timer_mqtt_fn, c->mgr);

Simulating sensor data

This is done with another repeating timer, initialized along with the one handling the MQTT client connection:

    mg_timer_add(c->mgr, 1000, MG_TIMER_REPEAT, timer_metrics_fn, c->mgr);

Its event handler function simply generates a random sample value and sends it, using the send_notification function we've seen above, to all WebSocket connections:

For more information on timers, check the timers tutorial.

Frontend implementation

This is an integration tutorial, for more details on some of the techniques used here consult the tutorials in the Web UI section, or those referenced further on in this text.

Overview

The UI is developed using the Preact framework, an alternative to React with the same modern API and a smaller footprint, ideal to fit on small devices with limited room for flash memory. Once served via the HTTP server, it will run on the browser.

The approach used here is to decouple the different subsections using a Publish/Subscribe architecture. This is done with an ad hoc simple broker, where all subscribed functions will receive all published messages, and they'll filter themselves those of interest for them.

An event handler function is initialized that keeps a connection to the WebSocket URI /api/watch always open. Any JSON message arriving is parsed and published to all subscribed handlers:

The chart handler subscribes to the ad hoc broker and filters the metrics messages, that contain the (simulated) sensor data and timestamp. It will then add that data to the chart:

The messages handler subscribes to the ad hoc broker and filters the message messages in order to show them on screen. If a message is introduced, it POSTs to the /api/message/send URI:

The initialization section subscribes to the ad hoc broker and filters the config messages in order to fire the function getconfig() when those are received:

    PubSub.subscribe(msg => msg.name == 'config' && getconfig());

This function will in turn fetch the new configuration by GETing the /api/config/get URI:

The configuration section only renders if the user is entitled to make changes on it

If the configuration is locally changed, its handler will POST to the /api/config/set URI the changed element:

For more detailed information on working with Preact, check the Preact-based UI tutorial

Login process

• The initialization section tries to GET the /api/login URI with no authorization headers; if it fails, it will render the login screen

• When credentials are entered at the login screen, the following GET will use proper authorization headers

• Once the REST API grants access and returns the user token; this token will be utilized for all remaning API transactions. A logout simply clears the token

For more detailed information on the login process, check the Web UI login tutorial

Browse latest code