Web UI: data push

Overview

This tutorial will show you how to push data from the device to a JavaScript-based user interface (UI) running on the browser; either using WebSocket or a REST-based API.

We'll concentrate here on the basics of the data push operation and the interactions with the UI, for other aspects of the development of a UI, please check the rest of the tutorials in the Web UI section.

Now:

• Follow the Build Tools tutorial to setup your development environment.

• Start a terminal in the project directory; if you've not already done so, clone the Mongoose Library repo

  git clone https://github.com/cesanta/mongoose
  

Data push using WebSocket

A WebSocket connection is a lightweight means of pushing data. The client just connects to the server and the latter pushes data over this connection when needed. The client just tries to keep the connection always open.

Build and try

• Build and run the example

  cd mongoose/examples/webui-push-ws
  make clean all
  

• Go to http://localhost:8000 in your browser; the JavaScript code loads and executes, connecting to the device and obtaining some simulated metrics; see the results in the log

• We can do manually what the UI is doing by connecting to /api/watch with a WebSocket client like for example wscat:

  wscat --connect ws://localhost:8000/api/watch
  Connected (press CTRL+C to quit)
  < {"name":"metrics","data":[1655755419, 11]}
  

Backend implementation

• When we get an HTTP request for the /api/watch URI, we upgrade the connection to WebSocket using a function call to mg_ws_upgrade(), and place a unique mark in it.

• When there is data to be pushed, we traverse our list of connections and print a response on those that have our unique mark on them. For that task, we use the function mg_ws_printf(), taking advantage of the identifier m and, through the macro MG_ESC(), function mg_print_esc(), to print double-quoted JSON-escaped strings.

For this demo, we simulate periodic data updates using a timer, check the timers tutorial for more information.

Frontend implementation

• The JavaScript code in the browser extracts the host URL and builds a WebSocket URL, in order to call our reconnect function

• This function then uses the browser's WebSocket API to connect to /api/watch. Data that is received is logged, and in the event of a disconnection, the function sets a timer to call itself and retry after a while

Data push using a REST API

Resource economy is implemented by versioning. Every time the data at the server is modified, its version will change. The client indicates in its request which version it has, and the server responds with either a short message indicating there are no changes, or a full message sending the new data and its version number.

A similar strategy can be used to handle huge responses on limited memory devices, check the tutorial for more information.

Build and try

• Build and run the example

  cd mongoose/examples/webui-push-rest
  make clean all
  

• Go to http://localhost:8000 in your browser; the JavaScript code loads and executes, periodically connecting to the device and querying for changes. When there is new data, the UI is updated; see the results in the log

• We can do manually what the UI is doing by connecting to /api/data with curl, and sending a version number:

  curl http://localhost:8000/api/data -d '{"version":0}'
  {"version":4,"data":[717370819,331448901,1470572844,832676375,1181518,1339528563,1082025075,656626282,2141179440,902560226]}
  curl http://localhost:8000/api/data -d '{"version":4}'
  {"status":"no change", "version":4}
  

Backend implementation

• When we get an HTTP request for the /api/data URI, we extract the version parameter from the JSON object. If the requested version is the same as our current version, our returned JSON object will contain a status indication telling that nothing has changed. Otherwise, it will contain the new data. We serve responses using mg_http_reply(). Here we use two convenient extensions to the syntax.
  • The specifier M is used to call a function, passing it the remaining parameter list. This function must return the number of bytes it has printed
  • The specifier m does the same, but also double-quotes what it prints. Here we use it to print JSON-escaped strings by calling mg_print_esc() through the macro MG_ESC().

• The function getparam() makes use of the function mg_json_get_num(), that parses a JSON string in a struct mg_str, searching for a valid numeric value at the JSON path json_path. If successful, it will store its value at placeholder dv and return true; returning false otherwise.

• Using a function call inside the print function allows us to have a clean code showing the message being sent, and handle the details of how much data is sent on a different function. The function been called will receive an output function pointer and its parameter, plus a variable argument list containing the remaining parameters; in our case, none. Inside this function, we'll use mg_xprintf(). It takes the very same parameters we received, prints where it has been told to do it, and returns how many bytes it printed.

Note that versions 7.7 and earlier used a different API for %M

For this demo, we simulate periodic data updates using a timer, check the timers tutorial for more information.

Frontend implementation

The JavaScript code in the browser gets the elements that are present on the page, then calls a function that calls the fetch() method to POST its current version to /api/data as a stringified JSON object; then extracts the returned JSON object. If that object contains a status key, then data version has not changed. Otherwise, the response contains new data so it updates the local copy of the version number and refreshes the UI. Then, the function sets a timer to call itself and repeat the operation after a while

Pushing live data on HTTP connections

If you need to send live data, like for example a live log, you can use chunked transfer encoding.

Build and try

• Build and run the example

  cd mongoose/examples/live-log
  make clean all
  

• Go to http://localhost:8000 in your browser; the JavaScript code loads and executes, connecting to the device. The left side will show the whole log; the right side will update as the device pushes new data using chunked transfer encoding.

• We can connect manually to /api/log/live with curl, and see the regular updates:

  curl http://localhost:8000/api/log/live
  Time is: 1658424026
  Time is: 1658424027
  

Backend implementation

• Reserve an API endpoint to serve live data; and mark connections to this endpoint:

  static void fn(struct mg_connection *c, int ev, void *ev_data) {
    ...
  

• Then, every time you need to update live data, traverse all marked connections and send data using chunked transfer encoding by calling mg_http_printf_chunk():

For this demo, we simulate periodic updates using a timer, check the timers tutorial for more information.

Frontend implementation

The JavaScript code in the browser calls the fetch() method to GET /api/log/static and extracts the text, showing it on the left side of the window. Then calls the fetch() method to GET /api/log/live and invokes the getReader() method on the response, obtaining a readable stream interface on which get regular updates. It then calls our function f() that will read on that stream and append read data to the right side of the window, calling itself again as long as the server keeps sending chunks.