HTTP server

Overview

This tutorial will show you how to configure a HTTP server, while you get familiar with the event manager and the server API.

We'll first cover the basics on HTTP, so feel free to skip ahead if you already handle that.

HTTP basics

The HTTP protocol is a simple request/response protocol. A HTTP client sends a request to a HTTP server, and the HTTP server replies. Each request/response transaction is stateless, i.e. it is completely self-contained and does not depend on previous request/response transactions. Usually, the HTTP client is a browser, and the HTTP server is a piece of software that runs on a workstation, a cloud server - or on an embedded device with Mongoose library!

Both HTTP request and HTTP response have a simple format:

a request line (for requests) or a status line (for responses)
- GET / HTTP/1.0\r\n - a request line
- HTTP/1.0 200 OK\r\n - a status line
zero or more HTTP headers. Each header has a NAME: VALUE\r\n format. Headers are terminated by an empty line, \r\n
HTTP body, which could be empty

Example HTTP transaction:

Data that is returned by a server could be just the contents of a directory on the server's filesystem. In this case, the server is called a "static server". Or, the data could instead be generated by the server "on the fly" - in which case the content is called "dynamic", and such a server is often architectured as a RESTful API server that returns data in JSON format. A hybrid approach is also common - for example, a device Web UI could have a static server with HTML, CSS, JavaScript and media files, as well as a dynamic RESTful API server that returns internal device state.

A minimal static server

Below is a minimal HTTP server that serves files from a given directory, taken straight from the user guide. It is pretty straightforward: we initialise an event manager, create one listening connection that listens on localhost:8000, and start an infinite event loop. An event handler function, fn, simply calls the mg_http_serve_dir() function which serves static content from a given directory:

#include "mongoose.h"

static void fn(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
  struct mg_http_serve_opts opts = {.root_dir = "."};   // Serve local dir
  if (ev == MG_EV_HTTP_MSG) mg_http_serve_dir(c, ev_data, &opts);
}

int main(int argc, char *argv[]) {
  struct mg_mgr mgr;
  mg_mgr_init(&mgr);                                        // Init manager
  mg_http_listen(&mgr, "http://localhost:8000", fn, &mgr);  // Setup listener
  for (;;) mg_mgr_poll(&mgr, 1000);                         // Event loop
  mg_mgr_free(&mgr);                                        // Cleanup
  return 0;
}

NOTE: If you use a relative path for your root directory, make sure to avoid double-dots, ... If you need to reference an upper-level directory, use an absolute path.

You can check examples/http-server for a more verbose version of this minimal example, it also follows some advice on signal handling we'll see below

If you've not already done so, clone the Mongoose Library repo
```
$ git clone https://github.com/cesanta/mongoose
```

Build and run the example

$ cd mongoose/examples/http-server
$ make clean all

Go to http://localhost:8000 in your browser

Dynamic RESTful server

A dynamic server would be very similar to a static one - the difference would be in the function that outputs data. Instead of mg_http_serve_dir(), we're going to use mg_http_reply(). Using mg_http_reply(), we can generate responses with specific headers and body. This snippet replies with a plain-text "Hello, world" on any URI:

static void fn(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
  if (ev == MG_EV_HTTP_MSG) {
    mg_http_reply(c, 200, "Content-Type: text/plain\r\n", "Hello, %s\n", "world");
  }
}

That is not a very useful server. Let's modify it to become a RESTful API server that can add two numbers on the URI /api/sum, and respond with an error on any other URI. For this, we must get more information about the request. First, we cast the function argument ev_data to a struct mg_http_message, which contains a parsed HTTP request. Then, we check the URI by calling mg_http_match_uri(). If no match, we return an error. If it matches, we assume that the client has sent us the numbers to add as a JSON array in the POST body, like [ 12.2, 4.7 ]. We fetch them using Mongoose's built-in JSON API, calculate the sum, and respond with a result:

static void fn(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
  if (ev == MG_EV_HTTP_MSG) {
    struct mg_http_message *hm = (struct mg_http_message *) ev_data;
    if (mg_http_match_uri(hm, "/api/sum")) {
      // Attempt to fetch parameters from the body, hm->body
      struct mg_str params = hm->body;
      double num1, num2;
      if (mg_json_get_num(params, "$[0]", &num1) &&
          mg_json_get_num(params, "$[1]", &num2)) {
        // Success! create a JSON response
        mg_http_reply(c, 200, "Content-Type: application/json\r\n", "{%Q:%g}\n",
                      "result", num1 + num2);
      } else {
        mg_http_reply(c, 500, NULL, "%s", "Parameters missing");
      }
    } else {
      mg_http_reply(c, 500, NULL, "%s", "Invalid URI");
    }
  }

Check the REST basics example to try this code with a user interface; and its tutorial for a more in deep explanation. Check the JSON-RPC tutorial for a much more involved implementation example.

Hybrid server

The best way to implement a Web UI for an application or device is to integrate a hybrid web server in it. The user's browser downloads the client part, which in turn makes RESTful requests and renders the UI on the browser:

In this case, the event handler function is going to handle certain URIs as dynamic, and for the rest of the URIs, just call mg_http_serve_dir(). Here's an example on how to implement such an event handler function:

You can check examples/http-restful-server for an expanded version of this minimal example, it also makes use of the chunked transfer encoding method we'll introduce below.

If you've not already done so, clone the Mongoose Library repo
```
$ git clone https://github.com/cesanta/mongoose
```

Build and run the example

$ cd mongoose/examples/http-restful-server
$ make clean all

Go to http://localhost:8000 in your browser to see directory contents
Go to http://localhost:8000/api/f2/this in your browser to see an example of an API URI
Go to http://localhost:8000/api/stats in your browser to see chunk-encoded dynamically generated statistics

How to get request data

As it was mentioned above, when Mongoose calls the user's event handler function with the MG_EV_HTTP_MSG event, the void *ev_data pointer points to a parsed HTTP message, struct mg_http_message:

static void fn(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
  if (ev == MG_EV_HTTP_MSG) {
    struct mg_http_message *hm = (struct mg_http_message *) ev_data;
    ...

This diagram below shows how different parts of the HTTP message map to that structure:

From that diagram it should be clear how to access any part of a HTTP request (or response, in case of a HTTP client). For example, this code snippet echoes back the requested URI: (in case of HTTP client - protocol, status code, and message):

static void fn(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
  if (ev == MG_EV_HTTP_MSG) {
    struct mg_http_message *hm = (struct mg_http_message *) ev_data;
    mg_http_reply(c, 200, NULL, "URI: %.*s", (int) hm->uri.len, hm->uri.ptr);
  }
}

How to iterate and send data

The following snippet echoes back all HTTP headers we described in this section above. Since we iterate over headers, we cannot use mg_http_reply(). We must send the HTTP response line and the headers first, and then append to the generated response body several times. In cases like this, it is best to use chunked transfer encoding, and we'll do that using mg_http_printf_chunk():

static void fn(struct mg_connection *c, int ev, void *ev_data, void *fn_data) {
  if (ev == MG_EV_HTTP_MSG) {
    struct mg_http_message *hm = (struct mg_http_message *) ev_data;
    size_t i, max = sizeof(hm->headers) / sizeof(hm->headers[0]);
    mg_printf(c, "%s%s%s\r\n",
              "HTTP/1.1 200 OK\r\n"               // Output response line
              "Transfer-Encoding: chunked\r\n";   // Chunked header
              "Content-Type: text/plain\r\n");    // and Content-Type header
    mg_printf(c, "Request headers:\n");
    // Iterate over request headers, and print them one by one
    for (i = 0; i < max && hm->headers[i].name.len > 0; i++) {
      struct mg_str *k = &hm->headers[i].name, *v = &hm->headers[i].value;
      mg_http_printf_chunk(c, "%.*s -> %.*s\n", (int) k->len, k->ptr, (int) v->len, v->ptr);
    }
    mg_http_write_chunk(c, "", 0);  // Final empty chunk
  }
}

Using mg_printf()

In the examples above, we have used the mg_printf() function to create a response. Note that mg_printf() is not HTTP specific; it is a generic function. It simply prints data to the connection's output buffer according to a printf()-like format specification.

Using mg_printf(), it is possible to create any custom response - in fact, the mg_http_reply() function is just a convenient wrapper around mg_printf() that adds a correct status message and a Content-Length header.

Things to remember

Set content length

Always set content length! Otherwise, your peer assumes that you'll stop sending data when you'll close the connection - so it will look like your peer receives your data after a huge delay (actually, when the connection times out). Therefore, either:

Use the mg_http_reply() function, which sets the Content-Length header automatically, or
Use chunked transfer encoding like shown in a previous section, or
If you're crafting your response manually, e.g. by using mg_printf(), don't forget to set the Content-Length header. Here is an illustration:

// Create JSON response using mjson library call
char *json = mjson_aprintf("{%Q: [%d, %g]}", "data", 123, 3.1415);

// Send response to the client
mg_printf(c, "%s", "HTTP/1.0 200 OK\r\n");                  // Response line
mg_printf(c, "%s", "Content-Type: application/json\r\n");   // One header
mg_printf(c, "Content-Length: %d\r\n", (int) strlen(json)); // Another header
mg_printf(c, "%s", "\r\n");                                 // End of headers
mg_printf(c, "%s", json);                                   // Body
free(json);                                                 // Don't forget!

Handle signals

This applies to UNIX or UNIX-like systems like MacOS. Especially, ignore the SIGPIPE signal to prevent accidental killing of your server when some client just abruptly closes a connection, and handle SIGINT to make your server behave well in container environments like Docker.

Add a signal handler function:

In the main() function, assign our signal handler function to handle SIGINT and SIGTERM signals:

Then, make the main event loop exit when a signal arrives:

Command line flags

Use command line flags to tune your server. For that, we declare change-able parameters as static variables:

And add a command line parsing snippet:

Embedded optimisation

Below is a short description of the typical request flow when a browser is used to show a web page. It is important to understand in order to optimise resource usage, especially on embedded systems. So, when a browser fetches a web page, it does it in following steps:

A browser fetches a page itself - e.g. index.html
Fetches a page icon, to display in a browser tab. If index.html has an explicit <link rel="icon" href="..." type="image/x-icon" /> then it fetches it, otherwise it fetches /favicon.ico
Then for CSS/JS/image local references present in the index.html, browsers start a pool of 4-5 connections to fetch those resources as fast as possible. In turn, those JS/CSS might reference other local resources, which get fetched sequentially using that pool of extra connections.

Therefore, a browser page request seldom come alone. Usually it is 4-5 connections, fetching many resources that are referenced by a page. For small embedded systems that could be a problem, especially if TLS is used. That burst of connections can easily take all RAM and kill a device.

To alleviate a multiple connections issue, you can amalgamate your page into once single page with no local references: just inline JS and CSS, and images too. You can make all images SVG and inline them, or make them PNG and base64-inline them too. This way, you can get away with a single HTTP connection.

In order to prevent a browser creating a connection for every new request (e.g. for POSTing new configuration), make sure to use HTTP/1.1 in all responses, and set up Content-Length properly (alternatively, use Transfer-Encoding: chunked). This way, your browser will know the boundary of the response and will re-use an existing connection for the next request.

Consider browser caching for static resources: add a "Etag" header for static content, and handle "If-None-Match" header. This way, you can simply response with 304 Not Modified on repeated requests to static resources.