Overview

Often it is required to uploads files. There are two scenarios possible:

  1. An uploaded file is small - significantly smaller than the amount of free RAM. For example, we're running Mongoose on an embedded Linux system with 64M of RAM, and uploading some JSON configuration about 1k in size. In this case, we can use a standard HTML form upload, and receive the whole file in one POST request.
  2. An uploaded file is large - compared to the amount of free RAM, or significantly more than that. For example, we want to upload a new filsystem image of size 512k to to ESP8266 device which has about 30k free RAM. In this case, there is no way to receive the whole file in memory. A file should be sent by small chunks, and Mongoose should receive each chunk, append it to the written file, until everything is received.

Form upload method

This method uses a standard HTML upload form, and works when file size is significantly less than free RAM.

  • Clone Mongoose Library repo
    $ git clone https://github.com/cesanta/mongoose
    
  • Build and run form upload example
    $ cd mongoose/examples/form-upload
    $ make clean all
    
  • Go to http://localhost:8000 in your browser
  • Click on "Choose file", select file to upload
  • Click on "Upload". You should see a "Thank you" message
  • Check the web server logs:
    2021-11-05 02:02:22 2 main.c:22:cb    New request to: [/upload], body size: 1991
    2021-11-05 02:02:22 2 main.c:28:cb    Chunk name: [field1] filename: [] length: 19 bytes
    2021-11-05 02:02:22 2 main.c:28:cb    Chunk name: [file1] filename: [hello.txt] length: 1698 bytes
    

Let's see how that worked.

  • A web server serves static content from web_root folder, which shows an upload form:

  • When a file is selected, and "Upload" button gets hit, a browser sends a request like this, where every form field is wrapped between "boundary markers":

    POST /upload HTTP/1.1
    ...other headers...
    Content-Type: multipart/form-data; boundary=----MyBound123
    
    ------MyBound123
    Content-Disposition: form-data; name="field1"
    
    type some text here
    ------MyBound123
    Content-Disposition: form-data; name="file1"; filename="hello.txt"
    Content-Type: text/x-chdr
    
    ... CONTENTS OF THE FILE hello.txt ...
    ------MyBound123--
    
  • Mongoose receives that request, fully buffers it in memory, and then iteratest over every form field using mg_http_next_multipart function:

  • There, the code simply logs everything. Alternatively, we could save an uploaded file, or write to a flash region, etcetera.

Streaming method

Mongoose always buffers a full HTTP message before invoking MG_EV_HTTP_MSG event. Big POST request require of lot of RAM to buffer everything. In order to upload large files to a memory-constrained system, use MG_EV_HTTP_CHUNK on a server side instead. It fires when a partial HTTP message has been received (or a chunk-encoded chunk). Use mg_http_delete_chunk() to release chunk memory. Extra information can be passed over via query string parameters.

When 0-sized chunk is received, that's the end of the message. Use MG_MAX_RECV_BUF_SIZE build constant to limit maximum chunk size on a server side.

  • Clone Mongoose Library repo
    $ git clone https://github.com/cesanta/mongoose
    
  • Build and run file upload example
    $ cd mongoose/examples/file-upload
    $ make clean all
    
  • Go to http://localhost:8000 in your browser
  • Click on "Choose file", select file to upload
  • Click on "Upload". You should see a result upload message with file size.
  • Check the web server logs: you should see file contents printed

Let's see how that worked.

  • A web server serves static content from web_root folder, which shows an upload form:

  • When a file is selected, a browser runs a JS snippet that splits file content and sends it over chunk by chunk:

  • Server side receives each chunk, prints it and deletes it. When last chunk is received, server sends a response.

  • If a file is small and fits into a single HTTP message, server uses MG_EV_HTTP_MSG instead and prints message body. Note that extra parameters, like file name, can be passed over the query string parameters: