Overview

This tutorial will show you how to upload a file to a Mongoose web server.

There are two possible scenarios:

  1. The file to be uploaded is small - significantly smaller than the amount of free RAM. For example, we're running Mongoose on an embedded Linux system with 64MB of RAM, and uploading some JSON configuration about 1KB in size. In this case, we can use a standard HTML form upload, and receive the whole file in one POST request.
  2. The file to be uploaded is large - compared to the amount of free RAM, or significantly more than that. For example, we want to upload a new filesystem image of size 512KB to an ESP8266 device which has about 30KB of free RAM. In this case, there is no way to hold the whole file in memory. It should be handled in small chunks, and Mongoose should receive each chunk and append it to the written file, until everything is received. Here we could follow three paths:
    • Stream the file, Mongoose handles partial HTTP requests as they are handled by the transport layer
    • Stream the file using chunked transfer encoding (not available in every browser), Mongoose processes every chunk
    • Send the file using multiple POSTS, Mongoose appends them

Form upload method

This method uses a standard HTML upload form, and works when file size is significantly less than free RAM.

  • If you've not already done so, clone the Mongoose Library repo
    $ git clone https://github.com/cesanta/mongoose
    
  • Build and run the form upload example
    $ cd mongoose/examples/file-upload-html-form
    $ make clean all
    
  • Go to http://localhost:8000 in your browser
  • Click on "Browse...", select a file to upload
  • Click on "submit form". You should see a "Thank you!" message
  • Check the web server logs:
    2021-11-05 02:02:22 2 main.c:22:cb    New request to: [/upload], body size: 1991
    2021-11-05 02:02:22 2 main.c:28:cb    Chunk name: [field1] filename: [] length: 19 bytes
    2021-11-05 02:02:22 2 main.c:28:cb    Chunk name: [file1] filename: [hello.txt] length: 1698 bytes
    

Let's see how that worked.

  • The web server serves static content from the web_root folder, which shows an upload form:

  • When a file is selected, and the "submit form" button gets hit, the browser sends a request like this, where every form field is wrapped between "boundary markers":

    POST /upload HTTP/1.1
    ...other headers...
    Content-Type: multipart/form-data; boundary=----MyBound123
    
    ------MyBound123
    Content-Disposition: form-data; name="field1"
    
    type some text here
    ------MyBound123
    Content-Disposition: form-data; name="file1"; filename="hello.txt"
    Content-Type: text/x-chdr
    
    ... CONTENTS OF THE FILE hello.txt ...
    ------MyBound123--
    
  • Mongoose receives that request, fully buffers it in memory, and then iterates over every form field using the mg_http_next_multipart() function:

  • There, the code simply logs everything. Alternatively, we could save an uploaded file, or write to a flash region, etcetera.

Browse latest code

Streaming method

Mongoose always buffers a full HTTP message before invoking the MG_EV_HTTP_MSG event. A big POST request would require a lot of RAM to buffer everything. In order to upload large files to a memory-constrained system, use MG_EV_HTTP_CHUNK instead. This event fires when a partial HTTP message or a chunk-encoded chunk has been received. Use mg_http_delete_chunk() to release chunk memory. Extra information can be passed over via query string parameters.

When a 0-sized chunk is received, that's the end of the message. Use the MG_MAX_RECV_BUF_SIZE build constant to limit the maximum chunk size on the server side.

  • If you've not already done so, clone the Mongoose Library repo
    $ git clone https://github.com/cesanta/mongoose
    
  • Build and run the single POST example
    $ cd mongoose/examples/file-upload-single-post
    $ make clean all
    
  • Go to http://localhost:8000 in your browser
  • Click on "choose file...", select a file to upload
  • You should see a result upload message showing the file size
  • Check the web server logs, you should see each chunk details printed:
    18149d3d440 2 main.c:17:cb              Got chunk len 1448
    18149d3d440 2 main.c:18:cb              Query string: [name=myfile]
    
    To have the file contents printed, just remove the comments in the example
  • You can also try using curl:
    $ curl -H Expect: --data-binary @filename http://localhost:8000/upload?name=myfile
    ok (chunked)
    
  • In particular, we can instruct curl to use chunked transfer encoding and it will send the file in chunks:
    $ curl -H Expect: -H "Transfer-Encoding: chunked" --data-binary @filename http://localhost:8000/upload?name=myfile
    ok (chunked)
    
    In this case, curl used 16372-byte chunks:
    18149d445c0 2 main.c:17:cb              Got chunk len 16372
    18149d445c0 2 main.c:18:cb              Query string: [name=myfile]
    

Let's see how that worked.

  • The web server serves static content from the web_root folder, which shows an upload form:

  • When a file is selected, the browser runs a JS snippet that sends it over to the server:

  • The server receives each chunk, processes it and deletes it. When the last chunk is received, sends a response.

  • If a file is small and fits into a single HTTP message, the server uses MG_EV_HTTP_MSG instead. Note that extra parameters, like file name, can be passed over using the query string parameters:

Browse latest code

Multiple POSTs method

Another way to upload a big file is to split the transfer into several POSTs.

The function mg_http_upload() can parse a POST request with extra information in the query string parameters.

  • If you've not already done so, clone the Mongoose Library repo
    $ git clone https://github.com/cesanta/mongoose
    
  • Build and run the multiple POSTs example
    $ cd mongoose/examples/file-upload-multiple-posts
    $ make clean all
    
  • Go to http://localhost:8000 in your browser
  • Click on "choose file...", select a file to upload
  • You should see a file upload progress indication
  • Check your file system, your file should be there

Let's see how that worked.

  • The web server serves static content from the web_root folder, which shows an upload form:

  • When a file is selected, the browser runs a JS snippet that splits the file content and sends it over in several POSTs, each 4KB in length. In the query string, we send the file name and the offset within the file as parameters:

  • The server receives each POST request and handles it to mg_http_upload(), that appends the contents to the specified file at the specified offset

Browse latest code