File Uploads

Overview

This tutorial will show you how to upload a file to a Mongoose web server.

Build and run

Follow the Build Tools tutorial to setup your development environment. Then start a command prompt / terminal and enter the following:

git clone https://github.com/cesanta/mongoose
cd mongoose/tutorials/http/file-upload-html-form
make

Possible approaches

There are two possible scenarios:

1. The file to be uploaded is small - significantly smaller than the amount of free RAM. For example, we're running Mongoose on an embedded Linux system with 64MB of RAM, and uploading some JSON configuration about 1KB in size. In this case, we can use a standard HTML form upload, and receive the whole file in one POST request.
2. The file to be uploaded is large - compared to the amount of free RAM, or significantly more than that. For example, we want to upload a new filesystem image of size 512KB to an ESP8266 device which has about 30KB of free RAM. In this case, there is no way to hold the whole file in memory. It should be handled in small chunks, and Mongoose should receive each chunk and append it to the written file, until everything is received. Here we could follow two paths:
   • Send a file using a single POST request, passing file content as-is as the POST body - i.e. use binary upload
   • Split a file in small chunks (e.g. 2Kb) on a client side, and send each chunk as an individual POST request

Form upload

This method uses a standard HTML upload form, and works when file size is significantly less than free RAM.

• Build and run the form upload example

  cd mongoose/tutorials/http/file-upload-html-form
  make clean all
  

• Go to http://localhost:8000 in your browser
• Click on "Browse...", select a file to upload
• Click on "submit form". You should see a "Thank you!" message
• Check the web server logs:

  2021-11-05 02:02:22 2 main.c:22:cb    New request to: [/upload], body size: 1991
  2021-11-05 02:02:22 2 main.c:28:cb    Chunk name: [field1] filename: [] length: 19 bytes
  2021-11-05 02:02:22 2 main.c:28:cb    Chunk name: [file1] filename: [hello.txt] length: 1698 bytes
  

Let's see how that worked.

• The web server serves static content from the web_root folder, which shows an upload form:

• When a file is selected, and the "submit form" button gets hit, the browser sends a request like this, where every form field is wrapped between "boundary markers":

  POST /upload HTTP/1.1
  ...other headers...
  Content-Type: multipart/form-data; boundary=----MyBound123
  
  ------MyBound123
  Content-Disposition: form-data; name="field1"
  
  type some text here
  ------MyBound123
  Content-Disposition: form-data; name="file1"; filename="hello.txt"
  Content-Type: text/x-chdr
  
  ... CONTENTS OF THE FILE hello.txt ...
  ------MyBound123--
  

• Mongoose receives that request, fully buffers it in memory, and then iterates over every form field using the mg_http_next_multipart() function:

• There, the code simply logs everything. Alternatively, we could save an uploaded file, or write to a flash region, etcetera.

The source code for this example is available at tutorials/http/file-upload-html-form

Binary upload, single POST

When Mongoose receives a large HTTP request, it buffers incoming data until the full HTTP message is received; then an MG_EV_HTTP_MSG event is generated.

To avoid buffering a large binary upload, catch MG_EV_HTTP_HDRS, which is generated after the headers have been parsed, and call mg_http_stream_body(). It streams the request body to a callback as data arrives.

mg_http_stream_body() takes ownership of the connection. It accepts POST and PUT requests with Content-Length; invalid requests receive an HTTP error automatically.

• Build and run the single POST example

  cd mongoose/tutorials/http/file-upload-single-post
  make clean all
  

• Go to http://localhost:8000 in your browser
• Click on "choose file...", select a file to upload
• You should see a result upload message showing the file size
• You can also try using curl:

  curl -H Expect: --data-binary @filename http://localhost:8000/fs/myfile
  

Windows users: the example writes the file to /tmp, create a TMP directory in your C: drive

Let's see how that worked.

• The web server serves static content from the web_root folder, which shows an upload form:

• When a file is selected, the browser runs a JS snippet that sends it over to the server:

• The server catches MG_EV_HTTP_HDRS, matches the upload URI, and starts the stream:

• The streaming callback opens the output file when hm is non-NULL, writes each data chunk, then closes the file when both hm and data are NULL. It returns false to report an error.

The source code for this example is available at tutorials/http/file-upload-single-post

This method does not handle HTTP chunked transfer encoding. Use it when the client sends a POST or PUT with Content-Length; otherwise, use another upload protocol such as multiple POSTs.

There is a full blown example containing both client and server using traffic shaping, available at tutorials/http/file-transfer

Binary upload, multiple POSTs

Another way to upload a big file is to split the transfer into several POSTs.

The function mg_http_upload() can parse a POST request with extra information in the query string parameters.

• Build and run the multiple POSTs example

  cd mongoose/tutorials/http/file-upload-multiple-posts
  make clean all
  

• Go to http://localhost:8000 in your browser
• Click on "choose file...", select a file to upload
• You should see a file upload progress indication
• Check your file system, your file should be there

Windows users: the example writes the file to /tmp, create a TMP directory in your C: drive

Let's see how that worked.

• The web server serves static content from the web_root folder, which shows an upload form:

• When a file is selected, the browser runs a JS snippet that splits the file content and sends it over in several POSTs, each 2KB in length. In the query string, we send the file name and the offset within the file as parameters:

• The server receives each POST request and handles it to mg_http_upload(), that appends the contents to the specified file at the specified offset

  This function, [mg_http_upload()](https://mongoose.ws/documentation/#mg_http_upload), will check the path for sanity and also limit the maximum file size to be below 100KB.

The source code for this example is available at tutorials/http/file-upload-multiple-posts