For file uploading, default behaviour will `ReadForm` and write data to new temporary file if oversize `maxMemory`.
If temporary directory size is too small, it will fail for uploading large file and even consume OS memory since copy to buffer.
Inspired by Flask, Flask `request.stream.read` will read underlying socket data directly if payload found. Refer to: https://werkzeug.palletsprojects.com/en/2.0.x/wsgi/#werkzeug.wsgi.LimitedStream
This method makes assumptions because the request payload exists for operating `POST/PATCH/PUT`.
A simple demo for uploading 5G large file in low memory based on docker:
```
[root@control-master tmp]# docker run -ti --rm --memory 1G --tmpfs /tmp:rw,size=1G,mode=1777 -v $(pwd)/go:/go golang:1.16.5 /bin/bash
root@b672a98e5314:/go/testmodules# cat main.go
package main
import (
"fmt"
"github.com/gin-gonic/gin"
"io"
"os"
"path/filepath"
)
func main() {
engine := gin.Default()
engine.POST("/streamupload", func(c *gin.Context) {
if c.GetHeader("Content-Type") != "application/octet-stream" {
err := fmt.Errorf("required octet-stream")
c.AbortWithStatusJSON(400, map[string]string{"message": err.Error()})
return
}
info, err := os.Create(filepath.Join("/home", "foo"))
if err != nil {
c.AbortWithStatusJSON(400, gin.H{"message": "Create: "+err.Error()})
return
}
defer info.Close()
_, err = io.Copy(info, c.Request.Body)
if err != nil {
c.AbortWithStatusJSON(400, gin.H{"message": "Copy: "+err.Error()})
return
}
c.JSON(200, map[string]string{"message": "ok stream"})
})
if err := engine.Run("0.0.0.0:19090"); err != nil {
panic(err)
}
}
root@b53810b3e294:/go/testmodules# GOTMPDIR=/opt/ go run main.go
[root@control-master ~]# dd if=/dev/zero of=5G bs=1M count=5170 status=progress
[root@control-master ~]# curl -vvv -H "Content-Type:application/octet-stream" -T 5G -X POST 172.17.0.2:19090/streamupload
```
Signed-off-by: Chenyang Yan <memory.yancy@gmail.com>
* Update tree.go (#2659)
delete more "()"
* updated comments for Get function for params (#2756)
* ci: add github action workflows (#2596)
* ci: add github action workflows
* test: fixed the TestUnixSocket test on windows (#20)
* ci: add github action workflows (#18)
* Remove .travis.yml
* ci: replace GITTER_ROOM_ID and upload coverage every time you go test
* ci: update coverage using codecov/codecov-action@v1
* Merge branch 'master' into github-actions
* repo: replace travis ci to github actions
* ci: add go version 1.16
* fix: go install requires a specific version
* chore(ci): remove go 1.12 support
* chore(ci): remove os windows-latest
Co-authored-by: thinkerou <thinkerou@gmail.com>
Co-authored-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* Setting trusted platform using an enum-like (#2739)
* gin.Context with fallback value from c.Request.Context()
* add test case
Co-authored-by: youzeliang <youzel@126.com>
Co-authored-by: Ashwani <ashwanisharma686@gmail.com>
Co-authored-by: Jeff <laojianzi1994@gmail.com>
Co-authored-by: thinkerou <thinkerou@gmail.com>
Co-authored-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
