elek · GitHub

Custom metadata is not saved when S3 MPU is used (in this example the threshold is 8MB)

As you can see in the second case the metadata is empty, but with real AWS it's foo: bar in both cases.

> aws s3 cp --endpoint http://192.168.80.4:7777 --metadata foo=bar 1mb s3://bucket1/1mb
upload: ./1mb to s3://bucket1/1mb
 > aws s3 cp --endpoint http://192.168.80.4:7777 --metadata foo=bar 20mb s3://bucket1/20mb
upload: ./20mb to s3://bucket1/20mb
> aws s3api get-object --endpoint http://192.168.80.4:7777 --bucket bucket1 --key 1mb x
{
    "AcceptRanges": "bytes",
    "LastModified": "Thu, 30 Sep 2021 11:36:10 GMT",
    "ContentLength": 1048576,
    "ETag": "\"f5f16eee8ad5a86a3439dc75429831ee\"",
    "ContentType": "application/octet-stream",
    "Metadata": {
        "foo": "bar"
    }
}
> aws s3api get-object --endpoint http://192.168.80.4:7777 --bucket bucket1 --key 20mb x
{
    "AcceptRanges": "bytes",
    "LastModified": "Thu, 30 Sep 2021 11:36:21 GMT",
    "ContentLength": 20971520,
    "ETag": "\"80f096bdc4c7fd9639307cd1174b398a-3\"",
    "ContentType": "application/octet-stream",
    "Metadata": {}
}

Credit:

The problem is identified by the rclone s3 unit tests during the conversation in rclone/rclone#5616 where @olefrost suggested to re-run s3 unit tests with storj gateway-mt and @ncw drew attention to this problem from many failures.

Read the original on github.com ↗