snapshot of https://github.com/seaweedfs/seaweedfs.git (master, 2026-09-16; blobs > 131072B omitted)
Some checks failed
go: build dev binaries / cleanup (push) Successful in 4s
Codespell / Check for spelling errors (push) Failing after 3s
docker: build dev containers / build-rust-binaries (amd64, x86_64-unknown-linux-musl) (push) Failing after 3s
docker: build dev containers / build-dev-containers (push) Has been skipped
docker: build dev containers / build-rust-binaries (arm64, true, aarch64-unknown-linux-musl) (push) Failing after 3s
End to End / FUSE Mount (push) Failing after 2s
EC Integration Tests / EC Integration Tests (push) Failing after 30s
FUSE DLM Integration Tests / FUSE DLM Integration Tests (push) Failing after 3s
FUSE Volume Server Failover Tests / FUSE Volume Server Failover (push) Failing after 3s
FUSE Integration Tests / FUSE Integration Testing (push) Failing after 2s
FUSE P2P Peer Chunk Sharing Integration Tests / FUSE P2P Peer Chunk Sharing (push) Failing after 3s
go: build binary / Go Vet (push) Failing after 2s
go: build binary / Go Vet 32-bit (push) Failing after 3s
go: build binary / Build (push) Failing after 4s
go: build binary / Build cross-platform (freebsd/amd64) (push) Failing after 3s
go: build binary / Build cross-platform (windows/amd64) (push) Failing after 3s
go: build binary / Build cross-platform (darwin/arm64) (push) Failing after 3s
go: build binary / Build cross-platform (windows/arm64) (push) Failing after 3s
go: build binary / Test (push) Failing after 2s
go: build binary / Test 32-bit (push) Failing after 3s
helm: lint and test charts / lint-test (push) Failing after 3s
Java Client Integration Tests / Java Integration Tests (11) (push) Failing after 3s
Java Client Integration Tests / Java Integration Tests (17) (push) Failing after 3s
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Failing after 2s
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Failing after 3s
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been skipped
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been skipped
Java Client Unit Tests / Java Unit Tests (11) (push) Failing after 3s
Java Client Unit Tests / Java Unit Tests (17) (push) Failing after 2s
Java Client Unit Tests / Java Unit Tests (21) (push) Failing after 3s
Java Client Unit Tests / Java Unit Tests (8) (push) Failing after 3s
Kafka Quick Test (Load Test with Schema Registry) / Kafka Client Load Test (Quick) (push) Failing after 3s
Kafka Gateway Tests / Kafka Unit Tests (unit-tests-1) (push) Failing after 11s
Kafka Gateway Tests / Kafka Integration Tests (Critical) (integration-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka End-to-End Tests (with SMQ) (e2e-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka Consumer Group Tests (Highly Isolated) (consumer-group-1) (push) Failing after 2s
Kafka Gateway Tests / Kafka Client Compatibility (with SMQ) (client-compat-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka SMQ Integration Tests (Full Stack) (smq-integration-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka Protocol Tests (Isolated) (protocol-1) (push) Failing after 3s
KMS Tests / KMS Provider Integration Tests (push) Failing after 2s
KMS Tests / S3 KMS End-to-End Tests (push) Failing after 3s
Master Cold Start Tests / Master Cold Start Tests (push) Failing after 4s
Metadata Subscribe Integration Tests / Metadata Subscribe Integration Tests (push) Failing after 3s
Multi-Master Tests / Multi-Master Failover Tests (push) Failing after 3s
Performance / CPU and Heap Profile (push) Failing after 2s
Performance / S3 Read/Write Benchmark (rust volume) (push) Failing after 3s
pjdfstest POSIX Compliance / pjdfstest (push) Failing after 2s
Plugin Worker Integration Tests / Plugin Worker: erasure_coding (push) Failing after 3s
Plugin Worker Integration Tests / Plugin Worker: vacuum (push) Failing after 3s
Rust Volume Server Tests / Rust Integration Tests (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 1) (push) Failing after 2s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 1) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 2) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 2) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 3) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Distributed Tests (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Performance Tests (push) Failing after 3s
Performance / Throughput Benchmark (go volume) (push) Failing after 3s
Performance / Throughput Benchmark (rust volume) (push) Failing after 2s
Performance / S3 Read/Write Benchmark (go volume) (push) Failing after 3s
Plugin Worker Integration Tests / Plugin Worker: volume_balance (push) Failing after 3s
PostgreSQL Gateway Tests / PostgreSQL Basic Tests (push) Failing after 3s
Rust Volume Server Tests / Rust Unit Tests (push) Failing after 2s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 3) (push) Failing after 3s
Rust Plugin Worker Tests / Rust Plugin Worker Build and Unit Tests (push) Failing after 3s
rust: build dev volume server binaries / cleanup (push) Successful in 3s
S3 ETag and ACL Tests / S3 ETag + ACL Integration Tests (push) Failing after 3s
S3 IAM Integration Tests / IAM Unit Tests (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (advanced) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (basic) (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Integration Tests (group) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (policy-enforcement) (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Integration Tests (sts) (push) Failing after 4s
S3 Keycloak Integration Tests / S3 Keycloak Integration Tests (push) Failing after 3s
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.11) (push) Failing after 3s
S3 PyArrow Parquet Tests / Go Unit Tests (Implicit Directory) (push) Has been cancelled
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.12) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-darwin (darwin-amd64, x86_64-apple-darwin) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-darwin (darwin-arm64, aarch64-apple-darwin) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-linux (linux-amd64, x86_64-unknown-linux-gnu) (push) Has been cancelled
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.9) (push) Has been cancelled
S3 Proxy Signature Tests / S3 Proxy Signature Verification Tests (push) Has been cancelled
S3 SDK V2 Route Disambiguation Tests / S3 SDK V2 Routing Tests (push) Has been cancelled
S3 SSE Tests / S3 SSE Integration Tests (quick) (push) Has been cancelled
S3 SSE Tests / S3 SSE Compatibility Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Metadata Persistence Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Copy Operations Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Multipart Upload Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Performance Test (push) Has been cancelled
S3 SSE Tests / S3 Volume Encryption Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Integration Tests (comprehensive) (push) Has been cancelled
Spark Integration Tests / Spark Integration Tests (push) Failing after 2s
Telemetry Integration Tests / telemetry-integration-test (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 1) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 1) (push) Failing after 3s
Ceph S3 tests / Basic S3 tests (KV store) (push) Has been cancelled
Ceph S3 tests / S3 Versioning & Object Lock tests (push) Has been cancelled
S3 Policy Integration Tests / S3 Policy Unit Tests (push) Failing after 3s
Ceph S3 tests / S3 CORS tests (push) Has been cancelled
Ceph S3 tests / SeaweedFS Custom S3 Copy tests (push) Has been cancelled
Ceph S3 tests / Basic S3 tests (SQL store) (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
TLS Rotation Integration Tests / TLS Rotation Integration Tests (push) Has been cancelled
Vacuum Integration Tests / Vacuum Integration Tests (push) Has been cancelled
S3 Policy Integration Tests / S3 Policy Variables Integration Tests (push) Failing after 3s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (advanced-policy) (push) Failing after 3s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (basic-policy) (push) Failing after 2s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (contextual-policy) (push) Failing after 4s
S3 Policy Integration Tests / Trusted Proxy Detection Tests (push) Failing after 2s
S3 Policy Integration Tests / S3 Policy Shell Integration Tests (push) Failing after 3s
Samba on FUSE Integration / samba-integration (push) Failing after 2s
SFTP Integration Tests / SFTP Integration Testing (push) Failing after 3s
terraform: validate and test modules / fmt, validate, plan-level tests (push) Failing after 3s
terraform: validate and test modules / local cluster smoke test (real weed) (push) Failing after 2s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 3) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 2) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 2) (push) Failing after 2s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 3) (push) Failing after 5s
mount: windows conformance / WinFsp conformance (push) Has been cancelled
mount: windows / Mount on Windows (push) Has been cancelled
Some checks failed
go: build dev binaries / cleanup (push) Successful in 4s
Codespell / Check for spelling errors (push) Failing after 3s
docker: build dev containers / build-rust-binaries (amd64, x86_64-unknown-linux-musl) (push) Failing after 3s
docker: build dev containers / build-dev-containers (push) Has been skipped
docker: build dev containers / build-rust-binaries (arm64, true, aarch64-unknown-linux-musl) (push) Failing after 3s
End to End / FUSE Mount (push) Failing after 2s
EC Integration Tests / EC Integration Tests (push) Failing after 30s
FUSE DLM Integration Tests / FUSE DLM Integration Tests (push) Failing after 3s
FUSE Volume Server Failover Tests / FUSE Volume Server Failover (push) Failing after 3s
FUSE Integration Tests / FUSE Integration Testing (push) Failing after 2s
FUSE P2P Peer Chunk Sharing Integration Tests / FUSE P2P Peer Chunk Sharing (push) Failing after 3s
go: build binary / Go Vet (push) Failing after 2s
go: build binary / Go Vet 32-bit (push) Failing after 3s
go: build binary / Build (push) Failing after 4s
go: build binary / Build cross-platform (freebsd/amd64) (push) Failing after 3s
go: build binary / Build cross-platform (windows/amd64) (push) Failing after 3s
go: build binary / Build cross-platform (darwin/arm64) (push) Failing after 3s
go: build binary / Build cross-platform (windows/arm64) (push) Failing after 3s
go: build binary / Test (push) Failing after 2s
go: build binary / Test 32-bit (push) Failing after 3s
helm: lint and test charts / lint-test (push) Failing after 3s
Java Client Integration Tests / Java Integration Tests (11) (push) Failing after 3s
Java Client Integration Tests / Java Integration Tests (17) (push) Failing after 3s
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Failing after 2s
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Failing after 3s
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been skipped
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been skipped
Java Client Unit Tests / Java Unit Tests (11) (push) Failing after 3s
Java Client Unit Tests / Java Unit Tests (17) (push) Failing after 2s
Java Client Unit Tests / Java Unit Tests (21) (push) Failing after 3s
Java Client Unit Tests / Java Unit Tests (8) (push) Failing after 3s
Kafka Quick Test (Load Test with Schema Registry) / Kafka Client Load Test (Quick) (push) Failing after 3s
Kafka Gateway Tests / Kafka Unit Tests (unit-tests-1) (push) Failing after 11s
Kafka Gateway Tests / Kafka Integration Tests (Critical) (integration-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka End-to-End Tests (with SMQ) (e2e-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka Consumer Group Tests (Highly Isolated) (consumer-group-1) (push) Failing after 2s
Kafka Gateway Tests / Kafka Client Compatibility (with SMQ) (client-compat-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka SMQ Integration Tests (Full Stack) (smq-integration-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka Protocol Tests (Isolated) (protocol-1) (push) Failing after 3s
KMS Tests / KMS Provider Integration Tests (push) Failing after 2s
KMS Tests / S3 KMS End-to-End Tests (push) Failing after 3s
Master Cold Start Tests / Master Cold Start Tests (push) Failing after 4s
Metadata Subscribe Integration Tests / Metadata Subscribe Integration Tests (push) Failing after 3s
Multi-Master Tests / Multi-Master Failover Tests (push) Failing after 3s
Performance / CPU and Heap Profile (push) Failing after 2s
Performance / S3 Read/Write Benchmark (rust volume) (push) Failing after 3s
pjdfstest POSIX Compliance / pjdfstest (push) Failing after 2s
Plugin Worker Integration Tests / Plugin Worker: erasure_coding (push) Failing after 3s
Plugin Worker Integration Tests / Plugin Worker: vacuum (push) Failing after 3s
Rust Volume Server Tests / Rust Integration Tests (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 1) (push) Failing after 2s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 1) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 2) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 2) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 3) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Distributed Tests (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Performance Tests (push) Failing after 3s
Performance / Throughput Benchmark (go volume) (push) Failing after 3s
Performance / Throughput Benchmark (rust volume) (push) Failing after 2s
Performance / S3 Read/Write Benchmark (go volume) (push) Failing after 3s
Plugin Worker Integration Tests / Plugin Worker: volume_balance (push) Failing after 3s
PostgreSQL Gateway Tests / PostgreSQL Basic Tests (push) Failing after 3s
Rust Volume Server Tests / Rust Unit Tests (push) Failing after 2s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 3) (push) Failing after 3s
Rust Plugin Worker Tests / Rust Plugin Worker Build and Unit Tests (push) Failing after 3s
rust: build dev volume server binaries / cleanup (push) Successful in 3s
S3 ETag and ACL Tests / S3 ETag + ACL Integration Tests (push) Failing after 3s
S3 IAM Integration Tests / IAM Unit Tests (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (advanced) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (basic) (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Integration Tests (group) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (policy-enforcement) (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Integration Tests (sts) (push) Failing after 4s
S3 Keycloak Integration Tests / S3 Keycloak Integration Tests (push) Failing after 3s
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.11) (push) Failing after 3s
S3 PyArrow Parquet Tests / Go Unit Tests (Implicit Directory) (push) Has been cancelled
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.12) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-darwin (darwin-amd64, x86_64-apple-darwin) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-darwin (darwin-arm64, aarch64-apple-darwin) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-linux (linux-amd64, x86_64-unknown-linux-gnu) (push) Has been cancelled
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.9) (push) Has been cancelled
S3 Proxy Signature Tests / S3 Proxy Signature Verification Tests (push) Has been cancelled
S3 SDK V2 Route Disambiguation Tests / S3 SDK V2 Routing Tests (push) Has been cancelled
S3 SSE Tests / S3 SSE Integration Tests (quick) (push) Has been cancelled
S3 SSE Tests / S3 SSE Compatibility Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Metadata Persistence Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Copy Operations Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Multipart Upload Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Performance Test (push) Has been cancelled
S3 SSE Tests / S3 Volume Encryption Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Integration Tests (comprehensive) (push) Has been cancelled
Spark Integration Tests / Spark Integration Tests (push) Failing after 2s
Telemetry Integration Tests / telemetry-integration-test (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 1) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 1) (push) Failing after 3s
Ceph S3 tests / Basic S3 tests (KV store) (push) Has been cancelled
Ceph S3 tests / S3 Versioning & Object Lock tests (push) Has been cancelled
S3 Policy Integration Tests / S3 Policy Unit Tests (push) Failing after 3s
Ceph S3 tests / S3 CORS tests (push) Has been cancelled
Ceph S3 tests / SeaweedFS Custom S3 Copy tests (push) Has been cancelled
Ceph S3 tests / Basic S3 tests (SQL store) (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
TLS Rotation Integration Tests / TLS Rotation Integration Tests (push) Has been cancelled
Vacuum Integration Tests / Vacuum Integration Tests (push) Has been cancelled
S3 Policy Integration Tests / S3 Policy Variables Integration Tests (push) Failing after 3s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (advanced-policy) (push) Failing after 3s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (basic-policy) (push) Failing after 2s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (contextual-policy) (push) Failing after 4s
S3 Policy Integration Tests / Trusted Proxy Detection Tests (push) Failing after 2s
S3 Policy Integration Tests / S3 Policy Shell Integration Tests (push) Failing after 3s
Samba on FUSE Integration / samba-integration (push) Failing after 2s
SFTP Integration Tests / SFTP Integration Testing (push) Failing after 3s
terraform: validate and test modules / fmt, validate, plan-level tests (push) Failing after 3s
terraform: validate and test modules / local cluster smoke test (real weed) (push) Failing after 2s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 3) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 2) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 2) (push) Failing after 2s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 3) (push) Failing after 5s
mount: windows conformance / WinFsp conformance (push) Has been cancelled
mount: windows / Mount on Windows (push) Has been cancelled
This commit is contained in:
349
weed/s3api/policy/post-policy_test.go
Normal file
349
weed/s3api/policy/post-policy_test.go
Normal file
@@ -0,0 +1,349 @@
|
||||
package policy
|
||||
|
||||
/*
|
||||
* MinIO Cloud Storage, (C) 2016, 2017, 2018 MinIO, Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import (
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
const (
|
||||
iso8601DateFormat = "20060102T150405Z"
|
||||
iso8601TimeFormat = "2006-01-02T15:04:05.000Z" // Reply date format with nanosecond precision.
|
||||
)
|
||||
|
||||
// if object matches reserved string, no need to encode them
|
||||
var reservedObjectNames = regexp.MustCompile("^[a-zA-Z0-9-_.~/]+$")
|
||||
|
||||
// EncodePath encode the strings from UTF-8 byte representations to HTML hex escape sequences
|
||||
//
|
||||
// This is necessary since regular url.Parse() and url.Encode() functions do not support UTF-8
|
||||
// non english characters cannot be parsed due to the nature in which url.Encode() is written
|
||||
//
|
||||
// This function on the other hand is a direct replacement for url.Encode() technique to support
|
||||
// pretty much every UTF-8 character.
|
||||
func EncodePath(pathName string) string {
|
||||
if reservedObjectNames.MatchString(pathName) {
|
||||
return pathName
|
||||
}
|
||||
var encodedPathname string
|
||||
for _, s := range pathName {
|
||||
if 'A' <= s && s <= 'Z' || 'a' <= s && s <= 'z' || '0' <= s && s <= '9' { // §2.3 Unreserved characters (mark)
|
||||
encodedPathname = encodedPathname + string(s)
|
||||
continue
|
||||
}
|
||||
switch s {
|
||||
case '-', '_', '.', '~', '/': // §2.3 Unreserved characters (mark)
|
||||
encodedPathname = encodedPathname + string(s)
|
||||
continue
|
||||
default:
|
||||
len := utf8.RuneLen(s)
|
||||
if len < 0 {
|
||||
// if utf8 cannot convert return the same string as is
|
||||
return pathName
|
||||
}
|
||||
u := make([]byte, len)
|
||||
utf8.EncodeRune(u, s)
|
||||
for _, r := range u {
|
||||
hex := hex.EncodeToString([]byte{r})
|
||||
encodedPathname = encodedPathname + "%" + strings.ToUpper(hex)
|
||||
}
|
||||
}
|
||||
}
|
||||
return encodedPathname
|
||||
}
|
||||
|
||||
// buildParsedPolicy is a small test helper that assembles a JSON policy
|
||||
// document with the supplied condition snippets and parses it into a
|
||||
// PostPolicyForm. Using ParsePostPolicyForm avoids having to construct the
|
||||
// anonymous struct in PostPolicyForm.Conditions.Policies directly.
|
||||
func buildParsedPolicy(t *testing.T, conditions string) PostPolicyForm {
|
||||
t.Helper()
|
||||
expiration := time.Now().UTC().Add(24 * time.Hour).Format(iso8601TimeFormat)
|
||||
raw := fmt.Sprintf(`{"expiration":"%s","conditions":[%s]}`, expiration, conditions)
|
||||
ppf, err := ParsePostPolicyForm(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("ParsePostPolicyForm failed: %v\npolicy: %s", err, raw)
|
||||
}
|
||||
return ppf
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_RejectsUnknownConditionKey verifies that a policy
|
||||
// containing a condition key that is neither in startsWithConds nor prefixed
|
||||
// with $x-amz- is rejected instead of being silently accepted.
|
||||
func TestCheckPostPolicy_RejectsUnknownConditionKey(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t, `["eq","$foo","bar"]`)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Foo", "bar")
|
||||
|
||||
err := CheckPostPolicy(form, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for unknown condition key, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "unknown condition key") {
|
||||
t.Fatalf("expected 'unknown condition key' error, got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "$foo") {
|
||||
t.Fatalf("expected error to name the offending key, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_RejectsExtraXAmzFormField verifies that stray X-Amz-*
|
||||
// form fields (beyond the reserved auth/signing ones) are rejected when no
|
||||
// matching policy condition is declared.
|
||||
func TestCheckPostPolicy_RejectsExtraXAmzFormField(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"]`)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
form.Set("X-Amz-Storage-Class", "STANDARD")
|
||||
|
||||
err := CheckPostPolicy(form, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for extra X-Amz-Storage-Class field, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "Extra input fields") {
|
||||
t.Fatalf("expected 'Extra input fields' error, got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "X-Amz-Storage-Class") {
|
||||
t.Fatalf("expected error to name the offending field, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_AllowsXAmzAuthFields verifies that the reserved
|
||||
// auth/signing X-Amz-* headers are accepted even when no policy condition
|
||||
// mentions them, because clients must always send these.
|
||||
func TestCheckPostPolicy_AllowsXAmzAuthFields(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"]`)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
form.Set("X-Amz-Signature", "deadbeef")
|
||||
form.Set("X-Amz-Credential", "AKIA/20260417/us-east-1/s3/aws4_request")
|
||||
form.Set("X-Amz-Algorithm", "AWS4-HMAC-SHA256")
|
||||
form.Set("X-Amz-Date", "20260417T000000Z")
|
||||
form.Set("X-Amz-Security-Token", "session-token")
|
||||
|
||||
if err := CheckPostPolicy(form, ppf); err != nil {
|
||||
t.Fatalf("expected no error with only reserved auth fields, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_AllowsMatchingXAmzField verifies that an X-Amz-* form
|
||||
// field is accepted when there is a matching equality policy condition.
|
||||
func TestCheckPostPolicy_AllowsMatchingXAmzField(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"],["eq","$x-amz-storage-class","STANDARD"]`)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
form.Set("X-Amz-Storage-Class", "STANDARD")
|
||||
|
||||
if err := CheckPostPolicy(form, ppf); err != nil {
|
||||
t.Fatalf("expected no error for matching X-Amz-Storage-Class, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_ExistingXAmzMetaCheckStillWorks is a regression test
|
||||
// for the pre-existing behavior: a stray X-Amz-Meta-* form field without a
|
||||
// matching condition is still rejected.
|
||||
func TestCheckPostPolicy_ExistingXAmzMetaCheckStillWorks(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"]`)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
form.Set("X-Amz-Meta-Foo", "bar")
|
||||
|
||||
err := CheckPostPolicy(form, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for extra X-Amz-Meta-Foo field, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "Extra input fields") {
|
||||
t.Fatalf("expected 'Extra input fields' error, got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "X-Amz-Meta-Foo") {
|
||||
t.Fatalf("expected error to name the offending field, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_AllowsStartsWithPrefixStem covers the AWS convention
|
||||
// where ["starts-with","$x-amz-meta-",""] permits any X-Amz-Meta-* form
|
||||
// field. Without prefix-stem handling, such fields would be wrongly
|
||||
// rejected as "Extra input fields".
|
||||
func TestCheckPostPolicy_AllowsStartsWithPrefixStem(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t,
|
||||
`["eq","$bucket","mybucket"],["starts-with","$x-amz-meta-",""]`,
|
||||
)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
form.Set("X-Amz-Meta-Foo", "bar")
|
||||
form.Set("X-Amz-Meta-Another", "baz")
|
||||
|
||||
if err := CheckPostPolicy(form, ppf); err != nil {
|
||||
t.Fatalf("expected no error for prefix-matched meta fields, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_PrefixStemDoesNotCoverOtherPrefixes ensures the
|
||||
// prefix allowance is scoped: a starts-with stem for x-amz-meta- must not
|
||||
// whitelist unrelated x-amz-* fields like x-amz-storage-class.
|
||||
func TestCheckPostPolicy_PrefixStemDoesNotCoverOtherPrefixes(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t,
|
||||
`["eq","$bucket","mybucket"],["starts-with","$x-amz-meta-",""]`,
|
||||
)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
form.Set("X-Amz-Storage-Class", "STANDARD")
|
||||
|
||||
err := CheckPostPolicy(form, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for X-Amz-Storage-Class not covered by meta prefix, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "X-Amz-Storage-Class") {
|
||||
t.Fatalf("expected error to name X-Amz-Storage-Class, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_PrefixStemEnforcesValuePrefix covers a starts-with
|
||||
// prefix-stem policy with a non-empty required value prefix: matching
|
||||
// fields must have values that satisfy the value prefix.
|
||||
func TestCheckPostPolicy_PrefixStemEnforcesValuePrefix(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t,
|
||||
`["eq","$bucket","mybucket"],["starts-with","$x-amz-meta-","pfx-"]`,
|
||||
)
|
||||
|
||||
// Value satisfies the required prefix: accepted.
|
||||
okForm := http.Header{}
|
||||
okForm.Set("Bucket", "mybucket")
|
||||
okForm.Set("X-Amz-Meta-Foo", "pfx-bar")
|
||||
if err := CheckPostPolicy(okForm, ppf); err != nil {
|
||||
t.Fatalf("expected no error when meta value matches required prefix, got: %v", err)
|
||||
}
|
||||
|
||||
// Value does not satisfy the required prefix: rejected as policy failure.
|
||||
badForm := http.Header{}
|
||||
badForm.Set("Bucket", "mybucket")
|
||||
badForm.Set("X-Amz-Meta-Foo", "other")
|
||||
err := CheckPostPolicy(badForm, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error when meta value misses required prefix, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "Policy Condition failed") {
|
||||
t.Fatalf("expected 'Policy Condition failed' error, got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "pfx-") {
|
||||
t.Fatalf("expected error to reference the required value prefix, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_ExactAndPrefixBothEnforced covers a field that is
|
||||
// simultaneously covered by an exact-key condition and a prefix-stem
|
||||
// condition. Both must be satisfied; exact-match alone does not let the
|
||||
// field skip the stem's value-prefix check.
|
||||
func TestCheckPostPolicy_ExactAndPrefixBothEnforced(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t,
|
||||
`["eq","$bucket","mybucket"],`+
|
||||
`["eq","$x-amz-meta-tag","gold"],`+
|
||||
`["starts-with","$x-amz-meta-","lvl-"]`,
|
||||
)
|
||||
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
// Satisfies the exact eq condition but not the starts-with stem.
|
||||
form.Set("X-Amz-Meta-Tag", "gold")
|
||||
|
||||
err := CheckPostPolicy(form, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error: exact-match field must still satisfy overlapping prefix stem, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "Policy Condition failed") {
|
||||
t.Fatalf("expected 'Policy Condition failed' error, got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "lvl-") {
|
||||
t.Fatalf("expected error to reference the stem's value prefix, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_MultiplePrefixStemsAllEnforced covers a field that
|
||||
// matches multiple starts-with prefix stems. All stems must hold; a value
|
||||
// that satisfies one but not the other must be rejected.
|
||||
func TestCheckPostPolicy_MultiplePrefixStemsAllEnforced(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t,
|
||||
`["eq","$bucket","mybucket"],`+
|
||||
`["starts-with","$x-amz-meta-","pfx-"],`+
|
||||
`["starts-with","$x-amz-meta-color-","red-"]`,
|
||||
)
|
||||
|
||||
// Satisfies the broader stem but not the color- stem's value prefix.
|
||||
form := http.Header{}
|
||||
form.Set("Bucket", "mybucket")
|
||||
form.Set("X-Amz-Meta-Color-Main", "pfx-green")
|
||||
|
||||
err := CheckPostPolicy(form, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error: field must satisfy every matching prefix stem, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "Policy Condition failed") {
|
||||
t.Fatalf("expected 'Policy Condition failed' error, got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "red-") {
|
||||
t.Fatalf("expected error to reference the failing value prefix, got: %v", err)
|
||||
}
|
||||
|
||||
// A policy with compatible stems (broad allows anything, narrow
|
||||
// requires "red-") accepts a value honoring both.
|
||||
compatible := buildParsedPolicy(t,
|
||||
`["eq","$bucket","mybucket"],`+
|
||||
`["starts-with","$x-amz-meta-",""],`+
|
||||
`["starts-with","$x-amz-meta-color-","red-"]`,
|
||||
)
|
||||
okForm := http.Header{}
|
||||
okForm.Set("Bucket", "mybucket")
|
||||
okForm.Set("X-Amz-Meta-Color-Main", "red-main")
|
||||
if err := CheckPostPolicy(okForm, compatible); err != nil {
|
||||
t.Fatalf("expected no error when value satisfies both compatible stems, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckPostPolicy_UnknownKeyErrorIncludesPolicyValue ensures the
|
||||
// unknown-condition error surfaces the policy value in its [op, key, value]
|
||||
// trailer so operators can tell which of several unknown keys failed.
|
||||
func TestCheckPostPolicy_UnknownKeyErrorIncludesPolicyValue(t *testing.T) {
|
||||
ppf := buildParsedPolicy(t, `["eq","$foo","custom-value"]`)
|
||||
|
||||
err := CheckPostPolicy(http.Header{}, ppf)
|
||||
if err == nil {
|
||||
t.Fatalf("expected error for unknown condition key, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "custom-value") {
|
||||
t.Fatalf("expected error to include policy.Value 'custom-value', got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "$foo") {
|
||||
t.Fatalf("expected error to include policy.Key '$foo', got: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "unknown condition key") {
|
||||
t.Fatalf("expected 'unknown condition key' suffix, got: %v", err)
|
||||
}
|
||||
}
|
||||
343
weed/s3api/policy/postpolicyform.go
Normal file
343
weed/s3api/policy/postpolicyform.go
Normal file
@@ -0,0 +1,343 @@
|
||||
package policy
|
||||
|
||||
/*
|
||||
* MinIO Cloud Storage, (C) 2015, 2016, 2017 MinIO, Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// startWithConds - map which indicates if a given condition supports starts-with policy operator
|
||||
var startsWithConds = map[string]bool{
|
||||
"$acl": true,
|
||||
"$bucket": false,
|
||||
"$cache-control": true,
|
||||
"$content-type": true,
|
||||
"$content-disposition": true,
|
||||
"$content-encoding": true,
|
||||
"$expires": true,
|
||||
"$key": true,
|
||||
"$success_action_redirect": true,
|
||||
"$redirect": true,
|
||||
"$success_action_status": false,
|
||||
"$x-amz-algorithm": false,
|
||||
"$x-amz-credential": false,
|
||||
"$x-amz-date": false,
|
||||
}
|
||||
|
||||
// Add policy conditionals.
|
||||
const (
|
||||
policyCondEqual = "eq"
|
||||
policyCondStartsWith = "starts-with"
|
||||
policyCondContentLength = "content-length-range"
|
||||
)
|
||||
|
||||
// toString - Safely convert interface to string without causing panic.
|
||||
func toString(val interface{}) string {
|
||||
switch v := val.(type) {
|
||||
case string:
|
||||
return v
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// toLowerString - safely convert interface to lower string
|
||||
func toLowerString(val interface{}) string {
|
||||
return strings.ToLower(toString(val))
|
||||
}
|
||||
|
||||
// toInteger _ Safely convert interface to integer without causing panic.
|
||||
func toInteger(val interface{}) (int64, error) {
|
||||
switch v := val.(type) {
|
||||
case float64:
|
||||
return int64(v), nil
|
||||
case int64:
|
||||
return v, nil
|
||||
case int:
|
||||
return int64(v), nil
|
||||
case string:
|
||||
i, err := strconv.Atoi(v)
|
||||
return int64(i), err
|
||||
default:
|
||||
return 0, errors.New("Invalid number format")
|
||||
}
|
||||
}
|
||||
|
||||
// isString - Safely check if val is of type string without causing panic.
|
||||
func isString(val interface{}) bool {
|
||||
_, ok := val.(string)
|
||||
return ok
|
||||
}
|
||||
|
||||
// ContentLengthRange - policy content-length-range field.
|
||||
type contentLengthRange struct {
|
||||
Min int64
|
||||
Max int64
|
||||
Valid bool // If content-length-range was part of policy
|
||||
}
|
||||
|
||||
// PostPolicyForm provides strict static type conversion and validation for Amazon S3's POST policy JSON string.
|
||||
type PostPolicyForm struct {
|
||||
Expiration time.Time // Expiration date and time of the POST policy.
|
||||
Conditions struct { // Conditional policy structure.
|
||||
Policies []struct {
|
||||
Operator string
|
||||
Key string
|
||||
Value string
|
||||
}
|
||||
ContentLengthRange contentLengthRange
|
||||
}
|
||||
}
|
||||
|
||||
// ParsePostPolicyForm - Parse JSON policy string into typed PostPolicyForm structure.
|
||||
func ParsePostPolicyForm(policy string) (ppf PostPolicyForm, e error) {
|
||||
// Convert po into interfaces and
|
||||
// perform strict type conversion using reflection.
|
||||
var rawPolicy struct {
|
||||
Expiration string `json:"expiration"`
|
||||
Conditions []interface{} `json:"conditions"`
|
||||
}
|
||||
|
||||
err := json.Unmarshal([]byte(policy), &rawPolicy)
|
||||
if err != nil {
|
||||
return ppf, err
|
||||
}
|
||||
|
||||
parsedPolicy := PostPolicyForm{}
|
||||
|
||||
// Parse expiry time.
|
||||
parsedPolicy.Expiration, err = time.Parse(time.RFC3339Nano, rawPolicy.Expiration)
|
||||
if err != nil {
|
||||
return ppf, err
|
||||
}
|
||||
|
||||
// Parse conditions.
|
||||
for _, val := range rawPolicy.Conditions {
|
||||
switch condt := val.(type) {
|
||||
case map[string]interface{}: // Handle key:value map types.
|
||||
for k, v := range condt {
|
||||
if !isString(v) { // Pre-check value type.
|
||||
// All values must be of type string.
|
||||
return parsedPolicy, fmt.Errorf("Unknown type %s of conditional field value %s found in POST policy form", reflect.TypeOf(condt).String(), condt)
|
||||
}
|
||||
// {"acl": "public-read" } is an alternate way to indicate - [ "eq", "$acl", "public-read" ]
|
||||
// In this case we will just collapse this into "eq" for all use cases.
|
||||
parsedPolicy.Conditions.Policies = append(parsedPolicy.Conditions.Policies, struct {
|
||||
Operator string
|
||||
Key string
|
||||
Value string
|
||||
}{
|
||||
policyCondEqual, "$" + strings.ToLower(k), toString(v),
|
||||
})
|
||||
}
|
||||
case []interface{}: // Handle array types.
|
||||
if len(condt) != 3 { // Return error if we have insufficient elements.
|
||||
return parsedPolicy, fmt.Errorf("Malformed conditional fields %s of type %s found in POST policy form", condt, reflect.TypeOf(condt).String())
|
||||
}
|
||||
switch toLowerString(condt[0]) {
|
||||
case policyCondEqual, policyCondStartsWith:
|
||||
for _, v := range condt { // Pre-check all values for type.
|
||||
if !isString(v) {
|
||||
// All values must be of type string.
|
||||
return parsedPolicy, fmt.Errorf("Unknown type %s of conditional field value %s found in POST policy form", reflect.TypeOf(condt).String(), condt)
|
||||
}
|
||||
}
|
||||
operator, matchType, value := toLowerString(condt[0]), toLowerString(condt[1]), toString(condt[2])
|
||||
if !strings.HasPrefix(matchType, "$") {
|
||||
return parsedPolicy, fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]", operator, matchType, value)
|
||||
}
|
||||
parsedPolicy.Conditions.Policies = append(parsedPolicy.Conditions.Policies, struct {
|
||||
Operator string
|
||||
Key string
|
||||
Value string
|
||||
}{
|
||||
operator, matchType, value,
|
||||
})
|
||||
case policyCondContentLength:
|
||||
min, err := toInteger(condt[1])
|
||||
if err != nil {
|
||||
return parsedPolicy, err
|
||||
}
|
||||
|
||||
max, err := toInteger(condt[2])
|
||||
if err != nil {
|
||||
return parsedPolicy, err
|
||||
}
|
||||
|
||||
parsedPolicy.Conditions.ContentLengthRange = contentLengthRange{
|
||||
Min: min,
|
||||
Max: max,
|
||||
Valid: true,
|
||||
}
|
||||
default:
|
||||
// Condition should be valid.
|
||||
return parsedPolicy, fmt.Errorf("Unknown type %s of conditional field value %s found in POST policy form",
|
||||
reflect.TypeOf(condt).String(), condt)
|
||||
}
|
||||
default:
|
||||
return parsedPolicy, fmt.Errorf("Unknown field %s of type %s found in POST policy form",
|
||||
condt, reflect.TypeOf(condt).String())
|
||||
}
|
||||
}
|
||||
return parsedPolicy, nil
|
||||
}
|
||||
|
||||
// checkPolicyCond returns a boolean to indicate if a condition is satisfied according
|
||||
// to the passed operator
|
||||
func checkPolicyCond(op string, input1, input2 string) bool {
|
||||
switch op {
|
||||
case policyCondEqual:
|
||||
return input1 == input2
|
||||
case policyCondStartsWith:
|
||||
return strings.HasPrefix(input1, input2)
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// xAmzPrefixRule captures a starts-with policy condition whose key ends in
|
||||
// "-" and therefore matches a prefix of form field names, e.g.
|
||||
// ["starts-with","$x-amz-meta-","pfx-"]. Any form field whose name starts
|
||||
// with namePrefix must have a value starting with valuePrefix.
|
||||
type xAmzPrefixRule struct {
|
||||
policyKey string
|
||||
namePrefix string
|
||||
valuePrefix string
|
||||
}
|
||||
|
||||
// postPolicyAuthFields enumerates the X-Amz-* form fields that clients are
|
||||
// required to send with every POST Object request for signing/authentication.
|
||||
// These must be accepted even when no matching policy condition is declared.
|
||||
var postPolicyAuthFields = map[string]bool{
|
||||
"X-Amz-Signature": true,
|
||||
"X-Amz-Credential": true,
|
||||
"X-Amz-Algorithm": true,
|
||||
"X-Amz-Date": true,
|
||||
"X-Amz-Security-Token": true,
|
||||
}
|
||||
|
||||
// CheckPostPolicy - apply policy conditions and validate input values.
|
||||
// (http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html)
|
||||
func CheckPostPolicy(formValues http.Header, postPolicyForm PostPolicyForm) error {
|
||||
// Check if policy document expiry date is still not reached
|
||||
if !postPolicyForm.Expiration.After(time.Now().UTC()) {
|
||||
return fmt.Errorf("Invalid according to Policy: Policy expired")
|
||||
}
|
||||
// Track $x-amz-* policy conditions in canonical form. A starts-with
|
||||
// condition on a key that itself ends with "-" (e.g.
|
||||
// ["starts-with","$x-amz-meta-","pfx-"]) is the AWS convention for
|
||||
// allowing any form field sharing that name prefix; capture the name
|
||||
// prefix together with its required value prefix so the extras check
|
||||
// can both accept matching fields and enforce the value constraint.
|
||||
policyXAmzKeys := make(map[string]bool)
|
||||
var policyXAmzPrefixes []xAmzPrefixRule
|
||||
for _, policy := range postPolicyForm.Conditions.Policies {
|
||||
if !strings.HasPrefix(policy.Key, "$x-amz-") {
|
||||
continue
|
||||
}
|
||||
formCanonicalName := http.CanonicalHeaderKey(strings.TrimPrefix(policy.Key, "$"))
|
||||
if policy.Operator == policyCondStartsWith && strings.HasSuffix(policy.Key, "-") {
|
||||
policyXAmzPrefixes = append(policyXAmzPrefixes, xAmzPrefixRule{
|
||||
policyKey: policy.Key,
|
||||
namePrefix: formCanonicalName,
|
||||
valuePrefix: policy.Value,
|
||||
})
|
||||
continue
|
||||
}
|
||||
policyXAmzKeys[formCanonicalName] = true
|
||||
}
|
||||
// Reject any X-Amz-* form field that has no matching policy condition,
|
||||
// except for the reserved auth/signing fields clients must always send.
|
||||
// A field may be covered by an exact-key condition, by any number of
|
||||
// prefix-stem rules, or both. Check every applicable rule: AWS requires
|
||||
// all matching conditions to be satisfied, so exact-match coverage does
|
||||
// not let a field skip the value-prefix enforcement of overlapping
|
||||
// starts-with stems, and multiple stems on the same field must all hold.
|
||||
// Prefix-stem conditions are then skipped in the main loop below because
|
||||
// no single form field corresponds to the prefix itself.
|
||||
for key := range formValues {
|
||||
if !strings.HasPrefix(key, "X-Amz-") {
|
||||
continue
|
||||
}
|
||||
if postPolicyAuthFields[key] {
|
||||
continue
|
||||
}
|
||||
matched := policyXAmzKeys[key]
|
||||
for _, rule := range policyXAmzPrefixes {
|
||||
if !strings.HasPrefix(key, rule.namePrefix) {
|
||||
continue
|
||||
}
|
||||
matched = true
|
||||
if !strings.HasPrefix(formValues.Get(key), rule.valuePrefix) {
|
||||
return fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]", policyCondStartsWith, rule.policyKey, rule.valuePrefix)
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
return fmt.Errorf("Invalid according to Policy: Extra input fields: %s", key)
|
||||
}
|
||||
}
|
||||
|
||||
// Flag to indicate if all policies conditions are satisfied
|
||||
var condPassed bool
|
||||
|
||||
// Iterate over policy conditions and check them against received form fields
|
||||
for _, policy := range postPolicyForm.Conditions.Policies {
|
||||
// Form fields names are in canonical format, convert conditions names
|
||||
// to canonical for simplification purpose, so `$key` will become `Key`
|
||||
formCanonicalName := http.CanonicalHeaderKey(strings.TrimPrefix(policy.Key, "$"))
|
||||
// Operator for the current policy condition
|
||||
op := policy.Operator
|
||||
// Prefix-stem x-amz-* conditions (key ending in "-" with starts-with)
|
||||
// are validated above against every matching form field; skip them
|
||||
// here so we do not fail on the non-existent literal form field whose
|
||||
// name equals the prefix itself.
|
||||
if strings.HasPrefix(policy.Key, "$x-amz-") && op == policyCondStartsWith && strings.HasSuffix(policy.Key, "-") {
|
||||
continue
|
||||
}
|
||||
// If the current policy condition is known
|
||||
if startsWithSupported, condFound := startsWithConds[policy.Key]; condFound {
|
||||
// Check if the current condition supports starts-with operator
|
||||
if op == policyCondStartsWith && !startsWithSupported {
|
||||
return fmt.Errorf("Invalid according to Policy: Policy Condition failed")
|
||||
}
|
||||
// Check if current policy condition is satisfied
|
||||
condPassed = checkPolicyCond(op, formValues.Get(formCanonicalName), policy.Value)
|
||||
if !condPassed {
|
||||
return fmt.Errorf("Invalid according to Policy: Policy Condition failed")
|
||||
}
|
||||
} else if strings.HasPrefix(policy.Key, "$x-amz-") {
|
||||
// Check if policy condition is satisfied
|
||||
condPassed = checkPolicyCond(op, formValues.Get(formCanonicalName), policy.Value)
|
||||
if !condPassed {
|
||||
return fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]", op, policy.Key, policy.Value)
|
||||
}
|
||||
} else {
|
||||
// Unknown condition key: neither in startsWithConds nor a $x-amz-*
|
||||
// prefixed key. AWS rejects these outright instead of silently
|
||||
// treating the condition as satisfied.
|
||||
return fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]: unknown condition key", op, policy.Key, policy.Value)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user