snapshot of https://github.com/seaweedfs/seaweedfs.git (master, 2026-09-16; blobs > 131072B omitted)
Some checks failed
go: build dev binaries / cleanup (push) Successful in 4s
Codespell / Check for spelling errors (push) Failing after 3s
docker: build dev containers / build-rust-binaries (amd64, x86_64-unknown-linux-musl) (push) Failing after 3s
docker: build dev containers / build-dev-containers (push) Has been skipped
docker: build dev containers / build-rust-binaries (arm64, true, aarch64-unknown-linux-musl) (push) Failing after 3s
End to End / FUSE Mount (push) Failing after 2s
EC Integration Tests / EC Integration Tests (push) Failing after 30s
FUSE DLM Integration Tests / FUSE DLM Integration Tests (push) Failing after 3s
FUSE Volume Server Failover Tests / FUSE Volume Server Failover (push) Failing after 3s
FUSE Integration Tests / FUSE Integration Testing (push) Failing after 2s
FUSE P2P Peer Chunk Sharing Integration Tests / FUSE P2P Peer Chunk Sharing (push) Failing after 3s
go: build binary / Go Vet (push) Failing after 2s
go: build binary / Go Vet 32-bit (push) Failing after 3s
go: build binary / Build (push) Failing after 4s
go: build binary / Build cross-platform (freebsd/amd64) (push) Failing after 3s
go: build binary / Build cross-platform (windows/amd64) (push) Failing after 3s
go: build binary / Build cross-platform (darwin/arm64) (push) Failing after 3s
go: build binary / Build cross-platform (windows/arm64) (push) Failing after 3s
go: build binary / Test (push) Failing after 2s
go: build binary / Test 32-bit (push) Failing after 3s
helm: lint and test charts / lint-test (push) Failing after 3s
Java Client Integration Tests / Java Integration Tests (11) (push) Failing after 3s
Java Client Integration Tests / Java Integration Tests (17) (push) Failing after 3s
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Failing after 2s
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Failing after 3s
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been skipped
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been skipped
Java Client Unit Tests / Java Unit Tests (11) (push) Failing after 3s
Java Client Unit Tests / Java Unit Tests (17) (push) Failing after 2s
Java Client Unit Tests / Java Unit Tests (21) (push) Failing after 3s
Java Client Unit Tests / Java Unit Tests (8) (push) Failing after 3s
Kafka Quick Test (Load Test with Schema Registry) / Kafka Client Load Test (Quick) (push) Failing after 3s
Kafka Gateway Tests / Kafka Unit Tests (unit-tests-1) (push) Failing after 11s
Kafka Gateway Tests / Kafka Integration Tests (Critical) (integration-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka End-to-End Tests (with SMQ) (e2e-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka Consumer Group Tests (Highly Isolated) (consumer-group-1) (push) Failing after 2s
Kafka Gateway Tests / Kafka Client Compatibility (with SMQ) (client-compat-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka SMQ Integration Tests (Full Stack) (smq-integration-1) (push) Failing after 3s
Kafka Gateway Tests / Kafka Protocol Tests (Isolated) (protocol-1) (push) Failing after 3s
KMS Tests / KMS Provider Integration Tests (push) Failing after 2s
KMS Tests / S3 KMS End-to-End Tests (push) Failing after 3s
Master Cold Start Tests / Master Cold Start Tests (push) Failing after 4s
Metadata Subscribe Integration Tests / Metadata Subscribe Integration Tests (push) Failing after 3s
Multi-Master Tests / Multi-Master Failover Tests (push) Failing after 3s
Performance / CPU and Heap Profile (push) Failing after 2s
Performance / S3 Read/Write Benchmark (rust volume) (push) Failing after 3s
pjdfstest POSIX Compliance / pjdfstest (push) Failing after 2s
Plugin Worker Integration Tests / Plugin Worker: erasure_coding (push) Failing after 3s
Plugin Worker Integration Tests / Plugin Worker: vacuum (push) Failing after 3s
Rust Volume Server Tests / Rust Integration Tests (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 1) (push) Failing after 2s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 1) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 2) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 2) (push) Failing after 3s
Rust Volume Server Tests / Go Tests with Rust Volume (grpc - Shard 3) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Distributed Tests (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Performance Tests (push) Failing after 3s
Performance / Throughput Benchmark (go volume) (push) Failing after 3s
Performance / Throughput Benchmark (rust volume) (push) Failing after 2s
Performance / S3 Read/Write Benchmark (go volume) (push) Failing after 3s
Plugin Worker Integration Tests / Plugin Worker: volume_balance (push) Failing after 3s
PostgreSQL Gateway Tests / PostgreSQL Basic Tests (push) Failing after 3s
Rust Volume Server Tests / Rust Unit Tests (push) Failing after 2s
Rust Volume Server Tests / Go Tests with Rust Volume (http - Shard 3) (push) Failing after 3s
Rust Plugin Worker Tests / Rust Plugin Worker Build and Unit Tests (push) Failing after 3s
rust: build dev volume server binaries / cleanup (push) Successful in 3s
S3 ETag and ACL Tests / S3 ETag + ACL Integration Tests (push) Failing after 3s
S3 IAM Integration Tests / IAM Unit Tests (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (advanced) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (basic) (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Integration Tests (group) (push) Failing after 3s
S3 IAM Integration Tests / S3 IAM Integration Tests (policy-enforcement) (push) Failing after 2s
S3 IAM Integration Tests / S3 IAM Integration Tests (sts) (push) Failing after 4s
S3 Keycloak Integration Tests / S3 Keycloak Integration Tests (push) Failing after 3s
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.11) (push) Failing after 3s
S3 PyArrow Parquet Tests / Go Unit Tests (Implicit Directory) (push) Has been cancelled
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.12) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-darwin (darwin-amd64, x86_64-apple-darwin) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-darwin (darwin-arm64, aarch64-apple-darwin) (push) Has been cancelled
rust: build dev volume server binaries / build-rust-volume-dev-linux (linux-amd64, x86_64-unknown-linux-gnu) (push) Has been cancelled
S3 PyArrow Parquet Tests / PyArrow Parquet Tests (Python 3.9) (push) Has been cancelled
S3 Proxy Signature Tests / S3 Proxy Signature Verification Tests (push) Has been cancelled
S3 SDK V2 Route Disambiguation Tests / S3 SDK V2 Routing Tests (push) Has been cancelled
S3 SSE Tests / S3 SSE Integration Tests (quick) (push) Has been cancelled
S3 SSE Tests / S3 SSE Compatibility Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Metadata Persistence Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Copy Operations Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Multipart Upload Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Performance Test (push) Has been cancelled
S3 SSE Tests / S3 Volume Encryption Test (push) Has been cancelled
S3 SSE Tests / S3 SSE Integration Tests (comprehensive) (push) Has been cancelled
Spark Integration Tests / Spark Integration Tests (push) Failing after 2s
Telemetry Integration Tests / telemetry-integration-test (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 1) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 1) (push) Failing after 3s
Ceph S3 tests / Basic S3 tests (KV store) (push) Has been cancelled
Ceph S3 tests / S3 Versioning & Object Lock tests (push) Has been cancelled
S3 Policy Integration Tests / S3 Policy Unit Tests (push) Failing after 3s
Ceph S3 tests / S3 CORS tests (push) Has been cancelled
Ceph S3 tests / SeaweedFS Custom S3 Copy tests (push) Has been cancelled
Ceph S3 tests / Basic S3 tests (SQL store) (push) Has been cancelled
test s3 over https using aws-cli / awscli-tests (push) Has been cancelled
TLS Rotation Integration Tests / TLS Rotation Integration Tests (push) Has been cancelled
Vacuum Integration Tests / Vacuum Integration Tests (push) Has been cancelled
S3 Policy Integration Tests / S3 Policy Variables Integration Tests (push) Failing after 3s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (advanced-policy) (push) Failing after 3s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (basic-policy) (push) Failing after 2s
S3 Policy Integration Tests / S3 Policy Enforcement Integration Tests (contextual-policy) (push) Failing after 4s
S3 Policy Integration Tests / Trusted Proxy Detection Tests (push) Failing after 2s
S3 Policy Integration Tests / S3 Policy Shell Integration Tests (push) Failing after 3s
Samba on FUSE Integration / samba-integration (push) Failing after 2s
SFTP Integration Tests / SFTP Integration Testing (push) Failing after 3s
terraform: validate and test modules / fmt, validate, plan-level tests (push) Failing after 3s
terraform: validate and test modules / local cluster smoke test (real weed) (push) Failing after 2s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 3) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (grpc - Shard 2) (push) Failing after 3s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 2) (push) Failing after 2s
Volume Server Integration Tests / Volume Server Integration Tests (http - Shard 3) (push) Failing after 5s
mount: windows conformance / WinFsp conformance (push) Has been cancelled
mount: windows / Mount on Windows (push) Has been cancelled

This commit is contained in:
upstream-mirror
2026-09-16 03:23:42 +05:00
commit 89f5be005d
3997 changed files with 961970 additions and 0 deletions

View File

@@ -0,0 +1,349 @@
package policy
/*
* MinIO Cloud Storage, (C) 2016, 2017, 2018 MinIO, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import (
"encoding/hex"
"fmt"
"net/http"
"regexp"
"strings"
"testing"
"time"
"unicode/utf8"
)
const (
iso8601DateFormat = "20060102T150405Z"
iso8601TimeFormat = "2006-01-02T15:04:05.000Z" // Reply date format with nanosecond precision.
)
// if object matches reserved string, no need to encode them
var reservedObjectNames = regexp.MustCompile("^[a-zA-Z0-9-_.~/]+$")
// EncodePath encode the strings from UTF-8 byte representations to HTML hex escape sequences
//
// This is necessary since regular url.Parse() and url.Encode() functions do not support UTF-8
// non english characters cannot be parsed due to the nature in which url.Encode() is written
//
// This function on the other hand is a direct replacement for url.Encode() technique to support
// pretty much every UTF-8 character.
func EncodePath(pathName string) string {
if reservedObjectNames.MatchString(pathName) {
return pathName
}
var encodedPathname string
for _, s := range pathName {
if 'A' <= s && s <= 'Z' || 'a' <= s && s <= 'z' || '0' <= s && s <= '9' { // §2.3 Unreserved characters (mark)
encodedPathname = encodedPathname + string(s)
continue
}
switch s {
case '-', '_', '.', '~', '/': // §2.3 Unreserved characters (mark)
encodedPathname = encodedPathname + string(s)
continue
default:
len := utf8.RuneLen(s)
if len < 0 {
// if utf8 cannot convert return the same string as is
return pathName
}
u := make([]byte, len)
utf8.EncodeRune(u, s)
for _, r := range u {
hex := hex.EncodeToString([]byte{r})
encodedPathname = encodedPathname + "%" + strings.ToUpper(hex)
}
}
}
return encodedPathname
}
// buildParsedPolicy is a small test helper that assembles a JSON policy
// document with the supplied condition snippets and parses it into a
// PostPolicyForm. Using ParsePostPolicyForm avoids having to construct the
// anonymous struct in PostPolicyForm.Conditions.Policies directly.
func buildParsedPolicy(t *testing.T, conditions string) PostPolicyForm {
t.Helper()
expiration := time.Now().UTC().Add(24 * time.Hour).Format(iso8601TimeFormat)
raw := fmt.Sprintf(`{"expiration":"%s","conditions":[%s]}`, expiration, conditions)
ppf, err := ParsePostPolicyForm(raw)
if err != nil {
t.Fatalf("ParsePostPolicyForm failed: %v\npolicy: %s", err, raw)
}
return ppf
}
// TestCheckPostPolicy_RejectsUnknownConditionKey verifies that a policy
// containing a condition key that is neither in startsWithConds nor prefixed
// with $x-amz- is rejected instead of being silently accepted.
func TestCheckPostPolicy_RejectsUnknownConditionKey(t *testing.T) {
ppf := buildParsedPolicy(t, `["eq","$foo","bar"]`)
form := http.Header{}
form.Set("Foo", "bar")
err := CheckPostPolicy(form, ppf)
if err == nil {
t.Fatalf("expected error for unknown condition key, got nil")
}
if !strings.Contains(err.Error(), "unknown condition key") {
t.Fatalf("expected 'unknown condition key' error, got: %v", err)
}
if !strings.Contains(err.Error(), "$foo") {
t.Fatalf("expected error to name the offending key, got: %v", err)
}
}
// TestCheckPostPolicy_RejectsExtraXAmzFormField verifies that stray X-Amz-*
// form fields (beyond the reserved auth/signing ones) are rejected when no
// matching policy condition is declared.
func TestCheckPostPolicy_RejectsExtraXAmzFormField(t *testing.T) {
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"]`)
form := http.Header{}
form.Set("Bucket", "mybucket")
form.Set("X-Amz-Storage-Class", "STANDARD")
err := CheckPostPolicy(form, ppf)
if err == nil {
t.Fatalf("expected error for extra X-Amz-Storage-Class field, got nil")
}
if !strings.Contains(err.Error(), "Extra input fields") {
t.Fatalf("expected 'Extra input fields' error, got: %v", err)
}
if !strings.Contains(err.Error(), "X-Amz-Storage-Class") {
t.Fatalf("expected error to name the offending field, got: %v", err)
}
}
// TestCheckPostPolicy_AllowsXAmzAuthFields verifies that the reserved
// auth/signing X-Amz-* headers are accepted even when no policy condition
// mentions them, because clients must always send these.
func TestCheckPostPolicy_AllowsXAmzAuthFields(t *testing.T) {
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"]`)
form := http.Header{}
form.Set("Bucket", "mybucket")
form.Set("X-Amz-Signature", "deadbeef")
form.Set("X-Amz-Credential", "AKIA/20260417/us-east-1/s3/aws4_request")
form.Set("X-Amz-Algorithm", "AWS4-HMAC-SHA256")
form.Set("X-Amz-Date", "20260417T000000Z")
form.Set("X-Amz-Security-Token", "session-token")
if err := CheckPostPolicy(form, ppf); err != nil {
t.Fatalf("expected no error with only reserved auth fields, got: %v", err)
}
}
// TestCheckPostPolicy_AllowsMatchingXAmzField verifies that an X-Amz-* form
// field is accepted when there is a matching equality policy condition.
func TestCheckPostPolicy_AllowsMatchingXAmzField(t *testing.T) {
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"],["eq","$x-amz-storage-class","STANDARD"]`)
form := http.Header{}
form.Set("Bucket", "mybucket")
form.Set("X-Amz-Storage-Class", "STANDARD")
if err := CheckPostPolicy(form, ppf); err != nil {
t.Fatalf("expected no error for matching X-Amz-Storage-Class, got: %v", err)
}
}
// TestCheckPostPolicy_ExistingXAmzMetaCheckStillWorks is a regression test
// for the pre-existing behavior: a stray X-Amz-Meta-* form field without a
// matching condition is still rejected.
func TestCheckPostPolicy_ExistingXAmzMetaCheckStillWorks(t *testing.T) {
ppf := buildParsedPolicy(t, `["eq","$bucket","mybucket"]`)
form := http.Header{}
form.Set("Bucket", "mybucket")
form.Set("X-Amz-Meta-Foo", "bar")
err := CheckPostPolicy(form, ppf)
if err == nil {
t.Fatalf("expected error for extra X-Amz-Meta-Foo field, got nil")
}
if !strings.Contains(err.Error(), "Extra input fields") {
t.Fatalf("expected 'Extra input fields' error, got: %v", err)
}
if !strings.Contains(err.Error(), "X-Amz-Meta-Foo") {
t.Fatalf("expected error to name the offending field, got: %v", err)
}
}
// TestCheckPostPolicy_AllowsStartsWithPrefixStem covers the AWS convention
// where ["starts-with","$x-amz-meta-",""] permits any X-Amz-Meta-* form
// field. Without prefix-stem handling, such fields would be wrongly
// rejected as "Extra input fields".
func TestCheckPostPolicy_AllowsStartsWithPrefixStem(t *testing.T) {
ppf := buildParsedPolicy(t,
`["eq","$bucket","mybucket"],["starts-with","$x-amz-meta-",""]`,
)
form := http.Header{}
form.Set("Bucket", "mybucket")
form.Set("X-Amz-Meta-Foo", "bar")
form.Set("X-Amz-Meta-Another", "baz")
if err := CheckPostPolicy(form, ppf); err != nil {
t.Fatalf("expected no error for prefix-matched meta fields, got: %v", err)
}
}
// TestCheckPostPolicy_PrefixStemDoesNotCoverOtherPrefixes ensures the
// prefix allowance is scoped: a starts-with stem for x-amz-meta- must not
// whitelist unrelated x-amz-* fields like x-amz-storage-class.
func TestCheckPostPolicy_PrefixStemDoesNotCoverOtherPrefixes(t *testing.T) {
ppf := buildParsedPolicy(t,
`["eq","$bucket","mybucket"],["starts-with","$x-amz-meta-",""]`,
)
form := http.Header{}
form.Set("Bucket", "mybucket")
form.Set("X-Amz-Storage-Class", "STANDARD")
err := CheckPostPolicy(form, ppf)
if err == nil {
t.Fatalf("expected error for X-Amz-Storage-Class not covered by meta prefix, got nil")
}
if !strings.Contains(err.Error(), "X-Amz-Storage-Class") {
t.Fatalf("expected error to name X-Amz-Storage-Class, got: %v", err)
}
}
// TestCheckPostPolicy_PrefixStemEnforcesValuePrefix covers a starts-with
// prefix-stem policy with a non-empty required value prefix: matching
// fields must have values that satisfy the value prefix.
func TestCheckPostPolicy_PrefixStemEnforcesValuePrefix(t *testing.T) {
ppf := buildParsedPolicy(t,
`["eq","$bucket","mybucket"],["starts-with","$x-amz-meta-","pfx-"]`,
)
// Value satisfies the required prefix: accepted.
okForm := http.Header{}
okForm.Set("Bucket", "mybucket")
okForm.Set("X-Amz-Meta-Foo", "pfx-bar")
if err := CheckPostPolicy(okForm, ppf); err != nil {
t.Fatalf("expected no error when meta value matches required prefix, got: %v", err)
}
// Value does not satisfy the required prefix: rejected as policy failure.
badForm := http.Header{}
badForm.Set("Bucket", "mybucket")
badForm.Set("X-Amz-Meta-Foo", "other")
err := CheckPostPolicy(badForm, ppf)
if err == nil {
t.Fatalf("expected error when meta value misses required prefix, got nil")
}
if !strings.Contains(err.Error(), "Policy Condition failed") {
t.Fatalf("expected 'Policy Condition failed' error, got: %v", err)
}
if !strings.Contains(err.Error(), "pfx-") {
t.Fatalf("expected error to reference the required value prefix, got: %v", err)
}
}
// TestCheckPostPolicy_ExactAndPrefixBothEnforced covers a field that is
// simultaneously covered by an exact-key condition and a prefix-stem
// condition. Both must be satisfied; exact-match alone does not let the
// field skip the stem's value-prefix check.
func TestCheckPostPolicy_ExactAndPrefixBothEnforced(t *testing.T) {
ppf := buildParsedPolicy(t,
`["eq","$bucket","mybucket"],`+
`["eq","$x-amz-meta-tag","gold"],`+
`["starts-with","$x-amz-meta-","lvl-"]`,
)
form := http.Header{}
form.Set("Bucket", "mybucket")
// Satisfies the exact eq condition but not the starts-with stem.
form.Set("X-Amz-Meta-Tag", "gold")
err := CheckPostPolicy(form, ppf)
if err == nil {
t.Fatalf("expected error: exact-match field must still satisfy overlapping prefix stem, got nil")
}
if !strings.Contains(err.Error(), "Policy Condition failed") {
t.Fatalf("expected 'Policy Condition failed' error, got: %v", err)
}
if !strings.Contains(err.Error(), "lvl-") {
t.Fatalf("expected error to reference the stem's value prefix, got: %v", err)
}
}
// TestCheckPostPolicy_MultiplePrefixStemsAllEnforced covers a field that
// matches multiple starts-with prefix stems. All stems must hold; a value
// that satisfies one but not the other must be rejected.
func TestCheckPostPolicy_MultiplePrefixStemsAllEnforced(t *testing.T) {
ppf := buildParsedPolicy(t,
`["eq","$bucket","mybucket"],`+
`["starts-with","$x-amz-meta-","pfx-"],`+
`["starts-with","$x-amz-meta-color-","red-"]`,
)
// Satisfies the broader stem but not the color- stem's value prefix.
form := http.Header{}
form.Set("Bucket", "mybucket")
form.Set("X-Amz-Meta-Color-Main", "pfx-green")
err := CheckPostPolicy(form, ppf)
if err == nil {
t.Fatalf("expected error: field must satisfy every matching prefix stem, got nil")
}
if !strings.Contains(err.Error(), "Policy Condition failed") {
t.Fatalf("expected 'Policy Condition failed' error, got: %v", err)
}
if !strings.Contains(err.Error(), "red-") {
t.Fatalf("expected error to reference the failing value prefix, got: %v", err)
}
// A policy with compatible stems (broad allows anything, narrow
// requires "red-") accepts a value honoring both.
compatible := buildParsedPolicy(t,
`["eq","$bucket","mybucket"],`+
`["starts-with","$x-amz-meta-",""],`+
`["starts-with","$x-amz-meta-color-","red-"]`,
)
okForm := http.Header{}
okForm.Set("Bucket", "mybucket")
okForm.Set("X-Amz-Meta-Color-Main", "red-main")
if err := CheckPostPolicy(okForm, compatible); err != nil {
t.Fatalf("expected no error when value satisfies both compatible stems, got: %v", err)
}
}
// TestCheckPostPolicy_UnknownKeyErrorIncludesPolicyValue ensures the
// unknown-condition error surfaces the policy value in its [op, key, value]
// trailer so operators can tell which of several unknown keys failed.
func TestCheckPostPolicy_UnknownKeyErrorIncludesPolicyValue(t *testing.T) {
ppf := buildParsedPolicy(t, `["eq","$foo","custom-value"]`)
err := CheckPostPolicy(http.Header{}, ppf)
if err == nil {
t.Fatalf("expected error for unknown condition key, got nil")
}
if !strings.Contains(err.Error(), "custom-value") {
t.Fatalf("expected error to include policy.Value 'custom-value', got: %v", err)
}
if !strings.Contains(err.Error(), "$foo") {
t.Fatalf("expected error to include policy.Key '$foo', got: %v", err)
}
if !strings.Contains(err.Error(), "unknown condition key") {
t.Fatalf("expected 'unknown condition key' suffix, got: %v", err)
}
}

View File

@@ -0,0 +1,343 @@
package policy
/*
* MinIO Cloud Storage, (C) 2015, 2016, 2017 MinIO, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"reflect"
"strconv"
"strings"
"time"
)
// startWithConds - map which indicates if a given condition supports starts-with policy operator
var startsWithConds = map[string]bool{
"$acl": true,
"$bucket": false,
"$cache-control": true,
"$content-type": true,
"$content-disposition": true,
"$content-encoding": true,
"$expires": true,
"$key": true,
"$success_action_redirect": true,
"$redirect": true,
"$success_action_status": false,
"$x-amz-algorithm": false,
"$x-amz-credential": false,
"$x-amz-date": false,
}
// Add policy conditionals.
const (
policyCondEqual = "eq"
policyCondStartsWith = "starts-with"
policyCondContentLength = "content-length-range"
)
// toString - Safely convert interface to string without causing panic.
func toString(val interface{}) string {
switch v := val.(type) {
case string:
return v
default:
return ""
}
}
// toLowerString - safely convert interface to lower string
func toLowerString(val interface{}) string {
return strings.ToLower(toString(val))
}
// toInteger _ Safely convert interface to integer without causing panic.
func toInteger(val interface{}) (int64, error) {
switch v := val.(type) {
case float64:
return int64(v), nil
case int64:
return v, nil
case int:
return int64(v), nil
case string:
i, err := strconv.Atoi(v)
return int64(i), err
default:
return 0, errors.New("Invalid number format")
}
}
// isString - Safely check if val is of type string without causing panic.
func isString(val interface{}) bool {
_, ok := val.(string)
return ok
}
// ContentLengthRange - policy content-length-range field.
type contentLengthRange struct {
Min int64
Max int64
Valid bool // If content-length-range was part of policy
}
// PostPolicyForm provides strict static type conversion and validation for Amazon S3's POST policy JSON string.
type PostPolicyForm struct {
Expiration time.Time // Expiration date and time of the POST policy.
Conditions struct { // Conditional policy structure.
Policies []struct {
Operator string
Key string
Value string
}
ContentLengthRange contentLengthRange
}
}
// ParsePostPolicyForm - Parse JSON policy string into typed PostPolicyForm structure.
func ParsePostPolicyForm(policy string) (ppf PostPolicyForm, e error) {
// Convert po into interfaces and
// perform strict type conversion using reflection.
var rawPolicy struct {
Expiration string `json:"expiration"`
Conditions []interface{} `json:"conditions"`
}
err := json.Unmarshal([]byte(policy), &rawPolicy)
if err != nil {
return ppf, err
}
parsedPolicy := PostPolicyForm{}
// Parse expiry time.
parsedPolicy.Expiration, err = time.Parse(time.RFC3339Nano, rawPolicy.Expiration)
if err != nil {
return ppf, err
}
// Parse conditions.
for _, val := range rawPolicy.Conditions {
switch condt := val.(type) {
case map[string]interface{}: // Handle key:value map types.
for k, v := range condt {
if !isString(v) { // Pre-check value type.
// All values must be of type string.
return parsedPolicy, fmt.Errorf("Unknown type %s of conditional field value %s found in POST policy form", reflect.TypeOf(condt).String(), condt)
}
// {"acl": "public-read" } is an alternate way to indicate - [ "eq", "$acl", "public-read" ]
// In this case we will just collapse this into "eq" for all use cases.
parsedPolicy.Conditions.Policies = append(parsedPolicy.Conditions.Policies, struct {
Operator string
Key string
Value string
}{
policyCondEqual, "$" + strings.ToLower(k), toString(v),
})
}
case []interface{}: // Handle array types.
if len(condt) != 3 { // Return error if we have insufficient elements.
return parsedPolicy, fmt.Errorf("Malformed conditional fields %s of type %s found in POST policy form", condt, reflect.TypeOf(condt).String())
}
switch toLowerString(condt[0]) {
case policyCondEqual, policyCondStartsWith:
for _, v := range condt { // Pre-check all values for type.
if !isString(v) {
// All values must be of type string.
return parsedPolicy, fmt.Errorf("Unknown type %s of conditional field value %s found in POST policy form", reflect.TypeOf(condt).String(), condt)
}
}
operator, matchType, value := toLowerString(condt[0]), toLowerString(condt[1]), toString(condt[2])
if !strings.HasPrefix(matchType, "$") {
return parsedPolicy, fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]", operator, matchType, value)
}
parsedPolicy.Conditions.Policies = append(parsedPolicy.Conditions.Policies, struct {
Operator string
Key string
Value string
}{
operator, matchType, value,
})
case policyCondContentLength:
min, err := toInteger(condt[1])
if err != nil {
return parsedPolicy, err
}
max, err := toInteger(condt[2])
if err != nil {
return parsedPolicy, err
}
parsedPolicy.Conditions.ContentLengthRange = contentLengthRange{
Min: min,
Max: max,
Valid: true,
}
default:
// Condition should be valid.
return parsedPolicy, fmt.Errorf("Unknown type %s of conditional field value %s found in POST policy form",
reflect.TypeOf(condt).String(), condt)
}
default:
return parsedPolicy, fmt.Errorf("Unknown field %s of type %s found in POST policy form",
condt, reflect.TypeOf(condt).String())
}
}
return parsedPolicy, nil
}
// checkPolicyCond returns a boolean to indicate if a condition is satisfied according
// to the passed operator
func checkPolicyCond(op string, input1, input2 string) bool {
switch op {
case policyCondEqual:
return input1 == input2
case policyCondStartsWith:
return strings.HasPrefix(input1, input2)
}
return false
}
// xAmzPrefixRule captures a starts-with policy condition whose key ends in
// "-" and therefore matches a prefix of form field names, e.g.
// ["starts-with","$x-amz-meta-","pfx-"]. Any form field whose name starts
// with namePrefix must have a value starting with valuePrefix.
type xAmzPrefixRule struct {
policyKey string
namePrefix string
valuePrefix string
}
// postPolicyAuthFields enumerates the X-Amz-* form fields that clients are
// required to send with every POST Object request for signing/authentication.
// These must be accepted even when no matching policy condition is declared.
var postPolicyAuthFields = map[string]bool{
"X-Amz-Signature": true,
"X-Amz-Credential": true,
"X-Amz-Algorithm": true,
"X-Amz-Date": true,
"X-Amz-Security-Token": true,
}
// CheckPostPolicy - apply policy conditions and validate input values.
// (http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html)
func CheckPostPolicy(formValues http.Header, postPolicyForm PostPolicyForm) error {
// Check if policy document expiry date is still not reached
if !postPolicyForm.Expiration.After(time.Now().UTC()) {
return fmt.Errorf("Invalid according to Policy: Policy expired")
}
// Track $x-amz-* policy conditions in canonical form. A starts-with
// condition on a key that itself ends with "-" (e.g.
// ["starts-with","$x-amz-meta-","pfx-"]) is the AWS convention for
// allowing any form field sharing that name prefix; capture the name
// prefix together with its required value prefix so the extras check
// can both accept matching fields and enforce the value constraint.
policyXAmzKeys := make(map[string]bool)
var policyXAmzPrefixes []xAmzPrefixRule
for _, policy := range postPolicyForm.Conditions.Policies {
if !strings.HasPrefix(policy.Key, "$x-amz-") {
continue
}
formCanonicalName := http.CanonicalHeaderKey(strings.TrimPrefix(policy.Key, "$"))
if policy.Operator == policyCondStartsWith && strings.HasSuffix(policy.Key, "-") {
policyXAmzPrefixes = append(policyXAmzPrefixes, xAmzPrefixRule{
policyKey: policy.Key,
namePrefix: formCanonicalName,
valuePrefix: policy.Value,
})
continue
}
policyXAmzKeys[formCanonicalName] = true
}
// Reject any X-Amz-* form field that has no matching policy condition,
// except for the reserved auth/signing fields clients must always send.
// A field may be covered by an exact-key condition, by any number of
// prefix-stem rules, or both. Check every applicable rule: AWS requires
// all matching conditions to be satisfied, so exact-match coverage does
// not let a field skip the value-prefix enforcement of overlapping
// starts-with stems, and multiple stems on the same field must all hold.
// Prefix-stem conditions are then skipped in the main loop below because
// no single form field corresponds to the prefix itself.
for key := range formValues {
if !strings.HasPrefix(key, "X-Amz-") {
continue
}
if postPolicyAuthFields[key] {
continue
}
matched := policyXAmzKeys[key]
for _, rule := range policyXAmzPrefixes {
if !strings.HasPrefix(key, rule.namePrefix) {
continue
}
matched = true
if !strings.HasPrefix(formValues.Get(key), rule.valuePrefix) {
return fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]", policyCondStartsWith, rule.policyKey, rule.valuePrefix)
}
}
if !matched {
return fmt.Errorf("Invalid according to Policy: Extra input fields: %s", key)
}
}
// Flag to indicate if all policies conditions are satisfied
var condPassed bool
// Iterate over policy conditions and check them against received form fields
for _, policy := range postPolicyForm.Conditions.Policies {
// Form fields names are in canonical format, convert conditions names
// to canonical for simplification purpose, so `$key` will become `Key`
formCanonicalName := http.CanonicalHeaderKey(strings.TrimPrefix(policy.Key, "$"))
// Operator for the current policy condition
op := policy.Operator
// Prefix-stem x-amz-* conditions (key ending in "-" with starts-with)
// are validated above against every matching form field; skip them
// here so we do not fail on the non-existent literal form field whose
// name equals the prefix itself.
if strings.HasPrefix(policy.Key, "$x-amz-") && op == policyCondStartsWith && strings.HasSuffix(policy.Key, "-") {
continue
}
// If the current policy condition is known
if startsWithSupported, condFound := startsWithConds[policy.Key]; condFound {
// Check if the current condition supports starts-with operator
if op == policyCondStartsWith && !startsWithSupported {
return fmt.Errorf("Invalid according to Policy: Policy Condition failed")
}
// Check if current policy condition is satisfied
condPassed = checkPolicyCond(op, formValues.Get(formCanonicalName), policy.Value)
if !condPassed {
return fmt.Errorf("Invalid according to Policy: Policy Condition failed")
}
} else if strings.HasPrefix(policy.Key, "$x-amz-") {
// Check if policy condition is satisfied
condPassed = checkPolicyCond(op, formValues.Get(formCanonicalName), policy.Value)
if !condPassed {
return fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]", op, policy.Key, policy.Value)
}
} else {
// Unknown condition key: neither in startsWithConds nor a $x-amz-*
// prefixed key. AWS rejects these outright instead of silently
// treating the condition as satisfied.
return fmt.Errorf("Invalid according to Policy: Policy Condition failed: [%s, %s, %s]: unknown condition key", op, policy.Key, policy.Value)
}
}
return nil
}