Uly.me

cloud engineer

AWS S3 Bucket Permission

I was getting this error when downloading a file from a S3 bucket.

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Turns out to be a permission issue. Use –acl bucket-owner-full-control.

# UPLOAD
aws s3 cp file.txt s3://bucket-name/dir/ --acl bucket-owner-full-control
upload: .\file.txt to s3://bucket-name/dir/fw.sh
# DOWNLOAD
aws s3 cp s3://bucket-name/dir/file.txt . --acl bucket-owner-full-control
download: s3://bucket-name/dir/file.txt to .\file.txt

You need to do for both upload and download.

AWS S3 Make Object Public

Copy object or file to S3 bucket.

aws s3 cp filename.ext s3://bucketname/ --profile your-profile

To make it publicly available, run this command.

aws s3api put-object-acl \
--bucket bucket-name \
--key filename.ext \
--acl public-read \
--profile your-profile

AWS S3 Acceleration CLI

Here is how to enable Amazon S3 Transfer Acceleration on a S3 bucket.

aws s3api put-bucket-accelerate-configuration \
--bucket bucketname \
--accelerate-configuration Status=Enabled \
--region us-east-1

Use an accelerate endpoint.

aws configure set default.s3.use_accelerate_endpoint true

To copy files to S3 you can use the default copy.

aws s3 cp file.txt s3://bucketname/keyname \
--region us-east-1

Or use the acceleration endpoint.

aws configure set s3.addressing_style virtual
aws s3 cp file.txt s3://bucketname/keyname \
--endpoint-url http://s3-accelerate.amazonaws.com \
--region us-east-1

AWS S3 Sync Between Accounts

Here’s how to sync S3 buckets between 2 different AWS accounts. Assuming buckets are already created.

  1. Setup bucket permissions in Account A
  2. Setup IAM user with permissions in Account B
  3. Setup bucket permissions in Account B
  4. Run S3 sync from Account B.

Account A bucket permissions. Account and user are from Account B.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DelegateS3Access",
            "Effect": "Allow",
            "Principal": {"AWS": "arn:aws:iam::222222222222:user/Jane"},
            "Action": ["s3:ListBucket","s3:GetObject"],
            "Resource": [
                "arn:aws:s3:::awsexamplesourcebucket/*",
                "arn:aws:s3:::awsexamplesourcebucket"
            ]
        }
    ]
}

Create IAM user (Jane) in Account B

aws iam create-user --user-name Jane

Give IAM user (Jane) access to both buckets.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::awsexamplesourcebucket",
                "arn:aws:s3:::awsexamplesourcebucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::awsexampledestinationbucket",
                "arn:aws:s3:::awsexampledestinationbucket/*"
            ]
        }
    ]
}

Sync the buckets

aws s3 sync s3://awsexamplesourcebucket s3://awsexampledestinationbucket

S3 Browser IAM Assume Role

S3 Browser is a free Windows client for accessing AWS S3 buckets. You can access S3 buckets with an interface similar to File Manager. A typical setup requires that you use access and secret keys to connect to the S3 browser. However, if the server is in the cloud, it’s better that use a machine profile with an assume role. The assume role gets temporary credentials until they expire. They are automatically rotated by AWS behind the scenes for you. This is much safer process than storing key credentials in the cloud. The good news is, S3 Browser support IAM assume roles. Here’s the setup instructions to get your S3 Browser configured using IAM assume role. Here’s IAM role explained.