Buckets are regional. Your source and destination buckets should be in different regions.
aws s3 sync s3://DOC-EXAMPLE-BUCKET-SOURCE s3://DOC-EXAMPLE-BUCKET-TARGET |
Use the sync command. It will copy new or modified files.
Copy S3 To Another Region
s3
S3FS
s3fs allows Linux to mount S3 buckets as a file system.
Install s3fs.
sudo apt install s3fs |
Setup credentials.
echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs chmod 600 ${HOME}/.passwd-s3fs |
Mount it.
s3fs bucketname /mountpoint -o passwd_file=${HOME}/.passwd-s3fs |
Mount it automatically.
bucketname /mountpoint fuse.s3fs _netdev,allow_other,passwd_file=/home/user/.passwd-s3fs,rw,uid=1000,gid=1000 0 0 |
AWS S3 Bucket Permission
I was getting this error when downloading a file from a S3 bucket.
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden |
Turns out to be a permission issue. Use –acl bucket-owner-full-control.
# UPLOAD aws s3 cp file.txt s3://bucket-name/dir/ --acl bucket-owner-full-control upload: .\file.txt to s3://bucket-name/dir/fw.sh # DOWNLOAD aws s3 cp s3://bucket-name/dir/file.txt . --acl bucket-owner-full-control download: s3://bucket-name/dir/file.txt to .\file.txt |
You need to do for both upload and download.
AWS S3 Make Object Public
Copy object or file to S3 bucket.
aws s3 cp filename.ext s3://bucketname/ --profile your-profile |
To make it publicly available, run this command.
aws s3api put-object-acl \ --bucket bucket-name \ --key filename.ext \ --acl public-read \ --profile your-profile |
AWS S3 Acceleration CLI
Here is how to enable Amazon S3 Transfer Acceleration on a S3 bucket.
aws s3api put-bucket-accelerate-configuration \ --bucket bucketname \ --accelerate-configuration Status=Enabled \ --region us-east-1 |
Use an accelerate endpoint.
aws configure set default.s3.use_accelerate_endpoint true |
To copy files to S3 you can use the default copy.
aws s3 cp file.txt s3://bucketname/keyname \ --region us-east-1 |
Or use the acceleration endpoint.
aws configure set s3.addressing_style virtual aws s3 cp file.txt s3://bucketname/keyname \ --endpoint-url http://s3-accelerate.amazonaws.com \ --region us-east-1 |