cloud engineer
s3fs allows Linux to mount S3 buckets as a file system.
Install s3fs.
sudo apt install s3fs |
sudo apt install s3fs
Setup credentials.
echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs chmod 600 ${HOME}/.passwd-s3fs |
echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs chmod 600 ${HOME}/.passwd-s3fs
Mount it.
s3fs bucketname /mountpoint -o passwd_file=${HOME}/.passwd-s3fs |
s3fs bucketname /mountpoint -o passwd_file=${HOME}/.passwd-s3fs
Mount it automatically.
bucketname /mountpoint fuse.s3fs _netdev,allow_other,passwd_file=/home/user/.passwd-s3fs,rw,uid=1000,gid=1000 0 0 |
bucketname /mountpoint fuse.s3fs _netdev,allow_other,passwd_file=/home/user/.passwd-s3fs,rw,uid=1000,gid=1000 0 0
I was getting this error when downloading a file from a S3 bucket.
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden |
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
Turns out to be a permission issue. Use –acl bucket-owner-full-control.
# UPLOAD aws s3 cp file.txt s3://bucket-name/dir/ --acl bucket-owner-full-control upload: .\file.txt to s3://bucket-name/dir/fw.sh # DOWNLOAD aws s3 cp s3://bucket-name/dir/file.txt . --acl bucket-owner-full-control download: s3://bucket-name/dir/file.txt to .\file.txt |
# UPLOAD aws s3 cp file.txt s3://bucket-name/dir/ --acl bucket-owner-full-control upload: .\file.txt to s3://bucket-name/dir/fw.sh # DOWNLOAD aws s3 cp s3://bucket-name/dir/file.txt . --acl bucket-owner-full-control download: s3://bucket-name/dir/file.txt to .\file.txt
You need to do for both upload and download.
Here are the commands to find out the bucket size in GCP.
gsutil du -s gs://bucket-name/ |
gsutil du -s gs://bucket-name/