Uly.me

cloud engineer

Home/Archives for s3

AWS CLI Create Bucket

Creating a bucket via the AWS Console takes only a few clicks. If you’re creating a bucket via shell script, you will need to use the AWS CLI (command line interface). So, here’s the script to create a S3 bucket via the AWS CLI within shell. Now the bucket name has to be globally unique similar to a domain name. If someone already has claimed it, you will need to use an alternative name.

aws s3api create-bucket --bucket bucket-name --region us-east-2 create-bucket-configuration LocationConstraint=us-east-2

AWS S3 Killedted

I was trying to copy a large file from AWS S3 bucket to the smallest Amazon Lightsail instance using the AWS CLI. It was throwing out an error saying “Killedted 3.8 Mib/113.4 Mib (5.0 Mib/s) with 1 file(s) remaining.” For some reason, the copy was being killed or terminated. I was thinking that maybe the small instance was not enough to copy or move large files. I’m glad I found a workaround. Here’s the command I was doing, and the fix.

The Error

$ aws s3 cp s3://bucketname/directory/largefile.mp4 largefile.mp4
$ Killedted 3.8 MiB/113.4 MiB (5.0 MiB/s) with 1 file(s) remaining

The Fix

$ aws configure set default.s3.max_concurrent_requests 4

The default is 10 which was too much for a small instance.

Install S3cmd on Ubuntu

If you want to backup your Linux server to Amazon S3 (Simple Storage Service), you need to install a utility called S3cmd which allows you to interact with Amazon’s cloud storage service. You’ll be able to create s3 buckets, upload and retrieve files from your Linux server to the S3.

You can install S3cmd from Ubuntu.

sudo apt-get install s3cmd

Next, you need to configure S3cmd using your S3 credentials. You’ll need an Access key and a Secret key from Amazon’s Security Credentials page which you can access from Amazon’s IAM Management Console.

sudo s3cmd --configure

For details on how to fully implement S3cmd, please take a look at this article.