Uly.me

cloud engineer

s3

Python Boto3 S3 List Objects

How to list S3 objects using Python Boto3

import boto3
from sys import argv
bucket=argv[1]
s3 = boto3.resource('s3')
mybucket = s3.Bucket(bucket)
def list_buckets():
    print("List of Buckets: ")
    for bucket in s3.buckets.all():
        print(bucket.name)
def list_files():
    print("Buckets: The objects in the \"" + bucket + "\" buckets are ... ")
    for files in mybucket.objects.all():
        print(files.key)
def main():
    #list_buckets()
    list_files()
if __name__ == "__main__":
    main()

Running command

$ python test.py bucket-name    # with python
$ ./test.py bucket-name         # without python

Results

Buckets: The objects in the "bucket-name" are ....
abc.txt
def.txt

Terraform AWS S3

How to create S3 bucket via Terraform.

erraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}
 
provider "aws" {
  profile = "default"
  region  = "us-east-1"
}
 
resource "aws_s3_bucket" "bucket" {
  bucket = "my-ulysses-bucket"
  acl    = "private"
 
  tags = {
    Name        = "My Ulysses bucket"
    Environment = "Dev"
  }
}
 
resource "aws_s3_bucket_public_access_block" "example" {
  bucket = aws_s3_bucket.bucket.id
  block_public_acls = true
  block_public_policy = true
  ignore_public_acls = true
  restrict_public_buckets = true
}

AWS Set Profile

If you find yourself using a named profile for every awscli command, you can set it temporarily.

Listing a bucket with a named profile.

aws s3 ls --profile=yourprofile

If you set AWS_PROFILE, you can then list a bucket without a named profile.

export AWS_PROFILE=yourprofile
aws s3 ls

After you are done, you can reset it back to default.

export AWS_PROFILE=default