• Skip to primary navigation
  • Skip to main content

Uly.me

cloud engineer

  • Home
  • About
  • Archives

awk

AWS S3 LS Recursive

by Ulysses · Apr 5, 2020

Here’s how to list S3 files recursively.

aws s3 ls s3://mybucket --recursive | awk '{print $4}'

aws s3 ls s3://mybucket --recursive | awk '{print $4}'

By default, the results display date, time, disk size and filename. Use awk to display the filename only.

Filed Under: Cloud Tagged With: awk, aws, ls, recursively, s3

GCP Projects List

by Ulysses · Mar 26, 2020

Here’s how to get a list of GCP projects.

gcloud projects list

gcloud projects list

Result:

PROJECT_ID            NAME     PROJECT_NUMBER
your-project-id-xxxx  servers  xxxxxxxxxxxx

PROJECT_ID NAME PROJECT_NUMBER your-project-id-xxxx servers xxxxxxxxxxxx

Use awk to display the project id only.

gcloud projects list | awk '{print $1}'

gcloud projects list | awk '{print $1}'

Result

PROJECT ID
your-project-id-xxxx

PROJECT ID your-project-id-xxxx

Filed Under: Cloud Tagged With: awk, gcp, id, list, projects

Clean Duplicates in File

by Ulysses · Dec 9, 2018

Here’s a nice command to get rid of line duplicates in a file. I’m sending the output to a temporary file then renaming it back to the original file. The reason is, it’s probably not a good idea to overwrite a file that you are editing. It’s much safer to send the output to another file.

$ awk '!seen[$0]++' backup.sh > backup01.sh
$ mv backup01.sh backup.sh

$ awk '!seen[$0]++' backup.sh > backup01.sh $ mv backup01.sh backup.sh

Filed Under: Linux Tagged With: awk, duplicates

Copyright © 2012–2021