• Skip to main content

Uly.me

cloud engineer

  • Home
  • About
  • Search

Archives for September 2019

GCP Service Accounts

September 25, 2019

Here’s how display the Service Account of a particular instance in Google Cloud.

gcloud compute instances describe server-name \
--zone us-central1-c \
--project project-id \
--format="flattened(serviceAccounts[].email)"

gcloud compute instances describe server-name \ --zone us-central1-c \ --project project-id \ --format="flattened(serviceAccounts[].email)"

Result is:

serviceAccounts[0].email: service-account-name@project-id.iam.gserviceaccount.com

serviceAccounts[0].email: service-account-name@project-id.iam.gserviceaccount.com

Filed Under: Cloud Tagged With: compute, describe, gcp, google, instances, service account

Elapse Time on Bash Script

September 24, 2019

Here’s a nice little Bash function in a script to display the elapse time. It’s a nice function for showing how long a process ran.

#!/bin/bash
# pass number of seconds as argument. 
# Example below calculates 1000s.
# elapse.sh 1000
function show_time () {
    num=$1
    min=0
    hour=0
    day=0
    if((num>59));then
        ((sec=num%60))
        ((num=num/60))
        if((num>59));then
            ((min=num%60))
            ((num=num/60))
            if((num>23));then
                ((hour=num%24))
                ((day=num/24))
            else
                ((hour=num))
            fi
        else
            ((min=num))
        fi
    else
        ((sec=num))
    fi
    echo "$day"d "$hour"h "$min"m "$sec"s
}
show_time $1

#!/bin/bash # pass number of seconds as argument. # Example below calculates 1000s. # elapse.sh 1000 function show_time () { num=$1 min=0 hour=0 day=0 if((num>59));then ((sec=num%60)) ((num=num/60)) if((num>59));then ((min=num%60)) ((num=num/60)) if((num>23));then ((hour=num%24)) ((day=num/24)) else ((hour=num)) fi else ((min=num)) fi else ((sec=num)) fi echo "$day"d "$hour"h "$min"m "$sec"s } show_time $1

Filed Under: Linux Tagged With: bash, days, elapse, hours, mins, process, seconds, time

Fpsync

September 23, 2019

Fpsync is command line tool for synchronizing directories in parallel using fpart and rsync tools. You can specify a number of concurrent sync jobs, number of files per sync job, and the maximum byte size per sync among other things. Fpsync is believed to be 4 to 5 times faster than rsync. Fpsync makes sense when syncing massive drives with thousands of directories and small files.

To install fpsync.

apt install fpart

apt install fpart

Fpsync with 8 parallel jobs.

log='/root/fpsync.log'
fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log

log='/root/fpsync.log' fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log

A sample Script with timestamps to display elapse time.

#!/bin/bash
log='/root/fpsync.log'
start=$(date)
begin=$(date +%s)
echo 'Start: '$start > $log
fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log
stop=$(date)
end=$(date +%s)
echo 'Stop: '$stop >> $log
elapse=$((end-begin))
 
function show_time () {
    num=$elapse
    min=0
    hour=0
    day=0
    if((num>59));then
        ((sec=num%60))
        ((num=num/60))
        if((num>59));then
            ((min=num%60))
            ((num=num/60))
            if((num>23));then
                ((hour=num%24))
                ((day=num/24))
            else
                ((hour=num))
            fi
        else
            ((min=num))
        fi
    else
        ((sec=num))
    fi
    echo "$day"d "$hour"h "$min"m "$sec"s
}
show_time $elapse >> $log

#!/bin/bash log='/root/fpsync.log' start=$(date) begin=$(date +%s) echo 'Start: '$start > $log fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log stop=$(date) end=$(date +%s) echo 'Stop: '$stop >> $log elapse=$((end-begin)) function show_time () { num=$elapse min=0 hour=0 day=0 if((num>59));then ((sec=num%60)) ((num=num/60)) if((num>59));then ((min=num%60)) ((num=num/60)) if((num>23));then ((hour=num%24)) ((day=num/24)) else ((hour=num)) fi else ((min=num)) fi else ((sec=num)) fi echo "$day"d "$hour"h "$min"m "$sec"s } show_time $elapse >> $log

For comparison, you can substitute fpsync with rysnc and see the performance difference.

fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log
# or
rsync -av /root/tmp1 /root/tmp2/ > /var/null

fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log # or rsync -av /root/tmp1 /root/tmp2/ > /var/null

Filed Under: Cloud, Linux Tagged With: concurrent, directories, fpart, fpsync, parallel, rsync, sync

GCP Create Service Account Key

September 22, 2019

Here’s how to create a key for the GCP Service Account.

Create Key.

gcloud iam service-accounts keys create ~/key.json \
  --iam-account service-name@project-id.iam.gserviceaccount.com

gcloud iam service-accounts keys create ~/key.json \ --iam-account service-name@project-id.iam.gserviceaccount.com

Activate Key

gcloud auth activate-service-account service-name@project-id.iam.gserviceaccount.com \
--key-file=key.json

gcloud auth activate-service-account service-name@project-id.iam.gserviceaccount.com \ --key-file=key.json

Revoke Key.

gcloud auth revoke service-account@project-id.iam.gserviceaccount.com

gcloud auth revoke service-account@project-id.iam.gserviceaccount.com

Delete Key

gcloud iam service-accounts keys delete key-id \
    --iam-account service-name@project-id.iam.gserviceaccount.com

gcloud iam service-accounts keys delete key-id \ --iam-account service-name@project-id.iam.gserviceaccount.com

Filed Under: Cloud Tagged With: activate, add, create, delete, gcp, key, revoke, service account

GCP Create Service Account

September 22, 2019

Here’s how to create a GCP Service Account.

gcloud beta iam service-accounts create service-account-name \
    --description "service account description" \
    --display-name "service account display name"

gcloud beta iam service-accounts create service-account-name \ --description "service account description" \ --display-name "service account display name"

Filed Under: Cloud Tagged With: create, gcp, iam, service account

  • Go to page 1
  • Go to page 2
  • Go to page 3
  • Interim pages omitted …
  • Go to page 6
  • Go to Next Page »
  • Home
  • About
  • Search

Copyright © 2023