• Skip to main content

Uly.me

cloud engineer

  • Home
  • About
  • Archives

Archives for September 2019

GCP Service Accounts

September 25, 2019

Here’s how display the Service Account of a particular instance in Google Cloud.

gcloud compute instances describe server-name \
--zone us-central1-c \
--project project-id \
--format="flattened(serviceAccounts[].email)"

gcloud compute instances describe server-name \ --zone us-central1-c \ --project project-id \ --format="flattened(serviceAccounts[].email)"

Result is:

serviceAccounts[0].email: service-account-name@project-id.iam.gserviceaccount.com

serviceAccounts[0].email: service-account-name@project-id.iam.gserviceaccount.com

Filed Under: Cloud Tagged With: compute, describe, gcp, google, instances, service account

Elapse Time on Bash Script

September 24, 2019

Here’s a nice little Bash function in a script to display the elapse time. It’s a nice function for showing how long a process ran.

#!/bin/bash
# pass number of seconds as argument. 
# Example below calculates 1000s.
# elapse.sh 1000
function show_time () {
    num=$1
    min=0
    hour=0
    day=0
    if((num>59));then
        ((sec=num%60))
        ((num=num/60))
        if((num>59));then
            ((min=num%60))
            ((num=num/60))
            if((num>23));then
                ((hour=num%24))
                ((day=num/24))
            else
                ((hour=num))
            fi
        else
            ((min=num))
        fi
    else
        ((sec=num))
    fi
    echo "$day"d "$hour"h "$min"m "$sec"s
}
show_time $1

#!/bin/bash # pass number of seconds as argument. # Example below calculates 1000s. # elapse.sh 1000 function show_time () { num=$1 min=0 hour=0 day=0 if((num>59));then ((sec=num%60)) ((num=num/60)) if((num>59));then ((min=num%60)) ((num=num/60)) if((num>23));then ((hour=num%24)) ((day=num/24)) else ((hour=num)) fi else ((min=num)) fi else ((sec=num)) fi echo "$day"d "$hour"h "$min"m "$sec"s } show_time $1

Filed Under: Linux Tagged With: bash, days, elapse, hours, mins, process, seconds, time

Fpsync

September 23, 2019

Fpsync is command line tool for synchronizing directories in parallel using fpart and rsync tools. You can specify a number of concurrent sync jobs, number of files per sync job, and the maximum byte size per sync among other things. Fpsync is believed to be 4 to 5 times faster than rsync. Fpsync makes sense when syncing massive drives with thousands of directories and small files.

To install fpsync.

apt install fpart

apt install fpart

Fpsync with 8 parallel jobs.

log='/root/fpsync.log'
fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log

log='/root/fpsync.log' fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log

A sample Script with timestamps to display elapse time.

#!/bin/bash
log='/root/fpsync.log'
start=$(date)
begin=$(date +%s)
echo 'Start: '$start > $log
fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log
stop=$(date)
end=$(date +%s)
echo 'Stop: '$stop >> $log
elapse=$((end-begin))
 
function show_time () {
    num=$elapse
    min=0
    hour=0
    day=0
    if((num>59));then
        ((sec=num%60))
        ((num=num/60))
        if((num>59));then
            ((min=num%60))
            ((num=num/60))
            if((num>23));then
                ((hour=num%24))
                ((day=num/24))
            else
                ((hour=num))
            fi
        else
            ((min=num))
        fi
    else
        ((sec=num))
    fi
    echo "$day"d "$hour"h "$min"m "$sec"s
}
show_time $elapse >> $log

#!/bin/bash log='/root/fpsync.log' start=$(date) begin=$(date +%s) echo 'Start: '$start > $log fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log stop=$(date) end=$(date +%s) echo 'Stop: '$stop >> $log elapse=$((end-begin)) function show_time () { num=$elapse min=0 hour=0 day=0 if((num>59));then ((sec=num%60)) ((num=num/60)) if((num>59));then ((min=num%60)) ((num=num/60)) if((num>23));then ((hour=num%24)) ((day=num/24)) else ((hour=num)) fi else ((min=num)) fi else ((sec=num)) fi echo "$day"d "$hour"h "$min"m "$sec"s } show_time $elapse >> $log

For comparison, you can substitute fpsync with rysnc and see the performance difference.

fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log
# or
rsync -av /root/tmp1 /root/tmp2/ > /var/null

fpsync -n 8 -v /root/tmp1/ /root/tmp2/ >> $log # or rsync -av /root/tmp1 /root/tmp2/ > /var/null

Filed Under: Cloud, Linux Tagged With: concurrent, directories, fpart, fpsync, parallel, rsync, sync

GCP Create Service Account Key

September 22, 2019

Here’s how to create a key for the GCP Service Account.

Create Key.

gcloud iam service-accounts keys create ~/key.json \
  --iam-account service-name@project-id.iam.gserviceaccount.com

gcloud iam service-accounts keys create ~/key.json \ --iam-account service-name@project-id.iam.gserviceaccount.com

Activate Key

gcloud auth activate-service-account service-name@project-id.iam.gserviceaccount.com \
--key-file=key.json

gcloud auth activate-service-account service-name@project-id.iam.gserviceaccount.com \ --key-file=key.json

Revoke Key.

gcloud auth revoke service-account@project-id.iam.gserviceaccount.com

gcloud auth revoke service-account@project-id.iam.gserviceaccount.com

Delete Key

gcloud iam service-accounts keys delete key-id \
    --iam-account service-name@project-id.iam.gserviceaccount.com

gcloud iam service-accounts keys delete key-id \ --iam-account service-name@project-id.iam.gserviceaccount.com

Filed Under: Cloud Tagged With: activate, add, create, delete, gcp, key, revoke, service account

GCP Create Service Account

September 22, 2019

Here’s how to create a GCP Service Account.

gcloud beta iam service-accounts create service-account-name \
    --description "service account description" \
    --display-name "service account display name"

gcloud beta iam service-accounts create service-account-name \ --description "service account description" \ --display-name "service account display name"

Filed Under: Cloud Tagged With: create, gcp, iam, service account

GCP Add IAM Role to Service Account

September 22, 2019

Here’s how to add an IAM role to a GCP Service Account.

gcloud projects add-iam-policy-binding your-project-id \
--member serviceAccount:service-account-name@your-project-id.iam.gserviceaccount.com \
--role roles/storage.objectAdmin

gcloud projects add-iam-policy-binding your-project-id \ --member serviceAccount:service-account-name@your-project-id.iam.gserviceaccount.com \ --role roles/storage.objectAdmin

Filed Under: Cloud Tagged With: add, gcp, iam, policy, projects, role

GCP SDK Update

September 22, 2019

Here’s how to update your Google Cloud Platform SDK (Software Development Kit) which is aka the CLI for interacting with Google’s API.

Get Current Version

gcloud version

gcloud version

Get the Components List

gcloud components list

gcloud components list

Update

gcloud components update

gcloud components update

Filed Under: Cloud Tagged With: cli, components, gcp, sdk, update

GCP Compute Disks Move

September 21, 2019

Here’s how to move a GCP disk between zones.

  1. Make sure disk is detached from instance.
  2. Use the GCloud Disk Move command.
# Detach disk from instance
gcloud compute instances detach-disk server --disk=server-boot
# Move disk from us-central1-b to us-central1-f
gcloud compute disks move example-disk-1 --zone us-central1-b --destination-zone us-central1-f

# Detach disk from instance gcloud compute instances detach-disk server --disk=server-boot # Move disk from us-central1-b to us-central1-f gcloud compute disks move example-disk-1 --zone us-central1-b --destination-zone us-central1-f

Filed Under: Misc

  • Go to page 1
  • Go to page 2
  • Go to page 3
  • Go to page 4
  • Go to Next Page »
  • Home
  • About
  • Archives

Copyright © 2023