Uly.me

cloud engineer

gcp

GCP Spot VM

by

Spot VMs are preemtible, they can be reclaimed by the Cloud provider.

However you can get deep discounts using them.

Launch a spot instance in GCP using Terraform.

provider "google" {
  project = "airy-totality-151318"
  zone    = "us-central1-c"
}
 
resource "google_compute_instance" "test" {
  name         = "test"
  machine_type = "e2-micro"
 
  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
    }
  }
 
  scheduling {
    preemptible       = true
    automatic_restart = false
  }
 
  network_interface {
    network = "default"
    access_config {
    }
  }
}

GCP Fix Hosts

by

This is a script that fixes /etc/hosts in GCP.

#!/bin/bash
 
# VARIABLES
FILE='/etc/hosts'
 
#BACKUP /etc/hosts before changes
cp -ipHR $FILE $FILE_$(date +%Y-%m-%d_%H-%M-%S)
 
# GET IP ADDRESS
IP=$(/sbin/ifconfig | grep 'inet' | grep -v 'inet6' | grep -v '127.0' | cut -d: -f2 | awk '{print $2}' | head -1)
 
# GET HOSTNAME
HN=$(/bin/hostname)
 
# GET PROJECT ID
echo "169.254.169.254 metadata.google.internal  # Added by Google" >> $FILE
PR=$(curl -f -s http://metadata.google.internal/computeMetadata/v1/project/project-id -H "Metadata-Flavor: Google")
 
# ADD HOSTNAME ENTRY TO /etc/hosts
STRING="$IP $HN.mydomain.com"
 
if grep -q "$STRING" "$FILE"
then
  echo "Valid host entry."
else
  echo "Adding host entry."
  echo $STRING $HN >> $FILE
fi
 
# DELETE GOOGLE ENTRIES.
sed -i '/# Added by Google/d' $FILE
 
# ADD BACK GOOGLE ENTRIES.
echo $IP $HN".c."$PR".internal" $HN"  # Added by Google" >> $FILE
echo "169.254.169.254 metadata.google.internal  # Added by Google" >> $FILE
 
# SET PERMISSIONS.
chmod 644 $FILE
 
# DISPLAY /etc/hosts
cat $FILE

Result

127.0.0.1       localhost
::1             localhost ip6-localhost ip6-loopback
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters
 
10.128.0.30 test.mydomain.com test
10.128.0.30 test.c.project-id.internal test  # Added by Google
169.254.169.254 metadata.google.internal  # Added by Google

It adds a proper host entry before the Google entries. /etc/hosts has top to bottom precedence.

GCP Spot Instance

by

You can save anywhere from 60-91% using a spot instance. The downside is, the instance can be preempted anytime.

Create a spot instance.

gcloud beta compute instances create spot-example \
--provisioning-model=SPOT \
--instance-termination-action=STOP \
--image-project=ubuntu-os-cloud \
--image-family=ubuntu-2004-lts \
--machine-type=e2-micro \
--project=your-project-id \
--zone=us-central1-a

Delete instance.

gcloud compute instances delete spot-example \
--project=your-project-id \
--zone=us-central1-a

GCP list firewall rule by name

by

Here’s how to list a GCP firewall rule by name.

gcloud compute firewall-rules list \
--filter="name=('wowza-rtmp')" \
--format=json

You can use different formats. Supported formats are:

config, csv, default, diff, disable, flattened, get, json, list, multi, none, object, table, text, value, yaml

Copy S3 to GCS

by

You can use gsutil to copy directly from a S3 bucket to a GCS bucket.

gsutil can read/write to a S3 bucket as long as it has access to your AWS credentials.

The -R option is recursive and -m is for multi-threaded or multi-processing.

gsutil -m cp -R s3://bucket/ gs://bucket

You can also use rsync instead of cp. Just be aware that rsync does not copy empty directory trees

gsutil -m rsync -r s3://bucket/ gs://bucket

If you plan to use -d, Use it with caution since it deletes content.

You may have to set the following to your defaults accounts.

export AWS_PROFILE=yourprofile
gcloud config set project your-project-id

This ensures that default profile and projects are used in both AWS and GCP.