Uly.me

cloud engineer

snapshot

AWS Create Volume From Snapshot with Tags

Here’s another script that creates a volume from a snapshot, but also add the tags.

#!/bin/bash
read -p "server     : " server
read -p "volumeId   : " volume
read -p "snapshotId : " snapshot
read -p "region     : " region
read -p "zone       : " zone
read -p "profile    : " profile
# get tags
tags1=$(aws ec2 describe-volumes --volume-ids $volume --query 'Volumes[].Tags[]' --region $region --profile $profile)
# remove quotes
tags2=$(echo "$tags1" | tr -d '"')
# remove spaces
tags3=$(echo $tags2 | sed 's/ //g')
# replace : with =
tags4=$(echo $tags3 | sed 's/:/=/g')
# if empty value replace with quotes
tags5=$(echo $tags4 | sed 's/Value=}/Value=""}/g')
# create volume
aws ec2 create-volume \
--availability-zone $zone \
--encrypted \
--iops 3000 \
--volume-type gp3 \
--snapshot-id $snapshot \
--tag-specifications 'ResourceType=volume,Tags='$tags5'' \
--region $region \
--profile $profile

AWS Create Volume From Snapshot

Here’s a bash script that creates a volume from a snapshot in AWS.

#!/bin/bash
read -p "snapshotId : " snapshot
read -p "server     : " server
read -p "tag1       : " tag1
read -p "tag2       : " tag2
read -p "region     : " region
read -p "zone       : " zone
read -p "profile    : " profile
aws ec2 create-volume \
--availability-zone $zone \
--encrypted \
--iops 3000 \
--volume-type gp3 \
--snapshot-id $snapshot \
--tag-specifications "ResourceType=volume,Tags=[{Key=Name,Value="$server"},{Key=tag1,Value="$tag1"},{Key=tag2,Value="$tag2"}]" \
--region $region \
--profile $profile

GCP Create Scheduled Snapshots

How to create scheduled snapshots in GCP.

gcloud compute resource-policies create snapshot-schedule hourly \
--description "my hourly schedule" \
--max-retention-days 7 \
--start-time 00:00 \
--hourly-schedule 1 \
--region us-central1 \
--on-source-disk-delete keep-auto-snapshots \
--storage-location US

Add snapshot schedule to a disk.

gcloud compute disks add-resource-policies disk-name \
--resource-policies hourly \
--zone us-central1-a
gcloud compute disks create disk-name \
--resource-policies hourly \
--zone us-central1-a

List snapshot schedules.

gcloud compute resource-policies list

Describe snapshot schedule.

gcloud compute resource-policies describe hourly

AWS Copy Snapshot to another Region

If you need to copy a snapshot from one region to another, here’s the AWS CLI command.

aws ec2 copy-snapshot \
    --region us-east-1 \
    --source-region us-east-2 \
    --source-snapshot-id snap-xxxxxxxxxxxxxxxxx \
    --description "This is the DR snapshot copy"

Output:

{
    "SnapshotId": "snap-xxxxxxxxxxxxxxxxx"
}

Create Instance With Alias IP

How to create an instance from a snapshot with alias IP and reserved IPs.

#!/bin/bash
gcloud beta compute instances create jump-server \
--network-interface=aliases=10.128.1.0/24,private-network-ip=jump-server,subnet=default \
--machine-type=n1-standard-1 \``
--network-tier=PREMIUM \
--maintenance-policy=MIGRATE \
--service-account=xxxxxxxxxxxxx-compute@developer.gserviceaccount.com \
--tags=int-webserver \
--image=debian-10-buster-v20201216 \
--image-project=debian-cloud \
--boot-disk-size=20GB \
--boot-disk-type=pd-standard \
--boot-disk-device-name=jump-server-1 \
--no-shielded-secure-boot \
--shielded-vtpm \
--shielded-integrity-monitoring \
--labels=name=jump-server \
--reservation-affinity=any \
--zone=us-central1-a \
--project=airy-totality-151318

The default command uses –private-network-ip and –subnet options separately.

--private-network-ip 10.0.0.24 \
--subnet=default \

But when dealing with aliases, reserved IPs and subnets, use a single –network-interface option instead.

--network-interface=aliases=10.128.1.0/24,private-network-ip=jump-server,subnet=default \

AWS RDS Backup Permission

Here’s the IAM policy to allow RDS Backup or create a snapshot on AWS.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "rds:RestoreDBClusterFromSnapshot",
		"rds:DescribeDBSnapshots",
		"rds:CopyDBSnapshot",
		"rds:CopyDBClusterSnapshot",
		"rds:DeleteDBSnapshot",
		"rds:DeleteDBClusterSnapshot",
		"rds:CreateDBSnapshot",
		"rds:RestoreDBInstanceFromDBSnapshot",
		"rds:CreateDBInstance",
		"rds:DescribeDBClusterSnapshots",
		"rds:DescribeDBInstances",
		"rds:DescribeDBClusters",
		"rds:DeleteDBInstance",
		"rds:CreateDBClusterSnapshot",
		"rds:ModifyDBSnapshotAttribute",
		"rds:ModifyDBClusterSnapshotAttribute",
		"rds:ListTagsForResource",
		"rds:DeleteDBCluster",
		"ec2:DescribeSecurityGroups",
		"ec2:DescribeRegions",
		"ec2:DescribeAvailabilityZones",
		"ec2:DescribeVpcs",
		"ec2:DescribeAccountAttributes",
		"ec2:DescribeSubnets",
		"iam:GetUser",
		"iam:GetAccountAuthorizationDetails",
		"kms:ReEncrypt*",
		"kms:GenerateDataKey*",
		"kms:CreateGrant",
		"kms:DescribeKey*",
		"kms:ListKeys",
		"kms:ListAliases",
		"kms:Encrypt",
		"kms:Decrypt",
		"kms:GenerateDataKeyWithoutPlaintext",
		"kms:ListKeys",
		"kms:ListAliases",
		"kms:ListResourceTags"
            ],
            "Resource": "*"
        }
    ]
}

GCP Create Instance From Snapshot

There are two steps in creating an instance from a snapshot.

  1. Create a disk from snapshot
  2. Create an instance from the disk

Create a disk from snapshot.

gcloud compute disks create "hostname-boot" \
--project "project-id" \
--zone "us-central1-a" \
--source-snapshot "snapshot-name" \
--type "pd-standard" \
--size "100"

Create an instance from disk.

gcloud beta compute instances create hostname \
--project=project-id \
--zone=us-central1-a \
--subnet=your-subnetwork \
--machine-type=n1-standard-1 \
--no-address \
--maintenance-policy=MIGRATE \
--service-account=service.account@developer.gserviceaccount.com \
--disk=name=instance-1,device-name=instance-1,mode=rw,boot=yes,auto-delete=yes \
--reservation-affinity=any \
--labels=builtby=john.doe \
--tags=web \
--scopes= \
--metadata=

GCP Convert Standard to SSD

How to convert Persistent Standard disk to SSD.

First create a snapshot of the disk.

gcloud compute disks snapshot your-server \
--snapshot-names manual-snapshot-disk-1 \
--project your-project \
--zone us-central1-c

Restore snapshot to SSD format.

gcloud compute disks create disk-1-ssd \
--source-snapshot manual-snapshot-disk-1 \
--project your-project \
--zone us-central1-a \
--type pd-ssd \
--size 10GB

Swap disks using GCP attach and detach.