cloud engineer
Here’s the script to backup GCP disks.
#!/bin/bash now=$(date +%s) disks=$(gcloud compute disks list --project project-id --filter="users:instance" --format="value(name)") for disk in $disks do gcloud compute disks snapshot $disk \ --snapshot-names=$disk-$now \ --zone=us-central1-a \ --project=project-id \ --async done |
Here’s how to list AWS Backup vaults and plans. You can filter the output by specifying a vault.
aws backup list-backup-vaults --query "BackupVaultList[?BackupVaultName=='my-vault']" --output json |
Output: (outputs are redacted for security reasons)
[ { "BackupVaultName": "my-vault", "BackupVaultArn": "arn:aws:backup:us-east-1:xxxxxxxxxxxx:backup-vault:my-vault", "CreationDate": "2019-02-10T11:38:42.556000-05:00", "EncryptionKeyArn": "arn:aws:kms:us-east-1:xxxxxxxxxxxx:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "CreatorRequestId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "NumberOfRecoveryPoints": 3 } ] |
Display the BackupPlanId of a specific backup plan.
aws backup list-backup-plans --query "BackupPlansList[?BackupPlanName=='my-backup-plan'].BackupPlanId" |
Output: (outputs are redacted for security reasons)
[ "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" ] |
Here’s a script that scans all EFS systems in several AWS accounts and regions and randomly assigns backup tags to EFS systems that are missing backup tags. This is assuming EFS is using AWS Backup service using tags to apply backup policies. If there are no backup tags, an EFS gets assigned a randomly picked backup policy.
#!/bin/bash # log file output="test.log" tmpfil="temp.txt" # empty file > $output # set random array arr[0]="efs-0000" arr[1]="efs-0400" arr[2]="efs-0800" arr[3]="efs-1200" arr[4]="efs-1600" arr[5]="efs-2000" rand=$[ $RANDOM %6 ] backup=${arr[$rand]} # set accounts and regions declare -a account=("default" "one" "two" "three" "four" "five") declare -a region=("us-east-1" "us-east-2" "us-west-1" "us-west-2") for i in "${account[@]}"; do echo "===================" >> $output echo $i >> $output echo "===================" >> $output for j in "${region[@]}"; do echo $j >> $output aws efs describe-file-systems \ --query "FileSystems[*].[FileSystemId,Tags[?Key=='aws-backup']|[0].Value]" \ --profile $i \ --region $j \ --output text > $tmpfil while read -r id tag; do if [[ $tag == "" ]]; then aws efs tag-resource \ --resource-id $id \ --tags Key="aws-backup",Value=${arr[$rand]} \ --profile $i \ --region $j >> $output echo "Added backup tag $backup to $id" >> $output elif [[ $tag == "no-backup" ]]; then echo "Backup tag is already set to no-backup on $id." >> $output else echo "No backup tag changes applied to $id." >> $output fi done < $tmpfil done done rm $tmpfil |
After you created a backup vault, it’s time to create a backup plan. Here are a few examples.
aws backup create-backup-plan \ --cli-input-json file://backup-plan.json \ --profile default \ --region us-east-1 |
File: backup-plan.json
{ "BackupPlan": { "BackupPlanName": "efs-0123", "Rules": [ { "RuleName": "efs-0123", "TargetBackupVaultName": "efs-vault", "ScheduleExpression": "cron(0 0 ? * * *)", "StartWindowMinutes": 60, "CompletionWindowMinutes": 10080, "Lifecycle": { "DeleteAfterDays": 7 } } ] } } |
Finally, create a backup selection.
aws backup create-backup-selection \ --backup-plan-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx \ --cli-input-json file://backup-selection.json \ --profile poc \ --region us-east-1 |
File: backup-selection.json
{ "BackupSelection": { "SelectionName": "efs-0123", "IamRoleArn": "arn:aws:iam::xxxxxxxxxxxx:role/service-role/AWSBackupDefaultServiceRole", "Resources": [], "ListOfTags": [ { "ConditionType": "STRINGEQUALS", "ConditionKey": "aws-backup", "ConditionValue": "efs-0123" } ] } } |
Here’s how to create a backup vault in AWS Backup.
aws backup create-backup-vault --backup-vault-name my-efs-vault --profile default --region us-east-1 |
To delete a vault, run this command.
aws backup delete-backup-vault --backup-vault-name my-efs-vault --profile default --region us-east-1 |
If you have an existing EFS that’s unencrypted, you can encrypt it be creating a snapshot using AWS Backup, and then restoring the file system to a new EFS with encryption. If you choose to restore in a directory in the same file system, it will not be encrypted. It has to be a new EFS. In addition, you’ll be asked to select which encryption key to use. The default key will work, unless you have your own.
Here’s the IAM policy to allow RDS Backup or create a snapshot on AWS.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "rds:RestoreDBClusterFromSnapshot", "rds:DescribeDBSnapshots", "rds:CopyDBSnapshot", "rds:CopyDBClusterSnapshot", "rds:DeleteDBSnapshot", "rds:DeleteDBClusterSnapshot", "rds:CreateDBSnapshot", "rds:RestoreDBInstanceFromDBSnapshot", "rds:CreateDBInstance", "rds:DescribeDBClusterSnapshots", "rds:DescribeDBInstances", "rds:DescribeDBClusters", "rds:DeleteDBInstance", "rds:CreateDBClusterSnapshot", "rds:ModifyDBSnapshotAttribute", "rds:ModifyDBClusterSnapshotAttribute", "rds:ListTagsForResource", "rds:DeleteDBCluster", "ec2:DescribeSecurityGroups", "ec2:DescribeRegions", "ec2:DescribeAvailabilityZones", "ec2:DescribeVpcs", "ec2:DescribeAccountAttributes", "ec2:DescribeSubnets", "iam:GetUser", "iam:GetAccountAuthorizationDetails", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:CreateGrant", "kms:DescribeKey*", "kms:ListKeys", "kms:ListAliases", "kms:Encrypt", "kms:Decrypt", "kms:GenerateDataKeyWithoutPlaintext", "kms:ListKeys", "kms:ListAliases", "kms:ListResourceTags" ], "Resource": "*" } ] } |
Here’s my MySQL backup script to the S3 Bucket.
Just a couple of things about the script. It’s using …
They must be setup and configured to work properly.
#!/bin/bash cd /root/database TIMESTAMP=$(date +%Y-%m-%d) S3FILE="s3://bucketname/sqlbackup/backup-$TIMESTAMP.sql" /usr/bin/mysqldump dbname > dbname.sql /usr/local/bin/aws s3 cp dbname.sql $S3FILE sleep 3s rm dbname.sql |
Finally, set the S3 bucket with a 7 day retention. Backups older than 7 days are automatically deleted.