Uly.me

cloud engineer

You are here: Home / 2019 / Archives for May 2019

Archives for May 2019

AWS Assume Role

If you have multiple AWS accounts, you can gain additional permission by authorizing it to a different AWS account via IAM. Here’s the assume role documentation from AWS. And here are some documentation if you need to grant someone to switch accounts. If you are going to be using the AWS CLI, you will need to run “aws configure” to setup multiple profiles.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Resource": [
                "arn:aws:iam::*:role/your-custom-role"
            ],
            "Effect": "Allow"
        }
    ]
}

AWS CLI Lightsail

Get instance details

aws lightsail get-instance --instance-name your-server

Get instance state

aws lightsail get-instance-state --instance-name your-server

Create a snapshot

aws lightsail create-instance-snapshot \
--instance-snapshot-name your-server-new-snapshot-201905192200 \
--instance-name your-server

Create disk from snapshot

aws lightsail create-disk-from-snapshot \
--disk-name your-server-new-boot-disk \
--disk-name your-server-new-snapshot-201905192200 \
--availability-zone us-east-1a
--size-in-gb 50

Attach Disk

aws lightsail attach-disk \
--disk-name your-server-new-boot-disk \
--instance-name your-server \
--disk-path /dev/sda1

Create instance from snapshot

aws lightsail create-instance-snapshot \
--instance-snaphot-name your-server-new-snapshot-201905192200
--instance-name your-server

Attach Static IP Address

aws lightsail attach-static-ip \
--static-ip-name your-ip-name \
--instance-name your-server

Split Text into Columns in Excel

How to Split Text into Columns in Excel.

  1. Select the cell or column that contains the text you want to split.
  2. Select Data > Text to Columns.
  3. In the Convert Text to Columns Wizard, select Delimited > Next.
  4. Select the Delimiters for your data. For example, Comma and Space.
  5. You can see a preview of your data in the Data preview window.
  6. Select Next.
  7. Select the Column data format or use what Excel chose for you.
  8. Select the Destination, which is where you want the split data to appear on your worksheet.
  9. Select Finish.

Count Number of Files in Directory

Get a count of the number of files in a directory.

# using ls
ls | wc -l
ls -Aq | wc -l
# include hidden files 
find . ! -name . -prune -print | grep -c /
find .//. ! -name . -print | grep -c //
# using tree if installed. yum install tree. apt-get install tree
tree

AWS Glacier Setup via S3

You can setup AWS Glacier via S3 bucket replication. Create a S3 bucket and slap this bucket policy.

{
    "Version": "2012-10-17",
    "Id": "S3PolicyId1",
    "Statement": [
        {
            "Sid": "IPAllow",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::your-bucket-storage-name/*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "10.0.0.0/8"
                }
            }
        },
        {
            "Sid": "DenyIncorrectEncryptionHeader",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::your-bucket-storage-name/*",
            "Condition": {
                "StringNotEquals": {
                    "s3:x-amz-server-side-encryption": "AES256"
                }
            }
        },
        {
            "Sid": "DenyUnEncryptedObjectUploads",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::your-bucket-storage-name/*",
            "Condition": {
                "Null": {
                    "s3:x-amz-server-side-encryption": "true"
                }
            }
        }
    ]
}

Add this policy to your IAM user or role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::cah-callcopy-storage/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:List*",
                "s3:Get*"
            ],
            "Resource": "arn:aws:s3:::*"
        }
    ]
}
Finally, add a lifecycle policy to move your files from Standard to Glacier storage type.

GCP Create Images

Here’s how to create a GCP Disk Image from a Boot Drive.

  • First, stop the instance.
  • If you can’t, stop the applications/database from writing to disks.
  • Run sudo sync .

Create From Disk

gcloud compute images create the-new-image-name \
   --source-disk the-source-boot-disk \
   --source-disk-zone us-central1-a \
   --family the-image-family \
   [--force]

Create From Image

gcloud compute images create the-new-image-name \
  --source-image the-source-image \
  --source-image-project the-project-where-the-image-is-located \
  --family the-image-family

Create From Snapshot

gcloud compute images create the-new-image-name \
  --source-snapshot the-source-snapshot

For more info.

Calculating the Ideal EFS File Size

AWS EFS runs on burst credits when choosing burst mode, which is about 99% of most implementations. The reason for this is that provisioned throughput is very expensive. It’s a $6000 per month commitment. In burst mode, if the EFS volume is too small with heavy IO usage, the burst credits will nose dive. You will eventually run out of burst credits resulting in very poor IO performance. It will be slower than a floppy disk drive. We all know how old that technology is. In order to sustain your burst credits, you’ll need to add data, because more data means you’ll earn burst credits at a higher clip. Here’s how to calculate the ideal file size for an EFS system.

  1. Identify your throughput needs by looking at your historical usage. From the Amazon CloudWatch console, check the sum statistic of the TotalIOBytes metric with daily aggregation, for the past 14 days. Identify the day with the largest value for TotalIOBytes.
  2. Divide this number by 24 hours, 60 minutes, 60 seconds, and 1024 bytes to get the average KiB/second your application required for that day.
  3. Calculate the file system size (in GiB) required to sustain this average throughput by dividing the average throughput number (in KiB/s) by the baseline throughput number (50 KiB/s/GiB) that EFS provides.

For example, let’s say you have a 500 Gib of TotalIOBytes.

500 x 1024 x 1024 x 1024 / (24 x 60 x 60 x 1024) = 6068.15 Kib/s

6068.15 / 50 Kib/s = 121 Gib

You will need about 121 GB of EFS volume size to maintain your burst credits.

GCP Change VPC of Instance

The following are steps on how to change the network of an instance from one VPC to another. This is assuming you want to keep the same server name, but will need to make a network change. Unfortunately, changing networks will require creating a new instance. The following are the steps to make that change.

  1. Keep track of all the disks.
  2. Keep all disks including boot.
  3. Stop the instance.
  4. Create an image of the boot drive. Wait for it to complete.
  5. Create a similar instance, but do not execute.
  6. Make the network changes. Save the GCloud CLI command.
  7. Delete the instance.
  8. Run the saved command from step 4. This creates a new instance.
  9. Stop the instance.
  10. Detach the boot disk.
  11. Attach all the other disks.
  12. Boot the instance.