cloud engineer
AWS has machine types that require ENA Support. You can run AWS CLI to find out if instance is ENA enabled.
aws ec2 describe-instances \ --instance-id i-xxxxxxxxxxxxxxxxx \ --profile default \ --region us-east-1 \ --query 'Reservations[].Instances[].EnaSupport' |
Login to the instance and verify.
sudo lspci | grep -i Amazon |
List driver details.
modinfo nvme |
Verify modules loaded at startup.
lsmod | grep nvme lsmod | grep ena |
If ENA drivers are missing, install them.
yum install pciutils |
Here’s a quick way to find the AWS Account ID via AWS CLI.
aws sts get-caller-identity --query Account --output text |
Output is a 12 digit number (redacted)
xxxxxxxxxxxx |
AWS Account ID is a 12 digit number unique to each AWS account. Occasionally, there are scripts and policies that need the ID of the account. The command above is one way of querying the account ID so they can be used for policies that need them.
Here’s how to list AWS Backup vaults and plans. You can filter the output by specifying a vault.
aws backup list-backup-vaults --query "BackupVaultList[?BackupVaultName=='my-vault']" --output json |
Output: (outputs are redacted for security reasons)
[ { "BackupVaultName": "my-vault", "BackupVaultArn": "arn:aws:backup:us-east-1:xxxxxxxxxxxx:backup-vault:my-vault", "CreationDate": "2019-02-10T11:38:42.556000-05:00", "EncryptionKeyArn": "arn:aws:kms:us-east-1:xxxxxxxxxxxx:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "CreatorRequestId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "NumberOfRecoveryPoints": 3 } ] |
Display the BackupPlanId of a specific backup plan.
aws backup list-backup-plans --query "BackupPlansList[?BackupPlanName=='my-backup-plan'].BackupPlanId" |
Output: (outputs are redacted for security reasons)
[ "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" ] |
If you need to copy a snapshot from one region to another, here’s the AWS CLI command.
aws ec2 copy-snapshot \
--region us-east-1 \
--source-region us-east-2 \
--source-snapshot-id snap-xxxxxxxxxxxxxxxxx \
--description "This is the DR snapshot copy" |
Output:
{ "SnapshotId": "snap-xxxxxxxxxxxxxxxxx" } |
Here’s how to list AWS AMI (machine images) owned by yourself.
aws ec2 describe-images \ --query "Images[*].[Name,PlatformDetails]" \ --owners self \ --profile default \ --region us-east-1 \ --output text |
Output:
icecast-0.1 Linux/UNIX jekyll-0.3 Linux/UNIX docker-0.1 Linux/UNIX |
Specifying owners as self only display images owned by you.
I was getting this error when downloading a file from a S3 bucket.
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden |
Turns out to be a permission issue. Use –acl bucket-owner-full-control.
# UPLOAD aws s3 cp file.txt s3://bucket-name/dir/ --acl bucket-owner-full-control upload: .\file.txt to s3://bucket-name/dir/fw.sh # DOWNLOAD aws s3 cp s3://bucket-name/dir/file.txt . --acl bucket-owner-full-control download: s3://bucket-name/dir/file.txt to .\file.txt |
You need to do for both upload and download.
I have an AMI with docker installed. Here’s how I launch a spot instance using Terraform.
Here’s my Terraform script.
terraform { required_providers { aws = { source = "hashicorp/aws" } } } provider "aws" { profile = "default" region = "us-east-1" } resource "aws_spot_instance_request" "docker" { ami = "ami-xxxxxxxxxxxxxxxx" spot_price = "0.0031" wait_for_fulfillment = "true" key_name = "servers" instance_type = "t3.micro" subnet_id = "subnet-xxxxxxxxxxxxxxxx" security_groups = ["sg-xxxxxxxxxxxxxxxxxx"] associate_public_ip_address = "true" user_data = <<-EOF #!/bin/bash hostnamectl set-hostname docker EOF tags = { Name = "docker-0.1" } } resource "aws_ec2_tag" "tagging" { resource_id = aws_spot_instance_request.docker.spot_instance_id key = "Name" value = "docker-0.1" } |
I use “aws_ec2_tag” resource to tag the instance properly.
In addition, I use user_data to run a script, to set the hostname.
To launch via Terraform, I run the following commands.
terraform init terraform apply |
When done, I could stop the instance to stop incurring charges. Or just simply destroy it via Terraform.
terraform destroy |
It’s not bad deal for an instance that costs only $0.0031 per hour.
If you have multiple AWS accounts, you can setup a customer-managed KMS (key management service) in the AWS Key Management Service, to secure requests or services between the two AWS accounts. The customer-managed KMS key is tied to an identity such as an IAM user or role. In addition to users and roles, other AWS accounts can be added to grant access. KMS can be symmetric or asymmetric. It’s symmetric be default. To grant access to the other account, you need to add the AWS Account Id to the key. It’s 12 digit number unique to each AWS account.
Once a key is created, the valid key ID can be used in a AWS SDK to access resources from the other AWS account.
AWS EC2 Firewall rules are defined within security groups. Security groups are attached to an instance. An instance can have up to 5 security groups. Essentially, this script gathers all the security groups associated with an instance, loops through them, and then outputs the ingress and egress rules of each security group to a file in a text format.
#!/bin/bash # set variables instanceid='i-xxxxxxxxxxxxxxxx' region='us-east-1' profile='sample' # log and temp files output="ec2-sg.log" tmpfil="ec2-sg.tmp" # empty log at start > $output # get sg ids aws ec2 describe-instances \ --instance-ids $instanceid \ --region $region \ --profile $profile \ --query 'Reservations[*].Instances[*].SecurityGroups[*].[GroupId]' --output text > $tmpfil while read -r id; do echo '============================================' >> $output echo $id >> $output echo '============================================' >> $output echo '---------------- INGRESS -------------------' >> $output aws ec2 describe-security-groups \ --group-ids $id \ --profile $profile \ --region $region \ --output text \ --query 'SecurityGroups[].IpPermissions[].[FromPort,ToPort,IpProtocol,IpRanges[].CidrIp[]|[0]]' >> $output echo '---------------- EGRESS --------------------' >> $output aws ec2 describe-security-groups \ --group-ids $id \ --profile $profile \ --region $region \ --output text \ --query 'SecurityGroups[].IpPermissionsEgress[].[FromPort,ToPort,IpProtocol,IpRanges[].CidrIp[]|[0]]' >> $output done < $tmpfil |
Here’s a sample output.
============================================ sg-xxxxxxxxxxxxxxx ============================================ ---------------- INGRESS ------------------- 5985 5985 tcp 10.0.0.220/32 10005 10005 tcp 10.0.0.164/32 ---------------- EGRESS -------------------- 80 80 tcp 10.0.0.14/32 40000 65535 udp 10.0.0.0/8 3389 3389 tcp 10.0.0.96/32 9389 9389 tcp 10.0.0.0/8 5985 5986 tcp 10.0.0.96/32 |
Here’s how to launch a LightSail instance using Terraform. Create a main.tf file.
terraform { required_providers { aws = { version = >= 3.22.0" source = "hashicorp/aws" } } } provider "aws" { profile = "default" region = "us-east-1" } resource "aws_lightsail_instance" "yourinstance" { name = "yourinstance" availability_zone = "us-east-1a" blueprint_id = "amazon_linux_2" bundle_id = "nano_2_0" } |
To launch, run the following Terraform commands.
terraform init terraform apply |