• Skip to main content

Uly.me

cloud engineer

  • Home
  • About
  • Archives

output

Bash Read File

December 1, 2021

How to read a file line by line in Bash.

#!/bin/bash
file=test.txt
while IFS= read -r line; do
  echo $line
done < $file

#!/bin/bash file=test.txt while IFS= read -r line; do echo $line done < $file

Contents of test.txt.

one
two
three

one two three

Results when running test.sh.

$ bash test.sh
one
two
three

$ bash test.sh one two three

To read multiple strings line by line.

#!/bin/bash
file=test.txt
while read -r a b; do
  echo $b
done < $file

#!/bin/bash file=test.txt while read -r a b; do echo $b done < $file

Modified contents of test.txt.

one 1
two 2
three 3

one 1 two 2 three 3

New results.

$ bash test.sh
1
2
3

$ bash test.sh 1 2 3

Filed Under: Linux Tagged With: bash, file, loop, output, read

AWS EC2 List Firewall Rules

January 26, 2021

AWS EC2 Firewall rules are defined within security groups. Security groups are attached to an instance. An instance can have up to 5 security groups. Essentially, this script gathers all the security groups associated with an instance, loops through them, and then outputs the ingress and egress rules of each security group to a file in a text format.

#!/bin/bash
# set variables
instanceid='i-xxxxxxxxxxxxxxxx'
region='us-east-1'
profile='sample'
# log and temp files
output="ec2-sg.log"
tmpfil="ec2-sg.tmp"
# empty log at start
> $output
# get sg ids
aws ec2 describe-instances \
--instance-ids $instanceid \
--region $region \
--profile $profile \
--query 'Reservations[*].Instances[*].SecurityGroups[*].[GroupId]' --output text > $tmpfil
while read -r id; do
  echo '============================================' >> $output
  echo $id >> $output
  echo '============================================' >> $output
  echo '---------------- INGRESS -------------------' >> $output
  aws ec2 describe-security-groups \
  --group-ids $id \
  --profile $profile \
  --region $region \
  --output text \
  --query 'SecurityGroups[].IpPermissions[].[FromPort,ToPort,IpProtocol,IpRanges[].CidrIp[]|[0]]' >> $output
  echo '---------------- EGRESS --------------------' >> $output
  aws ec2 describe-security-groups \
  --group-ids $id \
  --profile $profile \
  --region $region \
  --output text \
  --query 'SecurityGroups[].IpPermissionsEgress[].[FromPort,ToPort,IpProtocol,IpRanges[].CidrIp[]|[0]]' >> $output
done < $tmpfil

#!/bin/bash # set variables instanceid='i-xxxxxxxxxxxxxxxx' region='us-east-1' profile='sample' # log and temp files output="ec2-sg.log" tmpfil="ec2-sg.tmp" # empty log at start > $output # get sg ids aws ec2 describe-instances \ --instance-ids $instanceid \ --region $region \ --profile $profile \ --query 'Reservations[*].Instances[*].SecurityGroups[*].[GroupId]' --output text > $tmpfil while read -r id; do echo '============================================' >> $output echo $id >> $output echo '============================================' >> $output echo '---------------- INGRESS -------------------' >> $output aws ec2 describe-security-groups \ --group-ids $id \ --profile $profile \ --region $region \ --output text \ --query 'SecurityGroups[].IpPermissions[].[FromPort,ToPort,IpProtocol,IpRanges[].CidrIp[]|[0]]' >> $output echo '---------------- EGRESS --------------------' >> $output aws ec2 describe-security-groups \ --group-ids $id \ --profile $profile \ --region $region \ --output text \ --query 'SecurityGroups[].IpPermissionsEgress[].[FromPort,ToPort,IpProtocol,IpRanges[].CidrIp[]|[0]]' >> $output done < $tmpfil

Here’s a sample output.

============================================
sg-xxxxxxxxxxxxxxx
============================================
---------------- INGRESS -------------------
5985    5985    tcp     10.0.0.220/32
10005   10005   tcp     10.0.0.164/32
---------------- EGRESS --------------------
80      80      tcp     10.0.0.14/32
40000   65535   udp     10.0.0.0/8
3389    3389    tcp     10.0.0.96/32
9389    9389    tcp     10.0.0.0/8
5985    5986    tcp     10.0.0.96/32

============================================ sg-xxxxxxxxxxxxxxx ============================================ ---------------- INGRESS ------------------- 5985 5985 tcp 10.0.0.220/32 10005 10005 tcp 10.0.0.164/32 ---------------- EGRESS -------------------- 80 80 tcp 10.0.0.14/32 40000 65535 udp 10.0.0.0/8 3389 3389 tcp 10.0.0.96/32 9389 9389 tcp 10.0.0.0/8 5985 5986 tcp 10.0.0.96/32

Filed Under: Cloud Tagged With: aws, cli, ec2, firewall, output, security groups, text

AWS CLI Contains

January 16, 2021

The AWS CLI has a not so well-known comparison operator called “contains” which can be used to filter or query the output of your results. In this example, we want to show only instances that were not terminated.

Here’s a query containing “?!contains().”

aws ec2 describe-instances \
--query 'Reservations[*].Instances[?!contains(State.Name, `terminated`)].{Instance:InstanceId}'
--output text

aws ec2 describe-instances \ --query 'Reservations[*].Instances[?!contains(State.Name, `terminated`)].{Instance:InstanceId}' --output text

Filed Under: Cloud Tagged With: aws, cli, contains, describe-instances, output

Linux Script

June 30, 2020

The script command records all your keystrokes and output into a file called typescript.

You can start recording by simply typing script.

script
Script started, file is typescript
df -h
ls -l
exit

script Script started, file is typescript df -h ls -l exit

Typing exit stops the recording. Just cat typescript to see your recording.

cat typescript

cat typescript

Filed Under: Linux Tagged With: keystrokes, output, record, script, typescript

AWS EC2 Describe Snapshots

May 7, 2020

Here’s how to describe snapshots using query with tags.

aws ec2 describe-snapshots \
--owner-ids xxxxxxxxxxx \
--profile default \
--region us-east-2 \
--output text \
--query "Snapshots[*].[SnapshotId,Tags[?Key=='Name'].Value[] | [0]]"

aws ec2 describe-snapshots \ --owner-ids xxxxxxxxxxx \ --profile default \ --region us-east-2 \ --output text \ --query "Snapshots[*].[SnapshotId,Tags[?Key=='Name'].Value[] | [0]]"

Result.

snap-xxxxxxxxxxxxxxxxx	Server1
snap-xxxxxxxxxxxxxxxxx	Server2

snap-xxxxxxxxxxxxxxxxx Server1 snap-xxxxxxxxxxxxxxxxx Server2

Filed Under: Cloud Tagged With: aws, describe, ec2, output, snapshots, tags, text

Split A Large File

January 7, 2020

How to split a large file.

split -b 500MB httpd.log
ll -lh
total 1.9G
-rw-r--r-- 1 root root 954M Mar 25 12:35 httpd.log
-rw-r--r-- 1 root root 477M Mar 25 12:38 xaa
-rw-r--r-- 1 root root 477M Mar 25 12:38 xab

split -b 500MB httpd.log ll -lh total 1.9G -rw-r--r-- 1 root root 954M Mar 25 12:35 httpd.log -rw-r--r-- 1 root root 477M Mar 25 12:38 xaa -rw-r--r-- 1 root root 477M Mar 25 12:38 xab

Split with an output file.

split -b 200M httpd.log split.log
ll -lh
total 1.9G
-rw-r--r-- 1 root root 954M Mar 25 12:35 httpd.log
-rw-r--r-- 1 root root 200M Mar 25 12:52 split.logaa
-rw-r--r-- 1 root root 200M Mar 25 12:52 split.logab
-rw-r--r-- 1 root root 200M Mar 25 12:52 split.logac
-rw-r--r-- 1 root root 200M Mar 25 12:52 split.logad
-rw-r--r-- 1 root root 154M Mar 25 12:52 split.logae

split -b 200M httpd.log split.log ll -lh total 1.9G -rw-r--r-- 1 root root 954M Mar 25 12:35 httpd.log -rw-r--r-- 1 root root 200M Mar 25 12:52 split.logaa -rw-r--r-- 1 root root 200M Mar 25 12:52 split.logab -rw-r--r-- 1 root root 200M Mar 25 12:52 split.logac -rw-r--r-- 1 root root 200M Mar 25 12:52 split.logad -rw-r--r-- 1 root root 154M Mar 25 12:52 split.logae

Filed Under: Linux Tagged With: file, large, output, split

AWS CLI EC2 Describe Tags

August 26, 2019

Here’s how to get a list of EC2 tags.

aws ec2 describe-tags \
--filters "Name=resource-id,Values=i-xxxxxxxxxxxxx" \
--query 'Tags[][Key,Value]'  \
--profile default \
--region us-east-1 \
--output text

aws ec2 describe-tags \ --filters "Name=resource-id,Values=i-xxxxxxxxxxxxx" \ --query 'Tags[][Key,Value]' \ --profile default \ --region us-east-1 \ --output text

Filed Under: Cloud, Linux Tagged With: aws, cli, describe, ec2, output, tags, text

  • Home
  • About
  • Archives

Copyright © 2023