Uly.me

cloud engineer

Archives for March 2019

StatsMaker

A couple of personal statistics scripts.

# create stats. can be run from cron.
/var/www/statsmaker/create.sh
# fix. must supply date.
/var/www/statsmaker/fix.sh 190201

WP_Query Reverse Sort

Here’s how to reverse sort WP_Query. Order by ID. Reverse sort by ASC intead of DESC.

  $args = array (
    'cat' => 8, 
    'year' => $year, 
    'orderby' => 'ID',
    'order' => 'ASC', 
    'posts_per_page' => -1, 
    'paged' => $paged, 
    's' => $searchterm 
  );
  $category_posts = new WP_Query($args);

Create Group With ID

Here’s how to create a Linux group with a specific ID. Group name is nginx in this example. ID is 1000.

groupadd nginx -g 1000

Parsing Wowza Logs

I’ve been wanting to look at Wowza logs for a very long time. I just spent a couple of hours writing a Bash script that generates a stats report. It parses the Wowza log file for the IP addresses, and runs them and against a site called ipinfo.io to get the viewer’s geo location. The output is filtered, sorted, takes a count, and finally generates a webpage. Here’s the Bash script.

# Can be placed in cron to run.
create-wowza-report.sh wowzastreamingengine_access.log
#!/bin/bash
 
# check for missing argument
if [ -z "$1" ]
  then
    echo "No arguments supplied. Format: create-wowza-report.sh wowzastreamingengine_access.log"
    exit 1
fi
 
# get the ips from the wowza log file
awk '{print $17}' $1 > output1.txt
 
# exclude invalid ips 
grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' output1.txt > output2.txt
 
# sort unique ips
cat output2.txt | sort -u | uniq > ips.txt
 
# pause for 1s
sleep 1s
 
# empty file just in case
> output3.txt
 
# set filename
filename="ips.txt"
 
# read filename and get geo info
while read -r ipaddress; do
  curl ipinfo.io/"$ipaddress" 2>/dev/null | awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}' >> output3.txt
done < "$filename"
 
# cleanup empty geo locations
grep -v '^,' output3.txt > output4.txt
 
# sort output alphabetically
cat output4.txt | sort > output5.txt
 
# get a count of connections
count=$(wc -l < output5.txt)
 
# get today's date
filename=$(date +%Y%m%d)
date=`date +'%A %B %d, %Y'`
 
# rename file
mv output5.txt $filename.txt
 
# generate web page
cat <<EOF > $filename.php
<html>
<head>
<title>Wowza Streaming Stats on $date</title>
</head>
<body>
<h1>Live Streaming Stats</h1>
<p>Number of connections on $date: $count</p>
<?php include('$filename.txt');?>
</body>
</html>
EOF
 
# move files to web directory
cp $filename.* /var/www/html
 
# cleanup files
rm output*.txt
rm ips.txt

A month ago

How to get last month’s value. Useful with Bash scripts.

LASTMONTH = $(date +%Y%m -d '1 month ago')
echo $LASTMONTH
# Output is ...
201902

Steps to Encrypt Volumes

Here the steps to encrypt an unencrypted volume.

  1. Take a snapshot of the unencrypted volume.
  2. Make a copy of that snapshot and turn on encryption.
  3. Create a volume of the encrypted snapshot.
  4. Stop the instance.
  5. Detach the original unencrypted volume from the instance.
  6. Attach the newly created encrypted volume to the instance.
  7. Start the instance.

AWS CLI

# CREATE A SNAPSHOT
aws ec2 create-snapshot \
--volume-id vol-1234567890abcdef0 \
--description "This is my snapshot"
 
# COPY SNAPSHOT
aws ec2 copy-snapshot \
--source-region us-west-2 --source-snapshot-id snap-066877671789bd71b \
--region us-east-1 --description "This is my copied snapshot."
 
# CREATE A VOLUME
aws ec2 create-volume \
--region us-east-1 --availability-zone us-east-1a \
--snapshot-id snap-066877671789bd71b --volume-type io1 --iops 1000
 
# STOP AN INSTANCE
aws ec2 stop-instances --instance-ids i-1234567890abcdef0
 
# DETACH A VOLUME
aws ec2 detach-volume --volume-id vol-1234567890abcdef0
 
# ATTACH A VOLUME
aws ec2 attach-volume --volume-id vol-1234567890abcdef0 \
--instance-id i-01474ef662b89480 --device /dev/sdf
 
# START AN INSTANCE
aws ec2 start-instances --instance-ids i-1234567890abcdef0

List of Encrypted Volumes

Here’s the AWS CLI to get a list of encrypted or unencrypted volumes.

# list of encrypted volumes
aws ec2 describe-volumes \
--filters Name=encrypted,Values=true \
--region us-east-1 --profile default \
--query "Volumes[*].{ID:VolumeId}" --output text
# list of unencrypted volumes
aws ec2 describe-volumes \
--filters Name=encrypted,Values=false \
--region us-east-1 --profile default \
--query "Volumes[*].{ID:VolumeId}" --output text
# count the list of encrypted volumes. use wc -l to get a count.
aws ec2 describe-volumes \
--filters Name=encrypted,Values=true \
--region us-east-1 --profile default \
--query "Volumes[*].{ID:VolumeId}" --output text | wc -l