• Skip to main content

Uly.me

cloud engineer

  • Home
  • About
  • Archives

Archives for March 2019

StatsMaker

March 26, 2019

A couple of personal statistics scripts.

# create stats. can be run from cron.
/var/www/statsmaker/create.sh
# fix. must supply date.
/var/www/statsmaker/fix.sh 190201

# create stats. can be run from cron. /var/www/statsmaker/create.sh # fix. must supply date. /var/www/statsmaker/fix.sh 190201

Filed Under: Linux Tagged With: automation, create, stats, wowza

WP_Query Reverse Sort

March 25, 2019

Here’s how to reverse sort WP_Query. Order by ID. Reverse sort by ASC intead of DESC.

  $args = array (
    'cat' => 8, 
    'year' => $year, 
    'orderby' => 'ID',
    'order' => 'ASC', 
    'posts_per_page' => -1, 
    'paged' => $paged, 
    's' => $searchterm 
  );
  $category_posts = new WP_Query($args);

$args = array ( 'cat' => 8, 'year' => $year, 'orderby' => 'ID', 'order' => 'ASC', 'posts_per_page' => -1, 'paged' => $paged, 's' => $searchterm ); $category_posts = new WP_Query($args);

Filed Under: WP Tagged With: id, post, reverse, sort, wp_query

Create Group With ID

March 22, 2019

Here’s how to create a Linux group with a specific ID. Group name is nginx in this example. ID is 1000.

groupadd nginx -g 1000

groupadd nginx -g 1000

Filed Under: Linux Tagged With: create, group, id

Change VIM Colors

March 20, 2019

Vim is a text editor. Edit the ~.vimrc

# vim ~.vimrc
colo desert
syntax on

# vim ~.vimrc colo desert syntax on

Filed Under: Linux Tagged With: colors, desert, editor, terminal, vim

Parsing Wowza Logs

March 19, 2019

I’ve been wanting to look at Wowza logs for a very long time. I just spent a couple of hours writing a Bash script that generates a stats report. It parses the Wowza log file for the IP addresses, and runs them and against a site called ipinfo.io to get the viewer’s geo location. The output is filtered, sorted, takes a count, and finally generates a webpage. Here’s the Bash script.

# Can be placed in cron to run.
create-wowza-report.sh wowzastreamingengine_access.log

# Can be placed in cron to run. create-wowza-report.sh wowzastreamingengine_access.log

#!/bin/bash
 
# check for missing argument
if [ -z "$1" ]
  then
    echo "No arguments supplied. Format: create-wowza-report.sh wowzastreamingengine_access.log"
    exit 1
fi
 
# get the ips from the wowza log file
awk '{print $17}' $1 > output1.txt
 
# exclude invalid ips 
grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' output1.txt > output2.txt
 
# sort unique ips
cat output2.txt | sort -u | uniq > ips.txt
 
# pause for 1s
sleep 1s
 
# empty file just in case
> output3.txt
 
# set filename
filename="ips.txt"
 
# read filename and get geo info
while read -r ipaddress; do
  curl ipinfo.io/"$ipaddress" 2>/dev/null | awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}' >> output3.txt
done < "$filename"
 
# cleanup empty geo locations
grep -v '^,' output3.txt > output4.txt
 
# sort output alphabetically
cat output4.txt | sort > output5.txt
 
# get a count of connections
count=$(wc -l < output5.txt)
 
# get today's date
filename=$(date +%Y%m%d)
date=`date +'%A %B %d, %Y'`
 
# rename file
mv output5.txt $filename.txt
 
# generate web page
cat <<EOF > $filename.php
<html>
<head>
<title>Wowza Streaming Stats on $date</title>
</head>
<body>
<h1>Live Streaming Stats</h1>
<p>Number of connections on $date: $count</p>
<?php include('$filename.txt');?>
</body>
</html>
EOF
 
# move files to web directory
cp $filename.* /var/www/html
 
# cleanup files
rm output*.txt
rm ips.txt

#!/bin/bash # check for missing argument if [ -z "$1" ] then echo "No arguments supplied. Format: create-wowza-report.sh wowzastreamingengine_access.log" exit 1 fi # get the ips from the wowza log file awk '{print $17}' $1 > output1.txt # exclude invalid ips grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' output1.txt > output2.txt # sort unique ips cat output2.txt | sort -u | uniq > ips.txt # pause for 1s sleep 1s # empty file just in case > output3.txt # set filename filename="ips.txt" # read filename and get geo info while read -r ipaddress; do curl ipinfo.io/"$ipaddress" 2>/dev/null | awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}' >> output3.txt done < "$filename" # cleanup empty geo locations grep -v '^,' output3.txt > output4.txt # sort output alphabetically cat output4.txt | sort > output5.txt # get a count of connections count=$(wc -l < output5.txt) # get today's date filename=$(date +%Y%m%d) date=`date +'%A %B %d, %Y'` # rename file mv output5.txt $filename.txt # generate web page cat <<EOF > $filename.php <html> <head> <title>Wowza Streaming Stats on $date</title> </head> <body> <h1>Live Streaming Stats</h1> <p>Number of connections on $date: $count</p> <?php include('$filename.txt');?> </body> </html> EOF # move files to web directory cp $filename.* /var/www/html # cleanup files rm output*.txt rm ips.txt

Filed Under: Linux Tagged With: bash, live streaming, logs, stats, wowza

A month ago

March 19, 2019

How to get last month’s value. Useful with Bash scripts.

LASTMONTH = $(date +%Y%m -d '1 month ago')
echo $LASTMONTH
# Output is ...
201902

LASTMONTH = $(date +%Y%m -d '1 month ago') echo $LASTMONTH # Output is ... 201902

Filed Under: Linux Tagged With: bash, date, month, scripts

Steps to Encrypt Volumes

March 18, 2019

Here the steps to encrypt an unencrypted volume.

  1. Take a snapshot of the unencrypted volume.
  2. Make a copy of that snapshot and turn on encryption.
  3. Create a volume of the encrypted snapshot.
  4. Stop the instance.
  5. Detach the original unencrypted volume from the instance.
  6. Attach the newly created encrypted volume to the instance.
  7. Start the instance.

AWS CLI

# CREATE A SNAPSHOT
aws ec2 create-snapshot \
--volume-id vol-1234567890abcdef0 \
--description "This is my snapshot"
 
# COPY SNAPSHOT
aws ec2 copy-snapshot \
--source-region us-west-2 --source-snapshot-id snap-066877671789bd71b \
--region us-east-1 --description "This is my copied snapshot."
 
# CREATE A VOLUME
aws ec2 create-volume \
--region us-east-1 --availability-zone us-east-1a \
--snapshot-id snap-066877671789bd71b --volume-type io1 --iops 1000
 
# STOP AN INSTANCE
aws ec2 stop-instances --instance-ids i-1234567890abcdef0
 
# DETACH A VOLUME
aws ec2 detach-volume --volume-id vol-1234567890abcdef0
 
# ATTACH A VOLUME
aws ec2 attach-volume --volume-id vol-1234567890abcdef0 \
--instance-id i-01474ef662b89480 --device /dev/sdf
 
# START AN INSTANCE
aws ec2 start-instances --instance-ids i-1234567890abcdef0

# CREATE A SNAPSHOT aws ec2 create-snapshot \ --volume-id vol-1234567890abcdef0 \ --description "This is my snapshot" # COPY SNAPSHOT aws ec2 copy-snapshot \ --source-region us-west-2 --source-snapshot-id snap-066877671789bd71b \ --region us-east-1 --description "This is my copied snapshot." # CREATE A VOLUME aws ec2 create-volume \ --region us-east-1 --availability-zone us-east-1a \ --snapshot-id snap-066877671789bd71b --volume-type io1 --iops 1000 # STOP AN INSTANCE aws ec2 stop-instances --instance-ids i-1234567890abcdef0 # DETACH A VOLUME aws ec2 detach-volume --volume-id vol-1234567890abcdef0 # ATTACH A VOLUME aws ec2 attach-volume --volume-id vol-1234567890abcdef0 \ --instance-id i-01474ef662b89480 --device /dev/sdf # START AN INSTANCE aws ec2 start-instances --instance-ids i-1234567890abcdef0

Filed Under: Cloud, Linux Tagged With: encrypted, instance, snapshot, unencrypted, volumes

List of Encrypted Volumes

March 18, 2019

Here’s the AWS CLI to get a list of encrypted or unencrypted volumes.

# list of encrypted volumes
aws ec2 describe-volumes \
--filters Name=encrypted,Values=true \
--region us-east-1 --profile default \
--query "Volumes[*].{ID:VolumeId}" --output text
# list of unencrypted volumes
aws ec2 describe-volumes \
--filters Name=encrypted,Values=false \
--region us-east-1 --profile default \
--query "Volumes[*].{ID:VolumeId}" --output text
# count the list of encrypted volumes. use wc -l to get a count.
aws ec2 describe-volumes \
--filters Name=encrypted,Values=true \
--region us-east-1 --profile default \
--query "Volumes[*].{ID:VolumeId}" --output text | wc -l

# list of encrypted volumes aws ec2 describe-volumes \ --filters Name=encrypted,Values=true \ --region us-east-1 --profile default \ --query "Volumes[*].{ID:VolumeId}" --output text # list of unencrypted volumes aws ec2 describe-volumes \ --filters Name=encrypted,Values=false \ --region us-east-1 --profile default \ --query "Volumes[*].{ID:VolumeId}" --output text # count the list of encrypted volumes. use wc -l to get a count. aws ec2 describe-volumes \ --filters Name=encrypted,Values=true \ --region us-east-1 --profile default \ --query "Volumes[*].{ID:VolumeId}" --output text | wc -l

Filed Under: Cloud, Linux Tagged With: aws, cli, describe volumes, encrypted, unencrypted

  • Go to page 1
  • Go to page 2
  • Go to page 3
  • Go to Next Page »
  • Home
  • About
  • Archives

Copyright © 2023