Uly.me

cloud engineer

Run Shell Script From Your Website

Here’s how to run a shell script from your website. You’ll need 2 files.

Here’s the contents of foo.php. Wrap your output with ‘pre’ for better formatting.

<?php
$output = shell_exec('/var/www/html/bar.sh 2>&1');
echo "$output";

Here’s the content of bar.sh. Output will be displayed on web page.

#!/bin/bash
now="$(date +'%y%m%d')"
echo $now
aws s3 ls

MySQL Backup To S3 Bucket

Here’s my MySQL backup script to the S3 Bucket.

Just a couple of things about the script. It’s using …

  1. AWS CLI
  2. Mysqldump

They must be setup and configured to work properly.

#!/bin/bash
cd /root/database
TIMESTAMP=$(date +%Y-%m-%d)
S3FILE="s3://bucketname/sqlbackup/backup-$TIMESTAMP.sql"
/usr/bin/mysqldump dbname > dbname.sql
/usr/local/bin/aws s3 cp dbname.sql $S3FILE
sleep 3s
rm dbname.sql

Finally, set the S3 bucket with a 7 day retention. Backups older than 7 days are automatically deleted.

AWS LightSail Delete Snapshots

Here’s my script to delete AWS LightSail snapshots. It can delete daily or weekly backups and scheduled via crontab. If you want longer or shorter retention, adjust the expired value. It’s in seconds. 604800 is 1 week. 2592000 is 1 month.

#!/bin/bash
 
current=$(date +%s)
 
if [ $# -eq 0 ]; then
  exit 1
fi
 
if [[ $1 = 'daily' ]]; then
  prefix='daily'
  expired=$(($current-604800))
elif [[ $1 = 'weekly' ]]; then
  prefix='weekly'
  expired=$(($current-2592000))
else
  exit 1
fi
 
snaps='/root/snapshots/snapshots.json'
names='/root/snapshots/names.txt'
parse='/root/snapshots/parse.txt'
logfile='/root/snapshots/snapshots.log'
 
/usr/local/bin/aws lightsail get-instance-snapshots > $snaps
cat $snaps | jq -r '.instanceSnapshots[] | .name' > $names
cat $names | grep $prefix > $parse
 
while read -r line; do
 
  snapshot=$(echo $line | cut -d_ -f3)
  snapshotname=$line
 
  if [ `expr $snapshot + 1 2> /dev/null` ] ; then
 
    if [ $snapshot -le $expired ]; then
      echo 'Deleted: '$snapshotname >> $logfile
      /usr/local/bin/aws lightsail delete-instance-snapshot \
      --instance-snapshot-name $snapshotname
    else
      echo 'Nothing: '$snapshotname >> $logfile
    fi
 
  else
    echo $snapshot is not numeric > /dev/null
  fi
 
done < $parse
 
echo 'Current time: '$current >> $logfile
echo 'Expired time: '$expired >> $logfile
echo '-----------------------------------' >> $logfile

Schedule deletes via crontab.

# run daily at 5am
0 5 * * * /bin/bash /root/snapshots/delete-snapshot.sh daily 2>&1
# run weekly every sunday at 6am
0 6 * * 0 /bin/bash /root/snapshots/delete-snapshot.sh weekly 2>&1

Parsing Wowza Logs

I’ve been wanting to look at Wowza logs for a very long time. I just spent a couple of hours writing a Bash script that generates a stats report. It parses the Wowza log file for the IP addresses, and runs them and against a site called ipinfo.io to get the viewer’s geo location. The output is filtered, sorted, takes a count, and finally generates a webpage. Here’s the Bash script.

# Can be placed in cron to run.
create-wowza-report.sh wowzastreamingengine_access.log
#!/bin/bash
 
# check for missing argument
if [ -z "$1" ]
  then
    echo "No arguments supplied. Format: create-wowza-report.sh wowzastreamingengine_access.log"
    exit 1
fi
 
# get the ips from the wowza log file
awk '{print $17}' $1 > output1.txt
 
# exclude invalid ips 
grep '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' output1.txt > output2.txt
 
# sort unique ips
cat output2.txt | sort -u | uniq > ips.txt
 
# pause for 1s
sleep 1s
 
# empty file just in case
> output3.txt
 
# set filename
filename="ips.txt"
 
# read filename and get geo info
while read -r ipaddress; do
  curl ipinfo.io/"$ipaddress" 2>/dev/null | awk -F'"' '$2=="city"{printf("%s, ", $4)}$2=="region"{print $4}' >> output3.txt
done < "$filename"
 
# cleanup empty geo locations
grep -v '^,' output3.txt > output4.txt
 
# sort output alphabetically
cat output4.txt | sort > output5.txt
 
# get a count of connections
count=$(wc -l < output5.txt)
 
# get today's date
filename=$(date +%Y%m%d)
date=`date +'%A %B %d, %Y'`
 
# rename file
mv output5.txt $filename.txt
 
# generate web page
cat <<EOF > $filename.php
<html>
<head>
<title>Wowza Streaming Stats on $date</title>
</head>
<body>
<h1>Live Streaming Stats</h1>
<p>Number of connections on $date: $count</p>
<?php include('$filename.txt');?>
</body>
</html>
EOF
 
# move files to web directory
cp $filename.* /var/www/html
 
# cleanup files
rm output*.txt
rm ips.txt