Uly.me

cloud engineer

jq

JQ on GitBash

JQ is a json processor. It’s for slicing and dicing data.

Download

To install on GitBash, unzip the file and copy the executable to /usr/bin.

mv jq-win64.exe to /usr/bin/jq.exe

Create an alias. Edit ~/.bashrc.

alias jq='/usr/bin/jq.exe'

Using JQ.

jq --version
jq '.selfLink' vm.json

AWS Rotate IAM Keys

Here’s a script that will rotate AWS IAM keys.

#!/bin/bash
# set files
user='johndoe'
newkey='/root/new-access-key.json'
oldkey='/root/old-access-key.json'
credentials='/root/.aws/credentials'
# get old credentials
aws iam list-access-keys --user-name $user > $oldkey
okey=$(jq .AccessKeyMetadata[0].AccessKeyId $oldkey | tr -d \")
# create new key
aws iam create-access-key --user-name $user > $newkey
# get new access keys and new secret
nkey=$(jq .AccessKey.AccessKeyId $newkey | tr -d \")
nsecret=$(jq .AccessKey.SecretAccessKey $newkey | tr -d \")
# backup old credentials
cp /root/.aws/credentials /root/.aws/credentials-backup
# store the new key
echo '[default]' > $credentials
echo 'aws_access_key_id = ' $nkey >> $credentials
echo 'aws_secret_access_key = '$nsecret >> $credentials
sleep 10
# delete old key
aws iam delete-access-key --user-name $user --access-key-id $okey
rm $newkey
rm $oldkey

The script performs the following:

  1. Retrieves the current key
  2. Creates a new key
  3. Backup the current credentials file
  4. Create a new credentials file
  5. Deletes the old key
  6. Deletes the temp files
  7. Done

Remove Quotes From Strings

Here’s the command to remove quotes from a string in Bash by using the tr -d command. I’m using jq tool to extract the access key from a json file. Bash returns a string surrounded by quotes. To remove the quotes, just pipe the string to the tr -d command.

jq .AccessKey.AccessKeyId key.json

Result:

"ASDFSDFSDFASDFSDFSDFGHGDF"

Using tr -d.

jq .AccessKey.AccessKeyId key.json | tr -d \"

Result:

ASDFSDFSDFASDFSDFSDFGHGDF

JQ Proccessor

The AWS CLI spits out a JSON output after each successful execution. If you need to grab the result, assign it to a variable, and use it for your subsequent scripts, you need some kind of JSON parser. You can use a tool like jq which will process or filter out the result for you. From jq’s website,

jq is a tool for processing JSON inputs, applying the given filter to its JSON text inputs and producing the filter’s results as JSON on standard output. The simplest filter is ., which is the identity filter, copying jq’s input to its output unmodified (except for formatting).

In this example, we will use the AWS CLI to give us a list of running EC2 instances. We will then dump the output into a file called output.json. We will then filter out the “InstanceId” by running it through cat and the jq processor. We will then assign the result to a variable called INSTANCE, and then finally use that variable to associate our instance to an Elastic IP Address.

aws ec2 describe-instances --filters Name=instance-state-name,Values=running > output.json
INSTANCEID=$(cat output.json | jq '.Reservations[].Instances[] | {InstanceId} | .InstanceId' --raw-output)
aws ec2 associate-address --instance-id $INSTANCEID --public-ip xxx.xxx.xxx.xxx

Filtering a nested JSON can be a bit tricky. In this particular case, we are using a filter you’ll find inside the single quote right after the jq command. To remove quotes from our result, I’m using –raw-output switch. Finally, I then associate our instance to an elastic public IP address.

jq is a very handy tool.