Uly.me

cloud engineer

cloudformation

AWS CloudFormation Security Group

AWS CloudFormation to create security groups. Includes self-refencing ingress and egress rules.

AWSTemplateFormatVersion: '2010-09-09'
Description: my-security-groups
######################################
Parameters:
  EC2Vpc:
    ConstraintDescription: Must be a valid VpcId
    Description: Select the VPC to use
    Type: AWS::EC2::VPC::Id
##############################################################################
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
    - Label:
        default: VPC
      Parameters:
      - EC2Vpc
##############################################################################
Resources:
  EC2InstanceSecurityGroup1:
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: 
        Ref: EC2Vpc
      GroupDescription: my-security-group-1
      GroupName: my-security-group-1
      SecurityGroupIngress:
        - {CidrIp: 10.0.0.0/8,                        IpProtocol: tcp,  FromPort: '80',      ToPort: '80',    Description: 'HTTP'}  
      SecurityGroupEgress:
        - {CidrIp: 10.0.0.0/8,                        IpProtocol: udp,  FromPort: '123',     ToPort: '123',   Description: 'NTP'}  
        - {CidrIp: 10.0.0.0/8,                        IpProtocol: tcp,  FromPort: '53',      ToPort: '53',    Description: 'DNS'}
      Tags:
        - {Key: Name,         Value: 'my-security-group-1'}
  EC2InstanceSecurityGroup2:
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: 
        Ref: EC2Vpc
      GroupDescription: my-security-group-2
      GroupName: my-security-group-2
      SecurityGroupIngress:
        - {CidrIp: 0.0.0.0/0,                        IpProtocol: tcp,  FromPort: '443',     ToPort: '443',   Description: 'HTTP'}  
        - {CidrIp: 0.0.0.0/0,                        IpProtocol: icmp, FromPort: '-1',      ToPort: '-1',    Description: 'ICMP ping'}
      SecurityGroupEgress:
        - {CidrIp: 10.0.0.0/8,                       IpProtocol: udp,  FromPort: '123',     ToPort: '123',   Description: 'NTP'}  
        - {CidrIp: 10.0.0.0/8,                       IpProtocol: tcp,  FromPort: '53',      ToPort: '53',    Description: 'DNS'}
        - {CidrIp: 0.0.0.0/0,                        IpProtocol: icmp, FromPort: '-1',      ToPort: '-1',    Description: 'ICMP ping'}
      Tags:
        - {Key: Name,         Value: 'my-security-group-2'}
  MyIngressSelfAll:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      GroupId: !Ref EC2InstanceSecurityGroup2
      SourceSecurityGroupId: !GetAtt EC2InstanceSecurityGroup2.GroupId
      IpProtocol: -1
      FromPort: 0
      ToPort: 65535
  MyEgressSelfAll:
    Type: AWS::EC2::SecurityGroupEgress
    Properties: 
      GroupId: !Ref EC2InstanceSecurityGroup2
      DestinationSecurityGroupId: !GetAtt EC2InstanceSecurityGroup2.GroupId
      IpProtocol: -1 
      FromPort: 0
      ToPort: 65535
##############################################################################
Outputs:
  SecurityGroupId:
    Description: The Security Group that was created
    Value: {Ref: EC2InstanceSecurityGroup1}
    Value: {Ref: EC2InstanceSecurityGroup2}
  StackName:
    Description: Name of this stack for Fn::ImportValue use by children of top level stack
    Value: {Ref: 'AWS::StackName'}

CloudFormation Userdata

Here’s another way to add startup scripts to your instance during creation.

      UserData:
        Fn::Base64: !Sub |
          #cloud-config
          repo_upgrade: none
 
          runcmd:
            # Cloud init startup script
            - "bash /root/setup.sh"
 
          write_files:
            # Cloud init startup script
            - owner: root:root
              permissions: '0644'
              path: /root/setup.sh
              content: |
                #! /bin/bash -x
          write_files:
            # Cloud init startup script
            - owner: root:root
              permissions: '0644'
              path: /root/setup.sh
              content: |
                #! /bin/bash -x
                # run your bash commands here
                date > log.txt
                uptime >>  log.txt

The above script creates a file setup.sh and executes it during instance creation. The output is dumped to log.txt file.

EBS volume stuck in CloudFormation

When running CloudFormation, all the resources are being created with no problem. However it seems to be getting stuck at creating or mounting a volume. The CloudFormation fails and initiates a rollback. This is the error I am getting. 

Volume attachment between volume-id vol-xxxxxxxx and instance-id i-xxxxxxx at device /dev/xvda is attaching

This turned out to be a conflict on HVM EC2 instances because /dev/sda1 is being remapped to /dev/xvda. My second drive is also mapped to /dev/xvda. The fix was to simply to map it slightly different to avoid mapping conflict.

Here’s the original mapping.

Boot:   /dev/xvda
Device: /dev/xvda
Device: /dev/xvdb

Here’s the fix.

Boot:   /dev/xvda
Device: /dev/xvdb
Device: /dev/xvdc

WAF CloudFormation Template

Here’s the CloudFormation template for creating a WAF.

Here are some options that you’ll be asked during creation.

  • Activate SQL Injection Protection ( yes | no )
  • Activate Cross-site Scripting Protection ( yes | no )
  • Activate HTTP Flood Protection ( WAF rate | Lambda log parser | Athena log parser | no )
  • Activate Scanner & Probe Protection ( Lambda log parser | Athena log parser | no )
  • Activate Reputation List Protection ( yes | no )
  • Activate Bad Bot Protection ( yes | no )
  • Endpoint Type (CloudFront or ALB)
  • Application Access Log Bucket Name ( Leave blank if no S3 bucket)

The template creates 2 CloudFormation stacks.

EFS CloudFormation

Here’s a simple EFS CloudFormation template with a one mount target.

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Description": "EFS example setup",
	"Parameters": {
		"VPC": {
			"Description": "VPC ID",
			"Type": "AWS::EC2::VPC::Id"
		},
		"Subnet": {			
			"Description": "Subnet ID",
			"Type": "AWS::EC2::Subnet::Id"
		},
		"EC2SecurityGroup": {
			"Description": "Security Group for EC2 instance",
			"Type": "AWS::EC2::SecurityGroup::Id"
		}	
	},
	"Resources": {
		"EFSFileSystem": {
			"Type" : "AWS::EFS::FileSystem",
			"Properties" : {
				"FileSystemTags" : [
					{"Key" : "Name", "Value" : {"Ref": "AWS::StackName"}}
				]
			}
		},
		"EFSMountTarget": {
			"Type": "AWS::EFS::MountTarget",
			"Properties": {
				"FileSystemId": {"Ref": "EFSFileSystem"},
				"SubnetId": { "Ref": "Subnet" },
				"SecurityGroups": [{"Ref": "EFSSecurityGroup"}]        
			}
		},
		"EFSSecurityGroup": {
			"Type": "AWS::EC2::SecurityGroup",
			"Properties": {
				"GroupDescription": "Allowing access to EFS",
				"VpcId": {"Ref": "VPC"},
				"SecurityGroupIngress": [{
					"IpProtocol": "tcp",
					"FromPort": 2049,
					"ToPort": 2049,
					"SourceSecurityGroupId": {"Ref": "EC2SecurityGroup"}
				}]
			}
		}				
	}
}

YAML & Indentation

JSON and YAML are used in AWS CloudFormation. Some people prefer YAML for its simplicity. It’s easy to read and code. However, YAML has its own little quirks. It relies heavily on indentation for its structure. If you don’t use the proper spaces, it will fail. I ran into an issue when I ran CloudFormation in AWS. The example below display the contents of my userdata.

This piece of code will NOT work in CloudFormation.

Fn::Base64: !Sub |
#!/bin/bash -xe
yum update -y
mkdir /data
cd /data
aws s3 cp s3://efs-s3-backup-engine .
sleep 30s
initiate.sh

This is valid. It’s all about indentation.

Fn::Base64: !Sub |
  #!/bin/bash -xe
  yum update -y
  mkdir /data
  cd /data
  aws s3 cp s3://efs-s3-backup-engine .
  sleep 30s
  initiate.sh

By the way, tabs are to be avoided like the plague.