Skip to content

cli

AWS CLI command-line script - How to automatically delete the Default-VPCs in all AWS regions

AWS CLI command-line script to automatically delete all Default-VPCs in all AWS regions.

The script needs to have enough permissions to run the actions.

The script will fail if there are other dependencies than the ones dealt with in the script.

linux-bash-script

USE AT YOUR OWN RISK

#!/bin/bash

## uncomment to see the commands as they are executed.
#set -x

## gets a list of all AWS regions

LIST_OF_REGIONS=$(aws ec2 describe-regions --all-regions --query "Regions[].{Name:RegionName}" --output text)

## for-loop to cycle through all regions

for REGION in ${LIST_OF_REGIONS}; do
    echo "---------"
    echo "Region: ${REGION}"

    RESULT=$(aws ec2 describe-vpcs --region ${REGION} --query "Vpcs[].[VpcId,IsDefault]" --output text 2>/dev/null)
    if [ -z "${RESULT}" ];
        then
            echo "NULL - No Default-VPC in the region: ${REGION}"
        else
            echo "Not NULL - There is a Default-VPC in the region: ${REGION}"
            ##
            ## --- use AT YOUR OWN RISK ---
            ##
            ## Uncomment the `aws ec2 ...` lines below to delete the default VPC in all regions.
            ## The script still needs to have enough permission to run the commands.
            ##

            VPCID=${RESULT:0:-5}
            echo "${REGION} : ${VPCID}"

            ## We need to detach AND delete the correct Internet Gateway (IGW), before we can delete the Default-VPC.

            IGW=$(aws ec2 describe-internet-gateways --region ${REGION} --filters "Name=attachment.vpc-id,Values=${VPCID}" --query 'InternetGateways[].InternetGatewayId' --output text)

            ## IF IGW exists, then detach and delete the IGW from the Default-VPC
            if [ -z "${IGW}" ];
                then
                    echo "NULL - IGW already removed."
                else
                    echo "Removing and deleting the IGW: ${IGW}, from the Default-VPC: ${VPCID}."
                    #aws ec2 detach-internet-gateway --region ${REGION} --internet-gateway-id ${IGW} --vpc-id ${VPCID}
                    #aws ec2 delete-internet-gateway --region ${REGION} --internet-gateway-id ${IGW}
            fi

            ## From my own experience, also need to delete any subnets associated with the Default-VPC.

            LIST_OF_SUBNETS=$(aws ec2 describe-subnets --region ${REGION} --filters "Name=vpc-id,Values=${VPCID}" --query "Subnets[*].[SubnetId]" --output text)

            echo "List of subnets on the Default-VPC: ${LIST_OF_SUBNETS}"

            ## could add an if loop here too...

            for SUBNET in ${LIST_OF_SUBNETS}; do
                #aws ec2 delete-subnet --region ${REGION} --subnet-id ${SUBNET}
            done

            ## Finally, delete the Default-VPC.

            #aws ec2 delete-vpc --vpc-id ${VPCID} --region ${REGION} 2>/dev/null && echo "Default-VPC removed succesfully." || echo "Something is still not right..."
    fi

done

aws-documentation

Happy learning,

Antonio Feijao UK

Finding duplicated files using command line CLI in Linux MacOS Ubuntu

Easy way to find duplicated files in a folder or in all disk

find . -type f -name "*" -print0 | xargs -0 -I {} shasum -a 256 {}

Or course, output the above command to a file, cut -f 1, then sort and pipe it into uniq -c to count duplicates

find . -type f -name "*.JPG" -print0 | xargs -0 -I {} shasum -a 256 {} > finding-duplicates.txt

cat finding-duplicates.txt | cut -f 1 -d ' ' | sort | uniq -c | sort -rn | head -n10

grep 30848de6dba6f90bef4027fbf97a66dcc6f1f2eb3e8a6e47f0e9ce3fc411ce79 finding-duplicates.txt

of course, we now can automate this to keep the first file but move the duplicated into a backup folder before deleting them.

Example of my output on a folder with old photos that got duplicated over type... some photo is now 6x duplicated... time to automate tidying up!

Happy learning,

Antonio Feijao

AWS run multiple commands in multiple AWS accounts in the AWS Organizations at the same time

AWS run multiple commands in multiple AWS accounts in the AWS Organizations at the same time

** USE AT YOUR OWN RESPONSABILITY **

I needed to run the same AWS CLI command in multiple accounts within an AWS Organization.

I like code that is easy to read, does what is meant to do, is quick to run, is "safe" and efficient.

Might not be perfect for everyone, but it does well what I need it to do, and might be useful for others or for me again in the feature.

So, I built a script to do what I need in a way that can be re-used for multiple commands in multiple accounts at the same time.

If you have, for example +100 AWS Accounts, you might want to "slow down" this script.

for example, I sometimes use the for-loop below to "slow down" my script :)

for NUMBER in $(seq 0 9); do
    ./99-run-this-command-on-this-account.sh ${NUMBER} ; sleep 20
done

Script to run command multiple AWS account within the organisation

Important

cat 99-run-this-command-on-this-account.txt

#!/bin/bash


run_this_command () {
      account_id=$1
    account_name=$2
       role_name=$3

    echo "--------------------------------------------"
    echo "Going to run a command on ${account_id}, ${account_name}, using the role: ${role_name}"

    # assume a role in the account
    new_role=$(aws sts assume-role --role-arn "arn:aws:iam::${account_id}:role/${role_name}" --role-session-name ${account_id}-${role_name})

    AWS_ACCESS_KEY_ID=$(echo ${new_role} | jq -r '.Credentials.AccessKeyId' )
    AWS_SECRET_ACCESS_KEY=$(echo ${new_role} | jq -r '.Credentials.SecretAccessKey' )
    AWS_SESSION_TOKEN=$(echo ${new_role} | jq -r '.Credentials.SessionToken' )

    ACCOUNT_ID=$(echo ${new_role} | jq -r '.AssumedRoleUser.Arn' | cut -f 5 -d ':')

    export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
    export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
    export AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}

    export ACCOUNT_ID={ACCOUNT_ID}

    echo "Got key_id >>${AWS_ACCESS_KEY_ID}<< for account_id >>${account_id}, ${account_name}<< "

    #aws s3 ls

    aws securityhub update-standards-control \
        --standards-control-arn "arn:aws:securityhub:eu-west-2:${account_id}:control/cis-aws-foundations-benchmark/v/1.2.0/1.13" \
        --control-status "DISABLED" \
        --disabled-reason "SCP applied at the root level for the organisation will block any root actions."

    aws securityhub update-standards-control \
        --standards-control-arn "arn:aws:securityhub:eu-west-2:${account_id}:control/cis-aws-foundations-benchmark/v/1.2.0/1.14" \
        --control-status "DISABLED" \
        --disabled-reason "SCP applied at the root level for the organisation will block any root actions."

    aws securityhub update-standards-control \
        --standards-control-arn "arn:aws:securityhub:eu-west-2:${account_id}:control/aws-foundational-security-best-practices/v/1.0.0/IAM.6" \
        --control-status "DISABLED" \
        --disabled-reason "SCP applied at the root level for the organisation will block any root actions."

    #for bucket in $(aws s3api list-buckets | jq '.Buckets[].Name' | sed s/'"'//g) ; do
    #    echo "${account_id}, ${account_name}, enabling PublicAccessBlockConfiguration for bucket: ${bucket}" ;
    #    #aws s3api get-public-access-block --bucket ${MYBUCKET} | jq '.PublicAccessBlockConfiguration' ;
    #    aws s3api put-public-access-block \
    #          --bucket ${bucket} \
    #          --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"
    #done

}




number=$1
role_name=<ROLE_NAME_IN_HERE>
filter_name="prod"

# DO I NEED to filter by account name?
#
# DO I NEED a specific list of accounts?  >> command to list account - 'aws organizations list-accounts'
#
# Below, sample filter to list account by creation date.
# cat ./results/00-list-of-accounts-in-org.txt
# jq -r '.Accounts[] | "\(.JoinedTimestamp);\(.Id);\(.Name)" ' ./results/00-list-of-accounts-in-org.txt  | sort | grep '2022-03' | awk -F ';' '{print $2 ";" $3}'
# jq -r '.Accounts[] | "\(.JoinedTimestamp);\(.Id);\(.Name)" ' ./results/00-list-of-accounts-in-org.txt  | sort | grep '2022-03' | awk -F ';' '{print $2 ";" $3}' > work-on-this-list-of-accounts.txt
#
# pick your for-loop
#
#for account in $(jq '.Accounts[] | .Id + ";" + .Name' ./results/00-list-of-accounts-in-org.txt | tr -d '"' | sort | grep ^${number} | grep -v ${filter_name} ); do
#for account in $(jq '.Accounts[] | .Id + ";" + .Name' ./results/00-list-of-accounts-in-org.txt | tr -d '"' | sort | grep ^${number} ); do
 for account in $(cat ./work-on-this-list-of-acocunts.txt | tr -d '"' | sort | grep ^${number} ); do

    #sample account result: "000123456789;my-super-aws-account-name"
    account_id=$(  echo ${account} | cut -f1 -d ';' )
    account_name=$(echo ${account} | cut -f2 -d ';' )

    echo "Getting ready to run a command on this account : ${account_id}; ${account_name}
        "
    run_this_command ${account_id} ${account_name} ${role_name} &

done