Skip to content

Antonio Feijao UK

Learning by doing!

If any of my knowledge share notes helped you, please consider buying me a coffee ☕️ :)


blog posts

Opinions are my own

  • javascript increase, or change, the video playbackRate speed.

document.querySelector('video').playbackRate = 2.0;

document.getElementsByTagName('video')[0].playbackRate = 2.0;

Basics web scraping using Python3 with BeautifulSoup4 and then convert to Markdown

Basics web scraping using Python3 with BeautifulSoup4 and then converting to Markdown

Basic Python BeautifulSoup4 web scraping and then Markdown

pip install requests
pip install beautifulsoup4
pip install markdownify

import markdownify 

import requests
from bs4 import BeautifulSoup

def beautifulsoup_web_scrape_url(url):
  response = requests.get(url)
  soup = BeautifulSoup(response.content, 'html.parser')
  return str(soup)

url = "https://www.antoniofeijao.com/"

data = beautifulsoup_web_scrape_url(url)

print(data)



# convert html to markdown 
h = markdownify.markdownify(data, heading_style="ATX") 

print(h)


f = open("result.txt", "w")
f.write("##result file done. Woops! I have deleted the content!##")
f.write(h)
f.close()

#open and read the file after the overwriting:
f = open("result.txt", "r")
print(f.read())

inspiration-from


Happy learning

by Antonio Feijao UK

List all AWS VPCs or subnets with theirs tags and list them using jquery

Example AWS cli command with listing using [.jq(https://jqlang.github.io/jq/)].

This commands list all AWS VPCs within the account with their VpcId, CidrBlock and their Tags.

aws ec2 describe-vpcs | jq -r '.Vpcs[] | "\(.VpcId) \t \(.CidrBlock) \t \(.Tags[])" '

It is also possible to "select" a specific Tags.

aws ec2 describe-vpcs | jq -r '.Vpcs[] | "\(.VpcId) \t \(.CidrBlock) \t \(.Tags[] | select(.Key == "Application") | .Value)" '

example, select VPC name and sort by VPC name.

aws ec2 describe-vpcs | jq -r '.Vpcs[] | "\(.VpcId) \t \(.CidrBlock) \t \(.Tags[] | select(.Key == "Name")| .Value)" ' | sort -nk2

documentation for https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/describe-vpcs.html


List all Subnets with filter .key, .value data that I picked.

the sort -nk5 at the end, put on put the subnet with the least number of available IPs, taken from .AvailableIpAddressCount.

aws ec2 describe-subnets | jq -r '.Subnets[] | "\(.AvailabilityZone); \(.AvailabilityZoneId); \(.VpcId); \(.CidrBlock); \t \(.AvailableIpAddressCount); \t \(.Tags[] | select(.Key == "Name")| .Value)  "  ' | sort -nk5

documentation for https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/describe-subnets.html


Next, why not rotate through other AWS accounts in the Org if you have them?! and rotate through regions?! :)

I have been there, done that, so leave the challenge for you :)


Happy learning,

Antonio Feijao UK

lime-linux-ubuntu-step-by-step

LiME on Ubuntu Linux, live memory capture.

sources and learning material:


LiME step by step

My adaptation for manually testing LiME in a step-by-step method.

USE AT YOUR RISK

## check if LiME is installed

if [[ `lsmod|grep lime|wc -l` -gt 0 ]] ; then
    sudo rmmod lime.ko
fi

kernel_release=$(uname -r)
kernel_name=$(uname -s)

echo "
kernel_release : ${kernel_release}
kernel_name    : ${kernel_name}
"

## function - I executed one line at a time
installLimeApt() {
    sudo apt-get -y update
    sudo apt-get -y install git

    sudo apt-get install -y linux-headers-$1
    #sudo apt-get install -y linux-headers-${kernel_release}

    sudo apt-get install -y build-essential

    cd /tmp && sudo rm -rf LiME

    git clone https://github.com/504ensicsLabs/LiME
    # >> could not clone, so I copyed 1 file at a time <<

    cd LiME/src

    make

    lime_path=$(pwd)/lime-$1.ko
    #lime_path=$(pwd)/lime-${kernel_release}.ko
    echo "lime_path : ${lime_path}"
}


# I run the commands one by one
#installLimeApt $kernel_release

# loading the kernel module
sudo insmod $lime_path path=tcp:4444 format=lime localhostonly=1 &

# confirm the LiME kernel module is "listening" on port 4444
netstat -patnl | grep 4444

#sleep 120

if [[ `lsmod|grep lime|wc -l` -gt 0 ]] ; then
    echo "LiME has been loaded"
fi

MEMSIZE=`awk '/MemTotal/ {print $2/1024/1024}' /proc/meminfo`
echo "MEMSIZE: ${MEMSIZE}"

METADATA_FLAG="--metadata uncompressed-size=$MEMSIZE,kernel-name=$kernel_name,kernel-release=$kernel_release"
echo "METADATA_FLAG : ${METADATA_FLAG}"
# sample output >>> `METADATA_FLAG : --metadata uncompressed-size=31.0748,kernel-name=Linux,kernel-release=4.4.0-184-generic`


# copying memory dump into S3
#s3cp() {
# aws s3 cp - {{s3ArtifactLocation}}/linux_memcapture$1 $2 $3 $4
#}

# original command
# cat < /dev/tcp/127.0.0.1/4444 | tee >(gzip | s3cp \".lime.gz\" \"$EXPECTED_SIZE_FLAG\" \"$METADATA_FLAG\" \"$ACL_FLAG\") | sha256sum | s3cp \"_sha256.txt\" \"$ACL_FLAG\"",

# compressed memory
#cat < /dev/tcp/127.0.0.1/4444 | tee >(gzip > ./linux_memcapture.lime.gz)

# raw memory dump

cat < /dev/tcp/127.0.0.1/4444 > ./linux_memcapture.lime
sha256sum linux_memcapture.lime >> _sha256.txt

# remove the kernel module
# most of the time I tested, the kernel module `lime.ko` "removed" itself.

sudo rmmod lime.ko

Happy learning,

Antonio Feijao UK

aws sts decode-authorization-message

Decoding the aws sts decode-authorization-message

I included jq and tr commands to "clean up" and make the decoded message easier to read.

Note: - One of the command is jq, from jquery, which usually doesn't come installed by default in some OS, keep that in mind in case you might need to install it. - You need to have the sts permission to run the decode message - sts:DecodeAuthorizationMessage

  • TIP - I added the message to a variable, which makes it easier to read the command aws sts decode-authorization-message
enc_message="akjhkajshdkjahsdkjhakjshdais8duas8d7a98sd7a9s87da....example...." #replace with your encoded message

aws sts decode-authorization-message --encoded-message ${enc_message} | jq '.DecodedMessage' | tr -d '\\' | tr ',' '\n'

further reading from AWS documentation:


Happy learning,

Antonio Feijao UK

Linux dnsmasq options, dns-server and more

Linux dnsmasq options, dns-server and more.

More about dnsmasq in here - https://en.wikipedia.org/wiki/Dnsmasq


Starting with an example of a DHCP pool definition for dnsmasq

(...)

 dhcp-range=192.168.100.101,192.168.100.199,255.255.255.0,8h   # dhcp range
 dhcp-option=3,192.168.100.20                                  # default-gateway
 dhcp-option=6,1.1.1.3,1.0.0.3                                 # dns
 dhcp-option=15,mylocalnetwork.local                           # local-domain
 dhcp-option=44,0.0.0.0                                        # netbios server. Disabling NetBIOS over TCP/IP can improve security by reducing the attack surface of a system. However, it may also impact the functionality of certain legacy applications and networked devices that rely on NetBIOS

(...)

and with the command dnsmasq --help dhcp you get the below output, which shows what options are available and what they are.

Known DHCP options:
  1 netmask
  2 time-offset
  3 router
  6 dns-server
  7 log-server
  9 lpr-server
 13 boot-file-size
 15 domain-name
 16 swap-server
 17 root-path
 18 extension-path
 19 ip-forward-enable
 20 non-local-source-routing
 21 policy-filter
 22 max-datagram-reassembly
 23 default-ttl
 26 mtu
 27 all-subnets-local
 31 router-discovery
 32 router-solicitation
 33 static-route
 34 trailer-encapsulation
 35 arp-timeout
 36 ethernet-encap
 37 tcp-ttl
 38 tcp-keepalive
 40 nis-domain
 41 nis-server
 42 ntp-server
 44 netbios-ns
 45 netbios-dd
 46 netbios-nodetype
 47 netbios-scope
 48 x-windows-fs
 49 x-windows-dm
 58 T1
 59 T2
 60 vendor-class
 64 nis+-domain
 65 nis+-server
 66 tftp-server
 67 bootfile-name
 68 mobile-ip-home
 69 smtp-server
 70 pop3-server
 71 nntp-server
 74 irc-server
 77 user-class
 80 rapid-commit
 93 client-arch
 94 client-interface-id
 97 client-machine-id
119 domain-search
120 sip-server
121 classless-static-route
125 vendor-id-encap
150 tftp-server-address
255 server-ip-address

Happy learning,

Antonio Feijao UK

AWS CLI command-line script - How to automatically delete the Default-VPCs in all AWS regions

AWS CLI command-line script to automatically delete all Default-VPCs in all AWS regions.

The script needs to have enough permissions to run the actions.

The script will fail if there are other dependencies than the ones dealt with in the script.


linux-bash-script

USE AT YOUR OWN RISK

#!/bin/bash

## uncomment to see the commands as they are executed.
#set -x

## gets a list of all AWS regions

LIST_OF_REGIONS=$(aws ec2 describe-regions --all-regions --query "Regions[].{Name:RegionName}" --output text)

## for-loop to cycle through all regions

for REGION in ${LIST_OF_REGIONS}; do
    echo "---------"
    echo "Region: ${REGION}"

    RESULT=$(aws ec2 describe-vpcs --region ${REGION} --query "Vpcs[].[VpcId,IsDefault]" --output text 2>/dev/null)
    if [ -z "${RESULT}" ];
        then
            echo "NULL - No Default-VPC in the region: ${REGION}"
        else
            echo "Not NULL - There is a Default-VPC in the region: ${REGION}"
            ##
            ## --- use AT YOUR OWN RISK ---
            ##
            ## Uncomment the `aws ec2 ...` lines below to delete the default VPC in all regions.
            ## The script still needs to have enough permission to run the commands.
            ##

            VPCID=${RESULT:0:-5}
            echo "${REGION} : ${VPCID}"

            ## We need to detach AND delete the correct Internet Gateway (IGW), before we can delete the Default-VPC.

            IGW=$(aws ec2 describe-internet-gateways --region ${REGION} --filters "Name=attachment.vpc-id,Values=${VPCID}" --query 'InternetGateways[].InternetGatewayId' --output text)

            ## IF IGW exists, then detach and delete the IGW from the Default-VPC
            if [ -z "${IGW}" ];
                then
                    echo "NULL - IGW already removed."
                else
                    echo "Removing and deleting the IGW: ${IGW}, from the Default-VPC: ${VPCID}."
                    #aws ec2 detach-internet-gateway --region ${REGION} --internet-gateway-id ${IGW} --vpc-id ${VPCID}
                    #aws ec2 delete-internet-gateway --region ${REGION} --internet-gateway-id ${IGW}
            fi

            ## From my own experience, also need to delete any subnets associated with the Default-VPC.

            LIST_OF_SUBNETS=$(aws ec2 describe-subnets --region ${REGION} --filters "Name=vpc-id,Values=${VPCID}" --query "Subnets[*].[SubnetId]" --output text)

            echo "List of subnets on the Default-VPC: ${LIST_OF_SUBNETS}"

            ## could add an if loop here too...

            for SUBNET in ${LIST_OF_SUBNETS}; do
                #aws ec2 delete-subnet --region ${REGION} --subnet-id ${SUBNET}
            done

            ## Finally, delete the Default-VPC.

            #aws ec2 delete-vpc --vpc-id ${VPCID} --region ${REGION} 2>/dev/null && echo "Default-VPC removed succesfully." || echo "Something is still not right..."
    fi

done

aws-documentation


Happy learning,

Antonio Feijao UK

AWS boto3 credentials, boto session and boto3 available clients in python for the region the session was created.

About AWS credentials, boto3.session, list boto3 available clients in python3, load and access AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN.

Documentation here https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html.

My notes below.


aws-boto3-session

Example of AWS boto session credentials.

import boto3

help(boto3.session.Session)
  • output of help(...)
Help on class Session in module boto3.session:

class Session(builtins.object)
 |  Session(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, region_name=None, botocore_session=None, profile_name=None)
 |
 |  A session stores configuration state and allows you to create service
 |  clients and resources.
 |
 |  :type aws_access_key_id: string
 |  :param aws_access_key_id: AWS access key ID
 |  :type aws_secret_access_key: string
 |  :param aws_secret_access_key: AWS secret access key
 |  :type aws_session_token: string
 |  :param aws_session_token: AWS temporary session token
 |  :type region_name: string
 |  :param region_name: Default region when creating new connections
 |  :type botocore_session: botocore.session.Session
 |  :param botocore_session: Use this Botocore session instead of creating
 |                           a new default one.
 |  :type profile_name: string
 |  :param profile_name: The name of a profile to use. If not given, then
 |                       the default profile is used.
 |
 |  Methods defined here:

 (...)

creating-aws-boto3-session-with-aws-access-keys-secret-access-key-and-token

In this method, you must pass the AWS_ACCESS_KEY, SECRET and TOKEN through environment variables. It is not recommended to hard-code credentials.

session = boto3.session.Session(
    aws_access_key_id     = AWS_ACCESS_KEY_ID,
    aws_secret_access_key = AWS_SECRET_ACCESS_KEY,
    aws_session_token     = AWS_SESSION_TOKEN,
    region_name='eu-west-2',
    botocore_session=None,
    profile_name=None
)

In this method, the boto3, session will look for credentials in various locations based on predefined order, as described in the documentation https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html.

session = boto3.session.Session(
    region_name='eu-west-2'
)

using-the-session-list-available-clients

services = session.get_available_services()

print(services)
['accessanalyzer', 'account', 'acm', 'acm-pca', 'alexaforbusiness', 'amp', 'amplify', 'amplifybackend', 'amplifyuibuilder', 'apigateway', 'apigatewaymanagementapi', 'apigatewayv2', 'appconfig', 'appconfigdata', 'appflow', 'appintegrations', 'application-autoscaling', 'application-insights', 'applicationcostprofiler', 'appmesh', 'apprunner', 'appstream', 'appsync', 'arc-zonal-shift', 'athena', 'auditmanager', 'autoscaling', 'autoscaling-plans', 'backup', 'backup-gateway', 'backupstorage', 'batch', 'billingconductor', 'braket', 'budgets', 'ce', 'chime', 'chime-sdk-identity', 'chime-sdk-media-pipelines', 'chime-sdk-meetings', 'chime-sdk-messaging', 'chime-sdk-voice', 'cleanrooms', 'cloud9', 'cloudcontrol', 'clouddirectory', 'cloudformation', 'cloudfront', 'cloudhsm', 'cloudhsmv2', 'cloudsearch', 'cloudsearchdomain', 'cloudtrail', 'cloudtrail-data', 'cloudwatch', 'codeartifact', 'codebuild', 'codecatalyst', 'codecommit', 'codedeploy', 'codeguru-reviewer', 'codeguru-security', 'codeguruprofiler', 'codepipeline', 'codestar', 'codestar-connections', 'codestar-notifications', 'cognito-identity', 'cognito-idp', 'cognito-sync', 'comprehend', 'comprehendmedical', 'compute-optimizer', 'config', 'connect', 'connect-contact-lens', 'connectcampaigns', 'connectcases', 'connectparticipant', 'controltower', 'cur', 'customer-profiles', 'databrew', 'dataexchange', 'datapipeline', 'datasync', 'dax', 'detective', 'devicefarm', 'devops-guru', 'directconnect', 'discovery', 'dlm', 'dms', 'docdb', 'docdb-elastic', 'drs', 'ds', 'dynamodb', 'dynamodbstreams', 'ebs', 'ec2', 'ec2-instance-connect', 'ecr', 'ecr-public', 'ecs', 'efs', 'eks', 'elastic-inference', 'elasticache', 'elasticbeanstalk', 'elastictranscoder', 'elb', 'elbv2', 'emr', 'emr-containers', 'emr-serverless', 'es', 'events', 'evidently', 'finspace', 'finspace-data', 'firehose', 'fis', 'fms', 'forecast', 'forecastquery', 'frauddetector', 'fsx', 'gamelift', 'gamesparks', 'glacier', 'globalaccelerator', 'glue', 'grafana', 'greengrass', 'greengrassv2', 'groundstation', 'guardduty', 'health', 'healthlake', 'honeycode', 'iam', 'identitystore', 'imagebuilder', 'importexport', 'inspector', 'inspector2', 'internetmonitor', 'iot', 'iot-data', 'iot-jobs-data', 'iot-roborunner', 'iot1click-devices', 'iot1click-projects', 'iotanalytics', 'iotdeviceadvisor', 'iotevents', 'iotevents-data', 'iotfleethub', 'iotfleetwise', 'iotsecuretunneling', 'iotsitewise', 'iotthingsgraph', 'iottwinmaker', 'iotwireless', 'ivs', 'ivs-realtime', 'ivschat', 'kafka', 'kafkaconnect', 'kendra', 'kendra-ranking', 'keyspaces', 'kinesis', 'kinesis-video-archived-media', 'kinesis-video-media', 'kinesis-video-signaling', 'kinesis-video-webrtc-storage', 'kinesisanalytics', 'kinesisanalyticsv2', 'kinesisvideo', 'kms', 'lakeformation', 'lambda', 'lex-models', 'lex-runtime', 'lexv2-models', 'lexv2-runtime', 'license-manager', 'license-manager-linux-subscriptions', 'license-manager-user-subscriptions', 'lightsail', 'location', 'logs', 'lookoutequipment', 'lookoutmetrics', 'lookoutvision', 'm2', 'machinelearning', 'macie', 'macie2', 'managedblockchain', 'marketplace-catalog', 'marketplace-entitlement', 'marketplacecommerceanalytics', 'mediaconnect', 'mediaconvert', 'medialive', 'mediapackage', 'mediapackage-vod', 'mediapackagev2', 'mediastore', 'mediastore-data', 'mediatailor', 'memorydb', 'meteringmarketplace', 'mgh', 'mgn', 'migration-hub-refactor-spaces', 'migrationhub-config', 'migrationhuborchestrator', 'migrationhubstrategy', 'mobile', 'mq', 'mturk', 'mwaa', 'neptune', 'network-firewall', 'networkmanager', 'nimble', 'oam', 'omics', 'opensearch', 'opensearchserverless', 'opsworks', 'opsworkscm', 'organizations', 'osis', 'outposts', 'panorama', 'payment-cryptography', 'payment-cryptography-data', 'personalize', 'personalize-events', 'personalize-runtime', 'pi', 'pinpoint', 'pinpoint-email', 'pinpoint-sms-voice', 'pinpoint-sms-voice-v2', 'pipes', 'polly', 'pricing', 'privatenetworks', 'proton', 'qldb', 'qldb-session', 'quicksight', 'ram', 'rbin', 'rds', 'rds-data', 'redshift', 'redshift-data', 'redshift-serverless', 'rekognition', 'resiliencehub', 'resource-explorer-2', 'resource-groups', 'resourcegroupstaggingapi', 'robomaker', 'rolesanywhere', 'route53', 'route53-recovery-cluster', 'route53-recovery-control-config', 'route53-recovery-readiness', 'route53domains', 'route53resolver', 'rum', 's3', 's3control', 's3outposts', 'sagemaker', 'sagemaker-a2i-runtime', 'sagemaker-edge', 'sagemaker-featurestore-runtime', 'sagemaker-geospatial', 'sagemaker-metrics', 'sagemaker-runtime', 'savingsplans', 'scheduler', 'schemas', 'sdb', 'secretsmanager', 'securityhub', 'securitylake', 'serverlessrepo', 'service-quotas', 'servicecatalog', 'servicecatalog-appregistry', 'servicediscovery', 'ses', 'sesv2', 'shield', 'signer', 'simspaceweaver', 'sms', 'sms-voice', 'snow-device-management', 'snowball', 'sns', 'sqs', 'ssm', 'ssm-contacts', 'ssm-incidents', 'ssm-sap', 'sso', 'sso-admin', 'sso-oidc', 'stepfunctions', 'storagegateway', 'sts', 'support', 'support-app', 'swf', 'synthetics', 'textract', 'timestream-query', 'timestream-write', 'tnb', 'transcribe', 'transfer', 'translate', 'verifiedpermissions', 'voice-id', 'vpc-lattice', 'waf', 'waf-regional', 'wafv2', 'wellarchitected', 'wisdom', 'workdocs', 'worklink', 'workmail', 'workmailmessageflow', 'workspaces', 'workspaces-web', 'xray']

import-pretty-print-pprint-as-pp-for-list-ouput

```py from pprint import pprint as pp

pp(session.get_available_services()) ['accessanalyzer', 'account', 'acm', 'acm-pca', 'alexaforbusiness', 'amp', 'amplify', 'amplifybackend', 'amplifyuibuilder', 'apigateway', 'apigatewaymanagementapi', 'apigatewayv2', 'appconfig', 'appconfigdata', 'appflow', 'appintegrations', 'application-autoscaling', 'application-insights', 'applicationcostprofiler', 'appmesh', 'apprunner', 'appstream', 'appsync', 'arc-zonal-shift', 'athena', 'auditmanager', 'autoscaling', 'autoscaling-plans', 'backup', 'backup-gateway', 'backupstorage', 'batch', 'billingconductor', 'braket', 'budgets', 'ce', 'chime', 'chime-sdk-identity', 'chime-sdk-media-pipelines', 'chime-sdk-meetings', 'chime-sdk-messaging', 'chime-sdk-voice', 'cleanrooms', 'cloud9', 'cloudcontrol', 'clouddirectory', 'cloudformation', 'cloudfront', 'cloudhsm', 'cloudhsmv2', 'cloudsearch', 'cloudsearchdomain', 'cloudtrail', 'cloudtrail-data', 'cloudwatch', 'codeartifact', 'codebuild', 'codecatalyst', 'codecommit', 'codedeploy', 'codeguru-reviewer', 'codeguru-security', 'codeguruprofiler', 'codepipeline', 'codestar', 'codestar-connections', 'codestar-notifications', 'cognito-identity', 'cognito-idp', 'cognito-sync', 'comprehend', 'comprehendmedical', 'compute-optimizer', 'config', 'connect', 'connect-contact-lens', 'connectcampaigns', 'connectcases', 'connectparticipant', 'controltower', 'cur', 'customer-profiles', 'databrew', 'dataexchange', 'datapipeline', 'datasync', 'dax', 'detective', 'devicefarm', 'devops-guru', 'directconnect', 'discovery', 'dlm', 'dms', 'docdb', 'docdb-elastic', 'drs', 'ds', 'dynamodb', 'dynamodbstreams', 'ebs', 'ec2', 'ec2-instance-connect', 'ecr', 'ecr-public', 'ecs', 'efs', 'eks', 'elastic-inference', 'elasticache', 'elasticbeanstalk', 'elastictranscoder', 'elb', 'elbv2', 'emr', 'emr-containers', 'emr-serverless', 'es', 'events', 'evidently', 'finspace', 'finspace-data', 'firehose', 'fis', 'fms', 'forecast', 'forecastquery', 'frauddetector', 'fsx', 'gamelift', 'gamesparks', 'glacier', 'globalaccelerator', 'glue', 'grafana', 'greengrass', 'greengrassv2', 'groundstation', 'guardduty', 'health', 'healthlake', 'honeycode', 'iam', 'identitystore', 'imagebuilder', 'importexport', 'inspector', 'inspector2', 'internetmonitor', 'iot', 'iot-data', 'iot-jobs-data', 'iot-roborunner', 'iot1click-devices', 'iot1click-projects', 'iotanalytics', 'iotdeviceadvisor', 'iotevents', 'iotevents-data', 'iotfleethub', 'iotfleetwise', 'iotsecuretunneling', 'iotsitewise', 'iotthingsgraph', 'iottwinmaker', 'iotwireless', 'ivs', 'ivs-realtime', 'ivschat', 'kafka', 'kafkaconnect', 'kendra', 'kendra-ranking', 'keyspaces', 'kinesis', 'kinesis-video-archived-media', 'kinesis-video-media', 'kinesis-video-signaling', 'kinesis-video-webrtc-storage', 'kinesisanalytics', 'kinesisanalyticsv2', 'kinesisvideo', 'kms', 'lakeformation', 'lambda', 'lex-models', 'lex-runtime', 'lexv2-models', 'lexv2-runtime', 'license-manager', 'license-manager-linux-subscriptions', 'license-manager-user-subscriptions', 'lightsail', 'location', 'logs', 'lookoutequipment', 'lookoutmetrics', 'lookoutvision', 'm2', 'machinelearning', 'macie', 'macie2', 'managedblockchain', 'marketplace-catalog', 'marketplace-entitlement', 'marketplacecommerceanalytics', 'mediaconnect', 'mediaconvert', 'medialive', 'mediapackage', 'mediapackage-vod', 'mediapackagev2', 'mediastore', 'mediastore-data', 'mediatailor', 'memorydb', 'meteringmarketplace', 'mgh', 'mgn', 'migration-hub-refactor-spaces', 'migrationhub-config', 'migrationhuborchestrator', 'migrationhubstrategy', 'mobile', 'mq', 'mturk', 'mwaa', 'neptune', 'network-firewall', 'networkmanager', 'nimble', 'oam', 'omics', 'opensearch', 'opensearchserverless', 'opsworks', 'opsworkscm', 'organizations', 'osis', 'outposts', 'panorama', 'payment-cryptography', 'payment-cryptography-data', 'personalize', 'personalize-events', 'personalize-runtime', 'pi', 'pinpoint', 'pinpoint-email', 'pinpoint-sms-voice', 'pinpoint-sms-voice-v2', 'pipes', 'polly', 'pricing', 'privatenetworks', 'proton', 'qldb', 'qldb-session', 'quicksight', 'ram', 'rbin', 'rds', 'rds-data', 'redshift', 'redshift-data', 'redshift-serverless', 'rekognition', 'resiliencehub', 'resource-explorer-2', 'resource-groups', 'resourcegroupstaggingapi', 'robomaker', 'rolesanywhere', 'route53', 'route53-recovery-cluster', 'route53-recovery-control-config', 'route53-recovery-readiness', 'route53domains', 'route53resolver', 'rum', 's3', 's3control', 's3outposts', 'sagemaker', 'sagemaker-a2i-runtime', 'sagemaker-edge', 'sagemaker-featurestore-runtime', 'sagemaker-geospatial', 'sagemaker-metrics', 'sagemaker-runtime', 'savingsplans', 'scheduler', 'schemas', 'sdb', 'secretsmanager', 'securityhub', 'securitylake', 'serverlessrepo', 'service-quotas', 'servicecatalog', 'servicecatalog-appregistry', 'servicediscovery', 'ses', 'sesv2', 'shield', 'signer', 'simspaceweaver', 'sms', 'sms-voice', 'snow-device-management', 'snowball', 'sns', 'sqs', 'ssm', 'ssm-contacts', 'ssm-incidents', 'ssm-sap', 'sso', 'sso-admin', 'sso-oidc', 'stepfunctions', 'storagegateway', 'sts', 'support', 'support-app', 'swf', 'synthetics', 'textract', 'timestream-query', 'timestream-write', 'tnb', 'transcribe', 'transfer', 'translate', 'verifiedpermissions', 'voice-id', 'vpc-lattice', 'waf', 'waf-regional', 'wafv2', 'wellarchitected', 'wisdom', 'workdocs', 'worklink', 'workmail', 'workmailmessageflow', 'workspaces', 'workspaces-web', 'xray'] ```


Happy learning,

Antonio Feijao UK

AWS Console information. Reading the userInfo cookie information to display, alert, or anything else you want.
For example, include a banner on your AWS console with highlighting when you login as "AWS Administrator" role.
This can then be used in various applications.


sample code that "grabs" the userInfo and create an banner alert

javascript:(function () {
    function fullDecode (input) {
        let decoded = decodeURIComponent(input);
        return (decoded == input ? decoded : fullDecode(decoded))
    };
    let userInfo = document.cookie.replace(/(?:(?:^|.*;\s*)aws-userInfo\s*\=\s*([^;]*).*$)|^.*$/, "$1");
    alert(JSON.stringify(JSON.parse(fullDecode(userInfo)), null, 4))
})();

source code https://gist.github.com/ajkerrigan/0e2348d4ed960409b462e8aaca230d36


sample code that "grabs" the userInfo and outputs in the console

let userInfo = document.cookie.replace(/(?:(?:^|.*;\s*)aws-userInfo\s*\=\s*([^;]*).*$)|^.*$/, "$1");
let decoded = decodeURIComponent(userInfo);
JSON.stringify(JSON.parse(decoded), null, 1);

TBC...


Happy learning,

Antonio Feijao UK

Using a Raspberry Pi 4 as a router with iptables

Using a Raspberry Pi 4 as a router with iptables. With iptables we need to know more about what we are doing. ufw is great, it works as a leayer on top of iptables for with easy management, however you will not learn the real "thing", the network flow details, the beauty of "source" and "destination", and more...

So, I want to learn more, therefore I went on to learn the details of iptables and source destination IPs, NAT (MASQUERADE), source and destination ports, states...

the raspberry pi 4 basics

raspberry pi 4 updates

apt update && apt upgrade -y

apt autoremove -y

## useful

apt install dnsutils

raspberry pi 4 disable ipv6 at boot

vim /boot/cmdline.txt and add ipv6.disable=1 to the end of the line

example

console=tty1 root=PARTUUID=xxxxXXxx-xx rootfstype=ext4 fsck.repair=yes rootwait ipv6.disable=1

raspberry pi 4 enable IPv4 forward and disable IPv6

  • cat /etc/sysctl.d/local.conf
net.ipv4.ip_forward=1

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

raspberry pi 4 nat with iptables

  • cat reset-iptables.sh
#!/bin/bash -x
##
## source - https://www.linode.com/docs/guides/linux-router-and-ip-forwarding/
##
## 2023-06 - adapted and tweaked by AntonioFeijaoUK
##

## reset iptables
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT

## enable forwarding packets
iptables -A FORWARD -j ACCEPT

## detecting the default route interface
GATEWAY_INTERFACE=$(route -n | grep ^'0.0.0.0' | rev | cut -f 1 -d ' ' | rev | head -n1)
echo "your gateway interface is : ${GATEWAY_INTERFACE}"

## enable NAT on the outside interface for the internal subnet SOURCE_SUBNET
SOURCE_SUBNET="192.168.0.0/24"
iptables -t nat -s ${SOURCE_SUBNET} -I POSTROUTING -o ${GATEWAY_INTERFACE} -j MASQUERADE

## other good sources with details info
# - https://raspberrytips.com/raspberry-pi-firewall/
# - https://www.packetswitch.co.uk/raspberry/
## - enabled established connections - this is not needed as the FORWARD is set to default ACCEPT
#iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
#iptables -A FORWARD -i eth0 -d ${SOURCE_SUBNET} -m state --state RELATED,ESTABLISHED -j ACCEPT
#iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT

## save IPv4 iptables
iptables-save | sudo tee /etc/iptables/rules.v4


##################################################################
### similar but for IPv6 and to block by default
ip6tables -F
ip6tables -X
ip6tables -t nat -F
ip6tables -t nat -X
#ip6tables -t mangle -F
#ip6tables -t mangle -X
ip6tables -P INPUT DROP
ip6tables -P OUTPUT DROP
ip6tables -P FORWARD DROP

## enable forwarding packets
ip6tables -A FORWARD -j DROP

## enable NAT on the outside interface
#iptables -t nat -s 192.168.0.0/24 -I POSTROUTING -o enp0s3 -j MASQUERADE

## save IPv6 iptables
ip6tables-save | sudo tee /etc/iptables/rules.v6


## USEFUL COMMANDS

echo "

useful command to check your NAT MASQUERADE is working

    \`iptables -t nat -L -nv\`

"

raspberry pi 4 static IP, dhcp and gateway metrics

  • cat /etc/dhcpcd.conf | egrep -v '^#|^$'
hostname
clientid
persistent
option rapid_commit
option domain_name_servers, domain_name, domain_search, host_name
option classless_static_routes
option interface_mtu
require dhcp_server_identifier
slaac private
interface wlan0
metric 300
static domain_name_servers=94.140.14.15 94.140.15.16
interface eth0
domain antoniofeijaouk.local
search antoniofeijaouk.local
metric 200
static ip_address=192.168.0.4/24
static routers=192.168.0.1
static domain_name_servers=94.140.14.15 94.140.15.16

raspberry pi 4 - verify the dns upstream servers

resolvconf -l

resolvectl status

Happy learning,

Antonio Feijao UK