Sunday, September 9, 2018

Using AWS CLI



Amazon provide an AWS Command Line Interface which gives the ability to gather information from the command prompt, and potentially create text files and load these into a database for better reporting.


Download the installer and run it

Download and install Python V3.6 from https://www.python.org/downloads/release

Make sure you select ‘Add to path’

Once installed, run

pip install awscli

from the command prompt

Run this to upgrade to the latest version

pip install --user --upgrade awscli

You may get another prompt if you are not using the latest version of something – run the command that will be displayed

Close the command window and re-open

Log into the AWS console and select ‘My Security Credentials’
Click on ‘Access keys…’

Click on ‘Create New Access Key’

The key will be created – make a note and / or download the file

Go to the RDS instance in the console, and note the Availability Zone
Go to the command prompt, run


aws configure

And enter the information – note the default region name drops off the last character. If in doubt, go to https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html

If all the information is correct, you will now be able to run commands against your instances (if you are using multi-factor authentication, see below)


aws rds describe-db-instances



You can basically do anything


aws rds stop-db-instance –db-instance-identifier MGAMON




If you are using Multi Factor Authentication (MFA), then this won’t work. You need to run a command to generate temporary credentials by entering the code from the Google Authenticator app.

You need to know your ARN – go to IAM from the AWS console and navigate to your username



You then run the command, and put the token code in

aws sts get-session-token --serial-number arn:aws:iam::83XXXXXXXX64:mfa/user.name --token-code 537662

This will return a set of credentials

CREDENTIALS     ASIAXXXXXXXXXXXXXZ7JK5A    2018-09-06T17:22:59Z    i5gVyiw+LXRsbUnv1GD/XXXXXXXXXXXXXU6nLk0BtA        FQoGZXIvYXdzEHcaDJ1kZm7jTMh0OI95gCKwAVbTGneVDrFuTbJedKs/oir7D7nT+JQZasMGkhOWaO8X0XnopSFIB+XXXXXXXXXXXXXXXXXo9XOUa8nCodgjH4IWxR2YWdWPmNi3YD5z0FvI+TH0KYExCg0ScceGVxxxxxxxxxxxxxxxxxDgf9syGrVNwqnI9JP2GdgUVp+XXXXXXXJxsZAVjChAIRgEb82LCpsgtqtJWlXXXXXXXXXXXXXXPzwtwF

These are valid for 12 hours.

Set them as environment variables:

export AWS_ACCESS_KEY = ASIAXXXXXXXXXXXXXZ7JK5A
export AWS_SECRET_ACCESS_KEY= i5gVyiw+LXRsbUnv1GD/XXXXXXXXXXXXXU6nLk0BtA
export AWS_SESSION_TOKEN=FQoGZXIvYXdzEHcaDJ1kZm7jTMh0OI95gCKwAVbTGneVDrFuTbJedKs/oir7D7nT+JQZasMGkhOWaO8X0XnopSFIB+XXXXXXXXXXXXXXXXXo9XOUa8nCodgjH4IWxR2YWdWPmNi3YD5z0FvI+xxxxxxxxxxxxxxxxxGrVNwqnI9JP2GdgUVp+XXXXXXXJxsZAVjChAIRgEb82LCpsgtXXXXXXXXXXXXXXLPzwtwF

Create a profile in the .aws/config file with the target role

[default]
aws_access_key_id=AKIAXXXXXXXXXXXXXXXBJJQ
aws_secret_access_key=S+XXXXXXXXXXXXXXXXXXXXXDZk6deZjXXXWw
output = text
region = ap-southeast-2
[profile etrm]
role_arn=arn:aws:iam::32XXXXXXXXXX4:role/UsersMGA
source_profile=default
output=text
region=ap-southeast-2
mfa_serial=arn:aws:iam::83XXXXXXXXX4:mfa/user.name

Note the mfa_serial has “mfa” and not “user” – it won’t work otherwise

You can then run commands and it won’t prompt for the key

aws rds describe-db-instances --profile etrm

DBINSTANCES     5120    False   ap-southeast-2c 7       rds-ca-2015             True            arn:aws:rds:ap-southeast-2:32XXXXXXXX814:db:allegrordsprod       db.m4.4xlarge   allegrordsprod  available             0       db-HDLEXXXXXXXXXRGIXXXY   sqlserver-ee    12.00.5571.0.v1 arn:aws:logs:ap-southeast-2:3XXXXXX4:log-group:RDSOSMetrics:log-stream:db-HDLXXSP4XXXXXXXXX4Y     False    2017-12-05T22:45:06.679Z                2018-09-09T22:54:01Z    license-included        administrator   1       arn:aws:iam::32XXXXXXXXX4:role/rds-monitoring-role      True    False   22:00-22:30   sat:02:00-sat:08:00             False   ap-southeast-2a False   gp2     E. Australia Standard Time
DBPARAMETERGROUPS       prodetrmstacks-rds-paramgroups-allegroparams-1jbXXXXX5m      in-sync
DBSUBNETGROUP   Internal RDS Subnet     prodetrmstacks-rds-subnetgroups-rdsinternal-1XXXXXXXXX05       Complete        vpc-6XXXX05
SUBNETS subnet-dXXd Active
SUBNETAVAILABILITYZONE  ap-southeast-2a
SUBNETS subnet-0XXXXXX55 Active
SUBNETAVAILABILITYZONE  ap-southeast-2c
SUBNETS subnet-dXXXXXX9 Active
SUBNETAVAILABILITYZONE  ap-southeast-2b
ENDPOINT        allegrordsprod.cqmXXXXXX0f.ap-southeast-2.rds.amazonaws.com    Z32TXXXXXXXS0V  1433
OPTIONGROUPMEMBERSHIPS  sqlserverwithbackup     in-sync
VPCSECURITYGROUPS       active  sg-1XXX3

aws rds describe-db-snapshots --max-items 5 --profile etrm

DBSNAPSHOTS     4000    ap-southeast-2c allegrordsprod  arn:aws:rds:ap-southeast-2:32xxxxxxxxx4:snapshot:allegropreprodexport   allegropreprodexport    True    sqlserver-ee    12.00.5546.0.v1 False2017-12-05T22:45:06.679Z         arn:aws:kms:ap-southeast-2:325xxxxxx4:key/9bb411XXXXXXXXXXXXXXX15-95a1-9xxxxxac        license-included        administrator   sqlserverwithbackup     100     1433 2018-03-06T09:31:08.027Z manual  arn:aws:rds:ap-southeast-2:32xxxxxx4:snapshot:rds:allegrordsprod-2018-03-05-22-13    ap-southeast-2  available       gp2     E. Australia Standard Time      vpc-6XXXX
DBSNAPSHOTS     5120    ap-southeast-2c allegrordsprod  arn:aws:rds:ap-southeast-2:32xxxxxx4:snapshot:allegroprodexport      allegroprodexport       False   sqlserver-ee    12.00.5571.0.v1 False2017-12-05T22:45:06.679Z                 license-included        administrator   sqlserverwithbackup     100     1433    2018-09-09T22:33:04.270Z        manual  arn:aws:rds:ap-southeast-2:32xxxxxxx4:snapshot:rds:allegrordsprod-2018-09-08-22-13  ap-southeast-2  available       gp2     E. Australia Standard Time      vpc-6XXX
DBSNAPSHOTS     5120    ap-southeast-2b allegrordspreprod       arn:aws:rds:ap-southeast-2:32xxxxxx4:snapshot:allegrordspreprod-2018-08-13   allegrordspreprod-2018-08-13    False   sqlserver-ee 12.00.5571.0.v1  False   2018-08-07T01:22:50.677Z                        license-included        administrator   sqlserverwithbackup     100     1433    2018-08-13T07:35:26.984Z        manual       available        gp2     E. Australia Standard Time      vpc-6XXXXXXXX5
DBSNAPSHOTS     4500    ap-southeast-2a allegrordspreprod       arn:aws:rds:ap-southeast-2:32xxxxxxx4:snapshot:allegrordspreprod-final-snapshot       allegrordspreprod-final-snapshot        Falsesqlserver-ee     12.00.5546.0.v1 False   2018-03-07T04:39:20.230Z        20000           license-included        administrator   sqlserverwithbackup     100     1433    2018-03-15T05:01:18.538Z     manual                   available       io1     E. Australia Standard Time      vpc-61XXX5
DBSNAPSHOTS     5120    ap-southeast-2c allegrordsprod  arn:aws:rds:ap-southeast-2:32xxxxxxx4:snapshot:allegrordsprod-manual  allegrordsprod-manual   False   sqlserver-ee    12.00.5546.0.v1 False2017-12-05T22:45:06.679Z                 license-included        administrator   sqlserverwithbackup     100     1433    2018-06-13T05:31:03.012Z        manual  arn:aws:rds:ap-southeast-2:32xxxxxxxxx4:snapshot:rds:allegrordsprod-2018-06-12-22-13  ap-southeast-2  available       gp2     E. Australia Standard Time      vpc-6XXXXXX5
NEXTTOKEN       eyJNYXJrxxxxxxxxxxxxxxxxx1bnQiOiA1fQ==