CLI, CloudShell, CDK, SAM

Update the awscli regularly

[2022-Jun] aws-cli/2.7.9 Python/3.9.11 Windows/10 exe/AMD64 prompt/off

Explore further

• AWS CloudShell - https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html

AWS CLI

The AWS Command Line Interface (AWS CLI) enables you to interact with AWS services using commands in your command-line shell.

Setup

• Prerequisite: install unzip;

• Install - AWS CLI

• Configure - AWS CLI

• Review Docs - AWS CLI

Install unzip

$ sudo apt install unzip

Install - AWS CLI

Source: AWS Installing or updating the latest version of the AWS CLI

Linux

$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
# Install NEW!! See update below.
$ sudo ./aws/install
You can now run: /usr/local/bin/aws --version
$ aws --version
aws-cli/2.7.35 Python/3.9.11 Linux/5.10.16.3-microsoft-standard-WSL2 exe/x86_64.ubuntu.20 prompt/off
$ which aws
/usr/local/bin/aws
$ ls -l /usr/local/bin/aws
lrwxrwxrwx 1 root root 37 Sep 27 17:45 /usr/local/bin/aws -> /usr/local/aws-cli/v2/current/bin/aws
$

Windows

Update the awscli

$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip

$ sudo ./aws/install --bin-dir /usr/local/bin --install-dir /usr/local/aws-cli --update

aws-cli/2.17.14 Python/3.11.9 Linux/5.15.153.1-microsoft-standard-WSL2 exe/x86_64.ubuntu.22

Configure - AWS CLI

Source: AWS CLI - Quick Setup - New configuration quick setup

• For new & clean environment, configure the "default" config and profile

$ aws configure
AWS Access Key ID [None]: <replace-with-access-key-id>
AWS Secret Access Key [None]: <replace-with-access-key>
Default region name [None]: us-west-2
Default output format [None]: json

The configurations are stored in the ~/.aws folder for (macOS/Linux); or %USERPROFILE%.aws\ folder for (Windows), in two files: config and credentials

config

[default]
region=us-west-2

credentials

• [Optional] Configure for a specific "profile", for example: "produser"

$ aws configure --profile produser

Docs - AWS CLI

• AWS CLI Command Reference v2

• AWS Command Line Interface User Guide for Version 2

How-to

Get: account number, default region

// Get acct-number from AWS console or
$ aws sts get-caller-identity
$ aws sts get-caller-identity --query Account --output text

// Get the default region for the profile
$ aws configure get region

after windows update aws error: bus error

Resolution: reinstall aws cli; see Setup

Setup environment variables

AWS Environment variables to configure the AWS CLI

Get AWS Account Number using aws sts

aws sts - Secure Token Service

aws sts get-caller-identity
{
    "UserId": "AIDAJ<truncated>",
    "Account": "349<truncated>",
    "Arn": "arn:aws:iam::349<truncated>:user/gabe"
}

CloudShell

Source: AWS CloudShell

Command line access to AWS resources and tools directly from a browser.

There is no additional charge for AWS CloudShell.

Useful commands to explore:

• zsh; vim ~/.zshrc

CDK

The AWS Cloud Development Kit (AWS CDK) is a software development framework for defining your cloud application resources using familiar programming languages including: Python, Typescript, etc. Regardless of the programming languages, CDK uses the same back end which runs on Node.js. Therefore, the development computer needs to have the Node.js prerequisite installed.

Source: Getting started with the AWS CDK

Step-by-Step Instructions

• Prerequisite - Node.js & NPM; See SETUP > Dev Environment > Node.js

• Prerequisite - AWS CLI; See Configure - AWS CLI; and review CDK credential requirements below

• Install - CDK

• CDK Bootstrapping

• [Optional] Setup User Policy

• Review Docs - CDK

CDK credentials requirements

Ensure the development computer has been configured with the "region" and "credentials" in the config and credentials files, respectively. Both files are stored in the ~/.aws folder. For more details, see Configure - AWS CLI.

NOTE: although the AWS CDK uses credentials from the same configuration files as the AWS CLI, it behaves differently. If you use a named profile from the credentials file, the config file must have a profile of the same name specifying the region. In addition, do not use a profile named containing the word "default" (e.g. [profile default]). [see Getting started with the AWS CDK]

Install - CDK

$ npm install -g aws-cdk
$ cdk --version
2.29.0 (build 47d7ec4)
$ cdk --help

// To update aws-cdk
$ npm install -g aws-cdk

CDK Bootstrapping

Deploying stacks with the AWS CDK requires the AWS CloudFormation stack (called CDKToolkit) and a dedicated Amazon S3 buckets to be available during deployment. The process is called bootstrapping, and it's a onetime setup for the AWS Account. The S3 bucket and the CloudFormation "CDKToolkit" resources are reusable by any cdk projects.

$ cdk bootstrap aws://ACCOUNT-NUMBER/REGION

// Get acct-number from AWS console or
$ aws sts get-caller-identity

// Get the default region for the profile
$ aws configure get region

// cdk bootstrap output
 ⏳  Bootstrapping environment aws://349<truncated>/us-west-2...
Trusted accounts for deployment: (none)
Trusted accounts for lookup: (none)
Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
CDKToolkit: creating CloudFormation changeset...
 ✅  Environment aws://349<truncated>/us-west-2 bootstrapped.

// Go to AWS console > S3 to see the newly created bucket:
// cdk-hnb659fds-assets-349<truncate>-us-west-2

During "CDK bootstrapping", the following five IAM Roles are also created:

cdk-<uniqueId>-cfn-exec-role-<account-no>-<region>
cdk-<uniqueId>-deploy-role-<account-no>-<region>
cdk-<uniqueId>-file-publishing-role-<account-no>-<region>
cdk-<uniqueId>-image-publishing-role-<account-no>-<region>
cdk-<uniqueId>-lookup-role-<account-no>-<region>

CLEANUP

As mentioned above, the resources created by bootstrapping are reusable across cdk projects so they should not be deleted. However, to completely cleanup all the resources created during CDK bootstrapping:

• Delete the S3 bucket

• Delete the CloudFormation "CDKToolkit"

• Delete the five IAM roles; go to IAM console, and search for "cdk-"

[Optional] Setup User Policy

• Define user policy needed to deploy AWS S3.

If the user belongs to the Administrator group; then the following policy will not be needed.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "cloudformation:DescribeStacks",
                "cloudformation:CreateChangeSet",
                "cloudformation:DescribeChangeSet",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:DescribeStackEvents",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DeleteStack",
                "cloudformation:GetTemplate",
                "s3:CreateBucket"
            ],
            "Resource": "*"
        }
    ]
}

Tutorials

• Python - Python Tutorial

• Python - Youtube - Getting Started with AWS CDK and Python | Step by Step Tutorial

• aws-samples/aws-cdk-examples

Docs - CDK

• API - Python - PyPi.org - aws-cdk.aws-s3

• API - Typescript - tbd

• AWS Cloud Development Kit (CDK) v2 - Developer Guides

Troubleshooting

cdk deploy error: fail: No bucket named; Is account <acct#> bootstrapped?

One of the two resources created by the cdk bootstrap: the S3 bucket or the CloudFormation stack called "CDKToolkit" was not delete. Note: if you only delete the S3 bucket, the next cdk deploy will fail indicating that the account has not been "bootstrap" but attempt to cdk bootstrap will look like success with "no change" shown on the output, but the S3 bucket is not created and the cdk deploy will still fail, as shown below. The reason is because cdk bootstrap still detects the CloudFormation resource, i.e., "CDKToolkit".

$ cdk deploy MythicalMysfits-Website
✨  Synthesis time: 5.85s
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
[...]
Do you wish to deploy these changes (y/n)? y
MythicalMysfits-Website: deploying...
[0%] start: Publishing b7f327b4415410f319943b754edb274645a9d6850369ae4da9ba209858099210:current_account-current_region
[0%] start: Publishing f98b78092dcdd31f5e6d47489beb5f804d4835ef86a8085d0a2053cb9ae711da:current_account-current_region
[0%] start: Publishing e03a45d87457d42f35063ed0e30080682b7724c5d983d0a955d31010fee11946:current_account-current_region
[0%] start: Publishing 615e663697e0203efc01772c88062bfa12b8bc1d72c01e989686aab4416341a2:current_account-current_region
[25%] fail: No bucket named 'cdk-hnb659fds-assets-349327579537-us-west-2'. Is account 349327579537 bootstrapped?
[50%] fail: No bucket named 'cdk-hnb659fds-assets-349327579537-us-west-2'. Is account 349327579537 bootstrapped?
[75%] fail: No bucket named 'cdk-hnb659fds-assets-349327579537-us-west-2'. Is account 349327579537 bootstrapped?
[100%] fail: No bucket named 'cdk-hnb659fds-assets-349327579537-us-west-2'. Is account 349327579537 bootstrapped?

 ❌  MythicalMysfits-Website failed: Error: Failed to publish one or more assets. See the error messages above for more information.
    at publishAssets (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-cdk/lib/util/asset-publishing.ts:44:11)
    at CloudFormationDeployments.publishStackAssets (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:464:7)
    at CloudFormationDeployments.deployStack (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:339:7)
    at CdkToolkit.deploy (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:209:24)
    at initCommandLine (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-cdk/lib/cli.ts:341:12)

Failed to publish one or more assets. See the error messages above for more information.

$ cdk bootstrap aws://349327579537/us-west-2
 ⏳  Bootstrapping environment aws://349327579537/us-west-2...
Trusted accounts for deployment: (none)
Trusted accounts for lookup: (none)
Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
 ✅  Environment aws://349327579537/us-west-2 bootstrapped (no changes).
 

cdk deploy error: current credentials could not be used to assume

$ cdk deploy
✨  Synthesis time: 6.67s
[...]
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-lookup-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
(To get rid of this warning, please upgrade to bootstrap version >= 8)
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-deploy-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

IAM Statement Changes
[...]
Do you wish to deploy these changes (y/n)? y
CdkStack: deploying...
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-deploy-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
[0%] start: Publishing 990410bab4a39b07c4495c3b8fae2f3f8847daabc9e3fc1debf3fa050c25e302:current_account-current_region
[0%] start: Publishing b7f327b4415410f319943b754edb274645a9d6850369ae4da9ba209858099210:current_account-current_region
[0%] start: Publishing f98b78092dcdd31f5e6d47489beb5f804d4835ef86a8085d0a2053cb9ae711da:current_account-current_region
[0%] start: Publishing a45af0f103767cdfa62f52478cd2e33a0c44babd71e5a1e4d8068557f006ba09:current_account-current_region
[0%] start: Publishing ff834787ba68a959da6121bba92b2b17155fd7580a93cfa4eca9445eda33ab90:current_account-current_region
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-file-publishing-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-file-publishing-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-file-publishing-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-file-publishing-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
current credentials could not be used to assume 'arn:aws:iam::349327579537:role/cdk-hnb659fds-file-publishing-role-349327579537-us-west-2', but are for the right account. Proceeding anyway.
[20%] success: Published 990410bab4a39b07c4495c3b8fae2f3f8847daabc9e3fc1debf3fa050c25e302:current_account-current_region
[40%] success: Published f98b78092dcdd31f5e6d47489beb5f804d4835ef86a8085d0a2053cb9ae711da:current_account-current_region
[60%] success: Published a45af0f103767cdfa62f52478cd2e33a0c44babd71e5a1e4d8068557f006ba09:current_account-current_region
[80%] success: Published b7f327b4415410f319943b754edb274645a9d6850369ae4da9ba209858099210:current_account-current_region
[100%] success: Published ff834787ba68a959da6121bba92b2b17155fd7580a93cfa4eca9445eda33ab90:current_account-current_region
CdkStack: creating CloudFormation changeset...

 ❌  CdkStack failed: Error [ValidationError]: Role arn:aws:iam::349327579537:role/cdk-hnb659fds-cfn-exec-role-349327579537-us-west-2 is invalid or cannot be assumed
    at Request.extractError (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/protocol/query.js:50:29)
    at Request.callListeners (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/request.js:686:14)
    at Request.transition (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/request.js:688:12)
    at Request.callListeners (/home/gabe/.nvm/versions/node/v16.15.1/lib/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 'ValidationError',
  time: 2022-08-23T17:54:18.635Z,
  requestId: '93704c2c-2add-401e-ae3f-62ea93a49a7e',
  statusCode: 400,
  retryable: false,
  retryDelay: 704.8538544449583
}

Role arn:aws:iam::349327579537:role/cdk-hnb659fds-cfn-exec-role-349327579537-us-west-2 is invalid or cannot be assumed
$

AWS CLI SAM

Source: AWS Serverless Application Model - Installing the AWS SAM CLI

Prerequisite: AWS CLI version 2

Linux-x86-64

• Download the AWS SAM CLI .zip file to a directory

$ curl "https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip" \
        -o "awssamcli.zip"

• Verify the integrity and authenticity of the downloaded installer files by generating a hash value using the following command:

$ sha256sum aws-sam-cli-linux-x86_64.zip
24aa5893e35ce370b92b730fe163d902d248bdef4e3513eb3810bc6ff91a4efd  aws-sam-cli-linux-x86_64.zip

• Unzip the installation files into the sam-installation/ subdirectory

$ unzip aws-sam-cli-linux-x86_64.zip -d sam-installation

• Install the AWS SAM CLI

$ sudo ./sam-installation/install
$ sam --version
SAM CLI, version 1.73.0

Linux-arm

References

• [REF-1] AWS Configuration and credential file settings

• [REF-2] AWS Environment variables to configure the AWS CLI