Amazon VPC

VPC, Subnets, IGW and NAT

• VPC private network to deploy your resources (regional resource)

• Subnets allow you to partition your network inside VPC (AZ resource)

• A public subnet is a subnet that is accessible from the internet

• A private subnet is a subnet that is not accessible from the internet

• To define access to the internet and between subnet, we use Route Tables

• Internet Gateways helps our VPC instances connect with an internet

• Public Subnets have a route to the internet gateway

• NAT Gateways (AWS managed) & NAT instances(self managed) allow your instances in your Private subnet to access the internet while remaining private

Network ACL & Security Group, VPC flow logs

NACL:

• A firewall which controls traffic fro and to subnet
• Can have ALLOW and DENY rules
• rules are attached at the Subnet level
• Rules only include IP addresses

Security Groups:

• A firewall that controls traffic to and from an ENI / an EC2 instance
• Can have only ALLOW rules
• Rules include IP addresses and other security groups

VPC Flow Logs:

• Capture information about IP traffic going into your interface
• VPC flow logs
• Subnet Flow Logs
• Elastic Network Interface flow logs
• Helps to monitor and troubleshoot connectivity issues
• Capture network information from AWS managed interfaces too
• VPC flow logs data can go to S3, CloudWatch Logs, Kinesis Data Firehose

VPC Peering, Endpoints, VPN, DX

VPC Peering:

• Connect two VPC, private using AWS network
• Make them behave as if they were in the same network
• Must not have overlapping CIDR
• VPC Peering connection is not transitive (must be established between each VPC)

VPC Endpoint:

• Endpoints allow you to connect to AWS service using private network instead of public www network
• This gives you enhanced security and lower latency to AWS services
• VPC endpoint gateway: S3 & DynamoDB
• VPC Endpoint interface: the rest

Site to Site VPN & Direct Connect

Site to site VPN:

• Connect an on-premises VPN to AWS
• The connection is automatically encrypted
• Goes over the public internet

Direct connect:

• Establish a physical connection between on-premise and AWS
• The connection is private, secure and fast
• Goes over private network
• Takes at least a moth to establish

Site to site VPN and Direct connect cannot access VPC endpoints.

AWS API gateway

• AWS Lambda + API Gateway: No infra to manage
• Support for the WebSocket Protocol
• Handle API versioning
• Handle different env
• Handle security (authentication and authorization)
• Create API keys
• Swagger/ Open API import
• Transform and validate request and response
• Generate SDK and API specification
• Cache API responses

Integrations:

• Lambda function
• HTTP
• AWS service

Endpoint types:

• Edge optimized
• Regional
• Private

Security:

• IAM roles
• Cognito
• Custom authorizer (lambda)
• Custom domain name HTTPS and Amazon Certificate Manger (us-east-1 for global, same region for regional)

API Gateway - Deployment stages

• making changes in the API gateway doesn’t mean they are effective
• You need to make a deployment
• Changes are deployed to “stages”
• Use the naming you like for stages
• Each stage has its own configuration
• Stages can be rolled back as a history of deployment

API gateway - Stage Variables

• Stage variables are like like env variables
• Use them to change often changing values
• Format ${variable}

API gateway - integration types

• Integration type MOCK, returns the response without sending the request to back-end
• Integration type HTTP/AWS (Lambda or AWS services)
• Integration AWS_PROXY (Lambda proxy) incoming requests from the client is the input to lambda, no mapping template
• Integration HTTP_PROXY, no mapping template, request is passed to the back-end and response is passed to the client, possible to add HTTP headers

API gateway mapping templates

Can be used for modification of client request/response, modify query params, modify headers or body, add headers.

Only possible for Content-Type: application/json or application/xml.

Example: SOAP API integration when client send JSON and API gateway change it to XML or changing query params to body.

API gateway Open API

• Common way to define REST API, using API definition as code
• Import existing Open API v3 to API gateway
• Export API gateway as Open API
• Generate SDK
• Possible to set validation using JSON schema

API gateway caching

• Default is 300, min 0, max 3600s
• Cache is defined per stage
• Possible to overwrite cache per method
• Cache could be encrypted
• Cache size is 0.5GB - 237GB
• Cache is expensive
• Possible to invalidate from UI, or sending client headers
• Possible to invalidate for any client or authorized client

API gateway usage plans

Offer API as a paid option.

Usage plans:

• Who can access one or more stages or methods
• How much and how fast
• Use API keys to measure access
• Configure throttling limits and quotas

API keys:

• Distributed to customers
• Used with usage plans
• Throttling limits are applied
• Quotas limits are possible
• API key is passed in x-api-key header

API gateway logging and tracing

• Enable CloudWatch Logs, logging on the stage level (with log level ERROR, DEBUG, INFO)
• X-RAY
• CloudWatch Metrics (CacheHitCount, CacheMissCount, Count, IntegrationLatency, 4xx and 5xx errors)

API gateway throttling

Account limit:

• 10000 requests per second
• Soft limits can be increased upon request Possible to define
• Stage limits
• Method limits
• Or usage plans

API gateway CORS

Integrate OPTIONS method with correct response automatically.

Correct headers must be passed from the client side.

CORS is configured per domain from API gateway side.

This doesn’t work for PROXY integration types

API gateway security and IAM

• Create IAM policy authorization and attach to User|Role

• Authentication = IAM and Authorization = IAM Policy

• Good to provide access within AWS

• Leverage “Sig v4” capability where IAM credential are in headers

• Cognito user pool

• Lambda authorizer

AWS Lambda

Free tier 1 000 000 AWS Lambda request and 400 000 GBs of compute time.

Up to 10GB of RAM.

Increasing RAM will also improve CPU.

Integration for AWS lambda:

• API gateway
• Kinesis
• DynamoDB
• S3
• Cloud Front
• Event Bridge
• CloudWatch Logs
• SNS
• SQS
• Congnito

Default 128 MB of RAM.

Timeout 3 seconds - 15 mins.

Lambda Synchronous Invocation

Awaiting for response, next services

• ALB
• API gateway
• CloudFront (Lambda@Edge)
• S3 Batch
• Cognito
• Step function
• Lex
• Alexa
• Kinesis data firehose

Integration with ALB

Register lambda in a target group, translate ALB to Lambda, query, path, headers, body.

Lambda returns status code , headers and body.

Multi headers value:

name=foo&name=bar => { name: [foo, bar]}

Lambda Asynchronous Invocation

• S3
• SNS
• CloudWatch evens and bridge
• Code commit
• Code pipene
• SES

Once the event happen it’s placed in event queue and event is processed 3 times, with 1 and 2 mins delay.

You will see duplicated events in CloudWatchLogs.

DLQ - dead-later queue - SNS or SQS for failed processing.

S3 Events notification

It’s possible to filter using patters (e.g. ".jpg" suffix).

If you have 2 writes to a single non versioned object, it’s possible to miss one, so ensure you have versioning.

• S3 => SNS (Fun out) - multiple SQS
• S3 => SQS => Lambda
• S3 => Lambda => DLQ

Lambda - event source mapping

• Kinesis data streams
• SQS & SQS FIFO
• DynamoDB Streams

Records need to be polled from the source.

Scaling

Kinesis data stream & DynamoDB stream:

• One lambda per stream shard
• 10 batches per shard with parallelization

SQS:

• 60 instances per minute
• Up to 1000 batches of messages processed simultaneously

SQS FIFO:

• messages with same groupId processed in order
• number of active message groups

Event and context objects

Event object:

• json formatted document from event bridge or others
• First param in lambda

Context object:

• provider methods and properties that provide information about invocation function
• second param in lambda

Lambda destinations

Asynchronous invocation - can define destination for success:

• SQS
• SNS
• AWS lambda
• Amazon event bridge bus

Event source mapping:

• amazon SQS
• amazon SNS

Lambda permissions

Use IAM policies if you need Lambda communicate with other services.

Use resource based policies to give access to other accounts.

Or give permissions to other services.

Lambda ENV variable

Can be encrypted with KMS.

Lamda logging & monitoring

• Cloud watch logs is enabled by default, if there is a correct policy
• Cloud watch metrics
• X-Ray (just enable it and use X-Ray SDK) communication is done based on env variable

Lambda @ edge

Function that you run attached to CloudFront.

Minimal latency.

Cloud front function:

• can run millions of request
• max memory 2mb
• no access to request body and file system
• max execution time is 1ms
• 6 time cheaper
Lambda @ edge

Lambda @ edge:

• can run thouthand of request
• max memory 128MB to 10GB
• max execution time 5-10 seconds

Lambda in VPC

By default - no access to VPC, access to internet.

VPC can be defined, ENI elastic network interface is created to support it, no internet access, so you need to create NAT Gateway or Instance and NAT gives you IGW (internet gate way).

For accessing DynamoDB, possible to create VPC endpoint.

Lambda function configuration


RAM:

• 128MB to 10GB in 1MB increment
• 1792 MB of RAM = 1 vCPU
• Timeout 3 seconds, but could be set up to 15 minutes
• Execution context can be reused between functions
• Possible to use /tmp(Ephemeral storage) directory, max size is 10GB, can be reused

Lambda Layers

• Custom layers C++ or Rust, externalize dependencies to re-use those
• Function can reference layers

Lambda File System mounting

• Lambda function can access EFS in same VPC
• Must leverage EFS access points
• One connection per lambda

Lambda concurrency

• Up to 1000 concurrent executions, for account
• Reserved concurrency - at the function level, can be increased by creating ticket
• Cold start, first request takes more time
• Provision concurrency, concurrency is allocated, cold start is not happening.
• Can reserve concurrency, it will be deducted from the account concurrency

External dependencies

Must be loaded as archive, possible to use node_modules.

Lambda and Cloud Fomration

• Inline, defined in the policy (simple use case, no dependencies)
• Through s3, put to s3, refer from the policy

Lambda Container Images

Image up to 10Gb in ECR, pack complex dependencies in the container, base image implement runtime API.

Allows testing locally.

Lambda versions and Aliases

Versions are immutable.

Each version can be accessed.

Alias - point to the version.

Aliases are mutable.

Aliases enable Canary versions using weighting in alias configuration.

Alias can’t reference alias.

Code deploy

Codedeploy can help automate traffic shift for lambda aliases. Integrated in SAM framework.

Linear grow, grow every N minutes until reaches 100%. Canary X percent then 100%. AllAtOnce: immediate.

Rollback is possible.

AppSpec.yml can configure function name, alias current version and target version.

Lambda function URL

Possible for aliases and unpublished.

Possible to expose as public not changing URL, ipv4 and ipv6, only public internet, possible to set CORS.

Possible to use aliases.

Security

Resource-based policy:

• authorize other accounts / specific CIDR / IAM

CORS.

AuthType None - allow public and unathathenticated access.

AuthType AWS_IAM - IAM used to auth requests.

Codeguru profiling

Insight for your lambda, creates a profiler group.

• Adds code guru as a separate layer
• Adds env to your function
• iam policy assigned

Lambda limits

Execution:

• Executions 128mb - 10 gb
• Max execution 15 mins
• Env variable 4kb
• Disk capacity /tmp 512MB to 10Gb
• Concurrency 1000

Deployment:

• Size in complex zip 50mb
• Size of uncompressed code + dependencies 250MB
• Size of env varables 4kb

Tags

• technology