AWS certified developer associate hints. VPC, API Gateway, AWS Lambda (part IV)
Amazon VPC
VPC, Subnets, IGW and NAT
-
VPC private network to deploy your resources (regional resource)
-
Subnets allow you to partition your network inside VPC (AZ resource)
-
A public subnet is a subnet that is accessible from the internet
-
A private subnet is a subnet that is not accessible from the internet
-
To define access to the internet and between subnet, we use Route Tables
-
Internet Gateways helps our VPC instances connect with an internet
-
Public Subnets have a route to the internet gateway
-
NAT Gateways (AWS managed) & NAT instances(self managed) allow your instances in your Private subnet to access the internet while remaining private
Network ACL & Security Group, VPC flow logs
NACL:
- A firewall which controls traffic fro and to subnet
- Can have ALLOW and DENY rules
- rules are attached at the Subnet level
- Rules only include IP addresses
Security Groups:
- A firewall that controls traffic to and from an ENI / an EC2 instance
- Can have only ALLOW rules
- Rules include IP addresses and other security groups
VPC Flow Logs:
- Capture information about IP traffic going into your interface
- VPC flow logs
- Subnet Flow Logs
- Elastic Network Interface flow logs
- Helps to monitor and troubleshoot connectivity issues
- Capture network information from AWS managed interfaces too
- VPC flow logs data can go to S3, CloudWatch Logs, Kinesis Data Firehose
VPC Peering, Endpoints, VPN, DX
VPC Peering:
- Connect two VPC, private using AWS network
- Make them behave as if they were in the same network
- Must not have overlapping CIDR
- VPC Peering connection is not transitive (must be established between each VPC)
VPC Endpoint:
- Endpoints allow you to connect to AWS service using private network instead of public www network
- This gives you enhanced security and lower latency to AWS services
- VPC endpoint gateway: S3 & DynamoDB
- VPC Endpoint interface: the rest
Site to Site VPN & Direct Connect
Site to site VPN:
- Connect an on-premises VPN to AWS
- The connection is automatically encrypted
- Goes over the public internet
Direct connect:
- Establish a physical connection between on-premise and AWS
- The connection is private, secure and fast
- Goes over private network
- Takes at least a moth to establish
Site to site VPN and Direct connect cannot access VPC endpoints.
AWS API gateway
- AWS Lambda + API Gateway: No infra to manage
- Support for the WebSocket Protocol
- Handle API versioning
- Handle different env
- Handle security (authentication and authorization)
- Create API keys
- Swagger/ Open API import
- Transform and validate request and response
- Generate SDK and API specification
- Cache API responses
Integrations:
- Lambda function
- HTTP
- AWS service
Endpoint types:
- Edge optimized
- Regional
- Private
Security:
- IAM roles
- Cognito
- Custom authorizer (lambda)
- Custom domain name HTTPS and Amazon Certificate Manger (us-east-1 for global, same region for regional)
API Gateway - Deployment stages
- making changes in the API gateway doesn’t mean they are effective
- You need to make a deployment
- Changes are deployed to “stages”
- Use the naming you like for stages
- Each stage has its own configuration
- Stages can be rolled back as a history of deployment
API gateway - Stage Variables
- Stage variables are like like env variables
- Use them to change often changing values
- Format ${variable}
API gateway - integration types
- Integration type MOCK, returns the response without sending the request to back-end
- Integration type HTTP/AWS (Lambda or AWS services)
- Integration AWS_PROXY (Lambda proxy) incoming requests from the client is the input to lambda, no mapping template
- Integration HTTP_PROXY, no mapping template, request is passed to the back-end and response is passed to the client, possible to add HTTP headers
API gateway mapping templates
Can be used for modification of client request/response, modify query params, modify headers or body, add headers.
Only possible for Content-Type: application/json or application/xml.
Example: SOAP API integration when client send JSON and API gateway change it to XML or changing query params to body.
API gateway Open API
- Common way to define REST API, using API definition as code
- Import existing Open API v3 to API gateway
- Export API gateway as Open API
- Generate SDK
- Possible to set validation using JSON schema
API gateway caching
- Default is 300, min 0, max 3600s
- Cache is defined per stage
- Possible to overwrite cache per method
- Cache could be encrypted
- Cache size is 0.5GB - 237GB
- Cache is expensive
- Possible to invalidate from UI, or sending client headers
- Possible to invalidate for any client or authorized client
API gateway usage plans
Offer API as a paid option.
Usage plans:
- Who can access one or more stages or methods
- How much and how fast
- Use API keys to measure access
- Configure throttling limits and quotas
API keys:
- Distributed to customers
- Used with usage plans
- Throttling limits are applied
- Quotas limits are possible
- API key is passed in x-api-key header
API gateway logging and tracing
- Enable CloudWatch Logs, logging on the stage level (with log level ERROR, DEBUG, INFO)
- X-RAY
- CloudWatch Metrics (CacheHitCount, CacheMissCount, Count, IntegrationLatency, 4xx and 5xx errors)
API gateway throttling
Account limit:
- 10000 requests per second
- Soft limits can be increased upon request Possible to define
- Stage limits
- Method limits
- Or usage plans
API gateway CORS
Integrate OPTIONS method with correct response automatically.
Correct headers must be passed from the client side.
CORS is configured per domain from API gateway side.
This doesn’t work for PROXY integration types
API gateway security and IAM
-
Create IAM policy authorization and attach to User|Role
-
Authentication = IAM and Authorization = IAM Policy
-
Good to provide access within AWS
-
Leverage “Sig v4” capability where IAM credential are in headers
-
Cognito user pool
-
Lambda authorizer
AWS Lambda
Free tier 1 000 000 AWS Lambda request and 400 000 GBs of compute time.
Up to 10GB of RAM.
Increasing RAM will also improve CPU.
Integration for AWS lambda:
- API gateway
- Kinesis
- DynamoDB
- S3
- Cloud Front
- Event Bridge
- CloudWatch Logs
- SNS
- SQS
- Congnito
Default 128 MB of RAM.
Timeout 3 seconds - 15 mins.
Lambda Synchronous Invocation
Awaiting for response, next services
- ALB
- API gateway
- CloudFront (Lambda@Edge)
- S3 Batch
- Cognito
- Step function
- Lex
- Alexa
- Kinesis data firehose
Integration with ALB
Register lambda in a target group, translate ALB to Lambda, query, path, headers, body.
Lambda returns status code , headers and body.
Multi headers value:
name=foo&name=bar => { name: [foo, bar]}
Lambda Asynchronous Invocation
- S3
- SNS
- CloudWatch evens and bridge
- Code commit
- Code pipene
- SES
Once the event happen it’s placed in event queue and event is processed 3 times, with 1 and 2 mins delay.
You will see duplicated events in CloudWatchLogs.
DLQ - dead-later queue - SNS or SQS for failed processing.
S3 Events notification
It’s possible to filter using patters (e.g. ".jpg" suffix).
If you have 2 writes to a single non versioned object, it’s possible to miss one, so ensure you have versioning.
- S3 => SNS (Fun out) - multiple SQS
- S3 => SQS => Lambda
- S3 => Lambda => DLQ
Lambda - event source mapping
- Kinesis data streams
- SQS & SQS FIFO
- DynamoDB Streams
Records need to be polled from the source.
Scaling
Kinesis data stream & DynamoDB stream:
- One lambda per stream shard
- 10 batches per shard with parallelization
SQS:
- 60 instances per minute
- Up to 1000 batches of messages processed simultaneously
SQS FIFO:
- messages with same groupId processed in order
- number of active message groups
Event and context objects
Event object:
- json formatted document from event bridge or others
- First param in lambda
Context object:
- provider methods and properties that provide information about invocation function
- second param in lambda
Lambda destinations
Asynchronous invocation - can define destination for success:
- SQS
- SNS
- AWS lambda
- Amazon event bridge bus
Event source mapping:
- amazon SQS
- amazon SNS
Lambda permissions
Use IAM policies if you need Lambda communicate with other services.
Use resource based policies to give access to other accounts.
Or give permissions to other services.
Lambda ENV variable
Can be encrypted with KMS.
Lamda logging & monitoring
- Cloud watch logs is enabled by default, if there is a correct policy
- Cloud watch metrics
- X-Ray (just enable it and use X-Ray SDK) communication is done based on env variable
Lambda @ edge
Function that you run attached to CloudFront.
Minimal latency.
Cloud front function:
- can run millions of request
- max memory 2mb
- no access to request body and file system
- max execution time is 1ms
- 6 time cheaper Lambda @ edge
Lambda @ edge:
- can run thouthand of request
- max memory 128MB to 10GB
- max execution time 5-10 seconds
Lambda in VPC
By default - no access to VPC, access to internet.
VPC can be defined, ENI elastic network interface is created to support it, no internet access, so you need to create NAT Gateway or Instance and NAT gives you IGW (internet gate way).
For accessing DynamoDB, possible to create VPC endpoint.
Lambda function configuration
RAM:
- 128MB to 10GB in 1MB increment
- 1792 MB of RAM = 1 vCPU
- Timeout 3 seconds, but could be set up to 15 minutes
- Execution context can be reused between functions
- Possible to use /tmp(Ephemeral storage) directory, max size is 10GB, can be reused
Lambda Layers
- Custom layers C++ or Rust, externalize dependencies to re-use those
- Function can reference layers
Lambda File System mounting
- Lambda function can access EFS in same VPC
- Must leverage EFS access points
- One connection per lambda
Lambda concurrency
- Up to 1000 concurrent executions, for account
- Reserved concurrency - at the function level, can be increased by creating ticket
- Cold start, first request takes more time
- Provision concurrency, concurrency is allocated, cold start is not happening.
- Can reserve concurrency, it will be deducted from the account concurrency
External dependencies
Must be loaded as archive, possible to use node_modules.
Lambda and Cloud Fomration
- Inline, defined in the policy (simple use case, no dependencies)
- Through s3, put to s3, refer from the policy
Lambda Container Images
Image up to 10Gb in ECR, pack complex dependencies in the container, base image implement runtime API.
Allows testing locally.
Lambda versions and Aliases
Versions are immutable.
Each version can be accessed.
Alias - point to the version.
Aliases are mutable.
Aliases enable Canary versions using weighting in alias configuration.
Alias can’t reference alias.
Code deploy
Codedeploy can help automate traffic shift for lambda aliases. Integrated in SAM framework.
Linear grow, grow every N minutes until reaches 100%. Canary X percent then 100%. AllAtOnce: immediate.
Rollback is possible.
AppSpec.yml can configure function name, alias current version and target version.
Lambda function URL
Possible for aliases and unpublished.
Possible to expose as public not changing URL, ipv4 and ipv6, only public internet, possible to set CORS.
Possible to use aliases.
Security
Resource-based policy:
- authorize other accounts / specific CIDR / IAM
CORS.
AuthType None - allow public and unathathenticated access.
AuthType AWS_IAM - IAM used to auth requests.
Codeguru profiling
Insight for your lambda, creates a profiler group.
- Adds code guru as a separate layer
- Adds env to your function
- iam policy assigned
Lambda limits
Execution:
- Executions 128mb - 10 gb
- Max execution 15 mins
- Env variable 4kb
- Disk capacity /tmp 512MB to 10Gb
- Concurrency 1000
Deployment:
- Size in complex zip 50mb
- Size of uncompressed code + dependencies 250MB
- Size of env varables 4kb