AWS Training

AWS provides an enterprise agility in utilizing technology rapidly without incurring upfront capital expenditure.
Architecture for Failure 

It uses utility billing to charge for use cases.

S3 - Simple Storage Service

• S3 provides a way to store WORM - write once read many.
• Synchronized on different data centres
• it uses key value pair concept
• Updates are atomic, no partial updates, so its better to split files and upload.
• Eventual consistency vs immediate consistency - When a write takes place everyone can read it immediately (outside US region) 
• Standard S3 vs Reduced Redundancy Storage (RRS) - Approx 30% cheaper than standard
• RRS could be used for things that could be reproduced compared to S3 eg: RSS feeds
• S3 can contains buckets which can contain objects 
• S3 CloudBerry - Browsing S3
• Protocol/S3/Bucket/Key - http://s3.amazonaws.com/b1/a.jpg
• Access Control - Secure by default - ACL, Bucket Policies, IAM Policies
• ACL on bucket don't apply to objects
• Only grant permissions
• R/W/Read ACL/Write ACL
• Access to Owner/Specific AWS user/ All AWS Users/Any anonymous request
• Bucket Policies are programmatic and applies to contained objects, can allow or deny
• Can use AWS Policy Generator to generate policies.
• User Authentication via API  - (  )
• Complementary Services - 
• Version-ing - priced as separate object slightly cheaper, can enable versioning and latest version is provided for request of that object
• Access logging - pay for storage of log files, enable for any owned bucket, reports are delivered to nominated bucket, no log rotation - you have to delete old logs, designed for best effort delivery, log is for a bucket, can use of object expiry for the log file and remove it ?
• Multi-part uploading
• Import/Export Service - First time upload sent to Amazon/ or high load of data to be uploaded - SneakerNet term
• Pricing S3 storage <= $0.154 per GB month, Inbound data transfer free, Outbound charged, 10,000 GET requests <= $0.011 

Hands on Lab on S3

     Tasks Create S3 Bucket

     Upload a File

     Give permission to access the file

     Set bucket logging 

     Set bucket policies

     Set bucket rules etc Play with it

AWS CloudFront

• Content Delivery Network - client access closest copy of the data
• Low latency access to data / Improved availability / Increased throughput
• Users <==(DNS query / Routing Info / HTTP Query / Response)==>Edge locations <=(Data Transfer)=> Publisher's Content
• Uses lazy caching / Caching period of 1 hour
• CloudFront Cost - Each Request - GET + Data Transer to Edge Locations
• CloudFront is more lower in Cost / but edge locations in Asia cost more so maybe S3 is good in asia
• CloudFront can be used for static content as well as dynamic content - Offloading traffic from the server to cloud front
• Use proper HTTP headers 
• Caches content based on URL only - cookies ignored
• Cache Invalidation - Invalidation API - $0.005 per path invalidated (First 1000 invalidations per month free) Can only invalidate 3000 paths at once.
• Use for scenarios which are different
• Use object version-ing, proper headers etc.. rather than this
• Supports HTTPS (priced higher) - Not properly supported
• Supports CNAME for you distribution
• No Geo code blocking
• No advanced cache controls / 20 Gb per object

You can go to the cloudfront straight or S3 cloudfront essentially acts as a CDN

Hands on Lab on CloudFront

     Create cloudfront and access object through cloud front

Session 2:

   Amazon Elastic Compute Cloud: EC2 Overview

• Regions - multiple availability zones
• Instance types and Families - Size, CPU, memory, storage, Micro / Standard / High-CPU / High-Memory
• Pay simple hourly rate for use
• Cluster Computing Instances - provide HPC in the cloud - 2 instance types, Require special AMI
• Cluster GPU instances - CUDA Architecture 
• Pricing modes (Reserve / On Demand / Spot ) - Hybrid might be the appropriate - Reserved + On Demand 
• Windows vs Linux Pricing 30-70% more for linux
• AWS has a Price Calculator to play with, play and figure out the best low cost configuration which serves the purpose
• Security Group - Firewall
• Open ports as needed
• Default group nothing open
• Web Security Group / DB Security Group / App Security Group 
• Security Group control for inbound access control only
• Collection of access rules
• You may spin a instance which will be used to administrate other instances.
• AWS generated SSH keys - amazon gives you the keys
• User generated SSH keys - your private key is available only to you
• EC2 Addressing
• Each instance has a public IP and private IP
• Each instance has a public DNS and internal DNS
• Optionally can use a Elastic IP
• Externally for these you need to pay [Elastic IP / Public IP]
• Public DNS / Private IP / Private DNS - Free

Elastic Block Store - EBS

     Persistent storage outside the life cycle of any particular EC2 node

• S3 vs EBS
• EBS is a file system S3 is object store
• Good for files that change often 
• Create - Attach - Attached in Use - CreateSnapshot (back up to S3) - Detach - Delete
• EBS volumes are in a Single Availability zone
• EBS Interacts with EC2 and S3 - Create snapshot store in S3 copy to different availability zone for EC2 there
• EBS pricing for per GB-month & per I/O requests (block level or file level ?)

EC2 - AMI - Amazon Machine Images

• To spin EC2 instances - OS + Other things (Jboss or IBM etc.)
• Regional separations among regions on AMI
• AMI available in different flavours (Ubuntu AMI's -  http://alestic.com/ )

Approaches to use AMI

1. Golden approach - Choose a base, Install other software, make it golden and then make AMI out of it and use that
2. Deployment script approach - Pass the script to EC2 instance on startup - it can be a shell script or puppet
3. Hybrid approach - Choose a base image and install & configure, test until golden, use scripts to custom software

Should I make my own AMI ? depends! 

EC2 AMI Construction process - base image / launch on EC2 / Install configure / Test / Create EC2 bundle (S3) or EBS, make sure failure is tolerated, scripted or imaged to restore

Instance Store-Backed - Boot time in minutes

     Note: Root Drive is local disk - lost when instance terminated

     Maximum size 10 GB, no cost when stopped, I/O to root drive free, Not persisted when terminated

EBS Backed Instances - Boot time in seconds

    Note: Root drive is an EBS - instance can be suspended and resized charged for IO -

   Maximum 1TB, I/O to EBS costs, Instance can be suspended - EBS costs apply

Which to use: windows might be too big for instance store
   if possible, use instance store
   which ever way script the deployment and make it possible to restore from backup.

EC2 & EBS Hands on + Load Balancer (Included into this for ease)


create key pair
Use puttly - use puttygen to generate the private key using the file given by AWS
Add security group to open port 22, 8080, HTTP

Run a python script
  wget http://bit.ly/aws-ec2id
  bash aws-ec2id

Run mysql
  wget http://bit.ly/aws-mysql
  bash aws-mysql /dev/xvdf

  mysqlshow --user=root

wget http://bit.ly/aws-wiki
bash aws-wiki --init

ec2-50-17-153-95.compute-1.amazonaws.com
ec2-107-20-63-239.compute-1.amazonaws.com - ubuntu user name
make sure to select the key when using putty when connecting to the server

wget http://bit.ly/aws-wiki
bash aws-wiki -DNS of the next server make sure to give this so the app is talking to the mysql on the other server

Connect the second server to the loadbalancer and then try stoping first and going to each service it will be handled automatically

Elastic Load Balancer / AutoScaling / CloudWatch
   Quick to setup for EC2, specify protocols, ports AZ etc, Obtain DNS name, Scales, Fault-tolerant
   Routes traffics to healthy EC2 instances, More for external traffic not for internal traffic
   How does it handle web sockets ? 
   How does it handle sessions ?
   CloudWatch - Provide metrics for resources in the cloud - CPU/DiskI/O, CPU Utilization,Latency,
          Actions on metrics - Alarms - provide notifications, initiate auto scaling, SNS (System Notification
          System) can deliver to HTTP,HTTPS, Email, Queue
    Can make use of Auto Scaling - Scale groups based on collected metrics (Cloud Watch)
       3 Types - Manual Scaling (API Calls), Scaling by schedule (cronlike), Scaling by policy
       Elastic <> Instant - (Cloudwatch - alarm - Trigger fires - Boot time - ELB update - Look at the time for
                                      these it will not be instant initiating instances)
       Suspend/Resume and Delete - good for debuggin or dealing with external events
    
     CloudWatch ======> AutoScaling =====>  Elastic Load Balancer __
              ^                                                                                                         |
               |_____________________________________________________|                                                                                                   

   Takes some time to adjust fleet size, must use identical instances 

VPC - Virtual Private Cloud
    Create your own private network
    Different types of VPC's 4 different options

• Single public subnet (VPC - Public subnet)
• Public and Private subnets (Public subnet - NAT - Private subnet)
• Public and private subnets with h/w VPN (
• Private subnets with h/w VPN

How it works - Create an IP range / Create subnets in that range / Launch EC2 instances in those subnets

Configure Network ACLs (Inbound/Outbound) - Allow/Deny rules 

Configure security groups 

Use Cases : Accessing datacenter over a private connection / Batch processing / Test enviornment

Pricing - Secure VPN connection - $0.05 per hour

Other limitations exist

Relational Database Service (RDS)
   Managed service for cloud-based RDS
   Supported Databases - My SQL 5.1 / 5.5
                                       Oracle 11.2 / SE 1
   Without RDS you have to put things together!
   With RDS you have options of Multi-AZ deployments, Failover done by changing the CNAME to point
    to the standby, Other advantages provided by RDS. (read replication,perfect for scaling to support a
    heavy read load, version control, Amazon take care of applying patches)
   Price - Premium over EC2, Oracle (BYOL) etc...

IAM - Identity Access Management 
   Create users within your AWS account
   Each user has their own credential
   IAM Policies - Control access to and actions on AWS resources at the group level
                           Use the Access Policy Language
                           Can't use IAM users in S3 ACLs