Scheduling AWS Autoscaling to create Instance and run your tasks


      Introduction
      The purpose of this exercise is to not have a instance running for one off tasks but create the        
      instance dynamically using a pre configured AMI image.
      We use this insance to create reports and upload the reports to S3 bucket. The below details are           provided for configuring the instance and creating autoscaling groups which will fire up an AMI               image and run at a specified time to achieve the goal.
    Download and Configure Auto Scaling Command Line Tool 
     
      Google or use (http://ec2-downloads.s3.amazonaws.com/AutoScaling-2011-01-01.zip)
      Follow instruction in the Auto scaling tool "Readme.txt" which indicates to set the,
               Environment variables:
                 AWS_AUTO_SCALING_HOME - The directory where the deployment files were copied to
                 JAVA_HOME - Java Installation home directory
                 Add ${AWS_AUTO_SCALING_HOME}/bin
                   (in Windows: "%AWS_AUTO_SCALING_HOME%\bin") to your path
        Add your AWS Keys as indicated by the Readme.txt to a file
                ${AWS_AUTO_SCALING_HOME}/credential-file-path.template -> credential-file-path.test
        There are several ways to provide your credential information:
            a. Set the following environment variable:
                         AWS_CREDENTIAL_FILE=<the file created in above>
            b. Alternatively, provide the following option with every command
                         --aws-credential-file <the file created in above>
            c. Explicitly specify credentials on the command line: --I ACCESS_KEY --S SECRET_KEY

      Verify if the setup is working properly by running
              $ as-cmd --help
              $ as-describe-auto-scaling-groups --headers
              (You should see a header line. If you have auto scaling groups already configured, you will 
                see a description line for each auto scaling group)

     Using EC2 console Launch Instance and Configure
       Create your instance in EC2 and install software necessary for the task and create crontabs as
        necessary for schedules that need to be run. Make sure the crontabs are for "su" user or give rights for           "ec2-user" and setup the envirnment to create the AMI
 
       Note: The time on the EC2 instance will be UTC unless you configure it to EST or anyother and          
                 remember the Autoscaling schedules will be in UTC on amazon infrastructure maybe we can                         change that as well but in here we are not changing it.

    Create the AMI from AWS console for the instance configured -
        Right click on instance and choose "Create Image" and add name for created image,
          set "No reboot" true, and "delete on termination" true and create the image.

    Creating AutoScaling Group to Schedule creation of the Instance 
      Use the image id - AMI ID in the configuration of Autoscaling groups to create a schedule to run the   
      instance as necessary

Create Launch Configurations
as-create-launch-config <LaunchConfigName>
          --key <KeyToUseToConnectToInstance>
          --group <SecurityGroupForInstance>
          --region <RegionForAutoScale>
          --instance-type <InstanceType>
          --image-id <AMI_ID>
          --launch-config "<LaunchConfigName>"

       LaunchConfigName= REPORTSCHEDULELAUNCHCONFIG
       KeyToUseToConnectToInstance=
       SecurityGroupForInstance=
       AMI_ID=
       InstanceType= t1.micro
       RegionForAutoScale=

Create Auto Scaling Group
as-create-auto-scaling-group 
         --region <RegionForAutoScale>
         --auto-scaling-group "<AutoScalingGroupName>"
         --launch-configuration "<LaunchConfigName>"
         --availability-zones <AvailabilityZones>
         --min-size 0
         --max-size 0

AutoScalingGroupName=REPORTSCHEDULESCALINGGROUPNAME
AvailabilityZones=ap-southeast-2a

Create Suspend Process
as-suspend-processes 
         --region <RegionForAutoScale>
         --auto-scaling-group <AutoScalingGroupName>
         --processes ReplaceUnhealthy

Create Start and Stop Schedules
as-put-scheduled-update-group-action 
         --region ap-southeast-2
         --name "<StartScheduleName>"
         --auto-scaling-group "<AutoScalingGroupName>"
         --min-size 1
         --max-size 1
         --recurrence "0 1 * * *"

 StartScheduleName=startReportScheduleInstance

The schedule creates an instance at 1 AM everyday (UTC time)

as-put-scheduled-update-group-action 
         --region <RegionForAutoScale>
         --name "<StopScheduleName>"
         --auto-scaling-group "<AutoScalingGroupName>"
         --min-size 0
         --max-size 0
         --recurrence "57 1 * * *"

StopScheduleName=stopReportScheduleInstance

The schedule terminates the instance at 1:57AM everyday (UTC time)


Comments

Post a Comment

Popular posts from this blog

API design best practices

API Design summary 1. Use two base URLs per resource 2. Keep verbs out of your base URLs 3. Use HTTP verbs to operate on the collections and elements 4. Be consistent either use plural nouns or singular nouns 5. Use concreate names rather than abstract names     An intuitive API uses plural rather than singular nouns, and concrete rather than abstract names 6. keep your API intuitive by simplifying the associations between resources, and sweeping parameters and other complexities under the rug of the HTTP question mark. Handling errors Errors boil down to Everything worked - ok The application did something wrong – client error The API did something wrong – server error In summary, be verbose and use plain language descriptions. Add as many hints as your API team can think of about what's causing an error. eg: HTTP Status Code: 401 {"status" : "401", "message":"Authenticate","code": 20003, "...
Read more

DB Connection Issues

1. Application gives "1c86bdf0 SharedPool I J2CA0086W: Shareable connection MCWrapper id 686bbdf9 Managed connection com.ibm.ws.rsadapter.spi.WSRdbManagedConnectionImpl@6156bdf9 State:STATE_TRAN_WRAPPER_INUSE" message when trying to do a db update within loop which is querying a view and carrying out some operations. In IBM this problem is documented under the topic " Application hangs with prior J2CA0086W warning messages" The solutions are  1. The application must be modified to use serial reuse or 2. The connection pool can be changed to unshareable. Shareable connections are not allowed within a local transaction containment boundary. The above scenario suggests a possible problem in the application logic and how it is using the connection code. The J2CA0086 message is telling you that your application is using a shared connection in a LocalTransaction. Since the connection is enlisted in a Local Transaction, and not a Global Transaction, di...
Read more

Reading Excel Sheet on client side and processing it

Image
If you ever want to process excel sheets on the client side (on the browser) then this will be useful for you.  This uses  http://oss.sheetjs.com  coed and the source files are in ( https://github.com/SheetJS/js-xls ,  https://github.com/SheetJS/js-xlsx ) it is licensed under Apache Licence, Version 2.0 (refer attached licence in the libraries in github) Steps to Follow, JS Files required: Copy the code for following files and make corresponding files locally, or download them from github,                 https://raw.github.com/SheetJS/js-xlsx/master/xlsx.js                 https://raw.github.com/SheetJS/js-xls/master/xls.js                 http://stuk.github.io/jszip/ We will be taking some javascript already in oss.sheetjs.com for our example particularly on the main page of oss.sheetjs.com. Sample Excel Sheet: Our exce...
Read more