Tag Archives: aws

Moving to the Cloud with Amazon Web Services

Presenters

  • Ron Kraemer, VP and CIO, University of Notre Dame
  • Ryan Frazier, Director, System Engineering & Operations, Harvard Business School
  • Sarah Christen, Director of Community Platforms and CIO, Cornell University
  • Mike Chapple, Senior Director, IT Service Delivery, University of Notre Dame
  • Blake Chism, IT Transformation Sr., Amazon Web Services

Resource

Session Introduction

RC: we want to accomplish 1 major goal: roadmap and framework to take back to campus and “deal with the cloud in your culture and your world.”

It’s not perfect, and it’s a lot of work. BUT, it’s better service to our universities if we do it well.

SC: we’re a cloud-first institution. Lots of leadership change since that initiative started. We have 62 accounts under our master contract (master contract signed 18 months ago). Lots of accounts outside our contract. About $300K annual spend outside the IT org…we have a very distributed IT model.

We call the transformation “cloudification.” It’s a partnership with campus IT units. We refactor for most effective use of cloud technologies and containerization vs. “lift and shift.” Central IT must be the expert that campus wants to come to for help. We want to enable, not enforce (we do have SOME requirements to move to the master contract). We understand that if IaaS isn’t better with us, campus will make the move without us. We allow campus technologists to focus on unit differentiators central IT can help with the utilities.

Reqs for Cornell Master Contract

  • Onboarding discussion
  • Attestation
  • Shibboleth for authentication
  • DUO for multi-factor authentication for AWS Console access
  • Lock down root account, escrow with security office
  • Activation of AWS config
  • Activation of CloudTrail
  • CloudTrail logs sent to Security office
  • Activation of Cloudcheckr

What About Researcher AWS Accounts?

  • Easy onboarding without a lot of steps or complication
  • No interference with their research. No overhead (cost or performance)
  • Solutions for export control data and other compliance reqs.
  • Standard network config not always a good fit. “I am an island, not part of Cornell campus.”
  • Technical consultation options: docker, data storage, training, devops support

Today

  • All centrally hosted apps are being moved if possible
  • Infrastructure services are a large part of our on prem inventory
  • Campus units are moving more quickly than our central IT org

Biggest Challenge to Cloud Transformation: RESISTANCE TO CHANGE

RF: I’m director of Infrastructure Customer and Project Services. Initiated cloud strategy and planning when I was in the central IT division.

Cloud @Harvard

  • <2013: Exploration. Very early adopters at Harvard Medical School (research lab), pockets of uncoordinated use, little use within central and school-level IT departments.
  • 2013-2016: Alignment. We got enterprise agreement, direct billing and enterprise support services, laid technical foundations, brought on early adopters, developed cloud strategy.
  • 2016-?: Implementation. Accelerating adoption at all levels, i.e. labs, initiatives, schools, and central IT; shared service roadmaps; early adopters beginning to focus on optimization.

The Case for Cloud

  • Quality, cost, reliability, speed.
  • cloud.huit.harvard.edu
  • Our goal was to have 75% of our infrastructure at AWS by 2017. We’re currently at 31%.

HBX: Can We Deliver the Rich Interactive Experience of the Business School Online?

  • LET’S TRY
  • Move fast – 90 days to build, implement and launch application and registration system, < 1 year for complete course platform
  • Run independent of HBS IT – minimize impact on eisting services, enable new approaches to new needs
  • Be able to scale up or down rapidly – prepare for success or failure of the experiment

AWS Service Mix

  • 17 VPCs, 23 ELBs, 135 EC2 instances, 345 EBS volumes, 18TB instance storage, 4 Redshift Clusters, 18 RDS DBs, 30+ TB loaded via snowball, 78 TB object storage
  • Storage is a very small part of our spend (data transfer is 1%)
  • EC2 is about 58% of our spend

Notre Dame’s Journey to the Cloud

Why move at all? For us, we were sitting on an aging data center infrastructure. A capital investment – particularly cooling – had to happen if were going to continue. Tech demands from students, faculty and administrators outpaced our time and budget. In 2012, emergency communications were a critical concern.

2012

Originally we moved the web site as part of an emergency mitigation effort – “can we move the site in the event of an emergency?”

  • www.nd.edu
  • 3 web servers
  • load-driven autoscaling
  • Geographic diversity
  • It was really an easy move for us

2013

  • 435 web sites
  • 4 million monthly views
  • db as a service
  • ElastiCache

Cloud First

  • In 2013, we began having conversations about “why don’t we move everything over?”
  • We wanted to take advantage of what the cloud offers: 80% by the end of 2017; we’re at 59% today.
  • SaaS first, then PaaS, then IaaS, then on-prem.
  • Setting a goal created “a line in the sand,” that made it real for our people.

What We Learned

  • Rethink technical roles. NOBODY IS GOING TO LOSE THEIR JOB! However, you might not be doing the same job three years from now…
  • We were a very siloed organization prior to the cloud move. As a result of our move, those silos are breaking down.
  • Rethink security processes and tools (this was hard for us). We’re not mapping THINGS 1-to-1, we’re mapping OBJECTIVES.
  • Leverage automation – we’ve used ansible
  • Practical financial engineering. Our data center manager is now the guy who is our financial expert, who gives us insight into our costs. We’ve standardized on regions, instances (T2 class – about 3/4 of all our instances), use of reserve instances, etc.
  • Make a few choices and just go with them!

Cloud Transformation Maturity Model

  • Project Stage: limited knowledge, executive support, inability to purchase, limited confidence, no clear ownership or direction.
  • Foundation Stage
  • Migration Stage
  • Optimization Stage

Blake Chism from AWS: we developed this model to help you figure out where you are in the process. We’ve found that for most of our customers, procurement conversations are getting easier, but they’re still a challenge. If the central IT team helps take ownership, it can help organizations move forward more effectively, i.e. central IT not perceived as “being in the way.”

If your team has good processes now, your move will be much easier.

Project Stage

No matter what, you need to have a business case, a reason why you’re doing it. The roadmap helps describe how you’re doing it. Governance models evolve, and you get better at understanding them. Services change, and you need to have a plan about how you’ll integrate them (or not).

POC are much easier because if it doesn’t work, you can simply shut it off and you’re only out a few bucks. Try things out!

During the Project Stage, establish a “Cloud Center of Excellence” or “Cloud Competency Center” to get the organization moving in the right direction.

Foundation Stage

Lack of a detailed organizational transformation plan can be a challenge. Do a staff skills gap analysis to help you here.

Migration Stage

Should be as short as possible to get over the hump of hybrid and duplicate hosting. All-in will allow you to BEGIN doing new and exciting things. Imagine a space where the default state of, say, development environments, is OFF. All in is just the end of the adoption journey.

Were your enterprise systems like LMS, SIS, HR, Financials and the portal viewed as special and treated differently from smaller apps? Have you moved them yet?

  • Cornell: our KFS (Kuali) finance moved first (we dockerize ours) high availability on file shares was an early challenge (EFS – Elastic File Services are out now)
  • Harvard: IdM was first, we do Peoplesoft now, Oracle e-business is happening now
  • ND: ERP and LMS  – do not separate db servers and application servers!

AWS Cloud Adoption Journey

ALL: we use our AWS solutions architects extensively, and we’ve relied on AWS consulting almost exclusively for our migrations. These interactions have helped to accelerate our staff learning, because our staff are the ones who will need to maintain it long-term.

The professional services unit can help you figure out the high-level ecosystem you need for your particular situation. Enterprise support services is a bit pricey, but it’s useful in many cases.

SC: at Cornell, we created a 100 day training program that includes getting Amazon Solutions Architect certification. This is a good way to assure a certain level of competency. Some schools are using our model for training up their people, and they’re also using it as a way to network and learn new things, i.e. get names of people at other institutions that are going through the same problems.

Building the Roadmap – “Cloud Adoption Framework”

More details here: https://aws.amazon.com/education/movingtothecloudworkshop/

Organizes and describes the perspectives in planning, creating, managing, and supporting a modern IT service. Provides practical guidance and comprehensive guidelines for establishing, developing and running AWS cloud-enabled environments.

Don’t try to use all the components at once! Have your Cloud Center of Excellence (or whatever you choose to call it) do it in sprints by taking five or six of the elements and working through them.

In the private sector, the push to move to the cloud typically comes from the top. In higher ed IT, the push to move to the cloud typically comes from below. What we’ve often done is break off a small part of our budget, and use it to fund an “engineering SkunkWorks” where we can do the POCs and get staff buy-in. If the “where you do computing versus how you do computing” equation doesn’t click in your leadership’s minds, you’re going to have a hard time going anywhere.

The 2016 MMWCon Mega Post

Hey friends,

Once again, it’s time for another one of my conference “mega posts,” this time from the 2016 UCLA Mobile & Modern Web Conference. This mega post links to the notes I took from almost every session I attended, and proves that I was here and learned something. Some of this will be more useful than others…I drifted at times. There was some great work on display at this conference; I always meet awesome people and take away at least a couple gems that I didn’t know going in. As always, any mistakes, omissions or just plain crappy coverage are totally mine. Who knows, I may have even gotten something right 😉 Enjoy!

Wednesday, October 14

Thursday, October 15

Friday, October 16

 

Serverless Mobile Application & Game Development with Amazon Web Services (AWS)

Presenter

  • Curtis Bray, Manager of Solutions Architecture – US Education
  • Heather Matheson, Account Manager for UCLA
  • Amie Carobrese, Account Manager for UC & CSU

We’re deep into the public sector!  2,300+ govt agencies, 7,000+ educational institutions, etc.

When starting out, you pick a region, which is a location where your stuff (apps, data, whatever) will live. For example US-West-2 is a region. Within each region is a number of availability zones (AZs), separated by tens of miles.

How Do You Build a Mobile App Today?

  • Authorize
  • Authenticate
  • Synchronize
  • Store & Deliver Media
  • Analyze User Behavior
  • Send Push Notifications (bus arrival, quiz/test results, etc.)
  • Send Real Time Events
  • Store Shared Data (i.e. leaderboards)

Some of the products that’ll be covered:

  • Cognito: authentication
  • Kinesis Recorder: create live dashboards to track user events in real time
  • DynamoDB Mapper: store & query fast NoSQL data across users & devices
  • SNS Mobile Push: for push notifications
  • Lambda: “serverless” service that runs your functions

Authenticate Users: Amazon Cognito

What it does:

  1. Synchronize user’s data across devices / platforms
  2. Manage users as unique identities across identity providers
  3. Securely access all AWS services from Mobile device

It can provide temporary credentials to securely access your resources. It provides comprehensive support for identity use cases.

Synchronize data across devices: Amazon Cognito (Sync)

  • Store app data, preferences and staet
  • Cross-device Cross-OS Sync
  • Work Offline

Post processing can be done: push sync, events, streams

Store & Deliver Media Assets: S3 & CloudFront

  • S3 Connector transfer utility: multipart upload, fault tolerant download, no backend required, automatic retries, pause, resume, cancel functions.

Mobile Analytics

Collect, visualize and understand app usage data. Data stored can be put into a data warehouse if you want to.

Send Push Notifications: Amazon SNS Mobile Push

A single API can push messaging across multiple platforms.

Amazon DynamoDB Connector: Object Mapper

A NoSQL database that’s mobile aware

Which services should I use?

How do I connect them all together? We created the “AWS Mobile Hub” which ties into your AWS account. You create a project and add features you need to get up-and-running. The AWS Mobile Hub needs an IAM role so it can create resources on your behalf.

Working Mobile App project includes:

  • Xcode/Android Studio project
  • AWS resources
  • App Code
  • Helper “glue” code
  • SDKs
  • Detailed developer instructions

We then ran through an hour-long demonstration of the tools mentioned above…pretty powerful and easy-to-use stuff. The automated testing with AWS device farm was cool. You test against actual physical devices.