AWS re:Invent 2020: Executing a large-scale migration to AWS

hello welcome to ent 301 this is executing a large-scale migration to aws my name is jonathan allen i'm part of the enterprise strategy team and have had the privilege of helping hundreds of customers execute their own large migrations to aws as well as being a customer before i joined aws in 2017 this is going to be a level 300 presentation focusing on the deeper lessons to be learned covering the business drivers the migration phases a technical deep dive on the mental models and an overview of the migration acceleration program the focus throughout will be on lessons learned from the thousands of enterprise customers who have successfully migrated to aws and they cover nearly every imaginable industry segment from unicorns like airbnb pinterest through to large fsis like goldman sachs and capital one through to streaming providers like netflix disney comcast and the thousands of government departments and non-profits that now host aws trust aws to host their most important workloads now i have found that each customer has a multitude of reasons to use aws from the dramatic improvements in operational resiliency and security through to being able to compete globally but by far the most common thread that runs through all conversations is that of cost reduction the ability to migrate migrate and reduce the cost burden of the continual and intensive effort that is required to maintain those traditional on-premise systems and instead leverage globally on-demand secure building blocks that allow customers to focus far more of their precious human talents customer facing opportunity that is a powerful catalyst for example live nation delivered 58 cost savings increased their availability and saw a 10x improvement in their innovation pipeline allowing them to go faster than what was actually comfortable and the most common question i get in executive briefing sessions is this achieving that speed with the hindsight of others that have gone before is very important to our outcomes so with that in mind let's dive in now the most important lesson throughout this talk is that executive sponsorship is the biggest enabler that needs to be in place for any successful large migration to succeed in my experience this is best when a single senior leader empowers executive leads from the front and has established a bold declarative business goal to migrate to the cloud this single threaded leader as we refer to the mizan amazon has the ability to ensure holistic cross-executive engagement assign resources remove impediments and establish cadence this individual operating under the principle of all your assumed constraints are debatable can be a powerful binding tenet fabio veranuzzi annelle's head of infrastructure and technological services who have i've had the pleasure with working with spoke at reinvent 2016 about their large-scale migration he delivered clear unambiguous messages on their migration goals overall the initiative ran across 30 countries totaling about nine and a half thousand servers and the migration of six petabytes of storage achieving improved reliability greatly reduced provisioning times and delivering 21 compute savings and 60 saving on storage costs having this single threaded leader bring all of their peers on the journey is a crucial element to success if a silo tries to do it on their own problems will arise at its best it requires an holistic effort having leadership team meet weekly if not bi-weekly to remove any impediments get answers to questions and ensure consistent momentum is maintained is their shared goal now it was archimedes who said give me a lever long enough and a fulcrum on which to place it and i shall move the world the best fulcrum for cloud migration and adoption is by creating a cloud center of excellence fundamentally this means bringing together all of the technological performers to establish your first cloud patterns when using aws into a dedicated team the best team composition infrastructure engineers security engineers application engineers operations engineers a lead architect coming together with the right ccoe leader this team defines security objectives availability objectives reliability objectives and compliance objectives and getting the first workload if you're not already using aws into the cloud now at the start it can actually be a little intimidating when you're faced with thousands of applications maybe even hundreds of thousands of servers it can feel a little bit overwhelming but it doesn't have to be we can break the opportunity down into three phases which in turn can form waves of migrations assess mobilize migrate and modernize now within the assess phase there are three mechanisms available to you firstly the aws migration evaluator and or working with the aws cloud economics team to help establish your business case secondly working with the aws solutions or programs proserv architecture to carry out a migration readiness assessment and finally establishing the use of the aws migration hub is the central real-time location that can provide clear oversight of all of your migrations and their progress metrics let's dig a little deeper migration evaluator formerly tso logic is a complementary service to create data-driven business cases for aws cloud planning and migration technical deployment of the aws migration evaluator is straightforward needing a dedicated windows server 2012 or greater and it then uses agentless connectors to bring back the technical information in encrypted form to a central location you control it supports vmware hyper-v a number of bare metal servers and sql server alternatively if you already have your existing event inventory and utilization data you can provide these via flat files also the use of the migration evaluator is supported in all regions except for mainland china and the service is hosted in the u.s east region once it has completed data gathering it can then produce a number of different deliverables with different cost perspectives based on the choices you have available to you these outputs can then be directly used to support the business case for migrations a migration readiness assessment or mra is a step to objectively inform of your potential for migration success based on the six categories of the cloud adoption framework an aws solutions architect or pro serve leader will work with you to perform a one-day assessment of your cloud capabilities and readiness by taking through you through approximately 70 questions and coming to a consensus rating across your organization in my experience this provides super valuable insights about your readiness and can discover potential blockers the output of the mra is not just red yellow green rating but a recommended action plan for all critical observations that feeds the mobilized phase activities and is tailored to your specific needs the assess phase not only allows you to get your data and business case correct it is the fundamental input to ensuring you can make the correct migration choice for each type of workload and match to each r method which we're going to cover in some detail but at the same time as doing this we need to ensure that the landing zone is correctly in place what's a landing zone a landing zone is an aws environment with multiple accounts that are configured in a way to allow multiple people and teams across your organization to start to use aws in a secure scalable manner it allows you to experiment iterate and migrate workloads into it without having to worry about scalability or security concerns as your cloud footprint grows it's very normal for the ccoe team to work to establish this initially and then to make it broadly available working with your aws solutions architect accredited partner or aws pro serve guide they can help you put in place the basic parts of the landing zone including shared services log archive and security service foundations and then work with you to expand it if required with appropriate directory connectors for your business eventually culminating in a vending machine like setup where you are able to vend new accounts for new workloads and teams as required now just having one ccoe team who knows cloud is not going to scale for a large organization we need to really scale the reskilling in 2019 at reinvent i presented a separate session on how to achieve reskilling at scale with national australia bank and provide the links to this here scaling to have multiple folks available to migrate and skilled engineers and developers is the second greatest key to success after executive sponsorship let's go over some lessons learned from the assess and mobilize phase firstly firewalls actually i find a key having dedicated network and file resource assigned to the ccoe team and to assist with migration evaluator is crucial secondly side of the desk resource thing doesn't work in my experience agile teams that own the outcome does remember you don't have to change reporting lines to achieve a dedicated virtual team outcome data has gravity and applications typically require low latency to their data stores so consider this when grouping workloads establish people-centric mechanisms to reskilling check out those links from the previous slide crucially important to shortlist migration partners at this stage ask for their customer migration references scrutinize incumbent bidders and symbiotic with partner selection what is the tooling they have used with efficacy previously and finally the biggest lesson learned dependency matting dependency mapping matters dependency mapping is very typically a concern at this point now as a customer myself we worked with software partners like new relic and here app dynamics to actually give us the pictorial and accurate representation of our dependencies so as we move parts of the puzzle we formally understand what is talking to what and ensure we can replicate those communication pathways obviously you'll need to make your own call on whether you need to do this before we go into the different migration models if you haven't used the migration evaluator and need to gather your data of what's running already we also offer the aws data the aws application discovery service this can be used in both agentless for vmware and agent-based methods to discover your current deployed infrastructure and then populate that data into amazon athena which you can then query via amazon quicksite the data from the application discovery service can also be used to populate the aws migration hub aws migration hub is designed to be a single location to track the progress of application migrations across aws and partner solutions you can view server information and group servers into applications to help plan your migration you can track the progress of your migrations using the migration tools that best fit your needs it can manage migrations in any region that has the necessary migration tools available and it is available to all aws customers at no additional charge now with your inventory data in hand you can now break apart the opportunity into different methods aligned to different outcomes for your applications we call these methods the 7rs and the first one is relocate vmware cloud on aws is available in 17 regions around the world and supports different vsphere installations as well as a variety of tools and methods for hybrid migration this is an extremely rapid way to move your existing on-premise vsphere workloads to the cloud the second r is rehost this is where you're actually going to take the existing server operating system that's running the application or database and you're going to move them both off of the on-premise hypervisor and put it on top of ec2 now there are a lot of tools available to do this i'd also suggest that you do it manually to understand what is happening in this process the first time so you can clearly see how this works let's take a quick double click into re-hosting and the tooling aws offers two services to help here cloud endure migration and the aws service migration service each has different benefits depending on the requirements of your workloads with cloud endure being an agent-based migration tool offering block level replica replication and aws server migration service being agentless you can use both at no extra charge re-platforming is where you take the existing application or database and move it for example you could go from oracle web logic to apache tomcat and from oracle database to amazon aurora or you could just be changing out the underlying operating system from version 6 to version 7 while keeping the application database as is going a little deeper here you'll need to create a golden amazon machine image or army as we call them at the target operating system version that you need and then move the application and likely do this by reinstalling it with a software automation product such as aws ops works using either chef or puppet derivatives some top tips on re-platforming if you can keep the os at the same version to reduce kernel or registry impacts great the more you change the longer it's going to take you may have to create multiple golden armies at different versions to achieve this at pace the other top tip what does minimum viable refactoring look like many customers use a two-week rule if you can't re-platform the app in two weeks then re-host it instead keeping momentum going is the key don't let yourself get bogged down by by striving for perceived perfection what about re-platforming databases for this we have the aws database migration service so again you have your existing on-premise database types on the left-hand side here you can then use the dms service to migrate one of the many different types of target if some of your databases are very large maybe many terabytes and you don't want to be saturating your data connection link or your vpn link when you do this you have the ability to use dms alongside the aws snowball edge and to actually replicate the majority of your database in an encrypted format on the snow family device no encryption keys have to be kept on this device and it can then be securely shipped into a region and work in conjunction with the database migration service overall we've migrated over 300 000 databases using dms at the time of this recording these are the sources that the aws database migration supports and these are the targets for the dms service i would encourage you to always check the aws dms website as we are continually broadening the service scope now basic schema copy that comes with aws dms is great for doing a test migration or when you are migrating databases heterogeneously e.g oracle to mysql or sql server to oracle basic schema copy will not migrate secondary indexes foreign keys or stored procedures for that you need the aws schema conversion tool this makes heterogeneous database migrations predictable by automatically converting the source database schema and a majority of the database code objects including views stored procedures and functions to a format compatible with the target database any objects that cannot be automatically converted are clearly marked so that they can be manually converted to complete the migration sct can also scan your application source code for embedded sql statements and convert them as part of a database schema conversion project during this process sct performs cloud native code optimization by converting legacy oracle and sql server functions to their equivalent aws service thus helping you modernize the applications at the same time of database migration once schema conversion is complete sct can help migrate data from a range of data warehouses times and redshift using built-in database migration agents to complement the dms and sct tool we also have the database migration playbooks these are a series of guides focusing on best practices for creating successful heterogeneous database migration blueprints and currently cover five of the most common migrations we see the purpose of the database migration playbooks is to capture as much of the database business analyst or dba know-how part of the database migration equation as possible in easy readable reference documents those were available for download on the aws website a great example of a customer performing a database free platform is samsung migrating 1.1 billion users across continents from oracle to amazon aurora with the aws database migration service achieving a 44 reduction in monthly database costs some lessons here firstly get the code out of the database you need to be clear where business logic should live in modern stacks secondly one size does not fit all you need a kit bag of tools and guides next conduct drive runs can you replicate the data restore the data sets and the application with all the ports needed does it pass your testing measure performance understand the latencies before and after is crucial to meeting your customers performance expectations finally again finding the right partner that can do this with the right tooling is typically crucial to success what about repurchasing very often the conversations with the leaders and the engineers and the developers is what you really want to be focusing on building and then supporting what is going to truly differentiate your business where do you want to spend that precious engineer and developer resource innovating for your customers because if you're hosting systems which are not cool for your business and maintaining armies to that end you're going to struggle so when you're going through this really looking at the repurchasing angle is going to be crucial in the aws marketplace there are over 4 500 software solutions from more than 1400 isvs and this continues to grow and help customers migrate to the cloud so balance what you are building with what is already available in the marketplace refactoring what is refactoring very often the conversations i have with teams around the world strays rightly into that of devops we would like devops teams i would like continuous integration and continuous deployment of cloud native services for everything and i would like to have devops teams across the board so i'm going to refactor all my applications to be cloud native now in an ideal world and for new applications of course this would be great when you move up the stack as it were you get the best price points you get the very best efficiencies there is a lot of attraction but when you are migrating keeping momentum in place is really important so my advice here is to be really mindful about when to use refactoring in a migration now if you've some of the older non-x86 systems then refactoring is obvious putting teams together to tackle and move to cloud native services can be an extremely effective way of actually moving those older incumbent systems out of your data centers but be mindful about it you need to use all of these different r's in combination for an effective migration usually factoring thoughtfully to create truly cloud native architectures what about mainframes we have helped many customers achieve very successful migrations to aws cloud including coca-cola vanguard and many others there are now very established practices for re-platforming mainframes onto aws check out more in the session ent 307 mainframe workloads fast track to agility but ultimately what about retain are there some workloads you need to hang on to or need to sweat there may be latency specific workloads that physically just cannot move for that of course we have aws outpost that's now available in a number of regions for you to deploy on premise but also some customers need to hang on to some systems for a while longer but still want to close their data centers to achieve their cost goals well we do see some customers that want to rapidly move out of their data centers on incredibly aggressive timelines with some of these older systems actually move them we've had some customers choose to actually physically move systems to colos or even direct connect partners where they can get really low latency for those systems to still have access into the regions so we also see this coming along as an attractive option finally retire if it's not caught your business if it's maybe an application that supports monitoring in your existing data center and you'll be amazed how many you find of course you can label this with a retire or decommissioned label so using all of these seminars can be an incredibly clear way to actually label what you want to do now i spend a lot of time working with customers who at first can feel a little overwhelmed by their migration but literally taking their application portfolio and mapping it out with sticky notes using the seven hours mental model i find in a day you can create these swim lanes over a timeline and this can be very cathartic and provide a way three for leaders it's never going to be 100 accurate but it actually gives them a good basis to start from for their first migration wave it provokes lively conversation when suddenly leaders can see that if they migrate a particular group of applications faster they could potentially save more money by avoiding a capex spend for capacity or support reasons making their business case even stronger as they go so i get asked very often what sort of mix should i use when i'm doing a migration and there is no hard and fast rule for this the answer is you've got to look at the practicality of each one of these options and what's right for you but we typically see this as being a good guide and notice that rehost and re-platform works pretty effectively when you're looking at percent here now again i guarantee as engineers and developers who love technology there's a real propensity to want to choose refactoring or re-platforming but really again for that speed of migrating when we saw the tools available for rehosting actually rehosting gives you the best speed so here it is rightly indexed higher so use all of the r's but you'll be aware of some of the natural tendencies to want to build this crystal palace of code perfection as you go on when you're migrating momentum matters finally what about when you have vast amounts of data on premise and you want to migrate to the cloud for that you can use snow cone snow ball edge snowmobile to physically encrypt your data securely onto a device and then ship it straight to s3 or you can use aws data sync for secure continual replication now after executing thousands of migrations we've tried to package all of our learnings from all of those customers who are migrating hundreds of data center scale workloads and pull together what analysts are telling us is the industry's most comprehensive migration program that being the aws migration acceleration program it starts with the methodology packaging up all of these learnings investing and working with the isv ecosystem as well as our own tooling working with partners that we've vetted and enabling accredited partners professional services organizations who can act on the ground as experts to help guide you and alongside this re-skilling which i can't overemphasize enough being able being able to enable your organization to be able to do these types of migrations and in certain cases as customers may qualify for investments to help take care of some elements of the migration bubble that can sometimes arise as you instantiate your infrastructure in the cloud another customer i've had the privilege of working with is skyscanner skyscanner leveraged the migration acceleration program to accelerate the migration of their entire infrastructure operation to the cloud migrating thousands of vms and thousands of microservices and closed five data centers in three years so stepping back and looking at all the details of the map program the program can really accelerate each part of the journey with lessons learned from those that have gone before from helping you with your business case through your reskilling to choosing the right partner and finally to making the right technical migration decisions for your workloads and forming waves to execute these moves to achieve your goals in addition we come across specialized workloads that require careful consideration proven methodology tooling and partner ecosystem to support for this we have the map specialized workloads program covering storage windows vmware on cloud on aws sap database and finally amazon connect which very often is at the forefront of the ceo's desire to improve customer and agent experience although aws provides tooling of course our isv partners are a crucial part of this journey for customers you'll find many of their solutions available within the aws marketplace if not running on aws as software as a service some of which we've mentioned as we've gone through now throughout this presentation i have talked about partners and although aws have many thousands of partners only a small subset have achieved the sought after migration accredited partner status these are the partners who have a track record of successful customer migrations and audited annually against the competencies which you have seen as crucial to success with migrations now looking back on lessons learned at the migrate and modernize phase firstly the concept of minimum viable re-platforming is crucial be careful when opening pandora's box that you don't drown in too much work secondly forming special migration squads that have their goals aligned to working with application teams to achieve a timely and successful upgrade at scale can be highly effective evaluate what partners what tooling your partner uses should you be using it also probably if appropriate map credits can help smooth the way you don't need to re-org to migrate focus on migration velocity and savings you can achieve first make sure to visualize the waves of migrations via the swim lanes for everybody this plan will adapt to business situations and this is fine adapting to the agile environment we find ourselves in is crucial momentum and blockage removal is key and finally don't forget post migration cost optimization typically i see the most costs being saved when you right size compute and storage and use horizontal scaling compute instead of vertical scaling i'm going to leave you with a final thought from voltaire don't let perfect be the enemy of good enough thank you very much and please fill in our session survey this is incredibly important to us thank you

2021-02-09 00:40

Other news