How to Develop and Operate Cloud Native Data Platforms and Applications скачать в хорошем качестве

How to Develop and Operate Cloud Native Data Platforms and Applications

Data Orchestration Summit 2019 organized by Alluxio www.alluxio.io/data-orchestration-summit-2019 November 7, 2019 Speaker: Du Li (Architecture of Data Infrastructure, Electronic Arts) Today, one can easily launch or terminate services with hundreds or thousands of compute instances in just a few seconds on cloud services such as AWS. However, operating, monitoring and maintaining those resources could also easily become a nightmare if the corresponding systems were not designed in a cloud-native way. In this talk, we share our lessons in building and rebuilding our monitoring systems and data platforms at Electronic Arts (EA). In the first generation of the monitoring system, configurations were manually created for many individual software components and spread over all the resources. As services were started and terminated rapidly over time, it was extremely difficult to keep all configurations up to date. Consequently, on average we received over 1,000 alerts from thousands of machines on a daily basis, which stressed the operations team. We redesigned the system in late 2018 in a project called Monitoring As Code (MAC) emphasizing on version control and automation. MAC manages all the configurations using a GIT project in the same way as software code. Moreover, it establishes standards so that the configurations are automatically generated and deployed to keep everything in sync. As a result, it reduced the daily average number of alerts by two orders of magnitude. For more Alluxio events: https://www.alluxio.io/events/