At Scale Enterprise Computing

Presented on 20 May at CLOSER 2015

Keynote Lecturer: Chung-Sheng Li

Abstract: At scale computing” is becoming one of the most disruptive trends in recent technology history, and is becoming the central theme for the post distributed computing era and likely to become the primary driver for the innovation and IT transformation during the coming decade.  
At scale computing describes a computing environment that may involves very large amount of computation, transactions, large amount of data, large number of users, or any combinations of the above.  Recent examples of at scale computing include AWS/Google datacenters, Amazon/Alibaba e-Commerce activities, Facebook, Google search, Netflix, etc.  These are in direct contrast with more traditional at scale enterprise such as Walmart, UPS, VISA/Mastercard/Amex.
At scale computing is a paradigm shift from the traditional distributed computing.  It is both a blessing and a curse:  Very large scale of computation offers new opportunities to rethink how to achieve resiliency without having to pay for redundancy.  It definitely promotes the possibility of “fail in place” as opposed to having to perform field service of a failed component in the environment as soon as it happens.  It also offers great opportunity to amortize the potential investment involved in optimization and customization.  At scale does impose severe challenge on just about every aspect of a large scale system, including system architecture, hardware, software and the continuous operations.  It stresses the importance of continuous availability, extreme scalability, and maniac focus on cost efficiency (capex, opex) as every penny counts in this at scale environment.
In this keynote, we will discuss this new “at scale” era - which we believed has started, and perhaps well under way.   Nearly all exciting innovations for enterprise computing during the past decade originated from this environment. These innovative technologies were often motivated by at scale business models.  All of these at scale business models started small.  They invariably found the recipe for creating a virtuous cycle among the “addictive” services or content they provided, and the community and ecosystem created on top of it.  In such a virtuous cycle, addictive content or services attracted new members into the community (either as direct consumers or developers), which in turn generate more content or develop more services.  This virtuous cycle often leads to winner takes all in nearly all case studies.