Watching an Andy Jassy AWS re:Invent keynote can feel like attending the opening lecture of a killer college course where you’re taking notes so fast your brain can’t process all the information. Only after several days of rereading your notes and cross-referencing the course text does it begin to make sense.
Jassy, like Jensen Huang of NVIDIA, packs so much into keynotes that serve as highlight reels for an entire conference, it takes a while to unpack it all.
As I outlined in my last column, I saw four primary themes at re:Invent:
- Hybrid on premise-cloud infrastructure and services
- Enterprise IT management and governance services; making the cloud safe for the enterprise
- Machine learning services and performance optimization
- Bridging edge and IoT services with cloud services
Last time, I detailed AWS’s development of custom components and hybrid infrastructure, but most of the company’s announcements targeted machine learning (ML) and AI services, which is my focus here.
Jassy summarized the AWS ML portfolio as a three-level hierarchy that starts with ML frameworks and infrastructure, upon which it builds developer platform services, culminating with AI application services. I summarize these, with an homage to the familiar IaaS/PaaS/SaaS hierarchy, as infrastructure, platform and application (software) services and will highlight key announcements in each area below.
First, it’s critical to understand that AWS’s three-pronged strategy isn’t just designed for a few AI experts or budding specialists, but to, in the words of Google’s soon to be departing chief AI scientist Fei-Fei Li, democratize AI. There are many ways cloud services level the playing field for ML development, whether by providing rentable access to state-of-the-art compute power or streamlining the deployment of managed development frameworks like TensorFlow and MXNet. AWS, like Google Cloud and Azure, provides all of these, but many of its new services target the highest level of the ML service hierarchy, packaged applications. I believe these offer the greatest opportunity for AI democratization since they allow customers to use AI without fully understanding AI.
A statement from AWS VP of machine learning Swami Sivasubramanian nicely summarizes the company’s strategy (emphasis added):
We want to help all of our customers embrace machine learning, no matter their size, budget, experience, or skill level. Today’s announcements remove significant barriers to the successful adoption of machine learning, by reducing the cost of machine learning training and inference, introducing new SageMaker capabilities that make it easier for developers to build, train, and deploy machine learning models in the cloud and at the edge, and delivering new AI services based on our years of experience at Amazon.
AWS provides many flattering quotes from early customers that one must read through a selection-bias filter, however one from Tyson Foods, which was building an image recognition system for its processing plants, does illustrate the benefits of packaged AI services and development platforms (emphasis added),
When we first tried to setup our own labeling solution, it required a large amount of compute and a Frankenstein of open source solutions – even before creating the user interface for data labeling. With Amazon SageMaker Ground Truth, we were able to use the readymade template for bounding boxes and got a labeling job running in just a few clicks, quickly and easily.
The following summarizes the AI products and enhancements announced at re:Invent.
I covered most of these in my last column, notably the Inferentia custom SoC for ML inference, which should significantly reduce the cost of high-performance compute instances for running complex models. Inferentia won’t be available for a while, but a new Elastic infrastructure service provides immediate improvements to the flexibility and efficiency (and hence cost) of running GPU-accelerated EC2 instances and SageMaker environments. Elastic Inferences logically decouples the CPU and GPU configurations to let developers mix-and-match sizes, with support for NVIDIA-based GPU accelerators ranging from 1 to 32 trillion floating point operations per second (TFLOPS). AWS estimates that some configurations will save users 75 percent versus a fixed-configuration GPU instance.
AWS also introduced an evolutionary, albeit impressive, upgrade in top-end GPU performance via the EC2 P3dn instances that support up to 8 of NVIDIA’s fastest V100 GPUs alongside the latest Skylake CPUs. Significantly, these instances also are the first to use the fastest available network interface that, at 100 Gbps, can substantially improve the performance of distributed HPC simulations or deep learning model training that can be spread across as many as 64 instances.
As Jassy’s overview slide illustrates, SageMaker is the managed ML development platform AWS introduced at the last re:Invent and this year it’s been turned into a portfolio via several significant additions.
SageMaker Ground Truth can automate and outsource the tedious process of labeling datasets for ML model training. Labeling, namely, identifying and correctly tagging key features in training data, is a tedious, manual process that Ground Truth can farm out to either Amazon’s Mechanical Turk worker bees, an Amazon-vetted third-party vendor or one’s own employees. The service’s UI is designed to minimize human errors while its backend feeds the manually-assigned labels into a separate ML model used to eventually automate the labeling process, with less and less data sent to humans labelers.
SageMaker Neo is an ML compiler that enables models to run on a variety of supported devices via a Neo runtime engine managed by Greengrass. Neo is conceptually similar to Intel’s open source nGraph compiler and likewise supports the most popular ML development frameworks. Neo is designed to add ML intelligence to edge devices like video cameras, industrial sensors or any device with limited compute and memory since the Neo runtime only requires 2.5 MB.
SageMaker RL adds the ability to develop reinforcement learning (trial and error through with feedback on model decisions) models to SageMaker’s existing support of supervised learning (labeled data). RL allows SageMaker to be used on a broader set of problems, notably robotics control and language dialog (chatbots).
Two of the most compelling new AI services AWS introduced have their roots, like AWS itself, in Amazon’s use of technology to run its retail business. By commercializing and packaging some of the software that improves the Amazon shopping experience, AWS is tapping into a reservoir of ML expertise that few companies can match.
Amazon Forecast is a prediction engine for time-series data such as retail sales (surprise), other financial metrics (revenue, cash flow), traffic (either website or physical location) or various measures of IT infrastructure (server workload, network traffic/congestion). Unlike basic statistical regressions, Forecast uses deep learning algorithms that can include vectors of categorical (discrete values) features that can account for multiple parameters such as the size or color of a purchased item or if the weather is sunny, cloudy or rainy.
Amazon Personalize is a recommendation engine that can be used for shopping, digital or social media or any application where there might be a correlation between multiple parameters that would improve search results.
Besides these AWS managed services, it introduced an ML marketplace of more than 150 algorithms and model packages that work with SageMaker. These significantly expand the model portfolio with packages targeted for particular industries and use cases.
Although most of the new AWS ML services targeted the infrastructure and development platform layers of the service hierarchy, I think targeted applications like Forecast and Personalize will have the most long-term significance by broadening the base of ML developers. Recall that these join an existing portfolio of services for image, video and speech recognition, language and text translation and chatbot dialogs. While these aren’t packaged software titles and requires some software assembly to use, they significantly simplify the task of building ML into custom enterprise applications.
By encapsulating ML into packages targeting particular scenarios, AWS is following the footsteps of many consumer applications, notably phone cameras. Indeed, many of the best additions to the latest phones are based on AI modeling and analysis to provide features like low light image enhancement, portrait mode background blurring, multi-shot contrast enhancement and automatic facial enhancement. By hiding sophisticated deep learning models in an app people already know how to use, these AI-enhanced applications provide tremendous benefits to people that know nothing about AI.
While not quite as oversimplified, the new AWS application services provide similar benefits to enterprise developers that simply want to analyze data and solve a problem, not become AI experts. Such SaaS-level AI/ML packages as Forecast, Personalize and Rekognition are the next step in the democratization of AI.
Image credit – Amazon reInvent