From UC Berkeley RISE Labs : Clipper to deploy your models

The RISE Lab (Real-time Intelligent Secure Explainable) was formed in 2017  after the AMP lab was retired at UC Berkeley. AMP lamp produced the BDAS (Berkeley Data Analytics Stack) including Spark and MLLib which have become a mainstay for many Analytics solutions.

The Rise lab held an annual presentation on Oct 11th and 12th 2018.

One of their projects Clipper is talked about briefly here.

Organizations around the world are deploying machine learning in varied applications that require accurate, robust as well as real-time predictions, specifically when there is an overload of queries. But, most of the machine learning systems and frameworks do not address deployment and are restricted to model training only.

This is where Clipper comes into picture. Clipper is a prediction serving framework that is capable of processing high volumes of data with very minimal latency or delay. Clipper system interposes between end-user applications as well as a broad range of machine learning systems. It essentially gives rise to a modular framework that helps application users in simplifying deployment of different models across applications and frameworks. Read on to discover more about the Clipper system and its many advantages for the application users.

Clipper Explained

Clipper is basically a unique system that serves prediction and sits right in between user-facing apps as well as the widely utilized machine learning frameworks and models. Clipper also helps in the reduction of prediction delay or latency through the introduction of batching, caching as well as through model selecting techniques that are adaptive in nature. It also helps in improving the prediction accuracy, robustness and throughput without making any modifications in the machine learning systems.

What Clipper Does?

In simple terms, Clipper helps in simplifying the integration process of machine learning methods into user facing apps by simply giving a standard REST interface for feedback and prediction across a number of widely utilized machine learning systems. As a result, Clipper ensures that the members of the product team feel happy and satisfied.

Clipper also simplifies the deployment of model as well as helps in minimizing the bugs present in the system. It does this by utilizing the similar libraries and tools used at the time of the development of the model for the purpose of rendering the predictions that are made live.

In addition to the above, Clipper plays a pivotal role in improving the accuracy of prediction by launching the advanced bandit as well as ensemble techniques to choose and combine different predictions intelligently as well as complete personalization in real-time across multiple machine learning systems. Thus, Clipper also makes the users very happy.

The Importance of Clipper

Clipper is being built by a leading team of experienced researchers in The UC Berkeley Rise Lab who are currently learning more about the fundamental issues as well as challenges that will help them in taking the machine learning concept to production. The group has collaborated with prominent research and industrial organizations and identified that deployment of model will be one of the biggest issues facing the industry-wide adoption of Artificial Intelligence technologies.

The deployment of models based around trained machine learning within production requires time  and is also an ad-hoc procedure. The process is also prone to a number of errors, which results in massive impediment to creating as well as maintaining user-facing apps that revolve around machine learning.

Clipper helps in simplifying this particular procedure by way of decoupling of apps that take predictions via trained models that create predictions. Thus, Clipper is an extremely robust and high-performing serving framework capable of scaling to hundreds and thousands of requests every second. It offers responses, which easily meet the delay service objectives within milliseconds. This is why Clipper is very safe and can be easily incorporated within a stack that serves production without impacting the application delays.

Clipper enables data scientists to deploy the models that have been trained to production with ease. The simplification of the model deployment procedure enables data scientists to conduct experiments with fresh models and features to enhance the accuracy of applications quickly. It must be noted that a data scientist deploys different models to the system of Clipper using the similar code utilized for the purpose of training, getting rid of common bugs found in machine learning that surface as a result of the duplication of code. Clipper easily supports the trained models that have been deployed in numerous machine learning systems and furthermore implemented in a number of programming languages for supporting the rich and complex network of data science tools that are available in the present day scenario.

To sum up, Clipper lets you deploy the models that have been trained within your preferred choice of system to it with only a couple of code lines by utilizing an already existing model container or by using one that has been written by you. Clipper also allows you to quickly update or even roll back the models within the applications that are running. One can also set delay or latency service level goals for trusted query delays or latencies. With Clipper, you can run every model in a completely Docker container that is separate for managing clusters as well as allocating resources. You can also deploy different models that are operative on GPUs, CPUs, or even both of them within the same app.