DEVOPSdigest

Iguazio announced Nuclio ML Functions, broadening the serverless capabilities of Iguazio’s Data Science Platform for scalable machine learning training and data preparation.

Nuclio is an open source serverless framework that extends beyond event driven workloads to long lasting, parallel, and data-intensive jobs. This brings the benefits of serverless - allocating resources on demand, auto-scaling, and dev ops automation - to machine learning, data preparation and analytics workloads.

Serverless frameworks have so far only addressed challenges posed by the beginning and end of ML pipelines, namely data ingestion and model serving. Iguazio’s open source Nuclio offers performance and parallelism, support of GPUs at scale, multi-cloud deployment and native integration into the data science stack.

With Nuclio ML Functions, Iguazio now provides a layer of automation and monitoring for the widely used ML and analytics frameworks on top of Kubernetes, relying on its high-speed shared data layer for seamless scaling. ML function activities and data artifacts are automatically logged, allowing users to trace data and experiment results and re-run older jobs when needed. Nuclio ML Functions also leverages Kubeflow to speed up the running of ML pipelines.

With the introduction of these new serverless features, Iguazio enables full automation and CI/CD for ML workloads, cutting infrastructure costs, tedious development and operations tasks. The parallelism and auto-scaling capabilities are enabling Iguazio’s customers to process the same tasks at a fraction of the time, consuming server and GPU resources upon demand.

“Data preparation, experiments and devops are the most time-consuming tasks in data science. Our goal is to minimize the overhead so data science teams can focus on innovation and building new applications,” said Iguazio’s CTO, Yaron Haviv. “Iguazio is democratizing data science, enabling deployment either in the cloud or on prem with familiar open source tools.”

Nuclio ML Functions supports the following workloads:

- Real-time ingestion and APIs

- Analytics and data preparation - using Spark and Dask engines

- Machine learning - using Dask, XGBoost and Scikit

- Deep learning - using TensorFlow, PyTourch and Horovod

- Model Serving - using Nuclio Serving

Nuclio ML Functions will be generally available later this year across both cloud and edge versions of the Iguazio platform.