DEVOPSdigest

Conducto is launching a toolkit for simplifying complex CI/CD and data science pipelines, having raised $3 million in seed funding led by Jump Capital.

Conducto multiplies the productivity of DevOps, developers, data engineers, and data scientists, by making it easier than ever to automate their workloads. The tool is based on technology from Jump Trading, a quantitative trading firm, that powered billions of dollars in revenue generation.

"We view automation as an important part of increasing the efficiency and productivity of developers and data scientists. It was a key factor behind our own scale and success at Jump Trading," said Jonathan Marcus, co-founder and CEO of Conducto, who was a quantitative developer at Jump Trading for ten years.

"Conducto was originally going to be a data pipeline tool. We immediately switched to focus on CI/CD, and we're thrilled with how it has increased the productivity of our entire engineering team," said Matt Jachowski, CTO and co-founder of Conducto, who was also a long-tenured quantitative developer at Jump Trading.

"Conducto's pipeline as code offering built upon tree representation is truly differentiating. This approach has been battle tested in hyperscale environments and allows developers and data scientists to scale their pipelines to millions of nodes while maintaining speed and simplicity," said Saurabh Sharma, General Partner at Jump Capital, who will be joining Conducto's board.

Conducto pipelines are easier to write, scale, and debug than most other CI/CD or data pipeline tools.

- Writing pipelines: Conducto pipelines are written in code, not a configuration language. Pipeline developers can use Python (Javascript and other languages coming soon) – their own functions, abstractions, data sources, and business logic – to express their pipelines clearly and simply.

- Scaling: A medium-sized CI/CD pipeline can include hundreds of tasks, and data pipelines can easily have tens of thousands. Conducto pipelines are expressed as trees and not the directed acyclic graphs (DAGs) used by every competitor. DAGs become confusing after a few dozen nodes. At any scale, trees are simple to write, easy to visualize, and intuitive for humans to understand.

- Debugging errors: The debug cycle – finding errors, writing a fix, and getting the fix into the pipeline – is a major bottleneck for CI/CD and data pipeline development. Conducto has one of the best and fastest debugging experiences of any pipeline tool anywhere.

Each Conducto pipeline can handle millions of tasks, and Conducto's distributed architecture means it can scale horizontally nearly without limit. They have handled petabytes of data and petaflops of computing power.