DEVOPSdigest

Hortonworks launched Streams Messaging Manager (SMM), a new open-source operations monitoring and management tool that provides end-to-end visibility in enterprise Kafka environments.

It allows operations, DevOps/developers, and security/governance teams to gain clear insights about their Kafka clusters and understand the end-to-end flow of message streams from producers to topics to consumers.

Streams Messaging Manager also allows users to visually:

- Troubleshoot their Kafka environment to identify bottlenecks, throughputs, consumer patterns, traffic flow, etc.

- Analyze the stream dynamics between producers and consumers by using various filters.

- Optimize their Kafka environment based on key performance insights gathered from various brokers and topics.

- Gain complete data lineage across multiple Kafka topics, producers and consumers with powerful data flow visualization tightly integrated with Atlas.

Streams Messaging Manager is now available to customers through Hortonworks DataPlane Platform. This enables a single instance of SMM to manage multiple Kafka clusters and also allows for a hybrid cloud deployment model.

Hortonworks DataFlow (HDF) 3.2 was also released. The new release of HDF simplifies operations, delivers stronger integration and interoperability between HDF and Hortonworks Data Platform (HDP) 3.0 and significantly increases performance for data-in-motion in hybrid environments. Core improvements enhance HDF performance while delivering a single open-source platform that integrates governance, security and management across the entire data lifecycle from the edge to analytics to real-time decisions.

HDF 3.2 delivers the following for customers:

- Enhanced platform resiliency to help ensure smooth operations on large clusters and flows.

- More granular control in a multi-tenant environment by utilizing Kerberos keytab isolation.

- Increased performance for streaming to HDP via support for Apache Hive 3.0.

- Consistent management for HDF and HDP using the same operations, security and data governance instances.

HDF 3.2 is now available.