Confluent launches point-and-click canvas to build streaming data pipelines

To further reinforce our commitment to providing cutting-edge data technology coverage, VentureBeat is delighted to welcome Andrew Brust and Tony Baer as regular contributors. Watch for their posts in the data pipeline.

California-based Confluent, an Apache Kafka-based data streaming platform, today announced Stream Designer, a visual interface to help developers build, test, and deploy data pipelines online. real time in minutes. The company showcased the tool at its ongoing Current 2022 conference.

In recent years, real-time data has become essential to business growth and success. Batch data is still widely used, but it falls short of many of the challenges and use cases businesses encounter in their routine operations. As a result, streaming data technologies that were once at the edge are seeing increased adoption, including Apache Kafka.

Over 80% of Fortune 100 companies use Kafka to manage large volumes of real-time data. However, when it comes to building streaming data pipelines on open-source Kafka, teams have to rely on highly specialized engineering talent and deal with long development timelines spread across multiple tools. This makes ubiquitous data streaming beyond the reach of many organizations.

Confluent's new Stream Designer, now generally available, provides a point-and-click visual canvas for describing data flows and business logic in the Confluent Cloud UI. This way, instead of dealing with the headache of managing individual components on Kafka (each requiring its own boilerplate code), developers can build pipelines with the full Kafka ecosystem accessible in a single visual interface. . They can also iterate and test before deploying to production in a modular way.

MetaBeat 2022

MetaBeat will bring together thought leaders to advise on how metaverse technology will transform the way all industries communicate and do business on October 4 in San Francisco, CA.

“This is the industry's first visual interface for creating streaming pipelines natively on Kafka. other, often closed, proprietary messaging technologies," Jon Fancey, senior product manager at Confluent, told VentureBeat. "Stream Designer is built on top of Kafka, which is widely known as the standard for streaming data. is the platform of choice for developers due to its highly scalable and resilient architecture, as well as its strong open source foundation."

Fancey said they held a closed beta program for Stream Designer with some customers and received "very positive" feedback.

"This has reduced the ramp time for developers who are discovering...