Posted in Technical |
September 29, 2022 3 min learn



At this time, we’re excited to announce that DataFlow Capabilities (DFF), a function inside Cloudera DataFlow for the Public Cloud, is now typically accessible for AWS, Microsoft Azure, and Google Cloud Platform. DFF gives an environment friendly, price optimized, scalable approach to run NiFi flows in a totally serverless vogue. That is the primary full no-code, no-ops growth expertise for features, permitting customers to avoid wasting time and assets. 

Fig1: First no-code UI within the business to shortly develop and deploy features to cloud suppliers’ serverless compute companies.

First no-code UI for serverless features

Beforehand, builders needed to write code and depend on code samples to get began with features. Now, they’ll use DataFlow’s no-code UI to be extra productive – they’ll shortly design new NiFi flows after which run them as features in AWS Lambda, Azure Capabilities, and Google Cloud Capabilities.

Fig2: DataFlow Capabilities runtime environments can be found in
AWS Lambda, Azure Capabilities, and Google Cloud Capabilities.

Optimize price and remove infrastructure administration

For the reason that knowledge flows are operating in serverless environments within the public clouds, infrastructure administration is a factor of the previous. The stream is simply executed when an occasion triggers the operate, providing a really environment friendly manner of deploying event-driven use circumstances with out requiring builders to expend priceless assets on operational obligations. For example, a file touchdown in an object retailer (S3, ADLS, or GCS) triggers the execution of an information stream, which then processes the file and sends the consequence some other place.

Fig3: A pattern use case the place a file that lands in an object retailer triggers a operate that processes that file and sends outcomes to a vacation spot.

DataFlow Capabilities gives an environment friendly, price optimized, scalable approach to run NiFi flows in a totally serverless vogue for event-driven use circumstances.

The proper runtime on your use circumstances

There are actually two methods to run your Apache NiFi knowledge flows within the Cloudera DataFlow service: DataFlow deployments and DataFlow Capabilities:

• Deployments runtime is optimized for high-throughput, low-latency streaming use circumstances
• Capabilities runtime is greatest suited to event-driven, short-lived use circumstances 

Fig4: Runtime choices within the public cloud: DataFlow Deployments and DataFlow Capabilities

Under is a extra detailed breakdown of the 2 NiFi runtime choices within the public cloud: 

Abstract

DataFlow Capabilities gives a brand new, environment friendly approach to run your event-driven Apache NiFi knowledge flows.

With DataFlow Capabilities you possibly can deploy your stream functions in minutes by leveraging the serverless structure of all main public cloud suppliers (AWS, Azure, and Google Cloud Platform), and also you shouldn’t have to fret in regards to the operational overhead of managing and sustaining NiFi stream runtime environments.

To be taught extra on easy methods to arrange and run DataFlow Capabilities in AWS Lambda, Azure Capabilities, and Google Cloud Capabilities, checkout our technical weblog, or take a product tour for a light-weight step-by-step expertise.