Saying GA of DataFlow Capabilities







At this time, we’re excited to announce that DataFlow Capabilities (DFF), a function inside Cloudera DataFlow for the Public Cloud, is now typically accessible for AWS, Microsoft Azure, and Google Cloud Platform. DFF gives an environment friendly, price optimized, scalable approach to run NiFi flows in a totally serverless vogue. That is the primary full no-code, no-ops growth expertise for features, permitting customers to avoid wasting time and assets. 

Fig1: First no-code UI within the business to shortly develop and deploy features to cloud suppliers’ serverless compute companies.

First no-code UI for serverless features

Beforehand, builders needed to write code and depend on code samples to get began with features. Now, they’ll use DataFlow’s no-code UI to be extra productive – they’ll shortly design new NiFi flows after which run them as features in AWS Lambda, Azure Capabilities, and Google Cloud Capabilities.

Fig2: DataFlow Capabilities runtime environments can be found in
AWS Lambda, Azure Capabilities, and Google Cloud Capabilities.

Optimize price and remove infrastructure administration

For the reason that knowledge flows are operating in serverless environments within the public clouds, infrastructure administration is a factor of the previous. The stream is simply executed when an occasion triggers the operate, providing a really environment friendly manner of deploying event-driven use circumstances with out requiring builders to expend priceless assets on operational obligations. For example, a file touchdown in an object retailer (S3, ADLS, or GCS) triggers the execution of an information stream, which then processes the file and sends the consequence some other place.

Fig3: A pattern use case the place a file that lands in an object retailer triggers a operate that processes that file and sends outcomes to a vacation spot.

DataFlow Capabilities gives an environment friendly, price optimized, scalable approach to run NiFi flows in a totally serverless vogue for event-driven use circumstances.

The proper runtime on your use circumstances

There are actually two methods to run your Apache NiFi knowledge flows within the Cloudera DataFlow service: DataFlow deployments and DataFlow Capabilities:

  • Deployments runtime is optimized for high-throughput, low-latency streaming use circumstances
  • Capabilities runtime is greatest suited to event-driven, short-lived use circumstances 

Fig4: Runtime choices within the public cloud: DataFlow Deployments and DataFlow Capabilities

Under is a extra detailed breakdown of the 2 NiFi runtime choices within the public cloud: 

Runtime choices within the Public Cloud
Function DataFlow Deployments DataFlow Capabilities
Cloud Runtime NiFi clusters utilizing 


NiFi flows operating on cloud suppliers’ serverless compute companies (AWS Lambda, Azure Capabilities, and Google Cloud Capabilities)
Use Case Use circumstances that want low latency for prime throughput workloads requiring always-running NiFi flows Occasion pushed, micro-bursty use circumstances with no sub-second latency requirement the place NiFi flows don’t must run constantly
Advantages Auto-scaling Kubernetes clusters for lengthy operating workflows with centralized monitoring Environment friendly, price optimized, scalable approach to run NiFi flows serverless, permitting builders to deal with enterprise logic


DataFlow Capabilities gives a brand new, environment friendly approach to run your event-driven Apache NiFi knowledge flows.

With DataFlow Capabilities you possibly can deploy your stream functions in minutes by leveraging the serverless structure of all main public cloud suppliers (AWS, Azure, and Google Cloud Platform), and also you shouldn’t have to fret in regards to the operational overhead of managing and sustaining NiFi stream runtime environments.

To be taught extra on easy methods to arrange and run DataFlow Capabilities in AWS Lambda, Azure Capabilities, and Google Cloud Capabilities, checkout our technical weblog, or take a product tour for a light-weight step-by-step expertise.


Share this


What companies are using big data analytics

What do companies use big data for? What companies are using big data analytics. There are a multitude of reasons companies use big data, but...

How to use big data in healthcare

What is data quality and why is it important in healthcare? How to use big data in healthcare. In healthcare, data quality is important for...

How to build a big data platform

What is big data platform? How to build a big data platform. A big data platform is a powerful platform used to manage and analyze...

Recent articles

More like this