At this time, we’re excited to announce that DataFlow Capabilities (DFF), a function inside Cloudera DataFlow for the Public Cloud, is now typically accessible for AWS, Microsoft Azure, and Google Cloud Platform. DFF gives an environment friendly, price optimized, scalable approach to run NiFi flows in a totally serverless vogue. That is the primary full no-code, no-ops growth expertise for features, permitting customers to avoid wasting time and assets.
First no-code UI for serverless features
Beforehand, builders needed to write code and depend on code samples to get began with features. Now, they’ll use DataFlow’s no-code UI to be extra productive – they’ll shortly design new NiFi flows after which run them as features in AWS Lambda, Azure Capabilities, and Google Cloud Capabilities.
Optimize price and remove infrastructure administration
For the reason that knowledge flows are operating in serverless environments within the public clouds, infrastructure administration is a factor of the previous. The stream is simply executed when an occasion triggers the operate, providing a really environment friendly manner of deploying event-driven use circumstances with out requiring builders to expend priceless assets on operational obligations. For example, a file touchdown in an object retailer (S3, ADLS, or GCS) triggers the execution of an information stream, which then processes the file and sends the consequence some other place.
DataFlow Capabilities gives an environment friendly, price optimized, scalable approach to run NiFi flows in a totally serverless vogue for event-driven use circumstances.
The proper runtime on your use circumstances
There are actually two methods to run your Apache NiFi knowledge flows within the Cloudera DataFlow service: DataFlow deployments and DataFlow Capabilities:
- Deployments runtime is optimized for high-throughput, low-latency streaming use circumstances
- Capabilities runtime is greatest suited to event-driven, short-lived use circumstances
Under is a extra detailed breakdown of the 2 NiFi runtime choices within the public cloud:
|Runtime choices within the Public Cloud|
|Function||DataFlow Deployments||DataFlow Capabilities|
|Cloud Runtime||NiFi clusters utilizing
|NiFi flows operating on cloud suppliers’ serverless compute companies (AWS Lambda, Azure Capabilities, and Google Cloud Capabilities)|
|Use Case||Use circumstances that want low latency for prime throughput workloads requiring always-running NiFi flows||Occasion pushed, micro-bursty use circumstances with no sub-second latency requirement the place NiFi flows don’t must run constantly|
|Advantages||Auto-scaling Kubernetes clusters for lengthy operating workflows with centralized monitoring||Environment friendly, price optimized, scalable approach to run NiFi flows serverless, permitting builders to deal with enterprise logic|
DataFlow Capabilities gives a brand new, environment friendly approach to run your event-driven Apache NiFi knowledge flows.
With DataFlow Capabilities you possibly can deploy your stream functions in minutes by leveraging the serverless structure of all main public cloud suppliers (AWS, Azure, and Google Cloud Platform), and also you shouldn’t have to fret in regards to the operational overhead of managing and sustaining NiFi stream runtime environments.
To be taught extra on easy methods to arrange and run DataFlow Capabilities in AWS Lambda, Azure Capabilities, and Google Cloud Capabilities, checkout our technical weblog, or take a product tour for a light-weight step-by-step expertise.