Orchestrate the Lakehouse: Asserting Help for Refreshing SQL Queries, Dashboards and Alerts in Databricks Workflows







Databricks Workflows is the extremely obtainable, fully-managed orchestrator for all of your information, analytics, and AI workloads. In the present day, we’re completely satisfied to announce the general public preview of the brand new SQL process sort, which helps you to simply creator, schedule, examine, and function workflows that refresh Databricks SQL queries, dashboards and alerts. You may mix this new process sort with present sturdy assist for notebooks, Python scripts and dbt initiatives to orchestrate the complete energy of the lakehouse.

Actual-world workflows orchestrate many alternative actions throughout the lakehouse. Take into account a workflow that ingests information into Delta Lake and transforms it utilizing a Delta Dwell Desk. As soon as this information is up to date, now you can instruct Databricks to refresh a number of dashboards or execute SQL queries. It’s also possible to run alerts to examine numerous information high quality dimensions and ship notifications to Slack or Groups if specified circumstances are met. Orchestrating these SQL duties contained in the workflow the place information is ready has a number of advantages: not solely is the workflow simpler to motive about, it’s also straightforward to restore and decide up the place it failed.

A workflow refreshing multiple queries and a dashboard
A workflow refreshing a number of queries and a dashboard

With the intention to create a Databricks SQL process, merely go to Databricks Workflows and choose “SQL” from the Kind dropdown:

 Creating a SQL task to refresh a query
Making a SQL process to refresh a question

You additionally profit from the observability and stability advantages that include our jobs scheduler comparable to managing schedules and inspecting earlier runs.

Run details for a SQL task
Run particulars for a SQL process

For pricing and availability in your surroundings, see the Databricks SQL pricing web page (AWS, Azure).


Share this


What companies are using big data analytics

What do companies use big data for? What companies are using big data analytics. There are a multitude of reasons companies use big data, but...

How to use big data in healthcare

What is data quality and why is it important in healthcare? How to use big data in healthcare. In healthcare, data quality is important for...

How to build a big data platform

What is big data platform? How to build a big data platform. A big data platform is a powerful platform used to manage and analyze...

Recent articles

More like this