Connecting to Databricks

‍

¹ Create a Connection

On the Data page, click New Connection and choose Databricks as the type. Enter the fourcredentials you gathered, then save.

• Token — your Databricks access token
• Endpoint — your workspace URL
• AWS Region — your workspace region
• Catalog — the Unity Catalog name

‍

² Create a Dataset

Back on the Data page, click New, select source Connection, and pick the Databricks connection you just created. A Data Set points at a specific table and can carry its own query and refresh schedule.

‍

‍

³ Select a table, or write a query

Choose how Ridge reads from the catalog. A direct table reference is simplest; custom SQL lets you filter and aggregate at the source.

‍

Option A — Table reference

SELECT
  customer_id,
  SUM(amount) AS total_amount
FROM main.sales.transactions
WHERE year = 2024
GROUP BY customer_id

‍

Option B — Custom SQL

SELECT
  customer_id,
  SUM(amount) AS total_amount
FROM main.sales.transactions
WHERE year = 2024
GROUP BY customer_id

‍

⁴ Schedule ingestion

Set an ingestion schedule with standard cron syntax, then save. Ridge runs your query againstUnity Catalog and stores the results, ready for analysis.

‍

‍

What happens on each run
‍

1. Ridge executes your query against Databricks Unity Catalog
2. Results are exported to Parquet format
3. Data is stored in Ridge's object storage (R2)
4. The Data Set becomes available for visualization and the Data Agent