Prerequisites
Before starting, make sure you have:- Access to a Databricks workspace
- A workspace admin who can create or assign a service principal
- Permission to grant Can use on a SQL warehouse
- Access to Unity Catalog resources (catalogs, schemas, tables) that you want Sidecar to discover
- A catalog owner, metastore admin, or another principal that can grant Unity Catalog privileges
- An account admin or metastore admin if the required
systemschemas are not enabled - Can manage access to each AI/BI dashboard that you want Sidecar to discover
1. Get the host and HTTP path
- Log in to your Databricks workspace.
- Navigate to SQL Warehouses.
- Select the SQL Warehouse you want to use for the integration.
- Click Connection details.
2. Create a service principal and OAuth secret
- Click your username in the top bar and select Settings.
- Click the Identity and access tab.
- Next to Service principals, click Manage.
- Create a new service principal for Sidecar. Creating it from the workspace assigns it to that workspace.
- Click the created service principal, then click the Secrets tab.
- Click Generate secret.
- Set the secret’s lifetime in days (maximum 730 days).
- Click Generate.
- Copy the displayed secret and client ID, then click Done.
Use this one OAuth service principal for Sidecar’s SQL and workspace API access. Do not create a personal access token (PAT) for this integration. If you reuse an account-level service principal, assign it to this workspace before continuing.
3. Grant permissions
Grant your Sidecar service principal (the OAuth identity from step 2) the following before you test or save the integration in Sidecar. In the SQL commands below, replace<principal> with the service principal’s application ID, which is the Client ID you copied in step 2. Keep the surrounding backticks.
Grant SQL warehouse access
In Settings → Identity and access → Service principals → Manage, open the Sidecar service principal and confirm that the Databricks SQL access entitlement is enabled. In SQL Warehouses → your warehouse → Permissions, add the service principal with Can use.Grant access to catalogs Sidecar should scan
For each Unity Catalog catalog Sidecar should discover besidessystem, run the following commands. Replace <catalog> and <principal>.
USE CATALOG on the catalog and grant USE SCHEMA and SELECT only on the schemas or objects Sidecar should scan.
Grant access to required system tables
Sidecar reads the following system data:system.information_schemafor catalogs, schemas, tables, views, and columnssystem.access.table_lineageandsystem.access.column_lineagefor lineagesystem.query.historyfor query activitysystem.billing.usageandsystem.billing.list_pricesfor compute costs
access, query, and billing schemas exist under the system catalog. If one is missing, ask an account admin or metastore admin to enable that system schema.
Then run the following commands. Replace <principal>.
Create Sidecar’s write namespace
Sidecar stores derived metadata in the dedicatedsidecar.default namespace. Ask a metastore admin or a principal with CREATE CATALOG on the metastore to run the following commands before you save the integration. If the sidecar catalog already exists, its owner can create the schema and grant access. Replace <principal>.
Share AI/BI dashboards with Sidecar
For each AI/BI dashboard that Sidecar should discover:- Open the dashboard and click Share.
- Add the Sidecar service principal.
- Grant Can view.
4. Configure the integration
Enter the following information in the Sidecar integration setup page:
After entering the credentials:
- Click Test Connection.
- Click Save to finish the integration.
Result
After the integration is complete, Sidecar will be able to:- Connect to your Databricks SQL Warehouse
- Discover catalogs, schemas, and tables
- Discover shared AI/BI dashboards
- Query metadata from Unity Catalog
- Use
systemcatalog data for query history, lineage, and billing - Store derived metadata in
sidecar.default