Fix workspace owners unable to access Databricks workspace

[marrobi]

Description

After deploying the Databricks workspace service, workspace owners and researchers cannot access the Databricks workspace through the Azure portal. They receive an access denied error when trying to navigate to the workspace.

Root Cause

Azure Databricks uses Microsoft Entra ID SSO for authentication with Just-in-Time (JIT) user provisioning. However, users also need an Azure RBAC role on the Databricks workspace resource for the Azure portal to allow navigation to the workspace.

Currently, the Databricks workspace service does not assign any Azure RBAC roles to workspace owners or researchers, preventing portal access.

Changes

• Add roles.tf with Azure RBAC Contributor role assignments for workspace owners and researchers
• Add workspace_owners_group_id and workspace_researchers_group_id variables
• Move workspace group parameters to install pipeline only in template_schema.json
• Remove unused Databricks Terraform provider
• Upgrade AzureRM provider to 4.14.0

References

• Azure Databricks authentication and authorization

Fixes #4854

[github-actions]

Unit Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit 28d4445.

[marrobi]

Need to check with a custom role with minimal permissions:

{     "id": "/providers/Microsoft.Authorization/roleDefinitions/900b4ced-1ab8-4538-9c19-e2a182a8ee24",     "properties": {         "roleName": "Reseracher_test",         "description": "",         "assignableScopes": [             "/providers/Microsoft.Management/managementGroups/dfad657e-4803-4ebe-b30f-280bd491b655"         ],         "permissions": [             {                 "actions": [                     "*/read",                     "Microsoft.Databricks/register/action",                     "Microsoft.Databricks/accessConnectors/read",                     "Microsoft.Databricks/locations/getNetworkPolicies/action",                     "Microsoft.Databricks/locations/operationstatuses/read",                     "Microsoft.Databricks/workspaces/read",                     "Microsoft.Databricks/workspaces/refreshPermissions/action",                     "Microsoft.Databricks/workspaces/refreshWorkspaces/action",                     "Microsoft.Databricks/workspaces/dbWorkspaces/write",                     "Microsoft.Databricks/workspaces/outboundNetworkDependenciesEndpoints/read",                     "Microsoft.Databricks/workspaces/privateEndpointConnectionProxies/read",                     "Microsoft.Databricks/workspaces/privateEndpointConnectionProxies/validate/action",                     "Microsoft.Databricks/workspaces/privateLinkResources/read",                     "Microsoft.Databricks/workspaces/privateEndpointConnections/read",                     "Microsoft.Databricks/workspaces/virtualNetworkPeerings/read",                     "Microsoft.Databricks/operations/read"                 ],                 "notActions": [],                 "dataActions": [],                 "notDataActions": []             }         ]     } }

Not sure if need:

Write	Initialize Databricks workspace	Initializes the Databricks workspace (internal only)