---
title: "Databricks REPL"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Databricks REPL}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
eval = FALSE
)
library(brickster)
```
# Running Code Remotely
`{brickster}` provides mechanisms to run code against Databricks, below is an overview of the available of those in the package:
+--------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------------------------------------------------------------+
| Method | Compatible Compute | Notes |
+====================================================================================================================+=======================+===============================================================================+
| [SQL execution API](https://docs.databricks.com/api/workspace/statementexecution)\ | SQL Warehouse | Lower level functions that align 1:1 with API endpoints |
| ([`db_sql_exec_*`](https://databrickslabs.github.io/brickster/reference/index.html#sql-statement-execution)) | | |
+--------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------------------------------------------------------------+
| Command execution context manager\ | Clusters\ | Higher level `R6` class for command execution contexts. |
| (`db_context_manager`) | (Shared, Single User) | |
+--------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------------------------------------------------------------+
| [Command execution API](https://docs.databricks.com/api/workspace/commandexecution)\ | Clusters\ | Lower level functions that align 1:1 with API endpoints |
| ([`db_context_*`](https://databrickslabs.github.io/brickster/reference/index.html#execution-contexts)) | (Shared, Single User) | |
+--------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------------------------------------------------------------+
| [Databricks SQL Connector](https://docs.databricks.com/en/dev-tools/python-sql-connector.html) via `{reticulate}`\ | SQL Warehouse | Exposes `databricks.sql` connector to R. |
| ([`db_sql_client`](https://databrickslabs.github.io/brickster/reference/index.html#sql-connector)) | | |
+--------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------------------------------------------------------------+
| Databricks REPL\ | Clusters\ | Supports all notebook languages, R is only supported on single user clusters. |
| (`db_repl`) | (Shared, Single User) | |
+--------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------------------------------------------------------------+
Databricks [REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) (`db_repl()`) will be the focus of this article.
# What is the Databricks REPL?
The REPL temporarily connects the existing R console to a Databricks cluster (via [command execution APIs](https://docs.databricks.com/api/workspace/commandexecution)) and allows code in all supported languages to be sent interactively - as if it were running locally.
## Getting Started
Using the REPL is simple, to start just provide `cluster_id`:
```{r}
# start REPL
db_repl(cluster_id = "<insert cluster id>")
```
The REPL will check the clusters state and start the cluster if inactive. The default language is `R`.
After successfully connecting to the cluster you can run commands against the remote compute from the local session.
## Switching Languages
The REPL has a shortcut you can enter `:<language>` to change the active language. You can change between the following languages:
+---------------------------------+-----------------------------------+
| Language | Shortcut |
+=================================+===================================+
| R | `:r` |
+---------------------------------+-----------------------------------+
| Python | `:py` |
+---------------------------------+-----------------------------------+
| SQL | `:sql` |
+---------------------------------+-----------------------------------+
| Scala | `:scala` |
+---------------------------------+-----------------------------------+
| Shell | `:sh` |
+---------------------------------+-----------------------------------+
When you change between languages all variables should persist unless REPL is exited.
## RStudio Addins
There are three addins included with `{brickster}` for use with the REPL:
1. Send selection to console
2. Send current document to console
3. Send R script to console
It's recommended that (1) and (2) be bound to a keyboard shortcut if using the REPL frequently ('Tools' \> 'Modify Keyboard Shortcuts...' \> 'Search for `brickster`').
These addins behave slightly different to the standard shortcuts as they ensure that commands that span multiple lines are handled correctly via the REPL.
## Limitations
- Development environments (e.g. RStudio, Positron) won't display variables from the remote contexts in the environment pane
- HTML content will only render for Python, `{htmlwidgets}` rendering is restricted due to [notebook limitations that require a workaround](https://docs.databricks.com/en/visualizations/htmlwidgets.html) currently
- Not designed to work with interactive serverless compute
- Cannot persist or recover sessions