---
title: "origin"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{origin}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```
In contrast to other programming languages, R has no widely established and
undisputed style guide (e.g. PEP 8 for Python). As a data scientist, I helped
to establish a company wide R style guide. While it mainly relies on the
[tidyverse style guide](https://style.tidyverse.org/), we generally decided
to be more explicit in our coding practice. This includes that we always refer
to functions from non-native R packages with the double colon operator `::`. While it
is relatively easy to establish such a convention in new projects, it is
challenging to adapt ongoing projects and legacy code. `origin` allows for much
faster conversions of both legacy code as well as currently written code.
## Purpose of `origin`
The main purpose is to add `pkg::` to an R function call, i.e. it changes
code like this:
<img src="vignette_quickview.gif" width="650px" />
## Usage of `origin`
In general, you can either originize some selected text (more on that later
in Addins), a whole script, or a all scripts in a specific folder, e.g. your
project folder. There is a specifically designed function for each purpose yet
they all share the same options. Therefore, only `originize_file()` is
extensively presented as an example with its default options.
### Code Usage
```{r, eval=FALSE}
originize_file(file = "testscript.R",
pkgs = .packages(),
overwrite = TRUE,
ask_before_applying_changes = TRUE,
ignore_comments = TRUE,
check_conflicts = TRUE,
add_base_packages = FALSE,
check_base_conflicts = TRUE,
check_local_conflicts = TRUE,
excluded_functions = list(dplyr = c("%>%", "across"),
data.table = c(":=", "%like%"),
# exclude from all packages:
c("first", "last")),
verbose = TRUE,
use_markers = TRUE)
```
### Common Arguments
* `pkgs`: which packages to check for functions used in the code (see **Considered Packages**).
The default are all packages attached via `library` or `require`
* `overwrite`: actually insert `pkg::` into the code. Otherwise,
logging shows only what *would* happen. Note that `ask_before_applying_changes`
still allows to keep control over your code before `origin` changes anything.
* `ask_before_applying_changes`: whether changes should be applied
immediately or the user must approve them first.
* `check_conflicts`: should `origin` check for potential
namespace conflicts, i.e. a used function is defined in more than one considered
package. User input is required to solve the issue.
Strongly encouraged to be set to `TRUE`.
* `add_base_packages`: should base packages also be added, e.g. `base::sum()`.
* `check_base_conflicts`: Should origin also check for conflicts
with base R functions.
* `check_local_conflicts`: Should origin also check for conflicts
with locally defined functions anywhere in your project? Note that it does not
check the environment but solely parses files and scans them for function definitions.
* `excluded_functions`: a (named) list of functions to exclude from checking.
* `verbose`: some sort of logging is performed, either in the
console or via the markers tab in RStudio.
* `use_markers`: whether to use the Markers tab in RStudio.
* `filetypes`: Which filetypes to consider.
Currently, origin supports .R, .Rmd, and .Qmd (Quarto) files.
### Addins
Besides using regular R functions to originize files, there are also useful
addins delivered with `origin`. These addins are designed to be used on-the-fly
while coding. You can either originize selected text, the currently opened file,
or all scripts in the currently opened project. However, to have as much control
as when using functions, each function argument corresponds to an option that
can be set and used inside the addins, e.g.
```{r, eval=FALSE}
options(origin.pkgs = c("dplyr", "data.table"),
origin.overwrite = TRUE)
```
Actually, most function arguments of `origin` first check whether an option has
been declared and uses the assigned value as its default. This allows for equal
outcomes regardless whether you use the addin or a function sequentially.
### Safety Measures
Since `origin` changes files on disk, it is very important that the user has
full control over what happens and user input is required before critical steps.
#### Logging
Most importantly, the user must be aware of what the originized file(s)
would look like. For this, all changes **and** potential missed changes
are presented, either in the Markers tab (recommended) or in the console.
<img src="markers_logging.png" width="650px" />
<img src="console_logging.png" width="650px" />
* <span style="color: #00c7cc">insertion</span>: `pkg::` is inserted prior to a function
* <span style="color: #ff0000">missing</span>: an object that has the same name as a function
but not undoubtedly used as a function. In R it is usually no problem
to have variables that name like functions (data or df are popular examples).
While it is always clear when a function is directly used as one, functions
can also be arguments of other functions, most famously in functional programming
like the *apply family or purrr. `origin` highlights such cases in
the logging output.
* <span style="color: #ffa500">infix</span>: functions like `%>%` are exported by packages but cannot be called
with the `pkg::fun()` convention. Such functions are highlighted by default
to point the user that these stem from a package. When using
dplyr-style code, consider to exclude the pipe-operator via
`exclude_functions`.
#### Same Function Name in Multiple Packages
Due to the variety of R packages, function names must not be unique across all
packages out there. By default, R masks priorly imported functions by those
imported afterwards. `origin` mimics this rule by applying a higher priority
to those packages that are listed first. In case there is a conflict regarding
a **used** function, These functions are listed along with the packages from
which they stem.
```{asis, eval=FALSE}
Used functions in mutliple Packages!
filter: dplyr, stats
first: data.table, dplyr
Order in which relevant packges are evaluated:
data.table >> dplyr >> stats
Do you want to proceed?
1: YES
2: NO
```
#### Custom Functions Mask Exported Functions
As packages mask each others functions, the same applies to locally
defined custom functions. In case you defined your own `last` function
in your project, `origin` should **not** add `dplyr::` to it. Therefore,
your project is searched for function definitions and local functions
have higher priority than those exported by packages. Note that,
depending on the project size, this process can take quite some time.
In this case, set the argument/option `path_to_local_functions` to a
subdirectory or `check_local_conflicts` to `FALSE` to skip this feature.
```{asis, eval=FALSE}
Locally defined and used functions mask exported functions from packages
last: dplyr
Local functions have higher priority. In case you want to use an
exported version of a function listed above set pkg::fun manually
Got it?
1: YES
2: NO
3: Show files
```
#### Many Files Selected
When `originizing` a complete folder or project, many R scripts might be checked.
In case the user is unaware that there are many files in the selected folder,
resulting in a long run time of `origin`,
a warning is triggered and user input is required.
```{asis, eval=FALSE}
You are about to originize 99 files.
Proceed?
1: YES
2: NO
3: Show files
```
#### Final Check
Before the proposed changes are applied eventually, a final user input
is required.
```{asis, eval=FALSE}
Happy with the result? 😀
1: YES
2: NO
```
## Discussion
Whether or not to add `pkg::` to each (imported) function is a [controversial](https://stackoverflow.com/q/4372145/8107362)
[issue](https://stackoverflow.com/q/23232791/8107362) in the R community. While the tidyverse style guide does not mention explicit namespacing, [R Packages](https://r-pkgs.org) and the [Google R style guide](https://google.github.io/styleguide/Rguide.html#qualifying-namespaces) are in favor of it.
Pros
+ very explicit
+ completely avoid namespace conflicts
+ no need to attach the complete namespace of a package
+ keep track of which function belongs to which package
Cons
- (minimal) performance issue
- more writing required
- longer code
- infix functions like `%>%` cannot be called via `magrittr::%>%`
and workarounds are still required here. Either use
```
library(magrittr, include.only = "%>%")
`%>%` <- magrittr::`%>%`
```
- calling `library()` on top of a script clearly indicates which packages are
needed. A not yet installed package throws an error right away, not until
a function cannot be found later in the script. However, one can use
the `include_only` argument and set it to `NULL`. No functions are attached
into the search list then.
```
library(magrittr, include_only = NULL)
```
## Check Package Usage since origin 1.0.0
As a new feature origin origin exports the function `check_pkg_usage`.
Given you take over a project or just built a huge barrage of `library` calls
over time. Which of those are actually still needed.
Just run all those `library(...)` calls and then call `check_pkg_usage()`
### Interpreting the Output of check_pkg_usage
```
== Package Usage Report ================================================
-- Used Packages: 2 ----------------------------------------------------
v data.table
v testthat
-- Unused Packages: 1 --------------------------------------------------
i dplyr
-- Possible Namespace Conflicts: 1 -----------------------------------
x last data.table >> dplyr
-- Specifically (`pkg::fun()`) further used Packages: 2 ----------------
i purrr
-- Functions with unknown origin: 1 ------------------------------------
x map
```
The output shows
- we had attached 3 packages: {data.table}, {testthat}, and {dplyr}
- functions from {data.table} and {testthat} are used
- {dplyr} functions are not used
- a namespace conflict for the function `last` between {data.table} and {dplyr}
- additionally, we use purrr:: at some occasions
- we use the `map()` function that is not exported from {data.table},
{testthat}, or {dplyr}. Note that `map` is exported from {purrr} that is
used elsewhere but here our code would fail since {purrr} is not attached
and `map cannot be found.
A markers Tab shows all unknown functions and unknown packages that are
used explicitly
### Interpreting the Result of check_pkg_usage
Having a closer look into `result`
```
as.data.frame(result)
#> pkg fun n_calls namespaced conflict conflict_pkgs
#> 1 base %in% 53 FALSE FALSE NA
#> 2 base .packages 8 FALSE FALSE NA
#> 3 base Filter 3 FALSE FALSE NA
#> 4 base Map 1 FALSE FALSE NA
#> 5 base Reduce 5 FALSE FALSE NA
#> ...
```
It first shows a lot of base functions. That is, even though their are not explicitly attached, base r packages are always attached. The print output does not show them but if you want to deep dive into the functions that are used in the project they are available
```
#> pkg fun n_calls namespaced conflict conflict_pkgs
#> 110 data.table %like% 10 FALSE FALSE NA
#> 111 data.table := 1 FALSE FALSE NA
#> 112 data.table CJ 1 FALSE FALSE NA
#> 113 data.table as.data.table 1 FALSE FALSE NA
#> 114 data.table as.data.table 3 TRUE FALSE NA
#> 115 data.table last 2 TRUE FALSE NA
#> 116 data.table last 1 FALSE TRUE dplyr
```
Going further, there are a bunch of {data.table} functions that have been used.
Some are listed twice because they were sometimes called via `data.table::`,
sometimes not. Furthermore, `last` is marked with `conflict = TRUE`.
This is because {dplyr} does export a `last` function, as well. However, since
{data.table} has the higher priority than {dplyr} in this project, {origin}
considers it as an {data.table} function. Note that if a function is
namespaced via `::`, no conflict is given.
Finally, at the end of the output:
```
#> pkg fun n_calls namespaced conflict conflict_pkgs
#> 219 <NA> map 1 FALSE NA NA
#> 220 dplyr <NA> 0 NA NA NA
```
Here we see the `map` function that would not be assigned to one of the given packages
and the {dplyr} package that has not been used.
### Final Remarks
Locally defined functions are also detected via parsing. These also do have a
higher priority than exported function from other packages.