---
title: "Incorporating experimental designs"
author: "Mike Blazanin"
output: 
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 4
vignette: >
  %\VignetteIndexEntry{Incorporating experimental designs}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r global options, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
knitr::opts_knit$set(root.dir = tempdir())
```

# Where are we so far?

1. Introduction: `vignette("gc01_gcplyr")`
2. Importing and reshaping data: `vignette("gc02_import_reshape")`
3. **Incorporating experimental designs:** `vignette("gc03_incorporate_designs")`
4. Pre-processing and plotting your data: `vignette("gc04_preprocess_plot")`
5. Processing your data: `vignette("gc05_process")`
6. Analyzing your data: `vignette("gc06_analyze")`
7. Dealing with noise: `vignette("gc07_noise")`
8. Best practices and other tips: `vignette("gc08_conclusion")`
9. Working with multiple plates: `vignette("gc09_multiple_plates")`
10. Using make_design to generate experimental designs: `vignette("gc10_using_make_design")`

So far, we've imported and transformed our measures data into `R`. Now we're going to address how to incorporate our experimental design.

If you haven't already, load the necessary packages.

```{r setup}
library(gcplyr)
```

# Including design elements
We often want to combine our data with information about the experimental design. `gcplyr` enables incorporation of design elements in two ways:

1. Designs can be imported from files
2. Designs can be generated in `R` using `make_design`

If you're interested in generating your designs in `R`, see `vignette("gc10_using_make_design")`



When reading design elements from files, `gcplyr` can read block-shaped or tidy-shaped design files:

* If design files are block-shaped, they can be read with `import_blockdesigns`
* If design files are tidy-shaped, they can simply be read with `read_tidys`

# Importing block-shaped design files
To import block-shaped design files, use `import_blockdesigns`, which will return a tidy-shaped designs data frame (or list of data frames).

`import_blockdesigns` only requires a list of filenames (or relative file paths) and will return a data.frame (or list of data frames) in a **tidy format** that you can save in R.

## A basic example


Let's look at an example. First, we need to create an example file for the sake of this tutorial (normally you'd create this file in Excel)

```{r}
make_example(vignette = 3, example = 1, dir = ".")
```

Now let's take a look at what the file looks like:

```{r}
print_df(read.csv("mydesign.csv", header = FALSE, colClasses = "character"))
```

Here we can see that our design has Treatment 1 on the left-hand side of the plate (wells in columns 1 through 6), and Treatment 2 on the right-hand side of the plate (wells in columns 7 through 12). Let's import this design using `import_blockdesigns`, saving it with the column name `Treatment_numbers`.

```{r}
my_design <- import_blockdesigns(files = "mydesign.csv", 
                                 block_names = "Treatment_numbers")
head(my_design, 20)
```

## Importing multiple block-shaped design elements


What do you do if you have multiple designs? For instance, what if you have several strains each in several treatments? In that case, you have two options:

1. Save each design component as a separate file, or in separate blocks within a file
2. Save the design components pasted together in a single file

Regardless of which option you use, you can then import them all in one go with `import_blockdesigns`.

### Importing multiple block-shaped designs in separate files

First, let's create both our example designs files. Again, just imagine that you've created these files in Excel.

```{r}
make_example(vignette = 3, example = 1, dir = ".")
make_example(vignette = 3, example = 2, dir = ".")
```

Now let's take a look at what these files looks like:

```{r}
print_df(read.csv("mydesign.csv", header = FALSE, colClasses = "character"))
print_df(read.csv("mydesign2.csv", header = FALSE, colClasses = "character"))
```

As before, we have Treatment 1 on the left-hand side, and Treatment 2 on the right-hand side. In addition, we now also have Strain A in the first two rows, Strain B in the next two rows, and so on.

Let's now import both designs using `import_blockdesigns`, saving them to columns named `Treatment_numbers` and `Strain_letters`.

```{r}
my_design <- 
  import_blockdesigns(files = c("mydesign.csv", "mydesign2.csv"), 
                      block_names = c("Treatment_numbers", "Strain_letters"))
head(my_design, 20)
```

### Importing multiple separated block-shaped designs in one file

If you have your blocks separated but saved in the same file, you simply specify the location of each block within the file:

```{r}
make_example(vignette = 3, example = 3, dir = ".")

#Print what the file looks like
print_df(read.csv("mydesign_sep.csv", header = FALSE, colClasses = "character"))

#Read in the designs
my_design <- 
  import_blockdesigns(files = c("mydesign_sep.csv"), 
                      block_names = c("Treatment_numbers", "Strain_letters"),
                      startrow = c(1, 11), endrow = c(9, 19))
head(my_design, 20)
```

### Importing multiple pasted block-shaped designs

Alternative to saving your designs separated, often it may be easiest to save all the design information into a single block, separating the distinct components of the design with some character.

To demonstrate this, first let's create our example designs file. Again, just imagine that you've created this file in Excel.

```{r}
make_example(vignette = 3, example = 4, dir = ".")
```

Now let's take a look at what the file looks like:

```{r}
print_df(read.csv("mydesign_pasted.csv", header = FALSE, colClasses = "character")[, 1:10])
```

As before, we have Treatment 1 on the left-hand side, and Treatment 2 on the right-hand side, with Strain A in the first two rows, Strain B in the next two rows, and so on. However, this information is now pasted together, with "_" as the separating string (you can use any string as a separator).

To import this design with `import_blockdesigns`, we simply need to specify the `sep` string, as well as the output column names. Since the designs have been pasted together, the column names will result from splitting the designs apart. The easiest way to specify these split column names is to use the `into` argument passed to `separate_tidy` [if `into` is not specified, `import_blockdesigns` will attempt to split the `block_names` (either specified or inferred) with `sep` to generate the output column names].

```{r}
my_design <- 
  import_blockdesigns(files = "mydesign_pasted.csv", 
                      into = c("Treatment_numbers", "Strain_letters"),
                      sep = "_")
head(my_design, 20)
```

# Importing tidy-shaped design files
You can import tidy-shaped designs with `read_tidys`.

`read_tidys` only requires a filename (or vector of filenames, or relative file paths) and will return a `data.frame` (or list of data.frames) that you can save in R.

Once these design elements have been read into the `R` environment, they are ready to merge.


# Merging growth curve data with designs
Once we have both our design and data in `R` and tidy-shaped, we can merge them using `merge_dfs`.

To demonstrate this, we'll use the data in the `example_widedata_noiseless` dataset that is included with `gcplyr`, and which was the source for our previous examples with `import_blockmeasures` and `read_wides`.

In the `example_widedata_noiseless` dataset, we have 48 different bacterial strains. The left side of the plate has all 48 strains in a single well each, and the right side of the plate also has all 48 strains in a single well each:

Row names | Column 1   | ... | Column 6   | Column 7   | ... | Column 12
--------- | --------   | --- | --------   | --------   | --- | --------
Row A     | Strain #1  | ... | Strain #6  | Strain #1  | ... | Strain #6
Row B     | Strain #7  | ... | Strain #12 | Strain #7  | ... | Strain #12
...       | ...        | ... | ...        | ...        | ... |  ...
Row G     | Strain #37 | ... | Strain #42 | Strain #37 | ... |  Strain #42
Row H     | Strain #43 | ... | Strain #48 | Strain #43 | ... |  Strain #48

Then, on the right hand side of the plate a phage was also inoculated (while the left hand side remained bacteria-only):

Row names |Column 1 | ... | Column 6 | Column 7    | ... | Column 12
--------- |-------- | --- | -------- | --------    | --- | --------
Row A     |No Phage | ... | No Phage | Phage Added | ... | Phage Added
Row B     |No Phage | ... | No Phage | Phage Added | ... | Phage Added
...       |...      | ... | ...      | ...         | ... |  ... 
Row G     |No Phage | ... | No Phage | Phage Added | ... |  Phage Added
Row H     |No Phage | ... | No Phage | Phage Added | ... |  Phage Added

Let's transform the `example_widedata_noiseless` to tidy-shaped.

```{r}
example_tidydata <- trans_wide_to_tidy(example_widedata_noiseless,
                                       id_cols = "Time")
```

`gcplyr` also includes the design for this data for easy use:

```{r}
example_design <- example_design_tidy
head(example_design_tidy)
```

Now that we have our data and designs tidy-shaped, we merge the two using `merge_dfs`, saving the result to `ex_dat_mrg`, short for example_data_merged. `merge_dfs` merges using columns with the same name between the two data.frames.

```{r}
ex_dat_mrg <- merge_dfs(example_tidydata, example_design)

head(ex_dat_mrg)
```

# What's next?

Now that you've merged your data and designs, you can pre-process and plot your data

1. Introduction: `vignette("gc01_gcplyr")`
2. Importing and reshaping data: `vignette("gc02_import_reshape")`
3. Incorporating experimental designs: `vignette("gc03_incorporate_designs")`
4. **Pre-processing and plotting your data: `vignette("gc04_preprocess_plot")`**
5. Processing your data: `vignette("gc05_process")`
6. Analyzing your data: `vignette("gc06_analyze")`
7. Dealing with noise: `vignette("gc07_noise")`
8. Best practices and other tips: `vignette("gc08_conclusion")`
9. Working with multiple plates: `vignette("gc09_multiple_plates")`
10. Using make_design to generate experimental designs: `vignette("gc10_using_make_design")`