diff --git a/01_overview-getting-familiar-with-torch.Rmd b/01_overview-getting-familiar-with-torch.Rmd index d155f29..a19a892 100644 --- a/01_overview-getting-familiar-with-torch.Rmd +++ b/01_overview-getting-familiar-with-torch.Rmd @@ -4,6 +4,13 @@ - THESE ARE NICE TO HAVE BUT NOT ABSOLUTELY NECESSARY +## Setup {-} + +```{r install-torch} +Sys.setenv(TORCH_INSTALL = 1) +library(torch) +``` + ## SLIDE 1 {-} - ADD SLIDES AS SECTIONS (`##`). diff --git a/03_tensors.Rmd b/03_tensors.Rmd index 3d16e0d..0cfcf64 100644 --- a/03_tensors.Rmd +++ b/03_tensors.Rmd @@ -2,12 +2,406 @@ **Learning objectives:** -- THESE ARE NICE TO HAVE BUT NOT ABSOLUTELY NECESSARY +Learn how to: + +- Create and modify tensors +- Access parts of tensors +- Apply operations to tensors + +## What are tensors? {-} + +From the book: + +> "... *tensors* are 'just' multi-dimensional arrays optimized for fast computation - +not on the CPU only but also on specialized devices such as GPUs and TPUs. + +Load libraries: + +```{r message = FALSE, warning=FALSE} +library(torch) +library(dplyr) +library(ggplot2) +``` + +Create a tensor: + +```{r} +t1 <- torch_tensor(2) +t1$shape +``` + +Parameters of `torch_tensor()` (see help file for this function): + +* `data` +* `dtype` +* `device` +* `requires_grad` +* `pin_memory` + +Look at the attributes of the tensor: + +```{r} +t1$dtype +t1$device +t1$shape +summary(t1) +``` + +Change attributes: + +```{r} +# to integer: +t2 <- t1$to(dtype = torch_int()) +t2$dtype +``` + +```{r eval=FALSE} +# utilize GPU: +t2 <- t1$to(device = "cuda") +t2$device +``` + + +## Creating tensors + +### Tensors from values + +**Defaults**: long integer type and CPU device + +Let's test this: + +```{r} +torch_tensor(1:2) +``` +In the example above, the type is long integer. + +```{r} +torch_tensor(c(1,2)) +``` + +In this example, the type is float instead of a long integer. +My guess is that the difference is due to the difference in data types of `1:2` and `c(1,2)`: + +```{r} +class(c(1,2)) +class(1:2) +``` + +Explicitly set the type and device: + +```{r eval = FALSE} +torch_tensor(1:5, dtype = torch_float(), device = "cuda") +``` + +Two-dimensional tensor: + +```{r} +torch_tensor(matrix(1:9, ncol = 3, byrow = TRUE)) +``` + +Higher dimensions: + +```{r} +torch_tensor(array(1:24, dim = c(4,3,2))) +``` + +### Tensors from specifications + +```{r} +#normal distribution: +torch_randn(3, 3) + +#unfiorm distribution +torch_rand(3, 3) +``` + +```{r} +torch_zeros(2,2) +torch_ones(3,3) +torch_eye(3,3) +torch_diag(c(1,2,3)) +``` + +See full list at https://torch.mlverse.org/docs/reference/#tensor-creation-utilities. + +### Tensors from datasets + +We'll look at the `presidential` dataset from the `ggplot2` package: + +```{r} +data(presidential) # from ggplot2 +presidential |> glimpse() +``` + +```{r} +presidential |> + mutate(name = as.numeric(factor(name)), + start = as.numeric(start), + end = as.numeric(end), + party = as.numeric(factor(party))) |> + as.matrix() |> + torch_tensor() |> + print(5) +``` + +If your data contains `NA`'s, you'll need to convert them before training a neural network +(topic to be covered later). + +## Operations on Tensors + +Define two tensors: + +```{r} +t1 <- torch_tensor(c(1,2)) +t2 <- torch_tensor(c(3,4)) +``` + +Add: + +```{r} +torch_add(t1, t2) # t1 is NOT modified +t1$add(t2) # t1 is NOT modified +t1$add_(t2) # t1 IS modified +t1 +``` + +In general: + +* underscore appended to operation indicates modification in-place. +* torch does not distinguish between row and column vectors. +* Other examples of operations: `torch_t()`, `torch_dot()`, `torch_matmul()`, and `torch_multiply()`. +See https://torch.mlverse.org/docs/reference/#mathematical-operations-on-tensors. + +Another example: + +```{r} +t1 <- torch_tensor(1:3) +t2 <- torch_tensor(4:6) +``` + +```{r} +t1$dot(t2) +``` + +### Summary operations + +Create a matrix and a tensor using outer products: + +```{r} +m <- outer(1:3, 1:6) +t <- torch_outer(torch_tensor(1:3), torch_tensor(1:6)) +``` + +```{r} +apply(m, 1, sum) # row sums (of R matrix) +t$sum(dim = 2) # row sums (of tensor) +``` + +* In R, we *group* by row (dimension 1) for row summaries and by columns (dimension 2) for column summaries +* In `torch, we *collapse* the columns (dimension 2) for row summaries and the rows (dimension 1) for column summaries. + +Time series example: Two features collected three times for four individuals. + +* Dimension 1: Runs over individuals +* Dimension 2: Runs over points in time +* Dimension 3: Runs over features + +```{r} +t <- torch_randn(4, 3, 2) +t +``` + +Obtain averages of features, independent of subject (dimension 1) and time (dimension 2): + +```{r} +t$mean(dim = c(1, 2)) +``` + + +## Accessing Parts of a Tensor (indexing and slicing) + +Indexing in `torch` is 1-based. Very similar to functionality in R. + +Example: + +```{r} +t <- torch_tensor(matrix(1:9, ncol = 3, byrow = TRUE)) +t +t[1, ] # row 1. +t[1, , drop = FALSE] # Same as above except that dimensionality is preserved +``` + +Slicing example: + +```{r} +t <- torch_rand(3, 3, 3) +t[1:2, 2:3, c(1, 3)] +``` + +### Beyond R + +Access last element: + +```{r} +t <- torch_tensor(matrix(1:4, ncol = 2, byrow = TRUE)) +t[-1, -1] +``` + +```{r} +t <- torch_tensor(1:4) +t[-1] +``` + +Compare to R: + +```{r} +# matrix: +m <- matrix(1:4, ncol = 2, byrow = TRUE) +m[-1, -1] + +# vector: +m <- 1:4 +m[-1] +``` + +Step pattern: + +```{r} +t <- torch_tensor(matrix(1:20, ncol = 10, byrow = TRUE)) +t +t[ , 1:8:2] # every other value in columns 1 through 8 +``` + +Use `..` to designate all dimensions not explicitly referenced. + +```{r} +t2 <- torch_randn(2, 2, 2) +t2 +t2[2, ..] # 2nd element of the 1st dimension +``` + +## Reshaping Tensors + +Two methods: + +* `view`: Zero-copy reshape (by changing tensor metadata). Will fail if zero-copy reshape is not possible. +* `reshape`: Uses zero-copy reshape (via metadata) when possible. If not, then will make a copy. + +```{r} +t <- torch_tensor(matrix(1:15, nrow = 3, byrow = TRUE)) +t +t$stride() +``` + +`$stride()` tells us the jump necessary to go from one element to the next in a single dimension (e.g. the stride necessary to get from one row to the next). + +Next, change shape of `t` using `view()`: + +```{r} +t2 <- t$view(c(5,3)) +t2 +t2$stride() +``` + + +Check memory location: + +```{r} +t$storage()$data_ptr() +t2$storage()$data_ptr() # same location +``` + + +`view()` vs `reshape()`: + +When two operations that change the stride are done in sequence, this will likely fail. +Below is an example of transpose, `t()` followed by `view()`. + +```{r eval = FALSE} +t$t()$view(15) # error +``` + +However, `reshape()` makes a copy of the underlying data and does not fail. + +```{r} +t3 <- t$t()$reshape(15) # no error +t3 +``` + + +Now the memory locations of t3 and the original data are different: + +```{r} +t$storage()$data_ptr() +t3$storage()$data_ptr() # different location +``` + +Two additional functions that are zero-copy operations: + +* `squeeze()`: Removes singleton dimensions +* `unsqueeze()`: Adds a singleton dimension + +```{r} +t <- torch_rand(3) # one dimensional tensor of shape 3 +t +t2 <- t$unsqueeze(1) # two dimensional tensor of shape {1,3} +t2 +``` +Inspect location: + +```{r} +t$storage()$data_ptr() +t2$storage()$data_ptr() # same location +``` + + +## Broadcasting + +Example: Want to add two tensors of shape 3x7x1 and 1x5 + +``` +t1 shape: 3 7 1 +t2 shape: 1 5 +``` + +Broadcast (from right to left): + +``` +t1 shape: 3 7 5 +t2 shape: 7 5 +``` + +Next, virtual expansion: + +``` +t1 shape: 3 7 5 +t2 shape: 1 7 5 +``` + +And, broadcast again: + +``` +t1 shape: 3 7 5 +t2 shape: 3 7 5 +``` + +Let's see it in action: + +```{r} +t1 <- torch_ones(3, 7, 1) +t2 <- torch_zeros(1, 5) +torch_add(t1, t2) +``` + +The following will NOT work: + +```{r eval = FALSE} +t1 <- torch_ones(3, 7, 1) +t2 <- torch_zeros(6, 5) +torch_add(t1, t2) +``` -## SLIDE 1 {-} -- ADD SLIDES AS SECTIONS (`##`). -- TRY TO KEEP THEM RELATIVELY SLIDE-LIKE; THESE ARE NOTES, NOT THE BOOK ITSELF. ## Meeting Videos {-} diff --git a/DESCRIPTION b/DESCRIPTION index 7dd5b88..f9dc574 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -9,5 +9,8 @@ Depends: R (>= 3.1.0) Imports: bookdown, - rmarkdown + dplyr, + ggplot2, + rmarkdown, + torch Encoding: UTF-8