Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting cholesky representations to work in TF2 #642

Closed
njtierney opened this issue Jul 26, 2024 · 1 comment
Closed

Getting cholesky representations to work in TF2 #642

njtierney opened this issue Jul 26, 2024 · 1 comment
Milestone

Comments

@njtierney
Copy link
Collaborator

Current implementation in #637 removes representation approach. This is not as good, but right now I am really struggling to unfurl this problem and I'd rather get something that works. Here's a demo/comparison between branches

# In patch branch:
library(greta)
#> 
#> Attaching package: 'greta'
#> The following objects are masked from 'package:stats':
#> 
#>     binomial, cov2cor, poisson
#> The following objects are masked from 'package:base':
#> 
#>     %*%, apply, backsolve, beta, chol2inv, colMeans, colSums, diag,
#>     eigen, forwardsolve, gamma, identity, rowMeans, rowSums, sweep,
#>     tapply
n <- 3e3
Sigma <- wishart(n + 1, diag(n))
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> ✔ Initialising python and checking dependencies ... done!
#> 
chol_Sigma <- chol(Sigma)

Sigma2 <- wishart(n + 1, diag(n))
Sigma2_node <- greta:::get_node(Sigma2)
Sigma2_node$representations$cholesky <- NULL
Sigma2 <- greta:::as.greta_array(Sigma2_node)
chol_Sigma2 <- chol(Sigma2)

greta:::has_representation(Sigma, "cholesky")
#> [1] FALSE
greta:::has_representation(Sigma2, "cholesky")
#> [1] FALSE

bm_patch <- bench::mark(
  sigma_asis = calculate(chol_Sigma, nsim = 10),
  sigma_altered = calculate(chol_Sigma2, nsim = 10),
  check = FALSE
)
#> Warning: Some expressions had a GC in every iteration; so filtering is
#> disabled.

bm_patch
#> # A tibble: 2 × 6
#>   expression         min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr>    <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 sigma_asis         14s      14s    0.0717     831MB   0.0717
#> 2 sigma_altered    12.7s    12.7s    0.0785     824MB   0

Created on 2024-07-25 with reprex v2.1.0

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.4.0 (2024-04-24)
#>  os       macOS Sonoma 14.5
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Australia/Hobart
#>  date     2024-07-25
#>  pandoc   3.2.1 @ /opt/homebrew/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  backports     1.5.0      2024-05-23 [1] CRAN (R 4.4.0)
#>  base64enc     0.1-3      2015-07-28 [1] CRAN (R 4.4.0)
#>  bench         1.1.3      2023-05-04 [1] CRAN (R 4.4.0)
#>  callr         3.7.6      2024-03-25 [1] CRAN (R 4.4.0)
#>  cli           3.6.3      2024-06-21 [1] CRAN (R 4.4.0)
#>  coda          0.19-4.1   2024-01-31 [1] CRAN (R 4.4.0)
#>  codetools     0.2-20     2024-03-31 [2] CRAN (R 4.4.0)
#>  crayon        1.5.3      2024-06-20 [1] CRAN (R 4.4.0)
#>  digest        0.6.36     2024-06-23 [1] CRAN (R 4.4.0)
#>  evaluate      0.24.0     2024-06-10 [1] CRAN (R 4.4.0)
#>  fansi         1.0.6      2023-12-08 [1] CRAN (R 4.4.0)
#>  fastmap       1.2.0      2024-05-15 [1] CRAN (R 4.4.0)
#>  fs            1.6.4.9000 2024-06-26 [1] Github (r-lib/fs@714990b)
#>  future        1.33.2     2024-03-26 [1] CRAN (R 4.4.0)
#>  globals       0.16.3     2024-03-08 [1] CRAN (R 4.4.0)
#>  glue          1.7.0      2024-01-09 [1] CRAN (R 4.4.0)
#>  greta       * 0.4.5.9000 2024-07-25 [1] local
#>  hms           1.1.3      2023-03-21 [1] CRAN (R 4.4.0)
#>  htmltools     0.5.8.1    2024-04-04 [1] CRAN (R 4.4.0)
#>  jsonlite      1.8.8      2023-12-04 [1] CRAN (R 4.4.0)
#>  knitr         1.48       2024-07-07 [1] CRAN (R 4.4.0)
#>  lattice       0.22-6     2024-03-20 [2] CRAN (R 4.4.0)
#>  lifecycle     1.0.4      2023-11-07 [1] CRAN (R 4.4.0)
#>  listenv       0.9.1      2024-01-29 [1] CRAN (R 4.4.0)
#>  magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.4.0)
#>  Matrix        1.7-0      2024-03-22 [2] CRAN (R 4.4.0)
#>  parallelly    1.37.1     2024-02-29 [1] CRAN (R 4.4.0)
#>  pillar        1.9.0      2023-03-22 [1] CRAN (R 4.4.0)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.4.0)
#>  png           0.1-8      2022-11-29 [1] CRAN (R 4.4.0)
#>  prettyunits   1.2.0      2023-09-24 [1] CRAN (R 4.4.0)
#>  processx      3.8.4      2024-03-16 [1] CRAN (R 4.4.0)
#>  profmem       0.6.0      2020-12-13 [1] CRAN (R 4.4.0)
#>  progress      1.2.3      2023-12-06 [1] CRAN (R 4.4.0)
#>  ps            1.7.7      2024-07-02 [1] CRAN (R 4.4.0)
#>  purrr         1.0.2      2023-08-10 [1] CRAN (R 4.4.0)
#>  R.cache       0.16.0     2022-07-21 [1] CRAN (R 4.4.0)
#>  R.methodsS3   1.8.2      2022-06-13 [1] CRAN (R 4.4.0)
#>  R.oo          1.26.0     2024-01-24 [1] CRAN (R 4.4.0)
#>  R.utils       2.12.3     2023-11-18 [1] CRAN (R 4.4.0)
#>  R6            2.5.1      2021-08-19 [1] CRAN (R 4.4.0)
#>  Rcpp          1.0.12     2024-01-09 [1] CRAN (R 4.4.0)
#>  reprex        2.1.0      2024-01-11 [1] CRAN (R 4.4.0)
#>  reticulate    1.36.1     2024-04-22 [1] CRAN (R 4.4.0)
#>  rlang         1.1.4      2024-06-04 [1] CRAN (R 4.4.0)
#>  rmarkdown     2.27       2024-05-17 [1] CRAN (R 4.4.0)
#>  rstudioapi    0.16.0     2024-03-24 [1] CRAN (R 4.4.0)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.4.0)
#>  styler        1.10.3     2024-04-07 [1] CRAN (R 4.4.0)
#>  tensorflow    2.16.0     2024-04-15 [1] CRAN (R 4.4.0)
#>  tfautograph   0.3.2      2021-09-17 [1] CRAN (R 4.4.0)
#>  tfruns        1.5.3      2024-04-19 [1] CRAN (R 4.4.0)
#>  tibble        3.2.1      2023-03-20 [1] CRAN (R 4.4.0)
#>  utf8          1.2.4      2023-10-22 [1] CRAN (R 4.4.0)
#>  vctrs         0.6.5      2023-12-01 [1] CRAN (R 4.4.0)
#>  whisker       0.4.1      2022-12-05 [1] CRAN (R 4.4.0)
#>  withr         3.0.0      2024-01-16 [1] CRAN (R 4.4.0)
#>  xfun          0.45       2024-06-16 [1] CRAN (R 4.4.0)
#>  yaml          2.3.9      2024-07-05 [1] CRAN (R 4.4.0)
#> 
#>  [1] /Users/nick/Library/R/arm64/4.4/library
#>  [2] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
#> 
#> ─ Python configuration ───────────────────────────────────────────────────────
#>  python:         /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/bin/python
#>  libpython:      /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/libpython3.11.dylib
#>  pythonhome:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2:/Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2
#>  version:        3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:34:54) [Clang 16.0.6 ]
#>  numpy:          /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.11/site-packages/numpy
#>  numpy_version:  1.26.4
#>  tensorflow:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.11/site-packages/tensorflow
#>  
#>  NOTE: Python version was forced by use_python() function
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Compared to the TF2 poke fun branch: #534

library(greta)
#> 
#> Attaching package: 'greta'
#> The following objects are masked from 'package:stats':
#> 
#>     binomial, cov2cor, poisson
#> The following objects are masked from 'package:base':
#> 
#>     %*%, apply, backsolve, beta, chol2inv, colMeans, colSums, diag,
#>     eigen, forwardsolve, gamma, identity, rowMeans, rowSums, sweep,
#>     tapply
n <- 3e3
Sigma <- wishart(n + 1, diag(n))
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> ✔ Initialising python and checking dependencies ... done!
#> 
chol_Sigma <- chol(Sigma)

Sigma2 <- wishart(n + 1, diag(n))
Sigma2_node <- greta:::get_node(Sigma2)
Sigma2_node$representations$cholesky <- NULL
Sigma2 <- greta:::as.greta_array(Sigma2_node)
chol_Sigma2 <- chol(Sigma2)

greta:::has_representation(Sigma, "cholesky")
#> [1] TRUE
greta:::has_representation(Sigma2, "cholesky")
#> [1] FALSE

bm_patch <- bench::mark(
  sigma_asis = calculate(chol_Sigma, nsim = 10),
  sigma_altered = calculate(chol_Sigma2, nsim = 10),
  check = FALSE
)
#> Warning: Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Cannot use `calculate()` to sample a cholesky factor of a greta array
#> E.g., `x_chol <- chol(wishart(df = 4, Sigma = diag(3)))`
#> `calculate(x_chol)`
#> This is due to an internal issue with how greta handles cholesky
#> representations.
#> See issue here on github for more details:
#> <https://github.com/greta-dev/greta/issues/593>
#> Warning: Some expressions had a GC in every iteration; so filtering is
#> disabled.

bm_patch
#> # A tibble: 2 × 6
#>   expression         min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr>    <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 sigma_asis      49.5ms     56ms   16.0       76.6MB    2.00 
#> 2 sigma_altered    13.2s    13.2s    0.0759   824.1MB    0.152

Created on 2024-07-25 with reprex v2.1.0

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.4.0 (2024-04-24)
#>  os       macOS Sonoma 14.5
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Australia/Hobart
#>  date     2024-07-25
#>  pandoc   3.2.1 @ /opt/homebrew/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  backports     1.5.0      2024-05-23 [1] CRAN (R 4.4.0)
#>  base64enc     0.1-3      2015-07-28 [1] CRAN (R 4.4.0)
#>  bench         1.1.3      2023-05-04 [1] CRAN (R 4.4.0)
#>  callr         3.7.6      2024-03-25 [1] CRAN (R 4.4.0)
#>  cli           3.6.3      2024-06-21 [1] CRAN (R 4.4.0)
#>  coda          0.19-4.1   2024-01-31 [1] CRAN (R 4.4.0)
#>  codetools     0.2-20     2024-03-31 [2] CRAN (R 4.4.0)
#>  crayon        1.5.3      2024-06-20 [1] CRAN (R 4.4.0)
#>  digest        0.6.36     2024-06-23 [1] CRAN (R 4.4.0)
#>  evaluate      0.24.0     2024-06-10 [1] CRAN (R 4.4.0)
#>  fansi         1.0.6      2023-12-08 [1] CRAN (R 4.4.0)
#>  fastmap       1.2.0      2024-05-15 [1] CRAN (R 4.4.0)
#>  fs            1.6.4.9000 2024-06-26 [1] Github (r-lib/fs@714990b)
#>  future        1.33.2     2024-03-26 [1] CRAN (R 4.4.0)
#>  globals       0.16.3     2024-03-08 [1] CRAN (R 4.4.0)
#>  glue          1.7.0      2024-01-09 [1] CRAN (R 4.4.0)
#>  greta       * 0.4.5.9000 2024-07-25 [1] local
#>  hms           1.1.3      2023-03-21 [1] CRAN (R 4.4.0)
#>  htmltools     0.5.8.1    2024-04-04 [1] CRAN (R 4.4.0)
#>  jsonlite      1.8.8      2023-12-04 [1] CRAN (R 4.4.0)
#>  knitr         1.48       2024-07-07 [1] CRAN (R 4.4.0)
#>  lattice       0.22-6     2024-03-20 [2] CRAN (R 4.4.0)
#>  lifecycle     1.0.4      2023-11-07 [1] CRAN (R 4.4.0)
#>  listenv       0.9.1      2024-01-29 [1] CRAN (R 4.4.0)
#>  magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.4.0)
#>  Matrix        1.7-0      2024-03-22 [2] CRAN (R 4.4.0)
#>  parallelly    1.37.1     2024-02-29 [1] CRAN (R 4.4.0)
#>  pillar        1.9.0      2023-03-22 [1] CRAN (R 4.4.0)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.4.0)
#>  png           0.1-8      2022-11-29 [1] CRAN (R 4.4.0)
#>  prettyunits   1.2.0      2023-09-24 [1] CRAN (R 4.4.0)
#>  processx      3.8.4      2024-03-16 [1] CRAN (R 4.4.0)
#>  profmem       0.6.0      2020-12-13 [1] CRAN (R 4.4.0)
#>  progress      1.2.3      2023-12-06 [1] CRAN (R 4.4.0)
#>  ps            1.7.7      2024-07-02 [1] CRAN (R 4.4.0)
#>  purrr         1.0.2      2023-08-10 [1] CRAN (R 4.4.0)
#>  R.cache       0.16.0     2022-07-21 [1] CRAN (R 4.4.0)
#>  R.methodsS3   1.8.2      2022-06-13 [1] CRAN (R 4.4.0)
#>  R.oo          1.26.0     2024-01-24 [1] CRAN (R 4.4.0)
#>  R.utils       2.12.3     2023-11-18 [1] CRAN (R 4.4.0)
#>  R6            2.5.1      2021-08-19 [1] CRAN (R 4.4.0)
#>  Rcpp          1.0.12     2024-01-09 [1] CRAN (R 4.4.0)
#>  reprex        2.1.0      2024-01-11 [1] CRAN (R 4.4.0)
#>  reticulate    1.36.1     2024-04-22 [1] CRAN (R 4.4.0)
#>  rlang         1.1.4      2024-06-04 [1] CRAN (R 4.4.0)
#>  rmarkdown     2.27       2024-05-17 [1] CRAN (R 4.4.0)
#>  rstudioapi    0.16.0     2024-03-24 [1] CRAN (R 4.4.0)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.4.0)
#>  styler        1.10.3     2024-04-07 [1] CRAN (R 4.4.0)
#>  tensorflow    2.16.0     2024-04-15 [1] CRAN (R 4.4.0)
#>  tfautograph   0.3.2      2021-09-17 [1] CRAN (R 4.4.0)
#>  tfruns        1.5.3      2024-04-19 [1] CRAN (R 4.4.0)
#>  tibble        3.2.1      2023-03-20 [1] CRAN (R 4.4.0)
#>  utf8          1.2.4      2023-10-22 [1] CRAN (R 4.4.0)
#>  vctrs         0.6.5      2023-12-01 [1] CRAN (R 4.4.0)
#>  whisker       0.4.1      2022-12-05 [1] CRAN (R 4.4.0)
#>  withr         3.0.0      2024-01-16 [1] CRAN (R 4.4.0)
#>  xfun          0.45       2024-06-16 [1] CRAN (R 4.4.0)
#>  yaml          2.3.9      2024-07-05 [1] CRAN (R 4.4.0)
#> 
#>  [1] /Users/nick/Library/R/arm64/4.4/library
#>  [2] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
#> 
#> ─ Python configuration ───────────────────────────────────────────────────────
#>  python:         /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/bin/python
#>  libpython:      /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/libpython3.11.dylib
#>  pythonhome:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2:/Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2
#>  version:        3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:34:54) [Clang 16.0.6 ]
#>  numpy:          /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.11/site-packages/numpy
#>  numpy_version:  1.26.4
#>  tensorflow:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.11/site-packages/tensorflow
#>  
#>  NOTE: Python version was forced by use_python() function
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Overall we have removed the representation system by doing things this way.

@njtierney
Copy link
Collaborator Author

This has been resolved in #734

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant