Skip to content

Commit

Permalink
rerendered docs. Will get test coverage up again, next.
Browse files Browse the repository at this point in the history
  • Loading branch information
pat-alt committed Dec 1, 2022
1 parent 0ad8296 commit 767dd4f
Show file tree
Hide file tree
Showing 14 changed files with 9,245 additions and 61 deletions.
4 changes: 2 additions & 2 deletions _freeze/docs/src/classification/execute-results/md.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion _freeze/docs/src/index/execute-results/md.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"hash": "c56dcfed5fce5fece3f8dd90b08af0bd",
"result": {
"markdown": "```@meta\nCurrentModule = ConformalPrediction\n```\n\n# ConformalPrediction\n\nDocumentation for [ConformalPrediction.jl](https://github.com/pat-alt/ConformalPrediction.jl).\n\n\n\n`ConformalPrediction.jl` is a package for Uncertainty Quantification (UQ) through Conformal Prediction (CP) in Julia. It is designed to work with supervised models trained in [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) @blaom2020mlj. Conformal Prediction is distribution-free, easy-to-understand, easy-to-use and model-agnostic. \n\n# 📖 Background\n\nConformal Prediction is a scalable frequentist approach to uncertainty quantification and coverage control. It promises to be an easy-to-understand, distribution-free and model-agnostic way to generate statistically rigorous uncertainty estimates. Interestingly, it can even be used to complement Bayesian methods.\n\nThe animation below is lifted from a small blog post that introduces the topic and the package ([[TDS](https://towardsdatascience.com/conformal-prediction-in-julia-351b81309e30)], [[Quarto](https://www.paltmeyer.com/blog/posts/conformal-prediction/#fig-anim)]). It shows conformal prediction sets for two different samples and changing coverage rates. Standard conformal classifiers produce set-valued predictions: for ambiguous samples these sets are typically large (for high coverage) or empty (for low coverage).\n\n![Conformal Prediction in action: Prediction sets for two different samples and changing coverage rates. As coverage grows, so does the size of the prediction sets.](https://raw.githubusercontent.com/pat-alt/blog/main/posts/conformal-prediction/www/medium.gif)\n\n## 🚩 Installation \n\nYou can install the latest stable release from the general registry:\n\n```{.julia}\nusing Pkg\nPkg.add(\"ConformalPrediction\")\n```\n\nThe development version can be installed as follows:\n\n```{.julia}\nusing Pkg\nPkg.add(url=\"https://github.com/pat-alt/ConformalPrediction.jl\")\n```\n\n## 🔁 Status \n\nThis package is in its early stages of development and therefore still subject to changes to the core architecture and API. The following CP approaches have been implemented in the development version:\n\n**Regression**:\n\n- Inductive \n- Naive Transductive \n- Jackknife \n- Jackknife+ \n- Jackknife-minmax\n- CV+\n- CV-minmax\n\n**Classification**:\n\n- Inductive (LABEL [@sadinle2019least])\n- Adaptive Inductive\n\nThe package has been tested for the following supervised models offered by [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/).\n\n**Regression**:\n\n::: {.cell execution_count=2}\n``` {.julia .cell-code}\nusing ConformalPrediction\nkeys(tested_atomic_models[:regression])\n```\n\n::: {.cell-output .cell-output-display execution_count=3}\n```\nKeySet for a Dict{Symbol, Expr} with 4 entries. Keys:\n :nearest_neighbor\n :evo_tree\n :light_gbm\n :decision_tree\n```\n:::\n:::\n\n\n**Classification**:\n\n::: {.cell execution_count=3}\n``` {.julia .cell-code}\nkeys(tested_atomic_models[:classification])\n```\n\n::: {.cell-output .cell-output-display execution_count=4}\n```\nKeySet for a Dict{Symbol, Expr} with 4 entries. Keys:\n :nearest_neighbor\n :evo_tree\n :light_gbm\n :decision_tree\n```\n:::\n:::\n\n\n## 🔍 Usage Example \n\nTo illustrate the intended use of the package, let's have a quick look at a simple regression problem. Using [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) we first generate some synthetic data and then determine indices for our training, calibration and test data:\n\n::: {.cell execution_count=4}\n``` {.julia .cell-code}\nusing MLJ\nX, y = MLJ.make_regression(1000, 2)\ntrain, test = partition(eachindex(y), 0.4, 0.4)\n```\n:::\n\n\nWe then import a decision tree ([`EvoTrees.jl`](https://github.com/Evovest/EvoTrees.jl)) following the standard [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) procedure.\n\n::: {.cell execution_count=5}\n``` {.julia .cell-code}\nEvoTreeRegressor = @load EvoTreeRegressor pkg=EvoTrees\nmodel = EvoTreeRegressor() \n```\n:::\n\n\nTo turn our conventional model into a conformal model, we just need to declare it as such by using `conformal_model` wrapper function. The generated conformal model instance can wrapped in data to create a *machine*. Finally, we proceed by fitting the machine on training data using the generic `fit!` method:\n\n::: {.cell execution_count=6}\n``` {.julia .cell-code}\nusing ConformalPrediction\nconf_model = conformal_model(model)\nmach = machine(conf_model, X, y)\nfit!(mach, rows=train)\n```\n:::\n\n\nPredictions can then be computed using the generic `predict` method. The code below produces predictions for the first `n` samples. Each tuple contains the lower and upper bound for the prediction interval.\n\n::: {.cell execution_count=7}\n``` {.julia .cell-code}\nn = 5\nXtest = selectrows(X, first(test,n))\nytest = y[first(test,n)]\npredict(mach, Xtest)\n```\n\n::: {.cell-output .cell-output-display execution_count=8}\n```\n╭─────────────────────────────────────────────────────────╮\n│ │\n│ (1) (1.9785806677967583, 3.555027766731997) │\n│ (2) (3.902150239475792, 5.4785973384110305) │\n│ (3) (0.5223306700743977, 2.0987777690096365) │\n│ (4) (2.3013653650067107, 3.8778124639419493) │\n│ (5) (2.380170088915079, 3.9566171878503176) │\n│ │\n│ │\n╰───────────────────────────────────────────── 5 items ───╯\n```\n:::\n:::\n\n\n## 🛠 Contribute \n\nContributions are welcome! Please follow the [SciML ColPrac guide](https://github.com/SciML/ColPrac).\n\n## 🎓 References \n\n",
"markdown": "```@meta\nCurrentModule = ConformalPrediction\n```\n\n# ConformalPrediction\n\nDocumentation for [ConformalPrediction.jl](https://github.com/pat-alt/ConformalPrediction.jl).\n\n\n\n`ConformalPrediction.jl` is a package for Uncertainty Quantification (UQ) through Conformal Prediction (CP) in Julia. It is designed to work with supervised models trained in [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) @blaom2020mlj. Conformal Prediction is distribution-free, easy-to-understand, easy-to-use and model-agnostic. \n\n# 📖 Background\n\nConformal Prediction is a scalable frequentist approach to uncertainty quantification and coverage control. It promises to be an easy-to-understand, distribution-free and model-agnostic way to generate statistically rigorous uncertainty estimates. Interestingly, it can even be used to complement Bayesian methods.\n\nThe animation below is lifted from a small blog post that introduces the topic and the package ([[TDS](https://towardsdatascience.com/conformal-prediction-in-julia-351b81309e30)], [[Quarto](https://www.paltmeyer.com/blog/posts/conformal-prediction/#fig-anim)]). It shows conformal prediction sets for two different samples and changing coverage rates. Standard conformal classifiers produce set-valued predictions: for ambiguous samples these sets are typically large (for high coverage) or empty (for low coverage).\n\n![Conformal Prediction in action: Prediction sets for two different samples and changing coverage rates. As coverage grows, so does the size of the prediction sets.](https://raw.githubusercontent.com/pat-alt/blog/main/posts/conformal-prediction/www/medium.gif)\n\n## 🚩 Installation \n\nYou can install the latest stable release from the general registry:\n\n```{.julia}\nusing Pkg\nPkg.add(\"ConformalPrediction\")\n```\n\nThe development version can be installed as follows:\n\n```{.julia}\nusing Pkg\nPkg.add(url=\"https://github.com/pat-alt/ConformalPrediction.jl\")\n```\n\n## 🔁 Status \n\nThis package is in its early stages of development and therefore still subject to changes to the core architecture and API. The following CP approaches have been implemented in the development version:\n\n**Regression**:\n\n- Inductive \n- Naive Transductive \n- Jackknife \n- Jackknife+ \n- Jackknife-minmax\n- CV+\n- CV-minmax\n\n**Classification**:\n\n- Inductive (LABEL [@sadinle2019least])\n- Adaptive Inductive\n\nThe package has been tested for the following supervised models offered by [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/).\n\n**Regression**:\n\n::: {.cell execution_count=2}\n``` {.julia .cell-code}\nusing ConformalPrediction\nkeys(tested_atomic_models[:regression])\n```\n\n::: {.cell-output .cell-output-display execution_count=3}\n```\nKeySet for a Dict{Symbol, Expr} with 4 entries. Keys:\n :nearest_neighbor\n :evo_tree\n :light_gbm\n :decision_tree\n```\n:::\n:::\n\n\n**Classification**:\n\n::: {.cell execution_count=3}\n``` {.julia .cell-code}\nkeys(tested_atomic_models[:classification])\n```\n\n::: {.cell-output .cell-output-display execution_count=4}\n```\nKeySet for a Dict{Symbol, Expr} with 4 entries. Keys:\n :nearest_neighbor\n :evo_tree\n :light_gbm\n :decision_tree\n```\n:::\n:::\n\n\n## 🔍 Usage Example \n\nTo illustrate the intended use of the package, let's have a quick look at a simple regression problem. Using [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) we first generate some synthetic data and then determine indices for our training, calibration and test data:\n\n::: {.cell execution_count=4}\n``` {.julia .cell-code}\nusing MLJ\nX, y = MLJ.make_regression(1000, 2)\ntrain, test = partition(eachindex(y), 0.4, 0.4)\n```\n:::\n\n\nWe then import a decision tree ([`EvoTrees.jl`](https://github.com/Evovest/EvoTrees.jl)) following the standard [MLJ](https://alan-turing-institute.github.io/MLJ.jl/dev/) procedure.\n\n::: {.cell execution_count=5}\n``` {.julia .cell-code}\nEvoTreeRegressor = @load EvoTreeRegressor pkg=EvoTrees\nmodel = EvoTreeRegressor() \n```\n:::\n\n\nTo turn our conventional model into a conformal model, we just need to declare it as such by using `conformal_model` wrapper function. The generated conformal model instance can wrapped in data to create a *machine*. Finally, we proceed by fitting the machine on training data using the generic `fit!` method:\n\n::: {.cell execution_count=6}\n``` {.julia .cell-code}\nusing ConformalPrediction\nconf_model = conformal_model(model)\nmach = machine(conf_model, X, y)\nfit!(mach, rows=train)\n```\n:::\n\n\nPredictions can then be computed using the generic `predict` method. The code below produces predictions for the first `n` samples. Each tuple contains the lower and upper bound for the prediction interval.\n\n::: {.cell execution_count=7}\n``` {.julia .cell-code}\nn = 5\nXtest = selectrows(X, first(test,n))\nytest = y[first(test,n)]\npredict(mach, Xtest)\n```\n\n::: {.cell-output .cell-output-display execution_count=8}\n```\n╭─────────────────────────────────────────────────────────╮\n│ │\n│ (1) (1.2801183281465092, 2.0024286641173816) │\n│ (2) (0.8012756658949756, 1.5235860018658482) │\n│ (3) (1.1850387604493555, 1.9073490964202282) │\n│ (4) (1.1185514282818692, 1.8408617642527418) │\n│ (5) (1.1651738766694149, 1.8874842126402875) │\n│ │\n│ │\n╰───────────────────────────────────────────── 5 items ───╯\n```\n:::\n:::\n\n\n## 🛠 Contribute \n\nContributions are welcome! Please follow the [SciML ColPrac guide](https://github.com/SciML/ColPrac).\n\n## 🎓 References \n\n",
"supporting": [
"index_files"
],
Expand Down
Loading

0 comments on commit 767dd4f

Please sign in to comment.