Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exact Hamiltonian Monte Carlo for exponential sampling #142

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
3ef71df
implmentation of exact hmc walk
TolisChal Mar 6, 2021
932e2ae
implementation of boundary oracles for the hpolytope
TolisChal Mar 6, 2021
be84adc
add fake boundary oracles for the other polytope representations
TolisChal Mar 6, 2021
855f2af
implement c++ interface for exponential sampling
TolisChal Mar 6, 2021
462d132
improve boundary oracles and fix bugs
TolisChal Mar 6, 2021
da09805
add root finder for the degree two polynomial
TolisChal Mar 7, 2021
040efa6
complete the root computation and the failure check
TolisChal Mar 7, 2021
8e36064
improve R documentation and resolve gcc compile error
TolisChal Mar 7, 2021
436a780
improve R documentation
TolisChal Mar 7, 2021
eb9dab8
resolve PR reviews
TolisChal Mar 9, 2021
d1e9acf
fix compile errors
TolisChal Mar 9, 2021
f1e362a
Merge branch 'develop' into hmc_exponential
TolisChal Mar 11, 2021
b982f13
merge develop branch
TolisChal Mar 11, 2021
9c7735a
add burn in methods in exact hmc
TolisChal Mar 11, 2021
434de2f
resolve the reviews on the root computation
TolisChal Mar 12, 2021
1ac29c9
improve definition of macro RVOLESTI
TolisChal Mar 13, 2021
de8b3e0
add an example to sample from exponential exact HMC
TolisChal Mar 13, 2021
3442f5a
remove RVOLESTI macro
TolisChal Mar 19, 2021
60ed2bd
declare tol variable as static
TolisChal Mar 19, 2021
dff0c43
fix bug in variable declaration
TolisChal Mar 19, 2021
93d0658
change tolerance variable declaration
TolisChal Mar 19, 2021
29dfc59
Merge branch 'develop' into hmc_exponential
TolisChal Mar 21, 2021
dc1a67d
set upper bound for the number of relfections
TolisChal Mar 23, 2021
7bb8617
change the name of TOL to IN_INSIDE_BODY_TOLLERANCE
TolisChal Mar 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions R-proj/R/RcppExports.R
Original file line number Diff line number Diff line change
Expand Up @@ -302,18 +302,19 @@ rounding <- function(P, method = NULL, seed = NULL) {
#' \item{\code{starting_point} }{ A \eqn{d}-dimensional numerical vector that declares a starting point in the interior of the polytope for the random walk. The default choice is the center of the ball as that one computed by the function \code{inner_ball()}.}
#' \item{\code{BaW_rad} }{ The radius for the ball walk.}
#' \item{\code{L} }{ The maximum length of the billiard trajectory or the radius for the step of dikin, vaidya or john walk.}
#' \item{\code{solver}} {Specify ODE solver for logconcave sampling. Options are i) leapfrog, ii) euler iii) runge-kutta iv) richardson}
#' \item{\code{step_size} {Optionally chosen step size for logconcave sampling. Defaults to a theoretical value if not provided.}}
#' \item{\code{solver} }{ Specify ODE solver for logconcave sampling. Options are i) leapfrog, ii) euler iii) runge-kutta iv) richardson}
#' \item{\code{step_size }{ Optionally chosen step size for logconcave sampling. Defaults to a theoretical value if not provided.}}
#' }
#' @param distribution Optional. A list that declares the target density and some related parameters as follows:
#' \itemize{
#' \item{\code{density} }{ A string: (a) \code{'uniform'} for the uniform distribution or b) \code{'gaussian'} for the multidimensional spherical distribution. The default target distribution is uniform. c) Logconcave with form proportional to exp(-f(x)) where f(x) is L-smooth and m-strongly-convex. }
#' \item{\code{variance} }{ The variance of the multidimensional spherical gaussian. The default value is 1.}
#' \item{\code{density} }{ A string: (a) \code{'uniform'} for the uniform distribution or b) \code{'gaussian'} for the multidimensional spherical distribution c) \code{logconcave} with form proportional to exp(-f(x)) where f(x) is L-smooth and m-strongly-convex d) \code{'exponential'} for the exponential distribution. The default target distribution is uniform. }
#' \item{\code{variance} }{ The variance of the multidimensional spherical gaussian or the exponential distribution. The default value is 1.}
#' \item{\code{mode} }{ A \eqn{d}-dimensional numerical vector that declares the mode of the Gaussian distribution. The default choice is the center of the as that one computed by the function \code{inner_ball()}.}
#' \item{\code{L_}} { Smoothness constant (for logconcave). }
#' \item{\code{m}} { Strong-convexity constant (for logconcave). }
#' \item{\code{negative_logprob}} { Negative log-probability (for logconcave). }
#' \item{\code{negative_logprob_gradient}} { Negative log-probability gradient (for logconcave). }
#' \item{\code{bias} }{ The bias vector for the exponential distribution. The default vector is \eqn{c_1 = 1} and \eqn{c_i = 0} for \eqn{i \neq 1}.}
#' \item{\code{L_} }{ Smoothness constant (for logconcave). }
#' \item{\code{m} }{ Strong-convexity constant (for logconcave). }
#' \item{\code{negative_logprob} }{ Negative log-probability (for logconcave). }
#' \item{\code{negative_logprob_gradient} }{ Negative log-probability gradient (for logconcave). }
#' }
#' @param seed Optional. A fixed seed for the number generator.
#'
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/copula.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Apostolos Chalkis, as part of Google Summer of Code 2018 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/direct_sampling.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Apostolos Chalkis, as part of Google Summer of Code 2018 and 2019 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/exact_vol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
// Copyright (c) 2012-2019 Vissarion Fisikopoulos
// Copyright (c) 2018-2019 Apostolos Chalkis

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
1 change: 1 addition & 0 deletions R-proj/src/extractMatPoly.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
// Public License. If you did not receive this file along with HeaDDaCHe,
// see <http://www.gnu.org/licenses/>.

#define RVOLESTI

#ifndef EXTRACTMATPOLY_H
#define EXTRACTMATPOLY_H
Expand Down
1 change: 1 addition & 0 deletions R-proj/src/frustum_of_simplex.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
// Copyright (c) 2012-2018 Vissarion Fisikopoulos
// Copyright (c) 2018 Apostolos Chalkis

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/get_full_dimensional_polytope.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Alexandros Manochis, as part of Google Summer of Code 2020 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/geweke.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Alexandros Manochis, as part of Google Summer of Code 2020 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
1 change: 1 addition & 0 deletions R-proj/src/inner_ball.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// Copyright (c) 2012-2018 Vissarion Fisikopoulos
// Copyright (c) 2018 Apostolos Chalkis

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/ode_solve.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@

//Contributed and/or modified by Marios Papachristou, as part of Google Summer of Code 2018 and 2019 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/poly_gen.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Apostolos Chalkis, as part of Google Summer of Code 2018 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/polytopes_modules.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
// Copyright (c) 2012-2019 Vissarion Fisikopoulos
// Copyright (c) 2018-2019 Apostolos Chalkis

#define RVOLESTI

#include <Rcpp.h>


Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/psrf_multivariate.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Alexandros Manochis, as part of Google Summer of Code 2020 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/psrf_univariate.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Alexandros Manochis, as part of Google Summer of Code 2020 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/raftery.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Alexandros Manochis, as part of Google Summer of Code 2020 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/rotating.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Apostolos Chalkis, as part of Google Summer of Code 2018 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/rounding.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
//Contributed and/or modified by Apostolos Chalkis, as part of Google Summer of Code 2018 program.
//Contributed and/or modified by Alexandros Manochis, as part of Google Summer of Code 2020 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
59 changes: 41 additions & 18 deletions R-proj/src/sample_points.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@

// Contributed and/or modified by Alexandros Manochis, as part of Google Summer of Code 2020 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down Expand Up @@ -36,6 +38,7 @@ enum random_walks {
brdhr,
bcdhr,
hmc,
exponential_hmc,
uld
};

Expand All @@ -50,7 +53,7 @@ template <
>
void sample_from_polytope(Polytope &P, int type, RNGType &rng, PointList &randPoints,
unsigned int const& walkL, unsigned int const& numpoints,
bool const& gaussian, NT const& a, NT const& L,
bool const& gaussian, NT const& a, NT const& L, Point const& c,
Point const& StartingPoint, unsigned int const& nburns,
bool const& set_L, random_walks walk,
NegativeGradientFunctor *F=NULL, NegativeLogprobFunctor *f=NULL,
Expand Down Expand Up @@ -129,6 +132,15 @@ void sample_from_polytope(Polytope &P, int type, RNGType &rng, PointList &randPo
StartingPoint, nburns);
}
break;
case exponential_hmc:
if (set_L) {
ExponentialHamiltonianMonteCarloExactWalk WalkType(L);
exponential_sampling(randPoints, P, rng, WalkType, walkL, numpoints, c, a, StartingPoint, nburns);
} else {
exponential_sampling<ExponentialHamiltonianMonteCarloExactWalk>(randPoints, P, rng, walkL, numpoints, c, a,
StartingPoint, nburns);
}
break;
case ball_walk:
if (set_L) {
if (gaussian) {
Expand Down Expand Up @@ -228,18 +240,19 @@ void sample_from_polytope(Polytope &P, int type, RNGType &rng, PointList &randPo
//' \item{\code{starting_point} }{ A \eqn{d}-dimensional numerical vector that declares a starting point in the interior of the polytope for the random walk. The default choice is the center of the ball as that one computed by the function \code{inner_ball()}.}
//' \item{\code{BaW_rad} }{ The radius for the ball walk.}
//' \item{\code{L} }{ The maximum length of the billiard trajectory or the radius for the step of dikin, vaidya or john walk.}
//' \item{\code{solver}} {Specify ODE solver for logconcave sampling. Options are i) leapfrog, ii) euler iii) runge-kutta iv) richardson}
//' \item{\code{step_size} {Optionally chosen step size for logconcave sampling. Defaults to a theoretical value if not provided.}}
//' \item{\code{solver} }{ Specify ODE solver for logconcave sampling. Options are i) leapfrog, ii) euler iii) runge-kutta iv) richardson}
//' \item{\code{step_size }{ Optionally chosen step size for logconcave sampling. Defaults to a theoretical value if not provided.}}
//' }
//' @param distribution Optional. A list that declares the target density and some related parameters as follows:
//' \itemize{
//' \item{\code{density} }{ A string: (a) \code{'uniform'} for the uniform distribution or b) \code{'gaussian'} for the multidimensional spherical distribution. The default target distribution is uniform. c) Logconcave with form proportional to exp(-f(x)) where f(x) is L-smooth and m-strongly-convex. }
//' \item{\code{variance} }{ The variance of the multidimensional spherical gaussian. The default value is 1.}
//' \item{\code{density} }{ A string: (a) \code{'uniform'} for the uniform distribution or b) \code{'gaussian'} for the multidimensional spherical distribution c) \code{logconcave} with form proportional to exp(-f(x)) where f(x) is L-smooth and m-strongly-convex d) \code{'exponential'} for the exponential distribution. The default target distribution is uniform. }
//' \item{\code{variance} }{ The variance of the multidimensional spherical gaussian or the exponential distribution. The default value is 1.}
//' \item{\code{mode} }{ A \eqn{d}-dimensional numerical vector that declares the mode of the Gaussian distribution. The default choice is the center of the as that one computed by the function \code{inner_ball()}.}
//' \item{\code{L_}} { Smoothness constant (for logconcave). }
//' \item{\code{m}} { Strong-convexity constant (for logconcave). }
//' \item{\code{negative_logprob}} { Negative log-probability (for logconcave). }
//' \item{\code{negative_logprob_gradient}} { Negative log-probability gradient (for logconcave). }
//' \item{\code{bias} }{ The bias vector for the exponential distribution. The default vector is \eqn{c_1 = 1} and \eqn{c_i = 0} for \eqn{i \neq 1}.}
//' \item{\code{L_} }{ Smoothness constant (for logconcave). }
//' \item{\code{m} }{ Strong-convexity constant (for logconcave). }
//' \item{\code{negative_logprob} }{ Negative log-probability (for logconcave). }
//' \item{\code{negative_logprob_gradient} }{ Negative log-probability gradient (for logconcave). }
//' }
//' @param seed Optional. A fixed seed for the number generator.
//'
Expand Down Expand Up @@ -308,6 +321,8 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
rng.set_seed(seed_rcpp);
}

Point c(dim);

NT radius = 1.0, L;
bool set_mode = false, gaussian = false, logconcave = false,
set_starting_point = false, set_L = false;
Expand All @@ -330,6 +345,9 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
} else if (
Rcpp::as<std::string>(Rcpp::as<Rcpp::List>(distribution)["density"]).compare(std::string("gaussian")) == 0) {
gaussian = true;
} else if (
Rcpp::as<std::string>(Rcpp::as<Rcpp::List>(distribution)["density"]).compare(std::string("exponential")) == 0) {
walk = exponential_hmc;
} else if (
Rcpp::as<std::string>(Rcpp::as<Rcpp::List>(distribution)["density"]).compare(std::string("logconcave")) == 0) {
logconcave = true;
Expand All @@ -351,13 +369,18 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
NT a = 0.5;
if (Rcpp::as<Rcpp::List>(distribution).containsElementNamed("variance")) {
a = 1.0 / (2.0 * Rcpp::as<NT>(Rcpp::as<Rcpp::List>(distribution)["variance"]));
if (!gaussian) {
if (walk == exponential_hmc) a = Rcpp::as<NT>(Rcpp::as<Rcpp::List>(distribution)["variance"]);
if (!gaussian && walk != exponential_hmc) {
Rcpp::warning("The variance can be set only for Gaussian sampling!");
} else if (a <= 0.0) {
throw Rcpp::exception("The variance has to be positive!");
}
}

if (Rcpp::as<Rcpp::List>(distribution).containsElementNamed("bias")) {
c = Point(Rcpp::as<VT>(Rcpp::as<Rcpp::List>(distribution)["bias"]));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why you call it c and not bias?

}

if (Rcpp::as<Rcpp::List>(distribution).containsElementNamed("negative_logprob") &&
Rcpp::as<Rcpp::List>(distribution).containsElementNamed("negative_logprob_gradient")) {

Expand Down Expand Up @@ -422,7 +445,7 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
} else {
walk = rdhr;
}
} else {
} else if(walk != exponential_hmc) {
if (type == 1) {
walk = accelarated_billiard;
} else {
Expand Down Expand Up @@ -485,10 +508,10 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
walk = bcdhr;
} else if (Rcpp::as<std::string>(Rcpp::as<Rcpp::List>(random_walk)["walk"]).compare(std::string("HMC")) == 0) {
if (!logconcave) throw Rcpp::exception("HMC is not supported for non first-order sampling");
walk = hmc;
walk = hmc;
} else if (Rcpp::as<std::string>(Rcpp::as<Rcpp::List>(random_walk)["walk"]).compare(std::string("ULD")) == 0) {
if (!logconcave) throw Rcpp::exception("ULD is not supported for non first-order sampling");
walk = uld;
if (!logconcave) throw Rcpp::exception("ULD is not supported for non first-order sampling");
walk = uld;
} else {
throw Rcpp::exception("Unknown walk type!");
}
Expand Down Expand Up @@ -537,7 +560,7 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
StartingPoint = StartingPoint - mode;
HP.shift(mode.getCoefficients());
}
sample_from_polytope(HP, type, rng, randPoints, walkL, numpoints, gaussian, a, L,
sample_from_polytope(HP, type, rng, randPoints, walkL, numpoints, gaussian, a, L, c,
StartingPoint, nburns, set_L, walk, F, f, solver);
break;
}
Expand All @@ -558,7 +581,7 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
StartingPoint = StartingPoint - mode;
VP.shift(mode.getCoefficients());
}
sample_from_polytope(VP, type, rng, randPoints, walkL, numpoints, gaussian, a, L,
sample_from_polytope(VP, type, rng, randPoints, walkL, numpoints, gaussian, a, L, c,
StartingPoint, nburns, set_L, walk, F, f, solver);
break;
}
Expand All @@ -579,7 +602,7 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
StartingPoint = StartingPoint - mode;
ZP.shift(mode.getCoefficients());
}
sample_from_polytope(ZP, type, rng, randPoints, walkL, numpoints, gaussian, a, L,
sample_from_polytope(ZP, type, rng, randPoints, walkL, numpoints, gaussian, a, L, c,
StartingPoint, nburns, set_L, walk, F, f, solver);
break;
}
Expand All @@ -602,7 +625,7 @@ Rcpp::NumericMatrix sample_points(Rcpp::Nullable<Rcpp::Reference> P,
StartingPoint = StartingPoint - mode;
VPcVP.shift(mode.getCoefficients());
}
sample_from_polytope(VPcVP, type, rng, randPoints, walkL, numpoints, gaussian, a, L,
sample_from_polytope(VPcVP, type, rng, randPoints, walkL, numpoints, gaussian, a, L, c,
StartingPoint, nburns, set_L, walk, F, f, solver);
break;
}
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/spectrahedron.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

// Licensed under GNU LGPL.3, see LICENCE file

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <chrono>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/spectrahedron_module.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

// Licensed under GNU LGPL.3, see LICENCE file

#define RVOLESTI

#include <Rcpp.h>

class Spectrahedron {
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/volume.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

//Contributed and/or modified by Apostolos Chalkis, as part of Google Summer of Code 2018 and 2019 program.

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <boost/random.hpp>
Expand Down
2 changes: 2 additions & 0 deletions R-proj/src/zonotope_approximation.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
// Copyright (c) 2012-2018 Vissarion Fisikopoulos
// Copyright (c) 2018 Apostolos Chalkis

#define RVOLESTI

#include <Rcpp.h>
#include <RcppEigen.h>
#include <boost/random.hpp>
Expand Down
Loading