diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 86892fc30..5a8d248eb 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2025-01-22T16:02:48","documenter_version":"1.8.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2025-01-22T19:29:19","documenter_version":"1.8.0"}} \ No newline at end of file diff --git a/dev/GaussHermite/index.html b/dev/GaussHermite/index.html index c1c1675e8..e6e5fdd80 100644 --- a/dev/GaussHermite/index.html +++ b/dev/GaussHermite/index.html @@ -16,7 +16,7 @@ x=gh9[1], y=gh9[2], Geom.hair, Geom.point, Scale.y_log2, Guide.ylabel("Weight (log scale)"), Guide.xlabel(""), -)Example block output

The definition of MixedModels.GHnorm is similar to the gausshermitenorm function with some extra provisions for ensuring symmetry of the abscissae and the weights and for caching values once they have been calculated.

MixedModels.GHnorm — Function
GHnorm(k::Int)

Return the (unique) GaussHermiteNormalized{k} object.

The function values are stored (memoized) when first evaluated. Subsequent evaluations for the same k have very low overhead.

source
using MixedModels
+)
Example block output

The definition of MixedModels.GHnorm is similar to the gausshermitenorm function with some extra provisions for ensuring symmetry of the abscissae and the weights and for caching values once they have been calculated.

MixedModels.GHnorm — Function
GHnorm(k::Int)

Return the (unique) GaussHermiteNormalized{k} object.

The function values are stored (memoized) when first evaluated. Subsequent evaluations for the same k have very low overhead.

source
using MixedModels
 GHnorm(3)
MixedModels.GaussHermiteNormalized{3}([-1.7320508075688772, 0.0, 1.7320508075688772], [0.16666666666666666, 0.6666666666666666, 0.16666666666666666])

By the properties of the normal distribution, when $\mathcal{X}\sim\mathscr{N}(\mu, \sigma^2)$

\[\mathbb{E}[g(x)] \approx \sum_{i=1}^k g(\mu + \sigma z_i)\,w_i\]

For example, $\mathbb{E}[\mathcal{X}^2]$ where $\mathcal{X}\sim\mathcal{N}(2, 3^2)$ is

μ = 2; σ = 3; ghn3 = GHnorm(3);
 sum(@. ghn3.w * abs2(μ + σ * ghn3.z))  # should be μ² + σ² = 13
13.0

(In general a dot, '.', after the function name in a function call, as in abs2.(...), or before an operator creates a fused vectorized evaluation in Julia. The macro @. has the effect of vectorizing all operations in the subsequent expression.)

Application to a model for contraception use

A binary response is a "Yes"/"No" type of answer. For example, in a 1989 fertility survey of women in Bangladesh (reported in Huq, N. M. and Cleland, J., 1990) one response of interest was whether the woman used artificial contraception. Several covariates were recorded including the woman's age (centered at the mean), the number of live children the woman has had (in 4 categories: 0, 1, 2, and 3 or more), whether she lived in an urban setting, and the district in which she lived. The version of the data used here is that used in review of multilevel modeling software conducted by the Center for Multilevel Modelling, currently at University of Bristol (http://www.bristol.ac.uk/cmm/learning/mmsoftware/data-rev.html). These data are available as the :contra dataset.

contra = DataFrame(MixedModels.dataset(:contra))
 describe(contra)
5×7 DataFrame
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1distD01D610String
2urbanNY0String
3livch03+0String
4age0.00204757-13.56-1.5619.440Float64
5useNY0String

A smoothed scatterplot of contraception use versus age

plot(contra, x=:age, y=:use, Geom.smooth, Guide.xlabel("Centered age (yr)"),
@@ -1141,4 +1141,4 @@
   
 
 
-'/>
  • 1https://en.wikipedia.org/wiki/Gaussian_quadrature

+'/>
  • 1https://en.wikipedia.org/wiki/Gaussian_quadrature

diff --git a/dev/api/index.html b/dev/api/index.html
index 81f839c4b..8608668c6 100644
--- a/dev/api/index.html
+++ b/dev/api/index.html
@@ -1,36 +1,36 @@
 
-API · MixedModels
GitHub
API
In addition to its own functionality, MixedModels.jl also implements extensive support for the StatsAPI.StatisticalModel and StatsAPI.RegressionModel API.
Types
MixedModels.BlockDescription — Type
BlockDescription
Description of blocks of A and L in a LinearMixedModel
Fields
  • blknms: Vector{String} of block names
  • blkrows: Vector{Int} of the number of rows in each block
  • ALtypes: Matrix{String} of datatypes for blocks in A and L.
When a block in L is the same type as the corresponding block in A, it is described with a single name, such as Dense.  When the types differ the entry in ALtypes is of the form Diag/Dense, as determined by a shorttype method.
source
MixedModels.BlockedSparse — Type
BlockedSparse{Tv,S,P}
A SparseMatrixCSC whose nonzeros form blocks of rows or columns or both.
Members
  • cscmat: SparseMatrixCSC{Tv, Int32} representation for general calculations
  • nzasmat: nonzeros of cscmat as a dense matrix
  • colblkptr: pattern of blocks of columns
The only time these are created are as products of ReMats.
source
MixedModels.FeMat — Type
FeMat{T,S}
A matrix and a (possibly) weighted copy of itself.
Typically, an FeMat represents the fixed-effects model matrix with the response (y) concatenated as a final column.
Note
FeMat is not the same as FeTerm.
Fields
  • xy: original matrix, called xy b/c in practice this is hcat(fullrank(X), y)
  • wtxy: (possibly) weighted copy of xy (shares storage with xy until weights are applied)
Upon construction the xy and wtxy fields refer to the same matrix
source
MixedModels.FeTerm — Type
FeTerm{T,S}
Term with an explicit, constant matrix representation
Typically, an FeTerm represents the model matrix for the fixed effects.
Note
FeTerm is not the same as FeMat!
Fields
  • x: full model matrix
  • piv: pivot Vector{Int} for moving linearly dependent columns to the right
  • rank: computational rank of x
  • cnames: vector of column names
source
MixedModels.GaussHermiteNormalized — Type
GaussHermiteNormalized{K}
A struct with 2 SVector{K,Float64} members
  • z: abscissae for the K-point Gauss-Hermite quadrature rule on the Z scale
  • wt: Gauss-Hermite weights normalized to sum to unity
source
MixedModels.GeneralizedLinearMixedModel — Type
GeneralizedLinearMixedModel
Generalized linear mixed-effects model representation
Fields
  • LMM: a LinearMixedModel - the local approximation to the GLMM.
  • β: the pivoted and possibly truncated fixed-effects vector
  • β₀: similar to β. Used in the PIRLS algorithm if step-halving is needed.
  • θ: covariance parameter vector
  • b: similar to u, equivalent to broadcast!(*, b, LMM.Λ, u)
  • u: a vector of matrices of random effects
  • u₀: similar to u.  Used in the PIRLS algorithm if step-halving is needed.
  • resp: a GlmResp object
  • η: the linear predictor
  • wt: vector of prior case weights, a value of T[] indicates equal weights.
The following fields are used in adaptive Gauss-Hermite quadrature, which applies only to models with a single random-effects term, in which case their lengths are the number of levels in the grouping factor for that term.  Otherwise they are zero-length vectors.
  • devc: vector of deviance components
  • devc0: vector of deviance components at offset of zero
  • sd: approximate standard deviation of the conditional density
  • mult: multiplier
Properties
In addition to the fieldnames, the following names are also accessible through the . extractor
  • theta: synonym for θ
  • beta: synonym for β
  • σ or sigma: common scale parameter (value is NaN for distributions without a scale parameter)
  • lowerbd: vector of lower bounds on the combined elements of β and θ
  • formula, trms, A, L, and optsum: fields of the LMM field
  • X: fixed-effects model matrix
  • y: response vector
source
MixedModels.Grouping — Type
struct Grouping <: StatsModels.AbstractContrasts end
A placeholder type to indicate that a categorical variable is only used for grouping and not for contrasts.  When creating a CategoricalTerm, this skips constructing the contrasts matrix which makes it robust to large numbers of levels, while still holding onto the vector of levels and constructing the level-to-index mapping (invindex field of the ContrastsMatrix.).
Note that calling modelcols on a CategoricalTerm{Grouping} is an error.
Examples
julia> schema((; grp = string.(1:100_000)))
+API · MixedModels
GitHub
API
In addition to its own functionality, MixedModels.jl also implements extensive support for the StatsAPI.StatisticalModel and StatsAPI.RegressionModel API.
Types
MixedModels.BlockDescription — Type
BlockDescription
Description of blocks of A and L in a LinearMixedModel
Fields
  • blknms: Vector{String} of block names
  • blkrows: Vector{Int} of the number of rows in each block
  • ALtypes: Matrix{String} of datatypes for blocks in A and L.
When a block in L is the same type as the corresponding block in A, it is described with a single name, such as Dense.  When the types differ the entry in ALtypes is of the form Diag/Dense, as determined by a shorttype method.
source
MixedModels.BlockedSparse — Type
BlockedSparse{Tv,S,P}
A SparseMatrixCSC whose nonzeros form blocks of rows or columns or both.
Members
  • cscmat: SparseMatrixCSC{Tv, Int32} representation for general calculations
  • nzasmat: nonzeros of cscmat as a dense matrix
  • colblkptr: pattern of blocks of columns
The only time these are created are as products of ReMats.
source
MixedModels.FeMat — Type
FeMat{T,S}
A matrix and a (possibly) weighted copy of itself.
Typically, an FeMat represents the fixed-effects model matrix with the response (y) concatenated as a final column.
Note
FeMat is not the same as FeTerm.
Fields
  • xy: original matrix, called xy b/c in practice this is hcat(fullrank(X), y)
  • wtxy: (possibly) weighted copy of xy (shares storage with xy until weights are applied)
Upon construction the xy and wtxy fields refer to the same matrix
source
MixedModels.FeTerm — Type
FeTerm{T,S}
Term with an explicit, constant matrix representation
Typically, an FeTerm represents the model matrix for the fixed effects.
Note
FeTerm is not the same as FeMat!
Fields
  • x: full model matrix
  • piv: pivot Vector{Int} for moving linearly dependent columns to the right
  • rank: computational rank of x
  • cnames: vector of column names
source
MixedModels.GaussHermiteNormalized — Type
GaussHermiteNormalized{K}
A struct with 2 SVector{K,Float64} members
  • z: abscissae for the K-point Gauss-Hermite quadrature rule on the Z scale
  • wt: Gauss-Hermite weights normalized to sum to unity
source
MixedModels.GeneralizedLinearMixedModel — Type
GeneralizedLinearMixedModel
Generalized linear mixed-effects model representation
Fields
  • LMM: a LinearMixedModel - the local approximation to the GLMM.
  • β: the pivoted and possibly truncated fixed-effects vector
  • β₀: similar to β. Used in the PIRLS algorithm if step-halving is needed.
  • θ: covariance parameter vector
  • b: similar to u, equivalent to broadcast!(*, b, LMM.Λ, u)
  • u: a vector of matrices of random effects
  • u₀: similar to u.  Used in the PIRLS algorithm if step-halving is needed.
  • resp: a GlmResp object
  • η: the linear predictor
  • wt: vector of prior case weights, a value of T[] indicates equal weights.
The following fields are used in adaptive Gauss-Hermite quadrature, which applies only to models with a single random-effects term, in which case their lengths are the number of levels in the grouping factor for that term.  Otherwise they are zero-length vectors.
  • devc: vector of deviance components
  • devc0: vector of deviance components at offset of zero
  • sd: approximate standard deviation of the conditional density
  • mult: multiplier
Properties
In addition to the fieldnames, the following names are also accessible through the . extractor
  • theta: synonym for θ
  • beta: synonym for β
  • σ or sigma: common scale parameter (value is NaN for distributions without a scale parameter)
  • lowerbd: vector of lower bounds on the combined elements of β and θ
  • formula, trms, A, L, and optsum: fields of the LMM field
  • X: fixed-effects model matrix
  • y: response vector
source
MixedModels.Grouping — Type
struct Grouping <: StatsModels.AbstractContrasts end
A placeholder type to indicate that a categorical variable is only used for grouping and not for contrasts.  When creating a CategoricalTerm, this skips constructing the contrasts matrix which makes it robust to large numbers of levels, while still holding onto the vector of levels and constructing the level-to-index mapping (invindex field of the ContrastsMatrix.).
Note that calling modelcols on a CategoricalTerm{Grouping} is an error.
Examples
julia> schema((; grp = string.(1:100_000)))
 # out-of-memory error
 
-julia> schema((; grp = string.(1:100_000)), Dict(:grp => Grouping()))
source
MixedModels.LikelihoodRatioTest — Type
LikelihoodRatioTest
Results of MixedModels.likelihoodratiotest
Fields
  • formulas: Vector of model formulae
  • models: NamedTuple of the dof and deviance of the models
  • tests: NamedTuple of the sequential dofdiff, deviancediff,          and resulting pvalues
Properties
  • deviance : note that this is actually -2 log likelihood for linear models              (i.e. without subtracting the constant for a saturated model)
  • pvalues
source
MixedModels.LinearMixedModel — Type
LinearMixedModel(y, Xs, form, wts=[], σ=nothing, amalgamate=true)
Private constructor for a LinearMixedModel.
To construct a model, you only need the response (y), already assembled model matrices (Xs), schematized formula (form) and weights (wts). Everything else in the structure can be derived from these quantities.
Note
This method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.
source
MixedModels.LinearMixedModel — Type
LinearMixedModel
Linear mixed-effects model representation
Fields
  • formula: the formula for the model
  • reterms: a Vector{AbstractReMat{T}} of random-effects terms.
  • Xymat: horizontal concatenation of a full-rank fixed-effects model matrix X and response y as an FeMat{T}
  • feterm: the fixed-effects model matrix as an FeTerm{T}
  • sqrtwts: vector of square roots of the case weights.  Can be empty.
  • parmap : Vector{NTuple{3,Int}} of (block, row, column) mapping of θ to λ
  • dims : NamedTuple{(:n, :p, :nretrms),NTuple{3,Int}} of dimensions.  p is the rank of X, which may be smaller than size(X, 2).
  • A: a Vector{AbstractMatrix} containing the row-major packed lower triangle of hcat(Z,X,y)'hcat(Z,X,y)
  • L: the blocked lower Cholesky factor of Λ'AΛ+I in the same Vector representation as A
  • optsum: an OptSummary object
Properties
  • θ or theta: the covariance parameter vector used to form λ
  • β or beta: the fixed-effects coefficient vector
  • λ or lambda: a vector of lower triangular matrices repeated on the diagonal blocks of Λ
  • σ or sigma: current value of the standard deviation of the per-observation noise
  • b: random effects on the original scale, as a vector of matrices
  • u: random effects on the orthogonal scale, as a vector of matrices
  • lowerbd: lower bounds on the elements of θ
  • X: the fixed-effects model matrix
  • y: the response vector
source
MixedModels.LinearMixedModel — Method
LinearMixedModel(y, feterm, reterms, form, wts=[], σ=nothing; amalgamate=true)
Private constructor for a LinearMixedModel given already assembled fixed and random effects.
To construct a model, you only need a vector of FeMats (the fixed-effects model matrix and response), a vector of AbstractReMat (the random-effects model matrices), the formula and the weights. Everything else in the structure can be derived from these quantities.
Note
This method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.
source
MixedModels.MixedModel — Type
MixedModel
Abstract type for mixed models.  MixedModels.jl implements two subtypes: LinearMixedModel and GeneralizedLinearMixedModel.  See the documentation for each for more details.
This type is primarily used for dispatch in fit.  Without a distribution and link function specified, a LinearMixedModel will be fit.  When a distribution/link function is provided, a GeneralizedLinearModel is fit, unless that distribution is Normal and the link is IdentityLink, in which case the resulting GLMM would be equivalent to a LinearMixedModel anyway and so the simpler, equivalent LinearMixedModel will be fit instead.
source
MixedModels.MixedModelBootstrap — Type
MixedModelBootstrap{T<:AbstractFloat} <: MixedModelFitCollection{T}
Object returned by parametericbootstrap with fields
  • fits: the parameter estimates from the bootstrap replicates as a vector of named tuples.
  • λ: Vector{LowerTriangular{T,Matrix{T}}} containing copies of the λ field from ReMat model terms
  • inds: Vector{Vector{Int}} containing copies of the inds field from ReMat model terms
  • lowerbd: Vector{T} containing the vector of lower bounds (corresponds to the identically named field of OptSummary)
  • fcnames: NamedTuple whose keys are the grouping factor names and whose values are the column names
The schema of fits is, by default,
Tables.Schema:
+julia> schema((; grp = string.(1:100_000)), Dict(:grp => Grouping()))
source
MixedModels.LikelihoodRatioTest — Type
LikelihoodRatioTest
Results of MixedModels.likelihoodratiotest
Fields
  • formulas: Vector of model formulae
  • models: NamedTuple of the dof and deviance of the models
  • tests: NamedTuple of the sequential dofdiff, deviancediff,          and resulting pvalues
Properties
  • deviance : note that this is actually -2 log likelihood for linear models              (i.e. without subtracting the constant for a saturated model)
  • pvalues
source
MixedModels.LinearMixedModel — Type
LinearMixedModel(y, Xs, form, wts=[], σ=nothing, amalgamate=true)
Private constructor for a LinearMixedModel.
To construct a model, you only need the response (y), already assembled model matrices (Xs), schematized formula (form) and weights (wts). Everything else in the structure can be derived from these quantities.
Note
This method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.
source
MixedModels.LinearMixedModel — Type
LinearMixedModel
Linear mixed-effects model representation
Fields
  • formula: the formula for the model
  • reterms: a Vector{AbstractReMat{T}} of random-effects terms.
  • Xymat: horizontal concatenation of a full-rank fixed-effects model matrix X and response y as an FeMat{T}
  • feterm: the fixed-effects model matrix as an FeTerm{T}
  • sqrtwts: vector of square roots of the case weights.  Can be empty.
  • parmap : Vector{NTuple{3,Int}} of (block, row, column) mapping of θ to λ
  • dims : NamedTuple{(:n, :p, :nretrms),NTuple{3,Int}} of dimensions.  p is the rank of X, which may be smaller than size(X, 2).
  • A: a Vector{AbstractMatrix} containing the row-major packed lower triangle of hcat(Z,X,y)'hcat(Z,X,y)
  • L: the blocked lower Cholesky factor of Λ'AΛ+I in the same Vector representation as A
  • optsum: an OptSummary object
Properties
  • θ or theta: the covariance parameter vector used to form λ
  • β or beta: the fixed-effects coefficient vector
  • λ or lambda: a vector of lower triangular matrices repeated on the diagonal blocks of Λ
  • σ or sigma: current value of the standard deviation of the per-observation noise
  • b: random effects on the original scale, as a vector of matrices
  • u: random effects on the orthogonal scale, as a vector of matrices
  • lowerbd: lower bounds on the elements of θ
  • X: the fixed-effects model matrix
  • y: the response vector
source
MixedModels.LinearMixedModel — Method
LinearMixedModel(y, feterm, reterms, form, wts=[], σ=nothing; amalgamate=true)
Private constructor for a LinearMixedModel given already assembled fixed and random effects.
To construct a model, you only need a vector of FeMats (the fixed-effects model matrix and response), a vector of AbstractReMat (the random-effects model matrices), the formula and the weights. Everything else in the structure can be derived from these quantities.
Note
This method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.
source
MixedModels.MixedModel — Type
MixedModel
Abstract type for mixed models.  MixedModels.jl implements two subtypes: LinearMixedModel and GeneralizedLinearMixedModel.  See the documentation for each for more details.
This type is primarily used for dispatch in fit.  Without a distribution and link function specified, a LinearMixedModel will be fit.  When a distribution/link function is provided, a GeneralizedLinearModel is fit, unless that distribution is Normal and the link is IdentityLink, in which case the resulting GLMM would be equivalent to a LinearMixedModel anyway and so the simpler, equivalent LinearMixedModel will be fit instead.
source
MixedModels.MixedModelBootstrap — Type
MixedModelBootstrap{T<:AbstractFloat} <: MixedModelFitCollection{T}
Object returned by parametericbootstrap with fields
  • fits: the parameter estimates from the bootstrap replicates as a vector of named tuples.
  • λ: Vector{LowerTriangular{T,Matrix{T}}} containing copies of the λ field from ReMat model terms
  • inds: Vector{Vector{Int}} containing copies of the inds field from ReMat model terms
  • lowerbd: Vector{T} containing the vector of lower bounds (corresponds to the identically named field of OptSummary)
  • fcnames: NamedTuple whose keys are the grouping factor names and whose values are the column names
The schema of fits is, by default,
Tables.Schema:
  :objective  T
  :σ          T
  :β          NamedTuple{β_names}{NTuple{p,T}}
  :se         StaticArrays.SArray{Tuple{p},T,1,p}
- :θ          StaticArrays.SArray{Tuple{k},T,1,k}
where the sizes, p and k, of the β and θ elements are determined by the model.
Characteristics of the bootstrap replicates can be extracted as properties.  The σs and σρs properties unravel the σ and θ estimates into estimates of the standard deviations and correlations of the random-effects terms.
source
MixedModels.MixedModelProfile — Type
 MixedModelProfile{T<:AbstractFloat}
Type representing a likelihood profile of a LinearMixedModel, including associated interpolation splines.
The function profile is used for computing profiles, while confint provides a useful method for constructing confidence intervals from a MixedModelProfile.
Note
The exact fields and their representation are considered implementation details and are not part of the public API.
source
MixedModels.OptSummary — Type
OptSummary
Summary of an NLopt optimization
Fields
  • initial: a copy of the initial parameter values in the optimization
  • finitial: the initial value of the objective
  • lowerbd: lower bounds on the parameter values
  • ftol_rel: as in NLopt
  • ftol_abs: as in NLopt
  • xtol_rel: as in NLopt
  • xtol_abs: as in NLopt
  • initial_step: as in NLopt
  • maxfeval: as in NLopt (maxeval)
  • maxtime: as in NLopt
  • final: a copy of the final parameter values from the optimization
  • fmin: the final value of the objective
  • feval: the number of function evaluations
  • optimizer: the name of the optimizer used, as a Symbol
  • returnvalue: the return value, as a Symbol
  • xtol_zero_abs: the tolerance for a near zero parameter to be considered practically zero
  • ftol_zero_abs: the tolerance for change in the objective for setting a near zero parameter to zero
  • fitlog: A vector of tuples of parameter and objectives values from steps in the optimization
  • nAGQ: number of adaptive Gauss-Hermite quadrature points in deviance evaluation for GLMMs
  • REML: use the REML criterion for LMM fits
  • sigma: a priori value for the residual standard deviation for LMM
The last three fields are MixedModels functionality and not related directly to the NLopt package or algorithms.
Note
The internal storage of the parameter values within fitlog may change in the future to use a different subtype of AbstractVector (e.g., StaticArrays.SVector) for each snapshot without being considered a breaking change.
source
MixedModels.PCA — Type
PCA{T<:AbstractFloat}
Principal Components Analysis
Fields
  • covcorr covariance or correlation matrix
  • sv singular value decomposition
  • rnames rownames of the original matrix
  • corr is this a correlation matrix?
source
MixedModels.RaggedArray — Type
RaggedArray{T,I}
A "ragged" array structure consisting of values and indices
Fields
  • vals: a Vector{T} containing the values
  • inds: a Vector{I} containing the indices
For this application a RaggedArray is used only in its sum! method.
source
MixedModels.ReMat — Type
ReMat{T,S} <: AbstractMatrix{T}
A section of a model matrix generated by a random-effects term.
Fields
  • trm: the grouping factor as a StatsModels.CategoricalTerm
  • refs: indices into the levels of the grouping factor as a Vector{Int32}
  • levels: the levels of the grouping factor
  • cnames: the names of the columns of the model matrix generated by the left-hand side of the term
  • z: transpose of the model matrix generated by the left-hand side of the term
  • wtz: a weighted copy of z (z and wtz are the same object for unweighted cases)
  • λ: a LowerTriangular or Diagonal matrix of size S×S
  • inds: a Vector{Int} of linear indices of the potential nonzeros in λ
  • adjA: the adjoint of the matrix as a SparseMatrixCSC{T}
  • scratch: a Matrix{T}
source
MixedModels.TableColumns — Type
TableColumns
A structure containing the column names for the numeric part of the profile table.
The struct also contains a Dict giving the column ranges for Symbols like :σ and :β. Finally it contains a scratch vector used to accumulate to values in a row of the profile table.
Note
This is an internal structure used in MixedModelProfile. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.VarCorr — Type
VarCorr
Information from the fitted random-effects variance-covariance matrices.
Members
  • σρ: a NamedTuple of NamedTuples as returned from σρs
  • s: the estimate of the per-observation dispersion parameter
The main purpose of defining this type is to isolate the logic in the show method.
source
Exported Functions
LinearAlgebra.cond — Method
cond(m::MixedModel)
Return a vector of condition numbers of the λ matrices for the random-effects terms
source
LinearAlgebra.logdet — Method
logdet(m::LinearMixedModel)
Return the value of log(det(Λ'Z'ZΛ + I)) + m.optsum.REML * log(det(LX*LX')) evaluated in place.
Here LX is the diagonal term corresponding to the fixed-effects in the blocked lower Cholesky factor.
source
MixedModels.GHnorm — Method
GHnorm(k::Int)
Return the (unique) GaussHermiteNormalized{k} object.
The function values are stored (memoized) when first evaluated.  Subsequent evaluations for the same k have very low overhead.
source
MixedModels.coefpvalues — Method
coefpvalues(bsamp::MixedModelFitCollection)
Return a rowtable with columns (:iter, :coefname, :β, :se, :z, :p)
source
MixedModels.condVar — Method
condVar(m::LinearMixedModel)
Return the conditional variances matrices of the random effects.
The random effects are returned by ranef as a vector of length k, where k is the number of random effects terms.  The ith element is a matrix of size vᵢ × ℓᵢ  where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor.  Technically those values are the modes of the conditional distribution of the random effects given the observed data.
This function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ.  These are the diagonal blocks from the conditional variance-covariance matrix,
s² Λ(Λ'Z'ZΛ + I)⁻¹Λ'
source
MixedModels.condVartables — Method
condVartables(m::LinearMixedModel)
Return the conditional covariance matrices of the random effects as a NamedTuple of columntables
source
MixedModels.fitted! — Method
fitted!(v::AbstractArray{T}, m::LinearMixedModel{T})
Overwrite v with the fitted values from m.
See also fitted.
source
MixedModels.fixef — Method
fixef(m::MixedModel)
Return the fixed-effects parameter vector estimate of m.
In the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.
source
MixedModels.fixefnames — Method
fixefnames(m::MixedModel)
Return a (permuted and truncated in the rank-deficient case) vector of coefficient names.
source
MixedModels.fnames — Method
fnames(m::MixedModel)
Return the names of the grouping factors for the random-effects terms.
source
MixedModels.fulldummy — Method
fulldummy(term::CategoricalTerm)
Assign "contrasts" that include all indicator columns (dummy variables) and an intercept column.
This will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or "shrinkage", of the conditional modes.
The interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.
source
MixedModels.issingular — Function
issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)
Test whether the model m is singular if the parameter vector is θ.
Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.
Note
For GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.
source
MixedModels.issingular — Method
issingular(bsamp::MixedModelFitCollection;
-           atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))
Test each bootstrap sample for singularity of the corresponding fit.
Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.
See also issingular(::MixedModel).
source
MixedModels.lowerbd — Method
lowerbd{T}(A::ReMat{T})
Return the vector of lower bounds on the parameters, θ associated with A
These are the elements in the lower triangle of A.λ in column-major ordering. Diagonals have a lower bound of 0.  Off-diagonals have a lower-bound of -Inf.
source
MixedModels.objective! — Function
objective!(m::LinearMixedModel, θ)
-objective!(m::LinearMixedModel)
Equivalent to objective(updateL!(setθ!(m, θ))).
When m has a single, scalar random-effects term, θ can be a scalar.
The one-argument method curries and returns a single-argument function of θ.
Note that these methods modify m. The calling function is responsible for restoring the optimal θ.
source
MixedModels.parametricbootstrap — Method
parametricbootstrap([rng::AbstractRNG], nsamp::Integer, m::MixedModel{T}, ftype=T;
-    β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))
Perform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.
The default random number generator is Random.GLOBAL_RNG.
ftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.
Keyword Arguments
  • β, σ, and θ are the values of m's parameters for simulating the responses.
  • σ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for
families with a dispersion parameter.
  • progress controls whether the progress bar is shown. Note that the progress
bar is automatically disabled for non-interactive (i.e. logging) contexts.
  • optsum_overrides is used to override values of OptSummary in the models
fit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.
Note
All coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.
source
MixedModels.pirls! — Method
pirls!(m::GeneralizedLinearMixedModel)
Use Penalized Iteratively Reweighted Least Squares (PIRLS) to determine the conditional modes of the random effects.
When varyβ is true both u and β are optimized with PIRLS.  Otherwise only u is optimized and β is held fixed.
Passing verbose = true provides verbose output of the iterations.
source
MixedModels.profile — Method
profile(m::LinearMixedModel; threshold = 4)
Return a MixedModelProfile for the objective of m with respect to the fixed-effects coefficients.
m is refit! if !isfitted(m).
Profiling starts at the parameter estimate and continues until reaching a parameter bound or the absolute value of ζ exceeds threshold.
source
MixedModels.profilevc — Method
 profilevc(m::LinearMixedModel{T}, val::T, rowj::AbstractVector{T}) where {T}
Profile an element of the variance components.
Note
This method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.profileσ — Method
profileσ(m::LinearMixedModel, tc::TableColumns; threshold=4)
Return a Table of the profile of σ for model m.  The profile extends to where the magnitude of ζ exceeds threshold.
Note
This method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.ranef — Method
ranef(m::LinearMixedModel; uscale=false)
Return, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.
If uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.
For a named variant, see raneftables.
source
MixedModels.raneftables — Method
raneftables(m::MixedModel; uscale = false)
Return the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.
Note
The API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.
source
MixedModels.refit! — Method
refit!(m::GeneralizedLinearMixedModel[, y::Vector];
+ :θ          StaticArrays.SArray{Tuple{k},T,1,k}
where the sizes, p and k, of the β and θ elements are determined by the model.
Characteristics of the bootstrap replicates can be extracted as properties.  The σs and σρs properties unravel the σ and θ estimates into estimates of the standard deviations and correlations of the random-effects terms.
source
MixedModels.MixedModelProfile — Type
 MixedModelProfile{T<:AbstractFloat}
Type representing a likelihood profile of a LinearMixedModel, including associated interpolation splines.
The function profile is used for computing profiles, while confint provides a useful method for constructing confidence intervals from a MixedModelProfile.
Note
The exact fields and their representation are considered implementation details and are not part of the public API.
source
MixedModels.OptSummary — Type
OptSummary
Summary of an NLopt optimization
Fields
  • initial: a copy of the initial parameter values in the optimization
  • finitial: the initial value of the objective
  • lowerbd: lower bounds on the parameter values
  • ftol_rel: as in NLopt
  • ftol_abs: as in NLopt
  • xtol_rel: as in NLopt
  • xtol_abs: as in NLopt
  • initial_step: as in NLopt
  • maxfeval: as in NLopt (maxeval)
  • maxtime: as in NLopt
  • final: a copy of the final parameter values from the optimization
  • fmin: the final value of the objective
  • feval: the number of function evaluations
  • optimizer: the name of the optimizer used, as a Symbol
  • returnvalue: the return value, as a Symbol
  • xtol_zero_abs: the tolerance for a near zero parameter to be considered practically zero
  • ftol_zero_abs: the tolerance for change in the objective for setting a near zero parameter to zero
  • fitlog: A vector of tuples of parameter and objectives values from steps in the optimization
  • nAGQ: number of adaptive Gauss-Hermite quadrature points in deviance evaluation for GLMMs
  • REML: use the REML criterion for LMM fits
  • sigma: a priori value for the residual standard deviation for LMM
The last three fields are MixedModels functionality and not related directly to the NLopt package or algorithms.
Note
The internal storage of the parameter values within fitlog may change in the future to use a different subtype of AbstractVector (e.g., StaticArrays.SVector) for each snapshot without being considered a breaking change.
source
MixedModels.PCA — Type
PCA{T<:AbstractFloat}
Principal Components Analysis
Fields
  • covcorr covariance or correlation matrix
  • sv singular value decomposition
  • rnames rownames of the original matrix
  • corr is this a correlation matrix?
source
MixedModels.RaggedArray — Type
RaggedArray{T,I}
A "ragged" array structure consisting of values and indices
Fields
  • vals: a Vector{T} containing the values
  • inds: a Vector{I} containing the indices
For this application a RaggedArray is used only in its sum! method.
source
MixedModels.ReMat — Type
ReMat{T,S} <: AbstractMatrix{T}
A section of a model matrix generated by a random-effects term.
Fields
  • trm: the grouping factor as a StatsModels.CategoricalTerm
  • refs: indices into the levels of the grouping factor as a Vector{Int32}
  • levels: the levels of the grouping factor
  • cnames: the names of the columns of the model matrix generated by the left-hand side of the term
  • z: transpose of the model matrix generated by the left-hand side of the term
  • wtz: a weighted copy of z (z and wtz are the same object for unweighted cases)
  • λ: a LowerTriangular or Diagonal matrix of size S×S
  • inds: a Vector{Int} of linear indices of the potential nonzeros in λ
  • adjA: the adjoint of the matrix as a SparseMatrixCSC{T}
  • scratch: a Matrix{T}
source
MixedModels.TableColumns — Type
TableColumns
A structure containing the column names for the numeric part of the profile table.
The struct also contains a Dict giving the column ranges for Symbols like :σ and :β. Finally it contains a scratch vector used to accumulate to values in a row of the profile table.
Note
This is an internal structure used in MixedModelProfile. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.VarCorr — Type
VarCorr
Information from the fitted random-effects variance-covariance matrices.
Members
  • σρ: a NamedTuple of NamedTuples as returned from σρs
  • s: the estimate of the per-observation dispersion parameter
The main purpose of defining this type is to isolate the logic in the show method.
source
Exported Functions
LinearAlgebra.cond — Method
cond(m::MixedModel)
Return a vector of condition numbers of the λ matrices for the random-effects terms
source
LinearAlgebra.logdet — Method
logdet(m::LinearMixedModel)
Return the value of log(det(Λ'Z'ZΛ + I)) + m.optsum.REML * log(det(LX*LX')) evaluated in place.
Here LX is the diagonal term corresponding to the fixed-effects in the blocked lower Cholesky factor.
source
MixedModels.GHnorm — Method
GHnorm(k::Int)
Return the (unique) GaussHermiteNormalized{k} object.
The function values are stored (memoized) when first evaluated.  Subsequent evaluations for the same k have very low overhead.
source
MixedModels.coefpvalues — Method
coefpvalues(bsamp::MixedModelFitCollection)
Return a rowtable with columns (:iter, :coefname, :β, :se, :z, :p)
source
MixedModels.condVar — Method
condVar(m::LinearMixedModel)
Return the conditional variances matrices of the random effects.
The random effects are returned by ranef as a vector of length k, where k is the number of random effects terms.  The ith element is a matrix of size vᵢ × ℓᵢ  where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor.  Technically those values are the modes of the conditional distribution of the random effects given the observed data.
This function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ.  These are the diagonal blocks from the conditional variance-covariance matrix,
s² Λ(Λ'Z'ZΛ + I)⁻¹Λ'
source
MixedModels.condVartables — Method
condVartables(m::LinearMixedModel)
Return the conditional covariance matrices of the random effects as a NamedTuple of columntables
source
MixedModels.fitted! — Method
fitted!(v::AbstractArray{T}, m::LinearMixedModel{T})
Overwrite v with the fitted values from m.
See also fitted.
source
MixedModels.fixef — Method
fixef(m::MixedModel)
Return the fixed-effects parameter vector estimate of m.
In the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.
source
MixedModels.fixefnames — Method
fixefnames(m::MixedModel)
Return a (permuted and truncated in the rank-deficient case) vector of coefficient names.
source
MixedModels.fnames — Method
fnames(m::MixedModel)
Return the names of the grouping factors for the random-effects terms.
source
MixedModels.fulldummy — Method
fulldummy(term::CategoricalTerm)
Assign "contrasts" that include all indicator columns (dummy variables) and an intercept column.
This will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or "shrinkage", of the conditional modes.
The interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.
source
MixedModels.issingular — Function
issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)
Test whether the model m is singular if the parameter vector is θ.
Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.
Note
For GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.
source
MixedModels.issingular — Method
issingular(bsamp::MixedModelFitCollection;
+           atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))
Test each bootstrap sample for singularity of the corresponding fit.
Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.
See also issingular(::MixedModel).
source
MixedModels.lowerbd — Method
lowerbd{T}(A::ReMat{T})
Return the vector of lower bounds on the parameters, θ associated with A
These are the elements in the lower triangle of A.λ in column-major ordering. Diagonals have a lower bound of 0.  Off-diagonals have a lower-bound of -Inf.
source
MixedModels.objective! — Function
objective!(m::LinearMixedModel, θ)
+objective!(m::LinearMixedModel)
Equivalent to objective(updateL!(setθ!(m, θ))).
When m has a single, scalar random-effects term, θ can be a scalar.
The one-argument method curries and returns a single-argument function of θ.
Note that these methods modify m. The calling function is responsible for restoring the optimal θ.
source
MixedModels.parametricbootstrap — Method
parametricbootstrap([rng::AbstractRNG], nsamp::Integer, m::MixedModel{T}, ftype=T;
+    β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))
Perform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.
The default random number generator is Random.GLOBAL_RNG.
ftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.
Keyword Arguments
  • β, σ, and θ are the values of m's parameters for simulating the responses.
  • σ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for
families with a dispersion parameter.
  • progress controls whether the progress bar is shown. Note that the progress
bar is automatically disabled for non-interactive (i.e. logging) contexts.
  • optsum_overrides is used to override values of OptSummary in the models
fit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.
Note
All coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.
source
MixedModels.pirls! — Method
pirls!(m::GeneralizedLinearMixedModel)
Use Penalized Iteratively Reweighted Least Squares (PIRLS) to determine the conditional modes of the random effects.
When varyβ is true both u and β are optimized with PIRLS.  Otherwise only u is optimized and β is held fixed.
Passing verbose = true provides verbose output of the iterations.
source
MixedModels.profile — Method
profile(m::LinearMixedModel; threshold = 4)
Return a MixedModelProfile for the objective of m with respect to the fixed-effects coefficients.
m is refit! if !isfitted(m).
Profiling starts at the parameter estimate and continues until reaching a parameter bound or the absolute value of ζ exceeds threshold.
source
MixedModels.profilevc — Method
 profilevc(m::LinearMixedModel{T}, val::T, rowj::AbstractVector{T}) where {T}
Profile an element of the variance components.
Note
This method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.profileσ — Method
profileσ(m::LinearMixedModel, tc::TableColumns; threshold=4)
Return a Table of the profile of σ for model m.  The profile extends to where the magnitude of ζ exceeds threshold.
Note
This method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.ranef — Method
ranef(m::LinearMixedModel; uscale=false)
Return, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.
If uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.
For a named variant, see raneftables.
source
MixedModels.raneftables — Method
raneftables(m::MixedModel; uscale = false)
Return the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.
Note
The API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.
source
MixedModels.refit! — Method
refit!(m::GeneralizedLinearMixedModel[, y::Vector];
        fast::Bool = (length(m.θ) == length(m.optsum.final)),
        nAGQ::Integer = m.optsum.nAGQ,
-       kwargs...)
Refit the model m after installing response y.
If y is omitted the current response vector is used.
If not specified, the fast and nAGQ options from the previous fit are used. kwargs are the same as fit!
source
MixedModels.refit! — Method
refit!(m::LinearMixedModel[, y::Vector]; REML=m.optsum.REML, kwargs...)
Refit the model m after installing response y.
If y is omitted the current response vector is used. kwargs are the same as fit!.
source
MixedModels.replicate — Method
replicate(f::Function, n::Integer; progress=true)
Return a vector of the values of n calls to f() - used in simulations where the value of f is stochastic.
progress controls whether the progress bar is shown. Note that the progress bar is automatically disabled for non-interactive (i.e. logging) contexts.
source
MixedModels.restoreoptsum! — Method
restoreoptsum!(m::MixedModel, io::IO; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)
-restoreoptsum!(m::MixedModel, filename; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)
Read, check, and restore the optsum field from a JSON stream or filename.
source
MixedModels.restorereplicates — Method
restorereplicates(f, m::MixedModel{T})
+       kwargs...)
Refit the model m after installing response y.
If y is omitted the current response vector is used.
If not specified, the fast and nAGQ options from the previous fit are used. kwargs are the same as fit!
source
MixedModels.refit! — Method
refit!(m::LinearMixedModel[, y::Vector]; REML=m.optsum.REML, kwargs...)
Refit the model m after installing response y.
If y is omitted the current response vector is used. kwargs are the same as fit!.
source
MixedModels.replicate — Method
replicate(f::Function, n::Integer; progress=true)
Return a vector of the values of n calls to f() - used in simulations where the value of f is stochastic.
progress controls whether the progress bar is shown. Note that the progress bar is automatically disabled for non-interactive (i.e. logging) contexts.
source
MixedModels.restoreoptsum! — Method
restoreoptsum!(m::MixedModel, io::IO; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)
+restoreoptsum!(m::MixedModel, filename; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)
Read, check, and restore the optsum field from a JSON stream or filename.
source
MixedModels.restorereplicates — Method
restorereplicates(f, m::MixedModel{T})
 restorereplicates(f, m::MixedModel{T}, ftype::Type{<:AbstractFloat})
-restorereplicates(f, m::MixedModel{T}, ctype::Type{<:MixedModelFitCollection{S}})
Restore replicates from f, using m to create the desired subtype of MixedModelFitCollection.
f can be any entity supported by Arrow.Table. m does not have to be fitted, but it must have been constructed with the same structure as the source of the saved replicates.
The two-argument method constructs a MixedModelBootstrap with the same eltype as m. If an eltype is specified as the third argument, then a MixedModelBootstrap is returned. If a subtype of MixedModelFitCollection is specified as the third argument, then that is the return type.
See also savereplicates, restoreoptsum!.
source
MixedModels.saveoptsum — Method
saveoptsum(io::IO, m::MixedModel)
-saveoptsum(filename, m::MixedModel)
Save m.optsum (w/o the lowerbd field) in JSON format to an IO stream or a file
The reason for omitting the lowerbd field is because it often contains -Inf values that are not allowed in JSON.
source
MixedModels.sdest — Method
sdest(m::LinearMixedModel)
Return the estimate of σ, the standard deviation of the per-observation noise.
source
MixedModels.sdest — Method
sdest(m::GeneralizedLinearMixedModel)
Return the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.
For models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.
For Gaussian models, this parameter is often called σ.
source
MixedModels.setθ! — Method
setθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)
-setθ!(bsamp::MixedModelFitCollection, i::Integer)
Install the values of the i'th θ value of bsamp.fits in bsamp.λ
source
MixedModels.shortestcovint — Method
shortestcovint(bsamp::MixedModelFitCollection, level = 0.95)
Return the shortest interval containing level proportion for each parameter from bsamp.allpars.
Warning
Currently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.
source
MixedModels.simulate! — Method
simulate!([rng::AbstractRNG,] y::AbstractVector, m::MixedModel{T}[, newdata];
+restorereplicates(f, m::MixedModel{T}, ctype::Type{<:MixedModelFitCollection{S}})
Restore replicates from f, using m to create the desired subtype of MixedModelFitCollection.
f can be any entity supported by Arrow.Table. m does not have to be fitted, but it must have been constructed with the same structure as the source of the saved replicates.
The two-argument method constructs a MixedModelBootstrap with the same eltype as m. If an eltype is specified as the third argument, then a MixedModelBootstrap is returned. If a subtype of MixedModelFitCollection is specified as the third argument, then that is the return type.
See also savereplicates, restoreoptsum!.
source
MixedModels.saveoptsum — Method
saveoptsum(io::IO, m::MixedModel)
+saveoptsum(filename, m::MixedModel)
Save m.optsum (w/o the lowerbd field) in JSON format to an IO stream or a file
The reason for omitting the lowerbd field is because it often contains -Inf values that are not allowed in JSON.
source
MixedModels.sdest — Method
sdest(m::LinearMixedModel)
Return the estimate of σ, the standard deviation of the per-observation noise.
source
MixedModels.sdest — Method
sdest(m::GeneralizedLinearMixedModel)
Return the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.
For models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.
For Gaussian models, this parameter is often called σ.
source
MixedModels.setθ! — Method
setθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)
+setθ!(bsamp::MixedModelFitCollection, i::Integer)
Install the values of the i'th θ value of bsamp.fits in bsamp.λ
source
MixedModels.shortestcovint — Method
shortestcovint(bsamp::MixedModelFitCollection, level = 0.95)
Return the shortest interval containing level proportion for each parameter from bsamp.allpars.
Warning
Currently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.
source
MixedModels.simulate! — Method
simulate!([rng::AbstractRNG,] y::AbstractVector, m::MixedModel{T}[, newdata];
                 β = coef(m), σ = m.σ, θ = T[], wts=m.wts)
 simulate([rng::AbstractRNG,] m::MixedModel{T}[, newdata];
-                β = coef(m), σ = m.σ, θ = T[], wts=m.wts)
Simulate a new response vector, optionally overwriting a pre-allocated vector.
New data can be optionally provided in tabular format.
This simulation includes sampling new values for the random effects. Thus in contrast to predict, there is no distinction in between "new" and "old" / previously observed random-effects levels.
Unlike predict, there is no type parameter for GeneralizedLinearMixedModel because the noise term in the model and simulation is always on the response scale.
The wts argument is currently ignored except for GeneralizedLinearMixedModel models with a Binomial distribution.
Note
Note that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.
source
MixedModels.simulate! — Method
simulate!(rng::AbstractRNG, m::MixedModel{T}; β=fixef(m), σ=m.σ, θ=T[])
-simulate!(m::MixedModel; β=fixef(m), σ=m.σ, θ=m.θ)
Overwrite the response (i.e. m.trms[end]) with a simulated response vector from model m.
This simulation includes sampling new values for the random effects.
β can be specified either as a pivoted, full rank coefficient vector (cf. fixef) or as an unpivoted full dimension coefficient vector (cf. coef), where the entries corresponding to redundant columns will be ignored.
Note
Note that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.
source
MixedModels.sparseL — Method
sparseL(m::LinearMixedModel; fname::Symbol=first(fnames(m)), full::Bool=false)
Return the lower Cholesky factor L as a SparseMatrix{T,Int32}.
full indicates whether the parts of L associated with the fixed-effects and response are to be included.
fname specifies the first grouping factor to include. Blocks to the left of the block corresponding  to fname are dropped. The default is the first, i.e., leftmost block and hence all blocks.
source
MixedModels.stderror! — Method
stderror!(v::AbstractVector, m::LinearMixedModel)
Overwrite v with the standard errors of the fixed-effects coefficients in m
The length of v should be the total number of coefficients (i.e. length(coef(m))). When the model matrix is rank-deficient the coefficients forced to -0.0 have an undefined (i.e. NaN) standard error.
source
MixedModels.updateL! — Method
updateL!(m::LinearMixedModel)
Update the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)
This is the crucial step in evaluating the objective, given a new parameter value.
source
MixedModels.varest — Method
varest(m::LinearMixedModel)
Returns the estimate of σ², the variance of the conditional distribution of Y given B.
source
MixedModels.varest — Method
varest(m::GeneralizedLinearMixedModel)
Returns the estimate of ϕ², the variance of the conditional distribution of Y given B.
For models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.
For Gaussian models, this parameter is often called σ².
source
Statistics.std — Method
std(m::MixedModel)
Return the estimated standard deviations of the random effects as a Vector{Vector{T}}.
FIXME: This uses an old convention of isfinite(sdest(m)).  Probably drop in favor of m.σs
source
StatsAPI.confint — Method
confint(pr::MixedModelProfile; level::Real=0.95)
Compute profile confidence intervals for coefficients and variance components, with confidence level level (by default 95%).
Note
The API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.
Note
The "row names" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.
source
StatsAPI.confint — Method
confint(pr::MixedModelBootstrap; level::Real=0.95, method=:shortest)
Compute bootstrap confidence intervals for coefficients and variance components, with confidence level level (by default 95%).
The keyword argument method determines whether the :shortest, i.e. highest density, interval is used or the :equaltail, i.e. quantile-based, interval is used. For historical reasons, the default is :shortest, but :equaltail gives the interval that is most comparable to the profile and Wald confidence intervals.
Note
The API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.
Note
The "row names" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.
See also shortestcovint.
source
StatsAPI.confint — Method
confint(pr::MixedModelProfile; level::Real=0.95)
Compute profile confidence intervals for (fixed effects) coefficients, with confidence level level (by default 95%).
Note
The API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.
source
StatsAPI.deviance — Method
deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}
Return the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.
If the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, $u$, plus the determinant of $Λ'Z'WZΛ + I$, plus the sum of the squared deviance residuals.
source
StatsAPI.dof_residual — Method
dof_residual(m::MixedModel)
Return the residual degrees of freedom of the model.
Note
The residual degrees of freedom for mixed-effects models is not clearly defined due to partial pooling. The classical nobs(m) - dof(m) fails to capture the extra freedom granted by the random effects, but nobs(m) - nranef(m) would overestimate the freedom granted by the random effects. nobs(m) - sum(leverage(m)) provides a nice balance based on the relative influence of each observation, but is computationally expensive for large models. This problem is also fundamentally related to long-standing debates about the appropriate treatment of the denominator degrees of freedom for $F$-tests. In the future, MixedModels.jl may provide additional methods allowing the user to choose the computation to use.
Warning
Currently, the residual degrees of freedom is computed as nobs(m) - dof(m), but this may change in the future without being considered a breaking change because there is no canonical definition of the residual degrees of freedom in a mixed-effects model.
source
StatsAPI.fit! — Method
fit!(m::GeneralizedLinearMixedModel; fast=false, nAGQ=1,
+                β = coef(m), σ = m.σ, θ = T[], wts=m.wts)
Simulate a new response vector, optionally overwriting a pre-allocated vector.
New data can be optionally provided in tabular format.
This simulation includes sampling new values for the random effects. Thus in contrast to predict, there is no distinction in between "new" and "old" / previously observed random-effects levels.
Unlike predict, there is no type parameter for GeneralizedLinearMixedModel because the noise term in the model and simulation is always on the response scale.
The wts argument is currently ignored except for GeneralizedLinearMixedModel models with a Binomial distribution.
Note
Note that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.
source
MixedModels.simulate! — Method
simulate!(rng::AbstractRNG, m::MixedModel{T}; β=fixef(m), σ=m.σ, θ=T[])
+simulate!(m::MixedModel; β=fixef(m), σ=m.σ, θ=m.θ)
Overwrite the response (i.e. m.trms[end]) with a simulated response vector from model m.
This simulation includes sampling new values for the random effects.
β can be specified either as a pivoted, full rank coefficient vector (cf. fixef) or as an unpivoted full dimension coefficient vector (cf. coef), where the entries corresponding to redundant columns will be ignored.
Note
Note that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.
source
MixedModels.sparseL — Method
sparseL(m::LinearMixedModel; fname::Symbol=first(fnames(m)), full::Bool=false)
Return the lower Cholesky factor L as a SparseMatrix{T,Int32}.
full indicates whether the parts of L associated with the fixed-effects and response are to be included.
fname specifies the first grouping factor to include. Blocks to the left of the block corresponding  to fname are dropped. The default is the first, i.e., leftmost block and hence all blocks.
source
MixedModels.stderror! — Method
stderror!(v::AbstractVector, m::LinearMixedModel)
Overwrite v with the standard errors of the fixed-effects coefficients in m
The length of v should be the total number of coefficients (i.e. length(coef(m))). When the model matrix is rank-deficient the coefficients forced to -0.0 have an undefined (i.e. NaN) standard error.
source
MixedModels.updateL! — Method
updateL!(m::LinearMixedModel)
Update the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)
This is the crucial step in evaluating the objective, given a new parameter value.
source
MixedModels.varest — Method
varest(m::LinearMixedModel)
Returns the estimate of σ², the variance of the conditional distribution of Y given B.
source
MixedModels.varest — Method
varest(m::GeneralizedLinearMixedModel)
Returns the estimate of ϕ², the variance of the conditional distribution of Y given B.
For models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.
For Gaussian models, this parameter is often called σ².
source
Statistics.std — Method
std(m::MixedModel)
Return the estimated standard deviations of the random effects as a Vector{Vector{T}}.
FIXME: This uses an old convention of isfinite(sdest(m)).  Probably drop in favor of m.σs
source
StatsAPI.confint — Method
confint(pr::MixedModelProfile; level::Real=0.95)
Compute profile confidence intervals for coefficients and variance components, with confidence level level (by default 95%).
Note
The API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.
Note
The "row names" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.
source
StatsAPI.confint — Method
confint(pr::MixedModelBootstrap; level::Real=0.95, method=:shortest)
Compute bootstrap confidence intervals for coefficients and variance components, with confidence level level (by default 95%).
The keyword argument method determines whether the :shortest, i.e. highest density, interval is used or the :equaltail, i.e. quantile-based, interval is used. For historical reasons, the default is :shortest, but :equaltail gives the interval that is most comparable to the profile and Wald confidence intervals.
Note
The API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.
Note
The "row names" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.
See also shortestcovint.
source
StatsAPI.confint — Method
confint(pr::MixedModelProfile; level::Real=0.95)
Compute profile confidence intervals for (fixed effects) coefficients, with confidence level level (by default 95%).
Note
The API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.
source
StatsAPI.deviance — Method
deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}
Return the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.
If the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, $u$, plus the determinant of $Λ'Z'WZΛ + I$, plus the sum of the squared deviance residuals.
source
StatsAPI.dof_residual — Method
dof_residual(m::MixedModel)
Return the residual degrees of freedom of the model.
Note
The residual degrees of freedom for mixed-effects models is not clearly defined due to partial pooling. The classical nobs(m) - dof(m) fails to capture the extra freedom granted by the random effects, but nobs(m) - nranef(m) would overestimate the freedom granted by the random effects. nobs(m) - sum(leverage(m)) provides a nice balance based on the relative influence of each observation, but is computationally expensive for large models. This problem is also fundamentally related to long-standing debates about the appropriate treatment of the denominator degrees of freedom for $F$-tests. In the future, MixedModels.jl may provide additional methods allowing the user to choose the computation to use.
Warning
Currently, the residual degrees of freedom is computed as nobs(m) - dof(m), but this may change in the future without being considered a breaking change because there is no canonical definition of the residual degrees of freedom in a mixed-effects model.
source
StatsAPI.fit! — Method
fit!(m::GeneralizedLinearMixedModel; fast=false, nAGQ=1,
                                      verbose=false, progress=true,
                                      thin::Int=1,
-                                     init_from_lmm=Set())
Optimize the objective function for m.
When fast is true a potentially much faster but slightly less accurate algorithm, in which pirls! optimizes both the random effects and the fixed-effects parameters, is used.
If progress is true, the default, a ProgressMeter.ProgressUnknown counter is displayed. during the iterations to minimize the deviance.  There is a delay before this display is initialized and it may not be shown at all for models that are optimized quickly.
If verbose is true, then both the intermediate results of both the nonlinear optimization and PIRLS are also displayed on standard output.
At every thinth iteration  is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.
By default, the starting values for model fitting are taken from a (non mixed, i.e. marginal ) GLM fit. Experience with larger datasets (many thousands of observations and/or hundreds of levels of the grouping variables) has suggested that fitting a (Gaussian) linear mixed model on the untransformed data may provide better starting values and thus overall faster fits even though an entire LMM must be fit before the GLMM can be fit. init_from_lmm can be used to specify which starting values from an LMM to use. Valid options are any collection (array, set, etc.) containing one or more of :β and :θ, the default is the empty set.
Note
Initializing from an LMM requires fitting the entire LMM first, so when progress=true, there will be two progress bars: first for the LMM, then for the GLMM.
Warning
The init_from_lmm functionality is experimental and may change or be removed entirely without being considered a breaking change.
source
StatsAPI.fit! — Method
fit!(m::LinearMixedModel; progress::Bool=true, REML::Bool=m.optsum.REML,
+                                     init_from_lmm=Set())
Optimize the objective function for m.
When fast is true a potentially much faster but slightly less accurate algorithm, in which pirls! optimizes both the random effects and the fixed-effects parameters, is used.
If progress is true, the default, a ProgressMeter.ProgressUnknown counter is displayed. during the iterations to minimize the deviance.  There is a delay before this display is initialized and it may not be shown at all for models that are optimized quickly.
If verbose is true, then both the intermediate results of both the nonlinear optimization and PIRLS are also displayed on standard output.
At every thinth iteration  is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.
By default, the starting values for model fitting are taken from a (non mixed, i.e. marginal ) GLM fit. Experience with larger datasets (many thousands of observations and/or hundreds of levels of the grouping variables) has suggested that fitting a (Gaussian) linear mixed model on the untransformed data may provide better starting values and thus overall faster fits even though an entire LMM must be fit before the GLMM can be fit. init_from_lmm can be used to specify which starting values from an LMM to use. Valid options are any collection (array, set, etc.) containing one or more of :β and :θ, the default is the empty set.
Note
Initializing from an LMM requires fitting the entire LMM first, so when progress=true, there will be two progress bars: first for the LMM, then for the GLMM.
Warning
The init_from_lmm functionality is experimental and may change or be removed entirely without being considered a breaking change.
source
StatsAPI.fit! — Method
fit!(m::LinearMixedModel; progress::Bool=true, REML::Bool=m.optsum.REML,
                           σ::Union{Real, Nothing}=m.optsum.sigma,
-                          thin::Int=typemax(Int))
Optimize the objective of a LinearMixedModel.  When progress is true a ProgressMeter.ProgressUnknown display is shown during the optimization of the objective, if the optimization takes more than one second or so.
At every thinth iteration  is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.
source
StatsAPI.leverage — Method
leverage(::LinearMixedModel)
Return the diagonal of the hat matrix of the model.
For a linear model, the sum of the leverage values is the degrees of freedom for the model in the sense that this sum is the dimension of the span of columns of the model matrix.  With a bit of hand waving a similar argument could be made for linear mixed-effects models. The hat matrix is of the form $[ZΛ X][L L']⁻¹[ZΛ X]'$.
source
StatsAPI.modelmatrix — Method
modelmatrix(m::MixedModel)
Returns the model matrix X for the fixed-effects parameters, as returned by coef.
This is always the full model matrix in the original column order and from a field in the model struct.  It should be copied if it is to be modified.
source
StatsAPI.predict — Method
StatsAPI.predict(m::LinearMixedModel, newdata;
+                          thin::Int=typemax(Int))
Optimize the objective of a LinearMixedModel.  When progress is true a ProgressMeter.ProgressUnknown display is shown during the optimization of the objective, if the optimization takes more than one second or so.
At every thinth iteration  is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.
source
StatsAPI.leverage — Method
leverage(::LinearMixedModel)
Return the diagonal of the hat matrix of the model.
For a linear model, the sum of the leverage values is the degrees of freedom for the model in the sense that this sum is the dimension of the span of columns of the model matrix.  With a bit of hand waving a similar argument could be made for linear mixed-effects models. The hat matrix is of the form $[ZΛ X][L L']⁻¹[ZΛ X]'$.
source
StatsAPI.modelmatrix — Method
modelmatrix(m::MixedModel)
Returns the model matrix X for the fixed-effects parameters, as returned by coef.
This is always the full model matrix in the original column order and from a field in the model struct.  It should be copied if it is to be modified.
source
StatsAPI.predict — Method
StatsAPI.predict(m::LinearMixedModel, newdata;
                 new_re_levels=:missing)
 StatsAPI.predict(m::GeneralizedLinearMixedModel, newdata;
-                new_re_levels=:missing, type=:response)
Predict response for new data.
Note
Currently, no in-place methods are provided because these methods internally construct a new model and therefore allocate not just a response vector but also many other matrices.
Warning
newdata should contain a column for the response (dependent variable) initialized to some numerical value (not missing), because this is used to construct the new model used in computing the predictions. missing is not valid because missing data are dropped before constructing the model matrices.
Warning
These methods construct an entire MixedModel behind the scenes and as such may use a large amount of memory when newdata is large.
Warning
Rank-deficiency can lead to surprising but consistent behavior. For example, if there are two perfectly collinear predictors A and B (e.g. constant multiples of each other), then it is possible that A will be pivoted out in the fitted model and thus the associated coefficient is set to zero. If predictions are then generated on new data where B has been set to zero but A has not, then there will no contribution from neither A nor B in the resulting predictions.
The keyword argument new_re_levels specifies how previously unobserved values of the grouping variable are handled. Possible values are:
  • :population: return population values for the relevant grouping variable.  In other words, treat the associated random effect as 0.  If all grouping variables have new levels, then this is equivalent to  just the fixed effects.
  • :missing: return missing.
  • :error: error on this condition. The error type is an implementation detail:  you should not rely on a particular type of error being thrown.
If you want simulated values for unobserved levels of the grouping variable, consider the simulate! and simulate methods.
Predictions based purely on the fixed effects can be obtained by specifying previously unobserved levels of the random effects and setting new_re_levels=:population. Similarly, the contribution of any grouping variable can be excluded by specifying previously unobserved levels, while including previously observed levels of the other grouping variables. In the future, it may be possible to specify a subset of the grouping variables or overall random-effects structure to use, but not at this time.
Note
new_re_levels impacts only the behavior for previously unobserved random effects levels, i.e. new RE levels. For previously observed random effects levels, predictions take both the fixed and random effects into account.
For GeneralizedLinearMixedModel, the type parameter specifies whether the predictions should be returned on the scale of linear predictor (:linpred) or on the response scale (:response). If you don't know the difference between these terms, then you probably want type=:response.
Regression weights are not yet supported in prediction. Similarly, offsets are also not supported for GeneralizedLinearMixedModel.
source
StatsAPI.response — Method
response(m::MixedModel)
Return the response vector for the model.
For a linear mixed model this is a view of the last column of the XyMat field. For a generalized linear mixed model this is the m.resp.y field. In either case it should be copied if it is to be modified.
source
StatsAPI.vcov — Method
vcov(m::MixedModel; corr=false)
Returns the variance-covariance matrix of the fixed effects. If corr is true, the correlation of the fixed effects is returned instead.
source
Tables.columntable — Method
columntable(s::OptSummary, [stack::Bool=false])
Return s.fitlog as a Tables.columntable.
When stack is false (the default), there will be 3 columns in the result:
  • iter: the sample number
  • objective: the value of the objective at that sample
  • θ: the parameter vector at that sample
(The term sample here refers to the fact that when the thin argument to the fit or refit! call is greater than 1 only a subset of the iterations have results recorded.)
When stack is true, there will be 4 columns: iter, objective, par, and value where value is the stacked contents of the θ vectors (the equivalent of vcat(θ...)) and par is a vector of parameter numbers.
source
Methods from StatsAPI.jl, StatsBase.jl, StatsModels.jl and GLM.jl
aic
+                new_re_levels=:missing, type=:response)
Predict response for new data.
Note
Currently, no in-place methods are provided because these methods internally construct a new model and therefore allocate not just a response vector but also many other matrices.
Warning
newdata should contain a column for the response (dependent variable) initialized to some numerical value (not missing), because this is used to construct the new model used in computing the predictions. missing is not valid because missing data are dropped before constructing the model matrices.
Warning
These methods construct an entire MixedModel behind the scenes and as such may use a large amount of memory when newdata is large.
Warning
Rank-deficiency can lead to surprising but consistent behavior. For example, if there are two perfectly collinear predictors A and B (e.g. constant multiples of each other), then it is possible that A will be pivoted out in the fitted model and thus the associated coefficient is set to zero. If predictions are then generated on new data where B has been set to zero but A has not, then there will no contribution from neither A nor B in the resulting predictions.
The keyword argument new_re_levels specifies how previously unobserved values of the grouping variable are handled. Possible values are:
  • :population: return population values for the relevant grouping variable.  In other words, treat the associated random effect as 0.  If all grouping variables have new levels, then this is equivalent to  just the fixed effects.
  • :missing: return missing.
  • :error: error on this condition. The error type is an implementation detail:  you should not rely on a particular type of error being thrown.
If you want simulated values for unobserved levels of the grouping variable, consider the simulate! and simulate methods.
Predictions based purely on the fixed effects can be obtained by specifying previously unobserved levels of the random effects and setting new_re_levels=:population. Similarly, the contribution of any grouping variable can be excluded by specifying previously unobserved levels, while including previously observed levels of the other grouping variables. In the future, it may be possible to specify a subset of the grouping variables or overall random-effects structure to use, but not at this time.
Note
new_re_levels impacts only the behavior for previously unobserved random effects levels, i.e. new RE levels. For previously observed random effects levels, predictions take both the fixed and random effects into account.
For GeneralizedLinearMixedModel, the type parameter specifies whether the predictions should be returned on the scale of linear predictor (:linpred) or on the response scale (:response). If you don't know the difference between these terms, then you probably want type=:response.
Regression weights are not yet supported in prediction. Similarly, offsets are also not supported for GeneralizedLinearMixedModel.
source
StatsAPI.response — Method
response(m::MixedModel)
Return the response vector for the model.
For a linear mixed model this is a view of the last column of the XyMat field. For a generalized linear mixed model this is the m.resp.y field. In either case it should be copied if it is to be modified.
source
StatsAPI.vcov — Method
vcov(m::MixedModel; corr=false)
Returns the variance-covariance matrix of the fixed effects. If corr is true, the correlation of the fixed effects is returned instead.
source
Tables.columntable — Method
columntable(s::OptSummary, [stack::Bool=false])
Return s.fitlog as a Tables.columntable.
When stack is false (the default), there will be 3 columns in the result:
  • iter: the sample number
  • objective: the value of the objective at that sample
  • θ: the parameter vector at that sample
(The term sample here refers to the fact that when the thin argument to the fit or refit! call is greater than 1 only a subset of the iterations have results recorded.)
When stack is true, there will be 4 columns: iter, objective, par, and value where value is the stacked contents of the θ vectors (the equivalent of vcat(θ...)) and par is a vector of parameter numbers.
source
Methods from StatsAPI.jl, StatsBase.jl, StatsModels.jl and GLM.jl
aic
 aicc
 bic
 coef
@@ -77,9 +77,9 @@
 simulate
 simulate!
 stderrror!
-varest
Non-Exported Functions
Note that unless discussed elsewhere in the online documentation, non-exported functions should be considered implementation details.
Base.copy — Method
Base.copy(ReMat{T,S})
Return a shallow copy of ReMat.
A shallow copy shares as much internal storage as possible with the original ReMat. Only the vector λ and the scratch matrix are copied.
source
Base.size — Method
size(m::MixedModel)
Returns the size of a mixed model as a tuple of length four: the number of observations, the number of (non-singular) fixed-effects parameters, the number of conditional modes (random effects), the number of grouping variables
source
GLM.wrkresp! — Method
GLM.wrkresp!(v::AbstractVector{T}, resp::GLM.GlmResp{AbstractVector{T}})
A copy of a method from GLM that generalizes the types in the signature
source
MixedModels.LD — Method
LD(A::Diagonal)
+varest
Non-Exported Functions
Note that unless discussed elsewhere in the online documentation, non-exported functions should be considered implementation details.
Base.copy — Method
Base.copy(ReMat{T,S})
Return a shallow copy of ReMat.
A shallow copy shares as much internal storage as possible with the original ReMat. Only the vector λ and the scratch matrix are copied.
source
Base.size — Method
size(m::MixedModel)
Returns the size of a mixed model as a tuple of length four: the number of observations, the number of (non-singular) fixed-effects parameters, the number of conditional modes (random effects), the number of grouping variables
source
GLM.wrkresp! — Method
GLM.wrkresp!(v::AbstractVector{T}, resp::GLM.GlmResp{AbstractVector{T}})
A copy of a method from GLM that generalizes the types in the signature
source
MixedModels.LD — Method
LD(A::Diagonal)
 LD(A::HBlikDiag)
-LD(A::DenseMatrix)
Return log(det(tril(A))) evaluated in place.
source
MixedModels.adjA — Method
adjA(refs::AbstractVector, z::AbstractMatrix{T})
Returns the adjoint of an ReMat as a SparseMatrixCSC{T,Int32}
source
MixedModels.allpars — Method
allpars(bsamp::MixedModelFitCollection)
Return a tidy (column)table with the parameter estimates spread into columns of iter, type, group, name and value.
Warning
Currently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.
source
MixedModels.amalgamate — Method
amalgamate(reterms::Vector{AbstractReMat})
Combine multiple ReMat with the same grouping variable into a single object.
source
MixedModels.block — Method
block(i, j)
Return the linear index of the [i,j] position ("block") in the row-major packed lower triangle.
Use the row-major ordering in this case because the result depends only on i and j, not on the overall size of the array.
When i == j the value is the same as kp1choose2(i).
source
MixedModels.cholUnblocked! — Function
cholUnblocked!(A, Val{:L})
Overwrite the lower triangle of A with its lower Cholesky factor.
The name is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl] because these are part of the inner calculations in a blocked Cholesky factorization.
source
MixedModels.corrmat — Method
corrmat(A::ReMat)
Return the estimated correlation matrix for A.  The diagonal elements are 1 and the off-diagonal elements are the correlations between those random effect terms
Example
Note that trailing digits may vary slightly depending on the local platform.
julia> using MixedModels
+LD(A::DenseMatrix)
Return log(det(tril(A))) evaluated in place.
source
MixedModels.adjA — Method
adjA(refs::AbstractVector, z::AbstractMatrix{T})
Returns the adjoint of an ReMat as a SparseMatrixCSC{T,Int32}
source
MixedModels.allpars — Method
allpars(bsamp::MixedModelFitCollection)
Return a tidy (column)table with the parameter estimates spread into columns of iter, type, group, name and value.
Warning
Currently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.
source
MixedModels.amalgamate — Method
amalgamate(reterms::Vector{AbstractReMat})
Combine multiple ReMat with the same grouping variable into a single object.
source
MixedModels.block — Method
block(i, j)
Return the linear index of the [i,j] position ("block") in the row-major packed lower triangle.
Use the row-major ordering in this case because the result depends only on i and j, not on the overall size of the array.
When i == j the value is the same as kp1choose2(i).
source
MixedModels.cholUnblocked! — Function
cholUnblocked!(A, Val{:L})
Overwrite the lower triangle of A with its lower Cholesky factor.
The name is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl] because these are part of the inner calculations in a blocked Cholesky factorization.
source
MixedModels.corrmat — Method
corrmat(A::ReMat)
Return the estimated correlation matrix for A.  The diagonal elements are 1 and the off-diagonal elements are the correlations between those random effect terms
Example
Note that trailing digits may vary slightly depending on the local platform.
julia> using MixedModels
 
 julia> mod = fit(MixedModel,
                  @formula(rt_trunc ~ 1 + spkr + prec + load + (1 + spkr + prec | subj)),
@@ -97,13 +97,13 @@
 3×3 LinearAlgebra.Symmetric{Float64,Array{Float64,2}}:
   1.0        0.214816   -0.982948
   0.214816   1.0        -0.0315607
- -0.982948  -0.0315607   1.0
source
MixedModels.cpad — Method
cpad(s::AbstractString, n::Integer)
Return a string of length n containing s in the center (more-or-less).
source
MixedModels.densify — Function
densify(S::SparseMatrix, threshold=0.1)
Convert sparse S to Diagonal if S is diagonal or to Array(S) if the proportion of nonzeros exceeds threshold.
source
MixedModels.deviance! — Function
deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)
Update m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.
source
MixedModels.feL — Method
feL(m::LinearMixedModel)
Return the lower Cholesky factor for the fixed-effects parameters, as an LowerTriangular p × p matrix.
source
MixedModels.fixef! — Method
fixef!(v::Vector{T}, m::MixedModel{T})
Overwrite v with the pivoted fixed-effects coefficients of model m
For full-rank models the length of v must be the rank of X.  For rank-deficient models the length of v can be the rank of X or the number of columns of X.  In the latter case the calculated coefficients are padded with -0.0 out to the number of columns.
source
MixedModels.getθ! — Method
getθ!(v::AbstractVector{T}, A::ReMat{T}) where {T}
Overwrite v with the elements of the blocks in the lower triangle of A.Λ (column-major ordering)
source
MixedModels.isconstant — Method
isconstant(x::Array)
-isconstant(x::Tuple)
Are all elements of the iterator the same?  That is, is it constant?
source
MixedModels.isnested — Method
isnested(A::ReMat, B::ReMat)
Is the grouping factor for A nested in the grouping factor for B?
That is, does each value of A occur with just one value of B?
source
MixedModels.kchoose2 — Method
kchoose2(k)
The binomial coefficient k choose 2 which is the number of elements in the packed form of the strict lower triangle of a matrix.
source
MixedModels.kp1choose2 — Method
kp1choose2(k)
The binomial coefficient k+1 choose 2 which is the number of elements in the packed form of the lower triangle of a matrix.
source
MixedModels.cpad — Method
cpad(s::AbstractString, n::Integer)
Return a string of length n containing s in the center (more-or-less).
source
MixedModels.densify — Function
densify(S::SparseMatrix, threshold=0.1)
Convert sparse S to Diagonal if S is diagonal or to Array(S) if the proportion of nonzeros exceeds threshold.
source
MixedModels.deviance! — Function
deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)
Update m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.
source
MixedModels.feL — Method
feL(m::LinearMixedModel)
Return the lower Cholesky factor for the fixed-effects parameters, as an LowerTriangular p × p matrix.
source
MixedModels.fixef! — Method
fixef!(v::Vector{T}, m::MixedModel{T})
Overwrite v with the pivoted fixed-effects coefficients of model m
For full-rank models the length of v must be the rank of X.  For rank-deficient models the length of v can be the rank of X or the number of columns of X.  In the latter case the calculated coefficients are padded with -0.0 out to the number of columns.
source
MixedModels.getθ! — Method
getθ!(v::AbstractVector{T}, A::ReMat{T}) where {T}
Overwrite v with the elements of the blocks in the lower triangle of A.Λ (column-major ordering)
source
MixedModels.isconstant — Method
isconstant(x::Array)
+isconstant(x::Tuple)
Are all elements of the iterator the same?  That is, is it constant?
source
MixedModels.isnested — Method
isnested(A::ReMat, B::ReMat)
Is the grouping factor for A nested in the grouping factor for B?
That is, does each value of A occur with just one value of B?
source
MixedModels.kchoose2 — Method
kchoose2(k)
The binomial coefficient k choose 2 which is the number of elements in the packed form of the strict lower triangle of a matrix.
source
MixedModels.kp1choose2 — Method
kp1choose2(k)
The binomial coefficient k+1 choose 2 which is the number of elements in the packed form of the lower triangle of a matrix.
source
MixedModels.likelihoodratiotest — Method
likelihoodratiotest(m::MixedModel...)
 likelihoodratiotest(m0::LinearModel, m::MixedModel...)
 likelihoodratiotest(m0::GeneralizedLinearModel, m::MixedModel...)
 likelihoodratiotest(m0::TableRegressionModel{LinearModel}, m::MixedModel...)
-likelihoodratiotest(m0::TableRegressionModel{GeneralizedLinearModel}, m::MixedModel...)
Likeihood ratio test applied to a set of nested models.
Note
The nesting of the models is not checked.  It is incumbent on the user to check this. This differs from StatsModels.lrtest as nesting in mixed models, especially in the random effects specification, may be non obvious.
Note
For comparisons between mixed and non-mixed models, the deviance for the non-mixed model is taken to be -2 log likelihood, i.e. omitting the additive constant for the fully saturated model. This is in line with the computation of the deviance for mixed models.
This functionality may be deprecated in the future in favor of StatsModels.lrtest.
source
MixedModels.nranef — Method
nranef(A::ReMat)
Return the number of random effects represented by A.  Zero unless A is an ReMat.
source
MixedModels.nθ — Method
nθ(A::ReMat)
Return the number of free parameters in the relative covariance matrix λ
source
MixedModels.optsumj — Method
optsumj(os::OptSummary, j::Integer)
Return an OptSummary with the j'th component of the parameter omitted.
os.final with its j'th component omitted is used as the initial parameter.
source
MixedModels.parsej — Method
parsej(sym::Symbol)
Return the index from symbol names like :θ1, :θ01, etc.
Note
This method is internal.
source
MixedModels.pivot — Method
pivot(m::MixedModel)
-pivot(A::FeTerm)
Return the pivot associated with the FeTerm.
source
MixedModels.prfit! — Function
prfit!(m::LinearMixedModel)
Fit a mixed model using the PRIMA implementation of the BOBYQA optimizer.
Experimental feature
This function is an experimental feature that will go away in the future. Do not rely on it, unless you are willing to pin the precise MixedModels.jl version. The purpose of the function is to provide the MixedModels developers a chance to explore the performance of the PRIMA implementation without the large and potentially breaking changes it would take to fully replace the current NLopt backend with a PRIMA backend or a backend supporting a range of optimizers.
Package extension
In order to reduce the dependency burden, all methods of this function are implemented in a package extension and are only defined when PRIMA.jl is loaded by the user.
source
MixedModels.profileσs! — Method
 profileσs!(val::NamedTuple, tc::TableColumns{T}; nzlb=1.0e-8) where {T}
Profile the variance components.
Note
This method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.ranef! — Method
ranef!(v::Vector{Matrix{T}}, m::MixedModel{T}, β, uscale::Bool) where {T}
Overwrite v with the conditional modes of the random effects for m.
If uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale
β is the truncated, pivoted coefficient vector.
source
MixedModels.rankUpdate! — Function
rankUpdate!(C, A)
+likelihoodratiotest(m0::TableRegressionModel{GeneralizedLinearModel}, m::MixedModel...)
Likeihood ratio test applied to a set of nested models.
Note
The nesting of the models is not checked.  It is incumbent on the user to check this. This differs from StatsModels.lrtest as nesting in mixed models, especially in the random effects specification, may be non obvious.
Note
For comparisons between mixed and non-mixed models, the deviance for the non-mixed model is taken to be -2 log likelihood, i.e. omitting the additive constant for the fully saturated model. This is in line with the computation of the deviance for mixed models.
This functionality may be deprecated in the future in favor of StatsModels.lrtest.
source
MixedModels.nranef — Method
nranef(A::ReMat)
Return the number of random effects represented by A.  Zero unless A is an ReMat.
source
MixedModels.nθ — Method
nθ(A::ReMat)
Return the number of free parameters in the relative covariance matrix λ
source
MixedModels.optsumj — Method
optsumj(os::OptSummary, j::Integer)
Return an OptSummary with the j'th component of the parameter omitted.
os.final with its j'th component omitted is used as the initial parameter.
source
MixedModels.parsej — Method
parsej(sym::Symbol)
Return the index from symbol names like :θ1, :θ01, etc.
Note
This method is internal.
source
MixedModels.pivot — Method
pivot(m::MixedModel)
+pivot(A::FeTerm)
Return the pivot associated with the FeTerm.
source
MixedModels.prfit! — Function
prfit!(m::LinearMixedModel)
Fit a mixed model using the PRIMA implementation of the BOBYQA optimizer.
Experimental feature
This function is an experimental feature that will go away in the future. Do not rely on it, unless you are willing to pin the precise MixedModels.jl version. The purpose of the function is to provide the MixedModels developers a chance to explore the performance of the PRIMA implementation without the large and potentially breaking changes it would take to fully replace the current NLopt backend with a PRIMA backend or a backend supporting a range of optimizers.
OptSummary
As part of this experimental foray, the structure of OptSummary is not changed. This means that some fields of OptSummary are inaccurate when examining a model fit with PRIMA. The following fields are unused with PRIMA fits:
  • all tolerances (ftol_rel, ftol_abs, xtol_rel, xtol_abs)
  • optimization timeouts (maxfeval, maxtime)
  • initial_step
The following fields have a different meaning when used with PRIMA:
  • returnvalue is populated with a symbol representing the PRIMA   return value, which PRIMA represents as an enum.
  • optimizer is populated with a dummy value, indicating that a model was  fit with PRIMA. If you wish to refit the model with the NLOpt backend,  you will need to update the field with an appropriate NLOpt optimizer,  e.g. :LN_BOBYQA
Package extension
In order to reduce the dependency burden, all methods of this function are implemented in a package extension and are only defined when PRIMA.jl is loaded by the user.
source
MixedModels.profileσs! — Method
 profileσs!(val::NamedTuple, tc::TableColumns{T}; nzlb=1.0e-8) where {T}
Profile the variance components.
Note
This method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.
source
MixedModels.ranef! — Method
ranef!(v::Vector{Matrix{T}}, m::MixedModel{T}, β, uscale::Bool) where {T}
Overwrite v with the conditional modes of the random effects for m.
If uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale
β is the truncated, pivoted coefficient vector.
source
MixedModels.rankUpdate! — Function
rankUpdate!(C, A)
 rankUpdate!(C, A, α)
-rankUpdate!(C, A, α, β)
A rank-k update, C := αA'A + βC, of a Hermitian (Symmetric) matrix.
α and β both default to 1.0.  When α is -1.0 this is a downdate operation. The name rankUpdate! is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl]
source
MixedModels.rePCA — Method
rePCA(m::LinearMixedModel; corr::Bool=true)
Return a named tuple of the normalized cumulative variance of a principal components analysis of the random effects covariance matrices or correlation matrices when corr is true.
The normalized cumulative variance is the proportion of the variance for the first principal component, the first two principal components, etc.  The last element is always 1.0 representing the complete proportion of the variance.
source
MixedModels.reevaluateAend! — Method
reevaluateAend!(m::LinearMixedModel)
Reevaluate the last column of m.A from m.Xymat.  This function should be called after updating the response.
source
MixedModels.refitσ! — Method
refitσ!(m::LinearMixedModel{T}, σ::T, tc::TableColumns{T}, obj::T, neg::Bool)
Refit the model m with the given value of σ and return a NamedTuple of information about the fit.
obj and neg allow for conversion of the objective to the ζ scale and tc is used to return a NamedTuple
Note
This method is internal and may change or disappear in a future release without being considered breaking.
source
MixedModels.schematize — Function
schematize(f, tbl, contrasts::Dict{Symbol}, Mod=LinearMixedModel)
Find and apply the schema for f in a way that automatically uses Grouping() contrasts when appropriate.
Warn
This is an internal method.
source
MixedModels.sdcorr — Method
sdcorr(A::AbstractMatrix{T}) where {T}
Transform a square matrix A with positive diagonals into an NTuple{size(A,1), T} of standard deviations and a tuple of correlations.
A is assumed to be symmetric and only the lower triangle is used.  The order of the correlations is row-major ordering of the lower triangle (or, equivalently, column-major in the upper triangle).
source
MixedModels.setβθ! — Method
setβθ!(m::GeneralizedLinearMixedModel, v)
Set the parameter vector, :βθ, of m to v.
βθ is the concatenation of the fixed-effects, β, and the covariance parameter, θ.
source
MixedModels.ssqdenom — Method
ssqdenom(m::LinearMixedModel)
Return the denominator for penalized sums-of-squares.
For MLE, this value is the number of observations. For REML, this value is the number of observations minus the rank of the fixed-effects matrix. The difference is analogous to the use of n or n-1 in the denominator when calculating the variance.
source
MixedModels.statsrank — Method
statsrank(x::Matrix{T}, ranktol::Real=1e-8) where {T<:AbstractFloat}
Return the numerical column rank and a pivot vector.
The rank is determined from the absolute values of the diagonal of R from a pivoted QR decomposition, relative to the first (and, hence, largest) element of this vector.
In the full-rank case the pivot vector is collect(axes(x, 2)).
source
MixedModels.tidyβ — Method
tidyβ(bsamp::MixedModelFitCollection)
Return a tidy (row)table with the parameter estimates spread into columns of iter, coefname and β
source
MixedModels.tidyσs — Method
tidyσs(bsamp::MixedModelFitCollection)
Return a tidy (row)table with the estimates of the variance components (on the standard deviation scale) spread into columns of iter, group, column and σ.
source
MixedModels.unscaledre! — Function
unscaledre!(y::AbstractVector{T}, M::ReMat{T}) where {T}
-unscaledre!(rng::AbstractRNG, y::AbstractVector{T}, M::ReMat{T}) where {T}
Add unscaled random effects simulated from M to y.
These are unscaled random effects (i.e. they incorporate λ but not σ) because the scaling is done after the per-observation noise is added as a standard normal.
source
MixedModels.updateA! — Method
updateA!(m::LinearMixedModel)
Update the cross-product array, m.A, from m.reterms and m.Xymat
This is usually done after a reweight! operation.
source
MixedModels.updateη! — Method
updateη!(m::GeneralizedLinearMixedModel)
Update the linear predictor, m.η, from the offset and the B-scale random effects.
source
MixedModels.σvals! — Method
σvals!(v::AbstractVector, A::ReMat, sc::Number)
Overwrite v with the standard deviations of the random effects associated with A
source
MixedModels.σρ! — Method
σρ!(v, t, σ)
push! σ times the row lengths (σs) and the inner products of normalized rows (ρs) of t onto v
source
StatsModels.isnested — Method
isnested(m1::MixedModel, m2::MixedModel; atol::Real=0.0)
Indicate whether model m1 is nested in model m2, i.e. whether m1 can be obtained by constraining some parameters in m2. Both models must have been fitted on the same data. This check is conservative for MixedModels and may reject nested models with different parameterizations as being non nested.
source

+rankUpdate!(C, A, α, β)

A rank-k update, C := αA'A + βC, of a Hermitian (Symmetric) matrix.

α and β both default to 1.0. When α is -1.0 this is a downdate operation. The name rankUpdate! is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl]

source
MixedModels.rePCA — Method
rePCA(m::LinearMixedModel; corr::Bool=true)

Return a named tuple of the normalized cumulative variance of a principal components analysis of the random effects covariance matrices or correlation matrices when corr is true.

The normalized cumulative variance is the proportion of the variance for the first principal component, the first two principal components, etc. The last element is always 1.0 representing the complete proportion of the variance.

source
MixedModels.reevaluateAend! — Method
reevaluateAend!(m::LinearMixedModel)

Reevaluate the last column of m.A from m.Xymat. This function should be called after updating the response.

source
MixedModels.refitσ! — Method
refitσ!(m::LinearMixedModel{T}, σ::T, tc::TableColumns{T}, obj::T, neg::Bool)

Refit the model m with the given value of σ and return a NamedTuple of information about the fit.

obj and neg allow for conversion of the objective to the ζ scale and tc is used to return a NamedTuple

Note

This method is internal and may change or disappear in a future release without being considered breaking.

source
MixedModels.schematize — Function
schematize(f, tbl, contrasts::Dict{Symbol}, Mod=LinearMixedModel)

Find and apply the schema for f in a way that automatically uses Grouping() contrasts when appropriate.

Warn

This is an internal method.

source
MixedModels.sdcorr — Method
sdcorr(A::AbstractMatrix{T}) where {T}

Transform a square matrix A with positive diagonals into an NTuple{size(A,1), T} of standard deviations and a tuple of correlations.

A is assumed to be symmetric and only the lower triangle is used. The order of the correlations is row-major ordering of the lower triangle (or, equivalently, column-major in the upper triangle).

source
MixedModels.setβθ! — Method
setβθ!(m::GeneralizedLinearMixedModel, v)

Set the parameter vector, :βθ, of m to v.

βθ is the concatenation of the fixed-effects, β, and the covariance parameter, θ.

source
MixedModels.ssqdenom — Method
ssqdenom(m::LinearMixedModel)

Return the denominator for penalized sums-of-squares.

For MLE, this value is the number of observations. For REML, this value is the number of observations minus the rank of the fixed-effects matrix. The difference is analogous to the use of n or n-1 in the denominator when calculating the variance.

source
MixedModels.statsrank — Method
statsrank(x::Matrix{T}, ranktol::Real=1e-8) where {T<:AbstractFloat}

Return the numerical column rank and a pivot vector.

The rank is determined from the absolute values of the diagonal of R from a pivoted QR decomposition, relative to the first (and, hence, largest) element of this vector.

In the full-rank case the pivot vector is collect(axes(x, 2)).

source
MixedModels.tidyβ — Method
tidyβ(bsamp::MixedModelFitCollection)

Return a tidy (row)table with the parameter estimates spread into columns of iter, coefname and β

source
MixedModels.tidyσs — Method
tidyσs(bsamp::MixedModelFitCollection)

Return a tidy (row)table with the estimates of the variance components (on the standard deviation scale) spread into columns of iter, group, column and σ.

source
MixedModels.unfit! — Method
unfit!(model::MixedModel)

Mark a model as unfitted.

source
MixedModels.unscaledre! — Function
unscaledre!(y::AbstractVector{T}, M::ReMat{T}) where {T}
+unscaledre!(rng::AbstractRNG, y::AbstractVector{T}, M::ReMat{T}) where {T}

Add unscaled random effects simulated from M to y.

These are unscaled random effects (i.e. they incorporate λ but not σ) because the scaling is done after the per-observation noise is added as a standard normal.

source
MixedModels.updateA! — Method
updateA!(m::LinearMixedModel)

Update the cross-product array, m.A, from m.reterms and m.Xymat

This is usually done after a reweight! operation.

source
MixedModels.updateη! — Method
updateη!(m::GeneralizedLinearMixedModel)

Update the linear predictor, m.η, from the offset and the B-scale random effects.

source
MixedModels.σvals! — Method
σvals!(v::AbstractVector, A::ReMat, sc::Number)

Overwrite v with the standard deviations of the random effects associated with A

source
MixedModels.σρ! — Method
σρ!(v, t, σ)

push! σ times the row lengths (σs) and the inner products of normalized rows (ρs) of t onto v

source
StatsModels.isnested — Method
isnested(m1::MixedModel, m2::MixedModel; atol::Real=0.0)

Indicate whether model m1 is nested in model m2, i.e. whether m1 can be obtained by constraining some parameters in m2. Both models must have been fitted on the same data. This check is conservative for MixedModels and may reject nested models with different parameterizations as being non nested.

source
diff --git a/dev/benchmarks/index.html b/dev/benchmarks/index.html index d17c466f3..ab33d98ed 100644 --- a/dev/benchmarks/index.html +++ b/dev/benchmarks/index.html @@ -17,4 +17,4 @@ Load Avg: 1.4091796875 2.07080078125 1.63037109375 WORD_SIZE: 64 LIBM: libopenlibm - LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge) + LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge) diff --git a/dev/bootstrap/index.html b/dev/bootstrap/index.html index c87e6787e..2ebe0bde1 100644 --- a/dev/bootstrap/index.html +++ b/dev/bootstrap/index.html @@ -1,6 +1,6 @@ Parametric bootstrap for mixed-effects models · MixedModels
GitHub

Parametric bootstrap for mixed-effects models

Julia is well-suited to implementing bootstrapping and other simulation-based methods for statistical models. The parametricbootstrap function in the MixedModels package provides an efficient parametric bootstrap for mixed-effects models.

MixedModels.parametricbootstrap — Function
parametricbootstrap([rng::AbstractRNG], nsamp::Integer, m::MixedModel{T}, ftype=T;
-    β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))

Perform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.

The default random number generator is Random.GLOBAL_RNG.

ftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.

Keyword Arguments

  • β, σ, and θ are the values of m's parameters for simulating the responses.
  • σ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for

families with a dispersion parameter.

  • progress controls whether the progress bar is shown. Note that the progress

bar is automatically disabled for non-interactive (i.e. logging) contexts.

  • optsum_overrides is used to override values of OptSummary in the models

fit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.

Note

All coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.

source

The parametric bootstrap

Bootstrapping is a family of procedures for generating sample values of a statistic, allowing for visualization of the distribution of the statistic or for inference from this sample of values.

A parametric bootstrap is used with a parametric model, m, that has been fit to data. The procedure is to simulate n response vectors from m using the estimated parameter values and refit m to these responses in turn, accumulating the statistics of interest at each iteration.

The parameters of a LinearMixedModel object are the fixed-effects parameters, β, the standard deviation, σ, of the per-observation noise, and the covariance parameter, θ, that defines the variance-covariance matrices of the random effects.

For example, a simple linear mixed-effects model for the Dyestuff data in the lme4 package for R is fit by

using DataFrames
+    β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))

Perform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.

The default random number generator is Random.GLOBAL_RNG.

ftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.

Keyword Arguments

  • β, σ, and θ are the values of m's parameters for simulating the responses.
  • σ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for

families with a dispersion parameter.

  • progress controls whether the progress bar is shown. Note that the progress

bar is automatically disabled for non-interactive (i.e. logging) contexts.

  • optsum_overrides is used to override values of OptSummary in the models

fit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.

Note

All coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.

source

The parametric bootstrap

Bootstrapping is a family of procedures for generating sample values of a statistic, allowing for visualization of the distribution of the statistic or for inference from this sample of values.

A parametric bootstrap is used with a parametric model, m, that has been fit to data. The procedure is to simulate n response vectors from m using the estimated parameter values and refit m to these responses in turn, accumulating the statistics of interest at each iteration.

The parameters of a LinearMixedModel object are the fixed-effects parameters, β, the standard deviation, σ, of the per-observation noise, and the covariance parameter, θ, that defines the variance-covariance matrices of the random effects.

For example, a simple linear mixed-effects model for the Dyestuff data in the lme4 package for R is fit by

using DataFrames
 using Gadfly          # plotting package
 using MixedModels
 using Random
dyestuff = MixedModels.dataset(:dyestuff)
@@ -29,7 +29,7 @@
  17 │ 320.086  1517.67  43.218   35.9663   0.832207
  ⋮  │    ⋮        ⋮        ⋮        ⋮          ⋮

A density plot of the estimates of σ, the residual standard deviation, can be created as

plot(x = tbl.σ, Geom.density, Guide.xlabel("Parametric bootstrap estimates of σ"))
Example block output

or, for the intercept parameter

plot(x = tbl.β1, Geom.density, Guide.xlabel("Parametric bootstrap estimates of β₁"))
Example block output

A density plot of the estimates of the standard deviation of the random effects is obtained as

plot(x = tbl.σ1, Geom.density,
     Guide.xlabel("Parametric bootstrap estimates of σ₁"))
Example block output

Notice that this density plot has a spike, or mode, at zero. Although this mode appears to be diffuse, this is an artifact of the way that density plots are created. In fact, it is a pulse, as can be seen from a histogram.

plot(x = tbl.σ1, Geom.histogram,
-    Guide.xlabel("Parametric bootstrap estimates of σ₁"))
Example block output

The bootstrap sample can be used to generate intervals that cover a certain percentage of the bootstrapped values. We refer to these as "coverage intervals", similar to a confidence interval. The shortest such intervals, obtained with the shortestcovint extractor, correspond to a highest posterior density interval in Bayesian inference.

MixedModels.shortestcovint — Function
shortestcovint(v, level = 0.95)

Return the shortest interval containing level proportion of the values of v

source
shortestcovint(bsamp::MixedModelFitCollection, level = 0.95)

Return the shortest interval containing level proportion for each parameter from bsamp.allpars.

Warning

Currently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.

source

We generate these directly from the original bootstrap object:

Table(shortestcovint(samp))
Table with 5 columns and 3 rows:
+    Guide.xlabel("Parametric bootstrap estimates of σ₁"))
Example block output

The bootstrap sample can be used to generate intervals that cover a certain percentage of the bootstrapped values. We refer to these as "coverage intervals", similar to a confidence interval. The shortest such intervals, obtained with the shortestcovint extractor, correspond to a highest posterior density interval in Bayesian inference.

MixedModels.shortestcovint — Function
shortestcovint(v, level = 0.95)

Return the shortest interval containing level proportion of the values of v

source
shortestcovint(bsamp::MixedModelFitCollection, level = 0.95)

Return the shortest interval containing level proportion for each parameter from bsamp.allpars.

Warning

Currently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.

source

We generate these directly from the original bootstrap object:

Table(shortestcovint(samp))
Table with 5 columns and 3 rows:
      type  group     names        lower    upper
    ┌──────────────────────────────────────────────
  1 │ β     missing   (Intercept)  1492.54  1561.34
@@ -68,9 +68,9 @@
  (type = "ρ", group = "subj", names = "(Intercept), days", lower = -0.40535765760216863, upper = 1.0)
  (type = "σ", group = "residual", names = missing, lower = 22.656763019290786, upper = 28.43122140376578)

A histogram of the estimated correlations from the bootstrap sample has a spike at +1.

plot(x = tbl2.ρ1, Geom.histogram,
     Guide.xlabel("Parametric bootstrap samples of correlation of random effects"))
Example block output

or, as a count,

count(tbl2.ρ1 .≈ 1)
306

Close examination of the histogram shows a few values of -1.

count(tbl2.ρ1 .≈ -1)
2

Furthermore there are even a few cases where the estimate of the standard deviation of the random effect for the intercept is zero.

count(tbl2.σ1 .≈ 0)
5

There is a general condition to check for singularity of an estimated covariance matrix or matrices in a bootstrap sample. The parameter optimized in the estimation is θ, the relative covariance parameter. Some of the elements of this parameter vector must be non-negative and, when one of these components is approximately zero, one of the covariance matrices will be singular.

The issingular method for a MixedModel object that tests if a parameter vector θ corresponds to a boundary or singular fit.

This operation is encapsulated in a method for the issingular function.

count(issingular(samp2))
313

Reduced Precision Bootstrap

parametricbootstrap accepts an optional keyword argument optsum_overrides, which can be used to override the convergence criteria for bootstrap replicates. One possibility is setting ftol_rel=1e-8, i.e., considering the model converged when the relative change in the objective between optimizer iterations is smaller than 0.00000001. This threshold corresponds approximately to the precision from treating the value of the objective as a single precision (Float32) number, while not changing the precision of the intermediate computations. The resultant loss in precision will generally be smaller than the variation that the bootstrap captures, but can greatly speed up the fitting process for each replicates, especially for large models. More directly, lowering the fit quality for each replicate will reduce the quality of each replicate, but this may be more than compensated for by the ability to fit a much larger number of replicates in the same time.

t = @timed parametricbootstrap(MersenneTwister(42), 1000, m2; progress=false)
-t.time
0.800608407
optsum_overrides = (; ftol_rel=1e-8)
+t.time
0.788844129
optsum_overrides = (; ftol_rel=1e-8)
 t = @timed parametricbootstrap(MersenneTwister(42), 1000, m2; optsum_overrides, progress=false)
-t.time
0.671892825

Distributed Computing and the Bootstrap

Earlier versions of MixedModels.jl supported a multi-threaded bootstrap via the use_threads keyword argument. However, with improved BLAS multithreading, the Julia-level threads often wound up competing with the BLAS threads, leading to no improvement or even a worsening of performance when use_threads=true. Nonetheless, the bootstrap is a classic example of an embarrassingly parallel problem and so we provide a few convenience methods for combining results computed separately. In particular, there are vcat and an optimized reduce(::typeof(vcat)) methods for MixedModelBootstrap objects. For computers with many processors (as opposed to a single processor with several cores) or for computing clusters, these provide a convenient way to split the computation across nodes.

using Distributed
+t.time
0.644730112

Distributed Computing and the Bootstrap

Earlier versions of MixedModels.jl supported a multi-threaded bootstrap via the use_threads keyword argument. However, with improved BLAS multithreading, the Julia-level threads often wound up competing with the BLAS threads, leading to no improvement or even a worsening of performance when use_threads=true. Nonetheless, the bootstrap is a classic example of an embarrassingly parallel problem and so we provide a few convenience methods for combining results computed separately. In particular, there are vcat and an optimized reduce(::typeof(vcat)) methods for MixedModelBootstrap objects. For computers with many processors (as opposed to a single processor with several cores) or for computing clusters, these provide a convenient way to split the computation across nodes.

using Distributed
 # you already have 1 proc by default, so add the number of additional cores with `addprocs`
 # you need at least as many RNGs as cores you want to use in parallel
 # but you shouldn't use all of your cores because nested within this
@@ -112,4 +112,4 @@
  ρ1  │ -0.424371  1.0
  σ   │ 22.4485    28.2745
  σ1  │ 10.6217    32.5576
- σ2  │ 3.18136    7.74161
+ σ2 │ 3.18136 7.74161 diff --git a/dev/constructors/index.html b/dev/constructors/index.html index 57d746c28..8bdfec1b6 100644 --- a/dev/constructors/index.html +++ b/dev/constructors/index.html @@ -39,13 +39,13 @@ ────────────────────────────────────────────────

(If you are new to Julia you may find that this first fit takes an unexpectedly long time, due to Just-In-Time (JIT) compilation of the code. The subsequent calls to such functions are much faster.)

using BenchmarkTools
 dyestuff2 = MixedModels.dataset(:dyestuff2)
 @benchmark fit(MixedModel, $fm, $dyestuff2)
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
- Range (min … max):  141.724 μs …  14.983 ms  ┊ GC (min … max): 0.00% … 93.95%
- Time  (median):     147.366 μs               ┊ GC (median):    0.00%
- Time  (mean ± σ):   161.128 μs ± 366.484 μs  ┊ GC (mean ± σ):  6.34% ±  2.77%
+ Range (min … max):  143.276 μs …  14.358 ms  ┊ GC (min … max): 0.00% … 94.28%
+ Time  (median):     148.947 μs               ┊ GC (median):    0.00%
+ Time  (mean ± σ):   163.700 μs ± 369.017 μs  ┊ GC (mean ± σ):  6.31% ±  2.77%
 
-   ▂▄▇███▇▆▅▅▄▄▄▄▃▂▂▂▁▁▁   ▁▁   ▁                               ▂
-  ▆█████████████████████████████████████▇▇▆▅▆▅▄▅▅▅▆▃▅▅▄▅▄▄▅▅▄▄▄ █
-  142 μs        Histogram: log(frequency) by time        193 μs <
+   ▄▆███▇▆▅▅▄▄▃▃▃▂▂▂▂▂▁▁▂▁▁▁▁▂▁                                 ▂
+  ███████████████████████████████▇▇▇▇▆▅▆▆▆▆▆▆▅▅▆▅▆▆▅▆▆▆▇▇▇▆▇▇▆▆ █
+  143 μs        Histogram: log(frequency) by time        202 μs <
 
  Memory estimate: 53.70 KiB, allocs estimate: 955.

By default, the model is fit by maximum likelihood. To use the REML criterion instead, add the optional named argument REML=true to the call to fit

fm1reml = fit(MixedModel, fm, dyestuff, REML=true)
Linear mixed model fit by REML
  yield ~ 1 + (1 | batch)
@@ -286,7 +286,7 @@
 days: 7       62.0988     10.0922    6.15    <1e-09
 days: 8       79.9777     13.2713    6.03    <1e-08
 days: 9       94.1994     13.1757    7.15    <1e-12
-───────────────────────────────────────────────────

(Notice that the variance component for days: 1 is estimated as zero, so the correlations for this component are undefined and expressed as NaN, not a number.)

An alternative is to force all the levels of days as indicators using fulldummy encoding.

MixedModels.fulldummy — Function
fulldummy(term::CategoricalTerm)

Assign "contrasts" that include all indicator columns (dummy variables) and an intercept column.

This will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or "shrinkage", of the conditional modes.

The interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.

source
fit(MixedModel, @formula(reaction ~ 1 + days + (1 + fulldummy(days)|subj)), sleepstudy,
+───────────────────────────────────────────────────

(Notice that the variance component for days: 1 is estimated as zero, so the correlations for this component are undefined and expressed as NaN, not a number.)

An alternative is to force all the levels of days as indicators using fulldummy encoding.

MixedModels.fulldummy — Function
fulldummy(term::CategoricalTerm)

Assign "contrasts" that include all indicator columns (dummy variables) and an intercept column.

This will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or "shrinkage", of the conditional modes.

The interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.

source
fit(MixedModel, @formula(reaction ~ 1 + days + (1 + fulldummy(days)|subj)), sleepstudy,
     contrasts = Dict(:days => DummyCoding()))
Linear mixed model fit by maximum likelihood
  reaction ~ 1 + days + (1 + days | subj)
    logLik   -2 logLik     AIC       AICc        BIC    
@@ -458,24 +458,24 @@
 mode: want     0.706979     0.151006    4.68    <1e-05
 ──────────────────────────────────────────────────────

The canonical link, which is LogitLink for the Bernoulli distribution, is used if no explicit link is specified.

Note that, in keeping with convention in the GLM package, the distribution family for a binary (i.e. 0/1) response is the Bernoulli distribution. The Binomial distribution is only used when the response is the fraction of trials returning a positive, in which case the number of trials must be specified as the case weights.

Optional arguments to fit

An alternative approach is to create the GeneralizedLinearMixedModel object then call fit! on it. The optional arguments fast and/or nAGQ can be passed to the optimization process via both fit and fit! (i.e these optimization settings are not used nor recognized when constructing the model).

As the name implies, fast=true, provides a faster but somewhat less accurate fit. These fits may suffice for model comparisons.

gm1a = fit(MixedModel, verbaggform, verbagg, Bernoulli(), fast = true)
 deviance(gm1a) - deviance(gm1)
0.33800914130279125
@benchmark fit(MixedModel, $verbaggform, $verbagg, Bernoulli())
BenchmarkTools.Trial: 3 samples with 1 evaluation per sample.
- Range (min … max):  2.332 s …  2.337 s  ┊ GC (min … max): 0.00% … 0.00%
- Time  (median):     2.336 s             ┊ GC (median):    0.00%
- Time  (mean ± σ):   2.335 s ± 2.772 ms  ┊ GC (mean ± σ):  0.06% ± 0.11%
+ Range (min … max):  2.356 s …   2.454 s  ┊ GC (min … max): 0.23% … 0.00%
+ Time  (median):     2.367 s              ┊ GC (median):    0.00%
+ Time  (mean ± σ):   2.392 s ± 53.795 ms  ┊ GC (mean ± σ):  0.08% ± 0.13%
 
-  █                                         █            █  
-  █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
-  2.33 s        Histogram: frequency by time        2.34 s <
+  █     █                                                 █  
+  █▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
+  2.36 s         Histogram: frequency by time        2.45 s <
 
- Memory estimate: 40.73 MiB, allocs estimate: 403334.
@benchmark fit(MixedModel, $verbaggform, $verbagg, Bernoulli(), fast = true)
BenchmarkTools.Trial: 26 samples with 1 evaluation per sample.
- Range (min … max):  195.977 ms … 204.342 ms  ┊ GC (min … max): 0.00% … 1.83%
- Time  (median):     197.356 ms               ┊ GC (median):    0.00%
- Time  (mean ± σ):   198.205 ms ±   2.175 ms  ┊ GC (mean ± σ):  0.17% ± 0.49%
+ Memory estimate: 40.73 MiB, allocs estimate: 403347.
@benchmark fit(MixedModel, $verbaggform, $verbagg, Bernoulli(), fast = true)
BenchmarkTools.Trial: 25 samples with 1 evaluation per sample.
+ Range (min … max):  196.120 ms … 214.449 ms  ┊ GC (min … max): 0.00% … 1.28%
+ Time  (median):     198.385 ms               ┊ GC (median):    0.00%
+ Time  (mean ± σ):   200.143 ms ±   4.799 ms  ┊ GC (mean ± σ):  0.19% ± 0.53%
 
-  ▄ ▁     █▁▁         ▁                                          
-  █▁█▁▆▁▆▁███▁▆▁▁▁▆▆▁▁█▁▁▁▁▁▁▁▆▆▁▁▆▁▁▁▁▁▁▁▁▁▁▆▁▁▁▁▁▆▁▁▁▁▁▁▁▁▁▁▆ ▁
-  196 ms           Histogram: frequency by time          204 ms <
+  █▃█  ▃ █                                                       
+  ███▇▁█▇█▇▁▁▁▇▁▇▇▁▁▁▁▁▁▁▁▁▁▇▁▇▁▁▇▁▁▁▇▁▁▁▁▁▁▁▁▁▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇ ▁
+  196 ms           Histogram: frequency by time          214 ms <
 
- Memory estimate: 10.99 MiB, allocs estimate: 84351.

The optional argument nAGQ=k causes evaluation of the deviance function to use a k point adaptive Gauss-Hermite quadrature rule. This method only applies to models with a single, simple, scalar random-effects term, such as

contraception = MixedModels.dataset(:contra)
+ Memory estimate: 10.99 MiB, allocs estimate: 84350.

The optional argument nAGQ=k causes evaluation of the deviance function to use a k point adaptive Gauss-Hermite quadrature rule. This method only applies to models with a single, simple, scalar random-effects term, such as

contraception = MixedModels.dataset(:contra)
 contraform = @formula(use ~ 1 + age + abs2(age) + livch + urban + (1|dist));
 bernoulli = Bernoulli()
 deviances = Dict{Symbol,Float64}()
@@ -492,7 +492,7 @@
   :fast      => 2372.78
   :nAGQ      => 2372.46
   :nAGQ_fast => 2372.51

Extractor functions

LinearMixedModel and GeneralizedLinearMixedModel are subtypes of StatsAPI.RegressionModel which, in turn, is a subtype of StatsBase.StatisticalModel. Many of the generic extractors defined in the StatsBase package have methods for these models.

Model-fit statistics

The statistics describing the quality of the model fit include

StatsAPI.loglikelihood — Function
loglikelihood(model::StatisticalModel)
-loglikelihood(model::StatisticalModel, observation)

Return the log-likelihood of the model.

With an observation argument, return the contribution of observation to the log-likelihood of model.

If observation is a Colon, return a vector of each observation's contribution to the log-likelihood of the model. In other words, this is the vector of the pointwise log-likelihood contributions.

In general, sum(loglikehood(model, :)) == loglikelihood(model).

source
StatsAPI.aic — Function
aic(model::StatisticalModel)

Akaike's Information Criterion, defined as $-2 \log L + 2k$, with $L$ the likelihood of the model, and k its number of consumed degrees of freedom (as returned by dof).

source
StatsAPI.bic — Function
bic(model::StatisticalModel)

Bayesian Information Criterion, defined as $-2 \log L + k \log n$, with $L$ the likelihood of the model, $k$ its number of consumed degrees of freedom (as returned by dof), and $n$ the number of observations (as returned by nobs).

source
StatsAPI.dof — Function
dof(model::StatisticalModel)

Return the number of degrees of freedom consumed in the model, including when applicable the intercept and the distribution's dispersion parameter.

source
StatsAPI.nobs — Function
nobs(model::StatisticalModel)

Return the number of independent observations on which the model was fitted. Be careful when using this information, as the definition of an independent observation may vary depending on the model, on the format used to pass the data, on the sampling plan (if specified), etc.

source
loglikelihood(fm1)
-163.6635299405715
aic(fm1)
333.327059881143
bic(fm1)
337.5306520261295
dof(fm1)   # 1 fixed effect, 2 variances
3
nobs(fm1)  # 30 observations
30
loglikelihood(gm1)
-4067.916431282346

In general the deviance of a statistical model fit is negative twice the log-likelihood adjusting for the saturated model.

StatsAPI.deviance — Method
deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}

Return the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.

If the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, $u$, plus the determinant of $Λ'Z'WZΛ + I$, plus the sum of the squared deviance residuals.

source

Because it is not clear what the saturated model corresponding to a particular LinearMixedModel should be, negative twice the log-likelihood is called the objective.

MixedModels.objective — Function
objective(m::LinearMixedModel)

Return negative twice the log-likelihood of model m

source

This value is also accessible as the deviance but the user should bear in mind that this doesn't have all the properties of a deviance which is corrected for the saturated model. For example, it is not necessarily non-negative.

objective(fm1)
327.327059881143
deviance(fm1)
327.327059881143

The value optimized when fitting a GeneralizedLinearMixedModel is the Laplace approximation to the deviance or an adaptive Gauss-Hermite evaluation.

MixedModels.deviance! — Function
deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)

Update m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.

source
MixedModels.deviance!(gm1)
8135.832862564683

Fixed-effects parameter estimates

The coef and fixef extractors both return the maximum likelihood estimates of the fixed-effects coefficients. They differ in their behavior in the rank-deficient case. The associated coefnames and fixefnames return the corresponding coefficient names.

StatsAPI.coef — Function
coef(model::StatisticalModel)

Return the coefficients of the model.

source
StatsAPI.coefnames — Function
coefnames(model::StatisticalModel)

Return the names of the coefficients.

source
MixedModels.fixef — Function
fixef(m::MixedModel)

Return the fixed-effects parameter vector estimate of m.

In the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.

source
MixedModels.fixefnames — Function
fixefnames(m::MixedModel)

Return a (permuted and truncated in the rank-deficient case) vector of coefficient names.

source
coef(fm1)
+loglikelihood(model::StatisticalModel, observation)

Return the log-likelihood of the model.

With an observation argument, return the contribution of observation to the log-likelihood of model.

If observation is a Colon, return a vector of each observation's contribution to the log-likelihood of the model. In other words, this is the vector of the pointwise log-likelihood contributions.

In general, sum(loglikehood(model, :)) == loglikelihood(model).

source
StatsAPI.aic — Function
aic(model::StatisticalModel)

Akaike's Information Criterion, defined as $-2 \log L + 2k$, with $L$ the likelihood of the model, and k its number of consumed degrees of freedom (as returned by dof).

source
StatsAPI.bic — Function
bic(model::StatisticalModel)

Bayesian Information Criterion, defined as $-2 \log L + k \log n$, with $L$ the likelihood of the model, $k$ its number of consumed degrees of freedom (as returned by dof), and $n$ the number of observations (as returned by nobs).

source
StatsAPI.dof — Function
dof(model::StatisticalModel)

Return the number of degrees of freedom consumed in the model, including when applicable the intercept and the distribution's dispersion parameter.

source
StatsAPI.nobs — Function
nobs(model::StatisticalModel)

Return the number of independent observations on which the model was fitted. Be careful when using this information, as the definition of an independent observation may vary depending on the model, on the format used to pass the data, on the sampling plan (if specified), etc.

source
loglikelihood(fm1)
-163.6635299405715
aic(fm1)
333.327059881143
bic(fm1)
337.5306520261295
dof(fm1)   # 1 fixed effect, 2 variances
3
nobs(fm1)  # 30 observations
30
loglikelihood(gm1)
-4067.916431282346

In general the deviance of a statistical model fit is negative twice the log-likelihood adjusting for the saturated model.

StatsAPI.deviance — Method
deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}

Return the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.

If the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, $u$, plus the determinant of $Λ'Z'WZΛ + I$, plus the sum of the squared deviance residuals.

source

Because it is not clear what the saturated model corresponding to a particular LinearMixedModel should be, negative twice the log-likelihood is called the objective.

MixedModels.objective — Function
objective(m::LinearMixedModel)

Return negative twice the log-likelihood of model m

source

This value is also accessible as the deviance but the user should bear in mind that this doesn't have all the properties of a deviance which is corrected for the saturated model. For example, it is not necessarily non-negative.

objective(fm1)
327.327059881143
deviance(fm1)
327.327059881143

The value optimized when fitting a GeneralizedLinearMixedModel is the Laplace approximation to the deviance or an adaptive Gauss-Hermite evaluation.

MixedModels.deviance! — Function
deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)

Update m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.

source
MixedModels.deviance!(gm1)
8135.832862564683

Fixed-effects parameter estimates

The coef and fixef extractors both return the maximum likelihood estimates of the fixed-effects coefficients. They differ in their behavior in the rank-deficient case. The associated coefnames and fixefnames return the corresponding coefficient names.

StatsAPI.coef — Function
coef(model::StatisticalModel)

Return the coefficients of the model.

source
StatsAPI.coefnames — Function
coefnames(model::StatisticalModel)

Return the names of the coefficients.

source
MixedModels.fixef — Function
fixef(m::MixedModel)

Return the fixed-effects parameter vector estimate of m.

In the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.

source
MixedModels.fixefnames — Function
fixefnames(m::MixedModel)

Return a (permuted and truncated in the rank-deficient case) vector of coefficient names.

source
coef(fm1)
 coefnames(fm1)
1-element Vector{String}:
  "(Intercept)"
fixef(fm1)
 fixefnames(fm1)
1-element Vector{String}:
@@ -538,9 +538,9 @@
 subj (Intercept)  1.793543 1.339232
 item (Intercept)  0.117147 0.342267
 
-

Individual components are returned by other extractors

MixedModels.varest — Function
varest(m::LinearMixedModel)

Returns the estimate of σ², the variance of the conditional distribution of Y given B.

source
varest(m::GeneralizedLinearMixedModel)

Returns the estimate of ϕ², the variance of the conditional distribution of Y given B.

For models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.

For Gaussian models, this parameter is often called σ².

source
MixedModels.sdest — Function
sdest(m::LinearMixedModel)

Return the estimate of σ, the standard deviation of the per-observation noise.

source
sdest(m::GeneralizedLinearMixedModel)

Return the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.

For models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.

For Gaussian models, this parameter is often called σ.

source
varest(fm2)
654.9414514334794
sdest(fm2)
25.591823917678852
fm2.σ
25.591823917678852

Conditional modes of the random effects

The ranef extractor

MixedModels.ranef — Function
ranef(m::LinearMixedModel; uscale=false)

Return, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.

If uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.

For a named variant, see raneftables.

source
ranef(fm1)
1-element Vector{Matrix{Float64}}:
+

Individual components are returned by other extractors

MixedModels.varest — Function
varest(m::LinearMixedModel)

Returns the estimate of σ², the variance of the conditional distribution of Y given B.

source
varest(m::GeneralizedLinearMixedModel)

Returns the estimate of ϕ², the variance of the conditional distribution of Y given B.

For models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.

For Gaussian models, this parameter is often called σ².

source
MixedModels.sdest — Function
sdest(m::LinearMixedModel)

Return the estimate of σ, the standard deviation of the per-observation noise.

source
sdest(m::GeneralizedLinearMixedModel)

Return the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.

For models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.

For Gaussian models, this parameter is often called σ.

source
varest(fm2)
654.9414514334794
sdest(fm2)
25.591823917678852
fm2.σ
25.591823917678852

Conditional modes of the random effects

The ranef extractor

MixedModels.ranef — Function
ranef(m::LinearMixedModel; uscale=false)

Return, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.

If uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.

For a named variant, see raneftables.

source
ranef(fm1)
1-element Vector{Matrix{Float64}}:
  [-16.628221011733622 0.36951602248394705 … 53.57982326003441 -42.49434258554293]
fm1.b
1-element Vector{Matrix{Float64}}:
- [-16.628221011733622 0.36951602248394705 … 53.57982326003441 -42.49434258554293]

returns the conditional modes of the random effects given the observed data. That is, these are the values that maximize the conditional density of the random effects given the observed data. For a LinearMixedModel these are also the conditional means.

These are sometimes called the best linear unbiased predictors or BLUPs but that name is not particularly meaningful.

At a superficial level these can be considered as the "estimates" of the random effects, with a bit of hand waving, but pursuing this analogy too far usually results in confusion.

To obtain tables associating the values of the conditional modes with the levels of the grouping factor, use

MixedModels.raneftables — Function
raneftables(m::MixedModel; uscale = false)

Return the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.

Note

The API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.

source

as in

DataFrame(only(raneftables(fm1)))
6×2 DataFrame
Rowbatch(Intercept)
StringFloat64
1A-16.6282
2B0.369516
3C26.9747
4D-21.8014
5E53.5798
6F-42.4943

The corresponding conditional variances are returned by

MixedModels.condVar — Function
condVar(m::LinearMixedModel)

Return the conditional variances matrices of the random effects.

The random effects are returned by ranef as a vector of length k, where k is the number of random effects terms. The ith element is a matrix of size vᵢ × ℓᵢ where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor. Technically those values are the modes of the conditional distribution of the random effects given the observed data.

This function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ. These are the diagonal blocks from the conditional variance-covariance matrix,

s² Λ(Λ'Z'ZΛ + I)⁻¹Λ'
source
condVar(fm1)
1-element Vector{Array{Float64, 3}}:
+ [-16.628221011733622 0.36951602248394705 … 53.57982326003441 -42.49434258554293]

returns the conditional modes of the random effects given the observed data. That is, these are the values that maximize the conditional density of the random effects given the observed data. For a LinearMixedModel these are also the conditional means.

These are sometimes called the best linear unbiased predictors or BLUPs but that name is not particularly meaningful.

At a superficial level these can be considered as the "estimates" of the random effects, with a bit of hand waving, but pursuing this analogy too far usually results in confusion.

To obtain tables associating the values of the conditional modes with the levels of the grouping factor, use

MixedModels.raneftables — Function
raneftables(m::MixedModel; uscale = false)

Return the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.

Note

The API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.

source

as in

DataFrame(only(raneftables(fm1)))
6×2 DataFrame
Rowbatch(Intercept)
StringFloat64
1A-16.6282
2B0.369516
3C26.9747
4D-21.8014
5E53.5798
6F-42.4943

The corresponding conditional variances are returned by

MixedModels.condVar — Function
condVar(m::LinearMixedModel)

Return the conditional variances matrices of the random effects.

The random effects are returned by ranef as a vector of length k, where k is the number of random effects terms. The ith element is a matrix of size vᵢ × ℓᵢ where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor. Technically those values are the modes of the conditional distribution of the random effects given the observed data.

This function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ. These are the diagonal blocks from the conditional variance-covariance matrix,

s² Λ(Λ'Z'ZΛ + I)⁻¹Λ'
source
condVar(fm1)
1-element Vector{Array{Float64, 3}}:
  [362.3104675622471;;; 362.3104675622471;;; 362.3104675622471;;; 362.3104675622471;;; 362.3104675622471;;; 362.3104675622471]

Case-wise diagnostics and residual degrees of freedom

The leverage values

StatsAPI.leverage — Function
leverage(model::RegressionModel)

Return the diagonal of the projection matrix of the model.

source
leverage(fm1)
30-element Vector{Float64}:
  0.15650534082766315
  0.15650534082766315
@@ -612,4 +612,4 @@
                Coef.  Std. Error      z  Pr(>|z|)
 ─────────────────────────────────────────────────
 (Intercept)  22.9722    0.808572  28.41    <1e-99
-─────────────────────────────────────────────────
sum(leverage(fm4r))
27.472361767063312
+─────────────────────────────────────────────────
sum(leverage(fm4r))
27.472361767063312
diff --git a/dev/index.html b/dev/index.html index b2d5199c3..9853a08a8 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -MixedModels.jl Documentation · MixedModels
GitHub

MixedModels.jl Documentation

MixedModels.jl is a Julia package providing capabilities for fitting and examining linear and generalized linear mixed-effect models. It is similar in scope to the lme4 package for R.

+MixedModels.jl Documentation · MixedModels
GitHub

MixedModels.jl Documentation

MixedModels.jl is a Julia package providing capabilities for fitting and examining linear and generalized linear mixed-effect models. It is similar in scope to the lme4 package for R.

diff --git a/dev/mime/index.html b/dev/mime/index.html index d3ec70761..10d9cb810 100644 --- a/dev/mime/index.html +++ b/dev/mime/index.html @@ -82,4 +82,4 @@ Residual & 712.4038 & & & & & \\ \end{tabular}

This output can also be written directly to file:

open("model.md", "w") do io
     show(io, MIME("text/markdown"), kbm)
-end
+end diff --git a/dev/objects.inv b/dev/objects.inv index 177bbb532..d29406b5d 100644 Binary files a/dev/objects.inv and b/dev/objects.inv differ diff --git a/dev/optimization/index.html b/dev/optimization/index.html index af1b9790d..fc2ab72bd 100644 --- a/dev/optimization/index.html +++ b/dev/optimization/index.html @@ -126,8 +126,8 @@ ([0.8166315695343094, 0.011167254457244754, 0.28823768689703533], 1753.6956816568222)

A blocked Cholesky factor

A LinearMixedModel object contains two blocked matrices; a symmetric matrix A (only the lower triangle is stored) and a lower-triangular L which is the lower Cholesky factor of the updated and inflated A. In versions 4.0.0 and later of MixedModels only the blocks in the lower triangle are stored in A and L, as a Vector{AbstractMatrix{T}}.

BlockDescription shows the structure of the blocks

BlockDescription(fm2)
rows:     subj         fixed     
   36:   BlkDiag    
    3:    Dense         Dense     
-

Another change in v4.0.0 and later is that the last row of blocks is constructed from m.Xymat which contains the full-rank model matrix X with the response y concatenated on the right.

The operation of installing a new value of the variance parameters, θ, and updating L

MixedModels.setθ! — Function
setθ!(m::LinearMixedModel, v)

Install v as the θ parameters in m.

source
setθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)
-setθ!(bsamp::MixedModelFitCollection, i::Integer)

Install the values of the i'th θ value of bsamp.fits in bsamp.λ

source
MixedModels.updateL! — Function
updateL!(m::LinearMixedModel)

Update the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)

This is the crucial step in evaluating the objective, given a new parameter value.

source

is the central step in evaluating the objective (negative twice the log-likelihood).

Typically, the (1,1) block is the largest block in A and L and it has a special form, either Diagonal or UniformBlockDiagonal providing a compact representation and fast matrix multiplication or solutions of linear systems of equations.

Modifying the optimization process

The OptSummary object contains both input and output fields for the optimizer. To modify the optimization process the input fields can be changed after constructing the model but before fitting it.

Suppose, for example, that the user wishes to try a Nelder-Mead optimization method instead of the default BOBYQA (Bounded Optimization BY Quadratic Approximation) method.

fm2nm = LinearMixedModel(@formula(reaction ~ 1+days+(1+days|subj)), sleepstudy);
+

Another change in v4.0.0 and later is that the last row of blocks is constructed from m.Xymat which contains the full-rank model matrix X with the response y concatenated on the right.

The operation of installing a new value of the variance parameters, θ, and updating L

MixedModels.setθ! — Function
setθ!(m::LinearMixedModel, v)

Install v as the θ parameters in m.

source
setθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)
+setθ!(bsamp::MixedModelFitCollection, i::Integer)

Install the values of the i'th θ value of bsamp.fits in bsamp.λ

source
MixedModels.updateL! — Function
updateL!(m::LinearMixedModel)

Update the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)

This is the crucial step in evaluating the objective, given a new parameter value.

source

is the central step in evaluating the objective (negative twice the log-likelihood).

Typically, the (1,1) block is the largest block in A and L and it has a special form, either Diagonal or UniformBlockDiagonal providing a compact representation and fast matrix multiplication or solutions of linear systems of equations.

Modifying the optimization process

The OptSummary object contains both input and output fields for the optimizer. To modify the optimization process the input fields can be changed after constructing the model but before fitting it.

Suppose, for example, that the user wishes to try a Nelder-Mead optimization method instead of the default BOBYQA (Bounded Optimization BY Quadratic Approximation) method.

fm2nm = LinearMixedModel(@formula(reaction ~ 1+days+(1+days|subj)), sleepstudy);
 fm2nm.optsum.optimizer = :LN_NELDERMEAD;
 fit!(fm2nm; thin=1)
 fm2nm.optsum
Initial parameter vector: [1.0, 0.0, 1.0]
@@ -154,8 +154,8 @@
                            repeat(["BOBYQA"], length(bob))],
                    objective=[last.(nm); last.(bob)],
                    step=[1:length(nm); 1:length(bob)])
-plot(convdf, x=:step, y=:objective, color=:algorithm, Geom.line)
Example block output

Run time can be constrained with maxfeval and maxtime.

See the documentation for the NLopt package for details about the various settings.

Convergence to singular covariance matrices

To ensure identifiability of $\Sigma_\theta=\sigma^2\Lambda_\theta \Lambda_\theta$, the elements of $\theta$ corresponding to diagonal elements of $\Lambda_\theta$ are constrained to be non-negative. For example, in a trivial case of a single, simple, scalar, random-effects term as in fm1, the one-dimensional $\theta$ vector is the ratio of the standard deviation of the random effects to the standard deviation of the response. It happens that $-\theta$ produces the same log-likelihood but, by convention, we define the standard deviation to be the positive square root of the variance. Requiring the diagonal elements of $\Lambda_\theta$ to be non-negative is a generalization of using this positive square root.

If the optimization converges on the boundary of the feasible region, that is if one or more of the diagonal elements of $\Lambda_\theta$ is zero at convergence, the covariance matrix $\Sigma_\theta$ will be singular. This means that there will be linear combinations of random effects that are constant. Usually convergence to a singular covariance matrix is a sign of an over-specified model.

Singularity can be checked with the issingular predicate function.

MixedModels.issingular — Function
issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)

Test whether the model m is singular if the parameter vector is θ.

Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.

Note

For GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.

source
issingular(bsamp::MixedModelFitCollection;
-           atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))

Test each bootstrap sample for singularity of the corresponding fit.

Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.

See also issingular(::MixedModel).

source
issingular(fm2)
false

Generalized Linear Mixed-Effects Models

In a generalized linear model the responses are modelled as coming from a particular distribution, such as Bernoulli for binary responses or Poisson for responses that represent counts. The scalar distributions of individual responses differ only in their means, which are determined by a linear predictor expression $\eta=\bf X\beta$, where, as before, $\bf X$ is a model matrix derived from the values of covariates and $\beta$ is a vector of coefficients.

The unconstrained components of $\eta$ are mapped to the, possibly constrained, components of the mean response, $\mu$, via a scalar function, $g^{-1}$, applied to each component of $\eta$. For historical reasons, the inverse of this function, taking components of $\mu$ to the corresponding component of $\eta$ is called the link function and the more frequently used map from $\eta$ to $\mu$ is the inverse link.

A generalized linear mixed-effects model (GLMM) is defined, for the purposes of this package, by

\[\begin{aligned} +plot(convdf, x=:step, y=:objective, color=:algorithm, Geom.line)Example block output

Run time can be constrained with maxfeval and maxtime.

See the documentation for the NLopt package for details about the various settings.

Convergence to singular covariance matrices

To ensure identifiability of $\Sigma_\theta=\sigma^2\Lambda_\theta \Lambda_\theta$, the elements of $\theta$ corresponding to diagonal elements of $\Lambda_\theta$ are constrained to be non-negative. For example, in a trivial case of a single, simple, scalar, random-effects term as in fm1, the one-dimensional $\theta$ vector is the ratio of the standard deviation of the random effects to the standard deviation of the response. It happens that $-\theta$ produces the same log-likelihood but, by convention, we define the standard deviation to be the positive square root of the variance. Requiring the diagonal elements of $\Lambda_\theta$ to be non-negative is a generalization of using this positive square root.

If the optimization converges on the boundary of the feasible region, that is if one or more of the diagonal elements of $\Lambda_\theta$ is zero at convergence, the covariance matrix $\Sigma_\theta$ will be singular. This means that there will be linear combinations of random effects that are constant. Usually convergence to a singular covariance matrix is a sign of an over-specified model.

Singularity can be checked with the issingular predicate function.

MixedModels.issingular — Function
issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)

Test whether the model m is singular if the parameter vector is θ.

Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.

Note

For GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.

source
issingular(bsamp::MixedModelFitCollection;
+           atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))

Test each bootstrap sample for singularity of the corresponding fit.

Equality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.

See also issingular(::MixedModel).

source
issingular(fm2)
false

Generalized Linear Mixed-Effects Models

In a generalized linear model the responses are modelled as coming from a particular distribution, such as Bernoulli for binary responses or Poisson for responses that represent counts. The scalar distributions of individual responses differ only in their means, which are determined by a linear predictor expression $\eta=\bf X\beta$, where, as before, $\bf X$ is a model matrix derived from the values of covariates and $\beta$ is a vector of coefficients.

The unconstrained components of $\eta$ are mapped to the, possibly constrained, components of the mean response, $\mu$, via a scalar function, $g^{-1}$, applied to each component of $\eta$. For historical reasons, the inverse of this function, taking components of $\mu$ to the corresponding component of $\eta$ is called the link function and the more frequently used map from $\eta$ to $\mu$ is the inverse link.

A generalized linear mixed-effects model (GLMM) is defined, for the purposes of this package, by

\[\begin{aligned} (\mathcal{Y} | \mathcal{B}=\bf{b}) &\sim\mathcal{D}(\bf{g^{-1}(X\beta + Z b)},\phi)\\\\ \mathcal{B}&\sim\mathcal{N}(\bf{0},\Sigma_\theta) . \end{aligned}\]

where $\mathcal{D}$ indicates the distribution family parameterized by the mean and, when needed, a common scale parameter, $\phi$. (There is no scale parameter for Bernoulli or for Poisson. Specifying the mean completely determines the distribution.)

Distributions.Bernoulli — Type
Bernoulli(p)

A Bernoulli distribution is parameterized by a success rate p, which takes value 1 with probability p and 0 with probability 1-p.

\[P(X = k) = \begin{cases} @@ -283,4 +283,4 @@ btype: scold -1.05872 0.256803 -4.12 <1e-04 btype: shout -2.10528 0.258527 -8.14 <1e-15 situ: self -1.05558 0.210301 -5.02 <1e-06 -─────────────────────────────────────────────────────

This fit provided slightly better results (Laplace approximation to the deviance of 8151.400 versus 8151.583) but took 6 times as long. That is not terribly important when the times involved are a few seconds but can be important when the fit requires many hours or days of computing time.

+─────────────────────────────────────────────────────

This fit provided slightly better results (Laplace approximation to the deviance of 8151.400 versus 8151.583) but took 6 times as long. That is not terribly important when the times involved are a few seconds but can be important when the fit requires many hours or days of computing time.

diff --git a/dev/prediction/index.html b/dev/prediction/index.html index 97088cbc4..30930c9c0 100644 --- a/dev/prediction/index.html +++ b/dev/prediction/index.html @@ -193,4 +193,4 @@ ─────────────────────────────────────────────────── (Intercept) 259.607 7.53747 34.44 <1e-99 days 9.46755 0.783538 12.08 <1e-32 -───────────────────────────────────────────────────

For simulating from generalized linear mixed models, there is no type option because the observation-level always occurs at the level of the response and not of the linear predictor.

Warning

Simulating the model response in place may not yield the same result as simulating into a pre-allocated or new vector, depending on choice of pseudorandom number generator. Random number generation in Julia allows optimization based on type, and the internal storage type of the model response (currently a view into a matrix storing the concatenated fixed-effects model matrix and the response) may not match the type of a pre-allocated or new vector. See also discussion here.

Note

All the methods that take new data as a table construct an additional MixedModel behind the scenes, even when the new data is exactly the same as the data that the model was fitted to. For the simulation methods in particular, these thus form a convenience wrapper for constructing a new model and calling simulate without new data on that model with the parameters from the original model.

+───────────────────────────────────────────────────

For simulating from generalized linear mixed models, there is no type option because the observation-level always occurs at the level of the response and not of the linear predictor.

Warning

Simulating the model response in place may not yield the same result as simulating into a pre-allocated or new vector, depending on choice of pseudorandom number generator. Random number generation in Julia allows optimization based on type, and the internal storage type of the model response (currently a view into a matrix storing the concatenated fixed-effects model matrix and the response) may not match the type of a pre-allocated or new vector. See also discussion here.

Note

All the methods that take new data as a table construct an additional MixedModel behind the scenes, even when the new data is exactly the same as the data that the model was fitted to. For the simulation methods in particular, these thus form a convenience wrapper for constructing a new model and calling simulate without new data on that model with the parameters from the original model.

diff --git a/dev/rankdeficiency/index.html b/dev/rankdeficiency/index.html index 862e441c6..dbc27eb1d 100644 --- a/dev/rankdeficiency/index.html +++ b/dev/rankdeficiency/index.html @@ -52,4 +52,4 @@ spkr: old & load: yes 26.8642 21.7062 1.24 0.2159 prec: maintain & load: yes -18.6514 21.7062 -0.86 0.3902 spkr: old & prec: maintain & load: yes 15.4985 21.7062 0.71 0.4752 -──────────────────────────────────────────────────────────────────────────────

This may be useful when the PCA property suggests a random effects structure larger than only main effects but smaller than all interaction terms. This is also similar to the functionality provided by dummy in lme4, but as in the difference between zerocorr in Julia and || in R, there are subtle differences in how this expansion interacts with other terms in the random effects.

+──────────────────────────────────────────────────────────────────────────────

This may be useful when the PCA property suggests a random effects structure larger than only main effects but smaller than all interaction terms. This is also similar to the functionality provided by dummy in lme4, but as in the difference between zerocorr in Julia and || in R, there are subtle differences in how this expansion interacts with other terms in the random effects.

diff --git a/dev/search_index.js b/dev/search_index.js index a42222366..e346e1864 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"constructors/#Model-constructors","page":"Model constructors","title":"Model constructors","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The LinearMixedModel type represents a linear mixed-effects model. Typically it is constructed from a Formula and an appropriate Table type, usually a DataFrame.","category":"page"},{"location":"constructors/#Examples-of-linear-mixed-effects-model-fits","page":"Model constructors","title":"Examples of linear mixed-effects model fits","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For illustration, several data sets from the lme4 package for R are made available in .arrow format in this package. Often, for convenience, we will convert these to DataFrames. These data sets include the dyestuff and dyestuff2 data sets.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"using DisplayAs","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"using DataFrames, MixedModels, StatsModels\ndyestuff = MixedModels.dataset(:dyestuff)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"describe(DataFrame(dyestuff))","category":"page"},{"location":"constructors/#The-@formula-language-in-Julia","page":"Model constructors","title":"The @formula language in Julia","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.jl builds on the the Julia formula language provided by StatsModels.jl, which is similar to the formula language in R and is also based on the notation from Wilkinson and Rogers (1973). There are two ways to construct a formula in Julia. The first way is to enclose the formula expression in the @formula macro:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@formula","category":"page"},{"location":"constructors/#StatsModels.@formula","page":"Model constructors","title":"StatsModels.@formula","text":"@formula(ex)\n\nCapture and parse a formula expression as a Formula struct.\n\nA formula is an abstract specification of a dependence between left-hand and right-hand side variables as in, e.g., a regression model. Each side specifies at a high level how tabular data is to be converted to a numerical matrix suitable for modeling. This specification looks something like Julia code, is represented as a Julia Expr, but uses special syntax. The @formula macro takes an expression like y ~ 1 + a*b, transforms it according to the formula syntax rules into a lowered form (like y ~ 1 + a + b + a&b), and constructs a Formula struct which captures the original expression, the lowered expression, and the left- and right-hand-side.\n\nOperators that have special interpretations in this syntax are\n\n~ is the formula separator, where it is a binary operator (the first argument is the left-hand side, and the second is the right-hand side.\n+ concatenates variables as columns when generating a model matrix.\n& represents an interaction between two or more variables, which corresponds to a row-wise kronecker product of the individual terms (or element-wise product if all terms involved are continuous/scalar).\n* expands to all main effects and interactions: a*b is equivalent to a+b+a&b, a*b*c to a+b+c+a&b+a&c+b&c+a&b&c, etc.\n1, 0, and -1 indicate the presence (for 1) or absence (for 0 and -1) of an intercept column.\n\nThe rules that are applied are\n\nThe associative rule (un-nests nested calls to +, &, and *).\nThe distributive rule (interactions & distribute over concatenation +).\nThe * rule expands a*b to a+b+a&b (recursively).\nSubtraction is converted to addition and negation, so x-1 becomes x + -1 (applies only to subtraction of literal 1).\nSingle-argument & calls are stripped, so &(x) becomes the main effect x.\n\n\n\n\n\n","category":"macro"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The second way is to combine Terms with operators like +, &, ~, and others at \"run time\". This is especially useful if you wish to create a formula from a list a variable names. For instance, the following are equivalent:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@formula(y ~ 1 + a + b + a & b) == (term(:y) ~ term(1) + term(:a) + term(:b) + term(:a) & term(:b))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.jl provides additional formula syntax for representing random-effects terms. Most importantly, | separates random effects and their grouping factors (as in the formula extension used by the R package lme4. Much like with the base formula language, | can be used within the @formula macro and to construct a formula programmatically:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@formula(y ~ 1 + a + b + (1 + a + b | g))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"terms = sum(term(t) for t in [1, :a, :b])\ngroup = term(:g)\nresponse = term(:y)\nresponse ~ terms + (terms | group)","category":"page"},{"location":"constructors/#Models-with-simple,-scalar-random-effects","page":"Model constructors","title":"Models with simple, scalar random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A basic model with simple, scalar random effects for the levels of batch (the batch of an intermediate product, in this case) is declared and fit as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm = @formula(yield ~ 1 + (1|batch))\nfm1 = fit(MixedModel, fm, dyestuff)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"(If you are new to Julia you may find that this first fit takes an unexpectedly long time, due to Just-In-Time (JIT) compilation of the code. The subsequent calls to such functions are much faster.)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"using BenchmarkTools\ndyestuff2 = MixedModels.dataset(:dyestuff2)\n@benchmark fit(MixedModel, $fm, $dyestuff2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"By default, the model is fit by maximum likelihood. To use the REML criterion instead, add the optional named argument REML=true to the call to fit","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1reml = fit(MixedModel, fm, dyestuff, REML=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Floating-point-type-in-the-model","page":"Model constructors","title":"Floating-point type in the model","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The type of fm1","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"typeof(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"includes the floating point type used internally for the various matrices, vectors, and scalars that represent the model. At present, this will always be Float64 because the parameter estimates are optimized using the NLopt package which calls compiled C code that only allows for optimization with respect to a Float64 parameter vector.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"So in theory other floating point types, such as BigFloat or Float32, can be used to define a model but in practice only Float64 works at present.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In theory, theory and practice are the same. In practice, they aren't. – Anon","category":"page"},{"location":"constructors/#Simple,-scalar-random-effects","page":"Model constructors","title":"Simple, scalar random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A simple, scalar random effects term in a mixed-effects model formula is of the form (1|G). All random effects terms end with |G where G is the grouping factor for the random effect. The name or, more generally the expression, G, should evaluate to a categorical array that has a distinct set of levels. The random effects are associated with the levels of the grouping factor.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A scalar random effect is, as the name implies, one scalar value for each level of the grouping factor. A simple, scalar random effects term is of the form, (1|G). It corresponds to a shift in the intercept for each level of the grouping factor.","category":"page"},{"location":"constructors/#Models-with-vector-valued-random-effects","page":"Model constructors","title":"Models with vector-valued random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The sleepstudy data are observations of reaction time, reaction, on several subjects, subj, after 0 to 9 days of sleep deprivation, days. A model with random intercepts and random slopes for each subject, allowing for within-subject correlation of the slope and intercept, is fit as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sleepstudy = MixedModels.dataset(:sleepstudy)\nfm2 = fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Models-with-multiple,-scalar-random-effects-terms","page":"Model constructors","title":"Models with multiple, scalar random-effects terms","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A model for the Penicillin data incorporates random effects for the plate, and for the sample. As every sample is used on every plate these two factors are crossed.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"penicillin = MixedModels.dataset(:penicillin)\nfm3 = fit(MixedModel, @formula(diameter ~ 1 + (1|plate) + (1|sample)), penicillin)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In contrast, the cask grouping factor is nested within the batch grouping factor in the Pastes data.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"pastes = DataFrame(MixedModels.dataset(:pastes))\ndescribe(pastes)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"This can be expressed using the solidus (the \"/\" character) to separate grouping factors, read \"cask nested within batch\":","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm4a = fit(MixedModel, @formula(strength ~ 1 + (1|batch/cask)), pastes)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"If the levels of the inner grouping factor are unique across the levels of the outer grouping factor, then this nesting does not need to expressed explicitly in the model syntax. For example, defining sample to be the combination of batch and cask, yields a naming scheme where the nesting is apparent from the data even if not expressed in the formula. (That is, each level of sample occurs in conjunction with only one level of batch.) As such, this model is equivalent to the previous one.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"pastes.sample = (string.(pastes.cask, \"&\", pastes.batch))\nfm4b = fit(MixedModel, @formula(strength ~ 1 + (1|sample) + (1|batch)), pastes)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In observational studies it is common to encounter partially crossed grouping factors. For example, the InstEval data are course evaluations by students, s, of instructors, d. Additional covariates include the academic department, dept, in which the course was given and service, whether or not it was a service course.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"insteval = MixedModels.dataset(:insteval)\nfm5 = fit(MixedModel, @formula(y ~ 1 + service * dept + (1|s) + (1|d)), insteval)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Simplifying-the-random-effect-correlation-structure","page":"Model constructors","title":"Simplifying the random effect correlation structure","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.jl estimates not only the variance of the effects for each random effect level, but also the correlation between the random effects for different predictors. So, for the model of the sleepstudy data above, one of the parameters that is estimated is the correlation between each subject's random intercept (i.e., their baseline reaction time) and slope (i.e., their particular change in reaction time per day of sleep deprivation). In some cases, you may wish to simplify the random effects structure by removing these correlation parameters. This often arises when there are many random effects you want to estimate (as is common in psychological experiments with many conditions and covariates), since the number of random effects parameters increases as the square of the number of predictors, making these models difficult to estimate from limited data.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The special syntax zerocorr can be applied to individual random effects terms inside the @formula:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm2zerocorr_fm = fit(MixedModel, @formula(reaction ~ 1 + days + zerocorr(1 + days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Alternatively, correlations between parameters can be removed by including them as separate random effects terms:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj) + (days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Finally, for predictors that are categorical, MixedModels.jl will estimate correlations between each level. Notice the large number of correlation parameters if we treat days as a categorical variable by giving it contrasts:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Separating the 1 and days random effects into separate terms removes the correlations between the intercept and the levels of days, but not between the levels themselves:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj) + (days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"(Notice that the variance component for days: 1 is estimated as zero, so the correlations for this component are undefined and expressed as NaN, not a number.)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"An alternative is to force all the levels of days as indicators using fulldummy encoding.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fulldummy","category":"page"},{"location":"constructors/#MixedModels.fulldummy","page":"Model constructors","title":"MixedModels.fulldummy","text":"fulldummy(term::CategoricalTerm)\n\nAssign \"contrasts\" that include all indicator columns (dummy variables) and an intercept column.\n\nThis will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or \"shrinkage\", of the conditional modes.\n\nThe interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1 + fulldummy(days)|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"This fit produces a better fit as measured by the objective (negative twice the log-likelihood is 1610.8) but at the expense of adding many more parameters to the model. As a result, model comparison criteria such, as AIC and BIC, are inflated.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"But using zerocorr on the individual terms does remove the correlations between the levels:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + zerocorr(1 + days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj) + zerocorr(days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + zerocorr(1 + fulldummy(days)|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Fitting-generalized-linear-mixed-models","page":"Model constructors","title":"Fitting generalized linear mixed models","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"To create a GLMM representation, the distribution family for the response, and possibly the link function, must be specified.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"verbagg = MixedModels.dataset(:verbagg)\nverbaggform = @formula(r2 ~ 1 + anger + gender + btype + situ + mode + (1|subj) + (1|item));\ngm1 = fit(MixedModel, verbaggform, verbagg, Bernoulli())\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The canonical link, which is LogitLink for the Bernoulli distribution, is used if no explicit link is specified.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Note that, in keeping with convention in the GLM package, the distribution family for a binary (i.e. 0/1) response is the Bernoulli distribution. The Binomial distribution is only used when the response is the fraction of trials returning a positive, in which case the number of trials must be specified as the case weights.","category":"page"},{"location":"constructors/#Optional-arguments-to-fit","page":"Model constructors","title":"Optional arguments to fit","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"An alternative approach is to create the GeneralizedLinearMixedModel object then call fit! on it. The optional arguments fast and/or nAGQ can be passed to the optimization process via both fit and fit! (i.e these optimization settings are not used nor recognized when constructing the model).","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"As the name implies, fast=true, provides a faster but somewhat less accurate fit. These fits may suffice for model comparisons.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"gm1a = fit(MixedModel, verbaggform, verbagg, Bernoulli(), fast = true)\ndeviance(gm1a) - deviance(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@benchmark fit(MixedModel, $verbaggform, $verbagg, Bernoulli())","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@benchmark fit(MixedModel, $verbaggform, $verbagg, Bernoulli(), fast = true)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The optional argument nAGQ=k causes evaluation of the deviance function to use a k point adaptive Gauss-Hermite quadrature rule. This method only applies to models with a single, simple, scalar random-effects term, such as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"contraception = MixedModels.dataset(:contra)\ncontraform = @formula(use ~ 1 + age + abs2(age) + livch + urban + (1|dist));\nbernoulli = Bernoulli()\ndeviances = Dict{Symbol,Float64}()\nb = @benchmarkable deviances[:default] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli));\nrun(b)\nb = @benchmarkable deviances[:fast] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli, fast = true));\nrun(b)\nb = @benchmarkable deviances[:nAGQ] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli, nAGQ=9));\nrun(b)\nb = @benchmarkable deviances[:nAGQ_fast] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli, nAGQ=9, fast=true));\nrun(b)\nsort(deviances)","category":"page"},{"location":"constructors/#Extractor-functions","page":"Model constructors","title":"Extractor functions","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"LinearMixedModel and GeneralizedLinearMixedModel are subtypes of StatsAPI.RegressionModel which, in turn, is a subtype of StatsBase.StatisticalModel. Many of the generic extractors defined in the StatsBase package have methods for these models.","category":"page"},{"location":"constructors/#Model-fit-statistics","page":"Model constructors","title":"Model-fit statistics","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The statistics describing the quality of the model fit include","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"loglikelihood\naic\nbic\ndof\nnobs","category":"page"},{"location":"constructors/#StatsAPI.loglikelihood","page":"Model constructors","title":"StatsAPI.loglikelihood","text":"loglikelihood(model::StatisticalModel)\nloglikelihood(model::StatisticalModel, observation)\n\nReturn the log-likelihood of the model.\n\nWith an observation argument, return the contribution of observation to the log-likelihood of model.\n\nIf observation is a Colon, return a vector of each observation's contribution to the log-likelihood of the model. In other words, this is the vector of the pointwise log-likelihood contributions.\n\nIn general, sum(loglikehood(model, :)) == loglikelihood(model).\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.aic","page":"Model constructors","title":"StatsAPI.aic","text":"aic(model::StatisticalModel)\n\nAkaike's Information Criterion, defined as -2 log L + 2k, with L the likelihood of the model, and k its number of consumed degrees of freedom (as returned by dof).\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.bic","page":"Model constructors","title":"StatsAPI.bic","text":"bic(model::StatisticalModel)\n\nBayesian Information Criterion, defined as -2 log L + k log n, with L the likelihood of the model, k its number of consumed degrees of freedom (as returned by dof), and n the number of observations (as returned by nobs).\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.dof","page":"Model constructors","title":"StatsAPI.dof","text":"dof(model::StatisticalModel)\n\nReturn the number of degrees of freedom consumed in the model, including when applicable the intercept and the distribution's dispersion parameter.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.nobs","page":"Model constructors","title":"StatsAPI.nobs","text":"nobs(model::StatisticalModel)\n\nReturn the number of independent observations on which the model was fitted. Be careful when using this information, as the definition of an independent observation may vary depending on the model, on the format used to pass the data, on the sampling plan (if specified), etc.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"loglikelihood(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"aic(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"bic(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"dof(fm1) # 1 fixed effect, 2 variances","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"nobs(fm1) # 30 observations","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"loglikelihood(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In general the deviance of a statistical model fit is negative twice the log-likelihood adjusting for the saturated model.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"deviance(::StatisticalModel)","category":"page"},{"location":"constructors/#StatsAPI.deviance-Tuple{StatisticalModel}","page":"Model constructors","title":"StatsAPI.deviance","text":"deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}\n\nReturn the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.\n\nIf the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, u, plus the determinant of ΛZWZΛ + I, plus the sum of the squared deviance residuals.\n\n\n\n\n\n","category":"method"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Because it is not clear what the saturated model corresponding to a particular LinearMixedModel should be, negative twice the log-likelihood is called the objective.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"objective","category":"page"},{"location":"constructors/#MixedModels.objective","page":"Model constructors","title":"MixedModels.objective","text":"objective(m::LinearMixedModel)\n\nReturn negative twice the log-likelihood of model m\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"This value is also accessible as the deviance but the user should bear in mind that this doesn't have all the properties of a deviance which is corrected for the saturated model. For example, it is not necessarily non-negative.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"objective(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"deviance(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The value optimized when fitting a GeneralizedLinearMixedModel is the Laplace approximation to the deviance or an adaptive Gauss-Hermite evaluation.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.deviance!","category":"page"},{"location":"constructors/#MixedModels.deviance!","page":"Model constructors","title":"MixedModels.deviance!","text":"deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)\n\nUpdate m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.deviance!(gm1)","category":"page"},{"location":"constructors/#Fixed-effects-parameter-estimates","page":"Model constructors","title":"Fixed-effects parameter estimates","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The coef and fixef extractors both return the maximum likelihood estimates of the fixed-effects coefficients. They differ in their behavior in the rank-deficient case. The associated coefnames and fixefnames return the corresponding coefficient names.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coef\ncoefnames\nfixef\nfixefnames","category":"page"},{"location":"constructors/#StatsAPI.coef","page":"Model constructors","title":"StatsAPI.coef","text":"coef(model::StatisticalModel)\n\nReturn the coefficients of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.coefnames","page":"Model constructors","title":"StatsAPI.coefnames","text":"coefnames(model::StatisticalModel)\n\nReturn the names of the coefficients.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#MixedModels.fixef","page":"Model constructors","title":"MixedModels.fixef","text":"fixef(m::MixedModel)\n\nReturn the fixed-effects parameter vector estimate of m.\n\nIn the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#MixedModels.fixefnames","page":"Model constructors","title":"MixedModels.fixefnames","text":"fixefnames(m::MixedModel)\n\nReturn a (permuted and truncated in the rank-deficient case) vector of coefficient names.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coef(fm1)\ncoefnames(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fixef(fm1)\nfixefnames(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"An alternative extractor for the fixed-effects coefficient is the β property. Properties whose names are Greek letters usually have an alternative spelling, which is the name of the Greek letter.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1.β","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1.beta","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"gm1.β","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A full list of property names is returned by propertynames","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"propertynames(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"propertynames(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The variance-covariance matrix of the fixed-effects coefficients is returned by","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"vcov","category":"page"},{"location":"constructors/#StatsAPI.vcov","page":"Model constructors","title":"StatsAPI.vcov","text":"vcov(model::StatisticalModel)\n\nReturn the variance-covariance matrix for the coefficients of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"vcov(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"vcov(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The standard errors are the square roots of the diagonal elements of the estimated variance-covariance matrix of the fixed-effects coefficient estimators.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"stderror","category":"page"},{"location":"constructors/#StatsAPI.stderror","page":"Model constructors","title":"StatsAPI.stderror","text":"stderror(model::StatisticalModel)\n\nReturn the standard errors for the coefficients of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"stderror(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"stderror(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Finally, the coeftable generic produces a table of coefficient estimates, their standard errors, and their ratio. The p-values quoted here should be regarded as approximations.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coeftable","category":"page"},{"location":"constructors/#StatsAPI.coeftable","page":"Model constructors","title":"StatsAPI.coeftable","text":"coeftable(model::StatisticalModel; level::Real=0.95)\n\nReturn a table with coefficients and related statistics of the model. level determines the level for confidence intervals (by default, 95%).\n\nThe returned CoefTable object implements the Tables.jl interface, and can be converted e.g. to a DataFrame via using DataFrames; DataFrame(coeftable(model)).\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coeftable(fm2)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Covariance-parameter-estimates","page":"Model constructors","title":"Covariance parameter estimates","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The covariance parameters estimates, in the form shown in the model summary, are a VarCorr object","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"VarCorr(fm2)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"VarCorr(gm1)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Individual components are returned by other extractors","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"varest\nsdest","category":"page"},{"location":"constructors/#MixedModels.varest","page":"Model constructors","title":"MixedModels.varest","text":"varest(m::LinearMixedModel)\n\nReturns the estimate of σ², the variance of the conditional distribution of Y given B.\n\n\n\n\n\nvarest(m::GeneralizedLinearMixedModel)\n\nReturns the estimate of ϕ², the variance of the conditional distribution of Y given B.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ².\n\n\n\n\n\n","category":"function"},{"location":"constructors/#MixedModels.sdest","page":"Model constructors","title":"MixedModels.sdest","text":"sdest(m::LinearMixedModel)\n\nReturn the estimate of σ, the standard deviation of the per-observation noise.\n\n\n\n\n\nsdest(m::GeneralizedLinearMixedModel)\n\nReturn the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"varest(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sdest(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm2.σ","category":"page"},{"location":"constructors/#Conditional-modes-of-the-random-effects","page":"Model constructors","title":"Conditional modes of the random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The ranef extractor","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"ranef","category":"page"},{"location":"constructors/#MixedModels.ranef","page":"Model constructors","title":"MixedModels.ranef","text":"ranef(m::LinearMixedModel; uscale=false)\n\nReturn, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.\n\nIf uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.\n\nFor a named variant, see raneftables.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"ranef(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1.b","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"returns the conditional modes of the random effects given the observed data. That is, these are the values that maximize the conditional density of the random effects given the observed data. For a LinearMixedModel these are also the conditional means.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"These are sometimes called the best linear unbiased predictors or BLUPs but that name is not particularly meaningful.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"At a superficial level these can be considered as the \"estimates\" of the random effects, with a bit of hand waving, but pursuing this analogy too far usually results in confusion.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"To obtain tables associating the values of the conditional modes with the levels of the grouping factor, use","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"raneftables","category":"page"},{"location":"constructors/#MixedModels.raneftables","page":"Model constructors","title":"MixedModels.raneftables","text":"raneftables(m::MixedModel; uscale = false)\n\nReturn the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.\n\nnote: Note\nThe API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"as in","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"DataFrame(only(raneftables(fm1)))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The corresponding conditional variances are returned by","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"condVar","category":"page"},{"location":"constructors/#MixedModels.condVar","page":"Model constructors","title":"MixedModels.condVar","text":"condVar(m::LinearMixedModel)\n\nReturn the conditional variances matrices of the random effects.\n\nThe random effects are returned by ranef as a vector of length k, where k is the number of random effects terms. The ith element is a matrix of size vᵢ × ℓᵢ where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor. Technically those values are the modes of the conditional distribution of the random effects given the observed data.\n\nThis function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ. These are the diagonal blocks from the conditional variance-covariance matrix,\n\ns² Λ(Λ'Z'ZΛ + I)⁻¹Λ'\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"condVar(fm1)","category":"page"},{"location":"constructors/#Case-wise-diagnostics-and-residual-degrees-of-freedom","page":"Model constructors","title":"Case-wise diagnostics and residual degrees of freedom","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The leverage values","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"leverage","category":"page"},{"location":"constructors/#StatsAPI.leverage","page":"Model constructors","title":"StatsAPI.leverage","text":"leverage(model::RegressionModel)\n\nReturn the diagonal of the projection matrix of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"leverage(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"are used in diagnostics for linear regression models to determine cases that exert a strong influence on their own predicted response.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The documentation refers to a \"projection\". For a linear model without random effects the fitted values are obtained by orthogonal projection of the response onto the column span of the model matrix and the sum of the leverage values is the dimension of this column span. That is, the sum of the leverage values is the rank of the model matrix and n - sum(leverage(m)) is the degrees of freedom for residuals. The sum of the leverage values is also the trace of the so-called \"hat\" matrix, H. (The name \"hat matrix\" reflects the fact that hatmathbfy = mathbfH mathbfy. That is, H puts a hat on y.)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For a linear mixed model the sum of the leverage values will be between p, the rank of the fixed-effects model matrix, and p + q where q is the total number of random effects. This number does not represent a dimension (or \"degrees of freedom\") of a linear subspace of all possible fitted values because the projection is not an orthogonal projection. Nevertheless, it is a reasonable measure of the effective degrees of freedom of the model and n - sum(leverage(m)) can be considered the effective residual degrees of freedom.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For model fm1 the dimensions are","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"n, p, q, k = size(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"which implies that the sum of the leverage values should be in the range [1, 7]. The actual value is","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm1))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For model fm2 the dimensions are","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"n, p, q, k = size(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"providing a range of [2, 38] for the effective degrees of freedom for the model. The observed value is","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm2))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"When a model converges to a singular covariance, such as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm3 = fit(MixedModel, @formula(yield ~ 1+(1|batch)), MixedModels.dataset(:dyestuff2))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"the effective degrees of freedom is the lower bound.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm3))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Models for which the estimates of the variances of the random effects are large relative to the residual variance have effective degrees of freedom close to the upper bound.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm4 = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),\n MixedModels.dataset(:penicillin))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm4))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Also, a model fit by the REML criterion generally has larger estimates of the variance components and hence a larger effective degrees of freedom.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm4r = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),\n MixedModels.dataset(:penicillin), REML=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm4r))","category":"page"},{"location":"benchmarks/#Benchmark-Report-for-*/home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl*","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"","category":"section"},{"location":"benchmarks/#Job-Properties","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Job Properties","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Time of benchmark: 2 Oct 2018 - 13:42\nPackage commit: non gi\nJulia commit: 5d4eac\nJulia command flags: None\nEnvironment variables: None","category":"page"},{"location":"benchmarks/#Results","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Results","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Below is a table of this job's results, obtained by running the benchmarks. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The \"true\" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero.","category":"page"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"ID time GC time memory allocations\n`[\"crossed\", \"Assay:1+A+B*C+(1 G)+(1 H)\"]` 2.943 ms (5%) \n`[\"crossed\", \"Demand:1+U+V+W+X+(1 G)+(1 H)\"]` 2.775 ms (5%) \n`[\"crossed\", \"InstEval:1+A*I+(1 G)+(1 H)\"]` 1.247 s (5%) 114.131 ms\n`[\"crossed\", \"InstEval:1+A+(1 G)+(1 H)+(1 I)\"]` 1.999 s (5%)\n`[\"crossed\", \"Penicillin:1+(1 G)+(1 H)\"]` 2.697 ms (5%) \n`[\"crossed\", \"ScotsSec:1+A+U+V+(1 G)+(1 H)\"]` 4.833 ms (5%) \n`[\"crossed\", \"dialectNL:1+A+T+U+V+W+X+(1 G)+(1 H)+(1 I)\"]` 416.892 ms (5%)\n`[\"crossed\", \"egsingle:1+A+U+V+(1 G)+(1 H)\"]` 31.421 ms (5%) 3.427 ms\n`[\"crossed\", \"ml1m:1+(1 G)+(1 H)\"]` 36.714 s (5%) 225.872 ms\n`[\"crossed\", \"paulsim:1+S+T+U+(1 H)+(1 G)\"]` 14.097 ms (5%) \n`[\"crossedvector\", \"bs10:1+U+V+W+((1+U+V+W) G)+((1+U+V+W) H)\"]` 165.171 ms (5%) 3.149 ms\n`[\"crossedvector\", \"d3:1+U+((1+U) G)+((1+U) H)+((1+U) I)\"]` 49.023 s (5%)\n`[\"crossedvector\", \"d3:1+U+(1 G)+(1 H)+(1 I)\"]` 299.348 ms (5%)\n`[\"crossedvector\", \"gb12:1+S+T+U+V+W+X+Z+((1+S+U+W) G)+((1+S+T+V) H)\"]` 134.101 ms (5%) \n`[\"crossedvector\", \"kb07:1+S+T+U+V+W+X+Z+((1+S+T+U+V+W+X+Z) G)+((1+S+T+U+V+W+X+Z) H)\"]` 3.488 s (5%) 16.508 ms\n`[\"crossedvector\", \"kb07:1+S+T+U+V+W+X+Z+(1 G)+((0+S) G)+((0+T) G)+((0+U) G)+((0+V)\n`[\"nested\", \"Animal:1+(1 G)+(1 H)\"]` 1.261 ms (5%) \n`[\"nested\", \"Chem97:1+(1 G)+(1 H)\"]` 58.460 ms (5%) 6.975 ms\n`[\"nested\", \"Chem97:1+U+(1 G)+(1 H)\"]` 59.353 ms (5%) 7.019 ms\n`[\"nested\", \"Genetics:1+A+(1 G)+(1 H)\"]` 2.062 ms (5%) \n`[\"nested\", \"Pastes:1+(1 G)+(1 H)\"]` 2.298 ms (5%) \n`[\"nested\", \"Semi2:1+A+(1 G)+(1 H)\"]` 2.309 ms (5%) \n`[\"simplescalar\", \"Alfalfa:1+A*B+(1 G)\"]` 1.210 ms (5%) 208.80 KiB (1%)\n`[\"simplescalar\", \"Alfalfa:1+A+B+(1 G)\"]` 1.021 ms (5%) 168.47 KiB (1%)\n`[\"simplescalar\", \"AvgDailyGain:1+A*U+(1 G)\"]` 1.287 ms (5%) 193.33 KiB (1%)\n`[\"simplescalar\", \"AvgDailyGain:1+A+U+(1 G)\"]` 1.144 ms (5%) 169.59 KiB (1%)\n`[\"simplescalar\", \"BIB:1+A*U+(1 G)\"]` 1.574 ms (5%) 222.20 KiB (1%)\n`[\"simplescalar\", \"BIB:1+A+U+(1 G)\"]` 1.171 ms (5%) 171.31 KiB (1%)\n`[\"simplescalar\", \"Bond:1+A+(1 G)\"]` 958.770 μs (5%) 141.25 KiB (1%)\n`[\"simplescalar\", \"Cultivation:1+A*B+(1 G)\"]` 1.089 ms (5%) 173.38 KiB (1%)\n`[\"simplescalar\", \"Cultivation:1+A+(1 G)\"]` 1.138 ms (5%) 162.14 KiB (1%)\n`[\"simplescalar\", \"Cultivation:1+A+B+(1 G)\"]` 1.147 ms (5%) 173.47 KiB (1%)\n`[\"simplescalar\", \"Dyestuff2:1+(1 G)\"]` 830.840 μs (5%) 105.20 KiB (1%)\n`[\"simplescalar\", \"Dyestuff:1+(1 G)\"]` 974.091 μs (5%) 120.86 KiB (1%)\n`[\"simplescalar\", \"Exam:1+A*U+B+(1 G)\"]` 2.250 ms (5%) 1.17 MiB (1%)\n`[\"simplescalar\", \"Exam:1+A+B+U+(1 G)\"]` 2.133 ms (5%) 1.03 MiB (1%)\n`[\"simplescalar\", \"Gasoline:1+U+(1 G)\"]` 1.164 ms (5%) 162.03 KiB (1%)\n`[\"simplescalar\", \"Hsb82:1+A+B+C+U+(1 G)\"]` 3.048 ms (5%) 2.12 MiB (1%)\n`[\"simplescalar\", \"IncBlk:1+A+U+V+W+Z+(1 G)\"]` 1.226 ms (5%) 208.83 KiB (1%)\n`[\"simplescalar\", \"Mississippi:1+A+(1 G)\"]` 980.968 μs (5%) 145.75 KiB (1%)\n`[\"simplescalar\", \"PBIB:1+A+(1 G)\"]` 1.509 ms (5%) 234.47 KiB (1%)\n`[\"simplescalar\", \"Rail:1+(1 G)\"]` 1.251 ms (5%) 151.34 KiB (1%)\n`[\"simplescalar\", \"Semiconductor:1+A*B+(1 G)\"]` 1.313 ms (5%) 222.95 KiB (1%)\n`[\"simplescalar\", \"TeachingII:1+A+T+U+V+W+X+Z+(1 G)\"]` 1.483 ms (5%) 284.53 KiB (1%)\n`[\"simplescalar\", \"cake:1+A*B+(1 G)\"]` 1.606 ms (5%) 412.83 KiB (1%)\n`[\"simplescalar\", \"ergoStool:1+A+(1 G)\"]` 1.057 ms (5%) 155.59 KiB (1%)\n`[\"singlevector\", \"Early:1+U+U&A+((1+U) G)\"]` 20.373 ms (5%) 3.47 MiB (1%)\n`[\"singlevector\", \"HR:1+A*U+V+((1+U) G)\"]` 5.183 ms (5%) 915.00 KiB (1%)\n`[\"singlevector\", \"Oxboys:1+U+((1+U) G)\"]` 13.207 ms (5%) 1.93 MiB (1%)\n`[\"singlevector\", \"SIMS:1+U+((1+U) G)\"]` 61.675 ms (5%) 12.86 MiB (1%)\n`[\"singlevector\", \"WWheat:1+U+((1+U) G)\"]` 7.311 ms (5%) 902.31 KiB (1%)\n`[\"singlevector\", \"Weights:1+A*U+((1+U) G)\"]` 18.303 ms (5%) 3.20 MiB (1%)\n`[\"singlevector\", \"sleepstudy:1+U+((1+U) G)\"]` 4.829 ms (5%) 797.48 KiB (1%)\n`[\"singlevector\", \"sleepstudy:1+U+(1 G)+((0+U) G)\"]` 3.219 ms (5%) ","category":"page"},{"location":"benchmarks/#Benchmark-Group-List","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Group List","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Here's a list of all the benchmark groups executed by this job:","category":"page"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"[\"crossed\"]\n[\"crossedvector\"]\n[\"nested\"]\n[\"simplescalar\"]\n[\"singlevector\"]","category":"page"},{"location":"benchmarks/#Julia-versioninfo","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Julia versioninfo","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Julia Version 1.0.0\nCommit 5d4eaca0c9 (2018-08-08 20:58 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n Ubuntu 18.04.1 LTS\n uname: Linux 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64\n CPU: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz: \n speed user nice sys idle irq\n #1 1690 MHz 140498 s 134 s 18382 s 1495130 s 0 s\n #2 2513 MHz 131505 s 16 s 18277 s 1504212 s 0 s\n #3 1900 MHz 145131 s 581 s 18892 s 1485409 s 0 s\n #4 1682 MHz 190751 s 38 s 17941 s 1445446 s 0 s\n \n Memory: 15.554645538330078 GB (10502.1171875 MB free)\n Uptime: 16578.0 sec\n Load Avg: 1.4091796875 2.07080078125 1.63037109375\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge)","category":"page"},{"location":"api/#API","page":"API","title":"API","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"In addition to its own functionality, MixedModels.jl also implements extensive support for the StatsAPI.StatisticalModel and StatsAPI.RegressionModel API.","category":"page"},{"location":"api/#Types","page":"API","title":"Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Modules = [MixedModels]\nOrder = [:type]","category":"page"},{"location":"api/#MixedModels.BlockDescription","page":"API","title":"MixedModels.BlockDescription","text":"BlockDescription\n\nDescription of blocks of A and L in a LinearMixedModel\n\nFields\n\nblknms: Vector{String} of block names\nblkrows: Vector{Int} of the number of rows in each block\nALtypes: Matrix{String} of datatypes for blocks in A and L.\n\nWhen a block in L is the same type as the corresponding block in A, it is described with a single name, such as Dense. When the types differ the entry in ALtypes is of the form Diag/Dense, as determined by a shorttype method.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.BlockedSparse","page":"API","title":"MixedModels.BlockedSparse","text":"BlockedSparse{Tv,S,P}\n\nA SparseMatrixCSC whose nonzeros form blocks of rows or columns or both.\n\nMembers\n\ncscmat: SparseMatrixCSC{Tv, Int32} representation for general calculations\nnzasmat: nonzeros of cscmat as a dense matrix\ncolblkptr: pattern of blocks of columns\n\nThe only time these are created are as products of ReMats.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.FeMat","page":"API","title":"MixedModels.FeMat","text":"FeMat{T,S}\n\nA matrix and a (possibly) weighted copy of itself.\n\nTypically, an FeMat represents the fixed-effects model matrix with the response (y) concatenated as a final column.\n\nnote: Note\nFeMat is not the same as FeTerm.\n\nFields\n\nxy: original matrix, called xy b/c in practice this is hcat(fullrank(X), y)\nwtxy: (possibly) weighted copy of xy (shares storage with xy until weights are applied)\n\nUpon construction the xy and wtxy fields refer to the same matrix\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.FeTerm","page":"API","title":"MixedModels.FeTerm","text":"FeTerm{T,S}\n\nTerm with an explicit, constant matrix representation\n\nTypically, an FeTerm represents the model matrix for the fixed effects.\n\nnote: Note\nFeTerm is not the same as FeMat!\n\nFields\n\nx: full model matrix\npiv: pivot Vector{Int} for moving linearly dependent columns to the right\nrank: computational rank of x\ncnames: vector of column names\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.FeTerm-Tuple{SparseArrays.SparseMatrixCSC, AbstractVector{String}}","page":"API","title":"MixedModels.FeTerm","text":"FeTerm(X::SparseMatrixCSC, cnms)\n\nConvenience constructor for a sparse FeTerm assuming full rank, identity pivot and unit weights.\n\nNote: automatic rank deficiency handling may be added to this method in the future, as discussed in the vignette \"Rank deficiency in mixed-effects models\" for general FeTerm.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.FeTerm-Union{Tuple{T}, Tuple{AbstractMatrix{T}, Any}} where T","page":"API","title":"MixedModels.FeTerm","text":"FeTerm(X::AbstractMatrix, cnms)\n\nConvenience constructor for FeTerm that computes the rank and pivot with unit weights.\n\nSee the vignette \"Rank deficiency in mixed-effects models\" for more information on the computation of the rank and pivot.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.GaussHermiteNormalized","page":"API","title":"MixedModels.GaussHermiteNormalized","text":"GaussHermiteNormalized{K}\n\nA struct with 2 SVector{K,Float64} members\n\nz: abscissae for the K-point Gauss-Hermite quadrature rule on the Z scale\nwt: Gauss-Hermite weights normalized to sum to unity\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.GeneralizedLinearMixedModel","page":"API","title":"MixedModels.GeneralizedLinearMixedModel","text":"GeneralizedLinearMixedModel\n\nGeneralized linear mixed-effects model representation\n\nFields\n\nLMM: a LinearMixedModel - the local approximation to the GLMM.\nβ: the pivoted and possibly truncated fixed-effects vector\nβ₀: similar to β. Used in the PIRLS algorithm if step-halving is needed.\nθ: covariance parameter vector\nb: similar to u, equivalent to broadcast!(*, b, LMM.Λ, u)\nu: a vector of matrices of random effects\nu₀: similar to u. Used in the PIRLS algorithm if step-halving is needed.\nresp: a GlmResp object\nη: the linear predictor\nwt: vector of prior case weights, a value of T[] indicates equal weights.\n\nThe following fields are used in adaptive Gauss-Hermite quadrature, which applies only to models with a single random-effects term, in which case their lengths are the number of levels in the grouping factor for that term. Otherwise they are zero-length vectors.\n\ndevc: vector of deviance components\ndevc0: vector of deviance components at offset of zero\nsd: approximate standard deviation of the conditional density\nmult: multiplier\n\nProperties\n\nIn addition to the fieldnames, the following names are also accessible through the . extractor\n\ntheta: synonym for θ\nbeta: synonym for β\nσ or sigma: common scale parameter (value is NaN for distributions without a scale parameter)\nlowerbd: vector of lower bounds on the combined elements of β and θ\nformula, trms, A, L, and optsum: fields of the LMM field\nX: fixed-effects model matrix\ny: response vector\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.Grouping","page":"API","title":"MixedModels.Grouping","text":"struct Grouping <: StatsModels.AbstractContrasts end\n\nA placeholder type to indicate that a categorical variable is only used for grouping and not for contrasts. When creating a CategoricalTerm, this skips constructing the contrasts matrix which makes it robust to large numbers of levels, while still holding onto the vector of levels and constructing the level-to-index mapping (invindex field of the ContrastsMatrix.).\n\nNote that calling modelcols on a CategoricalTerm{Grouping} is an error.\n\nExamples\n\njulia> schema((; grp = string.(1:100_000)))\n# out-of-memory error\n\njulia> schema((; grp = string.(1:100_000)), Dict(:grp => Grouping()))\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LikelihoodRatioTest","page":"API","title":"MixedModels.LikelihoodRatioTest","text":"LikelihoodRatioTest\n\nResults of MixedModels.likelihoodratiotest\n\nFields\n\nformulas: Vector of model formulae\nmodels: NamedTuple of the dof and deviance of the models\ntests: NamedTuple of the sequential dofdiff, deviancediff, and resulting pvalues\n\nProperties\n\ndeviance : note that this is actually -2 log likelihood for linear models (i.e. without subtracting the constant for a saturated model)\npvalues\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LinearMixedModel","page":"API","title":"MixedModels.LinearMixedModel","text":"LinearMixedModel(y, Xs, form, wts=[], σ=nothing, amalgamate=true)\n\nPrivate constructor for a LinearMixedModel.\n\nTo construct a model, you only need the response (y), already assembled model matrices (Xs), schematized formula (form) and weights (wts). Everything else in the structure can be derived from these quantities.\n\nnote: Note\nThis method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LinearMixedModel-2","page":"API","title":"MixedModels.LinearMixedModel","text":"LinearMixedModel\n\nLinear mixed-effects model representation\n\nFields\n\nformula: the formula for the model\nreterms: a Vector{AbstractReMat{T}} of random-effects terms.\nXymat: horizontal concatenation of a full-rank fixed-effects model matrix X and response y as an FeMat{T}\nfeterm: the fixed-effects model matrix as an FeTerm{T}\nsqrtwts: vector of square roots of the case weights. Can be empty.\nparmap : Vector{NTuple{3,Int}} of (block, row, column) mapping of θ to λ\ndims : NamedTuple{(:n, :p, :nretrms),NTuple{3,Int}} of dimensions. p is the rank of X, which may be smaller than size(X, 2).\nA: a Vector{AbstractMatrix} containing the row-major packed lower triangle of hcat(Z,X,y)'hcat(Z,X,y)\nL: the blocked lower Cholesky factor of Λ'AΛ+I in the same Vector representation as A\noptsum: an OptSummary object\n\nProperties\n\nθ or theta: the covariance parameter vector used to form λ\nβ or beta: the fixed-effects coefficient vector\nλ or lambda: a vector of lower triangular matrices repeated on the diagonal blocks of Λ\nσ or sigma: current value of the standard deviation of the per-observation noise\nb: random effects on the original scale, as a vector of matrices\nu: random effects on the orthogonal scale, as a vector of matrices\nlowerbd: lower bounds on the elements of θ\nX: the fixed-effects model matrix\ny: the response vector\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LinearMixedModel-Union{Tuple{T}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm, Any}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm, Any, Any}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm, Any, Any, Any}} where T","page":"API","title":"MixedModels.LinearMixedModel","text":"LinearMixedModel(y, feterm, reterms, form, wts=[], σ=nothing; amalgamate=true)\n\nPrivate constructor for a LinearMixedModel given already assembled fixed and random effects.\n\nTo construct a model, you only need a vector of FeMats (the fixed-effects model matrix and response), a vector of AbstractReMat (the random-effects model matrices), the formula and the weights. Everything else in the structure can be derived from these quantities.\n\nnote: Note\nThis method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.MixedModel","page":"API","title":"MixedModels.MixedModel","text":"MixedModel\n\nAbstract type for mixed models. MixedModels.jl implements two subtypes: LinearMixedModel and GeneralizedLinearMixedModel. See the documentation for each for more details.\n\nThis type is primarily used for dispatch in fit. Without a distribution and link function specified, a LinearMixedModel will be fit. When a distribution/link function is provided, a GeneralizedLinearModel is fit, unless that distribution is Normal and the link is IdentityLink, in which case the resulting GLMM would be equivalent to a LinearMixedModel anyway and so the simpler, equivalent LinearMixedModel will be fit instead.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.MixedModelBootstrap","page":"API","title":"MixedModels.MixedModelBootstrap","text":"MixedModelBootstrap{T<:AbstractFloat} <: MixedModelFitCollection{T}\n\nObject returned by parametericbootstrap with fields\n\nfits: the parameter estimates from the bootstrap replicates as a vector of named tuples.\nλ: Vector{LowerTriangular{T,Matrix{T}}} containing copies of the λ field from ReMat model terms\ninds: Vector{Vector{Int}} containing copies of the inds field from ReMat model terms\nlowerbd: Vector{T} containing the vector of lower bounds (corresponds to the identically named field of OptSummary)\nfcnames: NamedTuple whose keys are the grouping factor names and whose values are the column names\n\nThe schema of fits is, by default,\n\nTables.Schema:\n :objective T\n :σ T\n :β NamedTuple{β_names}{NTuple{p,T}}\n :se StaticArrays.SArray{Tuple{p},T,1,p}\n :θ StaticArrays.SArray{Tuple{k},T,1,k}\n\nwhere the sizes, p and k, of the β and θ elements are determined by the model.\n\nCharacteristics of the bootstrap replicates can be extracted as properties. The σs and σρs properties unravel the σ and θ estimates into estimates of the standard deviations and correlations of the random-effects terms.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.MixedModelFitCollection","page":"API","title":"MixedModels.MixedModelFitCollection","text":"MixedModelFitCollection{T<:AbstractFloat}\n\nAbstract supertype for MixedModelBootstrap and related functionality in other packages.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.MixedModelProfile","page":"API","title":"MixedModels.MixedModelProfile","text":" MixedModelProfile{T<:AbstractFloat}\n\nType representing a likelihood profile of a LinearMixedModel, including associated interpolation splines.\n\nThe function profile is used for computing profiles, while confint provides a useful method for constructing confidence intervals from a MixedModelProfile.\n\nnote: Note\nThe exact fields and their representation are considered implementation details and are not part of the public API.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.OptSummary","page":"API","title":"MixedModels.OptSummary","text":"OptSummary\n\nSummary of an NLopt optimization\n\nFields\n\ninitial: a copy of the initial parameter values in the optimization\nfinitial: the initial value of the objective\nlowerbd: lower bounds on the parameter values\nftol_rel: as in NLopt\nftol_abs: as in NLopt\nxtol_rel: as in NLopt\nxtol_abs: as in NLopt\ninitial_step: as in NLopt\nmaxfeval: as in NLopt (maxeval)\nmaxtime: as in NLopt\nfinal: a copy of the final parameter values from the optimization\nfmin: the final value of the objective\nfeval: the number of function evaluations\noptimizer: the name of the optimizer used, as a Symbol\nreturnvalue: the return value, as a Symbol\nxtol_zero_abs: the tolerance for a near zero parameter to be considered practically zero\nftol_zero_abs: the tolerance for change in the objective for setting a near zero parameter to zero\nfitlog: A vector of tuples of parameter and objectives values from steps in the optimization\nnAGQ: number of adaptive Gauss-Hermite quadrature points in deviance evaluation for GLMMs\nREML: use the REML criterion for LMM fits\nsigma: a priori value for the residual standard deviation for LMM\n\nThe last three fields are MixedModels functionality and not related directly to the NLopt package or algorithms.\n\nnote: Note\nThe internal storage of the parameter values within fitlog may change in the future to use a different subtype of AbstractVector (e.g., StaticArrays.SVector) for each snapshot without being considered a breaking change.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.PCA","page":"API","title":"MixedModels.PCA","text":"PCA{T<:AbstractFloat}\n\nPrincipal Components Analysis\n\nFields\n\ncovcorr covariance or correlation matrix\nsv singular value decomposition\nrnames rownames of the original matrix\ncorr is this a correlation matrix?\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.RaggedArray","page":"API","title":"MixedModels.RaggedArray","text":"RaggedArray{T,I}\n\nA \"ragged\" array structure consisting of values and indices\n\nFields\n\nvals: a Vector{T} containing the values\ninds: a Vector{I} containing the indices\n\nFor this application a RaggedArray is used only in its sum! method.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.ReMat","page":"API","title":"MixedModels.ReMat","text":"ReMat{T,S} <: AbstractMatrix{T}\n\nA section of a model matrix generated by a random-effects term.\n\nFields\n\ntrm: the grouping factor as a StatsModels.CategoricalTerm\nrefs: indices into the levels of the grouping factor as a Vector{Int32}\nlevels: the levels of the grouping factor\ncnames: the names of the columns of the model matrix generated by the left-hand side of the term\nz: transpose of the model matrix generated by the left-hand side of the term\nwtz: a weighted copy of z (z and wtz are the same object for unweighted cases)\nλ: a LowerTriangular or Diagonal matrix of size S×S\ninds: a Vector{Int} of linear indices of the potential nonzeros in λ\nadjA: the adjoint of the matrix as a SparseMatrixCSC{T}\nscratch: a Matrix{T}\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.TableColumns","page":"API","title":"MixedModels.TableColumns","text":"TableColumns\n\nA structure containing the column names for the numeric part of the profile table.\n\nThe struct also contains a Dict giving the column ranges for Symbols like :σ and :β. Finally it contains a scratch vector used to accumulate to values in a row of the profile table.\n\nnote: Note\nThis is an internal structure used in MixedModelProfile. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.UniformBlockDiagonal","page":"API","title":"MixedModels.UniformBlockDiagonal","text":"UniformBlockDiagonal{T}\n\nHomogeneous block diagonal matrices. k diagonal blocks each of size m×m\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.VarCorr","page":"API","title":"MixedModels.VarCorr","text":"VarCorr\n\nInformation from the fitted random-effects variance-covariance matrices.\n\nMembers\n\nσρ: a NamedTuple of NamedTuples as returned from σρs\ns: the estimate of the per-observation dispersion parameter\n\nThe main purpose of defining this type is to isolate the logic in the show method.\n\n\n\n\n\n","category":"type"},{"location":"api/#Exported-Functions","page":"API","title":"Exported Functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Modules = [MixedModels]\nPrivate = false\nOrder = [:function]","category":"page"},{"location":"api/#LinearAlgebra.cond-Tuple{MixedModel}","page":"API","title":"LinearAlgebra.cond","text":"cond(m::MixedModel)\n\nReturn a vector of condition numbers of the λ matrices for the random-effects terms\n\n\n\n\n\n","category":"method"},{"location":"api/#LinearAlgebra.logdet-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"LinearAlgebra.logdet","text":"logdet(m::LinearMixedModel)\n\nReturn the value of log(det(Λ'Z'ZΛ + I)) + m.optsum.REML * log(det(LX*LX')) evaluated in place.\n\nHere LX is the diagonal term corresponding to the fixed-effects in the blocked lower Cholesky factor.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.GHnorm-Tuple{Int64}","page":"API","title":"MixedModels.GHnorm","text":"GHnorm(k::Int)\n\nReturn the (unique) GaussHermiteNormalized{k} object.\n\nThe function values are stored (memoized) when first evaluated. Subsequent evaluations for the same k have very low overhead.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.coefpvalues-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.coefpvalues","text":"coefpvalues(bsamp::MixedModelFitCollection)\n\nReturn a rowtable with columns (:iter, :coefname, :β, :se, :z, :p)\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.condVar-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.condVar","text":"condVar(m::LinearMixedModel)\n\nReturn the conditional variances matrices of the random effects.\n\nThe random effects are returned by ranef as a vector of length k, where k is the number of random effects terms. The ith element is a matrix of size vᵢ × ℓᵢ where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor. Technically those values are the modes of the conditional distribution of the random effects given the observed data.\n\nThis function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ. These are the diagonal blocks from the conditional variance-covariance matrix,\n\ns² Λ(Λ'Z'ZΛ + I)⁻¹Λ'\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.condVartables-Union{Tuple{MixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.condVartables","text":"condVartables(m::LinearMixedModel)\n\nReturn the conditional covariance matrices of the random effects as a NamedTuple of columntables\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fitted!-Union{Tuple{T}, Tuple{AbstractArray{T}, LinearMixedModel{T}}} where T","page":"API","title":"MixedModels.fitted!","text":"fitted!(v::AbstractArray{T}, m::LinearMixedModel{T})\n\nOverwrite v with the fitted values from m.\n\nSee also fitted.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fixef-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.fixef","text":"fixef(m::MixedModel)\n\nReturn the fixed-effects parameter vector estimate of m.\n\nIn the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fixefnames-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.fixefnames","text":"fixefnames(m::MixedModel)\n\nReturn a (permuted and truncated in the rank-deficient case) vector of coefficient names.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fnames-Tuple{MixedModel}","page":"API","title":"MixedModels.fnames","text":"fnames(m::MixedModel)\n\nReturn the names of the grouping factors for the random-effects terms.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fulldummy-Tuple{StatsModels.CategoricalTerm}","page":"API","title":"MixedModels.fulldummy","text":"fulldummy(term::CategoricalTerm)\n\nAssign \"contrasts\" that include all indicator columns (dummy variables) and an intercept column.\n\nThis will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or \"shrinkage\", of the conditional modes.\n\nThe interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.issingular","page":"API","title":"MixedModels.issingular","text":"issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\n\nTest whether the model m is singular if the parameter vector is θ.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nnote: Note\nFor GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.issingular-Tuple{MixedModels.MixedModelFitCollection}","page":"API","title":"MixedModels.issingular","text":"issingular(bsamp::MixedModelFitCollection;\n atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))\n\nTest each bootstrap sample for singularity of the corresponding fit.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nSee also issingular(::MixedModel).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.lowerbd-Union{Tuple{ReMat{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.lowerbd","text":"lowerbd{T}(A::ReMat{T})\n\nReturn the vector of lower bounds on the parameters, θ associated with A\n\nThese are the elements in the lower triangle of A.λ in column-major ordering. Diagonals have a lower bound of 0. Off-diagonals have a lower-bound of -Inf.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.objective!","page":"API","title":"MixedModels.objective!","text":"objective!(m::LinearMixedModel, θ)\nobjective!(m::LinearMixedModel)\n\nEquivalent to objective(updateL!(setθ!(m, θ))).\n\nWhen m has a single, scalar random-effects term, θ can be a scalar.\n\nThe one-argument method curries and returns a single-argument function of θ.\n\nNote that these methods modify m. The calling function is responsible for restoring the optimal θ.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.objective-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.objective","text":"objective(m::LinearMixedModel)\n\nReturn negative twice the log-likelihood of model m\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.parametricbootstrap-Union{Tuple{T}, Tuple{Random.AbstractRNG, Integer, MixedModel{T}}, Tuple{Random.AbstractRNG, Integer, MixedModel{T}, Type{<:AbstractFloat}}} where T","page":"API","title":"MixedModels.parametricbootstrap","text":"parametricbootstrap([rng::AbstractRNG], nsamp::Integer, m::MixedModel{T}, ftype=T;\n β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))\n\nPerform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.\n\nThe default random number generator is Random.GLOBAL_RNG.\n\nftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.\n\nKeyword Arguments\n\nβ, σ, and θ are the values of m's parameters for simulating the responses.\nσ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for\n\nfamilies with a dispersion parameter.\n\nprogress controls whether the progress bar is shown. Note that the progress\n\nbar is automatically disabled for non-interactive (i.e. logging) contexts.\n\noptsum_overrides is used to override values of OptSummary in the models\n\nfit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.\n\nnote: Note\nAll coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.pirls!-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}, Tuple{GeneralizedLinearMixedModel{T}, Any}, Tuple{GeneralizedLinearMixedModel{T}, Any, Any}} where T","page":"API","title":"MixedModels.pirls!","text":"pirls!(m::GeneralizedLinearMixedModel)\n\nUse Penalized Iteratively Reweighted Least Squares (PIRLS) to determine the conditional modes of the random effects.\n\nWhen varyβ is true both u and β are optimized with PIRLS. Otherwise only u is optimized and β is held fixed.\n\nPassing verbose = true provides verbose output of the iterations.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.profile-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.profile","text":"profile(m::LinearMixedModel; threshold = 4)\n\nReturn a MixedModelProfile for the objective of m with respect to the fixed-effects coefficients.\n\nm is refit! if !isfitted(m).\n\nProfiling starts at the parameter estimate and continues until reaching a parameter bound or the absolute value of ζ exceeds threshold.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.profilevc-Union{Tuple{T}, Tuple{LinearMixedModel{T}, T, AbstractVector{T}}} where T","page":"API","title":"MixedModels.profilevc","text":" profilevc(m::LinearMixedModel{T}, val::T, rowj::AbstractVector{T}) where {T}\n\nProfile an element of the variance components.\n\nnote: Note\nThis method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.profileσ-Union{Tuple{T}, Tuple{LinearMixedModel{T}, MixedModels.TableColumns{T}}} where T","page":"API","title":"MixedModels.profileσ","text":"profileσ(m::LinearMixedModel, tc::TableColumns; threshold=4)\n\nReturn a Table of the profile of σ for model m. The profile extends to where the magnitude of ζ exceeds threshold.\n\nnote: Note\nThis method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.pwrss-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.pwrss","text":"pwrss(m::LinearMixedModel)\n\nThe penalized, weighted residual sum-of-squares.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.ranef-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.ranef","text":"ranef(m::LinearMixedModel; uscale=false)\n\nReturn, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.\n\nIf uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.\n\nFor a named variant, see raneftables.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.raneftables-Union{Tuple{MixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.raneftables","text":"raneftables(m::MixedModel; uscale = false)\n\nReturn the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.\n\nnote: Note\nThe API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.refit!-Tuple{GeneralizedLinearMixedModel}","page":"API","title":"MixedModels.refit!","text":"refit!(m::GeneralizedLinearMixedModel[, y::Vector];\n fast::Bool = (length(m.θ) == length(m.optsum.final)),\n nAGQ::Integer = m.optsum.nAGQ,\n kwargs...)\n\nRefit the model m after installing response y.\n\nIf y is omitted the current response vector is used.\n\nIf not specified, the fast and nAGQ options from the previous fit are used. kwargs are the same as fit!\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.refit!-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.refit!","text":"refit!(m::LinearMixedModel[, y::Vector]; REML=m.optsum.REML, kwargs...)\n\nRefit the model m after installing response y.\n\nIf y is omitted the current response vector is used. kwargs are the same as fit!.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.replicate-Tuple{Function, Integer}","page":"API","title":"MixedModels.replicate","text":"replicate(f::Function, n::Integer; progress=true)\n\nReturn a vector of the values of n calls to f() - used in simulations where the value of f is stochastic.\n\nprogress controls whether the progress bar is shown. Note that the progress bar is automatically disabled for non-interactive (i.e. logging) contexts.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.restoreoptsum!-Tuple{MixedModel, Any}","page":"API","title":"MixedModels.restoreoptsum!","text":"restoreoptsum!(m::MixedModel, io::IO; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\nrestoreoptsum!(m::MixedModel, filename; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\n\nRead, check, and restore the optsum field from a JSON stream or filename.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.restorereplicates-Union{Tuple{T}, Tuple{Any, MixedModel{T}}, Tuple{Any, MixedModel{T}, Type{<:AbstractFloat}}} where T","page":"API","title":"MixedModels.restorereplicates","text":"restorereplicates(f, m::MixedModel{T})\nrestorereplicates(f, m::MixedModel{T}, ftype::Type{<:AbstractFloat})\nrestorereplicates(f, m::MixedModel{T}, ctype::Type{<:MixedModelFitCollection{S}})\n\nRestore replicates from f, using m to create the desired subtype of MixedModelFitCollection.\n\nf can be any entity supported by Arrow.Table. m does not have to be fitted, but it must have been constructed with the same structure as the source of the saved replicates.\n\nThe two-argument method constructs a MixedModelBootstrap with the same eltype as m. If an eltype is specified as the third argument, then a MixedModelBootstrap is returned. If a subtype of MixedModelFitCollection is specified as the third argument, then that is the return type.\n\nSee also savereplicates, restoreoptsum!.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.saveoptsum-Tuple{IO, MixedModel}","page":"API","title":"MixedModels.saveoptsum","text":"saveoptsum(io::IO, m::MixedModel)\nsaveoptsum(filename, m::MixedModel)\n\nSave m.optsum (w/o the lowerbd field) in JSON format to an IO stream or a file\n\nThe reason for omitting the lowerbd field is because it often contains -Inf values that are not allowed in JSON.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.savereplicates-Tuple{Any, MixedModels.MixedModelFitCollection}","page":"API","title":"MixedModels.savereplicates","text":"savereplicates(f, b::MixedModelFitCollection)\n\nSave the replicates associated with a MixedModelFitCollection, e.g. MixedModelBootstrap as an Arrow file.\n\nSee also restorereplicates, saveoptsum\n\nnote: Note\nOnly the replicates are saved, not the entire contents of the MixedModelFitCollection. restorereplicates requires a model compatible with the bootstrap to restore the full object.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.sdest-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.sdest","text":"sdest(m::LinearMixedModel)\n\nReturn the estimate of σ, the standard deviation of the per-observation noise.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.sdest-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.sdest","text":"sdest(m::GeneralizedLinearMixedModel)\n\nReturn the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.setθ!-Union{Tuple{T}, Tuple{LinearMixedModel{T}, AbstractVector}} where T","page":"API","title":"MixedModels.setθ!","text":"setθ!(m::LinearMixedModel, v)\n\nInstall v as the θ parameters in m.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.setθ!-Union{Tuple{T}, Tuple{MixedModels.MixedModelFitCollection{T}, AbstractVector{T}}} where T","page":"API","title":"MixedModels.setθ!","text":"setθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)\nsetθ!(bsamp::MixedModelFitCollection, i::Integer)\n\nInstall the values of the i'th θ value of bsamp.fits in bsamp.λ\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.shortestcovint","page":"API","title":"MixedModels.shortestcovint","text":"shortestcovint(v, level = 0.95)\n\nReturn the shortest interval containing level proportion of the values of v\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.shortestcovint-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}, Tuple{MixedModels.MixedModelFitCollection{T}, Any}} where T","page":"API","title":"MixedModels.shortestcovint","text":"shortestcovint(bsamp::MixedModelFitCollection, level = 0.95)\n\nReturn the shortest interval containing level proportion for each parameter from bsamp.allpars.\n\nwarning: Warning\nCurrently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.simulate","page":"API","title":"MixedModels.simulate","text":"See simulate!\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.simulate!-Tuple{Random.AbstractRNG, AbstractVector, LinearMixedModel, NamedTuple{names, T} where {N, names, T<:Tuple{Vararg{AbstractVector, N}}}}","page":"API","title":"MixedModels.simulate!","text":"simulate!([rng::AbstractRNG,] y::AbstractVector, m::MixedModel{T}[, newdata];\n β = coef(m), σ = m.σ, θ = T[], wts=m.wts)\nsimulate([rng::AbstractRNG,] m::MixedModel{T}[, newdata];\n β = coef(m), σ = m.σ, θ = T[], wts=m.wts)\n\nSimulate a new response vector, optionally overwriting a pre-allocated vector.\n\nNew data can be optionally provided in tabular format.\n\nThis simulation includes sampling new values for the random effects. Thus in contrast to predict, there is no distinction in between \"new\" and \"old\" / previously observed random-effects levels.\n\nUnlike predict, there is no type parameter for GeneralizedLinearMixedModel because the noise term in the model and simulation is always on the response scale.\n\nThe wts argument is currently ignored except for GeneralizedLinearMixedModel models with a Binomial distribution.\n\nnote: Note\nNote that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.simulate!-Union{Tuple{T}, Tuple{Random.AbstractRNG, LinearMixedModel{T}}} where T","page":"API","title":"MixedModels.simulate!","text":"simulate!(rng::AbstractRNG, m::MixedModel{T}; β=fixef(m), σ=m.σ, θ=T[])\nsimulate!(m::MixedModel; β=fixef(m), σ=m.σ, θ=m.θ)\n\nOverwrite the response (i.e. m.trms[end]) with a simulated response vector from model m.\n\nThis simulation includes sampling new values for the random effects.\n\nβ can be specified either as a pivoted, full rank coefficient vector (cf. fixef) or as an unpivoted full dimension coefficient vector (cf. coef), where the entries corresponding to redundant columns will be ignored.\n\nnote: Note\nNote that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.sparseL-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.sparseL","text":"sparseL(m::LinearMixedModel; fname::Symbol=first(fnames(m)), full::Bool=false)\n\nReturn the lower Cholesky factor L as a SparseMatrix{T,Int32}.\n\nfull indicates whether the parts of L associated with the fixed-effects and response are to be included.\n\nfname specifies the first grouping factor to include. Blocks to the left of the block corresponding to fname are dropped. The default is the first, i.e., leftmost block and hence all blocks.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.stderror!-Union{Tuple{T}, Tuple{Tv}, Tuple{AbstractVector{Tv}, LinearMixedModel{T}}} where {Tv, T}","page":"API","title":"MixedModels.stderror!","text":"stderror!(v::AbstractVector, m::LinearMixedModel)\n\nOverwrite v with the standard errors of the fixed-effects coefficients in m\n\nThe length of v should be the total number of coefficients (i.e. length(coef(m))). When the model matrix is rank-deficient the coefficients forced to -0.0 have an undefined (i.e. NaN) standard error.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.updateL!-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.updateL!","text":"updateL!(m::LinearMixedModel)\n\nUpdate the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)\n\nThis is the crucial step in evaluating the objective, given a new parameter value.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.varest-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.varest","text":"varest(m::LinearMixedModel)\n\nReturns the estimate of σ², the variance of the conditional distribution of Y given B.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.varest-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.varest","text":"varest(m::GeneralizedLinearMixedModel)\n\nReturns the estimate of ϕ², the variance of the conditional distribution of Y given B.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ².\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.zerocorr-Tuple{Any}","page":"API","title":"MixedModels.zerocorr","text":"zerocorr(term::RandomEffectsTerm)\n\nRemove correlations between random effects in term.\n\n\n\n\n\n","category":"method"},{"location":"api/#Statistics.std-Tuple{LinearMixedModel}","page":"API","title":"Statistics.std","text":"std(m::MixedModel)\n\nReturn the estimated standard deviations of the random effects as a Vector{Vector{T}}.\n\nFIXME: This uses an old convention of isfinite(sdest(m)). Probably drop in favor of m.σs\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.confint-Tuple{MixedModelProfile}","page":"API","title":"StatsAPI.confint","text":"confint(pr::MixedModelProfile; level::Real=0.95)\n\nCompute profile confidence intervals for coefficients and variance components, with confidence level level (by default 95%).\n\nnote: Note\nThe API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.\n\nnote: Note\nThe \"row names\" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.confint-Union{Tuple{MixedModelBootstrap{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.confint","text":"confint(pr::MixedModelBootstrap; level::Real=0.95, method=:shortest)\n\nCompute bootstrap confidence intervals for coefficients and variance components, with confidence level level (by default 95%).\n\nThe keyword argument method determines whether the :shortest, i.e. highest density, interval is used or the :equaltail, i.e. quantile-based, interval is used. For historical reasons, the default is :shortest, but :equaltail gives the interval that is most comparable to the profile and Wald confidence intervals.\n\nnote: Note\nThe API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.\n\nnote: Note\nThe \"row names\" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.\n\nSee also shortestcovint.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.confint-Union{Tuple{MixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.confint","text":"confint(pr::MixedModelProfile; level::Real=0.95)\n\nCompute profile confidence intervals for (fixed effects) coefficients, with confidence level level (by default 95%).\n\nnote: Note\nThe API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.deviance-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}, Tuple{GeneralizedLinearMixedModel{T}, Any}} where T","page":"API","title":"StatsAPI.deviance","text":"deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}\n\nReturn the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.\n\nIf the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, u, plus the determinant of ΛZWZΛ + I, plus the sum of the squared deviance residuals.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.dof_residual-Tuple{MixedModel}","page":"API","title":"StatsAPI.dof_residual","text":"dof_residual(m::MixedModel)\n\nReturn the residual degrees of freedom of the model.\n\nnote: Note\nThe residual degrees of freedom for mixed-effects models is not clearly defined due to partial pooling. The classical nobs(m) - dof(m) fails to capture the extra freedom granted by the random effects, but nobs(m) - nranef(m) would overestimate the freedom granted by the random effects. nobs(m) - sum(leverage(m)) provides a nice balance based on the relative influence of each observation, but is computationally expensive for large models. This problem is also fundamentally related to long-standing debates about the appropriate treatment of the denominator degrees of freedom for F-tests. In the future, MixedModels.jl may provide additional methods allowing the user to choose the computation to use.\n\nwarning: Warning\nCurrently, the residual degrees of freedom is computed as nobs(m) - dof(m), but this may change in the future without being considered a breaking change because there is no canonical definition of the residual degrees of freedom in a mixed-effects model.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.fit!-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.fit!","text":"fit!(m::GeneralizedLinearMixedModel; fast=false, nAGQ=1,\n verbose=false, progress=true,\n thin::Int=1,\n init_from_lmm=Set())\n\nOptimize the objective function for m.\n\nWhen fast is true a potentially much faster but slightly less accurate algorithm, in which pirls! optimizes both the random effects and the fixed-effects parameters, is used.\n\nIf progress is true, the default, a ProgressMeter.ProgressUnknown counter is displayed. during the iterations to minimize the deviance. There is a delay before this display is initialized and it may not be shown at all for models that are optimized quickly.\n\nIf verbose is true, then both the intermediate results of both the nonlinear optimization and PIRLS are also displayed on standard output.\n\nAt every thinth iteration is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.\n\nBy default, the starting values for model fitting are taken from a (non mixed, i.e. marginal ) GLM fit. Experience with larger datasets (many thousands of observations and/or hundreds of levels of the grouping variables) has suggested that fitting a (Gaussian) linear mixed model on the untransformed data may provide better starting values and thus overall faster fits even though an entire LMM must be fit before the GLMM can be fit. init_from_lmm can be used to specify which starting values from an LMM to use. Valid options are any collection (array, set, etc.) containing one or more of :β and :θ, the default is the empty set.\n\nnote: Note\nInitializing from an LMM requires fitting the entire LMM first, so when progress=true, there will be two progress bars: first for the LMM, then for the GLMM.\n\nwarning: Warning\nThe init_from_lmm functionality is experimental and may change or be removed entirely without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.fit!-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.fit!","text":"fit!(m::LinearMixedModel; progress::Bool=true, REML::Bool=m.optsum.REML,\n σ::Union{Real, Nothing}=m.optsum.sigma,\n thin::Int=typemax(Int))\n\nOptimize the objective of a LinearMixedModel. When progress is true a ProgressMeter.ProgressUnknown display is shown during the optimization of the objective, if the optimization takes more than one second or so.\n\nAt every thinth iteration is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.leverage-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.leverage","text":"leverage(::LinearMixedModel)\n\nReturn the diagonal of the hat matrix of the model.\n\nFor a linear model, the sum of the leverage values is the degrees of freedom for the model in the sense that this sum is the dimension of the span of columns of the model matrix. With a bit of hand waving a similar argument could be made for linear mixed-effects models. The hat matrix is of the form ZΛ XL L¹ZΛ X.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.modelmatrix-Tuple{MixedModel}","page":"API","title":"StatsAPI.modelmatrix","text":"modelmatrix(m::MixedModel)\n\nReturns the model matrix X for the fixed-effects parameters, as returned by coef.\n\nThis is always the full model matrix in the original column order and from a field in the model struct. It should be copied if it is to be modified.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.predict-Tuple{LinearMixedModel, NamedTuple{names, T} where {N, names, T<:Tuple{Vararg{AbstractVector, N}}}}","page":"API","title":"StatsAPI.predict","text":"StatsAPI.predict(m::LinearMixedModel, newdata;\n new_re_levels=:missing)\nStatsAPI.predict(m::GeneralizedLinearMixedModel, newdata;\n new_re_levels=:missing, type=:response)\n\nPredict response for new data.\n\nnote: Note\nCurrently, no in-place methods are provided because these methods internally construct a new model and therefore allocate not just a response vector but also many other matrices.\n\nwarning: Warning\nnewdata should contain a column for the response (dependent variable) initialized to some numerical value (not missing), because this is used to construct the new model used in computing the predictions. missing is not valid because missing data are dropped before constructing the model matrices.\n\nwarning: Warning\nThese methods construct an entire MixedModel behind the scenes and as such may use a large amount of memory when newdata is large.\n\nwarning: Warning\nRank-deficiency can lead to surprising but consistent behavior. For example, if there are two perfectly collinear predictors A and B (e.g. constant multiples of each other), then it is possible that A will be pivoted out in the fitted model and thus the associated coefficient is set to zero. If predictions are then generated on new data where B has been set to zero but A has not, then there will no contribution from neither A nor B in the resulting predictions.\n\nThe keyword argument new_re_levels specifies how previously unobserved values of the grouping variable are handled. Possible values are:\n\n:population: return population values for the relevant grouping variable. In other words, treat the associated random effect as 0. If all grouping variables have new levels, then this is equivalent to just the fixed effects.\n:missing: return missing.\n:error: error on this condition. The error type is an implementation detail: you should not rely on a particular type of error being thrown.\n\nIf you want simulated values for unobserved levels of the grouping variable, consider the simulate! and simulate methods.\n\nPredictions based purely on the fixed effects can be obtained by specifying previously unobserved levels of the random effects and setting new_re_levels=:population. Similarly, the contribution of any grouping variable can be excluded by specifying previously unobserved levels, while including previously observed levels of the other grouping variables. In the future, it may be possible to specify a subset of the grouping variables or overall random-effects structure to use, but not at this time.\n\nnote: Note\nnew_re_levels impacts only the behavior for previously unobserved random effects levels, i.e. new RE levels. For previously observed random effects levels, predictions take both the fixed and random effects into account.\n\nFor GeneralizedLinearMixedModel, the type parameter specifies whether the predictions should be returned on the scale of linear predictor (:linpred) or on the response scale (:response). If you don't know the difference between these terms, then you probably want type=:response.\n\nRegression weights are not yet supported in prediction. Similarly, offsets are also not supported for GeneralizedLinearMixedModel.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.response-Tuple{MixedModel}","page":"API","title":"StatsAPI.response","text":"response(m::MixedModel)\n\nReturn the response vector for the model.\n\nFor a linear mixed model this is a view of the last column of the XyMat field. For a generalized linear mixed model this is the m.resp.y field. In either case it should be copied if it is to be modified.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.vcov-Tuple{MixedModel}","page":"API","title":"StatsAPI.vcov","text":"vcov(m::MixedModel; corr=false)\n\nReturns the variance-covariance matrix of the fixed effects. If corr is true, the correlation of the fixed effects is returned instead.\n\n\n\n\n\n","category":"method"},{"location":"api/#Tables.columntable-Tuple{OptSummary}","page":"API","title":"Tables.columntable","text":"columntable(s::OptSummary, [stack::Bool=false])\n\nReturn s.fitlog as a Tables.columntable.\n\nWhen stack is false (the default), there will be 3 columns in the result:\n\niter: the sample number\nobjective: the value of the objective at that sample\nθ: the parameter vector at that sample\n\n(The term sample here refers to the fact that when the thin argument to the fit or refit! call is greater than 1 only a subset of the iterations have results recorded.)\n\nWhen stack is true, there will be 4 columns: iter, objective, par, and value where value is the stacked contents of the θ vectors (the equivalent of vcat(θ...)) and par is a vector of parameter numbers.\n\n\n\n\n\n","category":"method"},{"location":"api/#Methods-from-StatsAPI.jl,-StatsBase.jl,-StatsModels.jl-and-GLM.jl","page":"API","title":"Methods from StatsAPI.jl, StatsBase.jl, StatsModels.jl and GLM.jl","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"aic\naicc\nbic\ncoef\ncoefnames\ncoeftable\ndeviance\ndispersion\ndispersion_parameter\ndof\ndof_residual\nfit\nfit!\nfitted\nformula\nisfitted\nislinear\nleverage\nloglikelihood\nmeanresponse\nmodelmatrix\nmodel_response\nnobs\npredict\nresiduals\nresponse\nresponsename\nStatsModels.lrtest # not exported\nstd\nstderror\nvcov\nweights","category":"page"},{"location":"api/#MixedModels.jl-\"alternatives\"-and-extensions-to-StatsAPI-and-GLM-functions","page":"API","title":"MixedModels.jl \"alternatives\" and extensions to StatsAPI and GLM functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"The following are MixedModels.jl-specific functions and not simply methods for functions defined in StatsAPI and GLM.jl.","category":"page"},{"location":"api/","page":"API","title":"API","text":"coefpvalues\ncondVar\ncondVarTables\nfitted!\nfixef\nfixefnames\nlikelihoodratiotest # not exported\npwrss\nranef\nraneftables\nrefit!\nshortestcovint\nsdest\nsimulate\nsimulate!\nstderrror!\nvarest","category":"page"},{"location":"api/#Non-Exported-Functions","page":"API","title":"Non-Exported Functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Note that unless discussed elsewhere in the online documentation, non-exported functions should be considered implementation details.","category":"page"},{"location":"api/","page":"API","title":"API","text":"Modules = [MixedModels]\nPublic = false\nOrder = [:function]\nFilter = f -> !startswith(string(f), \"_\")","category":"page"},{"location":"api/#Base.copy-Union{Tuple{ReMat{T, S}}, Tuple{S}, Tuple{T}} where {T, S}","page":"API","title":"Base.copy","text":"Base.copy(ReMat{T,S})\n\nReturn a shallow copy of ReMat.\n\nA shallow copy shares as much internal storage as possible with the original ReMat. Only the vector λ and the scratch matrix are copied.\n\n\n\n\n\n","category":"method"},{"location":"api/#Base.size-Tuple{MixedModel}","page":"API","title":"Base.size","text":"size(m::MixedModel)\n\nReturns the size of a mixed model as a tuple of length four: the number of observations, the number of (non-singular) fixed-effects parameters, the number of conditional modes (random effects), the number of grouping variables\n\n\n\n\n\n","category":"method"},{"location":"api/#GLM.wrkresp!-Union{Tuple{T}, Tuple{AbstractVector{T}, GLM.GlmResp{Vector{T}}}} where T<:AbstractFloat","page":"API","title":"GLM.wrkresp!","text":"GLM.wrkresp!(v::AbstractVector{T}, resp::GLM.GlmResp{AbstractVector{T}})\n\nA copy of a method from GLM that generalizes the types in the signature\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.LD-Union{Tuple{LinearAlgebra.Diagonal{T, V} where V<:AbstractVector{T}}, Tuple{T}} where T<:Number","page":"API","title":"MixedModels.LD","text":"LD(A::Diagonal)\nLD(A::HBlikDiag)\nLD(A::DenseMatrix)\n\nReturn log(det(tril(A))) evaluated in place.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.adjA-Tuple{AbstractVector, AbstractMatrix}","page":"API","title":"MixedModels.adjA","text":"adjA(refs::AbstractVector, z::AbstractMatrix{T})\n\nReturns the adjoint of an ReMat as a SparseMatrixCSC{T,Int32}\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.allpars-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.allpars","text":"allpars(bsamp::MixedModelFitCollection)\n\nReturn a tidy (column)table with the parameter estimates spread into columns of iter, type, group, name and value.\n\nwarning: Warning\nCurrently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.amalgamate-Union{Tuple{Vector{<:AbstractReMat{T}}}, Tuple{T}} where T","page":"API","title":"MixedModels.amalgamate","text":"amalgamate(reterms::Vector{AbstractReMat})\n\nCombine multiple ReMat with the same grouping variable into a single object.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.average-Union{Tuple{T}, Tuple{T, T}} where T<:AbstractFloat","page":"API","title":"MixedModels.average","text":"average(a::T, b::T) where {T<:AbstractFloat}\n\nReturn the average of a and b\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.block-Tuple{Integer, Integer}","page":"API","title":"MixedModels.block","text":"block(i, j)\n\nReturn the linear index of the [i,j] position (\"block\") in the row-major packed lower triangle.\n\nUse the row-major ordering in this case because the result depends only on i and j, not on the overall size of the array.\n\nWhen i == j the value is the same as kp1choose2(i).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.cholUnblocked!","page":"API","title":"MixedModels.cholUnblocked!","text":"cholUnblocked!(A, Val{:L})\n\nOverwrite the lower triangle of A with its lower Cholesky factor.\n\nThe name is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl] because these are part of the inner calculations in a blocked Cholesky factorization.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.copyscaleinflate!","page":"API","title":"MixedModels.copyscaleinflate!","text":"copyscaleinflate!(L::AbstractMatrix, A::AbstractMatrix, Λ::ReMat)\n\nOverwrite L with Λ'AΛ + I\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.corrmat-Union{Tuple{ReMat{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.corrmat","text":"corrmat(A::ReMat)\n\nReturn the estimated correlation matrix for A. The diagonal elements are 1 and the off-diagonal elements are the correlations between those random effect terms\n\nExample\n\nNote that trailing digits may vary slightly depending on the local platform.\n\njulia> using MixedModels\n\njulia> mod = fit(MixedModel,\n @formula(rt_trunc ~ 1 + spkr + prec + load + (1 + spkr + prec | subj)),\n MixedModels.dataset(:kb07));\n\njulia> VarCorr(mod)\nVariance components:\n Column Variance Std.Dev. Corr.\nsubj (Intercept) 136591.782 369.583\n spkr: old 22922.871 151.403 +0.21\n prec: maintain 32348.269 179.856 -0.98 -0.03\nResidual 642324.531 801.452\n\njulia> MixedModels.corrmat(mod.reterms[1])\n3×3 LinearAlgebra.Symmetric{Float64,Array{Float64,2}}:\n 1.0 0.214816 -0.982948\n 0.214816 1.0 -0.0315607\n -0.982948 -0.0315607 1.0\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.cpad-Tuple{String, Integer}","page":"API","title":"MixedModels.cpad","text":"cpad(s::AbstractString, n::Integer)\n\nReturn a string of length n containing s in the center (more-or-less).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.densify","page":"API","title":"MixedModels.densify","text":"densify(S::SparseMatrix, threshold=0.1)\n\nConvert sparse S to Diagonal if S is diagonal or to Array(S) if the proportion of nonzeros exceeds threshold.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.deviance!","page":"API","title":"MixedModels.deviance!","text":"deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)\n\nUpdate m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.feL-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.feL","text":"feL(m::LinearMixedModel)\n\nReturn the lower Cholesky factor for the fixed-effects parameters, as an LowerTriangular p × p matrix.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fixef!-Union{Tuple{T}, Tuple{Tv}, Tuple{AbstractVector{Tv}, LinearMixedModel{T}}} where {Tv, T}","page":"API","title":"MixedModels.fixef!","text":"fixef!(v::Vector{T}, m::MixedModel{T})\n\nOverwrite v with the pivoted fixed-effects coefficients of model m\n\nFor full-rank models the length of v must be the rank of X. For rank-deficient models the length of v can be the rank of X or the number of columns of X. In the latter case the calculated coefficients are padded with -0.0 out to the number of columns.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fname-Tuple{ReMat}","page":"API","title":"MixedModels.fname","text":"fname(A::ReMat)\n\nReturn the name of the grouping factor as a Symbol\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.getθ!-Union{Tuple{T}, Tuple{AbstractVector{T}, ReMat{T}}} where T","page":"API","title":"MixedModels.getθ!","text":"getθ!(v::AbstractVector{T}, A::ReMat{T}) where {T}\n\nOverwrite v with the elements of the blocks in the lower triangle of A.Λ (column-major ordering)\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.getθ-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.getθ","text":"getθ(m::LinearMixedModel)\n\nReturn the current covariance parameter vector.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.indmat","page":"API","title":"MixedModels.indmat","text":"indmat(A::ReMat)\n\nReturn a Bool indicator matrix of the potential non-zeros in A.λ\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.isconstant-Tuple{Any}","page":"API","title":"MixedModels.isconstant","text":"isconstant(x::Array)\nisconstant(x::Tuple)\n\nAre all elements of the iterator the same? That is, is it constant?\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.isfullrank-Tuple{MixedModels.FeTerm}","page":"API","title":"MixedModels.isfullrank","text":"isfullrank(A::FeTerm)\n\nDoes A have full column rank?\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.isnested-Tuple{ReMat, ReMat}","page":"API","title":"MixedModels.isnested","text":"isnested(A::ReMat, B::ReMat)\n\nIs the grouping factor for A nested in the grouping factor for B?\n\nThat is, does each value of A occur with just one value of B?\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.kchoose2-Tuple{Any}","page":"API","title":"MixedModels.kchoose2","text":"kchoose2(k)\n\nThe binomial coefficient k choose 2 which is the number of elements in the packed form of the strict lower triangle of a matrix.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.kp1choose2-Tuple{Any}","page":"API","title":"MixedModels.kp1choose2","text":"kp1choose2(k)\n\nThe binomial coefficient k+1 choose 2 which is the number of elements in the packed form of the lower triangle of a matrix.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.likelihoodratiotest-Tuple{Vararg{MixedModel}}","page":"API","title":"MixedModels.likelihoodratiotest","text":"likelihoodratiotest(m::MixedModel...)\nlikelihoodratiotest(m0::LinearModel, m::MixedModel...)\nlikelihoodratiotest(m0::GeneralizedLinearModel, m::MixedModel...)\nlikelihoodratiotest(m0::TableRegressionModel{LinearModel}, m::MixedModel...)\nlikelihoodratiotest(m0::TableRegressionModel{GeneralizedLinearModel}, m::MixedModel...)\n\nLikeihood ratio test applied to a set of nested models.\n\nnote: Note\nThe nesting of the models is not checked. It is incumbent on the user to check this. This differs from StatsModels.lrtest as nesting in mixed models, especially in the random effects specification, may be non obvious.\n\nnote: Note\nFor comparisons between mixed and non-mixed models, the deviance for the non-mixed model is taken to be -2 log likelihood, i.e. omitting the additive constant for the fully saturated model. This is in line with the computation of the deviance for mixed models.\n\nThis functionality may be deprecated in the future in favor of StatsModels.lrtest.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.nranef-Tuple{ReMat}","page":"API","title":"MixedModels.nranef","text":"nranef(A::ReMat)\n\nReturn the number of random effects represented by A. Zero unless A is an ReMat.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.nθ-Tuple{ReMat}","page":"API","title":"MixedModels.nθ","text":"nθ(A::ReMat)\n\nReturn the number of free parameters in the relative covariance matrix λ\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.optsumj-Tuple{OptSummary, Integer}","page":"API","title":"MixedModels.optsumj","text":"optsumj(os::OptSummary, j::Integer)\n\nReturn an OptSummary with the j'th component of the parameter omitted.\n\nos.final with its j'th component omitted is used as the initial parameter.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.parsej-Tuple{Symbol}","page":"API","title":"MixedModels.parsej","text":"parsej(sym::Symbol)\n\nReturn the index from symbol names like :θ1, :θ01, etc.\n\nnote: Note\nThis method is internal.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.pivot-Tuple{MixedModel}","page":"API","title":"MixedModels.pivot","text":"pivot(m::MixedModel)\npivot(A::FeTerm)\n\nReturn the pivot associated with the FeTerm.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.prfit!","page":"API","title":"MixedModels.prfit!","text":"prfit!(m::LinearMixedModel)\n\nFit a mixed model using the PRIMA implementation of the BOBYQA optimizer.\n\nwarning: Experimental feature\nThis function is an experimental feature that will go away in the future. Do not rely on it, unless you are willing to pin the precise MixedModels.jl version. The purpose of the function is to provide the MixedModels developers a chance to explore the performance of the PRIMA implementation without the large and potentially breaking changes it would take to fully replace the current NLopt backend with a PRIMA backend or a backend supporting a range of optimizers.\n\nnote: Package extension\nIn order to reduce the dependency burden, all methods of this function are implemented in a package extension and are only defined when PRIMA.jl is loaded by the user.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.profileσs!-Union{Tuple{T}, Tuple{NamedTuple, MixedModels.TableColumns{T}}} where T","page":"API","title":"MixedModels.profileσs!","text":" profileσs!(val::NamedTuple, tc::TableColumns{T}; nzlb=1.0e-8) where {T}\n\nProfile the variance components.\n\nnote: Note\nThis method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.ranef!-Union{Tuple{T}, Tuple{Vector, LinearMixedModel{T}, AbstractArray{T}, Bool}} where T","page":"API","title":"MixedModels.ranef!","text":"ranef!(v::Vector{Matrix{T}}, m::MixedModel{T}, β, uscale::Bool) where {T}\n\nOverwrite v with the conditional modes of the random effects for m.\n\nIf uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale\n\nβ is the truncated, pivoted coefficient vector.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.rankUpdate!","page":"API","title":"MixedModels.rankUpdate!","text":"rankUpdate!(C, A)\nrankUpdate!(C, A, α)\nrankUpdate!(C, A, α, β)\n\nA rank-k update, C := αA'A + βC, of a Hermitian (Symmetric) matrix.\n\nα and β both default to 1.0. When α is -1.0 this is a downdate operation. The name rankUpdate! is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl]\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.rePCA-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.rePCA","text":"rePCA(m::LinearMixedModel; corr::Bool=true)\n\nReturn a named tuple of the normalized cumulative variance of a principal components analysis of the random effects covariance matrices or correlation matrices when corr is true.\n\nThe normalized cumulative variance is the proportion of the variance for the first principal component, the first two principal components, etc. The last element is always 1.0 representing the complete proportion of the variance.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.reevaluateAend!-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.reevaluateAend!","text":"reevaluateAend!(m::LinearMixedModel)\n\nReevaluate the last column of m.A from m.Xymat. This function should be called after updating the response.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.refitσ!-Union{Tuple{T}, Tuple{LinearMixedModel{T}, Any, MixedModels.TableColumns{T}, T, Bool}} where T","page":"API","title":"MixedModels.refitσ!","text":"refitσ!(m::LinearMixedModel{T}, σ::T, tc::TableColumns{T}, obj::T, neg::Bool)\n\nRefit the model m with the given value of σ and return a NamedTuple of information about the fit.\n\nobj and neg allow for conversion of the objective to the ζ scale and tc is used to return a NamedTuple\n\nnote: Note\nThis method is internal and may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.schematize","page":"API","title":"MixedModels.schematize","text":"schematize(f, tbl, contrasts::Dict{Symbol}, Mod=LinearMixedModel)\n\nFind and apply the schema for f in a way that automatically uses Grouping() contrasts when appropriate.\n\nwarn: Warn\nThis is an internal method.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.sdcorr-Union{Tuple{AbstractMatrix{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.sdcorr","text":"sdcorr(A::AbstractMatrix{T}) where {T}\n\nTransform a square matrix A with positive diagonals into an NTuple{size(A,1), T} of standard deviations and a tuple of correlations.\n\nA is assumed to be symmetric and only the lower triangle is used. The order of the correlations is row-major ordering of the lower triangle (or, equivalently, column-major in the upper triangle).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.setβθ!-Tuple{GeneralizedLinearMixedModel, Any}","page":"API","title":"MixedModels.setβθ!","text":"setβθ!(m::GeneralizedLinearMixedModel, v)\n\nSet the parameter vector, :βθ, of m to v.\n\nβθ is the concatenation of the fixed-effects, β, and the covariance parameter, θ.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.ssqdenom-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.ssqdenom","text":"ssqdenom(m::LinearMixedModel)\n\nReturn the denominator for penalized sums-of-squares.\n\nFor MLE, this value is the number of observations. For REML, this value is the number of observations minus the rank of the fixed-effects matrix. The difference is analogous to the use of n or n-1 in the denominator when calculating the variance.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.statsrank-Union{Tuple{AbstractMatrix{T}}, Tuple{T}} where T<:AbstractFloat","page":"API","title":"MixedModels.statsrank","text":"statsrank(x::Matrix{T}, ranktol::Real=1e-8) where {T<:AbstractFloat}\n\nReturn the numerical column rank and a pivot vector.\n\nThe rank is determined from the absolute values of the diagonal of R from a pivoted QR decomposition, relative to the first (and, hence, largest) element of this vector.\n\nIn the full-rank case the pivot vector is collect(axes(x, 2)).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.tidyβ-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.tidyβ","text":"tidyβ(bsamp::MixedModelFitCollection)\n\nReturn a tidy (row)table with the parameter estimates spread into columns of iter, coefname and β\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.tidyσs-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.tidyσs","text":"tidyσs(bsamp::MixedModelFitCollection)\n\nReturn a tidy (row)table with the estimates of the variance components (on the standard deviation scale) spread into columns of iter, group, column and σ.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.unfit!-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.unfit!","text":"unfit!(model::MixedModel)\n\nMark a model as unfitted.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.unscaledre!","page":"API","title":"MixedModels.unscaledre!","text":"unscaledre!(y::AbstractVector{T}, M::ReMat{T}) where {T}\nunscaledre!(rng::AbstractRNG, y::AbstractVector{T}, M::ReMat{T}) where {T}\n\nAdd unscaled random effects simulated from M to y.\n\nThese are unscaled random effects (i.e. they incorporate λ but not σ) because the scaling is done after the per-observation noise is added as a standard normal.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.updateA!-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.updateA!","text":"updateA!(m::LinearMixedModel)\n\nUpdate the cross-product array, m.A, from m.reterms and m.Xymat\n\nThis is usually done after a reweight! operation.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.updateη!-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.updateη!","text":"updateη!(m::GeneralizedLinearMixedModel)\n\nUpdate the linear predictor, m.η, from the offset and the B-scale random effects.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.σvals!-Tuple{AbstractVector, ReMat, Number}","page":"API","title":"MixedModels.σvals!","text":"σvals!(v::AbstractVector, A::ReMat, sc::Number)\n\nOverwrite v with the standard deviations of the random effects associated with A\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.σρ!-Union{Tuple{T}, Tuple{AbstractVector{<:Union{Missing, T}}, LinearAlgebra.LowerTriangular, Any}} where T","page":"API","title":"MixedModels.σρ!","text":"σρ!(v, t, σ)\n\npush! σ times the row lengths (σs) and the inner products of normalized rows (ρs) of t onto v\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsModels.isnested-Tuple{MixedModel, MixedModel}","page":"API","title":"StatsModels.isnested","text":"isnested(m1::MixedModel, m2::MixedModel; atol::Real=0.0)\n\nIndicate whether model m1 is nested in model m2, i.e. whether m1 can be obtained by constraining some parameters in m2. Both models must have been fitted on the same data. This check is conservative for MixedModels and may reject nested models with different parameterizations as being non nested.\n\n\n\n\n\n","category":"method"},{"location":"mime/#Alternative-display-and-output-formats","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"","category":"section"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"In the documentation, we have presented the output from MixedModels.jl in the same format you will see when working in the REPL. You may have noticed, however, that output from other packages received pretty printing. For example, DataFrames are converted into nice HTML tables. In MixedModels, we recently (v3.2.0) introduced limited support for such pretty printing. (For more details on how the print and display system in Julia works, check out this NextJournal post.)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"In particular, we have defined Markdown, HTML and LaTeX output, i.e. show methods, for our types. Note that the Markdown output can also be easily and more flexibly translated into HTML, LaTeX (e.g. with booktabs) or even a MS Word Document using tools such as pandoc. Packages like IJulia and Documenter can often detect the presence of these display options and use them automatically.","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"using MixedModels\nform = @formula(rt_trunc ~ 1 + spkr * prec * load +\n (1 + load | item) +\n (1 + spkr + prec + load | subj))\ncontr = Dict(:spkr => EffectsCoding(),\n :prec => EffectsCoding(),\n :load => EffectsCoding(),\n :item => Grouping(),\n :subj => Grouping())\nkbm = fit(MixedModel, form, MixedModels.dataset(:kb07); contrasts=contr)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"Note that the display here is more succinct than the standard REPL display:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"using DisplayAs\nkbm |> DisplayAs.Text","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"This brevity is intentional: we wanted these types to work well with traditional academic publishing constraints on tables. The summary for a model fit presented in the REPL does not mesh well with being treated as a single table (with columns shared between the random and fixed effects). In our experience, this leads to difficulties in typesetting the resulting tables. We nonetheless encourage users to report fit statistics such as the log likelihood or AIC as part of the caption of their table. If the correlation parameters in the random effects are of interest, then VarCorr can also be pretty printed:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"VarCorr(kbm)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"Similarly for BlockDescription, OptSummary and MixedModels.likelihoodratiotest:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"BlockDescription(kbm)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"kbm.optsum","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"m0 = fit(MixedModel, @formula(reaction ~ 1 + (1|subj)), MixedModels.dataset(:sleepstudy))\nm1 = fit(MixedModel, @formula(reaction ~ 1 + days + (1+days|subj)), MixedModels.dataset(:sleepstudy))\nMixedModels.likelihoodratiotest(m0,m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"To explicitly invoke this behavior, we must specify the right show method. (The raw and not rendered output is intentionally shown here.)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/markdown\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/markdown\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/html\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/html\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"Note for that LaTeX, the column labels for the random effects are slightly changed: σ is placed into math mode and escaped and the grouping variable is turned into a subscript. Similarly for the likelihood ratio test, the χ² is escaped into math mode. This transformation improves pdfLaTeX and journal compatibility, but also means that XeLaTeX and LuaTeX may use a different font at this point.","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/latex\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/latex\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"This escaping behavior can be disabled by specifying \"text/xelatex\" as the MIME type. (Note that other symbols may still be escaped, as the internal conversion uses the Markdown module from the standard library, which performs some escaping on its own.)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/xelatex\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/xelatex\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"This output can also be written directly to file:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"open(\"model.md\", \"w\") do io\n show(io, MIME(\"text/markdown\"), kbm)\nend","category":"page"},{"location":"optimization/#Details-of-the-parameter-estimation","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"","category":"section"},{"location":"optimization/#The-probability-model","page":"Details of the parameter estimation","title":"The probability model","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Maximum likelihood estimates are based on the probability model for the observed responses. In the probability model the distribution of the responses is expressed as a function of one or more parameters.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For a continuous distribution the probability density is a function of the responses, given the parameters. The likelihood function is the same expression as the probability density but regarding the observed values as fixed and the parameters as varying.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In general a mixed-effects model incorporates two random variables: mathcalB, the q-dimensional vector of random effects, and mathcalY, the n-dimensional response vector. The value, bf y, of mathcalY is observed; the value, bf b, of mathcalB is not.","category":"page"},{"location":"optimization/#Linear-Mixed-Effects-Models","page":"Details of the parameter estimation","title":"Linear Mixed-Effects Models","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In a linear mixed model the unconditional distribution of mathcalB and the conditional distribution, (mathcalY mathcalB=bfb), are both multivariate Gaussian distributions,","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"beginaligned\n (mathcalY mathcalB=bfb) simmathcalN(bf Xbeta + Z bsigma^2bfI)\n mathcalBsimmathcalN(bf0Sigma_theta) \nendaligned","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The conditional mean of mathcal Y, given mathcal B=bf b, is the linear predictor, bf Xbfbeta+bf Zbf b, which depends on the p-dimensional fixed-effects parameter, bf beta, and on bf b. The model matrices, bf X and bf Z, of dimension ntimes p and ntimes q, respectively, are determined from the formula for the model and the values of covariates. Although the matrix bf Z can be large (i.e. both n and q can be large), it is sparse (i.e. most of the elements in the matrix are zero).","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The relative covariance factor, Lambda_theta, is a qtimes q lower-triangular matrix, depending on the variance-component parameter, bftheta, and generating the symmetric qtimes q variance-covariance matrix, Sigma_theta, as","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Sigma_theta=sigma^2Lambda_thetaLambda_theta","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The spherical random effects, mathcalUsimmathcalN(bf0sigma^2bfI_q), determine mathcal B according to","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mathcalB=Lambda_thetamathcalU","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The penalized residual sum of squares (PRSS),","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"r^2(thetabetabfu)=bfy - bfXbeta -bfZLambda_thetabfu^2+bfu^2","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is the sum of the residual sum of squares, measuring fidelity of the model to the data, and a penalty on the size of bf u, measuring the complexity of the model. Minimizing r^2 with respect to bf u,","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"r^2_betatheta =min_bfuleft(bfy -bfXbeta -bfZLambda_thetabfu^2+bfu^2right)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is a direct (i.e. non-iterative) computation. The particular method used to solve this generates a blocked Choleksy factor, bfL_theta, which is an lower triangular qtimes q matrix satisfying","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"bfL_thetabfL_theta=Lambda_thetabfZbfZLambda_theta+bfI_q ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"where bf I_q is the qtimes q identity matrix.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Negative twice the log-likelihood of the parameters, given the data, bf y, is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"d(bfthetabfbetasigmabf y)\n=nlog(2pisigma^2)+log(bf L_theta^2)+fracr^2_betathetasigma^2","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"where bf L_theta denotes the determinant of bf L_theta. Because bf L_theta is triangular, its determinant is the product of its diagonal elements.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Because the conditional mean, bfmu_mathcal Ymathcal B=bf b=bf Xbfbeta+bf ZLambda_thetabf u, is a linear function of both bfbeta and bf u, minimization of the PRSS with respect to both bfbeta and bf u to produce","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"r^2_theta =min_bfbetabf uleft(bf y -bf Xbfbeta -bf ZLambda_thetabf u^2+bf u^2right)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is also a direct calculation. The values of bf u and bfbeta that provide this minimum are called, respectively, the conditional mode, tildebf u_theta, of the spherical random effects and the conditional estimate, widehatbfbeta_theta, of the fixed effects. At the conditional estimate of the fixed effects the objective is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"d(bfthetawidehatbeta_thetasigmabf y)\n=nlog(2pisigma^2)+log(bf L_theta^2)+fracr^2_thetasigma^2","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Minimizing this expression with respect to sigma^2 produces the conditional estimate","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"widehatsigma^2_theta=fracr^2_thetan","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"which provides the profiled log-likelihood on the deviance scale as","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"tilded(thetabf y)=d(thetawidehatbeta_thetawidehatsigma_thetabf y)\n=log(bf L_theta^2)+nleft1+logleft(frac2pi r^2_thetanright)right","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"a function of bftheta alone.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The MLE of bftheta, written widehatbftheta, is the value that minimizes this profiled objective. We determine this value by numerical optimization. In the process of evaluating tilded(widehatthetabf y) we determine widehatbeta=widehatbeta_widehattheta, tildebf u_widehattheta and r^2_widehattheta, from which we can evaluate widehatsigma=sqrtr^2_widehatthetan.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The elements of the conditional mode of mathcal B, evaluated at the parameter estimates,","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"tildebf b_widehattheta=Lambda_widehatthetatildebf u_widehattheta","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"are sometimes called the best linear unbiased predictors or BLUPs of the random effects. Although BLUPs an appealing acronym, I don’t find the term particularly instructive (what is a “linear unbiased predictor” and in what sense are these the “best”?) and prefer the term “conditional modes”, because these are the values of bf b that maximize the density of the conditional distribution mathcalB mathcalY = bf y. For a linear mixed model, where all the conditional and unconditional distributions are Gaussian, these values are also the conditional means.","category":"page"},{"location":"optimization/#Internal-structure-of-\\Lambda_\\theta-and-\\bf-Z","page":"Details of the parameter estimation","title":"Internal structure of Lambda_theta and bf Z","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In the types of LinearMixedModel available through the MixedModels package, groups of random effects and the corresponding columns of the model matrix, bf Z, are associated with random-effects terms in the model formula.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For the simple example","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"using DisplayAs","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"using BenchmarkTools, DataFrames, MixedModels","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"dyestuff = MixedModels.dataset(:dyestuff)\nfm1 = fit(MixedModel, @formula(yield ~ 1 + (1|batch)), dyestuff)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"the only random effects term in the formula is (1|batch), a simple, scalar random-effects term.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t1 = only(fm1.reterms);\nInt.(t1) # convert to integers for more compact display","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The matrix t1 is a sparse matrix, meaning that most of the elements are zero, and its transpose is stored in a sparse form.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"sparse(t1)'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"provides a compact representation of the positions of the non-zeros in this matrix.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"This RandomEffectsTerm contributes a block of columns to the model matrix bf Z and a diagonal block to Lambda_theta. In this case the diagonal block of Lambda_theta (which is also the only block) is a multiple of the 6times6 identity matrix where the multiple is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t1.λ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Because there is only one random-effects term in the model, the matrix bf Z is the indicators matrix shown as the result of Int.(t1), but stored in a special sparse format. Furthermore, there is only one block in Lambda_theta.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For a vector-valued random-effects term, as in","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"sleepstudy = MixedModels.dataset(:sleepstudy)\nfm2 = fit(MixedModel, @formula(reaction ~ 1+days+(1+days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"the model matrix bf Z is of the form","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t21 = only(fm2.reterms);\nsparse(t21)'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"and Lambda_theta is a 36times36 block diagonal matrix with 18 diagonal blocks, all of the form","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t21.λ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The theta vector is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"MixedModels.getθ(t21)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Random-effects terms in the model formula that have the same grouping factor are amalgamated into a single ReMat object.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fm3 = fit(MixedModel, @formula(reaction ~ 1+days+(1|subj) + (0+days|subj)), sleepstudy)\nt31 = only(fm3.reterms);\nsparse(t31)'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For this model the matrix bf Z is the same as that of model fm2 but the diagonal blocks of Lambda_theta are themselves diagonal.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t31.λ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"MixedModels.getθ(t31)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Random-effects terms with distinct grouping factors generate distinct elements of the reterms field of the LinearMixedModel object. Multiple ReMat objects are sorted by decreasing numbers of random effects.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"penicillin = MixedModels.dataset(:penicillin)\nfm4 = fit(MixedModel,\n @formula(diameter ~ 1 + (1|sample) + (1|plate)),\n penicillin)\nsparse(first(fm4.reterms))'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"sparse(last(fm4.reterms))'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Note that the first ReMat in fm4.reterms corresponds to grouping factor plate even though the term (1|plate) occurs in the formula after (1|sample).","category":"page"},{"location":"optimization/#Progress-of-the-optimization","page":"Details of the parameter estimation","title":"Progress of the optimization","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"By default a progress display is shown when fitting a model that takes a second or more to fit. (The optional named argument, progress=false, can be used to suppress this display.) The number of iterations performed, the average time per iteration and the current value of the objective are shown in this display.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"After the model has been fit, a summary of the optimization process is available as the optsum property of the LinearMixedModel.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fm2.optsum\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"More detailed information about the intermediate steps of the nonlinear optimizer can be obtained the fitlog field. By default, fitlog contains entries for only the initial and final steps, but additional information about every nth step can be obtained with the thin keyword-argument to fit, fit! and refit!:","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"refit!(fm2; thin=1)\nfm2.optsum.fitlog[1:10]\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/#A-blocked-Cholesky-factor","page":"Details of the parameter estimation","title":"A blocked Cholesky factor","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A LinearMixedModel object contains two blocked matrices; a symmetric matrix A (only the lower triangle is stored) and a lower-triangular L which is the lower Cholesky factor of the updated and inflated A. In versions 4.0.0 and later of MixedModels only the blocks in the lower triangle are stored in A and L, as a Vector{AbstractMatrix{T}}.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"BlockDescription shows the structure of the blocks","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"BlockDescription(fm2)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Another change in v4.0.0 and later is that the last row of blocks is constructed from m.Xymat which contains the full-rank model matrix X with the response y concatenated on the right.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The operation of installing a new value of the variance parameters, θ, and updating L","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"setθ!\nupdateL!","category":"page"},{"location":"optimization/#MixedModels.setθ!","page":"Details of the parameter estimation","title":"MixedModels.setθ!","text":"setθ!(m::LinearMixedModel, v)\n\nInstall v as the θ parameters in m.\n\n\n\n\n\nsetθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)\nsetθ!(bsamp::MixedModelFitCollection, i::Integer)\n\nInstall the values of the i'th θ value of bsamp.fits in bsamp.λ\n\n\n\n\n\n","category":"function"},{"location":"optimization/#MixedModels.updateL!","page":"Details of the parameter estimation","title":"MixedModels.updateL!","text":"updateL!(m::LinearMixedModel)\n\nUpdate the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)\n\nThis is the crucial step in evaluating the objective, given a new parameter value.\n\n\n\n\n\n","category":"function"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is the central step in evaluating the objective (negative twice the log-likelihood).","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Typically, the (1,1) block is the largest block in A and L and it has a special form, either Diagonal or UniformBlockDiagonal providing a compact representation and fast matrix multiplication or solutions of linear systems of equations.","category":"page"},{"location":"optimization/#Modifying-the-optimization-process","page":"Details of the parameter estimation","title":"Modifying the optimization process","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The OptSummary object contains both input and output fields for the optimizer. To modify the optimization process the input fields can be changed after constructing the model but before fitting it.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Suppose, for example, that the user wishes to try a Nelder-Mead optimization method instead of the default BOBYQA (Bounded Optimization BY Quadratic Approximation) method.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fm2nm = LinearMixedModel(@formula(reaction ~ 1+days+(1+days|subj)), sleepstudy);\nfm2nm.optsum.optimizer = :LN_NELDERMEAD;\nfit!(fm2nm; thin=1)\nfm2nm.optsum\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The parameter estimates are quite similar to those using :LN_BOBYQA but at the expense of 140 functions evaluations for :LN_NELDERMEAD versus 57 for :LN_BOBYQA. When plotting the progress of the individual fits, it becomes obvious that :LN_BOBYQA has fully converged by the time :LN_NELDERMEAD begins to approach the optimum.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"using Gadfly\nnm = fm2nm.optsum.fitlog\nbob = fm2.optsum.fitlog\nconvdf = DataFrame(algorithm=[repeat([\"NelderMead\"], length(nm));\n repeat([\"BOBYQA\"], length(bob))],\n objective=[last.(nm); last.(bob)],\n step=[1:length(nm); 1:length(bob)])\nplot(convdf, x=:step, y=:objective, color=:algorithm, Geom.line)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Run time can be constrained with maxfeval and maxtime.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"See the documentation for the NLopt package for details about the various settings.","category":"page"},{"location":"optimization/#Convergence-to-singular-covariance-matrices","page":"Details of the parameter estimation","title":"Convergence to singular covariance matrices","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"To ensure identifiability of Sigma_theta=sigma^2Lambda_theta Lambda_theta, the elements of theta corresponding to diagonal elements of Lambda_theta are constrained to be non-negative. For example, in a trivial case of a single, simple, scalar, random-effects term as in fm1, the one-dimensional theta vector is the ratio of the standard deviation of the random effects to the standard deviation of the response. It happens that -theta produces the same log-likelihood but, by convention, we define the standard deviation to be the positive square root of the variance. Requiring the diagonal elements of Lambda_theta to be non-negative is a generalization of using this positive square root.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"If the optimization converges on the boundary of the feasible region, that is if one or more of the diagonal elements of Lambda_theta is zero at convergence, the covariance matrix Sigma_theta will be singular. This means that there will be linear combinations of random effects that are constant. Usually convergence to a singular covariance matrix is a sign of an over-specified model.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Singularity can be checked with the issingular predicate function.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"issingular","category":"page"},{"location":"optimization/#MixedModels.issingular","page":"Details of the parameter estimation","title":"MixedModels.issingular","text":"issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\n\nTest whether the model m is singular if the parameter vector is θ.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nnote: Note\nFor GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.\n\n\n\n\n\nissingular(bsamp::MixedModelFitCollection;\n atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))\n\nTest each bootstrap sample for singularity of the corresponding fit.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nSee also issingular(::MixedModel).\n\n\n\n\n\n","category":"function"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"issingular(fm2)","category":"page"},{"location":"optimization/#Generalized-Linear-Mixed-Effects-Models","page":"Details of the parameter estimation","title":"Generalized Linear Mixed-Effects Models","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In a generalized linear model the responses are modelled as coming from a particular distribution, such as Bernoulli for binary responses or Poisson for responses that represent counts. The scalar distributions of individual responses differ only in their means, which are determined by a linear predictor expression eta=bf Xbeta, where, as before, bf X is a model matrix derived from the values of covariates and beta is a vector of coefficients.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The unconstrained components of eta are mapped to the, possibly constrained, components of the mean response, mu, via a scalar function, g^-1, applied to each component of eta. For historical reasons, the inverse of this function, taking components of mu to the corresponding component of eta is called the link function and the more frequently used map from eta to mu is the inverse link.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A generalized linear mixed-effects model (GLMM) is defined, for the purposes of this package, by","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"beginaligned\n (mathcalY mathcalB=bfb) simmathcalD(bfg^-1(Xbeta + Z b)phi)\n mathcalBsimmathcalN(bf0Sigma_theta) \nendaligned","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"where mathcalD indicates the distribution family parameterized by the mean and, when needed, a common scale parameter, phi. (There is no scale parameter for Bernoulli or for Poisson. Specifying the mean completely determines the distribution.)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Bernoulli\nPoisson","category":"page"},{"location":"optimization/#Distributions.Bernoulli","page":"Details of the parameter estimation","title":"Distributions.Bernoulli","text":"Bernoulli(p)\n\nA Bernoulli distribution is parameterized by a success rate p, which takes value 1 with probability p and 0 with probability 1-p.\n\nP(X = k) = begincases\n1 - p quad textfor k = 0 \np quad textfor k = 1\nendcases\n\nBernoulli() # Bernoulli distribution with p = 0.5\nBernoulli(p) # Bernoulli distribution with success rate p\n\nparams(d) # Get the parameters, i.e. (p,)\nsuccprob(d) # Get the success rate, i.e. p\nfailprob(d) # Get the failure rate, i.e. 1 - p\n\nExternal links:\n\nBernoulli distribution on Wikipedia\n\n\n\n\n\n","category":"type"},{"location":"optimization/#Distributions.Poisson","page":"Details of the parameter estimation","title":"Distributions.Poisson","text":"Poisson(λ)\n\nA Poisson distribution describes the number of independent events occurring within a unit time interval, given the average rate of occurrence λ.\n\nP(X = k) = fraclambda^kk e^-lambda quad text for k = 012ldots\n\nPoisson() # Poisson distribution with rate parameter 1\nPoisson(lambda) # Poisson distribution with rate parameter lambda\n\nparams(d) # Get the parameters, i.e. (λ,)\nmean(d) # Get the mean arrival rate, i.e. λ\n\nExternal links:\n\nPoisson distribution on Wikipedia\n\n\n\n\n\n","category":"type"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A GeneralizedLinearMixedModel object is generated from a formula, data frame and distribution family.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"verbagg = MixedModels.dataset(:verbagg)\nconst vaform = @formula(r2 ~ 1 + anger + gender + btype + situ + (1|subj) + (1|item));\nmdl = GeneralizedLinearMixedModel(vaform, verbagg, Bernoulli());\ntypeof(mdl)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A separate call to fit! can be used to fit the model. This involves optimizing an objective function, the Laplace approximation to the deviance, with respect to the parameters, which are beta, the fixed-effects coefficients, and theta, the covariance parameters. The starting estimate for beta is determined by fitting a GLM to the fixed-effects part of the formula","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.β","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"and the starting estimate for theta, which is a vector of the two standard deviations of the random effects, is chosen to be","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.θ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The Laplace approximation to the deviance requires determining the conditional modes of the random effects. These are the values that maximize the conditional density of the random effects, given the model parameters and the data. This is done using Penalized Iteratively Reweighted Least Squares (PIRLS). In most cases PIRLS is fast and stable. It is simply a penalized version of the IRLS algorithm used in fitting GLMs.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The distinction between the \"fast\" and \"slow\" algorithms in the MixedModels package (nAGQ=0 or nAGQ=1 in lme4) is whether the fixed-effects parameters, beta, are optimized in PIRLS or in the nonlinear optimizer. In a call to the pirls! function the first argument is a GeneralizedLinearMixedModel, which is modified during the function call. (By convention, the names of such mutating functions end in ! as a warning to the user that they can modify an argument, usually the first argument.) The second and third arguments are optional logical values indicating if beta is to be varied and if verbose output is to be printed.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"pirls!(mdl, true, false)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"deviance(mdl)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.β","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.θ # current values of the standard deviations of the random effects","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"If the optimization with respect to beta is performed within PIRLS then the nonlinear optimization of the Laplace approximation to the deviance requires optimization with respect to theta only. This is the \"fast\" algorithm. Given a value of theta, PIRLS is used to determine the conditional estimate of beta and the conditional mode of the random effects, b.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.b # conditional modes of b","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fit!(mdl, fast=true);\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The optimization process is summarized by","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.LMM.optsum\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"As one would hope, given the name of the option, this fit is comparatively fast.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"@btime fit(MixedModel, vaform, verbagg, Bernoulli(), fast=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The alternative algorithm is to use PIRLS to find the conditional mode of the random effects, given beta and theta and then use the general nonlinear optimizer to fit with respect to both beta and theta.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl1 = @btime fit(MixedModel, vaform, verbagg, Bernoulli())\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"This fit provided slightly better results (Laplace approximation to the deviance of 8151.400 versus 8151.583) but took 6 times as long. That is not terribly important when the times involved are a few seconds but can be important when the fit requires many hours or days of computing time.","category":"page"},{"location":"rankdeficiency/#Rank-deficiency-in-mixed-effects-models","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"using MixedModels\nusing DisplayAs","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"The (column) rank of a matrix refers to the number of linearly independent columns in the matrix. Clearly, the rank can never be more than the number of columns; however, the rank can be less than the number of columns. In a regression context, this corresponds to a (linear) dependency in the predictors. The simplest case of rank deficiency is a duplicated predictor or a predictor that is exactly a multiple of another predictor. However, rank deficiency can also arise in more subtle ways, such as from missing cells in a two-factor experimental design. Rank deficiency can also arise as an extreme case of multicollinearity. In all cases, it is important to remember that we can only assess the numerical rank of a matrix, which may be less than its theoretical rank, and that evaluation of this numerical rank requires setting some numerical tolerance levels. These choices are not always well defined. In other words, the rank of a matrix is well-defined in theory but in practice can be difficult to evaluate.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Rank deficiency can occur in two ways in mixed-effects models: in the fixed effects and in the random effects. The implications of rank deficiency and thus the handling of it differ between these.","category":"page"},{"location":"rankdeficiency/#Fixed-effects","page":"Rank deficiency in mixed-effects models","title":"Fixed effects","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"The consequences of rank deficiency in the fixed effects are similar to those in classical ordinary least squares (OLS) regression. If one or more predictors can be expressed as a linear combination of the other columns, then this column is redundant and the model matrix is rank deficient. Note however, that the redundant column is not defined uniquely. For example, in the case that of two columns a and b where b = 2a, then the rank deficiency can be handled by eliminating either a or b. While we defined b here in terms of a, it may be that b is actually the more 'fundamental' predictor and hence we may define a in terms of b as a = 0.5b. The user may of course possess this information, but the choice is not apparent to the modelling software. As such, the handling of rank deficiency in MixedModels.jl should not be taken as a replacement for thinking about the nature of the predictors in a given model.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"There is a widely accepted convention for how to make the coefficient estimates for these redundant columns well-defined: we set their value to zero and their standard errors to NaN (and thus also their z and p-values). The values that have been defined to be zero, as opposed to evaluating to zero, are displayed as -0.0 as an additional visual aid to distinguish them from the other coefficients. In practice the determination of rank and the redundant coefficients is done via a 'pivoting' scheme during a decomposition to move the surplus columns to the right side of the model matrix. In subsequent calculations, these columns are effectively ignored (as their estimates are zero and thus won't contribute to any other computations). For display purposes, this pivoting is unwound when the coef values are displayed.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Both the pivoted and unpivoted coefficients are available in MixedModels. The fixef extractor returns the pivoted, truncated estimates (i.e. the non redundant terms), while the coef extractor returns the unpivoted estimates (i.e. all terms, included the redundant ones). The same holds for the associated fixefnames and coefnames.","category":"page"},{"location":"rankdeficiency/#Pivoting-is-platform-dependent","page":"Rank deficiency in mixed-effects models","title":"Pivoting is platform dependent","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"In MixedModels.jl, we use standard numerical techniques to detect rank deficiency. We currently offer no guarantees as to which exactly of the standard techniques (pivoted QR decomposition, pivoted Cholesky decomposition, etc.) will be used. This choice should be viewed as an implementation detail. Similarly, we offer no guarantees as to which of columns will be treated as redundant. This choice may vary between releases and even between platforms (both in broad strokes of \"Linux\" vs. \"Windows\" and at the level of which BLAS options are loaded on a given processor architecture) for the same release. In other words, you should not rely on the order of the pivoted columns being consistent! when you switch to a different computer or a different operating system. If consistency in the pivoted columns is important to you, then you should instead determine your rank ahead of time and remove extraneous columns / predictors from your model specification.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"This lack of consistency guarantees arises from a more fundamental issue: numeric linear algebra is challenging and sensitive to the underlying floating point operations. Due to rounding error, floating point arithmetic is not associative:","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"0.1 + 0.1 + 0.1 - 0.3 == 0.1 + 0.1 + (0.1 - 0.3)","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"This means that \"nearly\" / numerically rank deficient matrices may or may not be detected as rank deficient, depending on details of the platform. Determining the rank of a matrix is the type of problem that is well-defined in theory but not in practice.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Currently, a coarse heuristic is applied to reduce the chance that the intercept column will be pivoted, but even this behavior is not guaranteed.","category":"page"},{"location":"rankdeficiency/#Undetected-Rank-Deficiency","page":"Rank deficiency in mixed-effects models","title":"Undetected Rank Deficiency","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Undetected rank deficiency in the fixed effects will lead to numerical issues, such as nonsensical estimates. A PosDefException may indicate rank deficiency because the covariance matrix will only be positive semidefinite and not positive definite (see Details of the parameter estimation). In other words, checking that the fixed effects are full rank is a great first step in debugging a PosDefException.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Note that PosDefException is not specific to rank deficiency and may arise in other ill-conditioned models. In any case, examining the model specification and the data to verify that they work together is the first step. For generalized linear mixed-effects models, it may also be worthwhile to try out fast=true instead of the default fast=false. See this GitHub issue and linked Discourse discussion for more information.","category":"page"},{"location":"rankdeficiency/#Random-effects","page":"Rank deficiency in mixed-effects models","title":"Random effects","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Rank deficiency presents less of a problem in the random effects than in the fixed effects because the \"estimates\" (more formally, the conditional modes of the random effects given the observed data) are determined as the solution to a penalized least squares problem. The shrinkage effect which moves the conditional modes (group-level predictions) towards the grand mean is a form of regularization, which provides well-defined \"estimates\" for overparameterized models. (For more reading on this general idea, see also this blog post on the model complexity myth.)","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"The nature of the penalty in the penalized least squares solution is such that the \"estimates\" are well-defined even when the covariance matrix of the random effects converges to a \"singular\" or \"boundary\" value. In other words, singularity of the covariance matrix for the random effects, which means that there are one or more directions in which there is no variability in the random effects, is different from singularity of the model matrix for the random effects, which would affect the ability to define uniquely these coefficients. The penalty term always provides a unique solution for the random-effects coefficients.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"In addition to handling naturally occurring rank deficiency in the random effects, the regularization allows us to fit explicitly overparameterized random effects. For example, we can use fulldummy to fit both an intercept term and n indicator variables in the random effects for a categorical variable with n levels instead of the usual n-1 contrasts.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"kb07 = MixedModels.dataset(:kb07)\ncontrasts = Dict(var => HelmertCoding() for var in (:spkr, :prec, :load))\nfit(MixedModel, @formula(rt_raw ~ spkr * prec * load + (1|subj) + (1+prec|item)), kb07; contrasts=contrasts)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"fit(MixedModel, @formula(rt_raw ~ spkr * prec * load + (1|subj) + (1+fulldummy(prec)|item)), kb07; contrasts=contrasts)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"This may be useful when the PCA property suggests a random effects structure larger than only main effects but smaller than all interaction terms. This is also similar to the functionality provided by dummy in lme4, but as in the difference between zerocorr in Julia and || in R, there are subtle differences in how this expansion interacts with other terms in the random effects.","category":"page"},{"location":"bootstrap/#Parametric-bootstrap-for-mixed-effects-models","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Julia is well-suited to implementing bootstrapping and other simulation-based methods for statistical models. The parametricbootstrap function in the MixedModels package provides an efficient parametric bootstrap for mixed-effects models.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"parametricbootstrap","category":"page"},{"location":"bootstrap/#MixedModels.parametricbootstrap","page":"Parametric bootstrap for mixed-effects models","title":"MixedModels.parametricbootstrap","text":"parametricbootstrap([rng::AbstractRNG], nsamp::Integer, m::MixedModel{T}, ftype=T;\n β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))\n\nPerform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.\n\nThe default random number generator is Random.GLOBAL_RNG.\n\nftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.\n\nKeyword Arguments\n\nβ, σ, and θ are the values of m's parameters for simulating the responses.\nσ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for\n\nfamilies with a dispersion parameter.\n\nprogress controls whether the progress bar is shown. Note that the progress\n\nbar is automatically disabled for non-interactive (i.e. logging) contexts.\n\noptsum_overrides is used to override values of OptSummary in the models\n\nfit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.\n\nnote: Note\nAll coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.\n\n\n\n\n\n","category":"function"},{"location":"bootstrap/#The-parametric-bootstrap","page":"Parametric bootstrap for mixed-effects models","title":"The parametric bootstrap","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Bootstrapping is a family of procedures for generating sample values of a statistic, allowing for visualization of the distribution of the statistic or for inference from this sample of values.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A parametric bootstrap is used with a parametric model, m, that has been fit to data. The procedure is to simulate n response vectors from m using the estimated parameter values and refit m to these responses in turn, accumulating the statistics of interest at each iteration.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"The parameters of a LinearMixedModel object are the fixed-effects parameters, β, the standard deviation, σ, of the per-observation noise, and the covariance parameter, θ, that defines the variance-covariance matrices of the random effects.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"For example, a simple linear mixed-effects model for the Dyestuff data in the lme4 package for R is fit by","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"using DataFrames\nusing Gadfly # plotting package\nusing MixedModels\nusing Random","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"dyestuff = MixedModels.dataset(:dyestuff)\nm1 = fit(MixedModel, @formula(yield ~ 1 + (1 | batch)), dyestuff)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"To bootstrap the model parameters, first initialize a random number generator then create a bootstrap sample and extract the tbl property, which is a Table - a lightweight dataframe-like object.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"const rng = MersenneTwister(1234321);\nsamp = parametricbootstrap(rng, 10_000, m1);\ntbl = samp.tbl","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A density plot of the estimates of σ, the residual standard deviation, can be created as","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.σ, Geom.density, Guide.xlabel(\"Parametric bootstrap estimates of σ\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"or, for the intercept parameter","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.β1, Geom.density, Guide.xlabel(\"Parametric bootstrap estimates of β₁\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A density plot of the estimates of the standard deviation of the random effects is obtained as","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.σ1, Geom.density,\n Guide.xlabel(\"Parametric bootstrap estimates of σ₁\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Notice that this density plot has a spike, or mode, at zero. Although this mode appears to be diffuse, this is an artifact of the way that density plots are created. In fact, it is a pulse, as can be seen from a histogram.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.σ1, Geom.histogram,\n Guide.xlabel(\"Parametric bootstrap estimates of σ₁\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"The bootstrap sample can be used to generate intervals that cover a certain percentage of the bootstrapped values. We refer to these as \"coverage intervals\", similar to a confidence interval. The shortest such intervals, obtained with the shortestcovint extractor, correspond to a highest posterior density interval in Bayesian inference.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"shortestcovint","category":"page"},{"location":"bootstrap/#MixedModels.shortestcovint","page":"Parametric bootstrap for mixed-effects models","title":"MixedModels.shortestcovint","text":"shortestcovint(v, level = 0.95)\n\nReturn the shortest interval containing level proportion of the values of v\n\n\n\n\n\nshortestcovint(bsamp::MixedModelFitCollection, level = 0.95)\n\nReturn the shortest interval containing level proportion for each parameter from bsamp.allpars.\n\nwarning: Warning\nCurrently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"function"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"We generate these directly from the original bootstrap object:","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Table(shortestcovint(samp))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A value of zero for the standard deviation of the random effects is an example of a singular covariance. It is easy to detect the singularity in the case of a scalar random-effects term. However, it is not as straightforward to detect singularity in vector-valued random-effects terms.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"For example, if we bootstrap a model fit to the sleepstudy data","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"sleepstudy = MixedModels.dataset(:sleepstudy)\ncontrasts = Dict(:subj => Grouping())\nm2 = let f = @formula reaction ~ 1+days+(1+days|subj)\n fit(MixedModel, f, sleepstudy; contrasts)\nend","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"samp2 = parametricbootstrap(rng, 10_000, m2);\ntbl2 = samp2.tbl","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"the singularity can be exhibited as a standard deviation of zero or as a correlation of pm1.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"shortestcovint(samp2)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A histogram of the estimated correlations from the bootstrap sample has a spike at +1.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl2.ρ1, Geom.histogram,\n Guide.xlabel(\"Parametric bootstrap samples of correlation of random effects\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"or, as a count,","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(tbl2.ρ1 .≈ 1)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Close examination of the histogram shows a few values of -1.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(tbl2.ρ1 .≈ -1)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Furthermore there are even a few cases where the estimate of the standard deviation of the random effect for the intercept is zero.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(tbl2.σ1 .≈ 0)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"There is a general condition to check for singularity of an estimated covariance matrix or matrices in a bootstrap sample. The parameter optimized in the estimation is θ, the relative covariance parameter. Some of the elements of this parameter vector must be non-negative and, when one of these components is approximately zero, one of the covariance matrices will be singular.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"The issingular method for a MixedModel object that tests if a parameter vector θ corresponds to a boundary or singular fit.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"This operation is encapsulated in a method for the issingular function.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(issingular(samp2))","category":"page"},{"location":"bootstrap/#Reduced-Precision-Bootstrap","page":"Parametric bootstrap for mixed-effects models","title":"Reduced Precision Bootstrap","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"parametricbootstrap accepts an optional keyword argument optsum_overrides, which can be used to override the convergence criteria for bootstrap replicates. One possibility is setting ftol_rel=1e-8, i.e., considering the model converged when the relative change in the objective between optimizer iterations is smaller than 0.00000001. This threshold corresponds approximately to the precision from treating the value of the objective as a single precision (Float32) number, while not changing the precision of the intermediate computations. The resultant loss in precision will generally be smaller than the variation that the bootstrap captures, but can greatly speed up the fitting process for each replicates, especially for large models. More directly, lowering the fit quality for each replicate will reduce the quality of each replicate, but this may be more than compensated for by the ability to fit a much larger number of replicates in the same time.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"t = @timed parametricbootstrap(MersenneTwister(42), 1000, m2; progress=false)\nt.time","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"optsum_overrides = (; ftol_rel=1e-8)\nt = @timed parametricbootstrap(MersenneTwister(42), 1000, m2; optsum_overrides, progress=false)\nt.time","category":"page"},{"location":"bootstrap/#Distributed-Computing-and-the-Bootstrap","page":"Parametric bootstrap for mixed-effects models","title":"Distributed Computing and the Bootstrap","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Earlier versions of MixedModels.jl supported a multi-threaded bootstrap via the use_threads keyword argument. However, with improved BLAS multithreading, the Julia-level threads often wound up competing with the BLAS threads, leading to no improvement or even a worsening of performance when use_threads=true. Nonetheless, the bootstrap is a classic example of an embarrassingly parallel problem and so we provide a few convenience methods for combining results computed separately. In particular, there are vcat and an optimized reduce(::typeof(vcat)) methods for MixedModelBootstrap objects. For computers with many processors (as opposed to a single processor with several cores) or for computing clusters, these provide a convenient way to split the computation across nodes.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"using Distributed\n# you already have 1 proc by default, so add the number of additional cores with `addprocs`\n# you need at least as many RNGs as cores you want to use in parallel\n# but you shouldn't use all of your cores because nested within this\n# is the multithreading of the linear algebra\n# addprocs(1)\n@info \"Currently using $(nprocs()) processors total and $(nworkers()) for work\"\n\n# Load the necessary packages on all workers\n# For clusters, you will also need to make sure that the Julia\n# environment (Project.toml) is set up and activated on each worker.\n@everywhere begin\n using ProgressMeter\n using MixedModels\nend\n# copy everything to workers\n@showprogress for w in workers()\n remotecall_fetch(() -> coefnames(m2), w)\nend\n\n# split the replicates across the workers\n# this rounds down, so if the number of workers doesn't divide the\n# number of replicates, you'll be a few replicates short!\nn_replicates = 1000\nn_rep_per_worker = n_replicates ÷ nworkers()\n# NB: You need a different seed/RNG for each worker otherwise you will\n# have copies of the same replicates and not independent replicates!\npb_map = @showprogress pmap(MersenneTwister.(1:nworkers())) do rng\n parametricbootstrap(rng, n_rep_per_worker, m2; optsum_overrides)\nend;\n\n# get rid of all the workers\n# rmprocs(workers())\n\nconfint(reduce(vcat, pb_map))","category":"page"},{"location":"prediction/#Prediction-and-simulation-in-Mixed-Effects-Models","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"","category":"section"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"We recommend the MixedModelsSim.jl package and associated documentation for useful tools in constructing designs to simulate. For now, we'll use the sleep study data as a starting point.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"using DataFrames\nusing MixedModels\nusing StatsBase\nusing DisplayAs # hide\n# use a DataFrame to make it easier to change things later\nslp = DataFrame(MixedModels.dataset(:sleepstudy))\nslpm = fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj)), slp)\nDisplayAs.Text(slpm) # hide","category":"page"},{"location":"prediction/#Prediction","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction","text":"","category":"section"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"The simplest form of prediction are the fitted values from the model: they are indeed the model's predictions for the observed data.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"When generalizing to new data, we need to consider what happens if there are new, previously unobserved levels of the grouping variable(s). MixedModels.jl provides three options:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":":error: error on encountering unobserved levels\n:population: use population values (i.e. only the fixed effects) for observations with unobserved levels\n:missing: return missing for observations with unobserved levels.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Providing either no prediction (:error, :missing) or providing the population-level values seem to be the most reasonable ways for predicting new values. For simulating new values based on previous estimates of the variance components, use simulate.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"In the case where there are no new levels of the grouping variable, all three of these methods provide the same results:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp; new_re_levels=:population) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp; new_re_levels=:missing) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp; new_re_levels=:error) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"In the case where there are new levels of the grouping variable, these methods differ.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"# create a new level\nslp2 = transform(slp, :subj => ByRow(x -> (x == \"S308\" ? \"NEW\" : x)) => :subj)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"try\n predict(slpm, slp2; new_re_levels=:error)\ncatch e\n show(e)\nend","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp2; new_re_levels=:missing)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp2; new_re_levels=:population)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"note: Note\nCurrently, we do not support predicting based on a subset of the random effects.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"note: Note\npredict is deterministic (within the constraints of floating point) and never adds noise to the result. If you want to construct prediction intervals, then simulate will generate new data with noise (including new values of the random effects).","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"For generalized linear mixed models, there is an additional keyword argument to predict: type specifies whether the predictions are returned on the scale of the linear predictor (:linpred) or on the level of the response (:response) (i.e. the level at which the values were originally observed).","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"cbpp = DataFrame(MixedModels.dataset(:cbpp))\ncbpp.rate = cbpp.incid ./ cbpp.hsz\ngm = fit(MixedModel, @formula(rate ~ 1 + period + (1|herd)), cbpp, Binomial(), wts=float(cbpp.hsz))\npredict(gm, cbpp; type=:response) ≈ fitted(gm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"logit(x) = log(x / (1 - x))\npredict(gm, cbpp; type=:linpred) ≈ logit.(fitted(gm))","category":"page"},{"location":"prediction/#Simulation","page":"Prediction and simulation in Mixed-Effects Models","title":"Simulation","text":"","category":"section"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"In contrast to predict, simulate and simulate! introduce randomness. This randomness occurs both at the level of the observation-level (residual) variance and at the level of the random effects, where new conditional modes are sampled based on the specified covariance parameter (θ; see Details of the parameter estimation), which defaults to the estimated value of the model. For reproducibility, we specify a pseudorandom generator here; if none is provided, the global PRNG is taken as the default.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"The simplest example of simulate takes a fitted model and generates a new response vector based on the existing model matrices combined with noise.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"using Random\nynew = simulate(MersenneTwister(42), slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"The simulated response can also be placed in a pre-allocated vector:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"ynew2 = zeros(nrow(slp))\nsimulate!(MersenneTwister(42), ynew2, slpm)\nynew2 ≈ ynew","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Or even directly replace the previous response vector in a model, at which point the model must be refit to the new values:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"slpm2 = deepcopy(slpm)\nrefit!(simulate!(MersenneTwister(42), slpm2))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"This inplace simulation actually forms the basis of parametricbootstrap.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Finally, we can also simulate the response from entirely new data.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"df = DataFrame(days = repeat(1:10, outer=20), subj=repeat(1:20, inner=10))\ndf[!, :subj] = string.(\"S\", lpad.(df.subj, 2, \"0\"))\ndf[!, :reaction] .= 0\ndf\nDisplayAs.Text(df) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"ysim = simulate(MersenneTwister(42), slpm, df)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Note that this is a convenience method for creating a new model and then using the parameters from the old model to call simulate on that model. In other words, this method incurs the cost of constructing a new model and then discarding it. If you could re-use that model (e.g., fitting that model as part of a simulation study), it often makes sense to do these steps to perform these steps explicitly and avoid the unnecessary construction and discarding of an intermediate model:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"msim = LinearMixedModel(@formula(reaction ~ 1 + days + (1|subj)), df)\nsimulate!(MersenneTwister(42), msim; θ=slpm.θ, β=slpm.β, σ=slpm.σ)\nresponse(msim) ≈ ysim","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"fit!(msim)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"For simulating from generalized linear mixed models, there is no type option because the observation-level always occurs at the level of the response and not of the linear predictor.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"warning: Warning\nSimulating the model response in place may not yield the same result as simulating into a pre-allocated or new vector, depending on choice of pseudorandom number generator. Random number generation in Julia allows optimization based on type, and the internal storage type of the model response (currently a view into a matrix storing the concatenated fixed-effects model matrix and the response) may not match the type of a pre-allocated or new vector. See also discussion here.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"note: Note\nAll the methods that take new data as a table construct an additional MixedModel behind the scenes, even when the new data is exactly the same as the data that the model was fitted to. For the simulation methods in particular, these thus form a convenience wrapper for constructing a new model and calling simulate without new data on that model with the parameters from the original model.","category":"page"},{"location":"GaussHermite/#Normalized-Gauss-Hermite-Quadrature","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"","category":"section"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Gaussian Quadrature rules provide sets of x values, called abscissae, and corresponding weights, w, to approximate an integral with respect to a weight function, g(x). For a kth order rule the approximation is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"int f(x)g(x)dx approx sum_i=1^k w_i f(x_i)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"For the Gauss-Hermite rule the weight function is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"g(x) = e^-x^2","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"and the domain of integration is (-infty infty). A slight variation of this is the normalized Gauss-Hermite rule for which the weight function is the standard normal density","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"g(z) = phi(z) = frace^-z^22sqrt2pi","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Thus, the expected value of f(z), where mathcalZsimmathscrN(01), is approximated as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"mathbbEf=int_-infty^infty f(z) phi(z)dzapproxsum_i=1^k w_if(z_i) ","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Naturally, there is a caveat. For the approximation to be accurate the function f(z) must behave like a low-order polynomial over the range of interest. More formally, a kth order rule is exact when f is a polynomial of order 2k-1 or less. [1]","category":"page"},{"location":"GaussHermite/#Evaluating-the-weights-and-abscissae","page":"Normalized Gauss-Hermite Quadrature","title":"Evaluating the weights and abscissae","text":"","category":"section"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"In the Golub-Welsch algorithm the abscissae for a particular Gaussian quadrature rule are determined as the eigenvalues of a symmetric tri-diagonal matrix and the weights are derived from the squares of the first row of the matrix of eigenvectors. For a kth order normalized Gauss-Hermite rule the tridiagonal matrix has zeros on the diagonal and the square roots of 1:k-1 on the super- and sub-diagonal, e.g.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using DisplayAs","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using DataFrames, LinearAlgebra, Gadfly\nsym3 = SymTridiagonal(zeros(3), sqrt.(1:2))\nev = eigen(sym3);\nev.values","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"abs2.(ev.vectors[1,:])","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"As a function of k this can be written as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"function gausshermitenorm(k)\n ev = eigen(SymTridiagonal(zeros(k), sqrt.(1:k-1)))\n ev.values, abs2.(ev.vectors[1,:])\nend;","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"providing","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"gausshermitenorm(3)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The weights and positions are often shown as a lollipop plot. For the 9th order rule these are","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"gh9=gausshermitenorm(9)\nplot(x=gh9[1], y=gh9[2], Geom.hair, Geom.point, Guide.ylabel(\"Weight\"), Guide.xlabel(\"\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Notice that the magnitudes of the weights drop quite dramatically away from zero, even on a logarithmic scale","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(\n x=gh9[1], y=gh9[2], Geom.hair, Geom.point,\n Scale.y_log2, Guide.ylabel(\"Weight (log scale)\"),\n Guide.xlabel(\"\"),\n)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The definition of MixedModels.GHnorm is similar to the gausshermitenorm function with some extra provisions for ensuring symmetry of the abscissae and the weights and for caching values once they have been calculated.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"GHnorm","category":"page"},{"location":"GaussHermite/#MixedModels.GHnorm","page":"Normalized Gauss-Hermite Quadrature","title":"MixedModels.GHnorm","text":"GHnorm(k::Int)\n\nReturn the (unique) GaussHermiteNormalized{k} object.\n\nThe function values are stored (memoized) when first evaluated. Subsequent evaluations for the same k have very low overhead.\n\n\n\n\n\n","category":"function"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using MixedModels\nGHnorm(3)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"By the properties of the normal distribution, when mathcalXsimmathscrN(mu sigma^2)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"mathbbEg(x) approx sum_i=1^k g(mu + sigma z_i)w_i","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"For example, mathbbEmathcalX^2 where mathcalXsimmathcalN(2 3^2) is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"μ = 2; σ = 3; ghn3 = GHnorm(3);\nsum(@. ghn3.w * abs2(μ + σ * ghn3.z)) # should be μ² + σ² = 13","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"(In general a dot, '.', after the function name in a function call, as in abs2.(...), or before an operator creates a fused vectorized evaluation in Julia. The macro @. has the effect of vectorizing all operations in the subsequent expression.)","category":"page"},{"location":"GaussHermite/#Application-to-a-model-for-contraception-use","page":"Normalized Gauss-Hermite Quadrature","title":"Application to a model for contraception use","text":"","category":"section"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A binary response is a \"Yes\"/\"No\" type of answer. For example, in a 1989 fertility survey of women in Bangladesh (reported in Huq, N. M. and Cleland, J., 1990) one response of interest was whether the woman used artificial contraception. Several covariates were recorded including the woman's age (centered at the mean), the number of live children the woman has had (in 4 categories: 0, 1, 2, and 3 or more), whether she lived in an urban setting, and the district in which she lived. The version of the data used here is that used in review of multilevel modeling software conducted by the Center for Multilevel Modelling, currently at University of Bristol (http://www.bristol.ac.uk/cmm/learning/mmsoftware/data-rev.html). These data are available as the :contra dataset.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"contra = DataFrame(MixedModels.dataset(:contra))\ndescribe(contra)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A smoothed scatterplot of contraception use versus age","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(contra, x=:age, y=:use, Geom.smooth, Guide.xlabel(\"Centered age (yr)\"),\n Guide.ylabel(\"Contraception use\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"shows that the proportion of women using artificial contraception is approximately quadratic in age.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A model with fixed-effects for age, age squared, number of live children and urban location and with random effects for district, is fit as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"const form1 = @formula use ~ 1 + age + abs2(age) + livch + urban + (1|dist);\nm1 = fit(MixedModel, form1, contra, Bernoulli(), fast=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"For a model such as m1, which has a single, scalar random-effects term, the unscaled conditional density of the spherical random effects variable, mathcalU, given the observed data, mathcalY=mathbfy_0, can be expressed as a product of scalar density functions, f_i(u_i) i=1dotsq. In the PIRLS algorithm, which determines the conditional mode vector, tildemathbfu, the optimization is performed on the deviance scale,","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"D(mathbfu)=-2sum_i=1^q log(f_i(u_i))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The objective, D, consists of two parts: the sum of the (squared) deviance residuals, measuring fidelity to the data, and the squared length of mathbfu, which is the penalty. In the PIRLS algorithm, only the sum of these components is needed. To use Gauss-Hermite quadrature the contributions of each of the u_ii=1dotsq should be separately evaluated.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"const devc0 = map!(abs2, m1.devc0, m1.u[1]); # start with uᵢ²\nconst devresid = m1.resp.devresid; # n-dimensional vector of deviance residuals\nconst refs = only(m1.LMM.reterms).refs; # n-dimensional vector of indices in 1:q\nfor (dr, i) in zip(devresid, refs)\n devc0[i] += dr\nend\nshow(devc0)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"One thing to notice is that, even on the deviance scale, the contributions of different districts can be of different magnitudes. This is primarily due to different sample sizes in the different districts.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using FreqTables\nfreqtable(contra, :dist)'","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Because the first district has one of the largest sample sizes and the third district has the smallest sample size, these two will be used for illustration. For a range of u values, evaluate the individual components of the deviance and store them in a matrix.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"const devc = m1.devc;\nconst xvals = -5.0:2.0^(-4):5.0;\nconst uv = vec(m1.u[1]);\nconst u₀ = vec(m1.u₀[1]);\nresults = zeros(length(devc0), length(xvals))\nfor (j, u) in enumerate(xvals)\n fill!(devc, abs2(u))\n fill!(uv, u)\n MixedModels.updateη!(m1)\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), devc)\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A plot of the deviance contribution versus u_1","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line, Guide.xlabel(\"u₁\"),\n Guide.ylabel(\"Deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"shows that the deviance contribution is very close to a quadratic. This is also true for u_3","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line, Guide.xlabel(\"u₃\"),\n Guide.ylabel(\"Deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The PIRLS algorithm provides the locations of the minima of these scalar functions, stored as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"m1.u₀[1]","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"the minima themselves, evaluated as devc0 above, and a horizontal scale, which is the inverse of diagonal of the Cholesky factor. As shown below, this is an estimate of the conditional standard deviations of the components of mathcalU.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using MixedModels: block\nconst s = inv.(m1.LMM.L[block(1,1)].diag);\ns'","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The curves can be put on a common scale, corresponding to the standard normal, as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"for (j, z) in enumerate(xvals)\n @. uv = u₀ + z * s\n MixedModels.updateη!(m1)\n @. devc = abs2(uv) - devc0\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), devc)\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₁\"),\n Guide.ylabel(\"Shifted deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₃\"),\n Guide.ylabel(\"Shifted deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"On the original density scale these become","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"for (j, z) in enumerate(xvals)\n @. uv = u₀ + z * s\n MixedModels.updateη!(m1)\n @. devc = abs2(uv) - devc0\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), @. exp(-devc/2))\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₁\"),\n Guide.ylabel(\"Conditional density\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₃\"),\n Guide.ylabel(\"Conditional density\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"and the function to be integrated with the normalized Gauss-Hermite rule is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"for (j, z) in enumerate(xvals)\n @. uv = u₀ + z * s\n MixedModels.updateη!(m1)\n @. devc = abs2(uv) - devc0\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), @. exp((abs2(z) - devc)/2))\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₁\"), Guide.ylabel(\"Kernel ratio\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₃\"), Guide.ylabel(\"Kernel ratio\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"[1]: https://en.wikipedia.org/wiki/Gaussian_quadrature","category":"page"},{"location":"#MixedModels.jl-Documentation","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"","category":"section"},{"location":"","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"CurrentModule = MixedModels","category":"page"},{"location":"","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"MixedModels.jl is a Julia package providing capabilities for fitting and examining linear and generalized linear mixed-effect models. It is similar in scope to the lme4 package for R.","category":"page"},{"location":"","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"Pages = [\n \"constructors.md\",\n \"optimization.md\",\n \"GaussHermite.md\",\n \"bootstrap.md\",\n \"rankdeficiency.md\",\n \"mime.md\",\n]\nDepth = 2","category":"page"}] +[{"location":"constructors/#Model-constructors","page":"Model constructors","title":"Model constructors","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The LinearMixedModel type represents a linear mixed-effects model. Typically it is constructed from a Formula and an appropriate Table type, usually a DataFrame.","category":"page"},{"location":"constructors/#Examples-of-linear-mixed-effects-model-fits","page":"Model constructors","title":"Examples of linear mixed-effects model fits","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For illustration, several data sets from the lme4 package for R are made available in .arrow format in this package. Often, for convenience, we will convert these to DataFrames. These data sets include the dyestuff and dyestuff2 data sets.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"using DisplayAs","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"using DataFrames, MixedModels, StatsModels\ndyestuff = MixedModels.dataset(:dyestuff)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"describe(DataFrame(dyestuff))","category":"page"},{"location":"constructors/#The-@formula-language-in-Julia","page":"Model constructors","title":"The @formula language in Julia","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.jl builds on the the Julia formula language provided by StatsModels.jl, which is similar to the formula language in R and is also based on the notation from Wilkinson and Rogers (1973). There are two ways to construct a formula in Julia. The first way is to enclose the formula expression in the @formula macro:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@formula","category":"page"},{"location":"constructors/#StatsModels.@formula","page":"Model constructors","title":"StatsModels.@formula","text":"@formula(ex)\n\nCapture and parse a formula expression as a Formula struct.\n\nA formula is an abstract specification of a dependence between left-hand and right-hand side variables as in, e.g., a regression model. Each side specifies at a high level how tabular data is to be converted to a numerical matrix suitable for modeling. This specification looks something like Julia code, is represented as a Julia Expr, but uses special syntax. The @formula macro takes an expression like y ~ 1 + a*b, transforms it according to the formula syntax rules into a lowered form (like y ~ 1 + a + b + a&b), and constructs a Formula struct which captures the original expression, the lowered expression, and the left- and right-hand-side.\n\nOperators that have special interpretations in this syntax are\n\n~ is the formula separator, where it is a binary operator (the first argument is the left-hand side, and the second is the right-hand side.\n+ concatenates variables as columns when generating a model matrix.\n& represents an interaction between two or more variables, which corresponds to a row-wise kronecker product of the individual terms (or element-wise product if all terms involved are continuous/scalar).\n* expands to all main effects and interactions: a*b is equivalent to a+b+a&b, a*b*c to a+b+c+a&b+a&c+b&c+a&b&c, etc.\n1, 0, and -1 indicate the presence (for 1) or absence (for 0 and -1) of an intercept column.\n\nThe rules that are applied are\n\nThe associative rule (un-nests nested calls to +, &, and *).\nThe distributive rule (interactions & distribute over concatenation +).\nThe * rule expands a*b to a+b+a&b (recursively).\nSubtraction is converted to addition and negation, so x-1 becomes x + -1 (applies only to subtraction of literal 1).\nSingle-argument & calls are stripped, so &(x) becomes the main effect x.\n\n\n\n\n\n","category":"macro"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The second way is to combine Terms with operators like +, &, ~, and others at \"run time\". This is especially useful if you wish to create a formula from a list a variable names. For instance, the following are equivalent:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@formula(y ~ 1 + a + b + a & b) == (term(:y) ~ term(1) + term(:a) + term(:b) + term(:a) & term(:b))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.jl provides additional formula syntax for representing random-effects terms. Most importantly, | separates random effects and their grouping factors (as in the formula extension used by the R package lme4. Much like with the base formula language, | can be used within the @formula macro and to construct a formula programmatically:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@formula(y ~ 1 + a + b + (1 + a + b | g))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"terms = sum(term(t) for t in [1, :a, :b])\ngroup = term(:g)\nresponse = term(:y)\nresponse ~ terms + (terms | group)","category":"page"},{"location":"constructors/#Models-with-simple,-scalar-random-effects","page":"Model constructors","title":"Models with simple, scalar random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A basic model with simple, scalar random effects for the levels of batch (the batch of an intermediate product, in this case) is declared and fit as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm = @formula(yield ~ 1 + (1|batch))\nfm1 = fit(MixedModel, fm, dyestuff)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"(If you are new to Julia you may find that this first fit takes an unexpectedly long time, due to Just-In-Time (JIT) compilation of the code. The subsequent calls to such functions are much faster.)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"using BenchmarkTools\ndyestuff2 = MixedModels.dataset(:dyestuff2)\n@benchmark fit(MixedModel, $fm, $dyestuff2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"By default, the model is fit by maximum likelihood. To use the REML criterion instead, add the optional named argument REML=true to the call to fit","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1reml = fit(MixedModel, fm, dyestuff, REML=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Floating-point-type-in-the-model","page":"Model constructors","title":"Floating-point type in the model","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The type of fm1","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"typeof(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"includes the floating point type used internally for the various matrices, vectors, and scalars that represent the model. At present, this will always be Float64 because the parameter estimates are optimized using the NLopt package which calls compiled C code that only allows for optimization with respect to a Float64 parameter vector.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"So in theory other floating point types, such as BigFloat or Float32, can be used to define a model but in practice only Float64 works at present.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In theory, theory and practice are the same. In practice, they aren't. – Anon","category":"page"},{"location":"constructors/#Simple,-scalar-random-effects","page":"Model constructors","title":"Simple, scalar random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A simple, scalar random effects term in a mixed-effects model formula is of the form (1|G). All random effects terms end with |G where G is the grouping factor for the random effect. The name or, more generally the expression, G, should evaluate to a categorical array that has a distinct set of levels. The random effects are associated with the levels of the grouping factor.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A scalar random effect is, as the name implies, one scalar value for each level of the grouping factor. A simple, scalar random effects term is of the form, (1|G). It corresponds to a shift in the intercept for each level of the grouping factor.","category":"page"},{"location":"constructors/#Models-with-vector-valued-random-effects","page":"Model constructors","title":"Models with vector-valued random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The sleepstudy data are observations of reaction time, reaction, on several subjects, subj, after 0 to 9 days of sleep deprivation, days. A model with random intercepts and random slopes for each subject, allowing for within-subject correlation of the slope and intercept, is fit as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sleepstudy = MixedModels.dataset(:sleepstudy)\nfm2 = fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Models-with-multiple,-scalar-random-effects-terms","page":"Model constructors","title":"Models with multiple, scalar random-effects terms","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A model for the Penicillin data incorporates random effects for the plate, and for the sample. As every sample is used on every plate these two factors are crossed.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"penicillin = MixedModels.dataset(:penicillin)\nfm3 = fit(MixedModel, @formula(diameter ~ 1 + (1|plate) + (1|sample)), penicillin)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In contrast, the cask grouping factor is nested within the batch grouping factor in the Pastes data.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"pastes = DataFrame(MixedModels.dataset(:pastes))\ndescribe(pastes)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"This can be expressed using the solidus (the \"/\" character) to separate grouping factors, read \"cask nested within batch\":","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm4a = fit(MixedModel, @formula(strength ~ 1 + (1|batch/cask)), pastes)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"If the levels of the inner grouping factor are unique across the levels of the outer grouping factor, then this nesting does not need to expressed explicitly in the model syntax. For example, defining sample to be the combination of batch and cask, yields a naming scheme where the nesting is apparent from the data even if not expressed in the formula. (That is, each level of sample occurs in conjunction with only one level of batch.) As such, this model is equivalent to the previous one.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"pastes.sample = (string.(pastes.cask, \"&\", pastes.batch))\nfm4b = fit(MixedModel, @formula(strength ~ 1 + (1|sample) + (1|batch)), pastes)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In observational studies it is common to encounter partially crossed grouping factors. For example, the InstEval data are course evaluations by students, s, of instructors, d. Additional covariates include the academic department, dept, in which the course was given and service, whether or not it was a service course.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"insteval = MixedModels.dataset(:insteval)\nfm5 = fit(MixedModel, @formula(y ~ 1 + service * dept + (1|s) + (1|d)), insteval)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Simplifying-the-random-effect-correlation-structure","page":"Model constructors","title":"Simplifying the random effect correlation structure","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.jl estimates not only the variance of the effects for each random effect level, but also the correlation between the random effects for different predictors. So, for the model of the sleepstudy data above, one of the parameters that is estimated is the correlation between each subject's random intercept (i.e., their baseline reaction time) and slope (i.e., their particular change in reaction time per day of sleep deprivation). In some cases, you may wish to simplify the random effects structure by removing these correlation parameters. This often arises when there are many random effects you want to estimate (as is common in psychological experiments with many conditions and covariates), since the number of random effects parameters increases as the square of the number of predictors, making these models difficult to estimate from limited data.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The special syntax zerocorr can be applied to individual random effects terms inside the @formula:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm2zerocorr_fm = fit(MixedModel, @formula(reaction ~ 1 + days + zerocorr(1 + days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Alternatively, correlations between parameters can be removed by including them as separate random effects terms:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj) + (days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Finally, for predictors that are categorical, MixedModels.jl will estimate correlations between each level. Notice the large number of correlation parameters if we treat days as a categorical variable by giving it contrasts:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Separating the 1 and days random effects into separate terms removes the correlations between the intercept and the levels of days, but not between the levels themselves:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj) + (days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"(Notice that the variance component for days: 1 is estimated as zero, so the correlations for this component are undefined and expressed as NaN, not a number.)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"An alternative is to force all the levels of days as indicators using fulldummy encoding.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fulldummy","category":"page"},{"location":"constructors/#MixedModels.fulldummy","page":"Model constructors","title":"MixedModels.fulldummy","text":"fulldummy(term::CategoricalTerm)\n\nAssign \"contrasts\" that include all indicator columns (dummy variables) and an intercept column.\n\nThis will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or \"shrinkage\", of the conditional modes.\n\nThe interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1 + fulldummy(days)|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"This fit produces a better fit as measured by the objective (negative twice the log-likelihood is 1610.8) but at the expense of adding many more parameters to the model. As a result, model comparison criteria such, as AIC and BIC, are inflated.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"But using zerocorr on the individual terms does remove the correlations between the levels:","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + zerocorr(1 + days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj) + zerocorr(days|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fit(MixedModel, @formula(reaction ~ 1 + days + zerocorr(1 + fulldummy(days)|subj)), sleepstudy,\n contrasts = Dict(:days => DummyCoding()))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Fitting-generalized-linear-mixed-models","page":"Model constructors","title":"Fitting generalized linear mixed models","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"To create a GLMM representation, the distribution family for the response, and possibly the link function, must be specified.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"verbagg = MixedModels.dataset(:verbagg)\nverbaggform = @formula(r2 ~ 1 + anger + gender + btype + situ + mode + (1|subj) + (1|item));\ngm1 = fit(MixedModel, verbaggform, verbagg, Bernoulli())\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The canonical link, which is LogitLink for the Bernoulli distribution, is used if no explicit link is specified.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Note that, in keeping with convention in the GLM package, the distribution family for a binary (i.e. 0/1) response is the Bernoulli distribution. The Binomial distribution is only used when the response is the fraction of trials returning a positive, in which case the number of trials must be specified as the case weights.","category":"page"},{"location":"constructors/#Optional-arguments-to-fit","page":"Model constructors","title":"Optional arguments to fit","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"An alternative approach is to create the GeneralizedLinearMixedModel object then call fit! on it. The optional arguments fast and/or nAGQ can be passed to the optimization process via both fit and fit! (i.e these optimization settings are not used nor recognized when constructing the model).","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"As the name implies, fast=true, provides a faster but somewhat less accurate fit. These fits may suffice for model comparisons.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"gm1a = fit(MixedModel, verbaggform, verbagg, Bernoulli(), fast = true)\ndeviance(gm1a) - deviance(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@benchmark fit(MixedModel, $verbaggform, $verbagg, Bernoulli())","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"@benchmark fit(MixedModel, $verbaggform, $verbagg, Bernoulli(), fast = true)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The optional argument nAGQ=k causes evaluation of the deviance function to use a k point adaptive Gauss-Hermite quadrature rule. This method only applies to models with a single, simple, scalar random-effects term, such as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"contraception = MixedModels.dataset(:contra)\ncontraform = @formula(use ~ 1 + age + abs2(age) + livch + urban + (1|dist));\nbernoulli = Bernoulli()\ndeviances = Dict{Symbol,Float64}()\nb = @benchmarkable deviances[:default] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli));\nrun(b)\nb = @benchmarkable deviances[:fast] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli, fast = true));\nrun(b)\nb = @benchmarkable deviances[:nAGQ] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli, nAGQ=9));\nrun(b)\nb = @benchmarkable deviances[:nAGQ_fast] = deviance(fit(MixedModel, $contraform, $contraception, $bernoulli, nAGQ=9, fast=true));\nrun(b)\nsort(deviances)","category":"page"},{"location":"constructors/#Extractor-functions","page":"Model constructors","title":"Extractor functions","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"LinearMixedModel and GeneralizedLinearMixedModel are subtypes of StatsAPI.RegressionModel which, in turn, is a subtype of StatsBase.StatisticalModel. Many of the generic extractors defined in the StatsBase package have methods for these models.","category":"page"},{"location":"constructors/#Model-fit-statistics","page":"Model constructors","title":"Model-fit statistics","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The statistics describing the quality of the model fit include","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"loglikelihood\naic\nbic\ndof\nnobs","category":"page"},{"location":"constructors/#StatsAPI.loglikelihood","page":"Model constructors","title":"StatsAPI.loglikelihood","text":"loglikelihood(model::StatisticalModel)\nloglikelihood(model::StatisticalModel, observation)\n\nReturn the log-likelihood of the model.\n\nWith an observation argument, return the contribution of observation to the log-likelihood of model.\n\nIf observation is a Colon, return a vector of each observation's contribution to the log-likelihood of the model. In other words, this is the vector of the pointwise log-likelihood contributions.\n\nIn general, sum(loglikehood(model, :)) == loglikelihood(model).\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.aic","page":"Model constructors","title":"StatsAPI.aic","text":"aic(model::StatisticalModel)\n\nAkaike's Information Criterion, defined as -2 log L + 2k, with L the likelihood of the model, and k its number of consumed degrees of freedom (as returned by dof).\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.bic","page":"Model constructors","title":"StatsAPI.bic","text":"bic(model::StatisticalModel)\n\nBayesian Information Criterion, defined as -2 log L + k log n, with L the likelihood of the model, k its number of consumed degrees of freedom (as returned by dof), and n the number of observations (as returned by nobs).\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.dof","page":"Model constructors","title":"StatsAPI.dof","text":"dof(model::StatisticalModel)\n\nReturn the number of degrees of freedom consumed in the model, including when applicable the intercept and the distribution's dispersion parameter.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.nobs","page":"Model constructors","title":"StatsAPI.nobs","text":"nobs(model::StatisticalModel)\n\nReturn the number of independent observations on which the model was fitted. Be careful when using this information, as the definition of an independent observation may vary depending on the model, on the format used to pass the data, on the sampling plan (if specified), etc.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"loglikelihood(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"aic(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"bic(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"dof(fm1) # 1 fixed effect, 2 variances","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"nobs(fm1) # 30 observations","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"loglikelihood(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"In general the deviance of a statistical model fit is negative twice the log-likelihood adjusting for the saturated model.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"deviance(::StatisticalModel)","category":"page"},{"location":"constructors/#StatsAPI.deviance-Tuple{StatisticalModel}","page":"Model constructors","title":"StatsAPI.deviance","text":"deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}\n\nReturn the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.\n\nIf the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, u, plus the determinant of ΛZWZΛ + I, plus the sum of the squared deviance residuals.\n\n\n\n\n\n","category":"method"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Because it is not clear what the saturated model corresponding to a particular LinearMixedModel should be, negative twice the log-likelihood is called the objective.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"objective","category":"page"},{"location":"constructors/#MixedModels.objective","page":"Model constructors","title":"MixedModels.objective","text":"objective(m::LinearMixedModel)\n\nReturn negative twice the log-likelihood of model m\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"This value is also accessible as the deviance but the user should bear in mind that this doesn't have all the properties of a deviance which is corrected for the saturated model. For example, it is not necessarily non-negative.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"objective(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"deviance(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The value optimized when fitting a GeneralizedLinearMixedModel is the Laplace approximation to the deviance or an adaptive Gauss-Hermite evaluation.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.deviance!","category":"page"},{"location":"constructors/#MixedModels.deviance!","page":"Model constructors","title":"MixedModels.deviance!","text":"deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)\n\nUpdate m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"MixedModels.deviance!(gm1)","category":"page"},{"location":"constructors/#Fixed-effects-parameter-estimates","page":"Model constructors","title":"Fixed-effects parameter estimates","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The coef and fixef extractors both return the maximum likelihood estimates of the fixed-effects coefficients. They differ in their behavior in the rank-deficient case. The associated coefnames and fixefnames return the corresponding coefficient names.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coef\ncoefnames\nfixef\nfixefnames","category":"page"},{"location":"constructors/#StatsAPI.coef","page":"Model constructors","title":"StatsAPI.coef","text":"coef(model::StatisticalModel)\n\nReturn the coefficients of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#StatsAPI.coefnames","page":"Model constructors","title":"StatsAPI.coefnames","text":"coefnames(model::StatisticalModel)\n\nReturn the names of the coefficients.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#MixedModels.fixef","page":"Model constructors","title":"MixedModels.fixef","text":"fixef(m::MixedModel)\n\nReturn the fixed-effects parameter vector estimate of m.\n\nIn the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.\n\n\n\n\n\n","category":"function"},{"location":"constructors/#MixedModels.fixefnames","page":"Model constructors","title":"MixedModels.fixefnames","text":"fixefnames(m::MixedModel)\n\nReturn a (permuted and truncated in the rank-deficient case) vector of coefficient names.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coef(fm1)\ncoefnames(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fixef(fm1)\nfixefnames(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"An alternative extractor for the fixed-effects coefficient is the β property. Properties whose names are Greek letters usually have an alternative spelling, which is the name of the Greek letter.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1.β","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1.beta","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"gm1.β","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"A full list of property names is returned by propertynames","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"propertynames(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"propertynames(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The variance-covariance matrix of the fixed-effects coefficients is returned by","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"vcov","category":"page"},{"location":"constructors/#StatsAPI.vcov","page":"Model constructors","title":"StatsAPI.vcov","text":"vcov(model::StatisticalModel)\n\nReturn the variance-covariance matrix for the coefficients of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"vcov(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"vcov(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The standard errors are the square roots of the diagonal elements of the estimated variance-covariance matrix of the fixed-effects coefficient estimators.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"stderror","category":"page"},{"location":"constructors/#StatsAPI.stderror","page":"Model constructors","title":"StatsAPI.stderror","text":"stderror(model::StatisticalModel)\n\nReturn the standard errors for the coefficients of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"stderror(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"stderror(gm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Finally, the coeftable generic produces a table of coefficient estimates, their standard errors, and their ratio. The p-values quoted here should be regarded as approximations.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coeftable","category":"page"},{"location":"constructors/#StatsAPI.coeftable","page":"Model constructors","title":"StatsAPI.coeftable","text":"coeftable(model::StatisticalModel; level::Real=0.95)\n\nReturn a table with coefficients and related statistics of the model. level determines the level for confidence intervals (by default, 95%).\n\nThe returned CoefTable object implements the Tables.jl interface, and can be converted e.g. to a DataFrame via using DataFrames; DataFrame(coeftable(model)).\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"coeftable(fm2)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/#Covariance-parameter-estimates","page":"Model constructors","title":"Covariance parameter estimates","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The covariance parameters estimates, in the form shown in the model summary, are a VarCorr object","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"VarCorr(fm2)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"VarCorr(gm1)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Individual components are returned by other extractors","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"varest\nsdest","category":"page"},{"location":"constructors/#MixedModels.varest","page":"Model constructors","title":"MixedModels.varest","text":"varest(m::LinearMixedModel)\n\nReturns the estimate of σ², the variance of the conditional distribution of Y given B.\n\n\n\n\n\nvarest(m::GeneralizedLinearMixedModel)\n\nReturns the estimate of ϕ², the variance of the conditional distribution of Y given B.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ².\n\n\n\n\n\n","category":"function"},{"location":"constructors/#MixedModels.sdest","page":"Model constructors","title":"MixedModels.sdest","text":"sdest(m::LinearMixedModel)\n\nReturn the estimate of σ, the standard deviation of the per-observation noise.\n\n\n\n\n\nsdest(m::GeneralizedLinearMixedModel)\n\nReturn the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"varest(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sdest(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm2.σ","category":"page"},{"location":"constructors/#Conditional-modes-of-the-random-effects","page":"Model constructors","title":"Conditional modes of the random effects","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The ranef extractor","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"ranef","category":"page"},{"location":"constructors/#MixedModels.ranef","page":"Model constructors","title":"MixedModels.ranef","text":"ranef(m::LinearMixedModel; uscale=false)\n\nReturn, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.\n\nIf uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.\n\nFor a named variant, see raneftables.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"ranef(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm1.b","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"returns the conditional modes of the random effects given the observed data. That is, these are the values that maximize the conditional density of the random effects given the observed data. For a LinearMixedModel these are also the conditional means.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"These are sometimes called the best linear unbiased predictors or BLUPs but that name is not particularly meaningful.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"At a superficial level these can be considered as the \"estimates\" of the random effects, with a bit of hand waving, but pursuing this analogy too far usually results in confusion.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"To obtain tables associating the values of the conditional modes with the levels of the grouping factor, use","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"raneftables","category":"page"},{"location":"constructors/#MixedModels.raneftables","page":"Model constructors","title":"MixedModels.raneftables","text":"raneftables(m::MixedModel; uscale = false)\n\nReturn the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.\n\nnote: Note\nThe API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"as in","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"DataFrame(only(raneftables(fm1)))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The corresponding conditional variances are returned by","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"condVar","category":"page"},{"location":"constructors/#MixedModels.condVar","page":"Model constructors","title":"MixedModels.condVar","text":"condVar(m::LinearMixedModel)\n\nReturn the conditional variances matrices of the random effects.\n\nThe random effects are returned by ranef as a vector of length k, where k is the number of random effects terms. The ith element is a matrix of size vᵢ × ℓᵢ where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor. Technically those values are the modes of the conditional distribution of the random effects given the observed data.\n\nThis function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ. These are the diagonal blocks from the conditional variance-covariance matrix,\n\ns² Λ(Λ'Z'ZΛ + I)⁻¹Λ'\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"condVar(fm1)","category":"page"},{"location":"constructors/#Case-wise-diagnostics-and-residual-degrees-of-freedom","page":"Model constructors","title":"Case-wise diagnostics and residual degrees of freedom","text":"","category":"section"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The leverage values","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"leverage","category":"page"},{"location":"constructors/#StatsAPI.leverage","page":"Model constructors","title":"StatsAPI.leverage","text":"leverage(model::RegressionModel)\n\nReturn the diagonal of the projection matrix of the model.\n\n\n\n\n\n","category":"function"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"leverage(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"are used in diagnostics for linear regression models to determine cases that exert a strong influence on their own predicted response.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"The documentation refers to a \"projection\". For a linear model without random effects the fitted values are obtained by orthogonal projection of the response onto the column span of the model matrix and the sum of the leverage values is the dimension of this column span. That is, the sum of the leverage values is the rank of the model matrix and n - sum(leverage(m)) is the degrees of freedom for residuals. The sum of the leverage values is also the trace of the so-called \"hat\" matrix, H. (The name \"hat matrix\" reflects the fact that hatmathbfy = mathbfH mathbfy. That is, H puts a hat on y.)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For a linear mixed model the sum of the leverage values will be between p, the rank of the fixed-effects model matrix, and p + q where q is the total number of random effects. This number does not represent a dimension (or \"degrees of freedom\") of a linear subspace of all possible fitted values because the projection is not an orthogonal projection. Nevertheless, it is a reasonable measure of the effective degrees of freedom of the model and n - sum(leverage(m)) can be considered the effective residual degrees of freedom.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For model fm1 the dimensions are","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"n, p, q, k = size(fm1)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"which implies that the sum of the leverage values should be in the range [1, 7]. The actual value is","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm1))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"For model fm2 the dimensions are","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"n, p, q, k = size(fm2)","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"providing a range of [2, 38] for the effective degrees of freedom for the model. The observed value is","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm2))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"When a model converges to a singular covariance, such as","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm3 = fit(MixedModel, @formula(yield ~ 1+(1|batch)), MixedModels.dataset(:dyestuff2))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"the effective degrees of freedom is the lower bound.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm3))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Models for which the estimates of the variances of the random effects are large relative to the residual variance have effective degrees of freedom close to the upper bound.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm4 = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),\n MixedModels.dataset(:penicillin))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm4))","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"Also, a model fit by the REML criterion generally has larger estimates of the variance components and hence a larger effective degrees of freedom.","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"fm4r = fit(MixedModel, @formula(diameter ~ 1+(1|plate)+(1|sample)),\n MixedModels.dataset(:penicillin), REML=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"constructors/","page":"Model constructors","title":"Model constructors","text":"sum(leverage(fm4r))","category":"page"},{"location":"benchmarks/#Benchmark-Report-for-*/home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl*","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"","category":"section"},{"location":"benchmarks/#Job-Properties","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Job Properties","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Time of benchmark: 2 Oct 2018 - 13:42\nPackage commit: non gi\nJulia commit: 5d4eac\nJulia command flags: None\nEnvironment variables: None","category":"page"},{"location":"benchmarks/#Results","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Results","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Below is a table of this job's results, obtained by running the benchmarks. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The \"true\" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero.","category":"page"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"ID time GC time memory allocations\n`[\"crossed\", \"Assay:1+A+B*C+(1 G)+(1 H)\"]` 2.943 ms (5%) \n`[\"crossed\", \"Demand:1+U+V+W+X+(1 G)+(1 H)\"]` 2.775 ms (5%) \n`[\"crossed\", \"InstEval:1+A*I+(1 G)+(1 H)\"]` 1.247 s (5%) 114.131 ms\n`[\"crossed\", \"InstEval:1+A+(1 G)+(1 H)+(1 I)\"]` 1.999 s (5%)\n`[\"crossed\", \"Penicillin:1+(1 G)+(1 H)\"]` 2.697 ms (5%) \n`[\"crossed\", \"ScotsSec:1+A+U+V+(1 G)+(1 H)\"]` 4.833 ms (5%) \n`[\"crossed\", \"dialectNL:1+A+T+U+V+W+X+(1 G)+(1 H)+(1 I)\"]` 416.892 ms (5%)\n`[\"crossed\", \"egsingle:1+A+U+V+(1 G)+(1 H)\"]` 31.421 ms (5%) 3.427 ms\n`[\"crossed\", \"ml1m:1+(1 G)+(1 H)\"]` 36.714 s (5%) 225.872 ms\n`[\"crossed\", \"paulsim:1+S+T+U+(1 H)+(1 G)\"]` 14.097 ms (5%) \n`[\"crossedvector\", \"bs10:1+U+V+W+((1+U+V+W) G)+((1+U+V+W) H)\"]` 165.171 ms (5%) 3.149 ms\n`[\"crossedvector\", \"d3:1+U+((1+U) G)+((1+U) H)+((1+U) I)\"]` 49.023 s (5%)\n`[\"crossedvector\", \"d3:1+U+(1 G)+(1 H)+(1 I)\"]` 299.348 ms (5%)\n`[\"crossedvector\", \"gb12:1+S+T+U+V+W+X+Z+((1+S+U+W) G)+((1+S+T+V) H)\"]` 134.101 ms (5%) \n`[\"crossedvector\", \"kb07:1+S+T+U+V+W+X+Z+((1+S+T+U+V+W+X+Z) G)+((1+S+T+U+V+W+X+Z) H)\"]` 3.488 s (5%) 16.508 ms\n`[\"crossedvector\", \"kb07:1+S+T+U+V+W+X+Z+(1 G)+((0+S) G)+((0+T) G)+((0+U) G)+((0+V)\n`[\"nested\", \"Animal:1+(1 G)+(1 H)\"]` 1.261 ms (5%) \n`[\"nested\", \"Chem97:1+(1 G)+(1 H)\"]` 58.460 ms (5%) 6.975 ms\n`[\"nested\", \"Chem97:1+U+(1 G)+(1 H)\"]` 59.353 ms (5%) 7.019 ms\n`[\"nested\", \"Genetics:1+A+(1 G)+(1 H)\"]` 2.062 ms (5%) \n`[\"nested\", \"Pastes:1+(1 G)+(1 H)\"]` 2.298 ms (5%) \n`[\"nested\", \"Semi2:1+A+(1 G)+(1 H)\"]` 2.309 ms (5%) \n`[\"simplescalar\", \"Alfalfa:1+A*B+(1 G)\"]` 1.210 ms (5%) 208.80 KiB (1%)\n`[\"simplescalar\", \"Alfalfa:1+A+B+(1 G)\"]` 1.021 ms (5%) 168.47 KiB (1%)\n`[\"simplescalar\", \"AvgDailyGain:1+A*U+(1 G)\"]` 1.287 ms (5%) 193.33 KiB (1%)\n`[\"simplescalar\", \"AvgDailyGain:1+A+U+(1 G)\"]` 1.144 ms (5%) 169.59 KiB (1%)\n`[\"simplescalar\", \"BIB:1+A*U+(1 G)\"]` 1.574 ms (5%) 222.20 KiB (1%)\n`[\"simplescalar\", \"BIB:1+A+U+(1 G)\"]` 1.171 ms (5%) 171.31 KiB (1%)\n`[\"simplescalar\", \"Bond:1+A+(1 G)\"]` 958.770 μs (5%) 141.25 KiB (1%)\n`[\"simplescalar\", \"Cultivation:1+A*B+(1 G)\"]` 1.089 ms (5%) 173.38 KiB (1%)\n`[\"simplescalar\", \"Cultivation:1+A+(1 G)\"]` 1.138 ms (5%) 162.14 KiB (1%)\n`[\"simplescalar\", \"Cultivation:1+A+B+(1 G)\"]` 1.147 ms (5%) 173.47 KiB (1%)\n`[\"simplescalar\", \"Dyestuff2:1+(1 G)\"]` 830.840 μs (5%) 105.20 KiB (1%)\n`[\"simplescalar\", \"Dyestuff:1+(1 G)\"]` 974.091 μs (5%) 120.86 KiB (1%)\n`[\"simplescalar\", \"Exam:1+A*U+B+(1 G)\"]` 2.250 ms (5%) 1.17 MiB (1%)\n`[\"simplescalar\", \"Exam:1+A+B+U+(1 G)\"]` 2.133 ms (5%) 1.03 MiB (1%)\n`[\"simplescalar\", \"Gasoline:1+U+(1 G)\"]` 1.164 ms (5%) 162.03 KiB (1%)\n`[\"simplescalar\", \"Hsb82:1+A+B+C+U+(1 G)\"]` 3.048 ms (5%) 2.12 MiB (1%)\n`[\"simplescalar\", \"IncBlk:1+A+U+V+W+Z+(1 G)\"]` 1.226 ms (5%) 208.83 KiB (1%)\n`[\"simplescalar\", \"Mississippi:1+A+(1 G)\"]` 980.968 μs (5%) 145.75 KiB (1%)\n`[\"simplescalar\", \"PBIB:1+A+(1 G)\"]` 1.509 ms (5%) 234.47 KiB (1%)\n`[\"simplescalar\", \"Rail:1+(1 G)\"]` 1.251 ms (5%) 151.34 KiB (1%)\n`[\"simplescalar\", \"Semiconductor:1+A*B+(1 G)\"]` 1.313 ms (5%) 222.95 KiB (1%)\n`[\"simplescalar\", \"TeachingII:1+A+T+U+V+W+X+Z+(1 G)\"]` 1.483 ms (5%) 284.53 KiB (1%)\n`[\"simplescalar\", \"cake:1+A*B+(1 G)\"]` 1.606 ms (5%) 412.83 KiB (1%)\n`[\"simplescalar\", \"ergoStool:1+A+(1 G)\"]` 1.057 ms (5%) 155.59 KiB (1%)\n`[\"singlevector\", \"Early:1+U+U&A+((1+U) G)\"]` 20.373 ms (5%) 3.47 MiB (1%)\n`[\"singlevector\", \"HR:1+A*U+V+((1+U) G)\"]` 5.183 ms (5%) 915.00 KiB (1%)\n`[\"singlevector\", \"Oxboys:1+U+((1+U) G)\"]` 13.207 ms (5%) 1.93 MiB (1%)\n`[\"singlevector\", \"SIMS:1+U+((1+U) G)\"]` 61.675 ms (5%) 12.86 MiB (1%)\n`[\"singlevector\", \"WWheat:1+U+((1+U) G)\"]` 7.311 ms (5%) 902.31 KiB (1%)\n`[\"singlevector\", \"Weights:1+A*U+((1+U) G)\"]` 18.303 ms (5%) 3.20 MiB (1%)\n`[\"singlevector\", \"sleepstudy:1+U+((1+U) G)\"]` 4.829 ms (5%) 797.48 KiB (1%)\n`[\"singlevector\", \"sleepstudy:1+U+(1 G)+((0+U) G)\"]` 3.219 ms (5%) ","category":"page"},{"location":"benchmarks/#Benchmark-Group-List","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Group List","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Here's a list of all the benchmark groups executed by this job:","category":"page"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"[\"crossed\"]\n[\"crossedvector\"]\n[\"nested\"]\n[\"simplescalar\"]\n[\"singlevector\"]","category":"page"},{"location":"benchmarks/#Julia-versioninfo","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Julia versioninfo","text":"","category":"section"},{"location":"benchmarks/","page":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","title":"Benchmark Report for /home/bates/.julia/packages/MixedModels/dn0WY/src/MixedModels.jl","text":"Julia Version 1.0.0\nCommit 5d4eaca0c9 (2018-08-08 20:58 UTC)\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n Ubuntu 18.04.1 LTS\n uname: Linux 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64\n CPU: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz: \n speed user nice sys idle irq\n #1 1690 MHz 140498 s 134 s 18382 s 1495130 s 0 s\n #2 2513 MHz 131505 s 16 s 18277 s 1504212 s 0 s\n #3 1900 MHz 145131 s 581 s 18892 s 1485409 s 0 s\n #4 1682 MHz 190751 s 38 s 17941 s 1445446 s 0 s\n \n Memory: 15.554645538330078 GB (10502.1171875 MB free)\n Uptime: 16578.0 sec\n Load Avg: 1.4091796875 2.07080078125 1.63037109375\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge)","category":"page"},{"location":"api/#API","page":"API","title":"API","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"In addition to its own functionality, MixedModels.jl also implements extensive support for the StatsAPI.StatisticalModel and StatsAPI.RegressionModel API.","category":"page"},{"location":"api/#Types","page":"API","title":"Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Modules = [MixedModels]\nOrder = [:type]","category":"page"},{"location":"api/#MixedModels.BlockDescription","page":"API","title":"MixedModels.BlockDescription","text":"BlockDescription\n\nDescription of blocks of A and L in a LinearMixedModel\n\nFields\n\nblknms: Vector{String} of block names\nblkrows: Vector{Int} of the number of rows in each block\nALtypes: Matrix{String} of datatypes for blocks in A and L.\n\nWhen a block in L is the same type as the corresponding block in A, it is described with a single name, such as Dense. When the types differ the entry in ALtypes is of the form Diag/Dense, as determined by a shorttype method.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.BlockedSparse","page":"API","title":"MixedModels.BlockedSparse","text":"BlockedSparse{Tv,S,P}\n\nA SparseMatrixCSC whose nonzeros form blocks of rows or columns or both.\n\nMembers\n\ncscmat: SparseMatrixCSC{Tv, Int32} representation for general calculations\nnzasmat: nonzeros of cscmat as a dense matrix\ncolblkptr: pattern of blocks of columns\n\nThe only time these are created are as products of ReMats.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.FeMat","page":"API","title":"MixedModels.FeMat","text":"FeMat{T,S}\n\nA matrix and a (possibly) weighted copy of itself.\n\nTypically, an FeMat represents the fixed-effects model matrix with the response (y) concatenated as a final column.\n\nnote: Note\nFeMat is not the same as FeTerm.\n\nFields\n\nxy: original matrix, called xy b/c in practice this is hcat(fullrank(X), y)\nwtxy: (possibly) weighted copy of xy (shares storage with xy until weights are applied)\n\nUpon construction the xy and wtxy fields refer to the same matrix\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.FeTerm","page":"API","title":"MixedModels.FeTerm","text":"FeTerm{T,S}\n\nTerm with an explicit, constant matrix representation\n\nTypically, an FeTerm represents the model matrix for the fixed effects.\n\nnote: Note\nFeTerm is not the same as FeMat!\n\nFields\n\nx: full model matrix\npiv: pivot Vector{Int} for moving linearly dependent columns to the right\nrank: computational rank of x\ncnames: vector of column names\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.FeTerm-Tuple{SparseArrays.SparseMatrixCSC, AbstractVector{String}}","page":"API","title":"MixedModels.FeTerm","text":"FeTerm(X::SparseMatrixCSC, cnms)\n\nConvenience constructor for a sparse FeTerm assuming full rank, identity pivot and unit weights.\n\nNote: automatic rank deficiency handling may be added to this method in the future, as discussed in the vignette \"Rank deficiency in mixed-effects models\" for general FeTerm.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.FeTerm-Union{Tuple{T}, Tuple{AbstractMatrix{T}, Any}} where T","page":"API","title":"MixedModels.FeTerm","text":"FeTerm(X::AbstractMatrix, cnms)\n\nConvenience constructor for FeTerm that computes the rank and pivot with unit weights.\n\nSee the vignette \"Rank deficiency in mixed-effects models\" for more information on the computation of the rank and pivot.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.GaussHermiteNormalized","page":"API","title":"MixedModels.GaussHermiteNormalized","text":"GaussHermiteNormalized{K}\n\nA struct with 2 SVector{K,Float64} members\n\nz: abscissae for the K-point Gauss-Hermite quadrature rule on the Z scale\nwt: Gauss-Hermite weights normalized to sum to unity\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.GeneralizedLinearMixedModel","page":"API","title":"MixedModels.GeneralizedLinearMixedModel","text":"GeneralizedLinearMixedModel\n\nGeneralized linear mixed-effects model representation\n\nFields\n\nLMM: a LinearMixedModel - the local approximation to the GLMM.\nβ: the pivoted and possibly truncated fixed-effects vector\nβ₀: similar to β. Used in the PIRLS algorithm if step-halving is needed.\nθ: covariance parameter vector\nb: similar to u, equivalent to broadcast!(*, b, LMM.Λ, u)\nu: a vector of matrices of random effects\nu₀: similar to u. Used in the PIRLS algorithm if step-halving is needed.\nresp: a GlmResp object\nη: the linear predictor\nwt: vector of prior case weights, a value of T[] indicates equal weights.\n\nThe following fields are used in adaptive Gauss-Hermite quadrature, which applies only to models with a single random-effects term, in which case their lengths are the number of levels in the grouping factor for that term. Otherwise they are zero-length vectors.\n\ndevc: vector of deviance components\ndevc0: vector of deviance components at offset of zero\nsd: approximate standard deviation of the conditional density\nmult: multiplier\n\nProperties\n\nIn addition to the fieldnames, the following names are also accessible through the . extractor\n\ntheta: synonym for θ\nbeta: synonym for β\nσ or sigma: common scale parameter (value is NaN for distributions without a scale parameter)\nlowerbd: vector of lower bounds on the combined elements of β and θ\nformula, trms, A, L, and optsum: fields of the LMM field\nX: fixed-effects model matrix\ny: response vector\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.Grouping","page":"API","title":"MixedModels.Grouping","text":"struct Grouping <: StatsModels.AbstractContrasts end\n\nA placeholder type to indicate that a categorical variable is only used for grouping and not for contrasts. When creating a CategoricalTerm, this skips constructing the contrasts matrix which makes it robust to large numbers of levels, while still holding onto the vector of levels and constructing the level-to-index mapping (invindex field of the ContrastsMatrix.).\n\nNote that calling modelcols on a CategoricalTerm{Grouping} is an error.\n\nExamples\n\njulia> schema((; grp = string.(1:100_000)))\n# out-of-memory error\n\njulia> schema((; grp = string.(1:100_000)), Dict(:grp => Grouping()))\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LikelihoodRatioTest","page":"API","title":"MixedModels.LikelihoodRatioTest","text":"LikelihoodRatioTest\n\nResults of MixedModels.likelihoodratiotest\n\nFields\n\nformulas: Vector of model formulae\nmodels: NamedTuple of the dof and deviance of the models\ntests: NamedTuple of the sequential dofdiff, deviancediff, and resulting pvalues\n\nProperties\n\ndeviance : note that this is actually -2 log likelihood for linear models (i.e. without subtracting the constant for a saturated model)\npvalues\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LinearMixedModel","page":"API","title":"MixedModels.LinearMixedModel","text":"LinearMixedModel(y, Xs, form, wts=[], σ=nothing, amalgamate=true)\n\nPrivate constructor for a LinearMixedModel.\n\nTo construct a model, you only need the response (y), already assembled model matrices (Xs), schematized formula (form) and weights (wts). Everything else in the structure can be derived from these quantities.\n\nnote: Note\nThis method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LinearMixedModel-2","page":"API","title":"MixedModels.LinearMixedModel","text":"LinearMixedModel\n\nLinear mixed-effects model representation\n\nFields\n\nformula: the formula for the model\nreterms: a Vector{AbstractReMat{T}} of random-effects terms.\nXymat: horizontal concatenation of a full-rank fixed-effects model matrix X and response y as an FeMat{T}\nfeterm: the fixed-effects model matrix as an FeTerm{T}\nsqrtwts: vector of square roots of the case weights. Can be empty.\nparmap : Vector{NTuple{3,Int}} of (block, row, column) mapping of θ to λ\ndims : NamedTuple{(:n, :p, :nretrms),NTuple{3,Int}} of dimensions. p is the rank of X, which may be smaller than size(X, 2).\nA: a Vector{AbstractMatrix} containing the row-major packed lower triangle of hcat(Z,X,y)'hcat(Z,X,y)\nL: the blocked lower Cholesky factor of Λ'AΛ+I in the same Vector representation as A\noptsum: an OptSummary object\n\nProperties\n\nθ or theta: the covariance parameter vector used to form λ\nβ or beta: the fixed-effects coefficient vector\nλ or lambda: a vector of lower triangular matrices repeated on the diagonal blocks of Λ\nσ or sigma: current value of the standard deviation of the per-observation noise\nb: random effects on the original scale, as a vector of matrices\nu: random effects on the orthogonal scale, as a vector of matrices\nlowerbd: lower bounds on the elements of θ\nX: the fixed-effects model matrix\ny: the response vector\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.LinearMixedModel-Union{Tuple{T}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm, Any}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm, Any, Any}, Tuple{AbstractArray, MixedModels.FeTerm{T}, AbstractVector{<:AbstractReMat{T}}, StatsModels.FormulaTerm, Any, Any, Any}} where T","page":"API","title":"MixedModels.LinearMixedModel","text":"LinearMixedModel(y, feterm, reterms, form, wts=[], σ=nothing; amalgamate=true)\n\nPrivate constructor for a LinearMixedModel given already assembled fixed and random effects.\n\nTo construct a model, you only need a vector of FeMats (the fixed-effects model matrix and response), a vector of AbstractReMat (the random-effects model matrices), the formula and the weights. Everything else in the structure can be derived from these quantities.\n\nnote: Note\nThis method is internal and experimental and so may change or disappear in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.MixedModel","page":"API","title":"MixedModels.MixedModel","text":"MixedModel\n\nAbstract type for mixed models. MixedModels.jl implements two subtypes: LinearMixedModel and GeneralizedLinearMixedModel. See the documentation for each for more details.\n\nThis type is primarily used for dispatch in fit. Without a distribution and link function specified, a LinearMixedModel will be fit. When a distribution/link function is provided, a GeneralizedLinearModel is fit, unless that distribution is Normal and the link is IdentityLink, in which case the resulting GLMM would be equivalent to a LinearMixedModel anyway and so the simpler, equivalent LinearMixedModel will be fit instead.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.MixedModelBootstrap","page":"API","title":"MixedModels.MixedModelBootstrap","text":"MixedModelBootstrap{T<:AbstractFloat} <: MixedModelFitCollection{T}\n\nObject returned by parametericbootstrap with fields\n\nfits: the parameter estimates from the bootstrap replicates as a vector of named tuples.\nλ: Vector{LowerTriangular{T,Matrix{T}}} containing copies of the λ field from ReMat model terms\ninds: Vector{Vector{Int}} containing copies of the inds field from ReMat model terms\nlowerbd: Vector{T} containing the vector of lower bounds (corresponds to the identically named field of OptSummary)\nfcnames: NamedTuple whose keys are the grouping factor names and whose values are the column names\n\nThe schema of fits is, by default,\n\nTables.Schema:\n :objective T\n :σ T\n :β NamedTuple{β_names}{NTuple{p,T}}\n :se StaticArrays.SArray{Tuple{p},T,1,p}\n :θ StaticArrays.SArray{Tuple{k},T,1,k}\n\nwhere the sizes, p and k, of the β and θ elements are determined by the model.\n\nCharacteristics of the bootstrap replicates can be extracted as properties. The σs and σρs properties unravel the σ and θ estimates into estimates of the standard deviations and correlations of the random-effects terms.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.MixedModelFitCollection","page":"API","title":"MixedModels.MixedModelFitCollection","text":"MixedModelFitCollection{T<:AbstractFloat}\n\nAbstract supertype for MixedModelBootstrap and related functionality in other packages.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.MixedModelProfile","page":"API","title":"MixedModels.MixedModelProfile","text":" MixedModelProfile{T<:AbstractFloat}\n\nType representing a likelihood profile of a LinearMixedModel, including associated interpolation splines.\n\nThe function profile is used for computing profiles, while confint provides a useful method for constructing confidence intervals from a MixedModelProfile.\n\nnote: Note\nThe exact fields and their representation are considered implementation details and are not part of the public API.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.OptSummary","page":"API","title":"MixedModels.OptSummary","text":"OptSummary\n\nSummary of an NLopt optimization\n\nFields\n\ninitial: a copy of the initial parameter values in the optimization\nfinitial: the initial value of the objective\nlowerbd: lower bounds on the parameter values\nftol_rel: as in NLopt\nftol_abs: as in NLopt\nxtol_rel: as in NLopt\nxtol_abs: as in NLopt\ninitial_step: as in NLopt\nmaxfeval: as in NLopt (maxeval)\nmaxtime: as in NLopt\nfinal: a copy of the final parameter values from the optimization\nfmin: the final value of the objective\nfeval: the number of function evaluations\noptimizer: the name of the optimizer used, as a Symbol\nreturnvalue: the return value, as a Symbol\nxtol_zero_abs: the tolerance for a near zero parameter to be considered practically zero\nftol_zero_abs: the tolerance for change in the objective for setting a near zero parameter to zero\nfitlog: A vector of tuples of parameter and objectives values from steps in the optimization\nnAGQ: number of adaptive Gauss-Hermite quadrature points in deviance evaluation for GLMMs\nREML: use the REML criterion for LMM fits\nsigma: a priori value for the residual standard deviation for LMM\n\nThe last three fields are MixedModels functionality and not related directly to the NLopt package or algorithms.\n\nnote: Note\nThe internal storage of the parameter values within fitlog may change in the future to use a different subtype of AbstractVector (e.g., StaticArrays.SVector) for each snapshot without being considered a breaking change.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.PCA","page":"API","title":"MixedModels.PCA","text":"PCA{T<:AbstractFloat}\n\nPrincipal Components Analysis\n\nFields\n\ncovcorr covariance or correlation matrix\nsv singular value decomposition\nrnames rownames of the original matrix\ncorr is this a correlation matrix?\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.RaggedArray","page":"API","title":"MixedModels.RaggedArray","text":"RaggedArray{T,I}\n\nA \"ragged\" array structure consisting of values and indices\n\nFields\n\nvals: a Vector{T} containing the values\ninds: a Vector{I} containing the indices\n\nFor this application a RaggedArray is used only in its sum! method.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.ReMat","page":"API","title":"MixedModels.ReMat","text":"ReMat{T,S} <: AbstractMatrix{T}\n\nA section of a model matrix generated by a random-effects term.\n\nFields\n\ntrm: the grouping factor as a StatsModels.CategoricalTerm\nrefs: indices into the levels of the grouping factor as a Vector{Int32}\nlevels: the levels of the grouping factor\ncnames: the names of the columns of the model matrix generated by the left-hand side of the term\nz: transpose of the model matrix generated by the left-hand side of the term\nwtz: a weighted copy of z (z and wtz are the same object for unweighted cases)\nλ: a LowerTriangular or Diagonal matrix of size S×S\ninds: a Vector{Int} of linear indices of the potential nonzeros in λ\nadjA: the adjoint of the matrix as a SparseMatrixCSC{T}\nscratch: a Matrix{T}\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.TableColumns","page":"API","title":"MixedModels.TableColumns","text":"TableColumns\n\nA structure containing the column names for the numeric part of the profile table.\n\nThe struct also contains a Dict giving the column ranges for Symbols like :σ and :β. Finally it contains a scratch vector used to accumulate to values in a row of the profile table.\n\nnote: Note\nThis is an internal structure used in MixedModelProfile. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.UniformBlockDiagonal","page":"API","title":"MixedModels.UniformBlockDiagonal","text":"UniformBlockDiagonal{T}\n\nHomogeneous block diagonal matrices. k diagonal blocks each of size m×m\n\n\n\n\n\n","category":"type"},{"location":"api/#MixedModels.VarCorr","page":"API","title":"MixedModels.VarCorr","text":"VarCorr\n\nInformation from the fitted random-effects variance-covariance matrices.\n\nMembers\n\nσρ: a NamedTuple of NamedTuples as returned from σρs\ns: the estimate of the per-observation dispersion parameter\n\nThe main purpose of defining this type is to isolate the logic in the show method.\n\n\n\n\n\n","category":"type"},{"location":"api/#Exported-Functions","page":"API","title":"Exported Functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Modules = [MixedModels]\nPrivate = false\nOrder = [:function]","category":"page"},{"location":"api/#LinearAlgebra.cond-Tuple{MixedModel}","page":"API","title":"LinearAlgebra.cond","text":"cond(m::MixedModel)\n\nReturn a vector of condition numbers of the λ matrices for the random-effects terms\n\n\n\n\n\n","category":"method"},{"location":"api/#LinearAlgebra.logdet-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"LinearAlgebra.logdet","text":"logdet(m::LinearMixedModel)\n\nReturn the value of log(det(Λ'Z'ZΛ + I)) + m.optsum.REML * log(det(LX*LX')) evaluated in place.\n\nHere LX is the diagonal term corresponding to the fixed-effects in the blocked lower Cholesky factor.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.GHnorm-Tuple{Int64}","page":"API","title":"MixedModels.GHnorm","text":"GHnorm(k::Int)\n\nReturn the (unique) GaussHermiteNormalized{k} object.\n\nThe function values are stored (memoized) when first evaluated. Subsequent evaluations for the same k have very low overhead.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.coefpvalues-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.coefpvalues","text":"coefpvalues(bsamp::MixedModelFitCollection)\n\nReturn a rowtable with columns (:iter, :coefname, :β, :se, :z, :p)\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.condVar-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.condVar","text":"condVar(m::LinearMixedModel)\n\nReturn the conditional variances matrices of the random effects.\n\nThe random effects are returned by ranef as a vector of length k, where k is the number of random effects terms. The ith element is a matrix of size vᵢ × ℓᵢ where vᵢ is the size of the vector-valued random effects for each of the ℓᵢ levels of the grouping factor. Technically those values are the modes of the conditional distribution of the random effects given the observed data.\n\nThis function returns an array of k three dimensional arrays, where the ith array is of size vᵢ × vᵢ × ℓᵢ. These are the diagonal blocks from the conditional variance-covariance matrix,\n\ns² Λ(Λ'Z'ZΛ + I)⁻¹Λ'\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.condVartables-Union{Tuple{MixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.condVartables","text":"condVartables(m::LinearMixedModel)\n\nReturn the conditional covariance matrices of the random effects as a NamedTuple of columntables\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fitted!-Union{Tuple{T}, Tuple{AbstractArray{T}, LinearMixedModel{T}}} where T","page":"API","title":"MixedModels.fitted!","text":"fitted!(v::AbstractArray{T}, m::LinearMixedModel{T})\n\nOverwrite v with the fitted values from m.\n\nSee also fitted.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fixef-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.fixef","text":"fixef(m::MixedModel)\n\nReturn the fixed-effects parameter vector estimate of m.\n\nIn the rank-deficient case the truncated parameter vector, of length rank(m) is returned. This is unlike coef which always returns a vector whose length matches the number of columns in X.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fixefnames-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.fixefnames","text":"fixefnames(m::MixedModel)\n\nReturn a (permuted and truncated in the rank-deficient case) vector of coefficient names.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fnames-Tuple{MixedModel}","page":"API","title":"MixedModels.fnames","text":"fnames(m::MixedModel)\n\nReturn the names of the grouping factors for the random-effects terms.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fulldummy-Tuple{StatsModels.CategoricalTerm}","page":"API","title":"MixedModels.fulldummy","text":"fulldummy(term::CategoricalTerm)\n\nAssign \"contrasts\" that include all indicator columns (dummy variables) and an intercept column.\n\nThis will result in an under-determined set of contrasts, which is not a problem in the random effects because of the regularization, or \"shrinkage\", of the conditional modes.\n\nThe interaction of fulldummy with complex random effects is subtle and complex with numerous potential edge cases. As we discover these edge cases, we will document and determine their behavior. Until such time, please check the model summary to verify that the expansion is working as you expected. If it is not, please report a use case on GitHub.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.issingular","page":"API","title":"MixedModels.issingular","text":"issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\n\nTest whether the model m is singular if the parameter vector is θ.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nnote: Note\nFor GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.issingular-Tuple{MixedModels.MixedModelFitCollection}","page":"API","title":"MixedModels.issingular","text":"issingular(bsamp::MixedModelFitCollection;\n atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))\n\nTest each bootstrap sample for singularity of the corresponding fit.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nSee also issingular(::MixedModel).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.lowerbd-Union{Tuple{ReMat{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.lowerbd","text":"lowerbd{T}(A::ReMat{T})\n\nReturn the vector of lower bounds on the parameters, θ associated with A\n\nThese are the elements in the lower triangle of A.λ in column-major ordering. Diagonals have a lower bound of 0. Off-diagonals have a lower-bound of -Inf.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.objective!","page":"API","title":"MixedModels.objective!","text":"objective!(m::LinearMixedModel, θ)\nobjective!(m::LinearMixedModel)\n\nEquivalent to objective(updateL!(setθ!(m, θ))).\n\nWhen m has a single, scalar random-effects term, θ can be a scalar.\n\nThe one-argument method curries and returns a single-argument function of θ.\n\nNote that these methods modify m. The calling function is responsible for restoring the optimal θ.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.objective-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.objective","text":"objective(m::LinearMixedModel)\n\nReturn negative twice the log-likelihood of model m\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.parametricbootstrap-Union{Tuple{T}, Tuple{Random.AbstractRNG, Integer, MixedModel{T}}, Tuple{Random.AbstractRNG, Integer, MixedModel{T}, Type{<:AbstractFloat}}} where T","page":"API","title":"MixedModels.parametricbootstrap","text":"parametricbootstrap([rng::AbstractRNG], nsamp::Integer, m::MixedModel{T}, ftype=T;\n β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))\n\nPerform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.\n\nThe default random number generator is Random.GLOBAL_RNG.\n\nftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.\n\nKeyword Arguments\n\nβ, σ, and θ are the values of m's parameters for simulating the responses.\nσ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for\n\nfamilies with a dispersion parameter.\n\nprogress controls whether the progress bar is shown. Note that the progress\n\nbar is automatically disabled for non-interactive (i.e. logging) contexts.\n\noptsum_overrides is used to override values of OptSummary in the models\n\nfit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.\n\nnote: Note\nAll coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.pirls!-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}, Tuple{GeneralizedLinearMixedModel{T}, Any}, Tuple{GeneralizedLinearMixedModel{T}, Any, Any}} where T","page":"API","title":"MixedModels.pirls!","text":"pirls!(m::GeneralizedLinearMixedModel)\n\nUse Penalized Iteratively Reweighted Least Squares (PIRLS) to determine the conditional modes of the random effects.\n\nWhen varyβ is true both u and β are optimized with PIRLS. Otherwise only u is optimized and β is held fixed.\n\nPassing verbose = true provides verbose output of the iterations.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.profile-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.profile","text":"profile(m::LinearMixedModel; threshold = 4)\n\nReturn a MixedModelProfile for the objective of m with respect to the fixed-effects coefficients.\n\nm is refit! if !isfitted(m).\n\nProfiling starts at the parameter estimate and continues until reaching a parameter bound or the absolute value of ζ exceeds threshold.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.profilevc-Union{Tuple{T}, Tuple{LinearMixedModel{T}, T, AbstractVector{T}}} where T","page":"API","title":"MixedModels.profilevc","text":" profilevc(m::LinearMixedModel{T}, val::T, rowj::AbstractVector{T}) where {T}\n\nProfile an element of the variance components.\n\nnote: Note\nThis method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.profileσ-Union{Tuple{T}, Tuple{LinearMixedModel{T}, MixedModels.TableColumns{T}}} where T","page":"API","title":"MixedModels.profileσ","text":"profileσ(m::LinearMixedModel, tc::TableColumns; threshold=4)\n\nReturn a Table of the profile of σ for model m. The profile extends to where the magnitude of ζ exceeds threshold.\n\nnote: Note\nThis method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.pwrss-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.pwrss","text":"pwrss(m::LinearMixedModel)\n\nThe penalized, weighted residual sum-of-squares.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.ranef-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.ranef","text":"ranef(m::LinearMixedModel; uscale=false)\n\nReturn, as a Vector{Matrix{T}}, the conditional modes of the random effects in model m.\n\nIf uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale.\n\nFor a named variant, see raneftables.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.raneftables-Union{Tuple{MixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.raneftables","text":"raneftables(m::MixedModel; uscale = false)\n\nReturn the conditional means of the random effects as a NamedTuple of Tables.jl-compliant tables.\n\nnote: Note\nThe API guarantee is only that the NamedTuple contains Tables.jl tables and not on the particular concrete type of each table.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.refit!-Tuple{GeneralizedLinearMixedModel}","page":"API","title":"MixedModels.refit!","text":"refit!(m::GeneralizedLinearMixedModel[, y::Vector];\n fast::Bool = (length(m.θ) == length(m.optsum.final)),\n nAGQ::Integer = m.optsum.nAGQ,\n kwargs...)\n\nRefit the model m after installing response y.\n\nIf y is omitted the current response vector is used.\n\nIf not specified, the fast and nAGQ options from the previous fit are used. kwargs are the same as fit!\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.refit!-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.refit!","text":"refit!(m::LinearMixedModel[, y::Vector]; REML=m.optsum.REML, kwargs...)\n\nRefit the model m after installing response y.\n\nIf y is omitted the current response vector is used. kwargs are the same as fit!.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.replicate-Tuple{Function, Integer}","page":"API","title":"MixedModels.replicate","text":"replicate(f::Function, n::Integer; progress=true)\n\nReturn a vector of the values of n calls to f() - used in simulations where the value of f is stochastic.\n\nprogress controls whether the progress bar is shown. Note that the progress bar is automatically disabled for non-interactive (i.e. logging) contexts.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.restoreoptsum!-Tuple{MixedModel, Any}","page":"API","title":"MixedModels.restoreoptsum!","text":"restoreoptsum!(m::MixedModel, io::IO; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\nrestoreoptsum!(m::MixedModel, filename; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\n\nRead, check, and restore the optsum field from a JSON stream or filename.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.restorereplicates-Union{Tuple{T}, Tuple{Any, MixedModel{T}}, Tuple{Any, MixedModel{T}, Type{<:AbstractFloat}}} where T","page":"API","title":"MixedModels.restorereplicates","text":"restorereplicates(f, m::MixedModel{T})\nrestorereplicates(f, m::MixedModel{T}, ftype::Type{<:AbstractFloat})\nrestorereplicates(f, m::MixedModel{T}, ctype::Type{<:MixedModelFitCollection{S}})\n\nRestore replicates from f, using m to create the desired subtype of MixedModelFitCollection.\n\nf can be any entity supported by Arrow.Table. m does not have to be fitted, but it must have been constructed with the same structure as the source of the saved replicates.\n\nThe two-argument method constructs a MixedModelBootstrap with the same eltype as m. If an eltype is specified as the third argument, then a MixedModelBootstrap is returned. If a subtype of MixedModelFitCollection is specified as the third argument, then that is the return type.\n\nSee also savereplicates, restoreoptsum!.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.saveoptsum-Tuple{IO, MixedModel}","page":"API","title":"MixedModels.saveoptsum","text":"saveoptsum(io::IO, m::MixedModel)\nsaveoptsum(filename, m::MixedModel)\n\nSave m.optsum (w/o the lowerbd field) in JSON format to an IO stream or a file\n\nThe reason for omitting the lowerbd field is because it often contains -Inf values that are not allowed in JSON.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.savereplicates-Tuple{Any, MixedModels.MixedModelFitCollection}","page":"API","title":"MixedModels.savereplicates","text":"savereplicates(f, b::MixedModelFitCollection)\n\nSave the replicates associated with a MixedModelFitCollection, e.g. MixedModelBootstrap as an Arrow file.\n\nSee also restorereplicates, saveoptsum\n\nnote: Note\nOnly the replicates are saved, not the entire contents of the MixedModelFitCollection. restorereplicates requires a model compatible with the bootstrap to restore the full object.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.sdest-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.sdest","text":"sdest(m::LinearMixedModel)\n\nReturn the estimate of σ, the standard deviation of the per-observation noise.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.sdest-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.sdest","text":"sdest(m::GeneralizedLinearMixedModel)\n\nReturn the estimate of the dispersion, i.e. the standard deviation of the per-observation noise.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ. For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.setθ!-Union{Tuple{T}, Tuple{LinearMixedModel{T}, AbstractVector}} where T","page":"API","title":"MixedModels.setθ!","text":"setθ!(m::LinearMixedModel, v)\n\nInstall v as the θ parameters in m.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.setθ!-Union{Tuple{T}, Tuple{MixedModels.MixedModelFitCollection{T}, AbstractVector{T}}} where T","page":"API","title":"MixedModels.setθ!","text":"setθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)\nsetθ!(bsamp::MixedModelFitCollection, i::Integer)\n\nInstall the values of the i'th θ value of bsamp.fits in bsamp.λ\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.shortestcovint","page":"API","title":"MixedModels.shortestcovint","text":"shortestcovint(v, level = 0.95)\n\nReturn the shortest interval containing level proportion of the values of v\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.shortestcovint-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}, Tuple{MixedModels.MixedModelFitCollection{T}, Any}} where T","page":"API","title":"MixedModels.shortestcovint","text":"shortestcovint(bsamp::MixedModelFitCollection, level = 0.95)\n\nReturn the shortest interval containing level proportion for each parameter from bsamp.allpars.\n\nwarning: Warning\nCurrently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.simulate","page":"API","title":"MixedModels.simulate","text":"See simulate!\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.simulate!-Tuple{Random.AbstractRNG, AbstractVector, LinearMixedModel, NamedTuple{names, T} where {N, names, T<:Tuple{Vararg{AbstractVector, N}}}}","page":"API","title":"MixedModels.simulate!","text":"simulate!([rng::AbstractRNG,] y::AbstractVector, m::MixedModel{T}[, newdata];\n β = coef(m), σ = m.σ, θ = T[], wts=m.wts)\nsimulate([rng::AbstractRNG,] m::MixedModel{T}[, newdata];\n β = coef(m), σ = m.σ, θ = T[], wts=m.wts)\n\nSimulate a new response vector, optionally overwriting a pre-allocated vector.\n\nNew data can be optionally provided in tabular format.\n\nThis simulation includes sampling new values for the random effects. Thus in contrast to predict, there is no distinction in between \"new\" and \"old\" / previously observed random-effects levels.\n\nUnlike predict, there is no type parameter for GeneralizedLinearMixedModel because the noise term in the model and simulation is always on the response scale.\n\nThe wts argument is currently ignored except for GeneralizedLinearMixedModel models with a Binomial distribution.\n\nnote: Note\nNote that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.simulate!-Union{Tuple{T}, Tuple{Random.AbstractRNG, LinearMixedModel{T}}} where T","page":"API","title":"MixedModels.simulate!","text":"simulate!(rng::AbstractRNG, m::MixedModel{T}; β=fixef(m), σ=m.σ, θ=T[])\nsimulate!(m::MixedModel; β=fixef(m), σ=m.σ, θ=m.θ)\n\nOverwrite the response (i.e. m.trms[end]) with a simulated response vector from model m.\n\nThis simulation includes sampling new values for the random effects.\n\nβ can be specified either as a pivoted, full rank coefficient vector (cf. fixef) or as an unpivoted full dimension coefficient vector (cf. coef), where the entries corresponding to redundant columns will be ignored.\n\nnote: Note\nNote that simulate! methods with a y::AbstractVector as the first argument (besides the RNG) and simulate methods return the simulated response. This is in contrast to simulate! methods with a m::MixedModel as the first argument, which modify the model's response and return the entire modified model.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.sparseL-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.sparseL","text":"sparseL(m::LinearMixedModel; fname::Symbol=first(fnames(m)), full::Bool=false)\n\nReturn the lower Cholesky factor L as a SparseMatrix{T,Int32}.\n\nfull indicates whether the parts of L associated with the fixed-effects and response are to be included.\n\nfname specifies the first grouping factor to include. Blocks to the left of the block corresponding to fname are dropped. The default is the first, i.e., leftmost block and hence all blocks.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.stderror!-Union{Tuple{T}, Tuple{Tv}, Tuple{AbstractVector{Tv}, LinearMixedModel{T}}} where {Tv, T}","page":"API","title":"MixedModels.stderror!","text":"stderror!(v::AbstractVector, m::LinearMixedModel)\n\nOverwrite v with the standard errors of the fixed-effects coefficients in m\n\nThe length of v should be the total number of coefficients (i.e. length(coef(m))). When the model matrix is rank-deficient the coefficients forced to -0.0 have an undefined (i.e. NaN) standard error.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.updateL!-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.updateL!","text":"updateL!(m::LinearMixedModel)\n\nUpdate the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)\n\nThis is the crucial step in evaluating the objective, given a new parameter value.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.varest-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.varest","text":"varest(m::LinearMixedModel)\n\nReturns the estimate of σ², the variance of the conditional distribution of Y given B.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.varest-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.varest","text":"varest(m::GeneralizedLinearMixedModel)\n\nReturns the estimate of ϕ², the variance of the conditional distribution of Y given B.\n\nFor models with a dispersion parameter ϕ, this is simply ϕ². For models without a dispersion parameter, this value is missing. This differs from disperion, which returns 1 for models without a dispersion parameter.\n\nFor Gaussian models, this parameter is often called σ².\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.zerocorr-Tuple{Any}","page":"API","title":"MixedModels.zerocorr","text":"zerocorr(term::RandomEffectsTerm)\n\nRemove correlations between random effects in term.\n\n\n\n\n\n","category":"method"},{"location":"api/#Statistics.std-Tuple{LinearMixedModel}","page":"API","title":"Statistics.std","text":"std(m::MixedModel)\n\nReturn the estimated standard deviations of the random effects as a Vector{Vector{T}}.\n\nFIXME: This uses an old convention of isfinite(sdest(m)). Probably drop in favor of m.σs\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.confint-Tuple{MixedModelProfile}","page":"API","title":"StatsAPI.confint","text":"confint(pr::MixedModelProfile; level::Real=0.95)\n\nCompute profile confidence intervals for coefficients and variance components, with confidence level level (by default 95%).\n\nnote: Note\nThe API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.\n\nnote: Note\nThe \"row names\" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.confint-Union{Tuple{MixedModelBootstrap{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.confint","text":"confint(pr::MixedModelBootstrap; level::Real=0.95, method=:shortest)\n\nCompute bootstrap confidence intervals for coefficients and variance components, with confidence level level (by default 95%).\n\nThe keyword argument method determines whether the :shortest, i.e. highest density, interval is used or the :equaltail, i.e. quantile-based, interval is used. For historical reasons, the default is :shortest, but :equaltail gives the interval that is most comparable to the profile and Wald confidence intervals.\n\nnote: Note\nThe API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.\n\nnote: Note\nThe \"row names\" indicating the associated parameter name are guaranteed to be unambiguous, but their precise naming scheme is not yet stable and may change in a future release without being considered breaking.\n\nSee also shortestcovint.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.confint-Union{Tuple{MixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.confint","text":"confint(pr::MixedModelProfile; level::Real=0.95)\n\nCompute profile confidence intervals for (fixed effects) coefficients, with confidence level level (by default 95%).\n\nnote: Note\nThe API guarantee is for a Tables.jl compatible table. The exact return type is an implementation detail and may change in a future minor release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.deviance-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}, Tuple{GeneralizedLinearMixedModel{T}, Any}} where T","page":"API","title":"StatsAPI.deviance","text":"deviance(m::GeneralizedLinearMixedModel{T}, nAGQ=1)::T where {T}\n\nReturn the deviance of m evaluated by the Laplace approximation (nAGQ=1) or nAGQ-point adaptive Gauss-Hermite quadrature.\n\nIf the distribution D does not have a scale parameter the Laplace approximation is the squared length of the conditional modes, u, plus the determinant of ΛZWZΛ + I, plus the sum of the squared deviance residuals.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.dof_residual-Tuple{MixedModel}","page":"API","title":"StatsAPI.dof_residual","text":"dof_residual(m::MixedModel)\n\nReturn the residual degrees of freedom of the model.\n\nnote: Note\nThe residual degrees of freedom for mixed-effects models is not clearly defined due to partial pooling. The classical nobs(m) - dof(m) fails to capture the extra freedom granted by the random effects, but nobs(m) - nranef(m) would overestimate the freedom granted by the random effects. nobs(m) - sum(leverage(m)) provides a nice balance based on the relative influence of each observation, but is computationally expensive for large models. This problem is also fundamentally related to long-standing debates about the appropriate treatment of the denominator degrees of freedom for F-tests. In the future, MixedModels.jl may provide additional methods allowing the user to choose the computation to use.\n\nwarning: Warning\nCurrently, the residual degrees of freedom is computed as nobs(m) - dof(m), but this may change in the future without being considered a breaking change because there is no canonical definition of the residual degrees of freedom in a mixed-effects model.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.fit!-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.fit!","text":"fit!(m::GeneralizedLinearMixedModel; fast=false, nAGQ=1,\n verbose=false, progress=true,\n thin::Int=1,\n init_from_lmm=Set())\n\nOptimize the objective function for m.\n\nWhen fast is true a potentially much faster but slightly less accurate algorithm, in which pirls! optimizes both the random effects and the fixed-effects parameters, is used.\n\nIf progress is true, the default, a ProgressMeter.ProgressUnknown counter is displayed. during the iterations to minimize the deviance. There is a delay before this display is initialized and it may not be shown at all for models that are optimized quickly.\n\nIf verbose is true, then both the intermediate results of both the nonlinear optimization and PIRLS are also displayed on standard output.\n\nAt every thinth iteration is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.\n\nBy default, the starting values for model fitting are taken from a (non mixed, i.e. marginal ) GLM fit. Experience with larger datasets (many thousands of observations and/or hundreds of levels of the grouping variables) has suggested that fitting a (Gaussian) linear mixed model on the untransformed data may provide better starting values and thus overall faster fits even though an entire LMM must be fit before the GLMM can be fit. init_from_lmm can be used to specify which starting values from an LMM to use. Valid options are any collection (array, set, etc.) containing one or more of :β and :θ, the default is the empty set.\n\nnote: Note\nInitializing from an LMM requires fitting the entire LMM first, so when progress=true, there will be two progress bars: first for the LMM, then for the GLMM.\n\nwarning: Warning\nThe init_from_lmm functionality is experimental and may change or be removed entirely without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.fit!-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.fit!","text":"fit!(m::LinearMixedModel; progress::Bool=true, REML::Bool=m.optsum.REML,\n σ::Union{Real, Nothing}=m.optsum.sigma,\n thin::Int=typemax(Int))\n\nOptimize the objective of a LinearMixedModel. When progress is true a ProgressMeter.ProgressUnknown display is shown during the optimization of the objective, if the optimization takes more than one second or so.\n\nAt every thinth iteration is recorded in fitlog, optimization progress is saved in m.optsum.fitlog.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.leverage-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"StatsAPI.leverage","text":"leverage(::LinearMixedModel)\n\nReturn the diagonal of the hat matrix of the model.\n\nFor a linear model, the sum of the leverage values is the degrees of freedom for the model in the sense that this sum is the dimension of the span of columns of the model matrix. With a bit of hand waving a similar argument could be made for linear mixed-effects models. The hat matrix is of the form ZΛ XL L¹ZΛ X.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.modelmatrix-Tuple{MixedModel}","page":"API","title":"StatsAPI.modelmatrix","text":"modelmatrix(m::MixedModel)\n\nReturns the model matrix X for the fixed-effects parameters, as returned by coef.\n\nThis is always the full model matrix in the original column order and from a field in the model struct. It should be copied if it is to be modified.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.predict-Tuple{LinearMixedModel, NamedTuple{names, T} where {N, names, T<:Tuple{Vararg{AbstractVector, N}}}}","page":"API","title":"StatsAPI.predict","text":"StatsAPI.predict(m::LinearMixedModel, newdata;\n new_re_levels=:missing)\nStatsAPI.predict(m::GeneralizedLinearMixedModel, newdata;\n new_re_levels=:missing, type=:response)\n\nPredict response for new data.\n\nnote: Note\nCurrently, no in-place methods are provided because these methods internally construct a new model and therefore allocate not just a response vector but also many other matrices.\n\nwarning: Warning\nnewdata should contain a column for the response (dependent variable) initialized to some numerical value (not missing), because this is used to construct the new model used in computing the predictions. missing is not valid because missing data are dropped before constructing the model matrices.\n\nwarning: Warning\nThese methods construct an entire MixedModel behind the scenes and as such may use a large amount of memory when newdata is large.\n\nwarning: Warning\nRank-deficiency can lead to surprising but consistent behavior. For example, if there are two perfectly collinear predictors A and B (e.g. constant multiples of each other), then it is possible that A will be pivoted out in the fitted model and thus the associated coefficient is set to zero. If predictions are then generated on new data where B has been set to zero but A has not, then there will no contribution from neither A nor B in the resulting predictions.\n\nThe keyword argument new_re_levels specifies how previously unobserved values of the grouping variable are handled. Possible values are:\n\n:population: return population values for the relevant grouping variable. In other words, treat the associated random effect as 0. If all grouping variables have new levels, then this is equivalent to just the fixed effects.\n:missing: return missing.\n:error: error on this condition. The error type is an implementation detail: you should not rely on a particular type of error being thrown.\n\nIf you want simulated values for unobserved levels of the grouping variable, consider the simulate! and simulate methods.\n\nPredictions based purely on the fixed effects can be obtained by specifying previously unobserved levels of the random effects and setting new_re_levels=:population. Similarly, the contribution of any grouping variable can be excluded by specifying previously unobserved levels, while including previously observed levels of the other grouping variables. In the future, it may be possible to specify a subset of the grouping variables or overall random-effects structure to use, but not at this time.\n\nnote: Note\nnew_re_levels impacts only the behavior for previously unobserved random effects levels, i.e. new RE levels. For previously observed random effects levels, predictions take both the fixed and random effects into account.\n\nFor GeneralizedLinearMixedModel, the type parameter specifies whether the predictions should be returned on the scale of linear predictor (:linpred) or on the response scale (:response). If you don't know the difference between these terms, then you probably want type=:response.\n\nRegression weights are not yet supported in prediction. Similarly, offsets are also not supported for GeneralizedLinearMixedModel.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.response-Tuple{MixedModel}","page":"API","title":"StatsAPI.response","text":"response(m::MixedModel)\n\nReturn the response vector for the model.\n\nFor a linear mixed model this is a view of the last column of the XyMat field. For a generalized linear mixed model this is the m.resp.y field. In either case it should be copied if it is to be modified.\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsAPI.vcov-Tuple{MixedModel}","page":"API","title":"StatsAPI.vcov","text":"vcov(m::MixedModel; corr=false)\n\nReturns the variance-covariance matrix of the fixed effects. If corr is true, the correlation of the fixed effects is returned instead.\n\n\n\n\n\n","category":"method"},{"location":"api/#Tables.columntable-Tuple{OptSummary}","page":"API","title":"Tables.columntable","text":"columntable(s::OptSummary, [stack::Bool=false])\n\nReturn s.fitlog as a Tables.columntable.\n\nWhen stack is false (the default), there will be 3 columns in the result:\n\niter: the sample number\nobjective: the value of the objective at that sample\nθ: the parameter vector at that sample\n\n(The term sample here refers to the fact that when the thin argument to the fit or refit! call is greater than 1 only a subset of the iterations have results recorded.)\n\nWhen stack is true, there will be 4 columns: iter, objective, par, and value where value is the stacked contents of the θ vectors (the equivalent of vcat(θ...)) and par is a vector of parameter numbers.\n\n\n\n\n\n","category":"method"},{"location":"api/#Methods-from-StatsAPI.jl,-StatsBase.jl,-StatsModels.jl-and-GLM.jl","page":"API","title":"Methods from StatsAPI.jl, StatsBase.jl, StatsModels.jl and GLM.jl","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"aic\naicc\nbic\ncoef\ncoefnames\ncoeftable\ndeviance\ndispersion\ndispersion_parameter\ndof\ndof_residual\nfit\nfit!\nfitted\nformula\nisfitted\nislinear\nleverage\nloglikelihood\nmeanresponse\nmodelmatrix\nmodel_response\nnobs\npredict\nresiduals\nresponse\nresponsename\nStatsModels.lrtest # not exported\nstd\nstderror\nvcov\nweights","category":"page"},{"location":"api/#MixedModels.jl-\"alternatives\"-and-extensions-to-StatsAPI-and-GLM-functions","page":"API","title":"MixedModels.jl \"alternatives\" and extensions to StatsAPI and GLM functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"The following are MixedModels.jl-specific functions and not simply methods for functions defined in StatsAPI and GLM.jl.","category":"page"},{"location":"api/","page":"API","title":"API","text":"coefpvalues\ncondVar\ncondVarTables\nfitted!\nfixef\nfixefnames\nlikelihoodratiotest # not exported\npwrss\nranef\nraneftables\nrefit!\nshortestcovint\nsdest\nsimulate\nsimulate!\nstderrror!\nvarest","category":"page"},{"location":"api/#Non-Exported-Functions","page":"API","title":"Non-Exported Functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Note that unless discussed elsewhere in the online documentation, non-exported functions should be considered implementation details.","category":"page"},{"location":"api/","page":"API","title":"API","text":"Modules = [MixedModels]\nPublic = false\nOrder = [:function]\nFilter = f -> !startswith(string(f), \"_\")","category":"page"},{"location":"api/#Base.copy-Union{Tuple{ReMat{T, S}}, Tuple{S}, Tuple{T}} where {T, S}","page":"API","title":"Base.copy","text":"Base.copy(ReMat{T,S})\n\nReturn a shallow copy of ReMat.\n\nA shallow copy shares as much internal storage as possible with the original ReMat. Only the vector λ and the scratch matrix are copied.\n\n\n\n\n\n","category":"method"},{"location":"api/#Base.size-Tuple{MixedModel}","page":"API","title":"Base.size","text":"size(m::MixedModel)\n\nReturns the size of a mixed model as a tuple of length four: the number of observations, the number of (non-singular) fixed-effects parameters, the number of conditional modes (random effects), the number of grouping variables\n\n\n\n\n\n","category":"method"},{"location":"api/#GLM.wrkresp!-Union{Tuple{T}, Tuple{AbstractVector{T}, GLM.GlmResp{Vector{T}}}} where T<:AbstractFloat","page":"API","title":"GLM.wrkresp!","text":"GLM.wrkresp!(v::AbstractVector{T}, resp::GLM.GlmResp{AbstractVector{T}})\n\nA copy of a method from GLM that generalizes the types in the signature\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.LD-Union{Tuple{LinearAlgebra.Diagonal{T, V} where V<:AbstractVector{T}}, Tuple{T}} where T<:Number","page":"API","title":"MixedModels.LD","text":"LD(A::Diagonal)\nLD(A::HBlikDiag)\nLD(A::DenseMatrix)\n\nReturn log(det(tril(A))) evaluated in place.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.adjA-Tuple{AbstractVector, AbstractMatrix}","page":"API","title":"MixedModels.adjA","text":"adjA(refs::AbstractVector, z::AbstractMatrix{T})\n\nReturns the adjoint of an ReMat as a SparseMatrixCSC{T,Int32}\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.allpars-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.allpars","text":"allpars(bsamp::MixedModelFitCollection)\n\nReturn a tidy (column)table with the parameter estimates spread into columns of iter, type, group, name and value.\n\nwarning: Warning\nCurrently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.amalgamate-Union{Tuple{Vector{<:AbstractReMat{T}}}, Tuple{T}} where T","page":"API","title":"MixedModels.amalgamate","text":"amalgamate(reterms::Vector{AbstractReMat})\n\nCombine multiple ReMat with the same grouping variable into a single object.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.average-Union{Tuple{T}, Tuple{T, T}} where T<:AbstractFloat","page":"API","title":"MixedModels.average","text":"average(a::T, b::T) where {T<:AbstractFloat}\n\nReturn the average of a and b\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.block-Tuple{Integer, Integer}","page":"API","title":"MixedModels.block","text":"block(i, j)\n\nReturn the linear index of the [i,j] position (\"block\") in the row-major packed lower triangle.\n\nUse the row-major ordering in this case because the result depends only on i and j, not on the overall size of the array.\n\nWhen i == j the value is the same as kp1choose2(i).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.cholUnblocked!","page":"API","title":"MixedModels.cholUnblocked!","text":"cholUnblocked!(A, Val{:L})\n\nOverwrite the lower triangle of A with its lower Cholesky factor.\n\nThe name is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl] because these are part of the inner calculations in a blocked Cholesky factorization.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.copyscaleinflate!","page":"API","title":"MixedModels.copyscaleinflate!","text":"copyscaleinflate!(L::AbstractMatrix, A::AbstractMatrix, Λ::ReMat)\n\nOverwrite L with Λ'AΛ + I\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.corrmat-Union{Tuple{ReMat{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.corrmat","text":"corrmat(A::ReMat)\n\nReturn the estimated correlation matrix for A. The diagonal elements are 1 and the off-diagonal elements are the correlations between those random effect terms\n\nExample\n\nNote that trailing digits may vary slightly depending on the local platform.\n\njulia> using MixedModels\n\njulia> mod = fit(MixedModel,\n @formula(rt_trunc ~ 1 + spkr + prec + load + (1 + spkr + prec | subj)),\n MixedModels.dataset(:kb07));\n\njulia> VarCorr(mod)\nVariance components:\n Column Variance Std.Dev. Corr.\nsubj (Intercept) 136591.782 369.583\n spkr: old 22922.871 151.403 +0.21\n prec: maintain 32348.269 179.856 -0.98 -0.03\nResidual 642324.531 801.452\n\njulia> MixedModels.corrmat(mod.reterms[1])\n3×3 LinearAlgebra.Symmetric{Float64,Array{Float64,2}}:\n 1.0 0.214816 -0.982948\n 0.214816 1.0 -0.0315607\n -0.982948 -0.0315607 1.0\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.cpad-Tuple{String, Integer}","page":"API","title":"MixedModels.cpad","text":"cpad(s::AbstractString, n::Integer)\n\nReturn a string of length n containing s in the center (more-or-less).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.densify","page":"API","title":"MixedModels.densify","text":"densify(S::SparseMatrix, threshold=0.1)\n\nConvert sparse S to Diagonal if S is diagonal or to Array(S) if the proportion of nonzeros exceeds threshold.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.deviance!","page":"API","title":"MixedModels.deviance!","text":"deviance!(m::GeneralizedLinearMixedModel, nAGQ=1)\n\nUpdate m.η, m.μ, etc., install the working response and working weights in m.LMM, update m.LMM.A and m.LMM.R, then evaluate the deviance.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.feL-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.feL","text":"feL(m::LinearMixedModel)\n\nReturn the lower Cholesky factor for the fixed-effects parameters, as an LowerTriangular p × p matrix.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fixef!-Union{Tuple{T}, Tuple{Tv}, Tuple{AbstractVector{Tv}, LinearMixedModel{T}}} where {Tv, T}","page":"API","title":"MixedModels.fixef!","text":"fixef!(v::Vector{T}, m::MixedModel{T})\n\nOverwrite v with the pivoted fixed-effects coefficients of model m\n\nFor full-rank models the length of v must be the rank of X. For rank-deficient models the length of v can be the rank of X or the number of columns of X. In the latter case the calculated coefficients are padded with -0.0 out to the number of columns.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.fname-Tuple{ReMat}","page":"API","title":"MixedModels.fname","text":"fname(A::ReMat)\n\nReturn the name of the grouping factor as a Symbol\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.getθ!-Union{Tuple{T}, Tuple{AbstractVector{T}, ReMat{T}}} where T","page":"API","title":"MixedModels.getθ!","text":"getθ!(v::AbstractVector{T}, A::ReMat{T}) where {T}\n\nOverwrite v with the elements of the blocks in the lower triangle of A.Λ (column-major ordering)\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.getθ-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.getθ","text":"getθ(m::LinearMixedModel)\n\nReturn the current covariance parameter vector.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.indmat","page":"API","title":"MixedModels.indmat","text":"indmat(A::ReMat)\n\nReturn a Bool indicator matrix of the potential non-zeros in A.λ\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.isconstant-Tuple{Any}","page":"API","title":"MixedModels.isconstant","text":"isconstant(x::Array)\nisconstant(x::Tuple)\n\nAre all elements of the iterator the same? That is, is it constant?\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.isfullrank-Tuple{MixedModels.FeTerm}","page":"API","title":"MixedModels.isfullrank","text":"isfullrank(A::FeTerm)\n\nDoes A have full column rank?\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.isnested-Tuple{ReMat, ReMat}","page":"API","title":"MixedModels.isnested","text":"isnested(A::ReMat, B::ReMat)\n\nIs the grouping factor for A nested in the grouping factor for B?\n\nThat is, does each value of A occur with just one value of B?\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.kchoose2-Tuple{Any}","page":"API","title":"MixedModels.kchoose2","text":"kchoose2(k)\n\nThe binomial coefficient k choose 2 which is the number of elements in the packed form of the strict lower triangle of a matrix.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.kp1choose2-Tuple{Any}","page":"API","title":"MixedModels.kp1choose2","text":"kp1choose2(k)\n\nThe binomial coefficient k+1 choose 2 which is the number of elements in the packed form of the lower triangle of a matrix.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.likelihoodratiotest-Tuple{Vararg{MixedModel}}","page":"API","title":"MixedModels.likelihoodratiotest","text":"likelihoodratiotest(m::MixedModel...)\nlikelihoodratiotest(m0::LinearModel, m::MixedModel...)\nlikelihoodratiotest(m0::GeneralizedLinearModel, m::MixedModel...)\nlikelihoodratiotest(m0::TableRegressionModel{LinearModel}, m::MixedModel...)\nlikelihoodratiotest(m0::TableRegressionModel{GeneralizedLinearModel}, m::MixedModel...)\n\nLikeihood ratio test applied to a set of nested models.\n\nnote: Note\nThe nesting of the models is not checked. It is incumbent on the user to check this. This differs from StatsModels.lrtest as nesting in mixed models, especially in the random effects specification, may be non obvious.\n\nnote: Note\nFor comparisons between mixed and non-mixed models, the deviance for the non-mixed model is taken to be -2 log likelihood, i.e. omitting the additive constant for the fully saturated model. This is in line with the computation of the deviance for mixed models.\n\nThis functionality may be deprecated in the future in favor of StatsModels.lrtest.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.nranef-Tuple{ReMat}","page":"API","title":"MixedModels.nranef","text":"nranef(A::ReMat)\n\nReturn the number of random effects represented by A. Zero unless A is an ReMat.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.nθ-Tuple{ReMat}","page":"API","title":"MixedModels.nθ","text":"nθ(A::ReMat)\n\nReturn the number of free parameters in the relative covariance matrix λ\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.optsumj-Tuple{OptSummary, Integer}","page":"API","title":"MixedModels.optsumj","text":"optsumj(os::OptSummary, j::Integer)\n\nReturn an OptSummary with the j'th component of the parameter omitted.\n\nos.final with its j'th component omitted is used as the initial parameter.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.parsej-Tuple{Symbol}","page":"API","title":"MixedModels.parsej","text":"parsej(sym::Symbol)\n\nReturn the index from symbol names like :θ1, :θ01, etc.\n\nnote: Note\nThis method is internal.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.pivot-Tuple{MixedModel}","page":"API","title":"MixedModels.pivot","text":"pivot(m::MixedModel)\npivot(A::FeTerm)\n\nReturn the pivot associated with the FeTerm.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.prfit!","page":"API","title":"MixedModels.prfit!","text":"prfit!(m::LinearMixedModel)\n\nFit a mixed model using the PRIMA implementation of the BOBYQA optimizer.\n\nwarning: Experimental feature\nThis function is an experimental feature that will go away in the future. Do not rely on it, unless you are willing to pin the precise MixedModels.jl version. The purpose of the function is to provide the MixedModels developers a chance to explore the performance of the PRIMA implementation without the large and potentially breaking changes it would take to fully replace the current NLopt backend with a PRIMA backend or a backend supporting a range of optimizers.\n\nnote: OptSummary\nAs part of this experimental foray, the structure of OptSummary is not changed. This means that some fields of OptSummary are inaccurate when examining a model fit with PRIMA. The following fields are unused with PRIMA fits:all tolerances (ftol_rel, ftol_abs, xtol_rel, xtol_abs)\noptimization timeouts (maxfeval, maxtime)\ninitial_stepThe following fields have a different meaning when used with PRIMA:returnvalue is populated with a symbol representing the PRIMA return value, which PRIMA represents as an enum.\noptimizer is populated with a dummy value, indicating that a model was fit with PRIMA. If you wish to refit the model with the NLOpt backend, you will need to update the field with an appropriate NLOpt optimizer, e.g. :LN_BOBYQA\n\nnote: Package extension\nIn order to reduce the dependency burden, all methods of this function are implemented in a package extension and are only defined when PRIMA.jl is loaded by the user.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.profileσs!-Union{Tuple{T}, Tuple{NamedTuple, MixedModels.TableColumns{T}}} where T","page":"API","title":"MixedModels.profileσs!","text":" profileσs!(val::NamedTuple, tc::TableColumns{T}; nzlb=1.0e-8) where {T}\n\nProfile the variance components.\n\nnote: Note\nThis method is called by profile and currently considered internal. As such, it may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.ranef!-Union{Tuple{T}, Tuple{Vector, LinearMixedModel{T}, AbstractArray{T}, Bool}} where T","page":"API","title":"MixedModels.ranef!","text":"ranef!(v::Vector{Matrix{T}}, m::MixedModel{T}, β, uscale::Bool) where {T}\n\nOverwrite v with the conditional modes of the random effects for m.\n\nIf uscale is true the random effects are on the spherical (i.e. u) scale, otherwise on the original scale\n\nβ is the truncated, pivoted coefficient vector.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.rankUpdate!","page":"API","title":"MixedModels.rankUpdate!","text":"rankUpdate!(C, A)\nrankUpdate!(C, A, α)\nrankUpdate!(C, A, α, β)\n\nA rank-k update, C := αA'A + βC, of a Hermitian (Symmetric) matrix.\n\nα and β both default to 1.0. When α is -1.0 this is a downdate operation. The name rankUpdate! is borrowed from [https://github.com/andreasnoack/LinearAlgebra.jl]\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.rePCA-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.rePCA","text":"rePCA(m::LinearMixedModel; corr::Bool=true)\n\nReturn a named tuple of the normalized cumulative variance of a principal components analysis of the random effects covariance matrices or correlation matrices when corr is true.\n\nThe normalized cumulative variance is the proportion of the variance for the first principal component, the first two principal components, etc. The last element is always 1.0 representing the complete proportion of the variance.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.reevaluateAend!-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.reevaluateAend!","text":"reevaluateAend!(m::LinearMixedModel)\n\nReevaluate the last column of m.A from m.Xymat. This function should be called after updating the response.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.refitσ!-Union{Tuple{T}, Tuple{LinearMixedModel{T}, Any, MixedModels.TableColumns{T}, T, Bool}} where T","page":"API","title":"MixedModels.refitσ!","text":"refitσ!(m::LinearMixedModel{T}, σ::T, tc::TableColumns{T}, obj::T, neg::Bool)\n\nRefit the model m with the given value of σ and return a NamedTuple of information about the fit.\n\nobj and neg allow for conversion of the objective to the ζ scale and tc is used to return a NamedTuple\n\nnote: Note\nThis method is internal and may change or disappear in a future release without being considered breaking.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.schematize","page":"API","title":"MixedModels.schematize","text":"schematize(f, tbl, contrasts::Dict{Symbol}, Mod=LinearMixedModel)\n\nFind and apply the schema for f in a way that automatically uses Grouping() contrasts when appropriate.\n\nwarn: Warn\nThis is an internal method.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.sdcorr-Union{Tuple{AbstractMatrix{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.sdcorr","text":"sdcorr(A::AbstractMatrix{T}) where {T}\n\nTransform a square matrix A with positive diagonals into an NTuple{size(A,1), T} of standard deviations and a tuple of correlations.\n\nA is assumed to be symmetric and only the lower triangle is used. The order of the correlations is row-major ordering of the lower triangle (or, equivalently, column-major in the upper triangle).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.setβθ!-Tuple{GeneralizedLinearMixedModel, Any}","page":"API","title":"MixedModels.setβθ!","text":"setβθ!(m::GeneralizedLinearMixedModel, v)\n\nSet the parameter vector, :βθ, of m to v.\n\nβθ is the concatenation of the fixed-effects, β, and the covariance parameter, θ.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.ssqdenom-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.ssqdenom","text":"ssqdenom(m::LinearMixedModel)\n\nReturn the denominator for penalized sums-of-squares.\n\nFor MLE, this value is the number of observations. For REML, this value is the number of observations minus the rank of the fixed-effects matrix. The difference is analogous to the use of n or n-1 in the denominator when calculating the variance.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.statsrank-Union{Tuple{AbstractMatrix{T}}, Tuple{T}} where T<:AbstractFloat","page":"API","title":"MixedModels.statsrank","text":"statsrank(x::Matrix{T}, ranktol::Real=1e-8) where {T<:AbstractFloat}\n\nReturn the numerical column rank and a pivot vector.\n\nThe rank is determined from the absolute values of the diagonal of R from a pivoted QR decomposition, relative to the first (and, hence, largest) element of this vector.\n\nIn the full-rank case the pivot vector is collect(axes(x, 2)).\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.tidyβ-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.tidyβ","text":"tidyβ(bsamp::MixedModelFitCollection)\n\nReturn a tidy (row)table with the parameter estimates spread into columns of iter, coefname and β\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.tidyσs-Union{Tuple{MixedModels.MixedModelFitCollection{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.tidyσs","text":"tidyσs(bsamp::MixedModelFitCollection)\n\nReturn a tidy (row)table with the estimates of the variance components (on the standard deviation scale) spread into columns of iter, group, column and σ.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.unfit!-Union{Tuple{LinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.unfit!","text":"unfit!(model::MixedModel)\n\nMark a model as unfitted.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.unscaledre!","page":"API","title":"MixedModels.unscaledre!","text":"unscaledre!(y::AbstractVector{T}, M::ReMat{T}) where {T}\nunscaledre!(rng::AbstractRNG, y::AbstractVector{T}, M::ReMat{T}) where {T}\n\nAdd unscaled random effects simulated from M to y.\n\nThese are unscaled random effects (i.e. they incorporate λ but not σ) because the scaling is done after the per-observation noise is added as a standard normal.\n\n\n\n\n\n","category":"function"},{"location":"api/#MixedModels.updateA!-Tuple{LinearMixedModel}","page":"API","title":"MixedModels.updateA!","text":"updateA!(m::LinearMixedModel)\n\nUpdate the cross-product array, m.A, from m.reterms and m.Xymat\n\nThis is usually done after a reweight! operation.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.updateη!-Union{Tuple{GeneralizedLinearMixedModel{T}}, Tuple{T}} where T","page":"API","title":"MixedModels.updateη!","text":"updateη!(m::GeneralizedLinearMixedModel)\n\nUpdate the linear predictor, m.η, from the offset and the B-scale random effects.\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.σvals!-Tuple{AbstractVector, ReMat, Number}","page":"API","title":"MixedModels.σvals!","text":"σvals!(v::AbstractVector, A::ReMat, sc::Number)\n\nOverwrite v with the standard deviations of the random effects associated with A\n\n\n\n\n\n","category":"method"},{"location":"api/#MixedModels.σρ!-Union{Tuple{T}, Tuple{AbstractVector{<:Union{Missing, T}}, LinearAlgebra.LowerTriangular, Any}} where T","page":"API","title":"MixedModels.σρ!","text":"σρ!(v, t, σ)\n\npush! σ times the row lengths (σs) and the inner products of normalized rows (ρs) of t onto v\n\n\n\n\n\n","category":"method"},{"location":"api/#StatsModels.isnested-Tuple{MixedModel, MixedModel}","page":"API","title":"StatsModels.isnested","text":"isnested(m1::MixedModel, m2::MixedModel; atol::Real=0.0)\n\nIndicate whether model m1 is nested in model m2, i.e. whether m1 can be obtained by constraining some parameters in m2. Both models must have been fitted on the same data. This check is conservative for MixedModels and may reject nested models with different parameterizations as being non nested.\n\n\n\n\n\n","category":"method"},{"location":"mime/#Alternative-display-and-output-formats","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"","category":"section"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"In the documentation, we have presented the output from MixedModels.jl in the same format you will see when working in the REPL. You may have noticed, however, that output from other packages received pretty printing. For example, DataFrames are converted into nice HTML tables. In MixedModels, we recently (v3.2.0) introduced limited support for such pretty printing. (For more details on how the print and display system in Julia works, check out this NextJournal post.)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"In particular, we have defined Markdown, HTML and LaTeX output, i.e. show methods, for our types. Note that the Markdown output can also be easily and more flexibly translated into HTML, LaTeX (e.g. with booktabs) or even a MS Word Document using tools such as pandoc. Packages like IJulia and Documenter can often detect the presence of these display options and use them automatically.","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"using MixedModels\nform = @formula(rt_trunc ~ 1 + spkr * prec * load +\n (1 + load | item) +\n (1 + spkr + prec + load | subj))\ncontr = Dict(:spkr => EffectsCoding(),\n :prec => EffectsCoding(),\n :load => EffectsCoding(),\n :item => Grouping(),\n :subj => Grouping())\nkbm = fit(MixedModel, form, MixedModels.dataset(:kb07); contrasts=contr)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"Note that the display here is more succinct than the standard REPL display:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"using DisplayAs\nkbm |> DisplayAs.Text","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"This brevity is intentional: we wanted these types to work well with traditional academic publishing constraints on tables. The summary for a model fit presented in the REPL does not mesh well with being treated as a single table (with columns shared between the random and fixed effects). In our experience, this leads to difficulties in typesetting the resulting tables. We nonetheless encourage users to report fit statistics such as the log likelihood or AIC as part of the caption of their table. If the correlation parameters in the random effects are of interest, then VarCorr can also be pretty printed:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"VarCorr(kbm)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"Similarly for BlockDescription, OptSummary and MixedModels.likelihoodratiotest:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"BlockDescription(kbm)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"kbm.optsum","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"m0 = fit(MixedModel, @formula(reaction ~ 1 + (1|subj)), MixedModels.dataset(:sleepstudy))\nm1 = fit(MixedModel, @formula(reaction ~ 1 + days + (1+days|subj)), MixedModels.dataset(:sleepstudy))\nMixedModels.likelihoodratiotest(m0,m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"To explicitly invoke this behavior, we must specify the right show method. (The raw and not rendered output is intentionally shown here.)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/markdown\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/markdown\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/html\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/html\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"Note for that LaTeX, the column labels for the random effects are slightly changed: σ is placed into math mode and escaped and the grouping variable is turned into a subscript. Similarly for the likelihood ratio test, the χ² is escaped into math mode. This transformation improves pdfLaTeX and journal compatibility, but also means that XeLaTeX and LuaTeX may use a different font at this point.","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/latex\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/latex\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"This escaping behavior can be disabled by specifying \"text/xelatex\" as the MIME type. (Note that other symbols may still be escaped, as the internal conversion uses the Markdown module from the standard library, which performs some escaping on its own.)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"show(MIME(\"text/xelatex\"), m1)","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"println(sprint(show, MIME(\"text/xelatex\"), kbm)) # hide","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"This output can also be written directly to file:","category":"page"},{"location":"mime/","page":"Alternative display and output formats","title":"Alternative display and output formats","text":"open(\"model.md\", \"w\") do io\n show(io, MIME(\"text/markdown\"), kbm)\nend","category":"page"},{"location":"optimization/#Details-of-the-parameter-estimation","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"","category":"section"},{"location":"optimization/#The-probability-model","page":"Details of the parameter estimation","title":"The probability model","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Maximum likelihood estimates are based on the probability model for the observed responses. In the probability model the distribution of the responses is expressed as a function of one or more parameters.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For a continuous distribution the probability density is a function of the responses, given the parameters. The likelihood function is the same expression as the probability density but regarding the observed values as fixed and the parameters as varying.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In general a mixed-effects model incorporates two random variables: mathcalB, the q-dimensional vector of random effects, and mathcalY, the n-dimensional response vector. The value, bf y, of mathcalY is observed; the value, bf b, of mathcalB is not.","category":"page"},{"location":"optimization/#Linear-Mixed-Effects-Models","page":"Details of the parameter estimation","title":"Linear Mixed-Effects Models","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In a linear mixed model the unconditional distribution of mathcalB and the conditional distribution, (mathcalY mathcalB=bfb), are both multivariate Gaussian distributions,","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"beginaligned\n (mathcalY mathcalB=bfb) simmathcalN(bf Xbeta + Z bsigma^2bfI)\n mathcalBsimmathcalN(bf0Sigma_theta) \nendaligned","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The conditional mean of mathcal Y, given mathcal B=bf b, is the linear predictor, bf Xbfbeta+bf Zbf b, which depends on the p-dimensional fixed-effects parameter, bf beta, and on bf b. The model matrices, bf X and bf Z, of dimension ntimes p and ntimes q, respectively, are determined from the formula for the model and the values of covariates. Although the matrix bf Z can be large (i.e. both n and q can be large), it is sparse (i.e. most of the elements in the matrix are zero).","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The relative covariance factor, Lambda_theta, is a qtimes q lower-triangular matrix, depending on the variance-component parameter, bftheta, and generating the symmetric qtimes q variance-covariance matrix, Sigma_theta, as","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Sigma_theta=sigma^2Lambda_thetaLambda_theta","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The spherical random effects, mathcalUsimmathcalN(bf0sigma^2bfI_q), determine mathcal B according to","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mathcalB=Lambda_thetamathcalU","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The penalized residual sum of squares (PRSS),","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"r^2(thetabetabfu)=bfy - bfXbeta -bfZLambda_thetabfu^2+bfu^2","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is the sum of the residual sum of squares, measuring fidelity of the model to the data, and a penalty on the size of bf u, measuring the complexity of the model. Minimizing r^2 with respect to bf u,","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"r^2_betatheta =min_bfuleft(bfy -bfXbeta -bfZLambda_thetabfu^2+bfu^2right)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is a direct (i.e. non-iterative) computation. The particular method used to solve this generates a blocked Choleksy factor, bfL_theta, which is an lower triangular qtimes q matrix satisfying","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"bfL_thetabfL_theta=Lambda_thetabfZbfZLambda_theta+bfI_q ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"where bf I_q is the qtimes q identity matrix.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Negative twice the log-likelihood of the parameters, given the data, bf y, is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"d(bfthetabfbetasigmabf y)\n=nlog(2pisigma^2)+log(bf L_theta^2)+fracr^2_betathetasigma^2","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"where bf L_theta denotes the determinant of bf L_theta. Because bf L_theta is triangular, its determinant is the product of its diagonal elements.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Because the conditional mean, bfmu_mathcal Ymathcal B=bf b=bf Xbfbeta+bf ZLambda_thetabf u, is a linear function of both bfbeta and bf u, minimization of the PRSS with respect to both bfbeta and bf u to produce","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"r^2_theta =min_bfbetabf uleft(bf y -bf Xbfbeta -bf ZLambda_thetabf u^2+bf u^2right)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is also a direct calculation. The values of bf u and bfbeta that provide this minimum are called, respectively, the conditional mode, tildebf u_theta, of the spherical random effects and the conditional estimate, widehatbfbeta_theta, of the fixed effects. At the conditional estimate of the fixed effects the objective is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"d(bfthetawidehatbeta_thetasigmabf y)\n=nlog(2pisigma^2)+log(bf L_theta^2)+fracr^2_thetasigma^2","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Minimizing this expression with respect to sigma^2 produces the conditional estimate","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"widehatsigma^2_theta=fracr^2_thetan","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"which provides the profiled log-likelihood on the deviance scale as","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"tilded(thetabf y)=d(thetawidehatbeta_thetawidehatsigma_thetabf y)\n=log(bf L_theta^2)+nleft1+logleft(frac2pi r^2_thetanright)right","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"a function of bftheta alone.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The MLE of bftheta, written widehatbftheta, is the value that minimizes this profiled objective. We determine this value by numerical optimization. In the process of evaluating tilded(widehatthetabf y) we determine widehatbeta=widehatbeta_widehattheta, tildebf u_widehattheta and r^2_widehattheta, from which we can evaluate widehatsigma=sqrtr^2_widehatthetan.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The elements of the conditional mode of mathcal B, evaluated at the parameter estimates,","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"tildebf b_widehattheta=Lambda_widehatthetatildebf u_widehattheta","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"are sometimes called the best linear unbiased predictors or BLUPs of the random effects. Although BLUPs an appealing acronym, I don’t find the term particularly instructive (what is a “linear unbiased predictor” and in what sense are these the “best”?) and prefer the term “conditional modes”, because these are the values of bf b that maximize the density of the conditional distribution mathcalB mathcalY = bf y. For a linear mixed model, where all the conditional and unconditional distributions are Gaussian, these values are also the conditional means.","category":"page"},{"location":"optimization/#Internal-structure-of-\\Lambda_\\theta-and-\\bf-Z","page":"Details of the parameter estimation","title":"Internal structure of Lambda_theta and bf Z","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In the types of LinearMixedModel available through the MixedModels package, groups of random effects and the corresponding columns of the model matrix, bf Z, are associated with random-effects terms in the model formula.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For the simple example","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"using DisplayAs","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"using BenchmarkTools, DataFrames, MixedModels","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"dyestuff = MixedModels.dataset(:dyestuff)\nfm1 = fit(MixedModel, @formula(yield ~ 1 + (1|batch)), dyestuff)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"the only random effects term in the formula is (1|batch), a simple, scalar random-effects term.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t1 = only(fm1.reterms);\nInt.(t1) # convert to integers for more compact display","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The matrix t1 is a sparse matrix, meaning that most of the elements are zero, and its transpose is stored in a sparse form.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"sparse(t1)'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"provides a compact representation of the positions of the non-zeros in this matrix.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"This RandomEffectsTerm contributes a block of columns to the model matrix bf Z and a diagonal block to Lambda_theta. In this case the diagonal block of Lambda_theta (which is also the only block) is a multiple of the 6times6 identity matrix where the multiple is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t1.λ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Because there is only one random-effects term in the model, the matrix bf Z is the indicators matrix shown as the result of Int.(t1), but stored in a special sparse format. Furthermore, there is only one block in Lambda_theta.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For a vector-valued random-effects term, as in","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"sleepstudy = MixedModels.dataset(:sleepstudy)\nfm2 = fit(MixedModel, @formula(reaction ~ 1+days+(1+days|subj)), sleepstudy)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"the model matrix bf Z is of the form","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t21 = only(fm2.reterms);\nsparse(t21)'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"and Lambda_theta is a 36times36 block diagonal matrix with 18 diagonal blocks, all of the form","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t21.λ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The theta vector is","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"MixedModels.getθ(t21)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Random-effects terms in the model formula that have the same grouping factor are amalgamated into a single ReMat object.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fm3 = fit(MixedModel, @formula(reaction ~ 1+days+(1|subj) + (0+days|subj)), sleepstudy)\nt31 = only(fm3.reterms);\nsparse(t31)'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"For this model the matrix bf Z is the same as that of model fm2 but the diagonal blocks of Lambda_theta are themselves diagonal.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"t31.λ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"MixedModels.getθ(t31)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Random-effects terms with distinct grouping factors generate distinct elements of the reterms field of the LinearMixedModel object. Multiple ReMat objects are sorted by decreasing numbers of random effects.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"penicillin = MixedModels.dataset(:penicillin)\nfm4 = fit(MixedModel,\n @formula(diameter ~ 1 + (1|sample) + (1|plate)),\n penicillin)\nsparse(first(fm4.reterms))'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"sparse(last(fm4.reterms))'","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Note that the first ReMat in fm4.reterms corresponds to grouping factor plate even though the term (1|plate) occurs in the formula after (1|sample).","category":"page"},{"location":"optimization/#Progress-of-the-optimization","page":"Details of the parameter estimation","title":"Progress of the optimization","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"By default a progress display is shown when fitting a model that takes a second or more to fit. (The optional named argument, progress=false, can be used to suppress this display.) The number of iterations performed, the average time per iteration and the current value of the objective are shown in this display.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"After the model has been fit, a summary of the optimization process is available as the optsum property of the LinearMixedModel.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fm2.optsum\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"More detailed information about the intermediate steps of the nonlinear optimizer can be obtained the fitlog field. By default, fitlog contains entries for only the initial and final steps, but additional information about every nth step can be obtained with the thin keyword-argument to fit, fit! and refit!:","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"refit!(fm2; thin=1)\nfm2.optsum.fitlog[1:10]\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/#A-blocked-Cholesky-factor","page":"Details of the parameter estimation","title":"A blocked Cholesky factor","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A LinearMixedModel object contains two blocked matrices; a symmetric matrix A (only the lower triangle is stored) and a lower-triangular L which is the lower Cholesky factor of the updated and inflated A. In versions 4.0.0 and later of MixedModels only the blocks in the lower triangle are stored in A and L, as a Vector{AbstractMatrix{T}}.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"BlockDescription shows the structure of the blocks","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"BlockDescription(fm2)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Another change in v4.0.0 and later is that the last row of blocks is constructed from m.Xymat which contains the full-rank model matrix X with the response y concatenated on the right.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The operation of installing a new value of the variance parameters, θ, and updating L","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"setθ!\nupdateL!","category":"page"},{"location":"optimization/#MixedModels.setθ!","page":"Details of the parameter estimation","title":"MixedModels.setθ!","text":"setθ!(m::LinearMixedModel, v)\n\nInstall v as the θ parameters in m.\n\n\n\n\n\nsetθ!(bsamp::MixedModelFitCollection, θ::AbstractVector)\nsetθ!(bsamp::MixedModelFitCollection, i::Integer)\n\nInstall the values of the i'th θ value of bsamp.fits in bsamp.λ\n\n\n\n\n\n","category":"function"},{"location":"optimization/#MixedModels.updateL!","page":"Details of the parameter estimation","title":"MixedModels.updateL!","text":"updateL!(m::LinearMixedModel)\n\nUpdate the blocked lower Cholesky factor, m.L, from m.A and m.reterms (used for λ only)\n\nThis is the crucial step in evaluating the objective, given a new parameter value.\n\n\n\n\n\n","category":"function"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"is the central step in evaluating the objective (negative twice the log-likelihood).","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Typically, the (1,1) block is the largest block in A and L and it has a special form, either Diagonal or UniformBlockDiagonal providing a compact representation and fast matrix multiplication or solutions of linear systems of equations.","category":"page"},{"location":"optimization/#Modifying-the-optimization-process","page":"Details of the parameter estimation","title":"Modifying the optimization process","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The OptSummary object contains both input and output fields for the optimizer. To modify the optimization process the input fields can be changed after constructing the model but before fitting it.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Suppose, for example, that the user wishes to try a Nelder-Mead optimization method instead of the default BOBYQA (Bounded Optimization BY Quadratic Approximation) method.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fm2nm = LinearMixedModel(@formula(reaction ~ 1+days+(1+days|subj)), sleepstudy);\nfm2nm.optsum.optimizer = :LN_NELDERMEAD;\nfit!(fm2nm; thin=1)\nfm2nm.optsum\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The parameter estimates are quite similar to those using :LN_BOBYQA but at the expense of 140 functions evaluations for :LN_NELDERMEAD versus 57 for :LN_BOBYQA. When plotting the progress of the individual fits, it becomes obvious that :LN_BOBYQA has fully converged by the time :LN_NELDERMEAD begins to approach the optimum.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"using Gadfly\nnm = fm2nm.optsum.fitlog\nbob = fm2.optsum.fitlog\nconvdf = DataFrame(algorithm=[repeat([\"NelderMead\"], length(nm));\n repeat([\"BOBYQA\"], length(bob))],\n objective=[last.(nm); last.(bob)],\n step=[1:length(nm); 1:length(bob)])\nplot(convdf, x=:step, y=:objective, color=:algorithm, Geom.line)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Run time can be constrained with maxfeval and maxtime.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"See the documentation for the NLopt package for details about the various settings.","category":"page"},{"location":"optimization/#Convergence-to-singular-covariance-matrices","page":"Details of the parameter estimation","title":"Convergence to singular covariance matrices","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"To ensure identifiability of Sigma_theta=sigma^2Lambda_theta Lambda_theta, the elements of theta corresponding to diagonal elements of Lambda_theta are constrained to be non-negative. For example, in a trivial case of a single, simple, scalar, random-effects term as in fm1, the one-dimensional theta vector is the ratio of the standard deviation of the random effects to the standard deviation of the response. It happens that -theta produces the same log-likelihood but, by convention, we define the standard deviation to be the positive square root of the variance. Requiring the diagonal elements of Lambda_theta to be non-negative is a generalization of using this positive square root.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"If the optimization converges on the boundary of the feasible region, that is if one or more of the diagonal elements of Lambda_theta is zero at convergence, the covariance matrix Sigma_theta will be singular. This means that there will be linear combinations of random effects that are constant. Usually convergence to a singular covariance matrix is a sign of an over-specified model.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Singularity can be checked with the issingular predicate function.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"issingular","category":"page"},{"location":"optimization/#MixedModels.issingular","page":"Details of the parameter estimation","title":"MixedModels.issingular","text":"issingular(m::MixedModel, θ=m.θ; atol::Real=0, rtol::Real=atol>0 ? 0 : √eps)\n\nTest whether the model m is singular if the parameter vector is θ.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nnote: Note\nFor GeneralizedLinearMixedModel, the entire parameter vector (including β in the case fast=false) must be specified if the default is not used.\n\n\n\n\n\nissingular(bsamp::MixedModelFitCollection;\n atol::Real=0, rtol::Real=atol>0 ? 0 : √eps))\n\nTest each bootstrap sample for singularity of the corresponding fit.\n\nEquality comparisons are used b/c small non-negative θ values are replaced by 0 in fit!.\n\nSee also issingular(::MixedModel).\n\n\n\n\n\n","category":"function"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"issingular(fm2)","category":"page"},{"location":"optimization/#Generalized-Linear-Mixed-Effects-Models","page":"Details of the parameter estimation","title":"Generalized Linear Mixed-Effects Models","text":"","category":"section"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"In a generalized linear model the responses are modelled as coming from a particular distribution, such as Bernoulli for binary responses or Poisson for responses that represent counts. The scalar distributions of individual responses differ only in their means, which are determined by a linear predictor expression eta=bf Xbeta, where, as before, bf X is a model matrix derived from the values of covariates and beta is a vector of coefficients.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The unconstrained components of eta are mapped to the, possibly constrained, components of the mean response, mu, via a scalar function, g^-1, applied to each component of eta. For historical reasons, the inverse of this function, taking components of mu to the corresponding component of eta is called the link function and the more frequently used map from eta to mu is the inverse link.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A generalized linear mixed-effects model (GLMM) is defined, for the purposes of this package, by","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"beginaligned\n (mathcalY mathcalB=bfb) simmathcalD(bfg^-1(Xbeta + Z b)phi)\n mathcalBsimmathcalN(bf0Sigma_theta) \nendaligned","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"where mathcalD indicates the distribution family parameterized by the mean and, when needed, a common scale parameter, phi. (There is no scale parameter for Bernoulli or for Poisson. Specifying the mean completely determines the distribution.)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"Bernoulli\nPoisson","category":"page"},{"location":"optimization/#Distributions.Bernoulli","page":"Details of the parameter estimation","title":"Distributions.Bernoulli","text":"Bernoulli(p)\n\nA Bernoulli distribution is parameterized by a success rate p, which takes value 1 with probability p and 0 with probability 1-p.\n\nP(X = k) = begincases\n1 - p quad textfor k = 0 \np quad textfor k = 1\nendcases\n\nBernoulli() # Bernoulli distribution with p = 0.5\nBernoulli(p) # Bernoulli distribution with success rate p\n\nparams(d) # Get the parameters, i.e. (p,)\nsuccprob(d) # Get the success rate, i.e. p\nfailprob(d) # Get the failure rate, i.e. 1 - p\n\nExternal links:\n\nBernoulli distribution on Wikipedia\n\n\n\n\n\n","category":"type"},{"location":"optimization/#Distributions.Poisson","page":"Details of the parameter estimation","title":"Distributions.Poisson","text":"Poisson(λ)\n\nA Poisson distribution describes the number of independent events occurring within a unit time interval, given the average rate of occurrence λ.\n\nP(X = k) = fraclambda^kk e^-lambda quad text for k = 012ldots\n\nPoisson() # Poisson distribution with rate parameter 1\nPoisson(lambda) # Poisson distribution with rate parameter lambda\n\nparams(d) # Get the parameters, i.e. (λ,)\nmean(d) # Get the mean arrival rate, i.e. λ\n\nExternal links:\n\nPoisson distribution on Wikipedia\n\n\n\n\n\n","category":"type"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A GeneralizedLinearMixedModel object is generated from a formula, data frame and distribution family.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"verbagg = MixedModels.dataset(:verbagg)\nconst vaform = @formula(r2 ~ 1 + anger + gender + btype + situ + (1|subj) + (1|item));\nmdl = GeneralizedLinearMixedModel(vaform, verbagg, Bernoulli());\ntypeof(mdl)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"A separate call to fit! can be used to fit the model. This involves optimizing an objective function, the Laplace approximation to the deviance, with respect to the parameters, which are beta, the fixed-effects coefficients, and theta, the covariance parameters. The starting estimate for beta is determined by fitting a GLM to the fixed-effects part of the formula","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.β","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"and the starting estimate for theta, which is a vector of the two standard deviations of the random effects, is chosen to be","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.θ","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The Laplace approximation to the deviance requires determining the conditional modes of the random effects. These are the values that maximize the conditional density of the random effects, given the model parameters and the data. This is done using Penalized Iteratively Reweighted Least Squares (PIRLS). In most cases PIRLS is fast and stable. It is simply a penalized version of the IRLS algorithm used in fitting GLMs.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The distinction between the \"fast\" and \"slow\" algorithms in the MixedModels package (nAGQ=0 or nAGQ=1 in lme4) is whether the fixed-effects parameters, beta, are optimized in PIRLS or in the nonlinear optimizer. In a call to the pirls! function the first argument is a GeneralizedLinearMixedModel, which is modified during the function call. (By convention, the names of such mutating functions end in ! as a warning to the user that they can modify an argument, usually the first argument.) The second and third arguments are optional logical values indicating if beta is to be varied and if verbose output is to be printed.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"pirls!(mdl, true, false)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"deviance(mdl)","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.β","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.θ # current values of the standard deviations of the random effects","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"If the optimization with respect to beta is performed within PIRLS then the nonlinear optimization of the Laplace approximation to the deviance requires optimization with respect to theta only. This is the \"fast\" algorithm. Given a value of theta, PIRLS is used to determine the conditional estimate of beta and the conditional mode of the random effects, b.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.b # conditional modes of b","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"fit!(mdl, fast=true);\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The optimization process is summarized by","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl.LMM.optsum\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"As one would hope, given the name of the option, this fit is comparatively fast.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"@btime fit(MixedModel, vaform, verbagg, Bernoulli(), fast=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"The alternative algorithm is to use PIRLS to find the conditional mode of the random effects, given beta and theta and then use the general nonlinear optimizer to fit with respect to both beta and theta.","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"mdl1 = @btime fit(MixedModel, vaform, verbagg, Bernoulli())\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"optimization/","page":"Details of the parameter estimation","title":"Details of the parameter estimation","text":"This fit provided slightly better results (Laplace approximation to the deviance of 8151.400 versus 8151.583) but took 6 times as long. That is not terribly important when the times involved are a few seconds but can be important when the fit requires many hours or days of computing time.","category":"page"},{"location":"rankdeficiency/#Rank-deficiency-in-mixed-effects-models","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"using MixedModels\nusing DisplayAs","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"The (column) rank of a matrix refers to the number of linearly independent columns in the matrix. Clearly, the rank can never be more than the number of columns; however, the rank can be less than the number of columns. In a regression context, this corresponds to a (linear) dependency in the predictors. The simplest case of rank deficiency is a duplicated predictor or a predictor that is exactly a multiple of another predictor. However, rank deficiency can also arise in more subtle ways, such as from missing cells in a two-factor experimental design. Rank deficiency can also arise as an extreme case of multicollinearity. In all cases, it is important to remember that we can only assess the numerical rank of a matrix, which may be less than its theoretical rank, and that evaluation of this numerical rank requires setting some numerical tolerance levels. These choices are not always well defined. In other words, the rank of a matrix is well-defined in theory but in practice can be difficult to evaluate.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Rank deficiency can occur in two ways in mixed-effects models: in the fixed effects and in the random effects. The implications of rank deficiency and thus the handling of it differ between these.","category":"page"},{"location":"rankdeficiency/#Fixed-effects","page":"Rank deficiency in mixed-effects models","title":"Fixed effects","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"The consequences of rank deficiency in the fixed effects are similar to those in classical ordinary least squares (OLS) regression. If one or more predictors can be expressed as a linear combination of the other columns, then this column is redundant and the model matrix is rank deficient. Note however, that the redundant column is not defined uniquely. For example, in the case that of two columns a and b where b = 2a, then the rank deficiency can be handled by eliminating either a or b. While we defined b here in terms of a, it may be that b is actually the more 'fundamental' predictor and hence we may define a in terms of b as a = 0.5b. The user may of course possess this information, but the choice is not apparent to the modelling software. As such, the handling of rank deficiency in MixedModels.jl should not be taken as a replacement for thinking about the nature of the predictors in a given model.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"There is a widely accepted convention for how to make the coefficient estimates for these redundant columns well-defined: we set their value to zero and their standard errors to NaN (and thus also their z and p-values). The values that have been defined to be zero, as opposed to evaluating to zero, are displayed as -0.0 as an additional visual aid to distinguish them from the other coefficients. In practice the determination of rank and the redundant coefficients is done via a 'pivoting' scheme during a decomposition to move the surplus columns to the right side of the model matrix. In subsequent calculations, these columns are effectively ignored (as their estimates are zero and thus won't contribute to any other computations). For display purposes, this pivoting is unwound when the coef values are displayed.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Both the pivoted and unpivoted coefficients are available in MixedModels. The fixef extractor returns the pivoted, truncated estimates (i.e. the non redundant terms), while the coef extractor returns the unpivoted estimates (i.e. all terms, included the redundant ones). The same holds for the associated fixefnames and coefnames.","category":"page"},{"location":"rankdeficiency/#Pivoting-is-platform-dependent","page":"Rank deficiency in mixed-effects models","title":"Pivoting is platform dependent","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"In MixedModels.jl, we use standard numerical techniques to detect rank deficiency. We currently offer no guarantees as to which exactly of the standard techniques (pivoted QR decomposition, pivoted Cholesky decomposition, etc.) will be used. This choice should be viewed as an implementation detail. Similarly, we offer no guarantees as to which of columns will be treated as redundant. This choice may vary between releases and even between platforms (both in broad strokes of \"Linux\" vs. \"Windows\" and at the level of which BLAS options are loaded on a given processor architecture) for the same release. In other words, you should not rely on the order of the pivoted columns being consistent! when you switch to a different computer or a different operating system. If consistency in the pivoted columns is important to you, then you should instead determine your rank ahead of time and remove extraneous columns / predictors from your model specification.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"This lack of consistency guarantees arises from a more fundamental issue: numeric linear algebra is challenging and sensitive to the underlying floating point operations. Due to rounding error, floating point arithmetic is not associative:","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"0.1 + 0.1 + 0.1 - 0.3 == 0.1 + 0.1 + (0.1 - 0.3)","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"This means that \"nearly\" / numerically rank deficient matrices may or may not be detected as rank deficient, depending on details of the platform. Determining the rank of a matrix is the type of problem that is well-defined in theory but not in practice.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Currently, a coarse heuristic is applied to reduce the chance that the intercept column will be pivoted, but even this behavior is not guaranteed.","category":"page"},{"location":"rankdeficiency/#Undetected-Rank-Deficiency","page":"Rank deficiency in mixed-effects models","title":"Undetected Rank Deficiency","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Undetected rank deficiency in the fixed effects will lead to numerical issues, such as nonsensical estimates. A PosDefException may indicate rank deficiency because the covariance matrix will only be positive semidefinite and not positive definite (see Details of the parameter estimation). In other words, checking that the fixed effects are full rank is a great first step in debugging a PosDefException.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Note that PosDefException is not specific to rank deficiency and may arise in other ill-conditioned models. In any case, examining the model specification and the data to verify that they work together is the first step. For generalized linear mixed-effects models, it may also be worthwhile to try out fast=true instead of the default fast=false. See this GitHub issue and linked Discourse discussion for more information.","category":"page"},{"location":"rankdeficiency/#Random-effects","page":"Rank deficiency in mixed-effects models","title":"Random effects","text":"","category":"section"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"Rank deficiency presents less of a problem in the random effects than in the fixed effects because the \"estimates\" (more formally, the conditional modes of the random effects given the observed data) are determined as the solution to a penalized least squares problem. The shrinkage effect which moves the conditional modes (group-level predictions) towards the grand mean is a form of regularization, which provides well-defined \"estimates\" for overparameterized models. (For more reading on this general idea, see also this blog post on the model complexity myth.)","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"The nature of the penalty in the penalized least squares solution is such that the \"estimates\" are well-defined even when the covariance matrix of the random effects converges to a \"singular\" or \"boundary\" value. In other words, singularity of the covariance matrix for the random effects, which means that there are one or more directions in which there is no variability in the random effects, is different from singularity of the model matrix for the random effects, which would affect the ability to define uniquely these coefficients. The penalty term always provides a unique solution for the random-effects coefficients.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"In addition to handling naturally occurring rank deficiency in the random effects, the regularization allows us to fit explicitly overparameterized random effects. For example, we can use fulldummy to fit both an intercept term and n indicator variables in the random effects for a categorical variable with n levels instead of the usual n-1 contrasts.","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"kb07 = MixedModels.dataset(:kb07)\ncontrasts = Dict(var => HelmertCoding() for var in (:spkr, :prec, :load))\nfit(MixedModel, @formula(rt_raw ~ spkr * prec * load + (1|subj) + (1+prec|item)), kb07; contrasts=contrasts)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"fit(MixedModel, @formula(rt_raw ~ spkr * prec * load + (1|subj) + (1+fulldummy(prec)|item)), kb07; contrasts=contrasts)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"rankdeficiency/","page":"Rank deficiency in mixed-effects models","title":"Rank deficiency in mixed-effects models","text":"This may be useful when the PCA property suggests a random effects structure larger than only main effects but smaller than all interaction terms. This is also similar to the functionality provided by dummy in lme4, but as in the difference between zerocorr in Julia and || in R, there are subtle differences in how this expansion interacts with other terms in the random effects.","category":"page"},{"location":"bootstrap/#Parametric-bootstrap-for-mixed-effects-models","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Julia is well-suited to implementing bootstrapping and other simulation-based methods for statistical models. The parametricbootstrap function in the MixedModels package provides an efficient parametric bootstrap for mixed-effects models.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"parametricbootstrap","category":"page"},{"location":"bootstrap/#MixedModels.parametricbootstrap","page":"Parametric bootstrap for mixed-effects models","title":"MixedModels.parametricbootstrap","text":"parametricbootstrap([rng::AbstractRNG], nsamp::Integer, m::MixedModel{T}, ftype=T;\n β = fixef(m), σ = m.σ, θ = m.θ, progress=true, optsum_overrides=(;))\n\nPerform nsamp parametric bootstrap replication fits of m, returning a MixedModelBootstrap.\n\nThe default random number generator is Random.GLOBAL_RNG.\n\nftype can be used to store the computed bootstrap values in a lower precision. ftype is not a named argument because named arguments are not used in method dispatch and thus specialization. In other words, having ftype as a positional argument has some potential performance benefits.\n\nKeyword Arguments\n\nβ, σ, and θ are the values of m's parameters for simulating the responses.\nσ is only valid for LinearMixedModel and GeneralizedLinearMixedModel for\n\nfamilies with a dispersion parameter.\n\nprogress controls whether the progress bar is shown. Note that the progress\n\nbar is automatically disabled for non-interactive (i.e. logging) contexts.\n\noptsum_overrides is used to override values of OptSummary in the models\n\nfit during the bootstrapping process. For example, optsum_overrides=(;ftol_rel=1e-08) reduces the convergence criterion, which can greatly speed up the bootstrap fits. Taking advantage of this speed up to increase n can often lead to better estimates of coverage intervals.\n\nnote: Note\nAll coefficients are bootstrapped. In the rank deficient case, the inestimatable coefficients are treated as -0.0 in the simulations underlying the bootstrap, which will generally result in their estimate from the simulated data also being being inestimable and thus set to -0.0. However this behavior may change in future releases to explicitly drop the extraneous columns before simulation and thus not include their estimates in the bootstrap result.\n\n\n\n\n\n","category":"function"},{"location":"bootstrap/#The-parametric-bootstrap","page":"Parametric bootstrap for mixed-effects models","title":"The parametric bootstrap","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Bootstrapping is a family of procedures for generating sample values of a statistic, allowing for visualization of the distribution of the statistic or for inference from this sample of values.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A parametric bootstrap is used with a parametric model, m, that has been fit to data. The procedure is to simulate n response vectors from m using the estimated parameter values and refit m to these responses in turn, accumulating the statistics of interest at each iteration.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"The parameters of a LinearMixedModel object are the fixed-effects parameters, β, the standard deviation, σ, of the per-observation noise, and the covariance parameter, θ, that defines the variance-covariance matrices of the random effects.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"For example, a simple linear mixed-effects model for the Dyestuff data in the lme4 package for R is fit by","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"using DataFrames\nusing Gadfly # plotting package\nusing MixedModels\nusing Random","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"dyestuff = MixedModels.dataset(:dyestuff)\nm1 = fit(MixedModel, @formula(yield ~ 1 + (1 | batch)), dyestuff)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"To bootstrap the model parameters, first initialize a random number generator then create a bootstrap sample and extract the tbl property, which is a Table - a lightweight dataframe-like object.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"const rng = MersenneTwister(1234321);\nsamp = parametricbootstrap(rng, 10_000, m1);\ntbl = samp.tbl","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A density plot of the estimates of σ, the residual standard deviation, can be created as","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.σ, Geom.density, Guide.xlabel(\"Parametric bootstrap estimates of σ\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"or, for the intercept parameter","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.β1, Geom.density, Guide.xlabel(\"Parametric bootstrap estimates of β₁\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A density plot of the estimates of the standard deviation of the random effects is obtained as","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.σ1, Geom.density,\n Guide.xlabel(\"Parametric bootstrap estimates of σ₁\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Notice that this density plot has a spike, or mode, at zero. Although this mode appears to be diffuse, this is an artifact of the way that density plots are created. In fact, it is a pulse, as can be seen from a histogram.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl.σ1, Geom.histogram,\n Guide.xlabel(\"Parametric bootstrap estimates of σ₁\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"The bootstrap sample can be used to generate intervals that cover a certain percentage of the bootstrapped values. We refer to these as \"coverage intervals\", similar to a confidence interval. The shortest such intervals, obtained with the shortestcovint extractor, correspond to a highest posterior density interval in Bayesian inference.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"shortestcovint","category":"page"},{"location":"bootstrap/#MixedModels.shortestcovint","page":"Parametric bootstrap for mixed-effects models","title":"MixedModels.shortestcovint","text":"shortestcovint(v, level = 0.95)\n\nReturn the shortest interval containing level proportion of the values of v\n\n\n\n\n\nshortestcovint(bsamp::MixedModelFitCollection, level = 0.95)\n\nReturn the shortest interval containing level proportion for each parameter from bsamp.allpars.\n\nwarning: Warning\nCurrently, correlations that are systematically zero are included in the the result. This may change in a future release without being considered a breaking change.\n\n\n\n\n\n","category":"function"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"We generate these directly from the original bootstrap object:","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Table(shortestcovint(samp))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A value of zero for the standard deviation of the random effects is an example of a singular covariance. It is easy to detect the singularity in the case of a scalar random-effects term. However, it is not as straightforward to detect singularity in vector-valued random-effects terms.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"For example, if we bootstrap a model fit to the sleepstudy data","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"sleepstudy = MixedModels.dataset(:sleepstudy)\ncontrasts = Dict(:subj => Grouping())\nm2 = let f = @formula reaction ~ 1+days+(1+days|subj)\n fit(MixedModel, f, sleepstudy; contrasts)\nend","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"samp2 = parametricbootstrap(rng, 10_000, m2);\ntbl2 = samp2.tbl","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"the singularity can be exhibited as a standard deviation of zero or as a correlation of pm1.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"shortestcovint(samp2)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"A histogram of the estimated correlations from the bootstrap sample has a spike at +1.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"plot(x = tbl2.ρ1, Geom.histogram,\n Guide.xlabel(\"Parametric bootstrap samples of correlation of random effects\"))","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"or, as a count,","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(tbl2.ρ1 .≈ 1)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Close examination of the histogram shows a few values of -1.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(tbl2.ρ1 .≈ -1)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Furthermore there are even a few cases where the estimate of the standard deviation of the random effect for the intercept is zero.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(tbl2.σ1 .≈ 0)","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"There is a general condition to check for singularity of an estimated covariance matrix or matrices in a bootstrap sample. The parameter optimized in the estimation is θ, the relative covariance parameter. Some of the elements of this parameter vector must be non-negative and, when one of these components is approximately zero, one of the covariance matrices will be singular.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"The issingular method for a MixedModel object that tests if a parameter vector θ corresponds to a boundary or singular fit.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"This operation is encapsulated in a method for the issingular function.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"count(issingular(samp2))","category":"page"},{"location":"bootstrap/#Reduced-Precision-Bootstrap","page":"Parametric bootstrap for mixed-effects models","title":"Reduced Precision Bootstrap","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"parametricbootstrap accepts an optional keyword argument optsum_overrides, which can be used to override the convergence criteria for bootstrap replicates. One possibility is setting ftol_rel=1e-8, i.e., considering the model converged when the relative change in the objective between optimizer iterations is smaller than 0.00000001. This threshold corresponds approximately to the precision from treating the value of the objective as a single precision (Float32) number, while not changing the precision of the intermediate computations. The resultant loss in precision will generally be smaller than the variation that the bootstrap captures, but can greatly speed up the fitting process for each replicates, especially for large models. More directly, lowering the fit quality for each replicate will reduce the quality of each replicate, but this may be more than compensated for by the ability to fit a much larger number of replicates in the same time.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"t = @timed parametricbootstrap(MersenneTwister(42), 1000, m2; progress=false)\nt.time","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"optsum_overrides = (; ftol_rel=1e-8)\nt = @timed parametricbootstrap(MersenneTwister(42), 1000, m2; optsum_overrides, progress=false)\nt.time","category":"page"},{"location":"bootstrap/#Distributed-Computing-and-the-Bootstrap","page":"Parametric bootstrap for mixed-effects models","title":"Distributed Computing and the Bootstrap","text":"","category":"section"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"Earlier versions of MixedModels.jl supported a multi-threaded bootstrap via the use_threads keyword argument. However, with improved BLAS multithreading, the Julia-level threads often wound up competing with the BLAS threads, leading to no improvement or even a worsening of performance when use_threads=true. Nonetheless, the bootstrap is a classic example of an embarrassingly parallel problem and so we provide a few convenience methods for combining results computed separately. In particular, there are vcat and an optimized reduce(::typeof(vcat)) methods for MixedModelBootstrap objects. For computers with many processors (as opposed to a single processor with several cores) or for computing clusters, these provide a convenient way to split the computation across nodes.","category":"page"},{"location":"bootstrap/","page":"Parametric bootstrap for mixed-effects models","title":"Parametric bootstrap for mixed-effects models","text":"using Distributed\n# you already have 1 proc by default, so add the number of additional cores with `addprocs`\n# you need at least as many RNGs as cores you want to use in parallel\n# but you shouldn't use all of your cores because nested within this\n# is the multithreading of the linear algebra\n# addprocs(1)\n@info \"Currently using $(nprocs()) processors total and $(nworkers()) for work\"\n\n# Load the necessary packages on all workers\n# For clusters, you will also need to make sure that the Julia\n# environment (Project.toml) is set up and activated on each worker.\n@everywhere begin\n using ProgressMeter\n using MixedModels\nend\n# copy everything to workers\n@showprogress for w in workers()\n remotecall_fetch(() -> coefnames(m2), w)\nend\n\n# split the replicates across the workers\n# this rounds down, so if the number of workers doesn't divide the\n# number of replicates, you'll be a few replicates short!\nn_replicates = 1000\nn_rep_per_worker = n_replicates ÷ nworkers()\n# NB: You need a different seed/RNG for each worker otherwise you will\n# have copies of the same replicates and not independent replicates!\npb_map = @showprogress pmap(MersenneTwister.(1:nworkers())) do rng\n parametricbootstrap(rng, n_rep_per_worker, m2; optsum_overrides)\nend;\n\n# get rid of all the workers\n# rmprocs(workers())\n\nconfint(reduce(vcat, pb_map))","category":"page"},{"location":"prediction/#Prediction-and-simulation-in-Mixed-Effects-Models","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"","category":"section"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"We recommend the MixedModelsSim.jl package and associated documentation for useful tools in constructing designs to simulate. For now, we'll use the sleep study data as a starting point.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"using DataFrames\nusing MixedModels\nusing StatsBase\nusing DisplayAs # hide\n# use a DataFrame to make it easier to change things later\nslp = DataFrame(MixedModels.dataset(:sleepstudy))\nslpm = fit(MixedModel, @formula(reaction ~ 1 + days + (1|subj)), slp)\nDisplayAs.Text(slpm) # hide","category":"page"},{"location":"prediction/#Prediction","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction","text":"","category":"section"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"The simplest form of prediction are the fitted values from the model: they are indeed the model's predictions for the observed data.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"When generalizing to new data, we need to consider what happens if there are new, previously unobserved levels of the grouping variable(s). MixedModels.jl provides three options:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":":error: error on encountering unobserved levels\n:population: use population values (i.e. only the fixed effects) for observations with unobserved levels\n:missing: return missing for observations with unobserved levels.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Providing either no prediction (:error, :missing) or providing the population-level values seem to be the most reasonable ways for predicting new values. For simulating new values based on previous estimates of the variance components, use simulate.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"In the case where there are no new levels of the grouping variable, all three of these methods provide the same results:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp; new_re_levels=:population) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp; new_re_levels=:missing) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp; new_re_levels=:error) ≈ fitted(slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"In the case where there are new levels of the grouping variable, these methods differ.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"# create a new level\nslp2 = transform(slp, :subj => ByRow(x -> (x == \"S308\" ? \"NEW\" : x)) => :subj)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"try\n predict(slpm, slp2; new_re_levels=:error)\ncatch e\n show(e)\nend","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp2; new_re_levels=:missing)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"predict(slpm, slp2; new_re_levels=:population)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"note: Note\nCurrently, we do not support predicting based on a subset of the random effects.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"note: Note\npredict is deterministic (within the constraints of floating point) and never adds noise to the result. If you want to construct prediction intervals, then simulate will generate new data with noise (including new values of the random effects).","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"For generalized linear mixed models, there is an additional keyword argument to predict: type specifies whether the predictions are returned on the scale of the linear predictor (:linpred) or on the level of the response (:response) (i.e. the level at which the values were originally observed).","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"cbpp = DataFrame(MixedModels.dataset(:cbpp))\ncbpp.rate = cbpp.incid ./ cbpp.hsz\ngm = fit(MixedModel, @formula(rate ~ 1 + period + (1|herd)), cbpp, Binomial(), wts=float(cbpp.hsz))\npredict(gm, cbpp; type=:response) ≈ fitted(gm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"logit(x) = log(x / (1 - x))\npredict(gm, cbpp; type=:linpred) ≈ logit.(fitted(gm))","category":"page"},{"location":"prediction/#Simulation","page":"Prediction and simulation in Mixed-Effects Models","title":"Simulation","text":"","category":"section"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"In contrast to predict, simulate and simulate! introduce randomness. This randomness occurs both at the level of the observation-level (residual) variance and at the level of the random effects, where new conditional modes are sampled based on the specified covariance parameter (θ; see Details of the parameter estimation), which defaults to the estimated value of the model. For reproducibility, we specify a pseudorandom generator here; if none is provided, the global PRNG is taken as the default.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"The simplest example of simulate takes a fitted model and generates a new response vector based on the existing model matrices combined with noise.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"using Random\nynew = simulate(MersenneTwister(42), slpm)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"The simulated response can also be placed in a pre-allocated vector:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"ynew2 = zeros(nrow(slp))\nsimulate!(MersenneTwister(42), ynew2, slpm)\nynew2 ≈ ynew","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Or even directly replace the previous response vector in a model, at which point the model must be refit to the new values:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"slpm2 = deepcopy(slpm)\nrefit!(simulate!(MersenneTwister(42), slpm2))\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"This inplace simulation actually forms the basis of parametricbootstrap.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Finally, we can also simulate the response from entirely new data.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"df = DataFrame(days = repeat(1:10, outer=20), subj=repeat(1:20, inner=10))\ndf[!, :subj] = string.(\"S\", lpad.(df.subj, 2, \"0\"))\ndf[!, :reaction] .= 0\ndf\nDisplayAs.Text(df) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"ysim = simulate(MersenneTwister(42), slpm, df)","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"Note that this is a convenience method for creating a new model and then using the parameters from the old model to call simulate on that model. In other words, this method incurs the cost of constructing a new model and then discarding it. If you could re-use that model (e.g., fitting that model as part of a simulation study), it often makes sense to do these steps to perform these steps explicitly and avoid the unnecessary construction and discarding of an intermediate model:","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"msim = LinearMixedModel(@formula(reaction ~ 1 + days + (1|subj)), df)\nsimulate!(MersenneTwister(42), msim; θ=slpm.θ, β=slpm.β, σ=slpm.σ)\nresponse(msim) ≈ ysim","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"fit!(msim)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"For simulating from generalized linear mixed models, there is no type option because the observation-level always occurs at the level of the response and not of the linear predictor.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"warning: Warning\nSimulating the model response in place may not yield the same result as simulating into a pre-allocated or new vector, depending on choice of pseudorandom number generator. Random number generation in Julia allows optimization based on type, and the internal storage type of the model response (currently a view into a matrix storing the concatenated fixed-effects model matrix and the response) may not match the type of a pre-allocated or new vector. See also discussion here.","category":"page"},{"location":"prediction/","page":"Prediction and simulation in Mixed-Effects Models","title":"Prediction and simulation in Mixed-Effects Models","text":"note: Note\nAll the methods that take new data as a table construct an additional MixedModel behind the scenes, even when the new data is exactly the same as the data that the model was fitted to. For the simulation methods in particular, these thus form a convenience wrapper for constructing a new model and calling simulate without new data on that model with the parameters from the original model.","category":"page"},{"location":"GaussHermite/#Normalized-Gauss-Hermite-Quadrature","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"","category":"section"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Gaussian Quadrature rules provide sets of x values, called abscissae, and corresponding weights, w, to approximate an integral with respect to a weight function, g(x). For a kth order rule the approximation is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"int f(x)g(x)dx approx sum_i=1^k w_i f(x_i)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"For the Gauss-Hermite rule the weight function is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"g(x) = e^-x^2","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"and the domain of integration is (-infty infty). A slight variation of this is the normalized Gauss-Hermite rule for which the weight function is the standard normal density","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"g(z) = phi(z) = frace^-z^22sqrt2pi","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Thus, the expected value of f(z), where mathcalZsimmathscrN(01), is approximated as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"mathbbEf=int_-infty^infty f(z) phi(z)dzapproxsum_i=1^k w_if(z_i) ","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Naturally, there is a caveat. For the approximation to be accurate the function f(z) must behave like a low-order polynomial over the range of interest. More formally, a kth order rule is exact when f is a polynomial of order 2k-1 or less. [1]","category":"page"},{"location":"GaussHermite/#Evaluating-the-weights-and-abscissae","page":"Normalized Gauss-Hermite Quadrature","title":"Evaluating the weights and abscissae","text":"","category":"section"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"In the Golub-Welsch algorithm the abscissae for a particular Gaussian quadrature rule are determined as the eigenvalues of a symmetric tri-diagonal matrix and the weights are derived from the squares of the first row of the matrix of eigenvectors. For a kth order normalized Gauss-Hermite rule the tridiagonal matrix has zeros on the diagonal and the square roots of 1:k-1 on the super- and sub-diagonal, e.g.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using DisplayAs","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using DataFrames, LinearAlgebra, Gadfly\nsym3 = SymTridiagonal(zeros(3), sqrt.(1:2))\nev = eigen(sym3);\nev.values","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"abs2.(ev.vectors[1,:])","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"As a function of k this can be written as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"function gausshermitenorm(k)\n ev = eigen(SymTridiagonal(zeros(k), sqrt.(1:k-1)))\n ev.values, abs2.(ev.vectors[1,:])\nend;","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"providing","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"gausshermitenorm(3)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The weights and positions are often shown as a lollipop plot. For the 9th order rule these are","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"gh9=gausshermitenorm(9)\nplot(x=gh9[1], y=gh9[2], Geom.hair, Geom.point, Guide.ylabel(\"Weight\"), Guide.xlabel(\"\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Notice that the magnitudes of the weights drop quite dramatically away from zero, even on a logarithmic scale","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(\n x=gh9[1], y=gh9[2], Geom.hair, Geom.point,\n Scale.y_log2, Guide.ylabel(\"Weight (log scale)\"),\n Guide.xlabel(\"\"),\n)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The definition of MixedModels.GHnorm is similar to the gausshermitenorm function with some extra provisions for ensuring symmetry of the abscissae and the weights and for caching values once they have been calculated.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"GHnorm","category":"page"},{"location":"GaussHermite/#MixedModels.GHnorm","page":"Normalized Gauss-Hermite Quadrature","title":"MixedModels.GHnorm","text":"GHnorm(k::Int)\n\nReturn the (unique) GaussHermiteNormalized{k} object.\n\nThe function values are stored (memoized) when first evaluated. Subsequent evaluations for the same k have very low overhead.\n\n\n\n\n\n","category":"function"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using MixedModels\nGHnorm(3)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"By the properties of the normal distribution, when mathcalXsimmathscrN(mu sigma^2)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"mathbbEg(x) approx sum_i=1^k g(mu + sigma z_i)w_i","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"For example, mathbbEmathcalX^2 where mathcalXsimmathcalN(2 3^2) is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"μ = 2; σ = 3; ghn3 = GHnorm(3);\nsum(@. ghn3.w * abs2(μ + σ * ghn3.z)) # should be μ² + σ² = 13","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"(In general a dot, '.', after the function name in a function call, as in abs2.(...), or before an operator creates a fused vectorized evaluation in Julia. The macro @. has the effect of vectorizing all operations in the subsequent expression.)","category":"page"},{"location":"GaussHermite/#Application-to-a-model-for-contraception-use","page":"Normalized Gauss-Hermite Quadrature","title":"Application to a model for contraception use","text":"","category":"section"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A binary response is a \"Yes\"/\"No\" type of answer. For example, in a 1989 fertility survey of women in Bangladesh (reported in Huq, N. M. and Cleland, J., 1990) one response of interest was whether the woman used artificial contraception. Several covariates were recorded including the woman's age (centered at the mean), the number of live children the woman has had (in 4 categories: 0, 1, 2, and 3 or more), whether she lived in an urban setting, and the district in which she lived. The version of the data used here is that used in review of multilevel modeling software conducted by the Center for Multilevel Modelling, currently at University of Bristol (http://www.bristol.ac.uk/cmm/learning/mmsoftware/data-rev.html). These data are available as the :contra dataset.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"contra = DataFrame(MixedModels.dataset(:contra))\ndescribe(contra)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A smoothed scatterplot of contraception use versus age","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(contra, x=:age, y=:use, Geom.smooth, Guide.xlabel(\"Centered age (yr)\"),\n Guide.ylabel(\"Contraception use\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"shows that the proportion of women using artificial contraception is approximately quadratic in age.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A model with fixed-effects for age, age squared, number of live children and urban location and with random effects for district, is fit as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"const form1 = @formula use ~ 1 + age + abs2(age) + livch + urban + (1|dist);\nm1 = fit(MixedModel, form1, contra, Bernoulli(), fast=true)\nDisplayAs.Text(ans) # hide","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"For a model such as m1, which has a single, scalar random-effects term, the unscaled conditional density of the spherical random effects variable, mathcalU, given the observed data, mathcalY=mathbfy_0, can be expressed as a product of scalar density functions, f_i(u_i) i=1dotsq. In the PIRLS algorithm, which determines the conditional mode vector, tildemathbfu, the optimization is performed on the deviance scale,","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"D(mathbfu)=-2sum_i=1^q log(f_i(u_i))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The objective, D, consists of two parts: the sum of the (squared) deviance residuals, measuring fidelity to the data, and the squared length of mathbfu, which is the penalty. In the PIRLS algorithm, only the sum of these components is needed. To use Gauss-Hermite quadrature the contributions of each of the u_ii=1dotsq should be separately evaluated.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"const devc0 = map!(abs2, m1.devc0, m1.u[1]); # start with uᵢ²\nconst devresid = m1.resp.devresid; # n-dimensional vector of deviance residuals\nconst refs = only(m1.LMM.reterms).refs; # n-dimensional vector of indices in 1:q\nfor (dr, i) in zip(devresid, refs)\n devc0[i] += dr\nend\nshow(devc0)","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"One thing to notice is that, even on the deviance scale, the contributions of different districts can be of different magnitudes. This is primarily due to different sample sizes in the different districts.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using FreqTables\nfreqtable(contra, :dist)'","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"Because the first district has one of the largest sample sizes and the third district has the smallest sample size, these two will be used for illustration. For a range of u values, evaluate the individual components of the deviance and store them in a matrix.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"const devc = m1.devc;\nconst xvals = -5.0:2.0^(-4):5.0;\nconst uv = vec(m1.u[1]);\nconst u₀ = vec(m1.u₀[1]);\nresults = zeros(length(devc0), length(xvals))\nfor (j, u) in enumerate(xvals)\n fill!(devc, abs2(u))\n fill!(uv, u)\n MixedModels.updateη!(m1)\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), devc)\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"A plot of the deviance contribution versus u_1","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line, Guide.xlabel(\"u₁\"),\n Guide.ylabel(\"Deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"shows that the deviance contribution is very close to a quadratic. This is also true for u_3","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line, Guide.xlabel(\"u₃\"),\n Guide.ylabel(\"Deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The PIRLS algorithm provides the locations of the minima of these scalar functions, stored as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"m1.u₀[1]","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"the minima themselves, evaluated as devc0 above, and a horizontal scale, which is the inverse of diagonal of the Cholesky factor. As shown below, this is an estimate of the conditional standard deviations of the components of mathcalU.","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"using MixedModels: block\nconst s = inv.(m1.LMM.L[block(1,1)].diag);\ns'","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"The curves can be put on a common scale, corresponding to the standard normal, as","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"for (j, z) in enumerate(xvals)\n @. uv = u₀ + z * s\n MixedModels.updateη!(m1)\n @. devc = abs2(uv) - devc0\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), devc)\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₁\"),\n Guide.ylabel(\"Shifted deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₃\"),\n Guide.ylabel(\"Shifted deviance contribution\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"On the original density scale these become","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"for (j, z) in enumerate(xvals)\n @. uv = u₀ + z * s\n MixedModels.updateη!(m1)\n @. devc = abs2(uv) - devc0\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), @. exp(-devc/2))\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₁\"),\n Guide.ylabel(\"Conditional density\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₃\"),\n Guide.ylabel(\"Conditional density\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"and the function to be integrated with the normalized Gauss-Hermite rule is","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"for (j, z) in enumerate(xvals)\n @. uv = u₀ + z * s\n MixedModels.updateη!(m1)\n @. devc = abs2(uv) - devc0\n for (dr, i) in zip(devresid, refs)\n devc[i] += dr\n end\n copyto!(view(results, :, j), @. exp((abs2(z) - devc)/2))\nend","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 1, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₁\"), Guide.ylabel(\"Kernel ratio\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"plot(x=xvals, y=view(results, 3, :), Geom.line,\n Guide.xlabel(\"Scaled and shifted u₃\"), Guide.ylabel(\"Kernel ratio\"))","category":"page"},{"location":"GaussHermite/","page":"Normalized Gauss-Hermite Quadrature","title":"Normalized Gauss-Hermite Quadrature","text":"[1]: https://en.wikipedia.org/wiki/Gaussian_quadrature","category":"page"},{"location":"#MixedModels.jl-Documentation","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"","category":"section"},{"location":"","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"CurrentModule = MixedModels","category":"page"},{"location":"","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"MixedModels.jl is a Julia package providing capabilities for fitting and examining linear and generalized linear mixed-effect models. It is similar in scope to the lme4 package for R.","category":"page"},{"location":"","page":"MixedModels.jl Documentation","title":"MixedModels.jl Documentation","text":"Pages = [\n \"constructors.md\",\n \"optimization.md\",\n \"GaussHermite.md\",\n \"bootstrap.md\",\n \"rankdeficiency.md\",\n \"mime.md\",\n]\nDepth = 2","category":"page"}] }