Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prediction intervals for mixed effects models #33

Open
SBuckerfield opened this issue Sep 27, 2019 · 3 comments
Open

Prediction intervals for mixed effects models #33

SBuckerfield opened this issue Sep 27, 2019 · 3 comments
Labels
question Further information is requested

Comments

@SBuckerfield
Copy link

What methods do people use to generate prediction intervals for mixed effects models?
I just used 'predictInterval' from the merTools package to produce 95% prediction intervals and didn't exactly get what I was hoping for.

I have two continuous predictors, one categorical variable, and two random effects that allow random intercepts, in this format:

ME<- lmer (Dependent_Variable ~ Categorical_pred+ scale(Continuous_pred1)+scale(Continuous_pred2)+(1|Random1)+(1|Random2))

I am interested in showing the prediction interval for one of the continuous predictors for each of the three levels of the categorical variable, whilst keeping the other continuous predictor constant and taking one group from each of the two random effects which has a mean close to the mean of the all the random effect group means.

I generated a new set of values from which to make predictions:
NEW<-expand.grid(Continuous_pred2=8.01,
Continuous_pred1=seq(from=0,to=10,by=0.01), Random1="Group7",
Random2="Group10", Categorical_pred=c("Cat1","Cat2","Cat3"))

I then used predictInterval to generate the prediction intervals with the new dataset using the model ME:

pred <- predictInterval(ME, NEW,
n.sims = 10000,
level = 0.95,
stat = 'mean')

When I then plot the results of this with the raw data, it looks ok (what I'd expect for a prediction interval) apart from the lines being very 'shaky', when I was hoping for smooth. If I increase the n.sims or discretisation of the continuous predictor in the new dataset from which the predictions are made they become smoother but based on the examples I've looked at I already have a lot of simulations....any advice/observations of some simple step I've missed? There seem to be multiple ways of doing this as I have discovered on stack exchange but this seemed perhaps the most simple, in principle.

@SBuckerfield SBuckerfield added the question Further information is requested label Sep 27, 2019
@jejoenje
Copy link
Member

jejoenje commented Oct 2, 2019

Hi, there are a few different ways of calculating prediction intervals for GLMMs.
I went through a few in the "Predicting from GLMMs" session last year (repo here). With multiple mixed effects, it's far from trivial to decide what is best and it very much depends on what you are trying to do.

Its hard to comment on your specific case without seeing the actual problem (not sure how to interpret "shaky lines"). Could you post a reproducible example?

@jejoenje
Copy link
Member

jejoenje commented Oct 2, 2019

Sorry I've just realised the link above doesn't have what I thought it did - bear with me while I find the right presentation.

@jejoenje
Copy link
Member

jejoenje commented Oct 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants