Getting at posterior trial-by-trial

[DavidBrainard]

Correspondence with David Jones:

Hi David,

Some pointers for something equivalent to QuestPdf would be very
helpful, (even if it only works in limited situations where the slope
and lapse rate are held constant). And of course, if I could generate
the PDF with sufficiently fine resolution, I could calculate the 90%
confidence interval directly from the PDF.

thanks,
David

On Mon, Oct 1, 2018 at 4:25 PM Brainard, David H
[email protected] wrote:

There are not currently replacement calls for QuestPdf or QuestQuantile.

The posterior density over the grid of parameters is updated and stored
after every trial, so that info is hiding indeed hiding in the structure that
gets passed around. When I get a moment I will jot down some
notes about what field it is and what the format is. I think it would be
pretty easy to write an equivalent for QuestPdf and QuestQuantile for
cases where there is one parameter, or where you cared about the
marginal distribution over a single parameter.

DB

On 10/1/18, 2:12 PM, "David Jones" [email protected] wrote:

 Hi David,

 I think I have an example working by following your example code.

 In the older version of Quest, I often found it useful to use ...
 QuestPdf and QuestQuantile ... to plot the probability density
 function of the threshold, and calculate the 90% confidence interval
 for the threshold, as a reality check the experiment went well.  If
 the confidence interval was unusually large, it might have been a bad
 run for some reason.  I would also run simulations before an
 experiment to estimate what the "expected" confidence interval would
 be, based on a "typical" model subject's threshold, so I could plan
 whether I should use 40, 50, 60 trials to get a suitably accurate
 threshold estimate.  For students, I also found it instructive to plot
 the PDF of the threshold estimate after each trial to illustrate how
 the Quest algorithm was getting a tighter estimate each step.

 Can you point me to any example code that would help me do these 2
 things with QuestPlus ?

 What are replacements for calls to QuestPdf and QuestQuantile ?
 How can I get a 90% confidence interval on the threshold (... or more
 generally, on any psi parameter)
 How can I get a probability density function for the threshold?

 Any pointers you can provide would be much appreciated!

 Thanks!
 David

[DavidBrainard]

If you run qpQuestPlusCoreFunctionDemo and put a break point on line 43, you can watch the questData structure change trial-by-trial.

There are two key fields. questData.psiParamsDomain is a matrix with column dimension equal to the number of parameters in the psychometric function, and number of rows equal to the number of combinations of the parameters in the parameter grid (all possible combinations of the parameters). Each row gives one set of parameters. Lets call the number of row nCombinations.

questData.posterior is an nCombinations by 1 vector where each entry is the posterior probability for the set of parameters in the corresponding row of questData.psiParamsDomain.

This info should let you write a qp equivalent for QuestPdf (easy) and QuestQuantile (pretty easy).

If you do, I can review and fold back into the master copy.