resolves #796 remove dep on pygam, use IsotonicRegression

[ras44]

Proposed changes

This removes the dependency on pygam and replaces the use of LogisticGAM() in causalml/propensity.py : calibrate() with IsotonicRegression(). Work shown here supports that using IsotonicRegression() produces comparable or improves log-loss and brier score loss values.

This removes the pygam dependency, which is acting as a constraint on other dependencies.

Types of changes

What types of changes does your code introduce to CausalML?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• I have read the CONTRIBUTING doc
• I have signed the CLA
• Lint and unit tests pass locally with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)
• Any dependent changes have been merged and published in downstream modules

[jeongyoonlee]

Thanks for the quick turnaround, @ras44. The test is failing, I guess, because the new calibrated propensity contains 0 or 1. Propensity scores, or treatment probabilities, should be within (0, 1), excluding 0 or 1, and binomial-like outputs are not ideal.

[jeongyoonlee]

I'm reposting my message from Slack. ICYMI: In your example notebook, treatment is supposed to be assigned randomly with a 75% probability. So, the logistic regression propensity and PyGAM calibrated scores look better than other calibrated scores, which show binomial distributions.

For a more interesting test, we should try a propensity model, of which outputs are not probabilities, e.g., GBM or RF, and then see how PyGAM calibration changes the distribution and find the calibration method that generates similar outputs.

A simple logistic regression - or sigmoid transformation might work better.

[ras44]

My apologies, I had notifications turned off. Will take a more detailed look at it 👍

[ras44]

@jeongyoonlee

I've added a new notebook in a dev branch that works through the calibration question at:

https://github.com/ras44/causalml/blob/ras44/remove_pygam_dep_796_dev/docs/examples/pygam_removal_and_testing_250121.ipynb

This notebook builds on the example from: https://scikit-learn.org/stable/auto_examples/calibration/plot_calibration.html#sphx-glr-auto-examples-calibration-plot-calibration-py

It reproduces that example and then adds examples to compare calibration using both pyGAM and Isotonic Regression.

Interestingly, calibration with isotonic regression seems to perform better at the discontinuities, with pyGAM showing high-frequency like effects:

This should help to motivate the transition to IsotonicRegression from pyGAM.

There are also a couple other modifications in the example code for compute_propensity_score(...):

• I would be inclined to return a null model if calibration is requested, because the model is not calibrated, only the scores.
• I've added a try/except to use predict_proba so that it can accept models like GaussianNB, but not sure if this handles all uses cases cleanly