Cookbook for DSPy with multimodal support

[Ranking666]

Are there any examples that support local multimodal large models? Including data format, model loading method

[pavan4]

Check #1495

[Ranking666]

Is there a complete example?

[isaacbmiller]

For now there isn't a great cookbook. MMMU.ipynb shows the basics.

I am working on a better cookbook. If there is something in particular you want to see let me know and I can see how to fit it in

[thiccvitalik]

An example with a list of input images, where the list size can vary, would be great.

[isaacbmiller]

What's the downstream use case here? (Not disagreeing just curious)

[thiccvitalik]

My use case is multi-page PDFs where the number of pages can vary, so I can’t predetermine a fixed number of input images (I convert multiple pages to a single image, but this still requires a flexible list size).

[jkpjkpjkp]

hi! am very curious about the followups on this one. did you solve it?

upd: i found this '#1801' and '#1763', the former being a work-in-progress and the latter handles the list case. will try them out. cheers~

upd2: pr/1801 worked like a charm. 1763 has issues when merged into latest.

[okhat]

@jkpjkpjkp You don't need any PRs for multi-modal support now. It's merged in the library.