synthsumm
Check out the SynthSumm collection for some (relatively) new model ideas:
Synthsumm is an experiment in harnessing the power of large language models (LLMs) to distill summarization power into a more compact and efficient text-to-text model. This model is fine-tuned exclusively on the synthsumm dataset, which consists of synthetic summaries generated by the GPT-3.5-turbo-1106 model and "curated" long context text from various domains, including "random" examples from pre-training datasets.
The goal of this fine-tuning approach is to leverage the high-quality summarization features of LLMs and embed them into a model that can handle long-context information, thereby providing robust summarization results without the need for extensive computational resources.
It turns out the models finetuned on it are pretty good, and require substantially less training examples than, say, the booksum models. Check it out and let me know what you think!