SDSignal Desk

IBM’s Palette shows a 20B open model can build on-brand slide decks nearly as well as NotebookLM

· Updated · Signal Desk newsroom

Analysis of reporting by Hugging Face. How we work

Image: Hugging Face

IBM Research fine-tuned a 20B model to follow the IBM Design Language, scoring 59.7 on PresentBench against NotebookLM’s 60.8. A render-and-check loop that keeps only verified improvements adds 3.5 points. Invented facts remain the hardest failure, and the benchmark doesn’t test brand adherence.

Key takeaways

  • Palette fine-tunes a 20B model with a LoRA adapter on synthetic IBM Design Language decks.
  • It ranked second overall on PresentBench, 59.7 versus NotebookLM’s 60.8, and first on Economics and Education.
  • A designer role writes an editable brief; a coder role writes each slide in parallel.
  • A checker keeps editor rewrites only if they improve overlaps, clipping or overflow, adding 3.5 points.
  • PresentBench doesn’t measure adherence to a specific design language.

Why decks are hard

Slide decks sound trivial until a company wants every one on its own template, or the content is too sensitive for an outside model. Frontier systems drift back to their house style; small self-hosted models usually produce broken slides with vanishing titles, overflowing text and stacked shapes. Palette, from IBM Research, fine-tunes a 20B model with a LoRA adapter on synthetic decks following the IBM Design Language. On PresentBench it ranks first on Economics and Education, third on Talk and second overall, 59.7 against NotebookLM’s 60.8. If that holds, owning a deck model that speaks your brand becomes realistic for organizations that can’t send content to a frontier API, part of a wider run of capable small open models (apex-flash-1, Mellum2.1).

How it’s built

Designer role
Writes a structured brief: palette, type, layout and a short rationale per slide, readable and editable so one slide can change without regenerating the deck.
Coder role
Writes each slide’s code in parallel, one call per slide.
Steering
Short bracketed directives from 12 categories such as layout, tone, audience and emphasis.
Training data
About 1,000 synthesized decks, roughly 10,000 slides, each with counterfactual reasoning about a rejected alternative.
Checker loop
Every slide is rendered and measured; flagged slides go to an editor model, and rewrites are kept only if the checker confirms improvement. Across 131 decks it adds 3.5 points.

The hard failure

A fix that can’t make things worse can run on everything, which makes the harness the part worth copying for charts, diagrams and UI layouts. The team is candid about the hardest failure: slides containing facts never in the input. Prompt rules barely moved the teacher model; fixing the reference decks did, a sharp lesson about data over instructions. Hallucinated numbers in a board deck are real harm, and a geometry checker can’t catch them, so fact tracing, which the team names as a next signal, is where effort should go.

Caveats

The training data is synthetic and generated by a large open teacher, so Palette’s taste is inherited. In benchmark runs, a separate frontier model drafted each plan from the source document before Palette built the deck. And PresentBench doesn’t measure adherence to a specific design language, the headline promise, as the authors acknowledge. Until that gap closes, “your design language” is a claim to test. The team plans to use checker signals as reinforcement learning rewards and is exploring a bring-your-own-template approach from existing .pptx files. More in our research coverage and tools coverage.

Who feels it

Enterprises with brand or confidentiality limits
A path to on-prem, on-template deck generation.
ML practitioners
Render-check-repair loops transfer to other visual outputs.
Presentation tool vendors
A 20B open model narrows the moat.

What to watch

  1. Checker signals used as RL rewards.
  2. A benchmark measuring design-language adherence.
  3. Bring-your-own-template from .pptx files.
  4. Figure extraction for figure-heavy decks.

FAQ

What is IBM Palette?

An IBM Research method fine-tuning a 20B open model to build slide decks in a company’s design language.

How does Palette compare to NotebookLM?

It scored 59.7 on PresentBench versus NotebookLM’s 60.8, ranking second overall.

Can AI slide tools invent facts?

Yes. IBM found invented facts were the hardest failure, improved mainly by fixing reference training decks.

Read the original

Continue at the source.

Hugging Face

More on AI

All AI coverage →