Skip to content

Studio: decide diffusion routing before the SWA resolver#6299

Merged
danielhanchen merged 2 commits into
mainfrom
studio-diffusion-detect-before-swa-resolver
Jun 13, 2026
Merged

Studio: decide diffusion routing before the SWA resolver#6299
danielhanchen merged 2 commits into
mainfrom
studio-diffusion-detect-before-swa-resolver

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Problem

Loading a DiffusionGemma GGUF in Studio could fail with llama-server failed to start. Check that the GGUF file is valid, even though llama-diffusion-cli runs the same model fine.

_read_gguf_metadata assigned self._is_diffusion after calling _resolve_swa_pattern, both inside one try/except. The resolver reaches into transformers/HF, which can raise for an architecture transformers does not know (diffusion-gemma). When it raised, the shared except swallowed it and left _is_diffusion False, so the model was routed to plain llama-server (which cannot serve a block-diffusion GGUF) instead of the diffusion runner. It only reproduced on hosts where the SWA pattern was not already cached or inline in the GGUF.

Fix

  • Set _is_diffusion right after the KV parse loop (it only needs arch / canvas_seen), before the resolver, so a resolver error can no longer drop the diffusion routing.
  • Skip the SWA resolver for diffusion models: they do not use Studio's SWA pattern (the visual server reads its own metadata), and it can raise for them.

Verification

Monkeypatching _resolve_swa_pattern to raise on a real DiffusionGemma GGUF: _is_diffusion is now True (was False), the resolver is invoked 0 times, and the remaining metadata (context_length, chat_template, reasoning, tools) is still read. No change for non-diffusion models.

danielhanchen and others added 2 commits June 13, 2026 11:14
Loading a DiffusionGemma GGUF could fail with "llama-server failed to start. Check that the GGUF file is valid" while llama-diffusion-cli ran the same model fine.

_read_gguf_metadata set self._is_diffusion after calling _resolve_swa_pattern, both inside one try/except. The resolver reaches into transformers/HF, which can raise for an architecture transformers does not know (diffusion-gemma); the shared except then swallowed it and left _is_diffusion False, so the model was routed to plain llama-server instead of the diffusion runner. It only reproduced where the SWA pattern was not already cached/inline.

Set _is_diffusion right after the KV parse loop (before the resolver) so a resolver error can no longer drop the routing, and skip the resolver for diffusion models, which do not use Studio's SWA pattern.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the _read_gguf_metadata method in llama_cpp.py to determine whether a model is a diffusion model earlier in the metadata parsing process. It also skips the sliding window attention (SWA) resolver for diffusion models to prevent potential exceptions on architectures that the resolver does not recognize. There are no review comments to assess, so no feedback is provided.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@danielhanchen danielhanchen merged commit a9a38da into main Jun 13, 2026
32 checks passed
@danielhanchen danielhanchen deleted the studio-diffusion-detect-before-swa-resolver branch June 13, 2026 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant