Studio: Video tab frontend#6875
Conversation
Add a Video generation page that mirrors the Images feature's create workflow. It loads a text-to-video model, generates a clip, and plays it back inline with the gallery of past clips. - src/features/video/api.ts: typed client for the /api/inference/video routes (load, load-progress, generate, generate-progress, cancel, status, unload, gallery CRUD, and an auth-protected MP4 blob fetch). - src/features/video/video-page.tsx: the page. Curated model picker (LTX 2.3 distilled GGUF, LTX 2 base pipeline), prompt and negative prompt, resolution preset select, duration select over the family's temporal lattice, fixed fps display, steps and guidance sliders seeded from per-model defaults, seed box. Generate polls per-step progress with a phase label and ETA and a Cancel button, then plays the result in a video player with a download button and an audio badge. Gallery strip below with per-card delete and clear all. Right-docked Advanced panel for memory, speed, attention, and step-cache with Auto badges fed from the resolved status. - Register the page: router child, /video route, sidebar nav item with a video icon after Images, and the __root keep-alive mount so an in-flight generation survives leaving the tab. - Add the text-to-video task to the model picker so video models never appear in the chat picker. - Add the video nav label to the en and zh-CN locales.
for more information, see https://pre-commit.ci
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new Video generation feature to the Studio frontend, adding a dedicated /video route, sidebar navigation, API integration, and a comprehensive VideoPage component with advanced load-time tuning, generation parameters, and a persistent video gallery. Feedback on the changes suggests refining the model tag detection in defaultsFor by splitting repository names into segments to avoid false positives, and checking isMounted.current before updating state in the asynchronous ensureSrc callback to prevent React state updates on unmounted components.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| function defaultsFor(repoId: string): { steps: number; guidance: number } { | ||
| const id = repoId.toLowerCase(); | ||
| return MODEL_DEFAULTS.find((d) => id.includes(d.match)) ?? DEFAULT_GEN; | ||
| } |
There was a problem hiding this comment.
Avoid using simple raw substring checks (id.includes(d.match)) to detect model types or tags, as this can lead to false positives (e.g., matching 'ltxify' or 'distilledmodel'). Instead, split the repository name into segments using delimiters like dashes, underscores, dots, and slashes, and verify if the tag appears as a whole segment.
| function defaultsFor(repoId: string): { steps: number; guidance: number } { | |
| const id = repoId.toLowerCase(); | |
| return MODEL_DEFAULTS.find((d) => id.includes(d.match)) ?? DEFAULT_GEN; | |
| } | |
| function defaultsFor(repoId: string): { steps: number; guidance: number } { | |
| const segments = repoId.toLowerCase().split(/[-_./\\]/); | |
| return MODEL_DEFAULTS.find((d) => segments.includes(d.match)) ?? DEFAULT_GEN; | |
| } |
References
- When statically detecting model types or tags from repository IDs or names, avoid simple raw substring checks to prevent false positives. Instead, split the repository name into segments using delimiters like dashes, underscores, and dots, and verify if the tag appears as a whole segment.
There was a problem hiding this comment.
The substring match is deliberate: "wan" has to match Wan2.2 repo ids, and the suggested segment split would stop it (segment "wan2" is not "wan"). These defaults only prefill steps and guidance for the curated picker entries, so a false positive costs nothing.
| const ensureSrc = useCallback(async (video: GalleryVideo) => { | ||
| if (galleryCache.srcById.has(video.id) || galleryCache.inflight.has(video.id)) return; | ||
| galleryCache.inflight.add(video.id); | ||
| try { | ||
| const url = await fetchGalleryVideoObjectUrl(video.url); | ||
| galleryCache.srcById.set(video.id, url); | ||
| setSrcById((prev) => ({ ...prev, [video.id]: url })); | ||
| } catch { | ||
| // Leave it without a src; the card shows a placeholder. | ||
| } finally { | ||
| galleryCache.inflight.delete(video.id); | ||
| } | ||
| }, []); |
There was a problem hiding this comment.
To prevent state updates on unmounted components, check isMounted.current before calling setSrcById inside the asynchronous ensureSrc callback.
const ensureSrc = useCallback(async (video: GalleryVideo) => {
if (galleryCache.srcById.has(video.id) || galleryCache.inflight.has(video.id)) return;
galleryCache.inflight.add(video.id);
try {
const url = await fetchGalleryVideoObjectUrl(video.url);
galleryCache.srcById.set(video.id, url);
if (isMounted.current) {
setSrcById((prev) => ({ ...prev, [video.id]: url }));
}
} catch {
// Leave it without a src; the card shows a placeholder.
} finally {
galleryCache.inflight.delete(video.id);
}
}, []);
There was a problem hiding this comment.
Added the isMounted guard to match the other async callbacks in the file.
|
@codex review |
The object URL still lands in the module cache either way; the setState call now checks isMounted like the other async callbacks in the file.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0f35bac673
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const dq = defaultsFor(id); | ||
| setSteps(dq.steps); | ||
| setGuidance(dq.guidance); |
There was a problem hiding this comment.
When a user picks a GGUF variant from unsloth/LTX-2.3-GGUF, defaultsFor(id) only sees the repo id, so it never matches distilled even though the chosen filename does. The backend derives defaults from state.gguf_filename, but these 40-step/CFG-4 values are then sent explicitly during generation, overriding the intended distilled 8-step/CFG-1 schedule and making default runs much slower and different from the loaded model's defaults. Use meta.ggufFilename here or apply status.defaults after load.
Useful? React with 👍 / 👎.
| // Video-generation pipeline tasks: handled by the Video page, never loadable as | ||
| // chat models. The backend reports "text-to-video" for video-diffusion GGUFs. The | ||
| // Video page reuses this as its picker's `task` filter, so it lives here. | ||
| export const VIDEO_GEN_TASKS = ["text-to-video"] as const; |
There was a problem hiding this comment.
The Video picker filters all cached/local On Device entries through passesTaskGate(..., VIDEO_GEN_TASKS), but the current backend cache classifier in routes/models.py::_arch_to_task does not emit text-to-video for supported LTX GGUFs; it tags ltxv as image-diffusion-unsupported. As a result, users who already have an LTX video GGUF in the HF cache, LM Studio, ./models, or a custom folder will not see it in the Video picker even though the new page is supposed to load on-device video GGUFs. Either emit text-to-video for supported video families or include the backend's current tag here.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
All five landed: ltxv and wan GGUF archs are now tagged text-to-video so cached checkpoints surface in the picker, the loaded family default clip length applies, the picked GGUF filename drives the steps and guidance prefill, Cancel no longer toasts as an error, and the Video nav item is disabled with a hint on chat-only hosts.
| setNumFrames((cur) => { | ||
| if (durationOptions.some((o) => o.frames === cur)) return cur; | ||
| // Prefer the ~3s preset (index 2) as a sensible default, else the first. | ||
| return durationOptions[2]?.frames ?? durationOptions[0]?.frames ?? cur; | ||
| }); |
There was a problem hiding this comment.
After a model loads, numFrames is still the fallback 25-frame value because it is already a valid LTX lattice option, so this effect keeps it and every generation sends num_frames: 25. That prevents the backend from using status.defaults.num_frames (121 frames for LTX-2), making the default Video run about 1s instead of the model's intended ~5s default unless the user manually changes the duration. Reset from status.defaults.num_frames when the loaded family changes.
Useful? React with 👍 / 👎.
| setSelectedId(res.video.id); | ||
| void ensureSrc(res.video); | ||
| } catch (err) { | ||
| toast.error(err instanceof Error ? err.message : "Video generation failed"); |
There was a problem hiding this comment.
When the user presses Cancel, the cancel endpoint only signals the backend; the in-flight generateVideo request then rejects with the backend's cancellation sentinel, and this catch displays it as an error toast. That makes the normal Cancel path look like a generation failure, so suppress or special-case the known cancellation message/status here while still reporting real failures.
Useful? React with 👍 / 👎.
| onClick={() => { | ||
| navigate({ to: "/video" }); | ||
| closeMobileIfOpen(); | ||
| }} |
There was a problem hiding this comment.
On chat-only hosts, __root.tsx does not include /video in the allowlist, so clicking this enabled sidebar item immediately hits the root guard and redirects back to /chat. In that environment the tab appears selectable but can never open; mirror the Train item by disabling/hiding it with a tooltip, or allow /video if it is meant to work there.
Useful? React with 👍 / 👎.
…ncel and chat-only polish Tag the ltxv and wan GGUF archs text-to-video so cached video checkpoints actually surface in the Video picker (they were classed unsupported and hidden everywhere). Adopt the loaded family's default clip length instead of silently keeping the 25-frame pre-load fallback, and derive steps and guidance from the picked GGUF filename so a distilled variant gets its few-step schedule. Suppress the error toast for the user's own Cancel and disable the Video nav item on chat-only hosts with a hint, matching Train.
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 35c1bc2135
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Video GGUF archs the video backend CAN load (LTX-2.x ships as "ltxv"; the Wan | ||
| # community GGUFs as "wan"). Tagged text-to-video so they surface in the Video | ||
| # picker (VIDEO_GEN_TASKS) and stay out of chat (NON_CHAT_TASKS). | ||
| _VIDEO_GGUF_ARCHS = frozenset({"ltxv", "wan"}) |
There was a problem hiding this comment.
When a cached Wan GGUF is present, _arch_to_task now labels its wan architecture as text-to-video, so the Video picker will offer it as loadable. I checked the video backend family registry (core/inference/video_families.py), and it currently only registers ltx-2; validate_load_request will therefore reject Wan picks as an unsupported text-to-video family. Please leave wan under the unsupported diffusion task until a Wan VideoFamily and loader are actually added.
Useful? React with 👍 / 👎.
| galleryCache.hasMore = page.has_more; | ||
| setVideos(page.videos); | ||
| setHasMore(page.has_more); | ||
| page.videos.forEach((video) => void ensureSrc(video)); |
There was a problem hiding this comment.
When the Video tab opens with existing clips, this calls ensureSrc for every first-page record, and ensureSrc uses authFetch(url).blob(), so up to 50 full MP4s are downloaded into browser memory before the user plays or selects them. Generated video files can be large, so galleries with many clips can stall the tab and consume hundreds of MB/GB immediately; fetch object URLs lazily for the selected/visible items instead.
Useful? React with 👍 / 👎.
| const dq2 = defaultsFor(id); | ||
| setSteps(dq2.steps); | ||
| setGuidance(dq2.guidance); | ||
| void handleLoad(dir, { kind: "gguf", filename }).then((started) => { |
There was a problem hiding this comment.
For direct local GGUF picks, the picker admits files based on their GGUF architecture, but this load request sends only the parent directory and filename. The video backend resolves the family from path/name tokens (_detect_load_family) rather than the GGUF metadata, so a valid ltxv file renamed to something like /models/video/model.gguf appears in the Video picker and then always fails validation as an unsupported text-to-video model. Pass a family override or have the backend infer the family from the selected GGUF metadata on this path.
Useful? React with 👍 / 👎.
| {videos.length > 0 && ( | ||
| <button | ||
| type="button" | ||
| onClick={() => void handleClearAll()} |
There was a problem hiding this comment.
When a user reaches the tail of the thumbnail strip and clicks this trash tile, handleClearAll() permanently deletes every saved video through the gallery DELETE endpoint with no confirmation or undo. Because this control is adjacent to normal gallery navigation/actions and its scope is the entire gallery rather than one clip, an accidental click can wipe all generated MP4s; add a confirmation step before invoking the clear-all request.
Useful? React with 👍 / 👎.
| onEject={status?.loaded ? handleUnload : undefined} | ||
| variant="ghost" | ||
| className="!h-[34px]" | ||
| task={VIDEO_GEN_TASKS} |
There was a problem hiding this comment.
With this task-scoped selector, the shared On Device GGUF rows expose their existing delete action for cached video variants. I checked /api/models/delete-cached: it refuses loaded chat and image/diffusion models, but it does not check the video backend, so a user can delete the GGUF backing the currently loaded Video model from this picker. Please disable model deletion while the video model is loaded or add the same loaded-model guard for get_video_backend().status().
Useful? React with 👍 / 👎.
1 participant
Stacked on #6874. Adds the Video tab UI over the video inference API.
What this adds
Videosidebar entry with the same keep-alive mount pattern as Images, so an in-flight generation survives tab switches.video-page.tsx: curated video model picker (LTX-2.3 GGUF variants first), load progress polling, phase-based generation progress with ETA (load, encode prompt, denoise, decode, export) so the bar never looks stuck outside the denoise callback window.Verification
npm run buildgreen.