Skip to content

fix: map None quant method to q8_0 before lowercasing in GGUF export#6889

Open
anxkhn wants to merge 1 commit into
unslothai:mainfrom
anxkhn:patch-3
Open

fix: map None quant method to q8_0 before lowercasing in GGUF export#6889
anxkhn wants to merge 1 commit into
unslothai:mainfrom
anxkhn:patch-3

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 5, 2026

Copy link
Copy Markdown
In `unsloth_save_pretrained_gguf` and `save_to_gguf_generic`, the loop that
normalizes the `quantization_method` list calls `quant_method.lower()` as its
first statement, before the branch that maps `None` to `"q8_0"`:

```python
for i, quant_method in enumerate(quantization_method):
    quant_method = quant_method.lower()        # runs first
    if quant_method == "not_quantized":
        quant_method = "f16"
    elif quant_method == "fast_quantized":
        quant_method = "q8_0"
    elif quant_method == "quantized":
        quant_method = "q4_k_m"
    elif quant_method is None:                 # unreachable
        quant_method = "q8_0"
    quantization_methods.append(quant_method.lower())

Because .lower() runs first, a None element inside the list (for example
quantization_method=[None] or ["q4_k_m", None]) raises
AttributeError: 'NoneType' object has no attribute 'lower', and the
elif quant_method is None: branch is dead code that never runs. A bare
quantization_method=None is fine, it is guarded by the surrounding
if quantization_method is not None:; the problem is specifically a None
element within the list.

The sibling loop in save_to_gguf already handles this correctly: it checks
None via elif and does not call .lower() up front. This change brings the
two loops in line with it.

Fix

Handle None before lowercasing and drop the now-redundant elif:

for i, quant_method in enumerate(quantization_method):
    if quant_method is None:
        quant_method = "q8_0"
    else:
        quant_method = quant_method.lower()
    if quant_method == "not_quantized":
        quant_method = "f16"
    elif quant_method == "fast_quantized":
        quant_method = "q8_0"
    elif quant_method == "quantized":
        quant_method = "q4_k_m"
    quantization_methods.append(quant_method.lower())

A None element now resolves to q8_0. Behavior is unchanged for every string
input.

Test

Adds tests/saving/test_quant_method_none_normalization.py. Importing
unsloth.save pulls in unsloth_zoo/Torch, so, following the existing
tests/saving/test_is_gpt_oss_detection.py pattern, the test extracts each
loop's source with ast and execs it against sample inputs. It asserts that
[None] maps to ["q8_0"], ["Q4_K_M", None] maps to ["q4_k_m", "q8_0"],
and that ordinary string inputs are unchanged. The test fails on the old
ordering (AttributeError) and passes with this fix (6 passed).

ruff check on the touched files passes.

In unsloth_save_pretrained_gguf and save_to_gguf_generic, the quant-method
normalization loop ran `quant_method = quant_method.lower()` as its first
statement, before the `elif quant_method is None: quant_method = "q8_0"` branch.
Because `.lower()` executed first, a None element in the quantization_method
list (e.g. quantization_method=[None] or ["q4_k_m", None]) raised
`AttributeError: 'NoneType' object has no attribute 'lower'`, and the intended
None -> q8_0 mapping was unreachable dead code.

Handle None before lowercasing, matching the sibling loop in save_to_gguf,
so a None element resolves to q8_0. Behavior is unchanged for string inputs.

Add tests/saving/test_quant_method_none_normalization.py, a CPU-only test that
extracts each loop via ast and checks that a None element maps to q8_0 instead
of raising, alongside strings.
@anxkhn anxkhn requested a review from danielhanchen as a code owner July 5, 2026 17:33

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes an AttributeError in unsloth/save.py that occurred when a None element was present in the quantization_method list. The fix ensures that None values are checked and mapped to 'q8_0' before calling .lower(). Additionally, a new CPU-only regression test has been added to verify this behavior using AST parsing and execution. There are no review comments to address.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant