Skip to content

Commit

Permalink
fixed linting
Browse files Browse the repository at this point in the history
  • Loading branch information
samuel100 committed Dec 31, 2024
1 parent 307c960 commit b474b8b
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions olive/passes/pytorch/gptq.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,10 +158,9 @@ def _run_for_config(
model_type = pytorch_model.config.model_type if hasattr(pytorch_model, "config") else ""
model_class = GPTQ_CAUSAL_LM_MODEL_MAP.get(model_type, BaseGPTQForCausalLM)
quantized_model: BaseGPTQForCausalLM = model_class(pytorch_model, False, quantize_config)

# explicitly move quantized model to CUDA device to avoid the "Expected all tensors to be
# on the same device" error in auto-gptq.
# see https://github.com/AutoGPTQ/AutoGPTQ/issues/729
# see https://github.com/AutoGPTQ/AutoGPTQ/issues/729
quantized_model.to("cuda")

fields_to_set = {
Expand Down

0 comments on commit b474b8b

Please sign in to comment.