Skip to content

Support full int8 quantization for diffusers#1193

Merged
IlyasMoutawwakil merged 6 commits into
huggingface:mainfrom
l-bat:lt/sd_ptq
Mar 21, 2025
Merged

Support full int8 quantization for diffusers#1193
IlyasMoutawwakil merged 6 commits into
huggingface:mainfrom
l-bat:lt/sd_ptq

Conversation

@l-bat

@l-bat l-bat commented Mar 7, 2025

Copy link
Copy Markdown
Contributor

What does this PR do?

  • Enable full quantization of denoiser models of Stable Diffusion pipelines via optimum-cli.
  • Supported --quant-mode options: int8, f8e4m3, f8e5m2.
  • Example command is below.
optimum-cli export openvino -m stabilityai/stable-diffusion-2 --quant-mode int8 --dataset conceptual_captions --num-samples 32 --smooth-quant-alpha 0.9 ./sd2

Fixes # (issue)

  • 163715

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@l-bat

l-bat commented Mar 7, 2025

Copy link
Copy Markdown
Contributor Author

@AlexKoff88, @nikita-savelyevv please take a look

Comment thread tests/openvino/test_exporters_cli.py Outdated
Comment thread tests/openvino/test_exporters_cli.py
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@AlexKoff88

Copy link
Copy Markdown
Contributor

@IlyasMoutawwakil, @echarlaix, please take a look.

@AlexKoff88

Copy link
Copy Markdown
Contributor

@IlyasMoutawwakil, @echarlaix, the PR is ready for your review

@nikita-savelyevv nikita-savelyevv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@l-bat please run make style to fix quality checks pre-commit failing

@IlyasMoutawwakil

Copy link
Copy Markdown
Member

@l-bat it's still failing, please do pip install .[quality] and then make style

@IlyasMoutawwakil

Copy link
Copy Markdown
Member

@l-bat please avoid applying black alone, the order of commands in the makefile is important, we apply black first and then ruff, if the make doesn't work for you, install it with apt-get install build-essential, or just run these two:

	black .
	ruff check . --fix

Comment thread optimum/commands/export/openvino.py Outdated
Comment thread optimum/intel/openvino/quantization.py Outdated
@IlyasMoutawwakil

Copy link
Copy Markdown
Member

I fixed it manually with code suggestions

@IlyasMoutawwakil IlyasMoutawwakil merged commit 8fa4ebd into huggingface:main Mar 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants