Cannot export `CIDAS/clipseg-rd16` model to ONNX with `optimum-cli`

#1
by sattwik21 - opened

Summary

Hello, I am trying to export the CIDAS/clipseg-rd16 model to ONNX using optimum-cli as given in the HuggingFace documentation. However, I get an error saying Unrecognized configuration classes ('AutoModelForImageSegmentation', 'AutoModelForSemanticSegmentation') do not match with the model type clipseg and task image-segmentation.

Setup

Environment

Spec Value
OS Linux
OS Version Ubuntu 22.04
Python 3.10.12
Python package manager Poetry

Files

pyproject.toml

[tool.poetry]
name = "hello-clipseg"
version = "0.1.0"
description = ""
authors = []
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
numpy = "^2.0.0"
matplotlib = "^3.9.0"
transformers = "^4.41.2"
optimum = {extras = ["exporters"], version = "^1.20.0"}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Error

$ optimum-cli export onnx --model CIDAS/clipseg-rd16 ./models/clipsed-rd16

Framework not specified. Using pt to export the model.
/home/macaw/sattwik/hello_clipseg/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
config.json: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 4.73k/4.73k [00:00<00:00, 18.0MB/s]
Traceback (most recent call last):
  File "/home/macaw/sattwik/hello_clipseg/.venv/bin/optimum-cli", line 8, in <module>
    sys.exit(main())
  File "/home/macaw/sattwik/hello_clipseg/.venv/lib/python3.10/site-packages/optimum/commands/optimum_cli.py", line 163, in main
    service.run()
  File "/home/macaw/sattwik/hello_clipseg/.venv/lib/python3.10/site-packages/optimum/commands/export/onnx.py", line 265, in run
    main_export(
  File "/home/macaw/sattwik/hello_clipseg/.venv/lib/python3.10/site-packages/optimum/exporters/onnx/__main__.py", line 280, in main_export
    model = TasksManager.get_model_from_task(
  File "/home/macaw/sattwik/hello_clipseg/.venv/lib/python3.10/site-packages/optimum/exporters/tasks.py", line 1918, in get_model_from_task
    model_class = TasksManager.get_model_class_for_task(
  File "/home/macaw/sattwik/hello_clipseg/.venv/lib/python3.10/site-packages/optimum/exporters/tasks.py", line 1378, in get_model_class_for_task
    raise ValueError(
ValueError: Unrecognized configuration classes ('AutoModelForImageSegmentation', 'AutoModelForSemanticSegmentation') do not match with the model type clipseg and task image-segmentation.

Debugging

This error is probably because the CLIPSeg model uses a custom loader CLIPSegForImageSegmentation and the AutoModelFor... classes do not support this. I am new to HuggingFace and don't know whether this is causing the problem or something else.


I need to complete this project urgently and your prompt help in this matter would be highly appreciated. Thank you

CAMPUS INSTITUTE DATA SCIENCE (CIDAS) org
β€’
edited Jul 4

Hi,

CLIPSeg is a bit of a special model that is not supported by the Auto API. It's recommended to open an issue on the Optimum library to make it be supported.

Else you could directly try to export it to ONNX without relying on Optimum as explained in this blog post.

Sign up or log in to comment