AttributeError: module 'mpmath' has no attribute 'rational'

[a5hwinjs]

System Info

• cpu arch x86
• cuda version - 12.2
• GPU - a10g

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Clone the Repo
Install the requirements.txt
Install ammo

run the quantize.py script in the example

python3 ./examples/quantization/quantize.py --model_dir /opt/ml/fine_tuned_model
--dtype float16
--qformat int4_awq
--awq_block_size 128
--output_dir /opt/ml/quantized_int4-awq
--calib_size 32

Expected behavior

Quantization should start

actual behavior

AttributeError: module 'mpmath' has no attribute 'rational',

additional notes

This is occurring because of not locking mpmath's version somewhere in the requirements . They released a latest version today 1.4.0a0 which is causing this issue. Downgrading back manually to 1.3.0 fixes the issue

[byshiue]

Thank you for the info and feedback. The issue is fixed in this PR #1146. Close this issue.

[Lauler]

@byshiue Same issue occurs when following the basic installation instruction with docker in the README. Should probably update the requirements.txt there too.

I cannot import the library followinging the instructions:

python3 -c "import tensorrt_llm"
# AttributeError: module 'mpmath' has no attribute 'rational

It seems the offending version is automatically being installed as a dependency of some other library:

root@b2da23167f04:/home# pip list | grep mpmath
mpmath                   1.4.0a0

After downgrading mpmath, I can import tensorrt_llm.

[byshiue]

We don't add mpmath in requirements.txt and it shouldn't affect. Do you install mpmath 1.4.0 in your environment originally? Or you have mpmath 1.3 and requirements.txt upgrades it?

[Lauler]

My assumption is that mpmath is a required dependency of some other library in your requirements.txt. And that library probably doesn't pin mpmath, leading to version 1.4.0 of mpmath being installed if a user follows the instructions in your README for setting up docker with TensorRT-LLM.

That import error comes from following the set up instructions in the README for docker. I made no other changes.

[IKACE]

Ran into the same problem when bootstrapping docker image provided in the official README. I can confirm uninstalling mpmath and reinstall version 1.3.0 of mpmath resolves the issue.

[fxmarty]

same