torchao: quantization schemes can't find ao lib

[jackzhxng]

🐛 Describe the bug

Anytime _load_torchao_aten_lib is called (namely for -qmode 8da[]w), the following error pops up:

Traceback (most recent call last):
File "/home/jackzhxng/.conda/envs/oss2/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/jackzhxng/.conda/envs/oss2/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/jackzhxng/torchrepos2/executorch/examples/models/llama/runner/eager.py", line 116, in
main() # pragma: no cover
File "/home/jackzhxng/torchrepos2/executorch/examples/models/llama/runner/eager.py", line 112, in main
execute_runner(EagerLlamaRunner)
File "/home/jackzhxng/torchrepos2/executorch/examples/models/llama/runner/eager.py", line 93, in execute_runner
runner = runner_class(args) # pyre-ignore: Missing argument [20]
File "/home/jackzhxng/torchrepos2/executorch/examples/models/llama/runner/eager.py", line 38, in init
manager: LLMEdgeManager = _prepare_for_llama_export(args)
File "/home/jackzhxng/.conda/envs/oss2/lib/python3.10/site-packages/executorch/examples/models/llama/export_llama_lib.py", line 601, in _prepare_for_llama_export
.source_transform(_get_source_transforms(args.model, dtype_override, args))
File "/home/jackzhxng/.conda/envs/oss2/lib/python3.10/site-packages/executorch/extension/llm/export/builder.py", line 154, in source_transform
self.model = transform(self.model)
File "/home/jackzhxng/.conda/envs/oss2/lib/python3.10/site-packages/executorch/examples/models/llama/source_transformation/quantize.py", line 101, in quantize
_load_torchao_aten_lib(libname="libtorchao_ops_aten")
File "/home/jackzhxng/.conda/envs/oss2/lib/python3.10/site-packages/executorch/examples/models/llama/source_transformation/quantize.py", line 823, in _load_torchao_aten_lib
assert (
AssertionError: Expected 1 library but got 0. If you installed the torchao ops in a non-standard location, please set CMAKE_INSTALL_PREFIX correctly.

Versions

PyTorch version: 2.7.0.dev20250131+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: CentOS Stream 9 (x86_64)
GCC version: (GCC) 11.5.0 20240719 (Red Hat 11.5.0-2)
Clang version: Could not collect
CMake version: version 3.30.5
Libc version: glibc-2.34

Python version: 3.10.0 (default, Mar 3 2022, 09:58:08) [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-6.4.3-0_fbk12_hardened_2624_g7d95a0297d81-x86_64-with-glibc2.34
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 40 bits physical, 57 bits virtual
Byte Order: Little Endian
CPU(s): 166
On-line CPU(s) list: 0-165
Vendor ID: AuthenticAMD
Model name: AMD EPYC-Milan Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 83
Socket(s): 1
Stepping: 1
BogoMIPS: 2399.99
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core invpcid_single ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512_bf16 clzero xsaveerptr wbnoinvd arat npt lbrv nrip_save tsc_scale vmcb_clean pausefilter pfthreshold avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid fsrm arch_capabilities
Virtualization: AMD-V
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 2.6 MiB (83 instances)
L1i cache: 2.6 MiB (83 instances)
L2 cache: 41.5 MiB (83 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-165
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Vulnerability Spectre v2: Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] executorch==0.6.0a0+bb60a20
[pip3] flake8==6.1.0
[pip3] flake8-breakpoint==1.1.0
[pip3] flake8-bugbear==24.4.26
[pip3] flake8-comprehensions==3.14.0
[pip3] flake8-plugin-utils==1.3.3
[pip3] flake8-pyi==23.5.0
[pip3] mypy==1.14.1
[pip3] mypy-extensions==1.0.0
[pip3] numpy==2.2.2
[pip3] torch==2.7.0.dev20250131+cpu
[pip3] torchao==0.10.0+git7d879462
[pip3] torchaudio==2.6.0.dev20250131+cpu
[pip3] torchsr==1.0.4
[pip3] torchtune==0.0.0
[pip3] torchvision==0.22.0.dev20250131+cpu
[conda] executorch 0.6.0a0+bb60a20 pypi_0 pypi
[conda] numpy 2.2.2 pypi_0 pypi
[conda] torch 2.7.0.dev20250131+cpu pypi_0 pypi
[conda] torchao 0.10.0+git7d879462 pypi_0 pypi
[conda] torchaudio 2.6.0.dev20250131+cpu pypi_0 pypi
[conda] torchfix 0.6.0 pypi_0 pypi
[conda] torchsr 1.0.4 pypi_0 pypi
[conda] torchtune 0.0.0 pypi_0 pypi
[conda] torchvision 0.22.0.dev20250131+cpu pypi_0 pypi

cc @kimishpatel @jerryzh168

[jackzhxng]

cc @metascroy @larryliu0820

[metascroy]

@jackzhxng The error is complaining that there is no op library for torchao kernels. Note that torchao kernels cannot be used in ET unless they are built first.

See instructions here for running torchao kernels in the ET llama runner here: #8571 (comment)

The key part is in step 2 when building the runner, you must set "-DEXECUTORCH_BUILD_TORCHAO=ON"

[jackzhxng]

Should we build these by default using install_executorch.sh?

[metascroy]

Should we build these by default using install_executorch.sh?

Potentially. I'll be doing a pin bump on torchao, and the AOT libs will be installed by install_executorch.py.

For the ET libs, I'll leave their build to the llama runner for now.

[metascroy]

Should we build these by default using install_executorch.sh?

Potentially. I'll be doing a pin bump on torchao, and the AOT libs will be installed by install_executorch.py.

For the ET libs, I'll leave their build to the llama runner for now.

@jackzhxng in #9396, these are installed by install_executorch.py. I also added a unit test, and instructions to the readme.