Building llava cpp runner fails

[spalatinate]

🐛 Describe the bug

I am trying to reproduce the llava example on an Raspberry Pi5. As a preliminary, I have exported the pte file already. Hence, the only objective is to build the llava cpp runner locally using ci/scripts/test_llava.sh. Unfortunately, I am running into the following error when executing .ci/scripts/test_llava.sh.

...
[  8%] Built target dumb_fht
[ 41%] Built target custom_ops
[ 75%] Built target extension_llm_runner
[ 91%] Built target llava_runner
[ 95%] Linking CXX executable llava_main
/usr/bin/ld: warning: libgfortran-4435c6db.so.5.0.0, needed by /home/miniconda3/envs/executorch/lib/python3.10/site-packages/torch/lib/../../torch.libs/libopenblasp-r0-bbd0e64a.3.28.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /home/miniconda3/envs/executorch/lib/python3.10/site-packages/torch/lib/../../torch.libs/libopenblasp-r0-bbd0e64a.3.28.so: undefined reference to `_gfortran_concat_string@GFORTRAN_8'
/usr/bin/ld: /home/miniconda3/envs/executorch/lib/python3.10/site-packages/torch/lib/../../torch.libs/libopenblasp-r0-bbd0e64a.3.28.so: undefined reference to `_gfortran_etime@GFORTRAN_8'
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/llava_main.dir/build.make:127: llava_main] Fehler 1
gmake[1]: *** [CMakeFiles/Makefile2:178: CMakeFiles/llava_main.dir/all] Fehler 2
gmake: *** [Makefile:136: all] Fehler 2

Versions

Collecting environment information...
PyTorch version: 2.7.0.dev20250131+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 12 (bookworm) (aarch64)
GCC version: (Debian 12.2.0-14) 12.2.0
Clang version: 14.0.6
CMake version: version 3.31.6
Libc version: glibc-2.36

Python version: 3.10.0 (default, Mar 3 2022, 09:51:40) [GCC 10.2.0] (64-bit runtime)
Python platform: Linux-6.6.74+rpt-rpi-v8-aarch64-with-glibc2.36
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: ARM
Model name: Cortex-A76
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 4
Socket(s): -
Cluster(s): 1
Stepping: r4p1
CPU(s) scaling MHz: 100%
CPU max MHz: 2400,0000
CPU min MHz: 1500,0000
BogoMIPS: 108,00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp
L1d cache: 256 KiB (4 instances)
L1i cache: 256 KiB (4 instances)
L2 cache: 2 MiB (4 instances)
L3 cache: 2 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] executorch==0.6.0a0+542480c
[pip3] numpy==2.2.3
[pip3] torch==2.7.0.dev20250131+cpu
[pip3] torchao==0.10.0+git7d879462
[pip3] torchaudio==2.6.0.dev20250131
[pip3] torchgen==0.0.1
[pip3] torchsr==1.0.4
[pip3] torchvision==0.22.0.dev20250131
[conda] executorch 0.6.0a0+542480c pypi_0 pypi
[conda] numpy 2.2.3 pypi_0 pypi
[conda] torch 2.7.0.dev20250131+cpu pypi_0 pypi
[conda] torchao 0.10.0+git7d879462 pypi_0 pypi
[conda] torchaudio 2.6.0.dev20250131 pypi_0 pypi
[conda] torchgen 0.0.1 pypi_0 pypi
[conda] torchsr 1.0.4 pypi_0 pypi
[conda] torchvision 0.22.0.dev20250131 pypi_0 pypi

cc @larryliu0820 @lucylq @mergennachin @cccclai @helunwencser @jackzhxng

[iseeyuan]

Thanks @spalatinate for reporting it! Have you tried install_executorch.sh --clean and reinstall install_executorch.sh? cc @larryliu0820 if you could help on this issue.

[larryliu0820]

Hi @spalatinate it looks like you need to install libgfortran based on this log:

libgfortran-4435c6db.so.5.0.0, needed by /home/miniconda3/envs/executorch/lib/python3.10/site-packages/torch/lib/../../torch.libs/libopenblasp-r0-bbd0e64a.3.28.so, not found (try using -rpath or -rpath-link)

Try the following

sudo apt-get update 
sudo apt-get install libgfortran5

[spalatinate]

@iseeyuan Yeah, I tried multiple times

install_requirements.dh --clean

I think @larryliu0820 solution sounds good. I will try it and will give asap feedback.

Thank you! Much appreciated.

[spalatinate]

@larryliu0820 I tried your approach. Unfortunately, the error remains the same. I also tried conda install conda-forge::libgfortran5.

As an alternative, I tried to build the llava runner on a VM on Azure ML. However, here the compilation gets stuck when creating the microkernels and I have to reboot the machine. Can you build the runner or is it just my set-up? I am strictly following the installation guidelines.

[spalatinate]

I found a potential root cause. I assume there is broken rpath in the ARM64 build of PyTorch (found similar issues, see below), which makes sense because the error remains although libgfortran5 is installed. But I am not sure how to fix it.

Sources:

• broken rpath in the ARM64 build of PyTorch
• Fix ARM64 builds #222

[larryliu0820]

For a quick hack, can you do

export LD_LIBRARY_PATH=<path to libgfortran>:$LD_LIBRARY_PATH

and then run the installation again?

[spalatinate]

@larryliu0820 Thanks for your response. So I tried the following:

I found libgfortran binaries in:

1. ./lib/aarch64-linux-gnu/libgfortran.so.5
2. ./lib/aarch64-linux-gnu/libgfortran.so.5.0.0

Hence, I tried:

export LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/:$LD_LIBRARY_PATHort LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/:$LD_LIBRARY_PATH

Unfortunately, the error message stays the same. In the error message it says libgfortran-4435c6db.so.5.0.0, needed by ... Is 4435c6db a hash?

Thank you for your support!!

[larryliu0820]

Can you alias your ./lib/aarch64-linux-gnu/libgfortran.so.5.0.0 to be libgfortran-4435c6db.so.5.0.0?

Just do a ln -s ./lib/aarch64-linux-gnu/libgfortran.so.5.0.0 ./lib/aarch64-linux-gnu/libgfortran-4435c6db.so.5.0.0 should be suffice.

[spalatinate]

@larryliu0820: It worked! Thanks for the hint though.Wasn't sure why this hash was attached to the file name.

[larryliu0820]

Great! closing feel free to open a new one if there's anything else