Does V100 support flash-attention?

[lingyezhixing]

My device environment is a 4060 laptop connected to a V100 via USB4. The community informed me that the V100 does not support flash-attention. However, when I include -fa in the parameters, regardless of whether the model is running on the 4060, the V100, or split across both GPUs, the speed is significantly improved, and the memory usage is also reduced considerably. Moreover, the output is normal without any repetition or interruption. What is going on here?

[JohannesGaessler]

I implemented FlashAttention as in the algorithm for CUDA hardware in llama.cpp/ggml. This implementation is independent of FlashAttention (2) as in the library. The ggml implementation supports all CUDA hardware.