Also in this world of accelerator programming, people are writing very specialized codes that target a specific architecture, datatype, and even input shape. So with that in mind, how useful is it to have a generic kernel? You still need to do all the targetted optimization to make it performant.

If you want portablitiy you need a machine learning compiler ala TorchInductor or TinyGrad or OpenXLA.