I recently built a brilliant work in my domain by directing LLMs. I used C++ and a knowledge of data structures and algorithms to achieve 3+ orders of magnitude of speed up. I generated custom data structures which I designed to have the fewest instructions possible in the hot path. I used novel non-locking communication schemes between threads.

The LLM did all the coding, but came up with none of this by default. None of the optimizations I envisioned emerged up when I prompted “how could we make this faster?” If I hadn’t been an expert in my field the output wouldn’t have been useful.

Keep studying!