To build an efficient machine learning system, it is important for people understanding hardware and parallel computation and people understanding the machine learning algorithm to work together.

One example of a hardware, systems, and ML engineers working together would be that for a specific ML algorithm the parallel version doesn't experience as much speed up due to expensive communication across nodes. The systems person says that increasing the batch size would produce better results, but neither the ML or systems person know how to due that without compromising speed or cache locality. Thus, all they can work with a hardware engineer to create special hardware that allows for a larger batch size.