PE are "processing elements", which are placed such that they are interconnected and can communicate with neighboring processing elements, which may reduce costs involved in data movements.

^with one specific cost being energy (communication to neighboring PEs uses little energy)

In lecture, Kunle mentioned that we don't use 3D structures for chips when placing memory close to cores as we need ways to dissipate heat. However, in this structure, it seems the units are larger and more spread out so we might not necessarily need a vertical heat dissipation method. In such a case, it might be interesting to create 3D systolic arrays. Even if we need to space it out more than 2D systolic arrays, it might provide benefits for higher level convolutions, eg on 3D data. While we normally convert convolutions to 2D matrices and multiply, I wonder if we could convert higher dimensional convolutions to higher dimensional matrices and use this 3D systolic array for faster computation even if it is more spaced out for heat reasons.

@mvpatel2000 this is a block diagram of TPU and on actual chip, each PE here is a flip flop (NAND gates) and they are very dense in area. The heat dissipation problem should still apply here.

In lecture, Kunle mentioned that these systolic arrays are, in practice, oftentimes around 256x256 in dimension.