What are some common criteria/metrics that architecture designers try to optimize when they decide on how to arrange the cores?

I think communication cost between cores is certainly one of the considerations, followed by thermal considerations.

Communication with off-chip memory is a major computation bottleneck; here's an interesting abstract that mentions the "bandwidth wall":https://dl.acm.org/doi/10.1145/1555815.1555801 . Also from a VLSI perspective the logic in the cores needs to meet setup and hold time constraints both of which are functions of the propagation latency. To meet this the designers iteratively go through the process of physical design adjusting floorplans, routes and cell placements until the timing constraints are met.

@ruilin @teapot Thermals are veryimportant, since cards will throttle performance if they run too hot. When building a PC, poor fan configuration (or none at all) can significantly reduce performance. A lot of extreme benchmarking and overclocking (e.g. increasing the amount of cycles per second that the CPU is factory-guaranteed to work with) on CPUs takes place with liquid nitrogen being used for cooling!