Out of interest, if the execution of the actual ALU represents around 6% of the total energy use in a general-purpose chip, how could we ever get 100-1000x preference per watt improvement? Is there something more fundamental that constitutes that 6%, like all the ALUs (including those unused) consuming power?

@harrymellsop Arithmetic operations represent 6% of power distribution on a general purpose processor. We can increase this distribution with things like SIMD (which amortizes the cost of the operations in the other 94%), but we obviously hit a ceiling.

The key with fixed function ASIC (and FPGAs as well, although the performance/watt of an FPGA would probably be somewhere between a CPU and an ASIC) is that we eliminate so much of the overhead from the other 94%. ASICs can deliver 100-1000x performance/watt because the power distribution on this chip gives much more than 6% to arithmetic computation. So much complexity that enables the "general purposeness" of a CPU is just gutted with fixed function hardware, allowing for this much higher performance/watt.