GPUs can almost certainly be more efficient for evaluating DNNs than CPUs are due to the fact that they are built to perform heaps of arithmetic in parallel. But, for effectively the same reasons that a CPU are considered sub-optimal for the task a GPU could also be considered sub-optimal. While GPUs have constraints about the type of workload in comparison to CPUs, they are still relatively general purpose. This is why hardware accelerators can still hugely increase efficiency by limiting the overhead present on a GPU in order to allow for many types of computations. In reality a lot of the work in evaluating DNNs can be boiled down to a few key pieces of arithmetic that will be repeated many times, and thus a hardware accelerator is extremely well-suited to the task.
GPUs can almost certainly be more efficient for evaluating DNNs than CPUs are due to the fact that they are built to perform heaps of arithmetic in parallel. But, for effectively the same reasons that a CPU are considered sub-optimal for the task a GPU could also be considered sub-optimal. While GPUs have constraints about the type of workload in comparison to CPUs, they are still relatively general purpose. This is why hardware accelerators can still hugely increase efficiency by limiting the overhead present on a GPU in order to allow for many types of computations. In reality a lot of the work in evaluating DNNs can be boiled down to a few key pieces of arithmetic that will be repeated many times, and thus a hardware accelerator is extremely well-suited to the task.