There are other dedicated hardwares for deep learning computations, such as TPU; what are the difference and similarities between GPU and TPU? Does TPU has their own API?

@suninhouse I may be incorrect but Google's TPUs do not have a publicly accessible API that is analagous to CUDA. The way to take advantage of TPUs is to run Tensorflow programs on Google's own cloud TPUs. Google does sell a lightweight TPU, the Edge TPU, to the public but again, the API's to access it are at a higher level of abstraction than CUDA.