CudaMemcpy is reminiscent of message passing. This is evidence that CUDA is in part, an example of the message passing model... not to spoil the next slide, but there are also shared memory addresses!

how are cudaMalloc/Memcpy implemented, since we can't access deviceA in the host's addr space? Are they implemented with system calls?

@uxfela that would be my guess... is it a blocking send?

@mziv According to the CUDA Toolkit Documentation, in most cases, it is a blocking send. See https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY.html#group__CUDART__MEMORY_1gc263dbe6574220cc776b45438fc351e8 for more details. Specifically, the 4th bullet point under the "Notes" section.