By changing the scheduler from the sequential to the pipelining, we can decrease the runtime as well as increase memory utilization. Pipelining is an essential technique to increase performance by using the same amount of resources as sequential.

There is a tradeoff when using pipelining instead of sequential scheduling. Pipelining can reduce the runtime, but it can also nontrivially increase memory consumption.

One downside to pipelining David mentioned in the chat is when multiple operators have dependencies between them. If you start pipelining, and one operation changes the shared state, then there’s a change where you may have to flush and restart the instructions, adding more time.