It was clarified in the chat that these messages are only sent out when the state of a line in the cache changes – this means that all of this communication doesn't necessarily have to happen on ever memory load/store, only on memory operations that change cache state (evicting a line, loading a new line, writing to a line that was previously clean). This means that good locality in code is still a desired quality – if most memory operations are on data that fits in a cache line then the CPU doesn't have to send many coherence messages.

What if a core had a cache line in the M state, then conducted a PrWr (which doesn't require a bus message since it's already in the M state), but at the same time another core called BusRdX which is supposed to invalidate the first core's M state? Since the first core didn't need a bus message, this conflict can't be arbitrated by seeing who seizes the interconnect first. So how is it handled?

In this system, is it also true that BusWB is called lazily, since it is only called when there is a BusRd/RdX on data in the modified state?

Each edge S1 ---(X / Y)---> S2 means that if the current cache state is S1 and the processor issued action X, the bus will do action Y and change the state to S2.

@arkhan. Good question. The coherence protocol requires processors to receive acknowledgement of the receipt of the BuxRdX before the generator of the BusRdX can proceed. In the situation you describe, this all occur over a sequence of steps

Starting state: P0 has line X in the M state.

Let's assume assume there is one shared bus for coherence requests and another for coherence responses.

• cycle i : P1 issues a BusRdX for line X on the request bus
• cycle i+1 : P0 perform the "snoop", checking its cache for the presence of X
• cycle i+2 : P0 flushes its line (let's assume an unrealistic single cycle flush for now)
• cycle i+3: : P0 asserts via the response bus that it has completed the snoop, and flushed.
• cycle i+4; : P1 proceeds to load the line in the M state.

@kayvonf When you say "flush" do you mean that P0's cache controller can send the cache line directly to P1's cache controller? Or does the cache line need to be written to main memory as a middle man?

Nevermind, it looks like cache controllers can communicate with each other directly. Slide 24 seems to demonstrate that this is the case.