PCIe transmited packet size

Comments and questions to the author of the "Down to the TLP" posts (Eli Billauer)

PCIe transmited packet size

Postby Guest » Thu Jun 06, 2013 11:42 pm

How PCIe decides when to initiate packet transaction and what size?
Lets assume that memcpy is used to copy 2K of data, No DMA, max packet size 512.
Case 1: PCIe will wait until write activity is done, to initiate 4x512 transaction?
Case 2: PCIe will wait until first 512 bytes are there to start sending?
Case 3: PCIe will start sending some packet size as fast as detected memory write?
How to configurable that?

Thanks for nice introduction to PCIe.

Re: PCIe transmited packet size

Postby support » Fri Jun 07, 2013 12:24 am

The general answer is that this is platform dependent. In theory, a root complex governed by the CPU may wait for some data to accumulate before generating a TLP for write. Even more theoretic, memcpy() could be implemented to hint the hardware that the data should be packed.

But practically, what I've seen is that each read/write operation directed towards a PCIe device results in a separate TLP. At best, if a 64-bit word is read or written, this TLP will span two DWORDs. I doubt anyone has seen larger TLPs resulting from data transfer initiated directly by software. And memcpy() is just an optimized loop of reads followed by writes.

There is a reason why there is no data packing: Software reads and writes to PCIe devices should be used only to access registers in a properly designed driver. The bulk data transfers should be done with DMA. Hence it doesn't make much sense to optimize non-DMA data transfers. I don't expect this to appear in future processors.

I hope this clarified this issue.
Posts: 619
Joined: Tue Apr 24, 2012 3:46 pm

Return to General PCIe