mem write from EP to RP, what address range?

Comments and questions to the author of the "Down to the TLP" posts (Eli Billauer)

mem write from EP to RP, what address range?

Postby mistercoffee » Mon Jul 04, 2016 8:37 pm


I am new to PCIe. I have read through a lot of the spec and have just bought the Mindshare book (PCI Express Technology), which is very good.

But I have one nagging question, and it is probably just something I'm not understanding.

The book talks about the 6 BARs in the EP and how you set the lower bits to specify the type and size of memory the system software needs to assign to it. It then talks about the upper bits then being written to by the system software with the base address for the EP. Any downstream TLPs within the range allocated to it will be processed by the EP.

I also see that the BARs in a switch have base and limit reg to help forward TLPs to an EP below.

But what about if I want to do an upstream mem write, and write to the system memory attached to the RP? What is the base and limit address for writing upstream? Where is this set and how do I find out what it is, so that I can set the correct address when I create a mem write TLP at the EP to go to the RP and system memory? I am basically trying to stream data to a PC from a PCIe card I have.

I must be missing something obvious...

Posts: 2
Joined: Mon Jul 04, 2016 8:29 pm

Re: mem write from EP to RP, what address range?

Postby support » Mon Jul 04, 2016 9:07 pm


First, let's start with the notion that it doesn't matter if an endpoint or root port initiates a write (or read) operation. All entities on the bus see the same memory map view, so all writes are treated equally. This allows, for example, peer-to-peer writes (from an endpoint to another). But that's not what you asked about.

Each switch in the network, including the root port switch, is configured to route packets of certain address ranges to certain downstream ports. If none of the downstream port covers the destination address, it routes the packet towards the upstream port.

In short: Routing towards the root port is the default. In a simple setting with no switches, the processor may have a fixed address range which is covered by the PCIe switch. Any write to that range will go to the PCIe bus. This is the case in some embedded processors. It's the operating system's job to set up the BAR addresses within this range.

So answer is: If you want to write to the processor's memory in a no-operating system scenario, get the address range from the processor's memory map. Just allocate a chunk in the RAM, and make sure that the PCIe endpoint writes to its (physical) addresses.

If you have an operating system, you'll need to allocate memory and request the physical address of that memory chunk from the OS. And then pass that information to the endpoint, possibly by writing to registers, which are mapped into the BAR space.

Hope this clarified things.

Posts: 482
Joined: Tue Apr 24, 2012 3:46 pm

Re: mem write from EP to RP, what address range?

Postby mistercoffee » Mon Jul 04, 2016 9:38 pm

Thank you so much for the detailed reply!

Our system will actually be linux based. There is another engineer who is responsible for the driver and application software that will collect the data I am sending, they just haven't started on the project yet...

So then, I will ask them to allocate a portion of the system memory for this specific purpose. I will then get them to send the base and limit values to the endpoint (probably writing to the bar space). I will then use these addresses when I do my bus master DMA TLPs. I also need them to set the bus master enable bit...

I think my mind was heading in this direction, but I was struggling to find confirmation.

Posts: 2
Joined: Mon Jul 04, 2016 8:29 pm

Return to General PCIe