PCIe Initialization

Comments and questions related to the "Down to the TLP" pages

PCIe Initialization

Postby Guest » Mon Apr 06, 2015 3:32 pm


I’d like to thank you for a higher level explanation of the PCIe Base Spec., it was a very good starting point for me doing PCIe development. Unfortunately I seem to continually run into issues as documentation seems lacking in places and sporadic. I’m at a loss as to where I can seek help/advice on such a task as experts and forums seem few. So I’m hoping that you might be able to share some of your expertise in the area and possibly give me some direction. If possible, thank you very much.

I am trying to forego the need to boot Linux on an embedded system (wish for as few software layers as possible), instead I just want to load my own application code. However I still require the use of PCIe. I basically have a device tree (DT) that would normally be passed off to Linux for boot completion and need to understand what PCIe setup needs to still be done and what information is required from the DT to do so. So my code will complete the PCIe setup and then launch my application.

However, I’m new to both kernel-level development and PCIe. I’ve made good progress and have learned a ton but my main feedback is simply “Does PCIe MMIO R/W work yet? No? Ok… look for something else I’ve missed.” Main resources I’ve been referencing are the PCI System Architecture textbook (Mindshare, Inc.), PCIe Base Specification 3.0, PCI-to-PCI Bridge Architecture Spec., and this website (http://devicetree.org/Device_Tree_Usage ... ranslation).

Details of what I’m attempting are as follows:

My hardware configuration is fixed and does not need to be discovered, so at this time I am trying to write simplified code based on assumptions I know about the system. The DT I have states that a “pcie” node has a “ranges” property with the following entry:

0x0200_0000 0x0000_0000 0x8000_0000 0x0000_3fe0 0x0000_0000 0x0000_0000 0x7fff_0000

I’ve discerned this to mean that a 32 bit memory mapped space should exist with local pcie address 0x8000_0000 which is mapped to parent cpu address 0x0000_3fe0_0000_0000 and of size 0x7fff_0000 (as per devicetree.org).

The PCIe network stemming from the PCIe Host Bridge is simply one PCIe slot with an FPGA endpoint. I have learned from the various PCIe sources above that I need to configure the PCIe endpoint as well as the root to recognize the endpoint BAR. As it stands I am programming 0x8000_0000 into BAR0 of the endpoint and enabling bus mastering and memory space access bits in the command register. In the root’s configuration space I also load 0x8000 into the Memory Base Register (interpreted as 0x8000_0000) and 0xFFF0 into the Memory Limit Register (interpreted as 0xFFFF_FFFF). The root’s command register is already set to bus master and have memory space access.

After this I attempt MMIO R/W by doing assembly load/store commands to address 0x0000_3fe0_0000_0000, assuming/hoping this has been mapped to pcie address 0x8000_0000 via the MMU and/or PCIe Host Bridge already by chip firmware. However, these always fail… the read returning a 0xFFFF value. ChipScope monitoring of the endpoint shows no PCIe traffic.

Thanks again for your time. Best regards.


Re: PCIe Initialization

Postby support » Mon Apr 06, 2015 4:25 pm


I have to admit that I've never configured a PCIe link manually, so I can't share any experience in that field. As far as I know, this is a common case with TI's DSPs that have a PCIe interface -- since the DSP itself can't run Linux, I've been told that TI offers library routines that do the setup. This can be a good starting point.

Anyhow, as it seems like your main concern is not to have excessive software layers, maybe you want to look into the possibility of booting up Linux, and then shutting it down, taking advantage of the hardware configuration it has made. The bonus is that this is more sustainable than setting it up manually. Maybe use kexec() to call your own software, or possibly just halt the kernel just before it's about to mount a root filesystem, and jump to your own software after halting. It's not an odd idea to use Linux as a bootloader.

I hope this helped a bit, even though I can't address your question directly.

Posts: 760
Joined: Tue Apr 24, 2012 3:46 pm

Re: PCIe Initialization

Postby Guest » Wed Apr 08, 2015 7:18 pm

Hello Eli,

Thanks very much for your feedback.

I wasn't sure you would be able to comment directly but thank you for the suggestion of booting Linux and using kexec. I am considering it. I'll also take a look at what TI can show me.

Thanks again!

Best Regards,

Return to General PCIe