PCIe 2.0 and lane size

Post a reply

Confirmation code
Enter the code exactly as it appears. All letters are case insensitive.
Smilies
:D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :arrow: :| :mrgreen: :geek: :ugeek:
BBCode is ON
[img] is ON
[flash] is OFF
[url] is ON
Smilies are ON
Topic review
   

Expand view Topic review: PCIe 2.0 and lane size

Re: PCIe 2.0 and lane size

Post by support »

Hello,

The only thing that looks fishy to me is the 900 MB/s rate at 4x Gen2. You should see about 1600 MB/s at least. As for the extender cable, you can see if there are hardware level problems quite easily (I wouldn't bet my money on that direction):

http://billauer.co.il/blog/2011/07/pcie ... yer-error/

Unfortunately, I don't have any experience with neither the AXI-PCIe bridge, nor the CDMA core. I suggest taking this to Xilinx' forum.

Regards,
Eli

Re: PCIe 2.0 and lane size

Post by Guest »

I tried it with vc707. When PCIE-AXI core configured in 4x mode it says what it should. 5GT/s when it's configured as 5GT/s and 2.5GT/s when it is 2.5GT/s.
When I transfer the data it is about 800MB/s for 2.5GT/s (approximately what it should be) and for 5GT/s it looks like it slightly faster (about 900MB/s). On FPGA side there is AXI-PCIE core and CDMA core which does all the data transfer. Might it be because of the PCIe extender cable?

With 8x there is another story. When I configure AXI-PCIE as 8x 2.5GT/s (doesn't support 5GT/s) LnkCap is 8x 2.5GT/s, but LnkSta is 4x 2.5GT/s. Can't say where is the problem. PCIe slot on the motherboard is PCIe Gen2 16x. J49 on FPGA jumper is in 8x mode.

Re: PCIe 2.0 and lane size

Post by support »

Hello,

What does lspci -vv (run as root) tell you about the device?

For example, my VGA card gives this:

Code: Select all
01:00.0 VGA compatible controller: ATI Technologies Inc RV710 [Radeon HD 4350] (prog-if 00 [VGA controller])
        Subsystem: Giga-byte Technology Device 21ac
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 48
        Region 0: Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Region 2: Memory at fbce0000 (64-bit, non-prefetchable) [size=64K]
        Region 4: I/O ports at ae00 [size=256]
        [virtual] Expansion ROM at fbc00000 [disabled] [size=128K]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee8800c  Data: 4172
        Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Kernel driver in use: radeon
        Kernel modules: radeon


Note the LnkCap and LnkSta entries: The first says what the device reports that it's capable of, and the second what was actually negotiated.

The terminology of different versions of lspci varies.

Regards,
Eli

PCIe 2.0 and lane size

Post by Guest »

Hi! Maybe it is wrong place to ask but I'll still try :).

I wrote a (DMA) linux driver for Xilinx deviced (AXI-PCI core) and tested it with zc706 and vc707 devices. It works fine, but it looks like that the speed is limited to PCIe Gen1 4lane and I have absolutely no idea why. It never goes above 800 MB/s. I can only slow down it in half by changing the switch position on the board.
Lane of course might be broken, but I don't undersand what is wrong with Gen1/Gen2 thing. I tried it on motherboards with PCIe Gen2 support and on boards with Gen1 only. No difference. I think that I miss something on the kernel side or on the fpga side. It looks like that I need to switch it somewhere somehow. In AXI-PCI core settings there is only one tab which seems related: the one, where you change number of lanes and the link speed. Changing these setting actually change absolutely nothing for me. It is always around 800MB/s or slower.

Top