Tandem PCIe - MCAP

Questions and discussions about the Xillybus IP core and drivers

Tandem PCIe - MCAP

Postby Guest » Wed Apr 26, 2017 11:40 am


I am experimenting with your xillybus Core + Windows Driver. When used without Tandem Configuration everything works fine.
However, when used with Tandem PCIe in combination with the new MCAP Interface for Ultrascale Devices,
after programming Stage 2 Bitream (which contains Xillybus), the Windows Driver Indicates "No Response from FPGA. Aborting."

Since the MCAP uses the Xilinx driver, the process is as follows:
1. Switch to MCAP Driver
2. Configure Stage 2 Bitstream using MCAP
3. Switch back to Xillybus Driver

The FPGA CONFIG_STATUS register indicates that configuration is done and the PXIe Core is also responsive.

As far as i know, the Windows driver is not open source, so i took a quick look at the Linux Driver.
It seems like "xillybus_endpoint_discovery" function times out waiting for the XILLYMSG_OPCODE_QUIESCEACK opcode MSI response from the FPGA.
So i am pretty sure the probing works similar on windows.

However, when Vivado is used to configure the Stage 2 instead of the MCAP Driver, everything works fine.
Also if a warm-boot is performed after a MCAP Bitstream write process, the xillybus driver works as expected again.

My rough guess is that the common ground is the PCIe core and how it is used/configured by the two drivers.

I would appreciate if you could give the problem a thought.

Re: Tandem PCIe - MCAP

Postby support » Wed Apr 26, 2017 1:05 pm


I'm personally not very fond of the Tandem concept. It's a complicated solution for a problem that is often easily solved with other means. Also, it often requires the host to respond properly to events that never happen with "normal" PCIe cards, in particular the coming and going of PCIe links. Even though PCIe hotplugging is mandatory per PCIe spec, real-life computer hardware often support what is practically encountered, rather than what it should support according to spec. So relying on Tandem configuration may lead to problems when changing the computer hardware, for no apparent reason.

What I usually suggest, is to hold the processor in reset until the FPGA is configured. It's very simple to implement: Let's say that the reset pin's asserted level is low. Pull the PC's reset pin down to GND with a resistor, and connect the reset pin directly to the FPGA. Then, in the FPGA's logic, assign the value of this pin constantly high. So while the FPGA isn't configured, its pins are high-Z, and hence the reset will be asserted due to the pulldown resistor. When the FPGA wakes up, the reset is deasserted.

It's simple and failsafe, except when you want to turn on the PC without the FPGA, in which case a simple jumper can be used to bypass this mechanism.

Another possible solution is to delay the reset deassertion with a timer chip. It depends on how you want the system to behave if the FPGA isn't configured due to a failure.

Both solutions are much simpler than Tandem configuration, and add a second or two to the PC's boot time.


Regardless of the above, these are my answers to your questions:

To begin with, yes, the Windows and Linux drivers for Xillybus are virtually the same, so using the Linux driver to learn what the Windows driver is up to makes sense.

The "No response from FPGA" occurs when the device driver is loaded (due to a match of its Vendor/Product IDs), but the FPGA didn't respond with an interrupt to the first command that is sent to its BAR space. Why this happens can have many reasons.

If you look at the LEDs while the computer boots with Xillybus configured properly on the FPGA, you'll see a brief flash on one of them. That indicates the transmission of an interrupt towards the host. Odds are that you won't see such a flash after the MCAP configuration. But if you do, it's clear that the host->FPGA write worked fine, and the interrupt was missed.

Since a warm reboot revived Xillybus, I would also suggest trying "Scan for hardware changes" in the Device Manager. This is actually the trick commonly used after configuring the FPGA while the PC is on (which is illegal, will probably leak kernel RAM, but yet it often works).

As for trying to figure out what's going on, I can think of two possible kinds of problems:

(1) Software: The handover of the PCIe interface in Windows wasn't done properly by Xilinx' driver, so Xillybus' driver didn't get access to the resources. I wouldn't bet my money on this one, because odds are that if the resources weren't obtained by Xillybus' driver, it would fail before attempting to talk with the hardware.

(2) Hardware: The switchover from one PCIe configuration to another was good enough to allow the retrieval of the updated Vendor/Product IDs, but failed shortly after. Maybe there's a need to set some parameter in Xilinx' PCIe block in Xillybus' bundle as well. I would also suggest adapting the first stage PCIe block link parameters to those in Xillybus' to smooth the transition from one to another.

Hope this helped.

Posts: 588
Joined: Tue Apr 24, 2012 3:46 pm

Return to Xillybus