questions regarding xillybus and DMA

Questions and discussions about the Xillybus IP core and drivers

questions regarding xillybus and DMA

Postby Guest »

Hi all, I am a newbie to embedded computing. I installed Xillinux1.1 in my ZedBoard and connected my logic design to Xillybus through the application FIFOs. A C project is also created for talking to the FPGA through the device files( xilllybus_read_32 and xillybus_write_32) . My logic design is for a halftoning algorithm. So far the prototyping has been completed and the testing results are satisfying. However, I am still confused about the xillybus architecture and its transport mechanism. The questions accumulated are listed as below.

1. In XPS, the xillybus IP core is instantiated in the PL. In ISE, there is another module called xillybus_core. I assume the xillybus IP in XPS is a PCIe IP with AXI interface while the xillybus_core in ISE packs and depacks the TLPs. Is my understanding correct?

2. I am not much into DMA. But at least, there should be a DMA controller for DMA. In XPS, the DMA controller( the green block named after "DMA8 Channel") seems not enabled (none of its interfaces are enabled). Where is the actual DMA controller for xiilybus loacted?

3.Is there any documentation specifying how the host driver works? I tried to read the driver source code xillybus.c. But it is too hard for me to understand such a complicated project. I am eager to know how the driver switches on/off the DMA transfer.

4. There is an interrupt from Xillybus to the host. When will the XIllybus trigger the interrupt? Is the interrupt handler included in the host driver?

5. Also there is another interrupt from the host to the Xillybus? When will the host trigger this interrupt? How does Xillybus respond?

6. I use plain read()/write() to send/receive data. My code follows the template of the C code example from http://xillybus.com/tutorials/vivado-hls-c-fpga-howto-2. A typical loop used in the C code is like the following :

while (donebytes < sizeof(struct packet) * N) {
rc = write(fdw, buf + donebytes, sizeof(struct packet) * N - donebytes);

if ((rc < 0) && (errno == EINTR))
continue;

if (rc <= 0) {
perror("write() failed");
exit(1);
}

donebytes += rc;
}

We keep doing read()/write() in a loop until the desired number of bytes is transferred. I am curious about when a single read()/write() returns? Take write() for instance, does it return when the DMA buffers are full?

7. When read() is called and the FPGA cannot send any data in response (the FIFO is empty), it seems that the current thread pauses until the arrivals of the data. Why could such a pause happen?

I am looking forward to any response regarding these questions. Thank you in advance.
Guest
 

Re: questions regarding xillybus and DMA

Postby support »

Hi,

As you've surely noticed, Xillybus is a complicated beast, so I won't be able to cover general "how does it work" questions. But I'll try. :)

1. In XPS, the xillybus IP core is instantiated in the PL. In ISE, there is another module called xillybus_core. I assume the xillybus IP in XPS is a PCIe IP with AXI interface while the xillybus_core in ISE packs and depacks the TLPs. Is my understanding correct?

xillybus_core is the actual IP core. xillybus.v or xillybus.vhd is just a wrapper, which is there to present a more convenient interface.

There is no PCIe logic at all in Xillybus for Zynq, and accordingly no TLP to pack or unpack. The core talks directly with the AXI bus.

2. I am not much into DMA. But at least, there should be a DMA controller for DMA. In XPS, the DMA controller( the green block named after "DMA8 Channel") seems not enabled (none of its interfaces are enabled). Where is the actual DMA controller for xiilybus loacted?

The DMA controllers in XPS relate to the processor's own controllers, which can be programmed for copying chunks of data. But this is by far not the most common way to perform DMA operations on an AXI bus. Xillybus, like most other peripherals, requests DMA cycles directly from the AXI bus by attaching to it through a master port.
3. Is there any documentation specifying how the host driver works? I tried to read the driver source code xillybus.c. But it is too hard for me to understand such a complicated project. I am eager to know how the driver switches on/off the DMA transfer.

Unfortunately, there is no such documentation. Anyhow, the driver doesn't switch DMA transfers on and off -- it's merely tells the hardware where the buffers are, and gives it permission to access them. It's the hardware that initiates the actual transfers.

4. There is an interrupt from Xillybus to the host. When will the XIllybus trigger the interrupt? Is the interrupt handler included in the host driver?

The interrupt handler in the driver is xillybus_isr(). The hardware triggers this interrupt, well, when it has something to tell the driver. As the mechanism is rather complex, I won't dive into the details.

5. Also there is another interrupt from the host to the Xillybus? When will the host trigger this interrupt? How does Xillybus respond?

No, there's no hardware from the host. The host writes to the hardware's registers when it needs to notify it about something.

6. I use plain read()/write() to send/receive data. My code follows the template of the C code example from http://xillybus.com/tutorials/vivado-hls-c-fpga-howto-2. A typical loop used in the C code is like the following :

[...]

We keep doing read()/write() in a loop until the desired number of bytes is transferred. I am curious about when a single read()/write() returns? Take write() for instance, does it return when the DMA buffers are full?

Assuming that you're using the demo bundle's IP core, write() just copies the data into a DMA buffer and returns immediately. The only reason it has not to return right away is if there's no space in any DMA buffer, in which case it waits (sleeps or "blocks") until such space is available. The actual data transfer takes place "in the background".

Please refer to the "Xillybus host application programming guide for Linux" guide, which deals with these issues in depth.

7. When read() is called and the FPGA cannot send any data in response (the FIFO is empty), it seems that the current thread pauses until the arrivals of the data. Why could such a pause happen?

Indeed, read() waits for data to arrive by blocking. I assume that you had one thread writing and one thread reading data. As your application logic handles its data at a certain pace, and the thread reading data just tries to slurp as much as it can, it's quite natural that it will finish up all data momentarily.

I hoped this covered most of what you wanted to know.

Regards,
Eli
support
 
Posts: 802
Joined:

Re: questions regarding xillybus and DMA

Postby Guest »

Eli,

Thank you so much for answering my questions. It cleared up most of my confusions. I really appreciate it a lot.

And I am still curious about the behavior of the xillybus interrupt. Would this interrupt be triggered during each read()/write() opertion? Or does it just work for special occasions?

In my original post, I forgot to include a question about changing the PL clock frequency.The current PL clock is 100MHz. However, its maximum ratio could be 250MHz. As mentioned in section 5.3 of http://xillybus.com/downloads/doc/xilly ... nq-1.1.pdf, the clock frequency change cannot be done by simply configuring the clock generation unit in XPS. It is necessary to change the value in the respective register which is hardcoded in arch/arm/mach-zynq/slcr.c. However I cannot find that source file at all (I am using Xillinux1.1). Could you specify the details of enhancing the PL clock in details?

Thank you again.
Guest
 

Re: questions regarding xillybus and DMA

Postby support »

Hi,

As I mentioned before, the interrupts are part of a non-trivial mechanism for passing information to the host. In most data flow conditions, they are not related directly to the read() or write() calls.

As for raising the clock frequency: There's a reason why it's set to 100 MHz, and it's quite unlikely that raising it will do any good for you. In particular, you will most probably not get a higher bandwidth performance, because it's limited, in most cases, by the processor, which is busy with synchronizing the cache. If you want to get a higher bandwidth, go for Xillinux-1.3, which is based upon the ACP AXI port.

Also, if you raise the clock beyond 150 MHz, you'll start having problems meeting the timing constraints during the implementation.

Anyhow, if you still want to do this, the slcr.c file in the Linux kernel sources. Please refer to /usr/src/xillinux/ in Xillinux' file system.

Regards,
Eli
support
 
Posts: 802
Joined:

Re: questions regarding xillybus and DMA

Postby Guest »

Hello Eli,
Thanks for your quick reply. I was off-line for the past two weeks. So I did not see your post until today.
I am using xillybus_write_32 and xillybus _read_32. Their original bandwidth is absolutely large enough for my application.
The current bottleneck of my project is actually the processing speed of the application logic in the FPGA. My logic architecture is using bus_clk as the input clock. Therefore, a higher clock rate will benefit its performance.
As I am using Xillinux1.1, I cannot find out slcr.c in the /usr/src/xillinux/. I also tried to search slcr.c in the whole file system. But I only got slcr.h and slcr.c never showed up.
Could you give me more instructions on changing the PL clock rate?
Thanks a ton!!
Bob
Guest
 

Re: questions regarding xillybus and DMA

Postby support »

Hello,

Changing the PL clock rate is a headache, and doing so is probably a bad idea. You want your own logic to run faster, not Xillybus.

What I suggest, is to use another clock for your logic. You can generate a clock with arbitrary frequency by adding an MMCM or PLL to the system, which can use either the PL clock or an external oscillator as a reference clock. If you use the PL clock, you'll have the timing constraints for this user clock added automatically.

As the interface with Xillybus takes place through a FIFO, you may replace the single-clock FIFO in the demo design with a dual-clock ("asynchronous") FIFO, which will bridge between the two clock domains.

Regards,
Eli
support
 
Posts: 802
Joined:

Re: questions regarding xillybus and DMA

Postby Guest »

Eli,
Thanks for your advice. I added a MMCM core to xillydemo today. This core was generated by xilinx clocking wizard 3.6. I used bus_clk as its input clock, the output clock is 800Mhz.
Asynchronous FIFOs are also applied for interfacing, The implementation process was successful. New bistream file and new boot.bin were also generated successfully. But afterwards xillinux failed to boot up. Only the green power light was on and other LEDs were off. The VGA had no outputs.
Then I tired to reload my original boot.bin to the SD card. The booting was successful again.
Do you have any clue what might cause the booting failure with the new architecture?
Thank you so much.
Bob
Guest
 

Re: questions regarding xillybus and DMA

Postby support »

Hello,

Always check if your design met timing (that is, the timing constaints were met) before applying it to the FPGA.

In your case, you picked a clock that is way too fast, so I have no doubt that timing failed. And when timing fails, everything is possible.

One could ask why the tools allow you to create a bitfile with a design that won't work anyhow -- well, that's how things work with all FPGA tools.

Regards,
Eli
support
 
Posts: 802
Joined:

Re: questions regarding xillybus and DMA

Postby Guest »

Eli,
Thank you for the reply.
You are right. The ISE did not terminate the bitstream generation process even if the design failed to satisfy the timing constraints.
I tried to set my new clock (output clock from the MMCM) to 200MHz. According to the consequent post-map timing report, there were still over 100 paths that failed to meet the timing constraints. Most of them had unacceptable routing delays. In order to get better timing,I was thinking of upgrading Xillinux from 1.1 to 1.3 and migrating my application logic design to Vivado2014.1.
The upgrade was successful. Before migrating the application logic, I want to customize xillybus in your IP factory for a better bandwidth. But the downloaded files for the customized xillybus IP core seem suitable for the ISE project only. Are these files also working for the Vivado project?
Regards,
Bob
Guest
 

Re: questions regarding xillybus and DMA

Postby support »

Hi,

The binaries from the IP core factory are suitable for Vivado 2014.1 as well.

However, upgrading to Xillinux 1.3 is not likely to solve your timing issue. Switching to Vivado may change things slightly, but odds are that the only way to fix the timing issue is to work with the design itself: Reduce the frequency, or make changes in the logic. There is almost never a quick fix for this.

Regards,
Eli
support
 
Posts: 802
Joined:


Return to Xillybus