Hello,
I wonder what is going to happen if I contentiously send data to the FPGA, but the FPGA is not consuming the xillybus fifo?
At some point the fifo will be filled and on host side the reserved memory will be full as well. What happens if I try to write more then?
Will the call be stalled until the fifo gets some space?
Another question: is there a limitation on the amount of streams? If I understood correctly the streams do not effect each others throughput as long as they dont send in the same timeframe.
Is that right, or is the maximum throughput statically divided? For my application I would like to use 8 asynchronous streams.
Thanks in advance!
What happens if the FPGA Fifo is read too slow
8 posts
• Page 1 of 1
Re: What happens if the FPGA Fifo is read too slow
by support »
Hello,
I'll take them one by one.
Exactly. On the host, the call to write() will block, which means that the process will sleep until there is buffering space to write the call's content to. This is exactly the way a TCP/IP link behaves if one side writes data and the other side doesn't consume it.
Note that you can control the host's DMA buffer size in the IP core factory, so there can be several hundreds of MBytes of data in the buffer before it blocks (if that's what you chose to have).
In theory, not. The practical limit is probably filling the FPGA, or possibly not meeting timing constraints because of the excessive logic. The core has been tested with 60 streams total, which is probably more than any real-life user requires. So it's practically unlimited.
Xillybus is designed to attempt dividing the bandwidth as clever as possible. In other words, as long as the sum of bandwidth requirements is below the total limit (which is published for each platform on the download page), no bandwidth limitation will be felt on either. If the total required bandwidth exceeds the limit, it's divided more or less evenly among the streams (with the exception of streams that require less than their fair share, which will get as much as needed, and leave the rest for the other streams).
So as long as the total momentary bandwidth requirement is below the limit, all streams flow with no limitations.
Regards,
Eli
I'll take them one by one.
stdoubleu wrote:I wonder what is going to happen if I contentiously send data to the FPGA, but the FPGA is not consuming the xillybus fifo?
At some point the fifo will be filled and on host side the reserved memory will be full as well. What happens if I try to write more then?
Will the call be stalled until the fifo gets some space?
Exactly. On the host, the call to write() will block, which means that the process will sleep until there is buffering space to write the call's content to. This is exactly the way a TCP/IP link behaves if one side writes data and the other side doesn't consume it.
Note that you can control the host's DMA buffer size in the IP core factory, so there can be several hundreds of MBytes of data in the buffer before it blocks (if that's what you chose to have).
stdoubleu wrote:Another question: is there a limitation on the amount of streams?
In theory, not. The practical limit is probably filling the FPGA, or possibly not meeting timing constraints because of the excessive logic. The core has been tested with 60 streams total, which is probably more than any real-life user requires. So it's practically unlimited.
stdoubleu wrote:If I understood correctly the streams do not effect each others throughput as long as they dont send in the same timeframe.
Is that right, or is the maximum throughput statically divided? For my application I would like to use 8 asynchronous streams.
Xillybus is designed to attempt dividing the bandwidth as clever as possible. In other words, as long as the sum of bandwidth requirements is below the total limit (which is published for each platform on the download page), no bandwidth limitation will be felt on either. If the total required bandwidth exceeds the limit, it's divided more or less evenly among the streams (with the exception of streams that require less than their fair share, which will get as much as needed, and leave the rest for the other streams).
So as long as the total momentary bandwidth requirement is below the limit, all streams flow with no limitations.
Regards,
Eli
- support
- Posts: 802
- Joined:
Re: What happens if the FPGA Fifo is read too slow
by Guest »
Great, thank you for your fast and adequate answer on all my questions.
All in all, sounds perfect for my application.
I will give it a try and report if I am happy or not
All in all, sounds perfect for my application.
I will give it a try and report if I am happy or not
- Guest
Re: What happens if the FPGA Fifo is read too slow
by Guest »
Hello,
so, I am half happy
A small sketch of my application:
I have 8 stream (to the FPGA and to host).
On all streams to the FPGA data is supplied continuously, which might cause the previously described blocking
because one stream to the FPGA is consumed less than others.
I cannot guarantee that all streams are consumed with the same priority.
Each stream is supplied by one thread in my host application.
On the other side one stream from the FPGA to the host is consumed.
At a certain event in this stream (FPGA to host) my application can be considered as "done".
But at this moment it might be that some of the threads (host to FPGA) are blocking.
And my problem now is how to close those streams as the thread is blocked drying to write more data.
In fact my threads are in Java, using a library to handle native calls to xillybus.
A simple stop() doesn't kill the thread.
Is there any c call to close the xillybus file with no regards for currently used resources?
(I know not the best approach, but I dont see any other approach which enables me to use interrupts)
so, I am half happy
A small sketch of my application:
I have 8 stream (to the FPGA and to host).
On all streams to the FPGA data is supplied continuously, which might cause the previously described blocking
because one stream to the FPGA is consumed less than others.
I cannot guarantee that all streams are consumed with the same priority.
Each stream is supplied by one thread in my host application.
On the other side one stream from the FPGA to the host is consumed.
At a certain event in this stream (FPGA to host) my application can be considered as "done".
But at this moment it might be that some of the threads (host to FPGA) are blocking.
And my problem now is how to close those streams as the thread is blocked drying to write more data.
In fact my threads are in Java, using a library to handle native calls to xillybus.
A simple stop() doesn't kill the thread.
Is there any c call to close the xillybus file with no regards for currently used resources?
(I know not the best approach, but I dont see any other approach which enables me to use interrupts)
- Guest
Re: What happens if the FPGA Fifo is read too slow
by support »
Hello,
A write() call returns when it has been fulfilled, but it may also return with an -EINTR return value (or with the number of bytes written in some cases) if the process/thread got a POSIX signal. It doesn't have to be a signal that would kill the process or stop it -- just any signal will make the call return immediately.
So in a way, it's quite odd that you fail to stop the threads. But Java has its ways.
What I may suggest is to attempt to stop the thread, and then send a signal to it. This might get the thread killing unstuck.
If you're using Windows, I think there's some signal emulation mechanism.
Note that an attempt to close a file when there's still data pending in the DMA buffers will block for up to a second, to give the FPGA a chance to fetch the data. If a second has elapsed, and there is still data in the buffers, the file is closed anyhow, and data is potentially lost. So you may see a second's delay -- that's normal.
Regards,
Eli
A write() call returns when it has been fulfilled, but it may also return with an -EINTR return value (or with the number of bytes written in some cases) if the process/thread got a POSIX signal. It doesn't have to be a signal that would kill the process or stop it -- just any signal will make the call return immediately.
So in a way, it's quite odd that you fail to stop the threads. But Java has its ways.
What I may suggest is to attempt to stop the thread, and then send a signal to it. This might get the thread killing unstuck.
If you're using Windows, I think there's some signal emulation mechanism.
Note that an attempt to close a file when there's still data pending in the DMA buffers will block for up to a second, to give the FPGA a chance to fetch the data. If a second has elapsed, and there is still data in the buffers, the file is closed anyhow, and data is potentially lost. So you may see a second's delay -- that's normal.
Regards,
Eli
- support
- Posts: 802
- Joined:
Re: What happens if the FPGA Fifo is read too slow
by stdoubleu »
Thank you again for your answer. In fact, I didnt try to kill each thread twice.
I guess the combination of using Java and native c calls somehow blocks the "clean" killing of the thread.
Thus, I changed my FPGA design a bit so that the buffers in the FPGA are still consumed after the application actually satisfied its task.
Clearly this causes that the write call will return eventually and my host application can react properly on the thread's interrupt and finally the xillybus streams can be closed.
I am just wondering about something else:
Do I really need these native calls to C to use xillybus?
I mean, the xillybus streams are used like files.
Can't I just open and use those files in the java fashion?
Yeah, I know, I could simply try, but maybe someone tried this before and time is rare
I guess the combination of using Java and native c calls somehow blocks the "clean" killing of the thread.
Thus, I changed my FPGA design a bit so that the buffers in the FPGA are still consumed after the application actually satisfied its task.
Clearly this causes that the write call will return eventually and my host application can react properly on the thread's interrupt and finally the xillybus streams can be closed.
I am just wondering about something else:
Do I really need these native calls to C to use xillybus?
I mean, the xillybus streams are used like files.
Can't I just open and use those files in the java fashion?
Yeah, I know, I could simply try, but maybe someone tried this before and time is rare
- stdoubleu
- Posts: 6
- Joined:
Re: What happens if the FPGA Fifo is read too slow
by support »
Hi,
Indeed, Xillybus can be used with any programming language, not just C. Perl, Python, C# and other languages I've never heard have been used successfully on both Linux and Windows.
The only possible problem is on Windows, if the language's implementation of open() checks if the file is a "regular file" or refuses to pass through something that starts with "\\.\". This is why Xillybus can't be accessed from Window's shell prompt, for example. With Linux, anything goes.
As for Java -- on Linux, no doubts it will work well. On Windows, that needs to be checked, if this is your case, but I would bet a pizza on that it will work. And surely, I wouldn't have bothered making C wrappers on my first attempt.
Regards,
Eli
Indeed, Xillybus can be used with any programming language, not just C. Perl, Python, C# and other languages I've never heard have been used successfully on both Linux and Windows.
The only possible problem is on Windows, if the language's implementation of open() checks if the file is a "regular file" or refuses to pass through something that starts with "\\.\". This is why Xillybus can't be accessed from Window's shell prompt, for example. With Linux, anything goes.
As for Java -- on Linux, no doubts it will work well. On Windows, that needs to be checked, if this is your case, but I would bet a pizza on that it will work. And surely, I wouldn't have bothered making C wrappers on my first attempt.
Regards,
Eli
- support
- Posts: 802
- Joined:
Re: What happens if the FPGA Fifo is read too slow
by stdoubleu »
eli wrote:[...]
And surely, I wouldn't have bothered making C wrappers on my first attempt.
Hehe, sure, but the C examples of Xillybus are so excellent that I had to integrate them even without thinking that a Java-only solution might be possible.
Sometime we like to make life harder than it is.
However, thanks again!
- stdoubleu
- Posts: 6
- Joined:
8 posts
• Page 1 of 1