64-bit data, 32-bit address seekable Xillybus

[Guest]

Hi:

I've created a Xillybus core with a bidirectional memory-mapped interface on a B core, 64-bits data with 32-bit address width on an Ultrascale system.

I understand that if I use lseek to go to a file position, I need to do it in multiples of 8: so if I want to go to the upper quarter of the address space (0xC000_0000), I would do something like

Code: Select all

off_t addr = 0x600000000;
lseek(fd, addr, SEEK_SET);


This is on a system where sizeof(off_t) == 8, so this should be fine. However, when I do that, in the FPGA I actually see the address come out as 0x4000_0000 with the addr_update. A little playing around with addresses seems like what happens is that if I send a 64-bit integer to lseek, for a 32-bit address space, bits[33:3] are output as user_xx_addr[30:0], but nothing I can do seems to ever set bit 31 - in other words, it looks like it's a 31-bit address space.

There's a bit of a note in the revision B user notes:

When using these wider data interfaces, it's important to observe that since the natural data element of the PCIe bus is a 32-bits, some software safeguards that prevent erroneous use of streams do not apply when the data width goes above 32 bits. For example, any read() or write() call for a 64-bit stream must have a lengh that is a multiple of 8. Likewise, the seek positions for a seekable 64-bit wide stream must be a multiple of 8 to achieve any meaningful result. The software will however only enforce that it's a multiple of 4.

which makes me wonder whether this might be intentional (like, the address space is 32-bits address, 32-bits data underneath, but mapped to 31-bit address, 64-bit data at the interface) but it'd be nice to confirm that.

[support]

Hello,

Using the full 32 bits on a 64-bit interface is a bit of an exotic use case, and it appears like the docs should be updated to cover the issue you've just encountered.

It goes like this: The host driver treats anything wider than 32 bits as a 32-bit interface. It therefore sends the FPGA the position divided by 4, as a 32-bit word. In the FPGA, this is corrected by shifting the position one bit to the right, so the address is divided overall by 8.

Hence when you chose the address 0x600000000, it was divided by 4, and became 0x180000000. But there are only 32 bits of addressing, so it was reduced to 0x8000000. Then, when the FPGA divided it further by 2, it ended up as 0x40000000, exactly like you saw.

So the bottom line is that when you work with a 64 bit interface, the address space is indeed just 31 bits.

I'll try to find the appropriate place to add a remark on this.

Apologies for the confusion.

Regards,
Eli

[support]

Follow up: A warning is now issued in the IP Core Factory when the address is limited because of this specific situation.

It's not documented elsewhere, because seekable streams wider than 32 bits are rarely requested. Adding a note on this specific case is likely to cause more confusion than help.

The original idea behind seekable streams was to access a register space or a block RAM, so it's not clear why 32 bits would be needed. But apparently, there's always someone who needs any possible use case.

Regards,
Eli

[barawn]

The warning note has a slight mistake in it: it says "Even though the address is defined as 32 bits, only the 31 upper bits will be effective" when it should say "only the 31 lower bits will be effective."

As for the use case, it's a similar situation to how 32-bit microcontrollers use their address space: once you've got the address bits, you might as well use them for something (and 16 bits is too small anyway). So, for instance, the top 8 bits of the space are used for individual byte write enables (avoiding read/modify/write patterns).

[support]

The warning note has a slight mistake in it: it says "Even though the address is defined as 32 bits, only the 31 upper bits will be effective" when it should say "only the 31 lower bits will be effective."

Ayee. Thanks, that has been corrected. I guess Friday the 13th isn't such a good day, after all.

Regards,
Eli