Hi,
Just curious to know how the switching of the packets happens in PCIe. Is it similar to the packet switching that happens in a Ethernet/IP Switch?
I briefly looked at the few resources on PCIe, they do not talk much about it.
Regards,
--Sreeni S
Reg. Switching Used in PCIe
2 posts
• Page 1 of 1
Re: Reg. Switching Used in PCIe
by support »
Hello,
PCIe switching typically takes place in, well, PCIe switches. These are chips that have one upstream port (pointing towards the root port, i.e. the host) and several downstream ports that go to devices (and possibly, but rarely, other switches).
The routing rule for any TLP packet that arrives from a downstream port will almost always forward it to the upstream port. The only exception is when one device sends a TLP to another (peer-to-peer) which is an allowed, however rather advanced feature.
Packets coming from the upstream port are routed to one of the downstream ports.
The routing rules depend on the type of addressing. Some TLP packets (e.g. those request a read or write operation) are routed according to the memory address (the PCIe switch is programmed to know which of its downstream port covers which address range during bus enumeration). Other TLP packets (read completions in particular, which contain the data is response to a read request) have bus addressing, saying which device should have the packet by its bus address (as it appears in lspci). Once again, the PCIe switch is informed about these addresses during enumeration.
In practice, PCIe packet switching doesn't happen all that much, because these PCIe switches are not so commonly used: Processors and their companion I/O controllers tend to supply enough PCIe slots directly. So the vast majority of PCIe devices are connected point-to-point with the I/O controller or even with the processor.
And even though the PCIe bus appears to be made of a root port and one large PCIe switch which splits the root port into several downstream ports, this is most likely an illusion presented to software. In reality, odds are that each port has a separate bridge from the processor specific internal bus to PCIe. It's easier to implement it this way.
Regards,
Eli
PCIe switching typically takes place in, well, PCIe switches. These are chips that have one upstream port (pointing towards the root port, i.e. the host) and several downstream ports that go to devices (and possibly, but rarely, other switches).
The routing rule for any TLP packet that arrives from a downstream port will almost always forward it to the upstream port. The only exception is when one device sends a TLP to another (peer-to-peer) which is an allowed, however rather advanced feature.
Packets coming from the upstream port are routed to one of the downstream ports.
The routing rules depend on the type of addressing. Some TLP packets (e.g. those request a read or write operation) are routed according to the memory address (the PCIe switch is programmed to know which of its downstream port covers which address range during bus enumeration). Other TLP packets (read completions in particular, which contain the data is response to a read request) have bus addressing, saying which device should have the packet by its bus address (as it appears in lspci). Once again, the PCIe switch is informed about these addresses during enumeration.
In practice, PCIe packet switching doesn't happen all that much, because these PCIe switches are not so commonly used: Processors and their companion I/O controllers tend to supply enough PCIe slots directly. So the vast majority of PCIe devices are connected point-to-point with the I/O controller or even with the processor.
And even though the PCIe bus appears to be made of a root port and one large PCIe switch which splits the root port into several downstream ports, this is most likely an illusion presented to software. In reality, odds are that each port has a separate bridge from the processor specific internal bus to PCIe. It's easier to implement it this way.
Regards,
Eli
- support
- Posts: 802
- Joined:
2 posts
• Page 1 of 1