Link Speed Change

[Guest]

HI Eli,

I have a scenario where both the RC and EP initially set speed change bit to 5GT/s successfully, then I see a speed change as a result of unreliable link shortly after that; both RC and EP drop to 2.5GT/s. At this point RC does not reinitiate speed change to a higher link rate at GT/s again and hence EP stays at 2.5 GT/s. Does the PCIe standard allow EP to initiate a speed change to 5GT/s? I am looking at the PCIe 2.0 specs and it's unclear to me.

please help.

-H

[support]

Hello,

Frankly speaking, I'm not familiar with the physical layer of PCIe, but the fact that the speed dropped doesn't sound like a good sign to me in general.

From section 3.5.2 of the PCIe 2.1 spec:

"If repeated attempts to transmit
a TLP are unsuccessful, the Transmitter will determine that the Link is not operating correctly, and
instruct the Physical Layer to retrain the Link"

Maybe this caused the link speed to drop. Or maybe the training failed, so the 5 Gb/s viewed was only for the training attempt (I'm not sure what you meant with "successfully").

Anyhow, if you want to try pushing the speed up again, I would suggest forcing the link to L1 or L2 (push the link to a low power state by a command from the host) and then return to L0. A retraining of the link is due. I didn't dig deep into the spec to see if the link should remember its bad experience with 5 Gb/s, but I guess this is your best shot.

Hope this helps.
Eli

[Guest]

Thanks so much Eli for your helpful explanation. I'll try your suggestion and will let you know. I'm also looking at the spec again but it's still unclear to me what the limitations are. At any rate, I appreciate your help. thanks again!

[Guest]

I was reading the standard again and the following sections are still unclear to me and would like your help.

Section 6.11. in PCI EXPRESS BASE SPECIFICATION, REV. 3.0
Link Speed Management says the following:

The Target Link Speed field in the Link Control 2 register sets the upper bound for the Link speed.
Except as described below, the Upstream component must attempt to maintain the Link at the
Target Link Speed, or at the highest speed supported by both components on the Link (as reported
by the values in the training sets – see Section 4.2.4.1), whichever is lower.


If the reliability of the Link is unacceptably low, then either component is permitted to lower the
Link speed by removing the unreliable Link speed from the list of supported speeds advertised in
the training sets the component transmits. The criteria for determination of acceptable Link
reliability are implementation specific, and are not dependent on the setting of the Hardware
Autonomous Speed Disable bit


When a component’s attempt to negotiate to a particular Link speed fails, that component is not
permitted to attempt negotiation to that Link speed, or to any higher Link speed, until 200 ms has
passed from the return to L0 following the failed attempt, or until the other component on the Link
advertises support for the higher Link speed through its transmitted training sets (with or without a
request to change the Link speed), whichever comes first.


the spec doesn't mention that it's not okay for EP to re-initiate or negotiate speed change after failing the first link
speed at 5.0 GT/s. so does it mean that it's okay to try to set it manually? will this violate the standard?

[support]

Reading the text above, I get the impression that it's OK to re-initiate link speed negotiations. 200 ms or if the partner says it wants to. But this is just my reading of the text.

The last paragraph goes "... not permitted to ... until ... or until ... whichever comes first". So it basically says it's permitted after 200 ms or when the other side advertises a higher speed.

[Guest]

gotcha and thanks for your great help, Eli. I guess the next step is to explore the 200ms and the suggestion from you. it'd be interesting to see the actual behavior on diff host environment.

[Guest]

Eli,

what would be the best way to find out the max link speed capability on the host side if one were to write a fw to control link speed changes?

[support]

Well, that depends on the host and what you can do with it. Why not just read the datasheet?

If you can run Linux on the host, lspci with different verbose levels will tell you that.

[Guest]

there's no datasheet for this and hence i'm a little stuck. Currently working with two units; one does a speed link change right away but the second unit doesn't know what to do and the speed link changes to 2.5 from 5 Gb/s. According to the standard it seems that EP can reinitiate speed link negotiations after 200ms but with the autonomous speed disable bit there could be a race condition if both are happening at the same time.

[support]

Frankly speaking, this is beyond me. I don't usually deal with anything below the transaction layer.

But if I had a system wiggling between the two speeds, I would, frankly speaking, limit it to 2.5 Gb/s to avoid inconsistent behavior, or solve the problem that causes that uncertainty. Unless it was really for a short lab.