PCIe® 5.0 and the Next Generation of Machine Learning and Artificial Intelligence

From social media feeds to autonomous vehicles and home appliances, artificial intelligence (AI) and the machine learning (ML) algorithms are more prevalent in our lives than ever before. Already valued at 100 billion USD, the AI and ML market is expected to grow twentyfold to reach two trillion by 2030. Powering this massive market are high data bandwidth and low latency transport channels enabled by the new PCI Express® (PCIe®) 5.0 architecture.

What’s new in the PCI Express® (PCIe®) 5.0 architecture?

Released in May 2019, PCIe 5.0 offers double the data transfer that PCIe 4.0 was capable of. This means that all PCIe peripherals such as graphics adapter cards, network interface cards (NICs), storage accelerator devices, solid-state drives (SSDs), and to a certain extent, graphic processing units (GPUs), can now enjoy higher performance.

How PCIe 5.0 supports ML and AI applications

To understand why PCIe 5.0 is significant when it comes to supporting ML and AI applications, we must first know what these next generation applications demand:

1. The capability to attach to as many compute chips and network devices as possible, which is why widely adopted interconnect protocols like PCIe are often selected for accelerator solution versatility.
2. Standard software and popular programming models for easy discovery, programming, and management.
3. Software must also be easily developed. In the case of PCIe, its widespread adoption allows devices to be deployed nearly instantly.

PCIe 5.0 also provides native support to carry additional protocols over low latency non-return to zero physical layers, which lets CPUs keep up with increasingly large volumes of data that will no doubt spike with the proliferation of AI and ML devices. 

How fast is PCIe 5.0?

Most processors and motherboards on the market are currently supported by PCIe Gen 4, a groundbreaking standard when it was first released in 2017. However, many users are starting to experience bottlenecks in its performance. For example, SSDs that come in the M.2 NVMe form factor can only facilitate 8GB/s of constant data transfer across four lanes. PCIe 5.0 SSDs, on the other hand, doubles that speed to 16 GB/s, with 32 lane devices exhibiting 128GB/s of bandwidth.

Required components of PCIe 5.0

Products such as AMD’s AM5 and Intel’s LGA 1700 motherboards are already announced to support PCIe 5.0, but many more new products are expected to hit the market. It’s important to note, however, that both ends of a connection must support PCIe 5.0 in order to unlock higher data speeds. Theoretically, a PCIe 5.0 SSD paired with a PCIe Gen4 motherboard will be capable of running at maximum Gen4 bandwidth, 16GT/s. Thus, it is capable of transmitting 64GB/s on x16 link width.

How PCIe 5.0 affects device development

Why are there so few PCIe 5.0 devices on the market?

While PCIe 5.0 is necessary for the development of faster devices, its existence alone does not improve GPUs or SSDs, especially ones that already have enough bandwidth to start with. In fact, there aren’t many PCIe 5.0 capable devices on the market right now even though the standard has been released for four years. 

That said, PCIe 5.0 will definitely play a pivotal role in expediting sequential reads and writes for higher transfer speeds, particularly when it comes to transmitting larger files. Crucial T700 and Seagate FireCuda 540 are some of the Gen 5 SSDs that already exhibit significant improvements over their Gen 4 counterparts. PCIe Gen 5 SSDs are also beginning to surface, although more affordable components will need to come to the forefront before PCIe 5.0 can truly become mainstream.

Cost limitations of PCIe 5.0

The main concern with upgrading to PCIe 5.0 lies in cost. In most cases, the motherboard, CPU, and even memory will need upgrades in order to be compatible with PCIe 5.0 devices. Different motherboards also have different levels of PCIe 5.0 support, making it difficult for users to decide if it’s worth the upgrade. Furthermore, Newer PCIe Gen 5 SSDs on the market demand heatsinks and even active cooling solutions like the Corsair MP700 Pro SSD. While heatsinks are entirely optional, it’s important to take note of the hardware maintenance that goes into these processes.

PCIe 6.0 release date

Considering that even the highest-end motherboards on the market today aren’t 100% PCIe 5.0, general consensus is that PCIe 6.0 will be released by the time PCIe 5.0 becomes the norm, creating a situation where consumers will continuously fall behind the latest technological specification on the market.

While there is still no set date for the release of PCIe 6.0, it’s safe to expect its release sometime between 2025 or 2026. The implication is that AM5 motherboards will likely not be impacted by PCIe 6.0, save for their very last chipset. 

Challenges in PCIe 5.0 design

Signal attenuation at higher frequencies

Signal attenuation caused by the channel insertion loss (IL) is the biggest challenge when it comes to PCIe 5.0 technology system design. To understand this, let’s first look at the original PAM-4 method, which is typically used for data transmission standards faster than 30 GT/s. 

While this method helps to reduce the signal's Nyquist frequency to one-quarter of the data rate, it comes at the cost of 9.5 dB signal-to-noise ratio (SNR). The PCIe 5.0 architecture, however, does things differently. Instead of PAM-4, PCIe 5.0 continues to use the non-return-to-zero (NRZ) signaling scheme, where the Nyquist frequency of the signal is one-half of the data rate. 

How PCIe 5.0 offsets error risks

The PCIe 5.0 specification has some guidelines for dealing with these challenges. It sets a limit on how much the signal can weaken during transmission (36 dB for 32 GT/s), and a bit error rate (BER) of less than 10-12. To tackle the problem of signal weakening or attenuation, PCIe 5.0 defines the reference receiver to ensure that the continuous-time linear equalizer (CTLE) model includes an ADC (adjustable DC gain) of -15 dB. This is opposed to the reference receiver for 16 GT/s which is only set to -12 dB. 

It’s also important to note that error frequency grows higher as data rate reaches 32GT/s, especially since DFE circuits play a critical role in the receiver’s overall equalization. Precoding within the PCIe 5.0 architecture helps to offset this risk. Enabling precoding at the transmitter side and decoding at the receiver side will significantly reduce the risk of burst errors and guarantee the robustness of PCIe 5.0 specification 32 GT/s Link.

PCIe 5.0 Technology Channel Insertion Loss Budget

The insertion loss budget for PCIe 4.0 architecture and PCIe 5.0 architecture are 16 GT/s and 32 GT/s respectively. After deducting 9 dB for the CPU package, the remaining budget is split into 9.5 dB for the AIC, 1.5 dB for CEM connectors, and 16dB for the system baseboard.

Insertion loss budget aside, engineers also need to take the following factors into account when designing PCIe 5.0:

1. Insertion loss of PCB trace grows higher as temperature rises
2. Further fluctuations in insertion loss can arise as a result of inconsistencies in PCB manufacturing, particularly when discrepancies in line widths are produced
3. There’s a need to leave additional insertion loss margin for the receiver as reflections, crosstalk, and power supply noise can all contribute to further degradation in SNR.

As a rule of thumb, hardware engineers and system designers leave approximately 10-20% of the overall channel insertion loss budget, which is approximately 4-7 dB for a 36-dB budget. More system topologies are expected to arise as AI and ML demand increases. It’s therefore important for engineers to familiarize themselves with PCB materials and PCIe 5.0 retimers to ensure a smooth upgrade to PCIe 5.0 architecture.

Get ahead of the PCIe curve with GRL

When it comes to AI and ML, engineers at GRL know the industry like the back of our hands. Experience the difference at our state-of-the-art testing facilities and extensive libraries are equipped to handle even the most intricate use cases. Get in touch with GRL today to guarantee your products meet the rigorous demands of the ever-changing industry.