TLDR:

The deeper you look at accelerated computing, the more a simple truth emerges -- silicon alone is not the moat. The real lock on the market is how those chips talk to each other.

NVLink, NVIDIA's proprietary high-bandwidth interconnect, is the glue that turns otherwise discrete GPUs into one giant, coherent accelerator.

In the process, NVIDIA gains control over the most lucrative workloads in Artificial Intelligence (AI), High-performance computing (HPC), and visualization.

The monopoly is not in the "brain". Rather it is in the "spine"

I try to distill a complex topic to the best that I can.

We start by exploring the bandwidth bottlenecks that plague PCIe-based systems and how NVLink's point-to-point links and non-blocking NV Switch fabric rewrite the rules for multi-GPU scaling.¹

From there we dive into NVLink Chip-to-Chip (C2C), the Grace Hopper coherence layer that lets GPUs sip from CPU memory while burning roughly 25× less energy per bit than a PCIe hop.²

Then, we track the real-world consequences -- faster LLM training runs, shared VRAM pools for cinematic renders, and scientific simulations that no longer crawl.³

Table of Contents

Key Glossary: Interconnects and Related Jargon

If you buy the argument that interconnect speed defines what problems you can solve, then NVLink stops being a specs table entry, and becomes the real monopoly.

Chapter 1: What Is NVLink?

NVLink is NVIDIA's proprietary, high-speed, point-to-point interconnect designed to link GPUs (and increasingly CPUs) into a unified complex that behaves as a single accelerator.

Instead of routing traffic through the CPU's PCI Express (PCIe) root complex, NVLink creates direct peer-to-peer paths between devices.

Each "link" is a bundle of high-speed differential lanes that aggregate into a mezzanine-style bridge between neighboring GPUs.

By the time you reach Blackwell-generation silicon, an NVLink endpoint can sustain up to 900 GB/s of bidirectional bandwidth per GPU.

The jump to 1.8 TB/s keeps a ≈14× headroom over PCIe Gen5 x16 (~128 GB/s), which is why trillion-parameter LLM training still requires NVLink-class collectives rather than commodity buses.

Switchless mesh, switch fabric, and scale

NVLink started out connecting GPUs together in simple mesh or ring layouts -- each GPU had direct connections to a few others, so they could send data straight across without waiting their turn on the PCIe bus.

Later, NVIDIA added a hardware switch (NV Switch) that let a lot more GPUs talk to each other at once, instead of just in small groups. For example, one NV Switch can link up to 16 GPUs in a box, and all of them can share data at full NVLink speed without getting in each other's way.

This architectural choice solves three headaches endemic to PCIe-based clusters:

• Bandwidth: PCIe 5.0 x16 gives about 128 GB/s both ways; NVLink gives much more, so GPUs don't have to wait for data.⁴
• Latency: NVLink skips the CPU and is much faster inside the server -- delays drop to just a few microseconds.
• Coherence: NVLink keeps memory in sync between GPUs, so code can access GPU or CPU memory directly instead of copying data around.⁵

Beyond GPU-to-GPU: NVLink C2C

NVLink C2C (Chip-to-Chip) lets a Grace CPU and Hopper GPU connect directly, sharing memory at much higher speed and lower energy use than PCIe.

This means a GPU can use the CPU's memory when it runs out of local VRAM, without a big slowdown. So, bigger models can run without running into memory or power problems.

TLDR

NVLink lets horizontal GPUs "vertically scale" in VRAM and throughput. This leads to better training costs, inference cost. That said, a linked GPU should not be confused to perform the equivalent of unified GPU

ChatImage

PM has entered the chat

ML engineer has entered the chat

ok, I don't really get what NVLink actually does. Isn't it just another connector?

nope

nvlink is like a private, super-fast network just for GPUs. With PCIe, GPUs have to go through the CPU to talk to each other.

with nvlink, they get direct high-speed links straight to their neighbors.

So what's the deal with this NV Switch? a bigger router or something?

Pretty much, the NV Switch brings all those NVLink connections together.

That way, up to sixteen GPUs can send like 14 terabytes per second between each other without any traffic jams.

And why do we keep hearing about Grace Hopper everywhere?

That's NVLink C2C

chip-to-chip.

It lets the Grace CPU and Hopper GPU share the same memory fabric. So, the GPU can access system memory with really low latency and a lot more efficiency than PCIe.

everyone online says CUDA is the real secret sauce. legit?

CUDA is super important

but honestly, the big challenge is moving data quickly and keeping everything in sync. That's the hard part.

NVLink is what makes accelerators scale smoothly, while others get stuck when a bunch of GPUs need to communicate.

Chapter 2: Performance, Latency, and Power Efficiency

The economic penalty for skipping NVLink shows up long before a purchase order clears. Data center operators who wire their clusters with commodity PCIe interconnects discover two brutal truths:

• scaling stalls well below linear
• every idle microsecond bleeds watts.

2.1 The Energy Penalty

NVLink is built for fast, efficient data transfer; PCIe uses more energy per bit to move the same data.⁶

When transfers are slow, GPUs sit idle but still use power, so you pay extra both for the inefficient transfer and for hardware doing nothing. On big clusters, energy waste per useful compute skyrockets.

2.2 The TCO Calculation

At first, picking PCIe instead of NVLink seems like a smart cost-saving move. PCIe hardware is much cheaper up front, and the specs sound fast enough. But as soon as you start training a big model, the real costs show up.

Here's what happens:

• Slower Training: PCIe can't move data fast enough between GPUs, so jobs take way longer to finish. This means you wait extra days -- or even weeks -- to reach your goals, and miss out on new revenue or research because everything slips.
• Wasted Hardware: While GPUs are waiting to send or receive data, they burn power but don't actually compute. So you pay top dollar for GPUs that sit idle for a big chunk of time.
• Huge Power Bills: Longer training + idling GPUs = much higher electricity bills. The more you try to save on hardware, the more money you lose on operations.

Add it all up, and the "discount" vanishes fast. Over just a few projects, paying less up front can actually cost you more than the pricey NVLink setup.

The hidden "tax" is wasted time, wasted energy, wasted hardware.

Everyone says NVIDIA has a monopoly because of CUDA, but that's not really the trap. The real moat is NVLink. Their fabric unlocks performance you just can't get with PCIe, so you end up paying NVIDIA's prices whether you like it or not.

That's what makes their market dominance so deceptive -- it hides in your power bills and your lost time, not just in hardware invoices.

ChatImage

We used PCIe for storage and network for years -- why isn't it enough for AI training too?

Think of AI training as a team in non-stop direct messaging

PCIe runs every message through a middleman (the CPU)

NVLink is like letting GPUs DM each other directly -- much faster and no detours, so everyone keeps up.

But what about when model size blows past VRAM limits?

we're already there. New AI models are too large for a single GPU

That's where NVLink C2C comes in. It lets the CPU and GPU share the same memory pool, so you can access system RAM at high speed without new code.

Its basically plug and play for big models.

Everyone says power costs pile up -- how much does the link type really matter?

PCIe makes GPUs wait, but they still burn power doing nothing.

with NVLink, data moves so fast you finish jobs sooner and save big on energy bills.

So going budget with PCIe-only GPUs isn't actually saving us?

yeah

Any hardware savings get eaten by waste, idle GPUs, lost efficiency, and higher OPEX.

In the end, the budget GPUs just costs more in the long run.

Chapter 3: NVLink Fusion vs. CXL

Having won the present on raw performance, Nvidia can harden its moat by licensing the fabric selectively and turning NVLink racks into the default blueprint for hyperscale AI factories.⁷

3.1 NVLink Fusion

NVLink Fusion is NVIDIA's new initiative that lets selected partners connect their own custom chips -- like CPUs, AI accelerators, or other devices -- directly into NVIDIA's ultra-high-speed GPU fabric using special NVLink chiplets and interface blocks.

NVLink Fusion is like an adapter that allow third-party silicon to join NVIDIA's super-fast hardware club, but on NVIDIA's terms.

Normally, different chips in a server talk to each other using standard connections like PCIe, which is much slower and struggles to keep up with AI workloads.

NVLink Fusion gives partners access to pieces of the NVLink technology -- such as chiplets (tiny, modular hardware blocks), SerDes (high-speed data links), and the rack-wide switch that ties everything together.

But crucially, NVIDIA does not hand over the full control of the underlying protocol, keeping the essential DNA of NVLink proprietary.⁸

3.2 The Challenger: CXL

CXL (Compute Express Link) is an industry-standard, high-speed interconnect protocol designed for coherently linking CPUs, GPUs, FPGAs, accelerators, and memory devices within servers and data centers.

It rides atop the PCIe physical layer (currently Gen5 and Gen6), but adds advanced memory coherency, direct load/store access, and fine-grained resource sharing between heterogeneous devices.

Unlike PCIe, which is optimized for general I/O and discrete transfers, CXL excels at pooling memory and connecting devices in a more unified, flexible memory space -- allowing system RAM, persistent memory, and accelerator memory to be shared or accessed transparently by multiple components.

This lets disaggregated servers and accelerators dynamically compose resources for workloads, particularly in cloud and inference environments, with much more flexibility than legacy architectures.

ChatImage

What's this CXL thing I keep hearing about?

Is it supposed to challenge NVLink?

CXL (Compute Express Link) is an open standard that rides on top of PCIe 5.0.

It lets CPUs, GPUs, accelerators, and even memory expanders pool memory and talk to each other almost like they're part of one big machine.

Sounds flexible -- so I could toss lots of different hardware into a rack and just have it all share memory?

yeaah CXL is great for "scale-out" setups

server farms, cloud clusters, and data centers running lots of smaller inference jobs.

You can add RAM, CPUs, or extra accelerators from various vendors, and CXL lets them all see and use memory together, efficiently.

Okay, but if it's so good, wouldnt it threaten Nvidia's nvlink model?

well, for the really big "scale-up" clusters -- the kind that train trillion-parameter models -- CXL still can't keep up

its still bottlenecked by PCIe bandwidth (about 128 GB/s per x16 slot)

its latency is also much higher -- usually tens of microseconds, compared to NVLink's single-digit microseconds.

So CXL is awesome for making CPUs and standard memory go further, but it's just not fast enough for top-end GPU training?

yeaah

CXL transforms how data centers pool resources for inference and general compute

but when it comes to the ultra-demanding, bandwidth-hungry world of flagship AI training, only NVLink delivers.

That high-speed, low-latency fabric is really what lets NVIDIA control the most valuable segment of the market.

3.3 Nvidia not dethroned anytime soon

NVLink's future is a two-track play:

• invite partners into the rack while keeping the only highway that makes the rack economic.¹²
• Fusion taps custom silicon, CXL absorbs the commodity edge
• NVIDIA keeps the 14× bandwidth differential that dictates performance-per-watt in training clusters.

ChatImage

oh but

If NVIDIA is licensing NVLink, doesn't that dilute the monopoly?

Fusion lets partners bolt on custom silicon, but the high-performance fabric still belongs to NVIDIA.

Can CXL replace NVLink for training?

maybe? but not today

CXL 3.0 rides PCIe, so bandwidth and latency land an order of magnitude behind NVLink's SerDes.

Then where does CXL fit?

Memory pooling, inference clusters, and heterogeneous CPU fabrics. It's perfect for scale-out, not the scale-up GPU core.

So the moat survives?

Yeah

The only economically viable path for massive training remains NVLink plus NVSwitch. Fusion just makes sure everyone else plugs into it instead of fighting it.

Conclusion

If you strip away the marketing wars and discussions on flops, CUDA, or even raw chip supply, then you notice the reason NVIDIA dominates AI isn't just better silicon or clever software lock-in -- its NVLink.

As other vendors talk about "openness", NVIDIA has built a network that bakes their advantage right into the server rack.

No competitor -- not CXL, not PCIe, not open accelerators -- can yet offer the seamless, ultra-low-latency, and high-bandwidth connections required for the largest AI models (and the biggest AI customers).

By licensing Fusion and letting others "plug in," NVIDIA appears open, but the only highway that makes hyperscale AI profitable is still theirs.

Even if you swap out the CPU or buy next year’s "open" accelerator, your data will still flow through NVLink’s lanes, routed by NVIDIA’s own switch fabric.

The AI race isn’t just about who makes the fastest chip -- its now about who owns the roads connecting them.