Explain The CMP 100-210 Bandwidth Issue?

[u/StellarWox]

I'm thinking of grabbing some of these, but people are saying they have very slow bandwidth due to being mining cards and this causes problems.

I don't fully understand what problems are that are caused from this, but my biggest question is: if I am able to load the entire model in VRAM with no offloading, wouldn't that solve the issue?

And, if there are multiple GPUs that load a bigger model by splitting the layers, wouldn't the bandwidth issue not be too much of a problem as it would only need to move the already computed forward passes to the next layer loaded in GPU?

Which wouldn't be as big as moving the entire model weights so I should still be able to load something like deepseek flash, provided I have enough cards for the VRAM needed, or is this wrong?

If someone could please explain this to me before I drop a grand buying a bunch of these cards, I would greatly appreciate it!

[u/DustNearby2848]

They are PCIE 2.0 2x or something and can be modded to 16x. PCIE 2.0 2x = 2GBs, 16x = 16 GBs. PCIE 5.0 16x = 128 GBs. I think those are bidirectional speeds though, so you half them. 

[u/StellarWox → u/DustNearby2848]

Can you explain more about this? I would love to know how to mod them

[u/DustNearby2848 → u/StellarWox]

You need to know how to do reflow soldering, it would take months of practice. But essentially some capacitors are missing, so the other lanes do not activate.

[u/a_beautiful_rhind → u/DustNearby2848]

Its not that bad. Cheat and buy a hot tweezer. I could probably do it by hand but the tweezer lets you properly position caps. You could do worse things under a microscope and people do all the time.

[u/An_Original_ID → u/StellarWox]

Only certain models can be modified. Without knowing the hardware ID of the card, you will likely get one that can't be modified.

The best modification, assuming it is a model that can be modified, is the v100 BIOS.

Stock cmp 100-210 is about the speed of a 3060. V100 BIOS cards are about 30% slower than a 3090. I'm guessing it's because the v100 BIOS enables the tensor cores.

Also, PCIe speed isn't a big deal if the weights are fully in VRAM. Offloading layers to system RAM can with but very big speed hit.

Don't let people complain about the "slow" load time with pcie x1. It takes like a minute to load the card, who cares.

TLDR Stock CMP 100-210 is okay, 16GB is nice, 3060 speeds. V100 CMP is great, much faster, a little harder to find. Don't expect to find a card you can physically modifying.

[u/darksteelsteed → u/An_Original_ID]

If the caps that need to be added are not under the bga package you could still manually solder them in under a microscope if you are brave enough. Your iron must be temperature controlled for this or you will ruin the pcb traces

[u/OvertaxedOne]

If the model is entirely in VRAM on one card (or you use pipeline parallel between cards) bandwidth doesn't matter much. You need serious bandwidth for tensor parallel, don't even consider that!

[u/Enough_Air_192]

slow bandwidth = no split mode tensor for you

[u/FearFactory2904]

I could be off but will try to explain with my best understanding:

Loading the model into the vram of the card when you load your model will take longer, but you wouldnt be doing that a lot unless your swapping back and forth between a lot of models.

Assuming your model fits on the one card then you are mostly bound by the compute on that card since you are not sending a lot back and forth over pcie that point.

Assuming you want to move to two or more cards, theres two ways to do it:

One method splits the layers to partially exist on one card and partially on the other. Once the first card does its work it sends the signal to the next card. At any given time you basically have the performance of one of whichever card is active so adding additional cards does not scale performance. This is what most people with restricted pcie lanes do.

If you have multiple cards and plenty of bandwidth between the cards you can run your workload in parallel where both cards are being utilized simultaneously for scaling the performance. This is what you are missing out on with most if the lane limited mining cards.

[u/PermanentLiminality]

This card is severely limited by the firmware. The interface is PCIe 1.0 x1. It will take a while to load a model. Compute is limited with f16 super slow.

Don't buy them.

Get the V100 instead. Basically the same card with x16 interface and all compute intact.

[u/sevendaysworth → u/PermanentLiminality]

I got two 8g with the gen 2 x16 mod and ran cmpunlocker to unlock both to 64gb each. It’s a hacky setup but mine have been flawless. Qwen 3.8 27b screams on each one with W4A16 and speculative decode

[u/Sporebattyl → u/sevendaysworth]

The Cmp 170hx? That is a different beast than what OP is talking about