40 GPU hosting terms in plain English, from hardware and networking to capacity, payouts and platform rules. The six most searched come first, with fuller answers.
This GPU glossary has GPU terms explained in plain English: 40 pieces of GPU terminology for anyone new to running or renting out GPUs. Think of it as a GPU hosting glossary. It answers the questions behind the most-searched terms first: what is neocloud, what is high bandwidth memory, what is NVLink, what is InfiniBand, what is SXM and what is GPUaaS. The rest follow in five groups: hardware, cloud, business, platform and money. Definitions are general industry meanings, so where a term depends on a platform's own rules, the entry says so.
Jump to the most searched terms
Neocloud | High bandwidth memory | NVLink | SXM | InfiniBand | GPUaaS
Or jump to a section
These six get a fuller answer, because they are the terms people look up most. Everything else in the glossary is one or two sentences.
A neocloud is a cloud provider built specifically for GPU compute and AI workloads, rather than a general-purpose cloud that added GPUs later. So what are neoclouds? The name is short for "new cloud," which is the neocloud meaning in one line. Commonly cited neocloud companies include CoreWeave, Crusoe, Lambda and Nebius, and neocloud providers like these sell depth in GPU compute where a hyperscaler such as AWS, Azure or Google Cloud sells breadth, which is the core of the neocloud vs hyperscaler difference, although individual neoclouds differ in architecture and services. A neocloud is still a public cloud, so using one next to a hyperscaler is multi-cloud, not hybrid. For a regional example, see neocloud providers in India.
HBM is stacked DRAM placed on the same package as the GPU and connected over a very wide bus, which gives far more memory bandwidth than ordinary graphics or system memory. The stacks are built by joining DRAM dies vertically with through-silicon vias. Bandwidth is how fast data moves and capacity is how much fits, and GPU spec sheets list them separately. What is HBM3e? It is the faster extension of HBM3: NVIDIA's H200 pairs 141 GB of HBM3e with 4.8 TB/s of bandwidth, against 80 GB of HBM3 at 3.35 TB/s on the H100. Because model weights and the KV cache have to fit in this memory, HBM capacity often decides how many GPUs a model needs. So what is HBM memory in plain terms? It is the fast on-package memory a GPU depends on.
NVLink is NVIDIA's high-speed link for moving data directly between GPUs, instead of routing it through the CPU and PCIe. NVIDIA lists 900 GB/s of GPU-to-GPU bandwidth for fourth-generation NVLink on the H100 SXM, against 128 GB/s for PCIe Gen5, and bandwidth and topology vary by GPU generation and server configuration. An NVLink bridge connects two PCIe-form-factor cards, as on the H100 NVL pair at 600 GB/s. What does NVLink do? NVLink explained in one line: it lets several GPUs share data fast enough to act like one larger pool, which matters most for large models split across GPUs. The packet.ai NVLink vs PCIe guide covers when each fits.
SXM is NVIDIA's socketed module form factor: the GPU mounts directly onto an HGX baseboard instead of plugging into a PCIe slot, which is what lets all the GPUs in a node connect over NVLink. SXM parts are also rated for more power; NVIDIA lists up to 700 W for the H200 in SXM against up to 600 W for the PCIe card. In short, SXM describes the GPU module, and the HGX baseboard around it is the wider server platform.
InfiniBand is an open-standard, high-bandwidth, low-latency network widely used to link GPU servers into one cluster, and it supports RDMA, which lets one machine read or write another machine's memory without involving either CPU. Bandwidth depends on the generation and hardware: NVIDIA's Quantum-2 (NDR) InfiniBand switches run 400 Gb/s per port, and NVIDIA pairs that networking with the H100 for scaling across nodes. It matters most for multi-node training, while single-node and inference-only providers often do not need it.
GPUaaS, or GPU as a service, means renting GPU compute from a provider by the hour or month instead of buying and running the hardware. That is the GPUaaS meaning, which is also the GPU as a service meaning, and GPUaaS providers range from neoclouds and hyperscalers to marketplaces. Customers typically pick a GPU model, a deployment type such as container, VM or bare metal, and a billing model.
The physical side of a GPU server. HBM, NVLink and SXM are covered above.
PCIe. PCIe (PCI Express) is the standard expansion bus that connects GPUs, network cards, storage and other devices to the CPU, and NVIDIA lists 128 GB/s for PCIe Gen5, well below NVLink.
NVSwitch. What is NVSwitch? It is a switch chip that joins NVLink connections into an all-to-all fabric, so in supported systems such as eight-GPU HGX nodes every GPU can talk to every other at full NVLink speed, and not every multi-GPU server has one.
VRAM. VRAM is the memory on the GPU itself, which has to hold a model's weights and working data, and on data center GPUs it is usually HBM, while other GPUs use types such as GDDR.
TDP. What is TDP in GPU specs? Thermal design power is a design rating for the sustained power a chip is built to draw and shed as heat, which sets your power and cooling budget, but it is not a measurement of actual draw under every workload, and NVIDIA lists up to 700 W for the H200 in SXM and up to 600 W for the PCIe card.
MIG (Multi-Instance GPU). What is MIG GPU partitioning? Multi-Instance GPU is NVIDIA's feature for partitioning a supported GPU into isolated instances, each with its own memory and compute, and the maximum number depends on the GPU model: NVIDIA's supported-GPU list shows up to seven on the A100 and H100 and four on the RTX PRO 6000 Blackwell.
How GPU capacity is packaged and shared. Neocloud, GPUaaS and InfiniBand are covered above.
Bare metal. Bare metal means renting a whole physical server with no hypervisor layer, so the customer gets direct hardware access with minimal virtualization overhead, and a bare metal GPU server is the usual choice for performance-sensitive training.
Virtual machine (VM). A virtual machine is a software-defined computer running on a hypervisor with its own operating system, so one physical server can host several isolated customers.
Container. A container packages an application with its dependencies and shares the host's operating system kernel, so it starts faster and uses fewer resources than a VM but isolates less strongly.
Multi-tenant. Multi-tenant means several customers share the same physical infrastructure, with software keeping their workloads and data separate.
GPU passthrough. GPU passthrough meaning: a hypervisor assigns a physical GPU to a virtual machine so it has direct control and performs close to bare metal, which is different from exposing a GPU to a container, and that GPU generally cannot be shared with other VMs while it is assigned.
How capacity is sold and measured.
Utilization. Utilization is the share of a GPU's time or capacity actually in use, and in a GPU business it can mean allocated, busy or billable hours, which are not the same thing (see GPU utilization explained).
Oversubscription. What is GPU oversubscription? It means selling or committing more capacity than physically exists on the bet that customers will not all use theirs at once, which is different from safely sharing a GPU through partitioning or scheduling, where the total sold still fits the hardware.
Reserved capacity. Reserved capacity is GPU capacity committed to a customer for a fixed term, usually at a lower rate and billed whether or not it is used, and whether the commitment guarantees availability, priority or only a rate depends on the provider's terms.
On-demand. On-demand capacity is pay-as-you-go GPU time with no commitment, which the customer can start and stop at any time.
Spot (interruptible) capacity. Spot instance meaning: spare capacity sold at a discount on the condition that the provider can take it back, often with little notice, and AWS, for example, gives a two-minute warning before it reclaims a Spot Instance.
Occupancy. In a rental business, occupancy is the share of a provider's available GPU-hours assigned to paying customers, a commercial capacity metric that is not the same as compute utilization, while in CUDA it means something different, the ratio of active warps to the maximum an SM can hold.
Fixed-cost break-even rate. The fixed-cost break-even rate is the price per billable GPU-hour that covers your modeled fixed monthly cost, and it falls as billable utilization rises (see the worked example in GPU utilization explained).
Wholesale capacity. Wholesale capacity is GPU capacity sold in bulk to a reseller or large buyer instead of to many small customers, usually at a lower rate per hour.
What a host runs into when listing capacity on a platform. These terms are used differently across platforms, so each entry notes where the platform sets the rules.
Verification. Verification is the set of checks a platform runs on a host's hardware and network before a machine can be listed, which is separate from ongoing reliability monitoring, and the specifics are each platform's own; on Vast.ai, for example, it is automated and includes minimum network speeds and a reliability threshold.
Tier. A tier is a platform's grouping of capacity by reliability, facility standard and price, such as community versus data center capacity.
SLA (service level agreement). A service level agreement is the written commitment on service quality, usually an uptime percentage, along with what the provider owes if it misses it, such as service credits, and the percentages and remedies depend on the specific agreement.
Uptime. Uptime is the share of time a service is available, and 99.9% allows about 44 minutes of downtime in an average month while 99% allows about 7 hours, though any commitment depends on the specific service agreement.
Host. A host is the person or company that owns and operates the GPU hardware offered on a platform.
Listing. A listing is a machine or block of capacity a host has put on a platform for rent, with its specs, price and availability.
Reliability score. A reliability score is a platform-calculated metric of how consistently a host's machine stays available, and each platform sets its own calculation; Vast.ai, for example, requires a score above 90% before a machine verifies.
Orchestration. Orchestration is the software that schedules and manages workloads across machines, for example Kubernetes or Slurm.
How money moves between customers, platforms and hosts. These are general definitions, and each platform sets its own rules.
Take rate. Take rate is the share of each transaction a marketplace or platform keeps, and the percentage and fee structure are set by the platform, so read the host agreement before you model income.
Payout. A payout is the transfer of a host's earnings from the platform to the host, and the schedule, method and fees vary by platform.
Minimum payout. A minimum payout is the smallest balance a host must accumulate before the platform will pay it out.
Chargeback. A chargeback is a reversal of a card payment after the cardholder disputes it with their bank, which can pull the money back from the platform and, depending on the terms, from the host.
Depreciation. Depreciation spreads a hardware purchase price across its useful life as an accounting figure, which is not the same as the hardware's market value, and the life you pick can change the monthly cost by about 60% across the lives modeled in what an idle GPU costs.
Committed power. Committed power is the kilowatts of power capacity reserved from a colocation provider and billed under the contract whether used or not, which is not necessarily the electricity actually consumed. What is GPU colocation? It is placing your own servers in a provider's data center, where rates are commonly quoted per kW per month.
Egress fee. An egress fee is a charge for data leaving a provider's network, to the internet or to another provider, where applicable, and some GPU clouds charge nothing, so check before moving large datasets.
Net terms. Net terms are the payment deadline on an invoice, such as net 30, which means payment is due 30 days after the invoice date.
If you own GPUs and are deciding whether to host them, the glossary maps onto four steps:
Retrieved October 9, 2026. Hardware figures come from NVIDIA's product pages and documentation; the rest are general industry definitions, and platform-specific examples reflect those platforms' documentation at the time of writing.
Last reviewed: October 9, 2026. These are general definitions for new hosts, not legal, financial or engineering advice, and platform rules and hardware specifications change. Check the current documentation for any platform or product before you rely on a figure here. For packet.ai's current provider requirements and terms, see the provider page.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →