H200 buyers care about memory. Here is why that helps you: H200 vs H100, a dated October 2026 price band with sources, who rents H200s and how to prepare your nodes.
H200 buyers care about memory. Here is why that helps you. If you hold H200s, the question behind h200 rental demand is who needs 141 GB of HBM3e per GPU, how that demand is changing, and what the going price looks like. This guide covers why memory decides inference economics, how the H200 compares with the H100 for buyers, a dated October 2026 price band with every source and its definition, who rents H200s, how to prepare your nodes and how to list. One disclosure: packet.ai lists supplier capacity, so we have an interest in this topic. The prices below are market figures from independent trackers, not what any supplier earns, and this guide sets out no packet.ai payout or fee terms.
Key takeaways
Serving a large language model has two phases with different limits. Prefill, which processes the prompt, is compute-bound. Decode, which generates tokens one at a time, is memory-bound: each new token needs the model's weights and the KV cache of everything generated so far. That is why GPU memory, not peak compute, often sets how many users one GPU can serve, and it is the thread running through HBM3e vs HBM2e, what KV cache is and prefill vs decode.
H200 memory is what changes. NVIDIA's datasheet lists 141 GB of HBM3e at 4.8 TB/s, which it describes as nearly double the capacity of the H100 with 1.4x the bandwidth. The compute silicon is the same Hopper design, so this is a Hopper GPU refresh, and the difference is the memory. A simple weights-only calculation shows what that means for a 70-billion-parameter model.
Arithmetic only: weights at 1 byte per parameter for FP8 and 2 bytes for FP16. It ignores activations and runtime overhead, the headroom is what is left for KV cache, and fitting in the weight budget does not mean a model will run at your context length, batch size or concurrency. It is not a benchmark.
Headroom is the point. A 70B model at FP8 leaves about 10 GB of KV cache on an H100 and about 71 GB on an H200, so the H200 can hold far more concurrent requests or longer contexts on one GPU. That is also how to read NVIDIA's headline numbers. Its datasheet claims up to 1.9x faster Llama 2 70B inference and 1.6x on GPT-3 175B, and the footnote for the 70B figure compares an H100 at batch size 8 with an H200 at batch size 32. The extra memory is what allows the larger batch, so the gain is real, but it is a memory story, not a clock-speed story.
Source: NVIDIA H100 and H200 product pages and datasheet, retrieved October 28, 2026.
For a buyer, the choice comes down to what limits the workload. If it is memory capacity or bandwidth, which describes long-context serving, large batches and 70B-class models, the H200 earns its premium. If it is raw compute or the model fits comfortably in 80 GB, the H100 is often enough, and the B200 vs H200 ROI framework covers when Blackwell is worth more. The NVL version matters for operators who cannot host SXM boards: it is a standard PCIe card that links in groups of two or four with NVLink bridges, which makes it a fit for commodity servers, and it is why h200 nvl is a term buyers search for on its own. The 141 GB figure is also why buyers search for the h200 141gb part by name.
The h200 cloud market has no single price: buyers who rent h200 capacity see published list prices, prices on completed deals, or a standardized rate. The same week, different trackers give different levels, so the table shows each with its type and definition. Snapshot dates run from October 16 to October 28, 2026, so the figures are not from a single day; retrieved October 28, 2026.
Read together, they say three things. Published H200 rental prices vary by provider, configuration and commitment, and advertised-rate trackers and transaction-based indices give different readings, so none of them is a universal market price: here they run from $3.47 (a standardized index) to $6.42 (completed deals). The extremes are wide: one tracker lists $2.09 at the bottom and $13.78 at the top, the lowest listings can come with conditions such as a region, a configuration, a commitment or limited availability, and hyperscaler list prices sit well above neoclouds. And the direction is up. Thunder Compute's tracker recorded recent on-demand rises of 26% at Crusoe (October 16), 15% at RunPod (October 9) and 20% at Nebius (September 29). A rising price does not by itself prove rising demand, because supply, provider mix and availability also move it. Several of these trackers are run by firms that sell or trade compute (Ornn, for instance, also sells reserved capacity), so treat each as one vendor's view. For buyers weighing the alternative, Thunder Compute's tracker puts the cost to buy h200 hardware at about $32,000 to $47,500 per GPU and around $293,000 for an 8-GPU HGX system.
Our view
These are what customers pay or are quoted, not what a supplier earns. A supplier's realized revenue depends on the platform's terms, on how many hours are actually billed, and on costs, which is the point of GPU utilization explained and what an idle GPU costs. Multiplying a list price by 730 hours overstates any real outcome. For planning, compare like-for-like configurations and use conservative realized-rate assumptions.
We do not have customer data on who rents H200s, so this section works from what the hardware does best. Memory-rich Hopper fits six kinds of work. Inference providers serving 70B-class open models want the batch headroom. Enterprises with long-context retrieval and agent workloads need room for large KV caches. Reasoning-model serving generates long outputs that grow the cache. Teams that want one GPU per 70B model at FP8 value the simplicity. Multi-model serving can use MIG to split a card into up to seven instances. And memory-bound HPC is a use NVIDIA itself highlights.
The direction of demand is the more interesting question. A 2026 academic analysis of inference scaling argues that reasoning workloads, with very long output sequences, shift serving into a capacity-bound regime in which the KV cache exhausts per-GPU HBM before compute is fully used. SemiAnalysis's InferenceX agentic benchmarks make a related point: long conversations stress HBM capacity and push operators to offload KV tensors to CPU memory and SSD. ClearML, citing Bloomberg Intelligence, puts inference at roughly two-thirds of AI compute. These reports describe workloads, not H200 demand, but they point toward memory pressure, which is where the H200 is strongest.
There are counterforces. KV-cache offloading in vLLM and NVIDIA Dynamo, prefill and decode disaggregation, and compression all reduce how much HBM a request needs, and Blackwell parts carry more memory per GPU (NVIDIA lists 1,440 GB across the eight B200s in a DGX B200). TrendForce notes that offloading itself creates demand for memory and storage, which suggests the pressure moves instead of disappearing.
Our view
This is a scenario, not an established outcome: we expect memory-rich Hopper to stay in demand for serving 70B-class models through the next year, because most deployments do not need Blackwell and long-context agents keep raising cache sizes. The risks are a faster shift to Blackwell, software that shrinks cache needs, and a fall in prices if supply catches up. The price trend above is a snapshot, not a forecast. Indicators that would change our view are realized rental rates rather than list prices, new-generation capacity coming online, utilization on H200 fleets, and the cost of serving the same workload on other hardware. The GPU supply outlook for 2027 covers the supply side.
Power and cooling. NVIDIA's DGX H200 datasheet lists up to 10.2 kW for an 8-GPU system, or up to 14.3 kW for the custom thermal version, on 200 to 240 V input with six 3.3 kW power supplies in a 4+2 arrangement. That is the DGX H200 system rating, not a per-GPU figure: an OEM HGX H200 server or an H200 NVL system has its own rating, so size power, cooling and networking from your actual server's spec sheet. For scale, one colocation operator's guide puts standard racks at 5 to 15 kW, so a single node can use most of a rack's allocation, which is where what colo owners earn as landlords versus operators matters. Form factor. SXM boards need an HGX or DGX-class chassis, while NVL cards are PCIe, dual-slot and air-cooled at up to 600 W, in systems of up to eight GPUs.
Networking. A DGX H200 ships with ten ConnectX-7 adapters at up to 400 Gb/s, but single-node inference does not need a cluster fabric, so match the network to what you plan to sell. Monitoring. Track allocated, busy and billable hours from day one, and benchmark with your own batch sizes and context lengths before you list, because that is what buyers will test. Licensing and warranty. The H200 is a data center GPU, not a GeForce part, but confirm the software terms and check that your warranty covers multi-tenant, around-the-clock use. Isolation. Keep tenant workloads separate, and keep your own workloads off any hardware you rent out.
Whether you want to rent out h200 capacity directly or list h200 nodes on a platform, there are four routes. List on a marketplace or supplier program, where the platform finds customers and handles billing. Sell reserved or long-term capacity directly to an inference provider. Resell through a white-label platform under your own brand. Or colocate and sell to whoever fits. The five ways to turn idle GPUs into revenue compares them, why neoclouds are adding a second demand channel covers why operators use more than one, and if you manage GPUs for clients, the guide for MSPs and integrators applies. Sibling guides in this series cover other cards: Rent out RTX PRO 6000 and Rent out A100 and L40S. In practice this is h200 gpu hosting: you keep the hardware running and the platform or buyer supplies the workload, so the h200 hosting basics above (power, cooling, monitoring and isolation) apply.
If you are ready to list, the H200 supplier page describes listing H200s, and you can apply with your H200s. For how the pieces fit together, the GPU provider glossary defines terms such as SXM, NVLink, HBM and MIG.
Retrieved October 28, 2026. Prices and indices change daily. Several trackers are run by GPU clouds, and index methodologies differ.
Last reviewed: October 28, 2026. This is general information, not financial or engineering advice. It contains no earnings figures and no guarantee of income, and the prices are market figures that change daily. For packet.ai's supplier information, see the supplier page.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start building →