The GPU selection decision for fine-tuning transformer models has shifted materially over the past twelve months. H100 supply has normalized, H200 is broadly available on CoreWeave, Lambda Labs, and AWS, and Blackwell pricing pressure is already pulling enterprise contracts toward better terms. Anyone doing the h100 vs a100 vs h200 fine tuning calculation today operates in a fundamentally different cost-availability landscape than 2023 — and the right answer depends more on model size than on raw compute specs.

This article resolves one specific decision: an ML engineer or technical founder needs to fine-tune a transformer model — LLaMA 3, Mistral, or a GPT-style architecture — and must choose between renting an A100 80GB, H100 SXM5, or H200 SXM5. The analysis covers memory capacity, bandwidth, compute throughput, actual cost per run, and ecosystem readiness. It does not cover inference-only workloads, the Blackwell B200, or multi-node training beyond 1T parameter scale.

---

The Five Dimensions That Actually Determine Fine-Tuning Outcomes

Before the GPU comparison, establish what you're actually measuring. Hourly rate is the wrong primary variable.

Memory capacity determines whether your model fits on a single GPU without quantization compromises. The H200's 141GB versus the H100 and A100's 80GB is the single most consequential spec difference for anyone fine-tuning a 70B model. Every workaround for insufficient VRAM — QLoRA, pipeline parallelism, model sharding — adds engineering overhead and typically degrades final model quality relative to full fine-tuning at larger batch sizes.

Memory bandwidth controls how fast weights and activations move during forward and backward passes. This is the primary throughput bottleneck in fine-tuning, not raw FLOPS. The progression from A100's 2.0 TB/s to H100's 3.35 TB/s to H200's 4.8 TB/s translates directly into tokens processed per second on long-context jobs. On 8K–128K context fine-tuning, the H200's 1.43× bandwidth advantage over H100 is measurable in wall-clock time.

Compute throughput in BF16 and FP8 explains the 3–6× real-world speedup H100 and H200 deliver over A100 on transformer architectures. The A100 delivers ~312 BF16 TFLOPS. The H100 and H200 each reach ~989 BF16 TFLOPS and ~1,979 FP8 TFLOPS through the Transformer Engine — a hardware-software co-design that dynamically selects FP8 precision for eligible operations. The A100 has no FP8 support, which means it cannot access this speedup path regardless of software configuration.

Cost per fine-tuning run — not hourly rate — is what determines your budget exposure. Current on-demand pricing on third-party providers runs approximately $1.10–$1.50/hr for A100, $1.99–$2.49/hr for H100, and $2.50–$3.50/hr for H200. But a fine-tuning job that takes 4 hours on H100 might take 14 hours on A100, inverting the apparent cost advantage. The calculation changes again by model size.

Ecosystem and availability governs reproducibility and scale-out. All three GPUs support CUDA 12.x, Hugging Face Transformers, DeepSpeed, and Axolotl. The practical difference is spot instance availability — H100 now has the deepest secondary market — and per-second billing, which Azure and GCP introduced for H100 in mid-2024 and which materially reduces cost on short iteration runs.

---

Specification Comparison

| Spec | A100 80GB | H100 SXM5 80GB | H200 SXM5 141GB | |---|---|---|---| | Architecture | Ampere (2020) | Hopper (2022) | Hopper Refresh (2024) | | Memory | 80GB HBM2e | 80GB HBM3 | 141GB HBM3e | | Memory Bandwidth | 2.0 TB/s | 3.35 TB/s | 4.8 TB/s | | BF16 TFLOPS | ~312 | ~989 | ~989 | | FP8 TFLOPS | None | ~1,979 | ~1,979 | | Transformer Engine | No | Yes | Yes | | Third-Party Price (2024) | $1.10–1.50/hr | $1.99–2.49/hr | $2.50–3.50/hr | | 2026 Projected Price | $0.50–0.80/hr | $1.00–1.50/hr | $1.80–2.50/hr | | Single-GPU 70B Full FT | No | No | Yes (with LoRA) |

---

A100 80GB: The Budget Workhorse With a Shrinking Window

The A100 remains the cheapest GPU of the three — $1.10–$1.50/hr on Lambda Labs and CoreWeave today, projected to fall to $0.50–$0.80/hr by 2026 as used inventory from replaced enterprise clusters enters the secondary market. For teams fine-tuning sub-13B models, that price point is genuinely attractive.

The architecture ceiling hits hard above that size threshold. Without FP8 support or a Transformer Engine, the A100 delivers ~312 BF16 TFLOPS — roughly one-third the throughput of H100 on the same transformer workload. On a LLaMA 3 7B fine-tuning job with a comparable batch size and sequence length, H100 completes the run in roughly one-third to one-fifth the wall-clock time. When you divide that time into the hourly rate, the A100's cheaper hourly figure rarely survives the math for models above 13B parameters.

For 70B models, the A100's 80GB ceiling becomes a structural problem. Running full fine-tuning on a 70B model requires an 8× A100 setup or aggressive QLoRA quantization. An 8× A100 configuration at $1.40/hr each costs $11.20/hr — more than three H200 instances, each of which handles the same workload on a single GPU. The A100's use case in 2026 is narrow but real: fine-tuning 7B–13B models on a constrained budget, running many parallel small experiments simultaneously, or operating within an existing institutional cluster where the hardware is already amortized.

---

H100 SXM5 80GB: The Current Price-Performance Standard

The H100 is the right GPU for most fine-tuning workloads running in 2026. Supply normalization has made it the most available high-end GPU on the market, third-party pricing has settled at $1.99–$2.49/hr on CoreWeave and Lambda Labs, and every major fine-tuning framework has been benchmarked extensively against it — which means cost estimates are reliable rather than theoretical.

The Transformer Engine with FP8 support is the architectural leap that matters. On LLaMA-style architectures, the H100 consistently delivers 3–6× the fine-tuning throughput of an A100 for transformer workloads. At $2.20/hr completing a job in 4 hours versus $1.30/hr taking 14 hours, the H100 costs $8.80 versus $18.20 — a 52% reduction in total run cost for the same output. That ratio holds roughly across 7B through 34B model sizes.

The 80GB memory ceiling is the H100's binding constraint. Full fine-tuning of a 70B model on a single H100 is not feasible. QLoRA reduces the memory requirement substantially, but imposes constraints on batch size and sequence length that affect final model quality at the margin. Teams that need to fine-tune 70B models repeatedly — running ablations, iterating on data mixtures — will spend meaningfully on multi-GPU H100 coordination overhead that a single H200 eliminates.

One forward-looking risk: H100 reserved contracts signed in 2025 at $1.50–$2.00/hr may look expensive by mid-2026 as Blackwell adoption scales and H100 spot prices fall toward $1.00–$1.50/hr. If your fine-tuning roadmap extends 18 months, negotiate contract flexibility or favor on-demand pricing despite the current premium.

---

H200 SXM5 141GB: The Large-Model Fine-Tuning Leader

The H200's defining characteristic is what it changes architecturally: single-GPU full fine-tuning of 70B models becomes operationally straightforward. LLaMA 3 70B with LoRA at production batch sizes fits within 141GB without quantization tricks. That eliminates multi-GPU coordination, reduces engineering complexity, and makes iteration faster — not because the GPU is faster, but because the workflow is simpler.

The performance distinction from H100 is almost entirely memory- and bandwidth-driven, not compute-driven. Both GPUs deliver the same ~989 BF16 TFLOPS and ~1,979 FP8 TFLOPS. Teams expecting 2× faster training on a 7B model will be disappointed — the speedup on small models is marginal. Where the H200's 4.8 TB/s bandwidth advantage over H100's 3.35 TB/s pays off concretely is long-context fine-tuning. On 32K–128K context jobs, the bandwidth gap translates to meaningful reductions in wall-clock time and approximately 40% lower power consumption per token relative to H100 nodes, attributable to higher memory efficiency and reduced inter-GPU communication overhead.

Current pricing at $2.50–$3.50/hr on third-party providers will decline. As Blackwell B200 nodes (192GB HBM3e, 8 TB/s) scale through 2025 and displace H200 at the high end, H200 pricing is projected to settle at $1.80–$2.50/hr by 2026. Teams fine-tuning 70B models who can wait six months will access substantially better economics. Teams who need to run those jobs today should accept the current rate — the single-GPU operational simplicity versus an 8× H100 configuration justifies the premium.

The H200 is overkill for sub-30B models. At current pricing, paying $3.00/hr for 141GB of memory when a 13B model consumes 26GB is waste, not performance. Use H100 for anything below 30B and reserve H200 for 70B workloads, very long context fine-tuning, or cases where a single clean GPU simplifies your deployment and reproducibility requirements.

---

The Decision Framework

Match GPU to model size first, then optimize for cost.

For 7B–13B models, the A100 at $0.50–$0.80/hr in 2026 is the rational choice if you're running high experiment volume and each run is short. For production fine-tuning runs where iteration speed matters, the H100 wins on total cost per completed run despite the higher hourly rate.

For 14B–34B models, the H100 is the clear answer. Sufficient memory headroom, Transformer Engine throughput, deep spot market availability, and pricing that will improve through 2026.

For 70B models, the decision hinges on how frequently you're running jobs. Occasional fine-tuning runs can be handled cost-effectively on 4× H100 with QLoRA. Teams running weekly 70B fine-tuning iterations should benchmark a single H200 against their multi-H100 setup — the reduced coordination overhead frequently offsets the higher per-GPU cost, and the operational simplicity compounds over time.

The A100's realistic role in 2026 is academic labs and cost-sensitive teams running sub-13B experiments at high parallelism. Anyone building a production fine-tuning pipeline today should center it on H100, with H200 as the upgrade path for 70B workloads — and budget for Blackwell rates when planning anything beyond a 12-month horizon.