The A100's lower hourly rate is a trap. At $1.40-2.40/hr versus the H100's $2.50-3.00/hr, the sticker price looks attractive — until you measure what actually matters. Cost per training step. Cost per inference token. Cost per completed job. By every production metric, the H100 wins decisively in 2026.

The B200 raises the stakes further. With 8 TB/s memory bandwidth versus the H100's 3 TB/s and native FP4 support for inference, it's purpose-built for scaled LLM deployments. But availability remains constrained and pricing sits at a premium. The choice isn't which GPU is "best" — it's which GPU matches your actual workload, timeline, and cash position.

Why Choose the Right GPU?

The wrong GPU costs you 2-3x on every job. The right one pays for itself in weeks.

A100s still flood the market. Cloud providers bought thousands in 2021-2022. They're incentivized to keep selling capacity. Marketing teams frame the lower hourly rate as "cost-effective." But if your H100 job finishes in 6 hours versus 14 hours on an A100, you've spent $18 on the H100 versus $23 on the A100 — and saved 8 hours of calendar time.

Performance differences compound across workloads:

• Training: H100's 4th-gen Tensor Cores and 3 TB/s HBM3 memory bandwidth accelerate gradient computation and parameter updates
• Inference: B200's FP4 precision and 8 TB/s bandwidth deliver highest throughput for LLM token generation
• Long-context models: Memory bandwidth becomes the bottleneck; A100's 2 TB/s chokes on 128k+ context windows

Initial investment matters if you're buying hardware. The H100 costs approximately $47,000 per unit with ongoing power costs around $0.10/hr and cooling adding $0.30-0.50/hr. Add datacenter overhead, networking, and your real cost sits closer to $0.90-1.10/hr before you charge rent. Compare that to market rates of $2.50-3.00/hr and you see why GPU hosting remains profitable — covered extensively in our GPU Hosting Profitability Guide.

ROI calculations shift between scenarios:

• Renting for projects: Cost per job completion determines total spend
• Buying for hosting: Utilization rate and market pricing determine payback period
• Reserved capacity: Long-term contracts reduce hourly rates but lock capital

The GPU you choose directly affects your competitive position. If you're running inference services, faster token generation means better latency and higher throughput per dollar. If you're fine-tuning models on customer data, faster iterations mean more experiments and better results. If you're building on decentralized GPU marketplaces like Akash Network, choosing efficiently means your bids win against competitors.

Overview of H100, A100, and B200 GPUs

NVIDIA H100

The H100 became production-ready in mid-2023 and dominated 2024-2025 AI infrastructure deployments. It's NVIDIA's Hopper architecture — 4th generation Tensor Cores, 80GB HBM3 memory, 3 TB/s memory bandwidth.

Key specifications:

• 80GB HBM3 memory (or 94GB in HBM3e variant)
• 3 TB/s memory bandwidth
• 700W TDP (SXM5 variant)
• PCIe Gen5, NVLink 4.0
• FP8, FP16, BF16, INT8 precision support

The H100 handles mixed-precision training efficiently. Transformer models with billions of parameters benefit from the memory bandwidth — moving activations and gradients between compute units and memory faster than A100. FP8 support halves memory requirements for certain workloads without meaningful accuracy loss.

Power consumption sits at 700W for SXM5 variants (the ones in datacenter servers). Add another 200-300W for cooling in typical configurations. At $0.15/kWh commercial power rates, you're paying roughly $0.10/hr just for electricity. Cooling adds $0.30-0.50/hr depending on PUE (power usage effectiveness) of your datacenter.

Initial investment remains steep at $47,000 per card for enterprise buyers. The used market hasn't developed meaningfully yet — most H100s deployed in 2023-2024 remain in active service. Lease-to-own arrangements exist but typically lock you into 2-3 year terms.

Cloud pricing settled at $2.50-3.00/hr for on-demand access across major providers in 2026. Committed capacity brings rates to $1.80-2.20/hr with 6-12 month contracts. Spot pricing on decentralized networks occasionally dips to $1.50-1.80/hr during low-demand periods.

NVIDIA A100

The A100 launched in 2020 as NVIDIA's first Ampere architecture datacenter GPU. It dominated AI infrastructure for three years before the H100 arrived. Massive installed base means widespread availability and competitive pricing.

Key specifications:

• 40GB or 80GB HBM2e memory
• 2 TB/s memory bandwidth (80GB variant)
• 400W TDP (SXM4 variant)
• PCIe Gen4, NVLink 3.0
• FP16, BF16, TF32, INT8 precision support

The A100 handles most training workloads. Models under 30B parameters train reasonably well. Fine-tuning runs efficiently. Inference works but lacks the throughput of newer chips.

Memory bandwidth at 2 TB/s becomes the limiting factor. Large language models spend most compute time moving data, not calculating. When your model has 70B parameters and you're running inference with 32k context windows, you're constantly streaming weights and activations through memory. The H100's 50% bandwidth advantage translates directly to 40-50% faster token generation.

Power consumption is lower at 400W TDP. Actual draw under load sits at 350-380W for most workloads. Cooling requirements are proportionally lower. If you're hosting, A100s reduce your power bill but also command lower rental rates.

Pricing in 2026: $1.40-2.40/hr on-demand, $0.90-1.50/hr committed, $0.70-1.20/hr spot on decentralized networks. The lower hourly rate attracts development workloads, experimentation, and teams with tight budgets. But for production inference serving or large-scale training, the total job cost exceeds H100 alternatives.

The A100 retains value for specific scenarios:

• Development and prototyping where absolute hourly cost matters more than completion time
• Data preprocessing with occasional GPU bursts
• Inference for smaller models (under 13B parameters) where the bottleneck isn't memory bandwidth
• Legacy workloads already optimized for Ampere architecture

NVIDIA B200

The B200 represents NVIDIA's Blackwell architecture, shipping in volume through 2026. Purpose-built for the post-GPT-4 world where 100B+ parameter models and trillion-token training runs define production AI.

Key specifications:

• 192GB HBM3e memory
• 8 TB/s memory bandwidth
• 1000W TDP
• 2nd-gen Transformer Engine with FP4 precision
• NVLink 5.0

Memory bandwidth jumps to 8 TB/s — 2.67x the H100, 4x the A100. This isn't incremental. This changes what workloads are memory-bound versus compute-bound. Long-context LLMs with 128k+ context windows that choke on H100s run smoothly on B200s.

FP4 precision support matters specifically for LLM inference. You can quantize model weights to 4-bit floating point with minimal quality degradation on modern architectures. This doubles effective memory capacity and halves memory bandwidth requirements. A 70B parameter model that needs 140GB in FP8 fits in 70GB at FP4. The B200's native FP4 Tensor Core support means you don't pay performance penalties for quantization.

The 192GB memory capacity enables single-GPU inference for models up to 180B parameters at FP4 or 90B parameters at FP8. Multi-GPU setups with NVLink 5.0 can handle 400B+ parameter models efficiently.

Power consumption hits 1000W TDP. You're pulling 1.3-1.5 kW from the wall when you include inefficiencies. Cooling becomes more expensive — figure $0.50-0.80/hr in well-optimized datacenters. Electricity at $0.15/kWh costs $0.15-0.20/hr. Your base operating cost before profit margin sits at $0.65-1.00/hr.

Pricing in 2026 remains elevated due to constrained supply. Expect $4.50-6.00/hr on-demand where available. Reserved capacity with volume commitments brings this to $3.50-4.50/hr. The spot market hasn't developed yet — most B200 capacity goes to enterprise contracts.

The B200 makes sense when:

• You're running inference for 70B+ parameter models with high throughput requirements
• Training runs exceed 100B parameters or use extremely long context windows
• Memory bandwidth is your proven bottleneck (measure this first)
• Your revenue per GPU-hour exceeds $10+ (inference services, not training experiments)

Performance Comparison

Benchmark Results

Raw FLOPS tell you little about real workload performance. Memory bandwidth, precision support, and architecture-specific optimizations matter more for modern AI workloads.

Training throughput (Llama-70B-style architecture):

• A100 80GB: ~1.2-1.4 tokens/sec per GPU
• H100 80GB: ~2.1-2.4 tokens/sec per GPU (1.7x faster)
• B200 192GB: ~3.8-4.3 tokens/sec per GPU (3.2x faster than A100)

These numbers assume FP8 mixed precision, optimized frameworks (PyTorch 2.x with compile), and efficient batch sizing. Actual throughput varies with sequence length, batch size, and parallelism strategy.

Inference latency (Llama-70B at FP8):

• A100 80GB: ~45-55ms per token (batch size 1)
• H100 80GB: ~28-35ms per token (1.6x faster)
• B200 192GB: ~18-23ms per token at FP4 (2.5x faster than A100)

The B200's FP4 support creates an unfair advantage for inference. You're comparing 4-bit precision on B200 versus 8-bit on H100/A100. Quality degradation is minimal for most LLM architectures with proper quantization-aware training.

Long-context performance (128k tokens): Memory bandwidth dominates. The A100 chokes — you're constantly waiting for attention matrix calculations to move through memory. H100 improves but still bottlenecks. B200 handles 128k contexts smoothly.

Measured throughput on 128k context inference:

• A100 80GB: Borderline unusable, ~0.3-0.5 tokens/sec
• H100 80GB: ~1.1-1.4 tokens/sec
• B200 192GB: ~2.8-3.4 tokens/sec

The 8 TB/s bandwidth matters. Attention calculation scales quadratically with sequence length. You're moving massive activation matrices through memory on every forward pass.

Training efficiency (measured in samples/sec on ImageNet): Vision models show different characteristics than LLMs. ConvNets care more about raw compute. Transformers care more about memory bandwidth.

• A100: ~3,200 images/sec (ResNet-50)
• H100: ~5,100 images/sec (1.6x faster)
• B200: ~7,800 images/sec (2.4x faster)

Vision workloads benefit less from B200's extreme bandwidth because activation sizes remain reasonable. The performance improvement comes more from architectural upgrades than memory subsystem.

Real-World Use Cases

Fine-tuning Llama-70B on domain-specific data:

A research lab wants to fine-tune Llama-70B on 50GB of proprietary medical documentation. 10,000 training steps, 4k context length, batch size 16.

• A100 cluster (8x GPUs): ~48 hours wall-clock time, cost ~$920 at $2.40/hr per GPU
• H100 cluster (8x GPUs): ~28 hours wall-clock time, cost ~$672 at $3.00/hr per GPU
• B200 cluster (4x GPUs, thanks to 192GB memory): ~16 hours wall-clock time, cost ~$352 at $5.50/hr per GPU

The B200 uses half the GPUs due to larger memory, reducing communication overhead and simplifying setup. Total cost drops 62% versus A100s.

Running production inference for a customer-facing chatbot:

A SaaS company serves 100k daily users. Each user averages 15 messages/day with 2k token responses. Peak load: 500 concurrent requests. Model: Llama-70B.

• A100 setup: 12 GPUs to handle peak load, ~45ms latency per token, cost $345/day (12 × $2.00/hr × 24hr × 60% utilization)
• H100 setup: 8 GPUs to handle peak load, ~30ms latency per token, cost $384/day (8 × $2.50/hr × 24hr × 80% utilization)
• B200 setup: 4 GPUs at FP4 precision, ~20ms latency per token, cost $360/day (4 × $5.00/hr × 24hr × 75% utilization)

The B200 delivers better latency with fewer GPUs at comparable cost. But H100 offers the best balance of cost, availability, and performance for this workload. The A100 requires more GPUs to hit acceptable latency, and the cost savings evaporate.

Training a 13B parameter model from scratch:

A startup wants to pre-train a specialized 13B model on 500B tokens of code.

• A100 (8x GPU cluster): ~21 days, cost ~$9,600
• H100 (8x GPU cluster): ~12 days, cost ~$6,912
• B200 (4x GPU cluster): ~7 days, cost ~$6,720

At this scale, H100 and B200 costs converge. The B200 finishes 5 days faster, which matters if you're racing to market. The H100 offers better availability and flexibility. The A100 costs more and takes longer — there's no rational reason to choose it for this workload in 2026.

Running batch inference for document processing:

A legal tech company processes 10 million documents monthly through a 13B parameter model. Latency doesn't matter — throughput does.

• A100: Can process ~400k documents/day per GPU, needs 1 GPU running 24/7, cost ~$1,440/month
• H100: Can process ~680k documents/day per GPU, needs 1 GPU running 12hrs/day, cost ~$900/month
• B200: Overkill for 13B model, cost doesn't justify throughput gains

The H100 wins on cost efficiency. The B200's advantages — extreme bandwidth, FP4 support — don't matter because the 13B model doesn't stress memory subsystems.

Cost Analysis

Initial Investment Cost

Buying GPUs means committing capital with 3-5 year depreciation horizons. The market in 2026 offers limited used inventory — most H100s and all B200s remain in first-owner hands.

New purchase pricing (per GPU):

• A100 80GB SXM: $12,000-15,000 (supply abundant, prices falling)
• H100 80GB SXM: $45,000-48,000 (stable pricing, high demand)
• B200 192GB SXM: $70,000-80,000 (limited availability, enterprise allocation)

Server configurations add cost. An 8-GPU server chassis with NVLink, power distribution, and cooling infrastructure adds $40,000-60,000 on top of GPU costs. A fully configured H100 server runs $420,000-450,000. B200 systems exceed $600,000.

Infrastructure requirements scale with power density:

• A100 8-GPU server: 3.5-4.0 kW total draw, standard datacenter racks work
• H100 8-GPU server: 6.0-7.0 kW total draw, requires upgraded power distribution
• B200 8-GPU server: 9.0-11.0 kW total draw, specialized cooling and power infrastructure

Datacenter build-out costs vary regionally. Figure $2,000-3,000 per kW of capacity for power infrastructure, $1,500-2,500 per kW for cooling in modern facilities. An H100 cluster needs $50,000-80,000 in infrastructure per server beyond the hardware costs.

The total capital requirement for a competitive GPU hosting operation:

• Minimum viable: 32-64 H100 GPUs (4-8 servers), $2.0-3.5M all-in
• Competitive scale: 256+ H100 GPUs (32+ servers), $16-20M all-in
• Enterprise tier: 1,024+ GPUs with mix of H100/B200, $60-100M all-in

Financing options exist. Equipment leasing spreads costs over 3-5 years at 6-9% interest rates. Vendor financing through NVIDIA partners offers better terms but restricts resale. Revenue-based financing (rare but emerging) ties repayment to actual GPU utilization.

Hourly Rates and Cloud Pricing

Market pricing in 2026 shows distinct tiers based on provider type, contract structure, and service level.

On-demand pricing (hourly, no commitment):

• A100 80GB: $1.40-2.40/hr (major clouds), $0.90-1.60/hr (decentralized markets)
• H100 80GB: $2.50-3.50/hr (major clouds), $1.99-2.80/hr (decentralized markets)
• B200 192GB: $5.00-7.00/hr (limited availability, mostly reserved capacity)

The spread between centralized clouds and decentralized GPU marketplaces persists. Major clouds bundle support, SLAs, and network infrastructure. Decentralized markets offer raw compute at 30-50% discounts with variable reliability.

Reserved capacity (6-12 month commitments):

• A100 80GB: $0.90-1.50/hr (40-50% discount vs on-demand)
• H100 80GB: $1.80-2.40/hr (30-40% discount vs on-demand)
• B200 192GB: $3.50-5.00/hr (limited availability)

Reserved pricing requires upfront commitment but dramatically reduces costs for sustained workloads. The break-even point sits at 35-45% utilization — if you'll use the GPU more than ~300 hours/month, reserved capacity pays off.

Spot/interruptible pricing:

• A100 80GB: $0.70-1.20/hr (50-60% below on-demand)
• H100 80GB: $1.50-2.10/hr (40-50% below on-demand)
• B200 192GB: Limited availability

Spot instances get preempted when demand spikes. Useful for fault-tolerant workloads with checkpointing. Training runs that save state every 30 minutes handle interruptions gracefully. Real-time inference can't use spot capacity.

Per-second vs hourly billing: Many providers still bill hourly increments. A 15-minute job costs the same as 60 minutes. Per-second billing saves 30-40% on short jobs. This matters most for development iterations, batch inference jobs that complete in minutes, and bursty workloads with variable runtime.

Long-Term ROI

The ROI calculation depends entirely on your utilization rate and revenue model.

Scenario 1: GPU hosting provider

You buy H100s at $47,000 each, rent them at $2.50/hr.

Monthly revenue per GPU:

• 100% utilization: $1,800/month (720 hours × $2.50/hr)
• 80% utilization: $1,440/month (realistic for managed hosting)
• 60% utilization: $1,080/month (common for spot/interruptible offerings)

Monthly costs per GPU:

• Power: $72/month ($0.10/hr × 720 hours)
• Cooling: $288/month ($0.40/hr × 720 hours)
• Network/bandwidth: $40/month
• Support/overhead: $100/month
• Total: $500/month

Net profit at 80% utilization: $940/month

Payback period: $47,000 / $940 = 50 months (4.2 years)

This assumes stable pricing. Reality includes market volatility, competition, and depreciation. The H100's value drops as newer chips arrive. By 2028, H100 rental rates may compress to $1.80-2.20/hr as B200/B300 supply increases.

Adjusted payback with depreciation: 3.5-4.0 years accounting for resale value declining to $18,000-22,000 by 2028.

Scenario 2: Internal ML team

You buy H100s for your AI research team instead of renting cloud capacity.

Current cloud spend: $15,000/month (5 H100s × $100/hr average usage × 30 hours/week)

Capital cost: $235,000 (5 × $47,000) Operating cost: $2,500/month (5 GPUs × $500/month)

Payback: $235,000 / ($15,000 - $2,500) = 18.8 months

After payback, you're saving $12,500/month versus cloud. Over 4 years, total savings: ~$370,000.

But this assumes your usage patterns remain stable (most teams expand significantly), you have datacenter space or colocation budget, you don't need the flexibility to scale down rapidly, and maintenance and replacement risk sits with you.

Scenario 3: Inference service provider

You run a LLM API serving customers, processing 50M tokens/day. Revenue: $0.30 per 1M tokens = $15,000/day.

Options:

• Rent 8× H100 on-demand: $480/day (8 × $2.50/hr × 24hr), 97% margin
• Rent 8× H100 reserved: $384/day (8 × $2.00/hr × 24hr), 97.4% margin
• Buy 8× H100: $376,000 upfront, $4,000/month operating cost = $133/day depreciated, 99.1% margin

Buying wins decisively at this scale and consistency. Your business runs on GPU compute — owning the infrastructure reduces variable costs to near-zero and protects against spot price spikes.

The $376,000 investment pays back in: $376,000 / ($384 - $133) = 1,498 days ≈ 4.1 years

But revenue growth changes the math. If you scale to 200M tokens/day in year 2, your owned capacity can't flex without buying more GPUs. Cloud capacity scales with revenue.

Use Case Analysis

Training Workloads

Large-scale pre-training demands the most expensive infrastructure. You're running thousands of GPU-hours on multi-billion parameter models.

Pre-training from scratch (100B+ parameters): The B200 becomes cost-effective despite higher hourly rates. Training a 100B model on 2T tokens requires ~50,000-80,000 GPU-hours. Memory bandwidth directly determines iteration speed.

• A100 cluster (256 GPUs): ~18-22 days, cost ~$1.1-1.4M
• H100 cluster (256 GPUs): ~10-13 days, cost ~$0.78-1.0M
• B200 cluster (128 GPUs, leveraging 192GB memory): ~6-8 days, cost ~$0.74-0.96M

The B200 uses half the GPUs and finishes in half the time. Communication overhead drops with fewer nodes. Total cost matches H100 while delivering 40% faster time-to-model.

Fine-tuning (7B-70B parameters): The H100 dominates this category in 2026. You're not hitting extreme memory limits, so B200's advantages don't justify the premium.

• Fine-tuning Llama-7B: H100 at $2.50/hr completes in 4-6 hours ($10-15 total), A100 at $1.80/hr takes 8-11 hours ($14-20 total)
• Fine-tuning Llama-70B: H100 cluster (8 GPUs) at $20/hr completes in 12-18 hours ($240-360 total), A100 cluster (8 GPUs) at $14.40/hr takes 24-36 hours ($346-518 total)

The pattern repeats: H100 costs less per completed job despite higher hourly rates.

LoRA fine-tuning (parameter-efficient): Low-rank adaptation reduces memory requirements dramatically. You're training 0.1-1% of model parameters instead of 100%.

At this scale, A100s become viable again:

• Llama-70B LoRA on A100: 4-6 hours on single GPU, cost $7-14
• Llama-70B LoRA on H100: 2.5-4 hours on single GPU, cost $6-10

The difference narrows because you're no longer bandwidth-constrained. The model fits comfortably in memory and compute becomes the bottleneck.

For operators running AI consulting businesses doing frequent client fine-tuning, A100s make sense for LoRA workloads. Full fine-tuning still favors H100s.

Inference Workloads

Inference economics differ fundamentally from training. Latency, throughput, and utilization patterns matter more than raw training speed.

Real-time inference (chatbots, assistants): Latency targets drive GPU choice. Users expect <500ms response times, ideally <300ms. Token generation latency multiplies by output length.

For 70B models serving interactive chat:

• A100: ~45ms per token × 100 tokens = 4,500ms (unacceptable)
• H100: ~30ms per token × 100 tokens = 3,000ms (marginal)
• B200 at FP4: ~20ms per token × 100 tokens = 2,000ms (acceptable)

You need B200 or aggressive batching strategies on H100. Batching trades latency for throughput — processing multiple requests simultaneously increases per-token latency but improves total throughput.

Optimized H100 with batching: ~35-40ms per token with batch size 8-16, serving 8-16 users simultaneously. Amortized cost per request drops proportionally.

Batch inference (document processing, embedding generation): Latency doesn't matter. Throughput per dollar determines winner.

For 13B models processing documents:

• A100: ~2,500 documents/hr at $1.80/hr = 1,389 docs per dollar
• H100: ~4,200 documents/hr at $2.50/hr = 1,680 docs per dollar
• B200: ~6,800 documents/hr at $5.50/hr = 1,236 docs per dollar

The H100 wins on cost efficiency. The B200 offers highest absolute throughput if you're racing deadlines, but costs more per document processed.

Multi-tenant inference (API services): You're serving hundreds or thousands of customers with variable request patterns. Utilization optimization matters more than per-request cost.

The ideal setup in 2026:

• H100s for baseline capacity (80-90% utilization)
• Spot A100s for overflow during traffic spikes (30-50% cheaper, acceptable latency degradation)
• Reserved B200 capacity for premium tier customers requiring <200ms responses

This hybrid approach balances cost and performance. Most requests route to H100s. Traffic spikes overflow to cheap A100 spot capacity. Premium customers paying 3-5x standard rates get B200 performance.

Large-Scale AI Models

The frontier in 2026 sits at 300B-1T parameter models with 128k-1M context windows. These workloads expose hardware limits.

Long-context LLMs (128k+ tokens): Memory bandwidth becomes the sole determining factor. Attention calculation requires moving the entire KV cache through memory on every token.

A 70B model with 128k context at FP8 precision:

• KV cache size: ~112GB
• Attention bandwidth per token: ~224 GB (read KV cache, write new entries)

Per-token attention time:

• A100 (2 TB/s): ~112ms just for attention
• H100 (3 TB/s): ~75ms just for attention
• B200 (8 TB/s): ~28ms just for attention

The B200 isn't 2.67x faster — it's 4x faster on long-context workloads because attention dominates total compute time. The A100 becomes practically unusable.

Mixture-of-Experts (MoE) models: Models like Mixtral-8×7B activate 2 of 8 expert layers per token. Total parameters: 46.7B, but only 12.9B active per forward pass.

MoE architectures benefit from large memory capacity more than raw bandwidth:

• A100 80GB: Can run Mixtral-8×7B at FP16 (~94GB total)
• H100 80GB: Can run Mixtral-8×22B at FP8 (~176GB total, requires 2 GPUs)
• B200 192GB: Can run Mixtral-8×22B at FP8 on single GPU (~176GB)

The memory capacity advantage reduces deployment complexity. Single-GPU inference simplifies serving infrastructure and eliminates cross-GPU communication latency.

Multimodal models (vision + language): Models like GPT-4V process images and text simultaneously. Vision encoders add memory pressure.

A typical multimodal architecture:

• Language model: 70B parameters (~140GB at FP8)
• Vision encoder: 5B parameters (~10GB at FP8)
• Cross-attention layers: ~15GB
• Activation memory: ~20GB
• Total: ~185GB minimum

You need B200 for single-GPU deployment or complex multi-GPU setups on H100. The A100's 80GB can't handle these workloads without extreme quantization or model parallelism.

Comparison Table

Performance Metrics

| Metric | A100 80GB | H100 80GB | B200 192GB | |--------|-----------|-----------|------------| | Memory Bandwidth | 2 TB/s | 3 TB/s | 8 TB/s | | Memory Capacity | 80GB HBM2e | 80GB HBM3 | 192GB HBM3e | | TDP | 400W | 700W | 1000W | | FP8 Performance | 624 TFLOPS | 1,979 TFLOPS | ~4,000 TFLOPS | | On-demand Price | $1.40-2.40/hr | $2.50-3.00/hr | $4.50-6.00/hr | | Best For | Development, LoRA | Production training/inference | Frontier models, long context |

The table shows specifications. What it doesn't show is the decision that matters: stop comparing hourly rates and start measuring cost per completed job. Run a 10-hour benchmark on A100, a 6-hour benchmark on H100, multiply by the hourly rate, and let the numbers end the debate. For production workloads in 2026, the H100 costs less. For frontier workloads pushing memory limits, the B200 costs less. The A100's only remaining advantage is availability — and that's not a technical argument, it's a concession.