Ollama Cloud Models: Running Ollama on Cloud Infrastructure

Ollama doesn’t provide cloud-hosted models as a managed API service. Ollama is a local inference engine that runs open-source models on your own hardware. However, you can deploy Ollama on cloud infrastructure—AWS EC2, Google Cloud, Azure VMs, or specialized GPU providers—to get cloud compute power while keeping Ollama’s simple interface. This approach gives you control over the runtime environment and can be more cost-effective than commercial APIs at high usage volumes.

What Ollama Is (and What It Isn’t)

Ollama is an open-source inference engine that runs large language models locally. It handles model downloads, quantization, and inference through a simple CLI and REST API. According to the official Ollama repository, it supports models from the Llama, Mistral, Gemma, Qwen, and DeepSeek families, among others.

Ollama does not operate as a cloud provider. It doesn’t host models on its own servers or charge per-token pricing. When you run ollama run llama3.3, the model runs on whatever machine executed the command—your laptop, a data center server, or a cloud VM you’re paying for separately.

The term “ollama cloud models” typically refers to one of three deployment patterns:

• Running Ollama on a cloud VM with GPU acceleration
• Deploying Ollama in a container orchestration platform (Kubernetes, ECS)
• Using Ollama on a dedicated GPU cloud instance for on-demand scaling

Running Ollama on Major Cloud Providers

AWS EC2 GPU Instances

AWS offers GPU-enabled EC2 instances in the G, P, and Inf families. For Ollama workloads, the g5.xlarge (1× NVIDIA A10G, 24 GB VRAM) starts at approximately $1.01/hour on-demand in us-east-1, while g5.12xlarge (4× A10G, 96 GB VRAM) runs about $5.67/hour.

To deploy Ollama on EC2:

# Launch Ubuntu 22.04 g5.xlarge instance with Deep Learning AMI
# SSH into instance
sudo apt update
curl -fsSL https://ollama.com/install.sh | sh

# Verify GPU is detected
nvidia-smi

# Run a model
ollama run llama3.3:70b

A Llama 3.3 70B model requires approximately 40 GB VRAM at 4-bit quantization, so you’d need a g5.12xlarge or larger. Llama 3.1 8B needs around 5 GB VRAM at 4-bit, fitting comfortably on a g5.xlarge.

Google Cloud Platform

GCP provides GPU instances through the Compute Engine N1 and A2 machine families. An n1-standard-4 with 1× NVIDIA T4 (16 GB VRAM) costs approximately $0.62/hour, while a2-highgpu-1g with 1× A100 (40 GB VRAM) runs about $3.67/hour in us-central1.

# Create instance with GPU
gcloud compute instances create ollama-instance 
  --zone=us-central1-a 
  --machine-type=n1-standard-4 
  --accelerator=type=nvidia-tesla-t4,count=1 
  --image-family=ubuntu-2204-lts 
  --image-project=ubuntu-os-cloud 
  --maintenance-policy=TERMINATE

# SSH and install
gcloud compute ssh ollama-instance --zone=us-central1-a
curl -fsSL https://ollama.com/install.sh | sh
ollama run mistral:7b

Microsoft Azure

Azure’s NC-series VMs provide NVIDIA GPUs for inference workloads. An NC6s_v3 (1× V100, 16 GB VRAM) costs approximately $3.06/hour, while NC24ads_A100_v4 (1× A100, 80 GB VRAM) runs about $3.67/hour in East US.

Installation follows the same pattern: provision a GPU-enabled Ubuntu VM, install NVIDIA drivers if not using a pre-configured image, and run the Ollama install script.

GPU-Focused Cloud Providers

Specialized GPU cloud providers often offer better price-performance than major clouds for Ollama deployments:

Provider	GPU	VRAM	Cost/Hour	Good For
Lambda Labs	A100	40 GB	$1.10	DeepSeek R1 Distill Llama 70B, Llama 3.3 70B
Vast.ai	RTX 4090	24 GB	$0.34-$0.54	Mistral 7B, Llama 3.1 8B, Phi-4
RunPod	A40	48 GB	$0.79	Mistral Large 3 (single instance), Llama 3.3 70B
Paperspace	A4000	16 GB	$0.76	Smaller models up to ~14B parameters

Lambda Labs provides the simplest setup—instances come with NVIDIA drivers and CUDA pre-installed. After launching an instance through their dashboard:

ssh ubuntu@<instance-ip>
curl -fsSL https://ollama.com/install.sh | sh
ollama run qwen3:32b

Vast.ai operates as a marketplace for unused GPU capacity, offering the lowest prices but variable availability. You bid on or rent instances through their web interface, then SSH in to install Ollama.

Cost Comparison: Cloud Ollama vs API Services

Whether cloud-hosted Ollama makes economic sense depends on your usage volume. Use the self-hosting vs API calculator to find your break-even point.

Model	API Cost (per 1M tokens)	Cloud GPU	Instance Cost/Hour	Tokens/Hour at Break-Even
Llama 3.3 70B	$0.10 in / $0.32 out	Lambda A100 40GB	$1.10	~3.4M output tokens
Mistral Large 3	$2.00 in / $6.00 out	RunPod 4×A40	$3.16	~530K output tokens
Qwen3 32B	$0.08 in / $0.28 out	Vast.ai RTX 4090	$0.54	~1.9M output tokens
DeepSeek R1	$0.50 in / $2.15 out	Lambda 4×A100	$4.40	~2M output tokens

If you’re processing more than the break-even token volume per hour, cloud Ollama becomes cheaper. For bursty workloads or low volumes, APIs like Claude Sonnet 5 ($2.00 in / $10.00 out per 1M tokens) or Gemini 3.6 Flash ($1.50 in / $7.50 out per 1M tokens) offer better economics with no idle time costs.

Deployment Patterns

On-Demand Instances

Start a GPU instance when you need it, run your workload, then terminate. This works well for batch processing, development, or infrequent inference. All major clouds and GPU providers support on-demand pricing.

Spot/Preemptible Instances

AWS Spot Instances, GCP Preemptible VMs, and Azure Spot VMs offer 60-90% discounts but can be terminated with 30 seconds to 2 minutes notice. Suitable for fault-tolerant batch workloads where you can checkpoint progress.

On Vast.ai, interruptible instances run at market rates with no termination guarantee, offering the lowest prices but requiring robust error handling.

Container Orchestration

For production deployments, run Ollama in Kubernetes with GPU node pools. This enables auto-scaling, load balancing, and high availability. Use the official Ollama Docker image:

docker pull ollama/ollama
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 ollama/ollama

Then deploy to your cluster with GPU resource requests defined in your pod spec.

Model Selection for Cloud Deployments

Choose models based on your GPU VRAM budget. Use the VRAM calculator to estimate requirements:

• 16-24 GB VRAM (T4, RTX 4090, A10G): Mistral 7B (~4.5 GB at 4-bit), Llama 3.1 8B (~5 GB), Phi-4 (~9 GB), Qwen3 14B (~9 GB), Gemma 3 12B (~8 GB)
• 40-48 GB VRAM (A100 40GB, A40): Llama 3.3 70B (~40 GB), DeepSeek R1 Distill Llama 70B (~40 GB), Mistral NeMo 12B with room for larger context
• 80+ GB VRAM (A100 80GB, H100): Llama 4 Scout (~65 GB), DeepSeek R1 (~400 GB requires 4×A100 or similar), Mistral Large 3 (~400 GB)

Models requiring more than 80 GB VRAM need multi-GPU setups or tensor parallelism, which Ollama doesn’t natively support as of version 0.3. For those models, consider frameworks like vLLM or TGI, or use commercial APIs instead.

Performance Optimization

Several settings affect Ollama’s inference speed on cloud GPUs:

• Quantization: Ollama defaults to 4-bit quantization (Q4_0). Use ollama pull model:q8_0 for 8-bit (better quality, 2× VRAM) or model:q2_K for 2-bit (faster, lower quality).
• Context window: Set via num_ctx parameter. Larger contexts consume more VRAM—a 32K context uses significantly more memory than 4K for the same model.
• Batch size: Increase num_batch for throughput-oriented workloads where latency matters less than tokens/second.
• GPU layers: Ollama automatically offloads all layers to GPU. On instances with limited VRAM, it will fall back to CPU for layers that don’t fit, drastically reducing speed.

Frequently Asked Questions

Does Ollama offer its own cloud hosting?

No. Ollama is self-hosted software that runs models on your own infrastructure. There is no official Ollama cloud service or managed API. When people refer to “ollama cloud models,” they mean running the Ollama software on cloud infrastructure they provision themselves through AWS, GCP, Azure, or GPU cloud providers.

Can I use Ollama’s API format with cloud models?

Yes. Once Ollama is running on a cloud VM, it exposes a REST API on port 11434 that’s compatible with OpenAI’s API format. You can point any OpenAI-compatible client at your cloud instance’s IP address and port. This lets you use Ollama-hosted models as drop-in replacements for commercial APIs in many applications, though you’re responsible for managing availability, scaling, and security.

Which cloud provider is cheapest for running Ollama?

GPU-focused providers like Lambda Labs ($1.10/hour for A100 40GB) and Vast.ai ($0.34-$0.54/hour for RTX 4090) significantly undercut AWS, GCP, and Azure for equivalent GPU memory. Lambda offers better reliability and support; Vast.ai offers the lowest prices but variable availability. For production workloads requiring SLAs, major clouds provide better guarantees at higher cost.

How much does it cost to run Llama 3.3 70B on the cloud vs APIs?

Llama 3.3 70B needs about 40 GB VRAM at 4-bit quantization. A Lambda Labs A100 40GB instance costs $1.10/hour. If you generate 3.4 million output tokens per hour (~945 tokens/second sustained), you break even with the $0.32 per million token API rate. Below that threshold, APIs are cheaper. Above it, the cloud instance wins. Most real-world workloads are bursty rather than sustained, favoring API pricing unless you’re running batch processing jobs continuously.

Can Ollama scale across multiple GPUs in the cloud?

Ollama automatically detects and uses multiple GPUs on a single instance, but it runs one model per GPU rather than distributing a single model across GPUs (tensor parallelism). This means a 4×A100 instance can run four separate model instances or handle four concurrent requests efficiently, but it cannot load a single 400 GB model like Mistral Large 3 that exceeds one GPU’s capacity. For multi-GPU model parallelism, use vLLM, TensorRT-LLM, or Text Generation Inference instead.

Is running Ollama in the cloud secure?

By default, Ollama binds to 0.0.0.0:11434, exposing its API to any network client. On cloud instances with public IPs, this means your inference endpoint is publicly accessible unless you configure firewall rules, security groups, or VPN access. Set OLLAMA_HOST=127.0.0.1:11434 to restrict access to localhost, then use SSH tunneling, a reverse proxy with authentication, or a VPN to secure remote access. Cloud deployments should also implement request logging, rate limiting, and input validation to prevent abuse.