{"id":2547,"date":"2026-09-05T20:17:41","date_gmt":"2026-09-05T20:17:41","guid":{"rendered":"https:\/\/convly.ai\/?p=2547"},"modified":"2026-09-05T20:17:41","modified_gmt":"2026-09-05T20:17:41","slug":"vllm-install","status":"publish","type":"post","link":"https:\/\/convly.ai\/pt\/vllm-install\/","title":{"rendered":"Como instalar o vLLM no Linux, Windows e macOS"},"content":{"rendered":"<div class=\"convly-tldr\">\n<ul>\n<li><strong>Linux + NVIDIA GPU:<\/strong> create a clean Python 3.12 environment, run <code>pip install vllm<\/code> (or <code>uv pip install vllm<\/code>), then <code>vllm serve Qwen\/Qwen3-8B<\/code>. You get an OpenAI-compatible API on port 8000.<\/li>\n<li><strong>Windows:<\/strong> there are no native Windows wheels. Use WSL2 with Ubuntu, or the <code>vllm\/vllm-openai<\/code> Docker image.<\/li>\n<li><strong>macOS:<\/strong> no published wheel. Apple silicon requires a source build and runs on CPU only \u2014 for a Mac laptop, Ollama or LM Studio is the practical choice.<\/li>\n<li><strong>Biggest install trap:<\/strong> vLLM pins its own PyTorch build. Installing it on top of an existing torch is the most common cause of import and CUDA errors. Always use a fresh virtual environment.<\/li>\n<\/ul>\n<\/div>\n<p>To install vLLM on Linux with an NVIDIA GPU, create a clean Python 3.12 environment and run <code>pip install vllm<\/code>, then start a server with <code>vllm serve Qwen\/Qwen3-8B<\/code>. That is the entire happy path. Windows requires WSL2 or Docker because vLLM publishes Linux-only wheels, and macOS requires a build from source that runs CPU-only inference.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_86 counter-flat ez-toc-counter ez-toc-container-direction\">\n<label for=\"ez-toc-cssicon-toggle-item-6a9ca005c7f05\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #000000;color:#000000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #000000;color:#000000\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a9ca005c7f05\"  aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Before_you_install_what_vLLM_actually_requires\" >Before you install: what vLLM actually requires<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Install_vLLM_on_Linux_with_an_NVIDIA_GPU\" >Install vLLM on Linux with an NVIDIA GPU<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Install_with_Docker_the_most_reproducible_option\" >Install with Docker (the most reproducible option)<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Install_vLLM_on_Windows_WSL2\" >Install vLLM on Windows (WSL2)<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Install_vLLM_on_macOS\" >Install vLLM on macOS<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#AMD_Intel_and_CPU-only_Linux\" >AMD, Intel and CPU-only Linux<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Start_the_server_and_test_it\" >Start the server and test it<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Will_the_model_fit_Size_it_before_you_install\" >Will the model fit? Size it before you install<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Common_vLLM_install_and_startup_errors\" >Common vLLM install and startup errors<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/convly.ai\/pt\/vllm-install\/#Frequently_asked_questions\" >Frequently asked questions<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Before_you_install_what_vLLM_actually_requires\"><\/span>Before you install: what vLLM actually requires<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<table>\n<thead>\n<tr>\n<th>Requirement<\/th>\n<th>What works<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Operating system<\/td>\n<td>Linux (x86_64 is the primary target; some releases also publish aarch64 wheels). Windows only via WSL2 or Docker. macOS via source build.<\/td>\n<\/tr>\n<tr>\n<td>Python<\/td>\n<td>3.9\u20133.12 covers releases through most of 2025, with 3.13 added in later versions. The supported range shifts between releases \u2014 check the release notes for the version you install.<\/td>\n<\/tr>\n<tr>\n<td>GPU<\/td>\n<td>NVIDIA GPUs with compute capability 7.0 or higher (V100, T4, RTX 20-series and newer, A10, L4, A100, H100, H200). AMD cards need the separate ROCm build.<\/td>\n<\/tr>\n<tr>\n<td>CUDA<\/td>\n<td>A current NVIDIA driver. The default PyPI wheel ships with the CUDA runtime that its bundled PyTorch build needs, so you do <em>not<\/em> need a system CUDA toolkit unless you compile from source.<\/td>\n<\/tr>\n<tr>\n<td>Disk<\/td>\n<td>The package itself is a few GB. Model weights dominate \u2014 they land in <code>~\/.cache\/huggingface<\/code> and range from a few GB to hundreds of GB.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The authoritative and continuously updated list lives in the project&#8217;s own documentation at <a href=\"https:\/\/docs.vllm.ai\/en\/latest\/\" rel=\"noopener\" target=\"_blank\">docs.vllm.ai<\/a>, and version-specific changes are recorded on the <a href=\"https:\/\/github.com\/vllm-project\/vllm\/releases\" rel=\"noopener\" target=\"_blank\">vLLM releases page<\/a>. Published wheels and their supported Python versions are listed on <a href=\"https:\/\/pypi.org\/project\/vllm\/\" rel=\"noopener\" target=\"_blank\">PyPI<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Install_vLLM_on_Linux_with_an_NVIDIA_GPU\"><\/span>Install vLLM on Linux with an NVIDIA GPU<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3>Option 1: uv (fastest, and what the vLLM docs now recommend)<\/h3>\n<pre><code>curl -LsSf https:\/\/astral.sh\/uv\/install.sh | sh\nuv venv vllm-env --python 3.12 --seed\nsource vllm-env\/bin\/activate\nuv pip install vllm --torch-backend=auto<\/code><\/pre>\n<p>The <code>--torch-backend=auto<\/code> flag lets uv detect your driver and pick a matching PyTorch\/CUDA build. If your driver is older than the wheel expects, replace <code>auto<\/code> with an explicit backend such as <code>cu126<\/code>. Which CUDA builds are available changes with each release, so consult the installation page rather than assuming a specific tag exists.<\/p>\n<h3>Option 2: plain pip and venv<\/h3>\n<pre><code>python3.12 -m venv ~\/vllm-env\nsource ~\/vllm-env\/bin\/activate\npip install --upgrade pip\npip install vllm<\/code><\/pre>\n<h3>Option 3: conda<\/h3>\n<pre><code>conda create -n vllm python=3.12 -y\nconda activate vllm\npip install vllm<\/code><\/pre>\n<p>Note that vLLM should be installed with <code>pip<\/code> even inside a conda environment. Do not install PyTorch separately first \u2014 vLLM will pull the exact torch build it was compiled against.<\/p>\n<h3>Verify the install<\/h3>\n<pre><code>vllm --version\npython -c \"import torch; print(torch.__version__, torch.cuda.is_available())\"\nnvidia-smi<\/code><\/pre>\n<p>If <code>torch.cuda.is_available()<\/code> prints <code>False<\/code>, stop here \u2014 the problem is your driver or environment, not vLLM, and no serve command will work until that reads <code>True<\/code>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Install_with_Docker_the_most_reproducible_option\"><\/span>Install with Docker (the most reproducible option)<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The project publishes an official OpenAI-compatible server image. This skips Python environment problems entirely and is the route worth taking on a shared or production box:<\/p>\n<pre><code>docker run --runtime nvidia --gpus all \n  -v ~\/.cache\/huggingface:\/root\/.cache\/huggingface \n  --env \"HF_TOKEN=$HF_TOKEN\" \n  -p 8000:8000 \n  --ipc=host \n  vllm\/vllm-openai:latest \n  --model Qwen\/Qwen3-8B<\/code><\/pre>\n<p>The <code>--ipc=host<\/code> flag matters: vLLM uses shared memory between processes, and Docker&#8217;s small default <code>\/dev\/shm<\/code> causes crashes with tensor parallelism. If you cannot use host IPC, pass <code>--shm-size=8g<\/code> instead. Mounting the Hugging Face cache means you download each model once rather than once per container. This requires the NVIDIA Container Toolkit on the host.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Install_vLLM_on_Windows_WSL2\"><\/span>Install vLLM on Windows (WSL2)<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>vLLM has no native Windows build. WSL2 is the supported path and works well:<\/p>\n<ol>\n<li>Install the standard NVIDIA <strong>Windows<\/strong> driver. Do not install a Linux display driver inside WSL \u2014 the WSL CUDA stack maps through the Windows driver. NVIDIA documents this in the <a href=\"https:\/\/docs.nvidia.com\/cuda\/wsl-user-guide\/index.html\" rel=\"noopener\" target=\"_blank\">CUDA on WSL user guide<\/a>.<\/li>\n<li>In PowerShell: <code>wsl --install -d Ubuntu-24.04<\/code>, then reboot if prompted.<\/li>\n<li>Inside Ubuntu, run <code>nvidia-smi<\/code>. If your GPU is not listed, fix that before continuing.<\/li>\n<li>Install <code>python3.12-venv<\/code>, create a venv, and follow the Linux instructions above.<\/li>\n<\/ol>\n<p>Two WSL-specific gotchas: WSL caps RAM by default, so add a <code>[wsl2]<\/code> section with <code>memory=<\/code> to <code>C:Users&lt;you&gt;.wslconfig<\/code> if model loading gets killed; and model weights stored on the Windows filesystem (<code>\/mnt\/c\/...<\/code>) load noticeably slower than weights inside the WSL filesystem.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Install_vLLM_on_macOS\"><\/span>Install vLLM on macOS<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There is no macOS wheel on PyPI. Apple silicon support is a CPU-only source build:<\/p>\n<pre><code>xcode-select --install\ngit clone https:\/\/github.com\/vllm-project\/vllm.git\ncd vllm\npip install -r requirements\/cpu.txt\npip install -e .<\/code><\/pre>\n<p>The requirements file path has moved between versions (it was <code>requirements-cpu.txt<\/code> in older releases), so check the repository tree for the tag you check out. Critically, this build does not use Metal or the Apple GPU \u2014 inference runs on CPU and is far slower than a CUDA machine. vLLM&#8217;s design goal is high-throughput batched serving on server GPUs, which is not what a Mac laptop is. If your goal is running a model locally on macOS, use the <a href=\"https:\/\/convly.ai\/what-is-ollama-complete-guide-2026\/\">Ollama<\/a> route or <a href=\"https:\/\/convly.ai\/lm-studio-complete-guide-2026\/\">LM Studio<\/a>, both of which use Metal properly.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"AMD_Intel_and_CPU-only_Linux\"><\/span>AMD, Intel and CPU-only Linux<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>ROCm (AMD), Intel GPU\/XPU and CPU-only x86 all have their own installation paths, generally either a prebuilt Docker image or a source build with a target-device environment variable such as <code>VLLM_TARGET_DEVICE=cpu<\/code>. These backends move faster than the CUDA path and the exact commands change between releases, so follow the hardware-specific page in the current docs rather than copying a command from a tutorial.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Start_the_server_and_test_it\"><\/span>Start the server and test it<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<pre><code>vllm serve Qwen\/Qwen3-8B \n  --max-model-len 8192 \n  --gpu-memory-utilization 0.90 \n  --port 8000<\/code><\/pre>\n<p><a href=\"https:\/\/huggingface.co\/Qwen\/Qwen3-8B\" rel=\"noopener\" target=\"_blank\">Qwen3-8B<\/a> is a good first target because it is ungated, downloads quickly, and fits a single 24 GB card at bf16. Then test:<\/p>\n<pre><code>curl http:\/\/localhost:8000\/v1\/models\n\ncurl http:\/\/localhost:8000\/v1\/chat\/completions \n  -H \"Content-Type: application\/json\" \n  -d '{\"model\": \"Qwen\/Qwen3-8B\", \"messages\": [{\"role\": \"user\", \"content\": \"Say hi\"}]}'<\/code><\/pre>\n<p><strong>Security note:<\/strong> <code>vllm serve<\/code> starts with no authentication. Pass <code>--api-key<\/code> and keep port 8000 behind a firewall or reverse proxy \u2014 an exposed vLLM endpoint is an open, uncapped inference bill on your own hardware.<\/p>\n<p>Useful flags on first run: <code>--tensor-parallel-size N<\/code> to shard across N GPUs, <code>--max-model-len<\/code> to cap context (the single most effective lever against startup OOM), <code>--quantization<\/code> for pre-quantized checkpoints, <code>--served-model-name<\/code> to expose a shorter alias to clients, and <code>--enforce-eager<\/code> to skip CUDA graph capture when you want faster startup while debugging.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Will_the_model_fit_Size_it_before_you_install\"><\/span>Will the model fit? Size it before you install<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>vLLM loads weights in bf16 by default, so budget roughly 2 GB of VRAM per billion parameters, plus KV cache \u2014 and vLLM preallocates that cache aggressively (90% of the card at the default <code>--gpu-memory-utilization<\/code>). An 8B model is about 16 GB of weights at bf16. Quantized to 4 bits, the same model drops to roughly 5 GB, per the Convly <a href=\"https:\/\/convly.ai\/models\/\">models database<\/a>:<\/p>\n<table>\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Context<\/th>\n<th>~VRAM at 4-bit<\/th>\n<th>Realistic single-node setup<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Qwen3 8B<\/td>\n<td>128K<\/td>\n<td>~5 GB<\/td>\n<td>One 12\u201324 GB consumer card<\/td>\n<\/tr>\n<tr>\n<td>Llama 3.1 8B<\/td>\n<td>128K<\/td>\n<td>~5 GB<\/td>\n<td>One 12\u201324 GB consumer card<\/td>\n<\/tr>\n<tr>\n<td>Gemma 3 27B<\/td>\n<td>128K<\/td>\n<td>~16 GB<\/td>\n<td>One 24 GB card at 4-bit<\/td>\n<\/tr>\n<tr>\n<td>Qwen3 32B<\/td>\n<td>128K<\/td>\n<td>~20 GB<\/td>\n<td>One 24 GB card at 4-bit, tight on KV cache<\/td>\n<\/tr>\n<tr>\n<td>Llama 3.3 70B<\/td>\n<td>128K<\/td>\n<td>~40 GB<\/td>\n<td>2\u00d7 24 GB with <code>--tensor-parallel-size 2<\/code>, or one 48 GB card<\/td>\n<\/tr>\n<tr>\n<td>DeepSeek R1<\/td>\n<td>128K<\/td>\n<td>~400 GB<\/td>\n<td>Multi-GPU server, not a workstation<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>For a figure specific to your context length and batch size, use the <a href=\"https:\/\/convly.ai\/llm-vram-calculator\/\">VRAM calculator<\/a>; the fuller per-model breakdown is in the <a href=\"https:\/\/convly.ai\/vram-requirements-every-major-llm-2026\/\">VRAM requirements guide<\/a>. If you are still choosing hardware, see <a href=\"https:\/\/convly.ai\/best-gpus-for-local-llms-2026\/\">best GPUs for local LLMs<\/a>. And before you buy anything, it is worth running the numbers with the <a href=\"https:\/\/convly.ai\/self-hosting-vs-api-calculator\/\">self-hosting vs API calculator<\/a> \u2014 Llama 3.3 70B costs $0.10 in \/ $0.32 out per 1M tokens from a hosted provider, which is hard to beat on your own electricity below fairly high sustained utilization.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Common_vLLM_install_and_startup_errors\"><\/span>Common vLLM install and startup errors<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<table>\n<thead>\n<tr>\n<th>Symptom<\/th>\n<th>Cause and fix<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code>ImportError<\/code> on <code>vllm._C<\/code>, or an ABI\/symbol error from torch<\/td>\n<td>vLLM was installed over an incompatible PyTorch. Delete the environment, recreate it clean, install vLLM first.<\/td>\n<\/tr>\n<tr>\n<td>&#8220;The model&#8217;s max seq len is larger than the maximum number of tokens that can be stored in KV cache&#8221;<\/td>\n<td>Not enough free VRAM for the requested context. Lower <code>--max-model-len<\/code>, raise <code>--gpu-memory-utilization<\/code>, or use a quantized checkpoint.<\/td>\n<\/tr>\n<tr>\n<td><code>CUDA out of memory<\/code> while loading weights<\/td>\n<td>The weights themselves do not fit. Shard with <code>--tensor-parallel-size<\/code> or pick a smaller model.<\/td>\n<\/tr>\n<tr>\n<td><code>Found no NVIDIA driver<\/code><\/td>\n<td>No GPU visible to the process. In WSL, the driver belongs on the Windows side; in Docker, you are missing <code>--gpus all<\/code>.<\/td>\n<\/tr>\n<tr>\n<td>401\/403 downloading a model<\/td>\n<td>Gated repository. Accept the licence on Hugging Face, then authenticate (<code>hf auth login<\/code> in current versions of the Hugging Face CLI, <code>huggingface-cli login<\/code> in older ones) or set <code>HF_TOKEN<\/code>.<\/td>\n<\/tr>\n<tr>\n<td>Long pause before the server accepts requests<\/td>\n<td>Normal: CUDA graph capture and compilation. Use <code>--enforce-eager<\/code> to skip it during debugging.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2><span class=\"ez-toc-section\" id=\"Frequently_asked_questions\"><\/span>Frequently asked questions<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3>Can I install vLLM natively on Windows?<\/h3>\n<p>No. vLLM publishes Linux wheels only, and <code>pip install vllm<\/code> in Windows Python will not give you a working GPU server. Use WSL2 with an Ubuntu distribution, or run the official Docker image. Both are fully supported and give near-native performance on the same hardware.<\/p>\n<h3>Do I need to install the CUDA toolkit first?<\/h3>\n<p>Not for the default wheel. It bundles the CUDA runtime through its pinned PyTorch build, so a reasonably current NVIDIA driver is enough. You only need a full toolkit with <code>nvcc<\/code> if you compile vLLM from source or build custom kernels.<\/p>\n<h3>How do I install a specific vLLM version or the nightly build?<\/h3>\n<p>Pin it like any package: <code>pip install vllm==&lt;version&gt;<\/code>, choosing from the versions listed on PyPI. Nightly and per-commit wheels are published separately by the project and installed with an extra index URL \u2014 the current address is documented on the installation page, and it has changed before, so read it there rather than copying an old command.<\/p>\n<h3>Why does vLLM take up my whole GPU?<\/h3>\n<p>By design. It preallocates a large KV cache block pool at startup \u2014 controlled by <code>--gpu-memory-utilization<\/code>, which defaults to 0.9 \u2014 because paged attention is what makes its throughput high under concurrency. Lower the value if you need to share the card, and expect fewer concurrent requests as a result.<\/p>\n<h3>Should I use vLLM or Ollama?<\/h3>\n<p>Ollama is a single cross-platform installer aimed at one user on one machine; see the <a href=\"https:\/\/convly.ai\/how-to-install-ollama-2026\/\">Ollama install guide<\/a> if that describes you. vLLM is a serving engine built for many concurrent requests per GPU, with continuous batching, tensor parallelism and an OpenAI-compatible API. Install vLLM when you are serving an application, not when you are chatting locally.<\/p>\n<h3>Which model should I serve first?<\/h3>\n<p>Start with something small and ungated so you are debugging the install rather than the download \u2014 an 8B-class model at around 5 GB in 4-bit is ideal. Once the server responds to <code>\/v1\/models<\/code>, move up. The <a href=\"https:\/\/convly.ai\/llm-leaderboard\/\">LLM leaderboard<\/a> is a reasonable way to shortlist candidates by capability, price and context length before you commit VRAM to one.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Linux + NVIDIA GPU: create a clean Python 3.12 environment, run pip install vllm (or uv pip install vllm), then [\u2026]<\/p>\n","protected":false},"author":1,"featured_media":2548,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[7],"tags":[],"class_list":["post-2547","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-news"],"_links":{"self":[{"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/posts\/2547","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/comments?post=2547"}],"version-history":[{"count":1,"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/posts\/2547\/revisions"}],"predecessor-version":[{"id":2549,"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/posts\/2547\/revisions\/2549"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/media\/2548"}],"wp:attachment":[{"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/media?parent=2547"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/categories?post=2547"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/convly.ai\/pt\/wp-json\/wp\/v2\/tags?post=2547"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}