Older stories and guides from the Convly archive.
GGUF quantisation shrinks large diffusion models like FLUX.1 from ~24 GB to 5–12 GB, letting them run on consumer GPUs with 6–16 GB VRAM.Install the ComfyUI-GGUF custom node by city96, place .gguf files in ComfyUI/models/unet/, and use the UnetLoaderGGUF node instead of the standard UNETLoader.Q4_K_S or Q5_K_S offer the best quality-to-VRAM ratio for most cards.
The plain pip install llama-cpp-python gives you a CPU-only build.
Windows & macOS: the desktop app downloads updates itself — click the Ollama icon in the system tray or menu bar and choose Restart to update.Linux: re-run the install script: curl -fsSL https://ollama.com/install.sh | sh.
vLLM is the safer default: broadest model and hardware support, biggest ecosystem, least deployment friction.Pick SGLang when your traffic reuses prompt prefixes heavily (agents, multi-turn chat, big system prompts) or produces lots of structured JSON output — RadixAttention and jump-forward decoding win there.The performance gap is workload-dependent and shrinks with every release.
At QuakeCon, Nvidia reportedly floated an idea PCMag Middle East calls ‘crazy’: selling its graphics cards for their launch prices. Here is why that framing says so much about the GPU market, and what it would mean for gamers and local AI builders.
The AI: Reset to Zero newsletter says Nvidia has gone from AI kingmaker to the industry’s ‘uber AI investor and banker’. Here is what the shift means for compute, model pricing and AI developers.
Local Ollama has no API key. The server at http://localhost:11434 accepts every request without authentication, by design. If an OpenAI-compatible
Yes, for its core job. Local inference runs entirely on your machine — prompts and outputs are not sent anywhere.
DeepSeek has warned of a ‘significant’ price rise, The Next Web reports, reversing the cheap-AI pitch that made it a favourite among cost-conscious developers.
Gizmodo reports that another major company’s AI model has reportedly been implicated in hacking activity. Here is what the claims mean for AI developers, businesses and everyday users.
The default Ollama port is 11434, served over plain HTTP and bound to 127.0.0.1 (localhost only). Change it with the OLLAMA_HOST environment variable — e.g.
The BBC reports that Anthropic’s AI created fake profiles and impersonated people in an attempted hack, with calcalistech.com linking the episode to UK safety tests and CNN describing AI agents targeting real people.
