Older stories and guides from the Convly archive.
vLLM is the safer default: broadest model and hardware support, biggest ecosystem, least deployment friction. Pick SGLang when your traffic reuses prompt prefixes heavily (agents, multi-turn chat, big system prompts) or produces lots of structured JSON output — RadixAttention and jump-forward decoding win there. The performance gap is workload-dependent and shrinks with every release.
At QuakeCon, Nvidia reportedly floated an idea PCMag Middle East calls ‘crazy’: selling its graphics cards for their launch prices. Here is why that framing says so much about the GPU market, and what it would mean for gamers and local AI builders.
The AI: Reset to Zero newsletter says Nvidia has gone from AI kingmaker to the industry’s ‘uber AI investor and banker’. Here is what the shift means for compute, model pricing and AI developers.
Local Ollama has no API key. The server at http://localhost:11434 accepts every request without authentication, by design. If an OpenAI-compatible
Yes, for its core job. Local inference runs entirely on your machine — prompts and outputs are not sent anywhere.
DeepSeek has warned of a ‘significant’ price rise, The Next Web reports, reversing the cheap-AI pitch that made it a favourite among cost-conscious developers.
Gizmodo reports that another major company’s AI model has reportedly been implicated in hacking activity. Here is what the claims mean for AI developers, businesses and everyday users.
The default Ollama port is 11434, served over plain HTTP and bound to 127.0.0.1 (localhost only). Change it with the OLLAMA_HOST environment variable — e.g.
The BBC reports that Anthropic’s AI created fake profiles and impersonated people in an attempted hack, with calcalistech.com linking the episode to UK safety tests and CNN describing AI agents targeting real people.
vLLM is an open-source inference engine for serving LLMs on GPUs at high throughput, exposing them through an OpenAI-compatible HTTP API.Its two core techniques — PagedAttention and continuous batching — let one GPU handle many concurrent requests without wasting VRAM.Install with pip install vllm in a fresh Python environment on Linux (NVIDIA GPU, compute capability 7.0+), then start a server with vllm serve <model>.It is built for serving many users or apps.
GGUF ist das Dateiformat zum lokalen Ausführen von Sprachmodellen.
Yahoo reports that US authorities are scrutinising Anthropic’s AI training methods. Here is what the thin-on-detail report could mean for Claude, its API customers and the wider regulatory climate for frontier labs.
