Open sourceAIIntermediate
vLLM
An LLM inference engine — the place to understand KV caching and continuous batching.
vLLMAbout 2 hrpythoncuda
Browse by topic
推理部署
18 items
Open sourceAIBeginnerReady to start
[Bug]: RankLLMRerank cannot be imported without vllm installed, regardless of reranking backend
RankLLMRerank cannot be imported, unless vllm is installed. This is true even when using a backend that has nothing to do with vLLM. Reason: llama_index.postprocessor.rankllm_rerank.base imports PromptMode and Reranker from rank_llm.rerank at module level: In the current release 0.25.7, importing anything from rank_ll…
run-llama/llama_indexAbout 4 hr推理部署后端
Open sourceDataBeginnerReady to start
Sensitive data from extra fields is visible in Edit Connection window
3.0.3 After upgrading Airflow from version 3.0.2 to 3.0.3 sensitive data in extra fields and extra fields JSON is visible in edit connection page. Sensitive data should be masked with Enter Edit Connection window in Airflow UI. Ubuntu 24.04 Other Docker-based deployment Airflow deployed with Docker Swarm. Settings in…
apache/airflowAbout 4 hr推理部署数据工程
Open sourceAIBeginnerReady to start
[Feature]: Tracking Whisper feature requests
This issue is for keeping track of the recurrent Whisper asks as well as the linked on-going efforts to support that feature, if any. When a feature request has no linked PR, feel free to claim the work here if you want to help! - Related issues: https://github.com/vllm-project/vllm/issues/19556, https://github.com/vl…
vllm-project/vllmAbout 4 hr语音推理部署
AI updateAI
Better prompt caching for GPT-6
Learn how GPT-6 improves prompt caching with higher cache hit rates, new diagnostics, explicit breakpoints, and controls that reduce latency and costs.
OpenAIPublished Sep 23, 2026
Official source · OpenAI NewsVerified Oct 7, 2026
AI updateAI
Offloaded inference for real-world physical AI robotics
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared…
Microsoft ResearchPublished Sep 24, 2026
Official source · Microsoft ResearchVerified Oct 10, 2026
AI updateAI
EvoLib: Turning experience into evolving knowledge
LLMs do not get smarter just by remembering more. EvoLib turns experience into evolving knowledge, taking reusable skills and insights that help models learn and adapt across tasks long after deployment. The post EvoLib: Turning experience into evolving knowledge appeared first on Microsoft Research .
Microsoft ResearchPublished Jul 31, 2026
Official source · Microsoft ResearchVerified Sep 29, 2026
AI updateAI
Rapidly scaling online storage to serve over 1 billion ChatGPT users
Learn how OpenAI evolved Habitat from a Python library into a globally distributed storage platform serving 1 billion ChatGPT users and 22M requests per second.
OpenAIPublished Sep 11, 2026
Official source · OpenAI NewsVerified Sep 24, 2026
AI updateAI
Safety overview: GPT-6 Astra
GPT-6 Astra is our most capable broadly deployed model and our first to reach the Critical level of cybersecurity capability under our Preparedness Framework.
OpenAIPublished Sep 3, 2026
Official source · OpenAI NewsVerified Sep 17, 2026
AI updateAI
Inside our 353,000-person vibe coding course
Kaggle’s AI Agents Intensive with Google brought learners together in a no-cost course to build and deploy the next frontier of AI.
GooglePublished Aug 3, 2026
Official source · Google AIVerified Aug 28, 2026
AI updateAI
Jalapeño’s first results show industry-leading speed and efficiency in AI inference
Jalapeño is a custom inference chip from OpenAI that delivers faster, more power-efficient AI inference, with higher throughput and lower latency for modern models.
OpenAIPublished Aug 25, 2026
Official source · OpenAI NewsVerified Sep 9, 2026
AI updateAI
Up to 3.2x Faster Inference with LFM2.5-DSpark
How does DSpark work The decode phase in LLM inference is traditionally memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation. Speculative decoding addresses this by using a lightweight draft model to produce candidate tokens, then having the target model verify them…
Hugging FacePublished Aug 21, 2026
Cross-checked · Hugging Face BlogVerified Sep 9, 2026
AI updateAI
Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS
By the time a user hears your application respond, you've already spent precious milliseconds capturing audio, transcribing speech, running an LLM, retrieving context, and generating a response. Text-to-speech (TTS) is the final step — and the one users notice most. If speech generation is slow, the whole experience f…
Hugging FacePublished Aug 11, 2026
Cross-checked · Hugging Face BlogVerified Sep 9, 2026
AI updateAI
Record, train, and deploy from one place with Strands Agents, LeRobot, and Hugging Face Storage Buckets
You have an agent that can already record a demonstration and push it to the Hugging Face Hub . Now you want to run that loop continuously: collect episodes through the day, train a policy on the growing dataset, deploy it, and pull the next batch back to improve it. Run that loop once and every piece works. Run it ev…
Hugging FacePublished Aug 14, 2026
Cross-checked · Hugging Face BlogVerified Sep 9, 2026
AI updateAI
Wire It, Run It, Deploy It: AI Workflows in Gradio
gr.Workflow , built right into Gradio, makes the pipeline the interface . You describe your steps as a graph of typed nodes, and Gradio serves a drag-and-drop canvas where every node is runnable and every intermediate result is visible. The same graph is also a REST API and a one-command deploy to HF Mirror Spaces.
Hugging FacePublished Aug 25, 2026
Cross-checked · Hugging Face BlogVerified Sep 9, 2026
AI updateAI
Baseten on Hugging Face Inference Providers 🔥
We're thrilled to share that Baseten is now a supported Inference Provider on the HF Mirror Hub!
Hugging FacePublished Aug 6, 2026
Cross-checked · Hugging Face BlogVerified Sep 9, 2026
AI updateAI
Bringing Nunchaku 4-bit Diffusion Inference to Diffusers
Most of these backends are weight-only . This means that they store the weights in low precision and dequantize them back to high precision at compute time. This reduces memory usage significantly, but it usually does not make inference faster, and can even add a small latency overhead.
Hugging FacePublished Jul 23, 2026
Cross-checked · Hugging Face BlogVerified Sep 4, 2026
AI updateAI
How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code
Of course, making AI research accessible requires a powerful search engine, so that humans and agents can quickly find relevant and related work, either through the website or the pwc search CLI command , which agents can use via the Skill .
Hugging FacePublished Aug 21, 2026
Cross-checked · Hugging Face BlogVerified Sep 9, 2026