gitpulse

llm-serving

5 kho được công bố.

Kho
1
vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
97k +0.0%+0.4%+3.4% 71
2
ray-project/ray Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
44.3k +0.0%+0.1%+0.6% 12
3
liguodongiot/llm-action 本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)
25.3k +0.0%+0.1%+0.5% 20
4
NVIDIA/TensorRT-LLM TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
15.1k +0.0%+0.3%+1.8% 13
5
bojieli/ai-infra-book 《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
7.8k +2.1%+5.5%+20.3% 92