llm-serving
5 kho được công bố.
| Kho | |||||
|---|---|---|---|---|---|
| 1 |
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
|
97k | +0.0% | +0.4% | 71 |
| 2 |
ray-project/ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
|
44.3k | +0.0% | +0.1% | 12 |
| 3 |
liguodongiot/llm-action
本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)
|
25.3k | +0.0% | +0.1% | 20 |
| 4 |
NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
|
15.1k | +0.0% | +0.3% | 13 |
| 5 |
bojieli/ai-infra-book
《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
|
7.8k | +2.1% | +5.5% | 92 |