Pinned Loading

1. ModelCloud/GPTQModel ModelCloud/GPTQModel Public

   LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

   Python 1k 163

2. ModelCloud/Tokenicer ModelCloud/Tokenicer Public

   A (nicer) tokenizer you want to use for model inference and training: with all known peventable gotchas normalized or auto-fixed.

   Python 11 4

3. ModelCloud/Device-SMI ModelCloud/Device-SMI Public

   Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it y...

   Python 15 1

4. sgl-project/sglang sgl-project/sglang Public

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python 24.1k 4.6k

5. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 71.9k 13.9k

6. ModelCloud/LogBar ModelCloud/LogBar Public

   A unified Logger and ProgressBar util with zero dependencies.

   Python 8