A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

🧑‍🚀 全世界最好的LLM资料总结 | Summary of the world's best LLM resources.

FCC China open source codebase and curriculum. Learn to code and help nonprofits.

freeCodeCamp.org's open-source codebase and curriculum. Learn to code for free.

brpc is an Industrial-grade RPC framework using C++ Language, which is often used in high performance system such as Search, Storage, Machine learning, Advertisement, Recommendation etc. "brpc" mea…

A curated list of awesome Machine Learning frameworks, libraries and software.

LLM serving cluster simulator

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

模型压缩的小白入门教程

《开源大模型食用指南》基于Linux环境快速部署开源大模型，更适合中国宝宝的部署教程

cube studio开源云原生一站式机器学习/深度学习/大模型AI平台，支持sso登录，多租户，大数据平台对接，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU，边缘计算，serverless，标注平台，自动化标注，数据集管理，大模型微调，vllm大模型推理，llmops，私有知识库，AI模型应用商店，支持模型一键开发/推理/微调，…

Collaborative Training of Large Language Models in an Efficient Way

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

LLM training code for Databricks foundation models

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

DeepSeek Coder: Let the Code Write Itself

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

LightSeq: A High Performance Library for Sequence Processing and Generation

Best practice for training LLaMA models in Megatron-LM

GPT-Fathom is an open-source and reproducible LLM evaluation suite, benchmarking 10+ leading open-source and closed-source LLMs as well as OpenAI's earlier models on 20+ curated benchmarks under al…

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents

Large Language Model Text Generation Inference

Official repository for Spyder - The Scientific Python Development Environment

Hackable and optimized Transformers building blocks, supporting a composable construction.

A comparison of pretraining framework for LLM

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks