rasbt/LLMs-from-scratch

What it does

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Summary based on the GitHub description (opens in a new tab)

What the README covers

• Build a Large Language Model (From Scratch)
• Table of Contents
• Prerequisites
• Hardware Requirements
• Video Course
• Companion Book / Sequel
• Exercises
• Bonus Material
• Questions, Feedback, and Contributing to This Repository
• Citation

Getting started

Excerpt from the README · Checked 15 Oct 2026

git clone --depth 1 https://github.com/rasbt/LLMs-from-scratch.git

Shown exactly as published. Top 200 Repos has not run or tested it. View in README (opens in a new tab)

Ranking trend

Global rank in each recorded snapshot. A lower number is a higher rank.

From #118 to #124 across 2 snapshots since 15 Oct 2026.

What people build with it

Related repositories

• huggingface/transformers#52

  🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

  Shared topics: natural-language-processing, pytorch, deep-learning, machine-learning

• microsoft/generative-ai-for-beginners#95

  21 Lessons, Get Started Building with Generative AI

  Shared topics: language-model, transformers, generative-ai, gpt

• f/prompts.chat#49

  f.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source — self-host for your organization with complete privacy.

  Shared topics: artificial-intelligence, gpt, machine-learning, llm

• Significant-Gravitas/AutoGPT#43

  AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

  Shared topics: artificial-intelligence, gpt, llm, ai

• langchain-ai/langchain#64

  The agent engineering platform.

  Shared topics: generative-ai, llm, ai, python

• mlabonne/llm-course#198

  Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

  Shared topics: large-language-models, machine-learning, llm