8 How It Works
To use vibe coding properly, you need at least a rough sense of what’s running underneath it. This chapter explains it without math, but accurately: the basic structure of next-token prediction, the anatomy of a coding agent, and why this approach has worked best specifically for code.
8.1 From Next-Token Prediction to Code
Pre-draft. Check TOPICS.md at the repository root for owner and status.
Brief. The basic behavior of an LLM is nothing more than predicting the next token. This section’s task is to explain precisely, without formulas, how the ability to write code emerges from this simple mechanism, and why code is a particularly good fit for this mechanism compared to natural language. It examines the role played by each of the following: the syntactic regularity of code, the vast training data available from public repositories, and the signal provided by executability. At the same time, it addresses where this mechanism breaks down, that is, what form hallucination takes in code generation. The actual boundary between the claim that “the model understands” and the claim that “it merely mimics patterns” forms the outline for this entire chapter (Chapter 8).
Seed Questions
- Through what specific pathway does the ability to write code emerge from the simple training objective of next-token prediction?
- Why is code generation more favorable for training than natural language generation? Which contributes more: syntactic regularity or data scale?
- What specific forms does hallucination take in code generation? Do fabricating a nonexistent API and using the wrong argument share the same cause?
- What grounds does each of the claims “the model understands code” and “it merely mimics patterns” rest on?
- How has the expansion of context window size translated into performance improvements in actual coding tasks?
Research Pointers
- Reread Karpathy’s original “Software 2.0” (2017) as the origin of the perspective that treats code as data.
- Select and cite popular yet accurate explanatory material on next-token prediction and the transformer architecture.
- Prioritize searching for academic papers addressing hallucination in code generation (empirical research is the basic stance of Part 3 as a whole).
I’ll translate this Quarto markdown file from Korean to English, preserving the structure exactly as specified.
8.2 Anatomy of a Coding Agent
Pre-draft. Check the repository root’s TOPICS.md for owner and status.
Brief. Break down what a coding agent actually does in a single turn. Trace, step by step, how user input is assembled into a prompt, how tool calls (reading and writing files, executing commands) proceed, how the feedback loop by which execution results flow back into context works, and how many times this loop repeats before the task ends. The core work is to use Anthropic’s “Building effective agents” and its distinction between workflows and agents as the theoretical axis, and to cross-check it against the actual source code of open-source coding agents to identify the gap between theory and implementation.
Seed Questions
- If you break down a single turn of a coding agent from user input to final response, step by step, what do you get?
- What does tool use mean on the model side, and what does the harness side need to prepare for it to work?
- Where is the boundary between the “workflow” and “agent” that Anthropic distinguishes, and which are actual coding agents closer to?
- In the source code of open-source coding agents, how is the main loop actually implemented, and does it match the documented explanation?
- What changes if there is no feedback loop by which execution results (test failures, error messages, shell output) flow back into the model’s input?
Research Pointers
- Read closely Anthropic’s “Building effective agents” (2024-12) and use it as this chapter’s theoretical axis
- Read the main loop code directly from the GitHub repositories of several open-source coding agents (CLI-type tools)
- Find materials related to harness engineering and cross-reference them with §3.4 and the agent harness entry in the glossary
8.3 Verification Asymmetry: Why Code, of All Things
Pre-draft. See TOPICS.md at the repository root for ownership and status.
Brief. One reason code became the first major domain of AI generation is verifiability. When you run code, you can immediately tell, at least in part, whether it is right or wrong, but this is not true of prose or images. This section covers why this asymmetry arises, what concrete role automated verification signals such as tests, compilation, and type checks actually play in model training and agent loop design, and what conditions would be needed for this verification advantage to generalize to writing or other creative domains. This section forms a counterpart to the “artifacts that are hard to verify” discussion in §6.1 (Writing).
Seed Questions
- What exactly does the claim that “code is verified by running it” verify, and what does it fail to verify?
- How are automated verification signals (passing tests, successful compilation, type checks) concretely reflected in agent loop design?
- What empirical research supports or refutes the verification asymmetry hypothesis?
- What conditions (formalizability, the existence of gradeable criteria) would be needed for this verification advantage to generalize to other outputs such as writing, images, or data analysis?
- What long-term bias does automation progressing preferentially on verifiable tasks create?
Research Pointers
- Find and review empirical research on learning based on verifiable rewards.
- Cross-reference with §5.4 (Creating Verifiable Units) and §6.1 (Writing) to divide roles without overlap.
- Identify where software testing theory intersects with the AI evaluation (evals) literature.