How to Build an LLM Agent in Python: The Tool-Calling Loop Explained

Build a minimal LLM agent in Python: tool/function calling, memory, and the agent loop — plus when to switch to a framework. Grounded in 2026 sources.

An LLM agent is a loop, not a magic box: the model decides which tool to call, your code runs the tool, and the result goes back to the model until the task is done. The collected sources for this guide — a Medium walkthrough by Katsiaryna Ruksha, Real Python’s LLM learning path, the Langroid framework, and an OpenDataScience open-source guide — all describe the same core pieces. This article walks through those pieces in Python so you can build a minimal agent and understand what the frameworks do for you.

What an LLM agent actually is

Across the collected sources, an agent has three recurring components:

• A model with function/tool calling — the LLM is given tool schemas and decides when to call them.
• Tools — plain functions (search, calculator, database query) the agent can invoke.
• Memory — conversation state and, optionally, retrieved context (RAG) passed back into each call.

The Medium source frames the agent as a loop around these three: the model proposes an action, your code executes it, and the output is fed back. That loop is the whole idea.

Prerequisites

• Python 3.10+ and a virtual environment.
• An LLM API key (the sources use OpenAI; the same pattern applies to other providers’ tool-calling APIs).
• The provider SDK installed (for example, pip install openai).

If you want to avoid API costs, the OpenDataScience source shows the identical loop running against open-source models through Ollama and Hugging Face.

The agent loop, step by step

1. Define tools as functions and describe each with a JSON schema (name, description, parameters). The description is what the model uses to decide when to call it, so write it clearly.
2. Send the user message plus the tool schemas to the model.
3. Inspect the response. If the model returned a tool call, parse the arguments and run the matching Python function.
4. Append the tool result to the message history and call the model again.
5. Repeat until the model responds with a final answer instead of a tool call.

That five-step loop is the same one the Medium and Real Python sources build by hand, and the same one frameworks wrap. Keeping a turn limit on the loop prevents runaway tool-calling.

When to use a framework instead

Building the loop yourself is the best way to understand it, but the sources point to frameworks once you need more. The Langroid source positions itself as a lightweight, multi-agent Python framework for exactly this; Real Python’s path adds retrieval-augmented generation and MCP connectivity on top of the basic loop. Reach for a framework when you need persistent memory, multiple cooperating agents, or standardized tool access through MCP — and stay hand-rolled while you are still learning the mechanics.

Verify it works

A working agent should: call the right tool for a prompt that needs one, answer directly when no tool is needed, and stop cleanly at your turn limit. Test each tool in isolation first, then test the loop with one tool, then add more. If the model never calls a tool, the usual cause is a vague tool description — tighten it and retry.

Note: APIs, SDK versions, and model tool-calling formats change. Confirm the current syntax in your provider’s official docs before building.