How to Build a Local AI Agent That Calls Python Tools (No API Needed)

Most AI content shows you how to ask better questions.
I wanted to build something that answers by doing.

Here’s a minimal AI agent that runs locally, reasons, and calls external tools — no APIs, no cloud.

A tool-calling agent using:

• granite3-dense:2b by IBM (open-source, runs on CPU)

• LangChain to wire everything

• Ollama for local inference

The agent can:

• Decide when a tool is needed

• Call a Python function (like getting the current time)

• Return the result as part of the final answer

You’re not just sending prompts.
You’re running logic. Calling functions. Building systems that think and act.

This is how AI becomes a backend, not just an interface.

Tools are regular Python functions. The agent chooses whether to call them based on the prompt and user input.

python from langchain.agents import tool
import datetime
@tool
def get_current_datetime(format: str = "%Y-%m-%d %H:%M:%S") -> str:
    """
    Returns the current date and time, formatted according to the provided Python strftime format string.
    Use this tool whenever the user asks for the current date, time, or both.
    Example format strings: '%Y-%m-%d' for date, '%H:%M:%S' for time.
    If no format is specified, defaults to '%Y-%m-%d %H:%M:%S'.
    """
    try:
        return datetime.datetime.now().strftime(format)
    except Exception as e:
        return f"Error formatting date/time: {e}"

I used granite3-dense:2b, served locally via Ollama.

python
def get_agent_llm(model_name=MODEL, temperature=0):
    """Initializes the ChatOllama model for the agent."""
    llm = ChatOllama(
        model=model_name,
        temperature=temperature
    )
    return llm

Make sure the Ollama server is running (ollama serve) and the model is pulled (ollama pull granite3-dense).

I skipped LangChain Hub and wrote a ReAct-style prompt from scratch.
It gives me full control over the reasoning flow and tool usage.

python

def get_agent_prompt():
    """Custom ReAct-style prompt with required agent_scratchpad variable."""
    system_msg = SystemMessagePromptTemplate.from_template(
    "You are an AI agent that can use external tools to answer user  questions.\n\n"
    "Available tool:\n"
    "- `get_current_datetime`: Returns the current date or time in a specified format. Use only when the user explicitly asks for the current date or time.\n\n"
    "Only use the tools listed above. If no tool is needed, respond immediately after your Thought with a Final Answer. Do not invent tools or actions.\n\n"
    "Follow this exact format:\n"
    "Thought: your reasoning\n"
    "Action: the action to take\n"
    "Action Input: the input to the action\n"
    "Observation: the result of the action\n"
    "... (repeat as needed)\n"
    "Final Answer: the final response to the user"
    )
    human_msg = HumanMessagePromptTemplate.from_template("{input}\n\n{agent_scratchpad}")
    return ChatPromptTemplate.from_messages([system_msg, human_msg])

Now we combine the model, tools, and prompt into a runnable agent.

python
from langchain.agents import create_tool_calling_agent
def build_agent(llm, tools, prompt):
    """Builds the tool-calling agent runnable."""
    agent = create_tool_calling_agent(llm, tools, prompt)
    print("Agent runnable created.")
    return agent

This component manages the interaction loop: the agent receives input, decides whether to call a tool, and returns an answer.

python
from langchain.agents import AgentExecutor
def create_agent_executor(agent, tools):
    """Creates the agent executor."""
    agent_executor = AgentExecutor(
        agent=agent,
        tools=tools,
        verbose=True # Set to True to see agent thoughts and tool calls
    )
    print("Agent executor created.")
    return agent_executor

Let’s run the agent with a few prompts.

python
def run_agent(executor, user_input):
    """Runs the agent executor with the given input."""
    response = executor.invoke({"input": user_input})

#Main
  run_agent(executor, "What is the current date?")
  run_agent(executor, "Give me the time in HH:MM format")
  run_agent(executor, "Tell me a joke")  # should skip tool use

Output

Full code’s on GitHub: agent-ollama-basic
It’s minimal, modular, and ready to extend — APIs, logic, full workflows.

I’ll extend this agent to call real APIs, chain multiple tools, and integrate with MCPs.
But even this tiny POC shows what’s possible when you move beyond “chat”.

AI that acts > AI that chats.

Until then, clone the repo. See what your agent can actually do.

Nina