08 — Dynamic Tool Ecosystem: How to Add ANY Tool
🎯 What This Chapter Covers
- How tools are registered in smolagents
- How to add new tools without retraining the model
- The tool marketplace concept (1000+ MCP servers)
- How the agent discovers and uses new tools automatically
- Architecture for a "tool marketplace" in our agent harness
🧩 The Core Principle: Pattern Over Specifics
The #1 insight from our research: Our 1.7B model doesn't need to know about SPECIFIC tools. It needs to know the PATTERN of using tools.
Think of it like this:
- Bad approach: Train model on "how to use Tool A, Tool B, Tool C..."
- Good approach: Train model on "how to write Python code that solves problems"
The model already knows Python (Qwen3 was trained on code). We just need to teach it to:
- Break problems into steps
- Use available Python libraries/functions
- Handle errors and try alternatives
Result: You can add ANY new tool (any Python function) and the model will figure out how to use it.
🔧 How Tool Registration Works in smolagents
The Simple Way: @tool Decorator
from smolagents import tool
@tool
def my_awesome_tool(input_param: str) -> str:
"""
What this tool does (this becomes the "instruction manual" for the LLM).
Args:
input_param: What this parameter means
"""
# Your code here
result = do_something(input_param)
return result
That's it. The @tool decorator automatically:
- Reads the function name → becomes the tool name
- Reads the docstring → becomes the tool description (shown to the LLM)
- Reads type hints → becomes the parameter schema
- Registers it in the agent's "toolbox"
Example: Adding a Weather Tool
from smolagents import tool
import requests
@tool
def get_weather(city: str, country_code: str = "") -> str:
"""
Get current weather for a city. Returns temperature, conditions, and forecast.
Args:
city: The city name (e.g., "London", "New York")
country_code: Optional 2-letter country code (e.g., "US", "GB")
"""
url = f"https://api.openweathermap.org/data/2.5/weather"
params = {
"q": f"{city},{country_code}" if country_code else city,
"appid": "YOUR_API_KEY",
"units": "metric"
}
response = requests.get(url, params=params)
data = response.json()
temp = data["main"]["temp"]
conditions = data["weather"][0]["description"]
humidity = data["main"]["humidity"]
return f"Weather in {city}: {temp}°C, {conditions}, humidity {humidity}%"
What the LLM sees:
You have access to the following tools:
- get_weather(city: str, country_code: str = "")
Get current weather for a city. Returns temperature, conditions, and forecast.
Args:
city: The city name (e.g., "London", "New York")
country_code: Optional 2-letter country code (e.g., "US", "GB")
The LLM learns to use it from the description alone. No training needed!
📦 The "Tool Marketplace" Concept
How It Works
┌─────────────────────────────────────────────┐
│ Tool Marketplace │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Weather │ │ Finance │ │ Social │ │
│ │ Tool │ │ Tool │ │ Media │ │
│ │ (Free) │ │ (Free) │ │ (Free) │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Browser │ │ GitHub │ │ Image │ │
│ │ Tool │ │ Tool │ │ Gen Tool │ │
│ │ (Built) │ │ (Built) │ │ (Built) │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Database │ │ Email │ │ Calendar │ │
│ │ Tool │ │ Tool │ │ Tool │ │
│ │ (Built) │ │ (Built) │ │ (Built) │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ └──────┬──────┴──────┬────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────────┐ │
│ │ Agent Tool Loader │ │
│ │ (User picks which │ │
│ │ tools to enable) │ │
│ └────────────┬────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────┐ │
│ │ CodeAgent with Tools │ │
│ │ (Model sees all enabled│ │
│ │ tool descriptions) │ │
│ └─────────────────────────┘ │
└─────────────────────────────────────────────┘
Adding a Tool Is Just Installing a Package
# Install weather tool
pip install some-weather-library
# Add to agent config
# (The tool is auto-registered via @tool decorator)
Or for MCP servers:
# Install MCP server
npm install -g @some-org/mcp-weather
# Register in agent
# Agent discovers tools from the MCP server's tool definitions
🛠️ Building Our Tool Ecosystem
Phase 1: Core Tools (Built Into Agent)
These are always available — the foundation:
# core_tools.py
from smolagents import tool
import os, subprocess, json
@tool
def read_file(file_path: str, max_chars: int = 10000) -> str:
"""Read contents of a file."""
with open(file_path, 'r') as f:
return f.read()[:max_chars]
@tool
def write_file(file_path: str, content: str) -> str:
"""Write content to a file."""
os.makedirs(os.path.dirname(file_path) or '.', exist_ok=True)
with open(file_path, 'w') as f:
f.write(content)
return f"Written {len(content)} chars to {file_path}"
@tool
def list_directory(path: str = '.') -> str:
"""List files and folders in a directory."""
entries = os.listdir(path)
return "\n".join(sorted(entries))
@tool
def execute_shell(command: str) -> str:
"""Execute a shell command safely."""
result = subprocess.run(command, shell=True, capture_output=True, text=True, timeout=30)
return result.stdout + result.stderr
Phase 2: Web Tools (Internet Access)
# web_tools.py
from smolagents import tool
import requests
from bs4 import BeautifulSoup
@tool
def fetch_webpage(url: str) -> str:
"""Fetch and extract text content from a webpage."""
response = requests.get(url, timeout=30)
soup = BeautifulSoup(response.text, 'html.parser')
# Remove scripts and styles
for script in soup(["script", "style"]):
script.decompose()
return soup.get_text()[:10000]
@tool
def web_search(query: str, num_results: int = 5) -> str:
"""Search the web using DuckDuckGo."""
# Using DuckDuckGo's HTML interface
response = requests.get(f"https://html.duckduckgo.com/html/?q={query}")
# Parse results...
return formatted_results
Phase 3: Analysis Tools (Data Processing)
# analysis_tools.py
from smolagents import tool
import pandas as pd
import matplotlib.pyplot as plt
@tool
def analyze_csv(file_path: str, query: str) -> str:
"""Load a CSV file and answer questions about it using pandas."""
df = pd.read_csv(file_path)
# Agent writes code to analyze
# Could generate summary stats, charts, etc.
return str(df.describe())
@tool
def create_chart(data_source: str, chart_type: str, output_path: str) -> str:
"""Create a chart from data."""
# chart_type: "bar", "line", "pie", "scatter"
# Agent writes matplotlib code
# Saves to output_path
return output_path
Phase 4: Creative Tools (Generation)
# creative_tools.py
from smolagents import tool
from diffusers import StableDiffusionPipeline
import torch
# Load model once at startup
pipe = StableDiffusionPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5",
torch_dtype=torch.float16
).to("cuda")
@tool
def generate_image(prompt: str, output_path: str = "generated.png") -> str:
"""Generate an image from a text description."""
image = pipe(prompt, num_inference_steps=20).images[0]
image.save(output_path)
return output_path
@tool
def generate_code(language: str, task: str, output_path: str) -> str:
"""Generate code for a specific task."""
# Uses the LLM itself to generate code
# Then saves to file
return output_path
🧠 How the Agent Uses Tools It Never Saw Before
Example: User Adds a "Crypto Price" Tool
Step 1: User installs the tool
pip install crypto-price-library
Step 2: User writes the tool wrapper
from smolagents import tool
import crypto_price
@tool
def get_crypto_price(symbol: str, currency: str = "USD") -> str:
"""
Get the current price of a cryptocurrency.
Args:
symbol: The cryptocurrency symbol (e.g., "BTC", "ETH", "SOL")
currency: The currency to convert to (default: "USD")
"""
price = crypto_price.get_current(symbol, currency)
return f"{symbol}: ${price} {currency}"
Step 3: Register with agent
from smolagents import CodeAgent
from my_tools import get_crypto_price, read_file, write_file
agent = CodeAgent(
model=my_model,
tools=[get_crypto_price, read_file, write_file],
)
Step 4: The agent automatically learns
User: "What's the price of Bitcoin and should I invest?"
Agent's system prompt now includes:
"- get_crypto_price(symbol: str, currency: str = 'USD')
Get the current price of a cryptocurrency."
Agent thinks (via CodeAgent pattern):
"User wants Bitcoin price. I have get_crypto_price tool.
I'll call it with symbol='BTC'."
Agent generates Python code:
```python
btc_price = get_crypto_price("BTC", "USD")
print(btc_price)
Result: "BTC: $67,420 USD"
Agent then might: "User also asked if they should invest. I should do more research. Let me search for recent Bitcoin news and analysis."
Agent generates:
news = web_search("Bitcoin investment analysis 2025")
print(news)
Agent synthesizes final answer: "Bitcoin is currently $67,420. Recent analysis suggests... [summary of research]"
**No retraining needed.** The model learns to use the tool from its name, description, and parameter hints.
---
## 🌐 The MCP Server Universe
### What Are MCP Servers?
MCP (Model Context Protocol) servers are **pre-built tool packages** that expose tools in a standard format. Think of them like "apps" for your agent.
There are **1000+ MCP servers** covering every domain:
| Category | Example Servers | What They Do |
|----------|----------------|--------------|
| **Web** | firecrawl, browser-use, playwright | Web scraping, browsing |
| **Code** | github, git, code-index | Repo analysis, code search |
| **Data** | postgres, sqlite, duckdb | Database queries |
| **Memory** | chroma, mem0 | Long-term memory |
| **Comm** | slack, gmail, discord | Messaging |
| **Dev** | kubernetes, docker, aws | Infrastructure |
| **Creative** | comfyui, image-gen | Image/video generation |
| **Research** | perplexity, arxiv | Academic search |
### How to Use MCP Servers
```python
# Install MCP server
# npm install -g @modelcontextprotocol/server-filesystem
# In Python, use the MCP client
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
# Connect to MCP server
server_params = StdioServerParameters(
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem", "/home/user"]
)
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
# List available tools
tools = await session.list_tools()
# Call a tool
result = await session.call_tool("read_file", {"path": "/home/user/doc.txt"})
The MCP server exposes its tools as Python functions that smolagents can use.
🎛️ Dynamic Tool Loading: The "Plugin System"
Architecture for Loading Tools at Runtime
# tool_loader.py
import os
import importlib
from smolagents import tool, CodeAgent
def load_tools_from_directory(directory: str):
"""Dynamically load all tools from a directory."""
tools = []
for filename in os.listdir(directory):
if filename.endswith('_tools.py'):
module_name = filename[:-3] # Remove .py
module = importlib.import_module(f"tools.{module_name}")
# Find all @tool decorated functions
for attr_name in dir(module):
attr = getattr(module, attr_name)
if hasattr(attr, '_is_smolagents_tool'):
tools.append(attr)
return tools
# Usage
custom_tools = load_tools_from_directory('./tools')
agent = CodeAgent(
model=my_model,
tools=custom_tools + [read_file, write_file], # Core + custom
)
Tool Configuration File
Users can enable/disable tools via a config:
{
"agent_name": "My Mini-Manus",
"enabled_tools": [
"core:read_file",
"core:write_file",
"core:execute_shell",
"web:fetch_webpage",
"web:web_search",
"analysis:analyze_csv",
"creative:generate_image",
"mcp:github",
"mcp:slack"
],
"max_iterations": 10,
"model": "muhammadtlha944/MCP-Agent-1.7B"
}
📊 Tool Complexity vs Model Capability
What a 1.7B Model Can Handle
| Tool Complexity | Can Use? | Notes |
|---|---|---|
| Simple function (1 param, 1 return) | ✅ Yes | Easy — model gets it from description |
| Multi-param function (3-5 params) | ✅ Yes | With clear descriptions |
| Chain of 2-3 tools | ✅ Yes | With ReAct loop |
| Chain of 5+ tools | ⚠️ Maybe | Depends on context length |
| Complex logic (loops, if/else) | ✅ Yes | CodeAgent handles this well |
| API calls with auth | ✅ Yes | If keys are pre-configured |
| Browser automation | ✅ Yes | With Helium/Selenium abstraction |
| Vision/image understanding | ⚠️ Maybe | Needs vision model (adds VRAM) |
| Real-time streaming | ❌ No | Too complex for 1.7B |
| Multi-agent coordination | ⚠️ Maybe | smolagents multi-agent can help |
Rule of Thumb
If you can describe the tool in 2-3 sentences and it has 1-5 parameters, a 1.7B model can learn to use it from the description alone.
🚀 The Complete Architecture
┌───────────────────────────────────────────────────────────┐
│ User Interface │
│ (Gradio Web App) │
└───────────────────┬───────────────────────────────────────┘
│
▼
┌───────────────────────────────────────────────────────────┐
│ Agent Controller │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ CodeAgent (Qwen3-1.7B) │ │
│ │ │ │
│ │ System Prompt: │ │
│ │ "You are an AI assistant. Use available tools │ │
│ │ to solve problems. Write Python code." │ │
│ │ │ │
│ │ Memory: Conversation history + tool results │ │
│ └─────────────────────────────────────────────────────┘ │
│ │ │
│ ┌────────────────────────┼────────────────────────────┐ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────────────┐ │ │
│ │ │ Tool Registry │ │ │
│ │ │ │ │ │
│ │ │ Core Tools Custom Tools │ │ │
│ │ │ ├─ read_file ├─ get_weather │ │ │
│ │ │ ├─ write_file ├─ fetch_webpage │ │ │
│ │ │ ├─ list_dir ├─ analyze_csv │ │ │
│ │ │ ├─ shell_exec ├─ generate_image │ │ │
│ │ │ ├─ web_search ├─ create_presentation │ │ │
│ │ │ └─ python_exec └─ [user adds more!] │ │ │
│ │ │ │ │ │
│ │ │ MCP Servers (external): │ │ │
│ │ │ ├─ github-mcp-server │ │ │
│ │ │ ├─ slack-mcp-server │ │ │
│ │ │ └─ [any MCP server] │ │ │
│ │ └─────────────────────────────────────────────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌─────────────────────────────────────────────┐ │ │
│ │ │ Tool Implementations │ │ │
│ │ │ │ │ │
│ │ │ Python Libraries: │ │ │
│ │ │ ├─ requests (HTTP) │ │ │
│ │ │ ├─ pandas (data) │ │ │
│ │ │ ├─ matplotlib (charts) │ │ │
│ │ │ ├─ selenium/helium (browser) │ │ │
│ │ │ ├─ diffusers (image gen) │ │ │
│ │ │ └─ [any Python library!] │ │ │
│ │ │ │ │ │
│ │ │ System Tools: │ │ │
│ │ │ ├─ git │ │ │
│ │ │ ├─ ffmpeg │ │ │
│ │ │ └─ [any CLI tool!] │ │ │
│ │ └─────────────────────────────────────────────┘ │ │
│ └────────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────┘
📝 Summary: Adding Tools Is Just Python
| Step | What You Do | Time |
|---|---|---|
| 1 | Write a Python function with @tool decorator |
5 min |
| 2 | Write a good docstring (this teaches the LLM!) | 5 min |
| 3 | Add it to your agent's tools list | 1 min |
| 4 | Test it | 5 min |
| Total | 16 min per tool |
No retraining. No model changes. Just write Python.
🎓 Key Takeaways
- Tools are just Python functions — write them, decorate with
@tool, done - The LLM learns from docstrings — the description teaches the model how to use it
- No retraining needed — add/remove tools anytime
- MCP servers = pre-built tools — 1000+ available, install and use
- CodeAgent writes Python to use tools — more flexible than JSON tool calls
- 1.7B model handles 90% of tools — anything with clear description + 1-5 params
- Dynamic loading — tool marketplace concept: enable/disable tools via config
🔜 How This Changes Our Project
Original Plan
- Train model to generate JSON tool calls (MCP format)
- Build manual ReAct loop
- Hardcode tool registry
- Limited to trained tools
New Plan (Based on Research)
- Train model to solve problems by writing Python (it already knows Python!)
- Use smolagents CodeAgent (handles ReAct loop)
- Dynamic tool registration via
@tooldecorator - Unlimited tools — add any Python function anytime
- Leverage 1000+ MCP servers
- Use built-in GradioUI for the web app
Training Focus Changes
Instead of teaching: "Generate JSON tool calls in MCP format" We teach: "Break problems into steps, write Python code, use available functions"
Benefits:
- Less training data needed (model already knows Python)
- More flexible (any tool works, not just trained ones)
- Easier to add tools later (just write Python)
- More natural for the model (code is easier than JSON schemas)
This is the final piece of our planning. You now have the complete picture: vision, research, architecture, training, dataset, execution plan, tool ecosystem, and dynamic tool loading.
When you're ready: say "START" and we build! 🚀