How to build a dataset from scratch on a budget

While building our 4 part course - “Build your own Siri” we spent over $1,000 for the dataset generation.

We built this optimization guide to improve on past mistakes and reduce costs drastically.

Think of this as a bonus article to the course (where we broke down how to build a Siri-like assistant locally on your phone).

Quick reminder, in the last 4 articles, we covered:

1. How to prepare a function-calling dataset using Salesforce's xlam-function-calling-60k dataset as few-shot examples for the AI;

2. Fine-tuning LLaMA 3.1 8B with the generated dataset using Unsloth;

3. Quantization of the model so that it can be implemented on a edge device.

4. Edge deployment

For this bonus course, however, we will go back to the beginning, to the part where we prepared the dataset.

Check out the FREE lessons here:

1. What needs to change

2. Let’s generate a new & better dataset

Here’s the downside of the current implementation: cost.

The model we are currently using is Claude-3-5-Sonnet, which is payable, meaning we pay for every token generated. For the dataset we generated initially, the cost was $1,000+, which is a significant disadvantage.

That's why we’ve switched out to a small language model (SLM).

Interestingly, this implementation uses the same model as the one from fine-tuning: LLaMA 3.1 8B. It's a reliable model, and using it within Ollama is FREE of charge, which is what we want.

Now, when generating a dataset, it is essential to understand the requirements.

For this implementation, the requirements are:

• a model that can generate a relevant query for a function-calling dataset;

• a dataset based on the same concept.

We already have the Salesforce xlam-function-calling-60k dataset for the second condition, but is it actually necessary to use the Claude-3-5-Sonnet model for the model, or can we achieve the same results with an SLM?

Well… yes, we can!

Just think about it: If you're going on a trip with two people, do you really need to rent a seven-seater car, or would your five-seater car be enough? The question is basic and the answer is clear, as it is with our models.

There's no need to use a big, well-known model to achieve something that doesn't require a lot of context or a very specific query when there's a free, small model that can achieve the same results.

That’s why I opted for the LLaMA 3.1 8B. It's not a large model, so generating the dataset does not take ages. It's also not such a small model that would invite hallucinations and incorrect queries.

What's more, it's already been trained on function-calling data, making it the perfect fit for us.

If you’re following the course, change the model in the settings.py file.

class DatasetSettings(BaseAppSettings):
    """LLM settings for dataset generation."""
    # Ollama configuration
    LLM_MODEL: str = "ollama/llama3.1:8b"  # Use Ollama with Llama 3.1 8B (smarter, good with 16GB RAM)
    OLLAMA_BASE_URL: str = "http://localhost:11434"
...

We used Ollama because it runs models locally, keeping data private and costs predictable (no per-token fees), but feel free to use something else.

For the prompt, I made a few modifications because it was too big, which caused the model to generate each query very slowly, despite it being very reliable. But I still focused on giving the model a very detailed prompt so that it can generate exactly what’s needed.

I injected the model with a few shots from the Salesforce xlam-function-calling-60k dataset, providing only the query as there was no need for the other information. I also increased the number of shots given to the AI to 10, providing it with a reasonable amount of information from which it could generate a response.

The prompt looks like this:

    # Parse the tool_payload to extract the specific tool calls that were generated
    # tool_payload format: ["- search_google({'query': 'pizza recipes'}) ➜ Searches Google for a query."]
    tool_calls = []
    for payload_line in tool_payload:
        # Extract the tool call part before the arrow
        if '➜' in payload_line:
            tool_call = payload_line.split('➜')[0].strip('- ').strip()
            tool_calls.append(tool_call)
    tool_calls_text = " and ".join(tool_calls)
    prompt_multi_tool = f"""Write a natural user request that would need these exact tool calls:
{tool_calls_text}
Example requests:
- "Search for Italian recipes and save them to my cooking notes"
- "Set volume to 70% then lock the screen for security"
- "Check my battery level and create a note about it"
❌ DON'T write:
- "Here's a request..."
- "The user wants..."
- Any explanations
✅ Write the direct user request:"""
    if use_hf_examples:
        # Extract only the actual user queries, not full JSON
        best_practices = extract_datapoints_hf_dataset(num_datapoints=num_examples)
        # Parse and extract just the user queries from the JSON
        actual_queries = []
        for practice in best_practices:
            try:
                import json
                if isinstance(practice, str):
                    data = json.loads(practice)
                    if 'query' in data:
                        actual_queries.append(data['query'])
            except:
                continue
        if actual_queries:
            prompt_multi_tool += f"\n\nSimilar examples:\n"
            for query in actual_queries:
                prompt_multi_tool += f"- \"{query}\"\n"
    return prompt_multi_tool.strip()

The results after generating the dataset are the same as for the last implementation. This demonstrates that using an SLM rather than an LLM for this kind of task is effective.

Execution validation...
✅ Tested 10 function calls
📊 Success rate: 100.0%
✔️ Passed: 10
❌ Failed: 0

Now that all the necessary modifications have been made, we can generate the dataset.

First, we need to generate the entire dataset without removing any duplicated or contaminated data.

uv run python src/dataset/create_dataset.py --dataset-name "name_of_the_dataset.json" # if you are using uv
#or
python src/dataset/create_dataset.py --dataset-name "name_of_the_dataset.json"

Afterwards, just run the next command and your dataset is prepared.

uv run python validate_dataset.py data/name_of_the_dataset.json
#or
python validate_dataset.py data/name_of_the_dataset.json

The only downside is that this generation takes more time and involves a few more duplication problems, but it is undoubtedly the best approach.

Face it: would you rather give money or time? The answers may vary, but the vast majority wouldn't pay $1,000 for a dataset when there's a better alternative that only takes a few extra minutes or hours and is free.

P.S. As you might expect, all the code and modifications are available on GitHub. Go and check them out for yourself!

If you’ve followed this far (drop a 🔥 in the comments, it’s been a journey), you've gone through the complete process of building a free of charge and reliable function-calling data generator. By this point, you should have:

• A solid understanding of why it is unnecessary to use an LLM for every task when a better approach using an SLM is available.

• A clear understanding of how to write a short but effective prompt.

• A function-tooling dataset generated for free

🔗 Check out the code on GitHub and support us with a ⭐️