Fine-Tuning Local Models

Advanced guide • 20 min read • Python/ML experience helpful

Stock models are generic. Fine-tuning teaches them your domain. Your proprietary recruiting rubric, your sales voice, your company's culture—baked into the model.

What Is Fine-Tuning?

Fine-tuning is training a model on your data. It takes a general model (Llama 2) and optimizes it for your specific task.

Stock Llama: "Write a cold email about software sales."
Fine-tuned on your emails: "Write a cold email about software sales" → generates emails in YOUR voice, YOUR structure, YOUR messaging.

When to Fine-Tune

Good use cases:

Recruiting screener (specific rubric for your role)
Customer support responses (your tone, your products)
Sales email generator (your pitch, your ICP)
Content writer (your brand voice)

Not worth it:

One-off tasks (fine-tuning costs time/GPU)
Tasks where generic models work well
If you have <100 examples

Data Prep: The Hard Part

Fine-tuning needs examples. You need 100-1000 input/output pairs.

For recruiting screening:
Input: Resume text
Output: Scoring (technical: 8, culture: 7, growth: 6, recommendation: interview)

For sales emails:
Input: Prospect company, industry, pain point
Output: Personalized cold email in your style

Collect these from your past work. Recruiting team: compile 200 screened resumes with scores. Sales team: export 300 best emails sent.

The Fine-Tuning Process

Step 1: Format Your Data

{ "messages": [ {"role": "user", "content": "Screen this resume for a Senior Engineer role: [resume text]"}, {"role": "assistant", "content": "Technical fit: 8/10. Growth: 7/10. Culture: 8/10. Recommendation: Schedule technical interview."} ] }

Step 2: Use a Fine-Tuning Tool

Options:

Ollama (local): Fine-tune directly on your laptop (free but slow)
Hugging Face (cloud): $10-50 per model (faster)
Together AI (cloud): Fine-tune any open model cheaply

Step 3: Train & Test

Training takes 1-4 hours depending on model size and data. Then test on 10 examples and iterate.

Cost & ROI

One-time cost: $50-200 (cloud fine-tuning)
Monthly cost: $0 (run locally after tuning)
ROI: If your fine-tuned model saves 2 hours/week, ROI is 1-2 months.

Example: Recruiting Screener

You collected 300 resumes you screened over the past year. Format them:

[300 examples of: Resume → Your scoring notes] Then fine-tune on this data and get a model that screens like YOU.

Result: A model that scores candidates using your exact rubric, in your voice, with your judgment. Deploy it and let it screen all incoming resumes.

Deployment: Running Your Fine-Tuned Model

Once fine-tuned, export it and run locally:

ollama run my-custom-recruiter

Now you have a production-grade screening tool, zero monthly cost, completely private.

Limitations

Fine-tuning is only as good as your training data
Quality gap remains (your fine-tuned Llama is better, but not better than Claude)
Takes time to collect 100+ examples
Requires some technical setup (or willingness to learn)

Next Level: Continuous Learning

Build a feedback loop:

Model screens candidates
You review top 10 and add them to training data
Re-fine-tune weekly
Model gets smarter over time

This is your proprietary hiring advantage.

Ready to build a domain-specific AI model for your team? Let's architect it.

← Back to Home