Fine-Tuned Classifier
Case Study Summary
Client: Marketing agency Industry: Marketing & Social Media
Impact Metrics:
- 98.4% accuracy (up from 78% base model, 95% previous LLM solution)
- 81% reduction in inference costs
- 50,000+ posts classified per day
- <200ms average latency per query
Marketing agency reduced sentiment analysis costs by 81% while improving accuracy from 95% to 98.4% by deploying a fine-tuned model for real-time social media classification.
Challenge
The agency needed to classify sentiment across high volumes of social media posts for their clients. Off-the-shelf models were either too expensive at scale, too slow for real-time monitoring, or lacked the accuracy needed for nuanced social media language — slang, sarcasm, and brand-specific context.
Our Approach
After evaluating multiple architectures, we selected Qwen 3 1.7B for its strong baseline performance at a compact size. We trained a LoRA adapter on domain-specific social media data, deployed on our own hardware to eliminate per-query API costs and maintain low latency. The lightweight architecture allows real-time classification without compromising accuracy. For data labelling, we created a lightweight interface to label real examples with ground truth, across multiple languages.
Results & Impact
- Fine-tuned model achieved 98.4% accuracy on sentiment classification, up from 78% on the base model and 95% on the previously deployed LLM solution
- System processes 50,000+ posts per day at an average latency of <200ms per query
- Running on RunPod serverless cut inference costs by 81% compared to API-based alternatives
Tech Stack
- Qwen 3 1.7B with LoRA fine-tuning
- Hugging Face, Virtual Nvidia A100
- Python, RunPod serverless
Team Expertise
Our team brings deep expertise in model selection, fine-tuning, and efficient inference optimisation. We evaluated multiple architectures before landing on the optimal balance of performance, cost, and latency — then trained and deployed the model end-to-end on proprietary infrastructure, giving the agency full ownership of their ML pipeline with zero vendor dependency.
-
Let's have a virtual coffee together!
Want to see if we're a match? Let's have a chat and find out. Schedule a free 30-minute strategy session to discuss your ML use case and explore how we can work together.