A foundation model is a large AI model trained on a broad, varied dataset: think billions of words of text from the internet, books, code and more. The model learns a general understanding of language, reasoning and structure. Well-known foundation models include GPT-4 (OpenAI), Claude (Anthropic) and Gemini (Google). You can deploy them directly for a wide variety of tasks without any additional training.

Fine-tuning is the process of training an existing foundation model further on a smaller, specific dataset. You adapt the model to a particular domain, a specific writing style or a well-defined task. A model fine-tuned on legal contracts will respond differently to legal questions than a generic model. Fine-tuning changes the weights of the model itself: it is not the same as prompting or adding context through RAG.

In many cases, a foundation model is more than enough. With good prompts, a clear system instruction and optionally retrieval-augmented generation (RAG), you can make a foundation model perform well on a specific domain without retraining it. This is cheaper, faster and easier to maintain. If your results are already acceptable with smart prompting, fine-tuning is often unnecessary.

Fine-tuning adds value in a limited number of situations. First, when you need a very specific output style that is difficult to control through prompts. Second, when your model must consistently respond in a strict format, such as when generating structured JSON or following a fixed report structure. Third, when latency is a concern: a smaller fine-tuned model can be faster and cheaper than a large generic model with a long prompt. But be honest: in most business use cases, fine-tuning is not the first answer.

Fine-tuning sounds appealing, but comes with costs that are often underestimated. You need high-quality training data: drafting and validating that takes time. The training process itself costs money. And when the foundation model is updated, you need to evaluate whether your fine-tuned model is still current or needs to be retrained. That makes fine-tuning an ongoing investment, not a one-off action.

For many organisations, RAG is a more practical alternative to fine-tuning. You connect the foundation model to a knowledge source: internal documents, product catalogues, FAQs: and the model retrieves relevant information with each query. This is more flexible, easier to update and requires no retraining. Well-designed system prompts can also achieve a large part of the desired behavioural adjustment without additional training.

Consider fine-tuning if you meet several of the following criteria: you have more than a thousand high-quality examples available, the task is strictly defined, you have a clear quality metric to measure against, and prompting demonstrably delivers insufficient results. If you only meet one or two of those criteria, fine-tuning is probably not the best first step.

Foundation models are an excellent starting point for most business applications. Fine-tuning is a useful tool for specific situations, but requires an honest cost-benefit analysis. At Mach8, we help organisations make that assessment based on their actual situation, not based on technical trends. Explore our AI agent services or get in touch to discuss your use case.