Choose an AI technique after defining the failure you need to fix. Missing reference information and an unsuitable learned response pattern are different problems. Retrieval and fine-tuning can be combined, but they should not be treated as interchangeable names for making a model better.
Anthropic’s retrieval explanation describes supplying relevant reference material. Hugging Face describes fine-tuning as continuing training on task- or domain-specific data. Neither definition establishes which option will work best on your own workload.
Choose one representative output and describe the improvement the task needs. Relevant reference material, a particular format and a consistent response style are distinct requirements. Giving each a clear description helps connect the proposed technique with the work it is meant to improve.
Identify the problem precisely
| Observed problem | First question |
|---|---|
| Missing current document details | Can the required reference be retrieved and supplied? |
| Inconsistent task behaviour | Are instructions, examples and evaluation criteria clear? |
| Repeated domain-specific output requirements | Would a suitable training dataset address the observed gap? |
| Conflicting or low-quality inputs | Does the evidence need fixing before either approach? |
These are investigation routes, not an automatic decision tree. A better prompt or corrected source material may be worth testing before a larger system change.
Define a baseline
Keep a set of representative tasks and an independent scoring process. Compare the proposed change with the existing system using the same requirements. Record failures involving unsupported answers, unavailable references and changed source material.
Include operation in the comparison
For retrieval, ask who maintains the corpus and checks whether the right passages arrive. For fine-tuning, ask who prepares the data, evaluates changes and maintains the adapted model. Keep access controls and deletion requirements in view. Choose only after the measured result and ongoing responsibilities are clear; a more elaborate architecture is not itself evidence of improvement.
Classify a fictional support failure
Suppose an assistant is asked about a current return policy. If the relevant policy was not supplied, begin by examining how reference material is found and provided. If the policy is available but the answer repeatedly misses a required structure, examine the instructions and examples governing the response.
These observations identify questions to investigate. A workflow may need more than one improvement, so keep the reference-information and response-behaviour checks separate.
Compare the change with a baseline
Create a small set of fictional questions with the expected policy points and source version recorded. Run the current arrangement, save the results and record the missing or incorrect elements. Evaluate a proposed change against the same cases.
Include a case that the reference does not answer. Agree how the assistant should handle that gap, such as requesting clarification or marking the issue for review. This checks whether the workflow can represent uncertainty as well as answer covered questions.
For a changing reference set, add a maintenance question: who updates it, and how is the assistant’s use of the new version checked? For a training-based change, record the training data scope and the evaluation needed afterwards. The comparison is complete when it connects the observed problem, intervention and ongoing responsibilities.
For a fictional support task, keep a dated question, expected answer and supporting source together. Use that same case when assessing a proposed change, and include responsibility for keeping the information current. The example can then serve as both a comparison and a maintenance reference.
Sources consulted
- Introducing Contextual Retrieval — Anthropic; consulted 3 November 2026.
- Fine-tuning — Hugging Face; consulted 3 November 2026.