LLM Adaptation Techniques
The ladder from a plain prompt to a fine-tuned model — few-shot, chain-of-thought, tool use, retrieval-augmented generation, fine-tuning — and what each rung costs before it buys anything.
What it is
A general-purpose model can be pointed at a specialised task without anyone training a new one. The techniques for doing that form a ladder, and the useful discipline is to climb it only as far as the task actually requires, because each rung costs more than the one below it.
Few-shot learning. Put a handful of worked examples in the prompt. Cheapest thing that works, and the fastest way to pin down output format — which matters when the output feeds another program. The catch is that the exemplars are a sample: choose them badly and their biases become the model’s.
Chain-of-thought. Ask for the reasoning before the answer. Costs one line of prompt, usually improves accuracy on multi-step problems, and produces an explanation you can inspect — which in a clinical setting is worth as much as the accuracy. The cost is output that is harder to parse, mitigated by asking for structured output alongside.
Tool learning. Give the model APIs it can call. This is how a model gets access to functions it cannot do reliably in its head — arithmetic, a database lookup, a code execution. Value depends entirely on which tools you expose.
Retrieval-augmented generation (RAG). Retrieve relevant documents and put them in the prompt, so the answer is grounded in a specific corpus rather than in whatever the model absorbed during training. This is the standard answer to two problems at once: knowledge that postdates training, and hallucination. Its ceiling is retrieval quality — a RAG system that retrieves the wrong passage answers confidently from the wrong passage, so the retrieval step needs evaluating in its own right, separately from the generation.
Fine-tuning. Train on annotated examples. Buys the most performance and shortens prompts, and costs annotation effort plus compute. It is also the only rung that produces an artefact you must then version, store and re-validate.
The ordering matters more than any individual rung. Reaching for fine-tuning before trying few-shot is the common expensive mistake, and RAG solves a different problem from fine-tuning — currency and grounding versus behaviour and format — so they are not alternatives.
Why it matters for my work
The group already runs LLMs over pathology reports: LLM Report-Quality Checking scores reports against CAP/ICCR elements, and Report Text Extraction turns narrative into fields. Both are built as tools without an explicit account of which adaptation technique they use, which is why that page’s open questions — is the model version recorded, are prompts under version control — are still open. Naming the technique is the first step to versioning it.
Two mappings are worth drawing. CAP/ICCR compliance checking is a structurization task with a fixed external standard, which is exactly the shape few-shot plus structured output handles well, and where the required elements of a protocol edition are a natural retrieval corpus for RAG. And the trial-matching example in the source tutorial — match a patient description against eligibility criteria — is structurally identical to matching a report against a protocol checklist, so its published prompting patterns transfer directly.
The caution that carries over from the source: a model chosen on a headline benchmark score is chosen on a proxy. MedQA is a multiple-choice exam; performance there says little about whether a model structures a Turkish pathology report correctly. That is the same reasoning as Class Imbalance and Accuracy — one number standing in for a task it does not resemble.
How it connects
LLM Report-Quality Checking — the group’s existing LLM layer; this page supplies the vocabulary its open questions need, since you cannot version a prompting strategy you have not named.
Report Text Extraction — the rule-based-versus-LLM choice sits directly below this ladder: where a rule is crisp, a regex is more auditable than any rung here.
Synoptic Reporting — CAP/ICCR required elements are both the target of the structurization task and the obvious retrieval corpus for a RAG approach to checking.
Interobserver Agreement — the right framework for validating any of these outputs: treat the model as another rater and measure agreement against expert review.
Class Imbalance and Accuracy — the same proxy-metric trap, in a different setting: a single benchmark number standing in for the task you actually care about.
askLLM — the retrieval rung with a number attached: someone else’s jamovi module that measured its own fabrication rate, grounded the model in a scanned catalogue, and measured again.
Jin 2026 — Guidance on the Use of LLMs for Medical Research — the NIH/NLM tutorial this ladder is drawn from, laying out the same progression as five phases: formulate, choose, prompt, fine-tune, deploy.
Open questions
- Which rung does the existing report checker actually sit on — plain prompting, few-shot, or something else? Not recorded anywhere.
[unverified] - Would RAG over the CAP protocol text beat putting required elements directly in the prompt? Testable, and it would also make protocol-edition changes a data update rather than a prompt rewrite. There is now an external data point on the closest analogous problem. askLLM faced models inventing jamovi menu paths — a fixed, enumerable vocabulary that has to be quoted rather than composed, which is structurally what a CAP required-element list is. Sending the real scanned catalogue and demanding verbatim quotation took it from at least three structural fabrications to 18 of 18 paths matching the catalogue exactly, on a deliberately small sample the author flags himself. Not proof, but it is evidence, and the mechanical check it enables — compare every emitted reference against the list — is the part worth copying regardless.
- Is there enough annotated Turkish report data to make fine-tuning worth considering, or is few-shot the ceiling for the bilingual case?
- No page here yet on evaluating retrieval quality separately from generation quality, which is the part most RAG deployments skip.