Jin 2026 — Guidance on the Use of LLMs for Medical Research

An NIH/NLM tutorial laying out a five-phase workflow for applying large language models to medical tasks: formulate, choose, prompt, fine-tune, deploy.

An NIH/NLM tutorial laying out a five-phase workflow for applying large language models to medical tasks — formulate the task, choose the model, engineer the prompt, fine-tune if needed, deploy with monitoring — aimed at clinicians rather than engineers.

Read status. The Nature Protocols article is paywalled; only the abstract was obtained. The companion repository (ncbi-nlp/LLM-Medicine-Primer) was cloned and its README read in full — 111 lines, four figures — and the README states it is an extension of the paper. So the structure, the three tables and the tutorial links below are firsthand; anything about how the paper argues a point is not, and is marked [unverified].

Who wrote it

Twenty-three authors, led from the National Library of Medicine at NIH (Qiao Jin, Zhiyong Lu), with Illinois Urbana-Champaign, Virginia, Columbia, Yale, Florida State, the NIH Clinical Center, and Weill Cornell. Zhiyong Lu’s NCBI group is the one behind PubMed’s own text-mining infrastructure, which is why the worked example is clinical trial matching against PubMed.

The workflow

Five sequential phases: formulate the task → choose the model → engineer the prompt → fine-tune → deploy.

Task formulation. Five categories of thing an LLM is actually good at: structurization, summarization, translation, knowledge and reasoning, and multi-modal processing. The point of the taxonomy is to check that the medical need maps onto one of them before any tooling is chosen.

Model choice. Four axes: interface (web, API, local), data modality, context length, and medical capability. Table 1 ranks models by MedQA-USMLE — GPT-5 at 95.8%, o1-preview 94.9%, Gemini 3.0 Pro 94.6%, DeepSeek-R1 92.0%, Claude 4.5 91.4%, GPT-4 90.2%, with open-weight options (Llama 3.1, Qwen3) noted for local deployment.

Adaptation. Table 2 is the useful one — five techniques with what each demands and costs:

Technique	Needs	Buys you	Costs
Few-shot learning	A handful of exemplars	Edge cases, output style	Exemplars can inject bias
Tool learning	APIs	Domain functions	Depends on tool curation
Chain-of-thought	One line of prompt	Explanations, accuracy	Output harder to parse
Retrieval-augmented generation	A knowledge base	Current facts, fewer hallucinations	Only as good as retrieval
Fine-tuning	Annotations and compute	Performance, shorter prompts	Expensive

Deployment. Regulatory compliance, ethical guidelines, and continuous monitoring for fairness and bias.

The companion repository

ncbi-nlp/LLM-Medicine-Primer — README, licence, four figures. Public domain under the NLM notice (a US Government work), so unusually free to reuse.

Its five hands-on tutorials all use clinical trial matching as the worked task: LLM basics (tokenization, prompting, temperature), chain-of-thought, few-shot learning, RAG against the PubMed API, and data preparation for fine-tuning.

Reservations

1. A benchmark leaderboard inside a tutorial dates immediately. Table 1 is a snapshot of MedQA-USMLE scores; the repository was retitled two days after publication, and the model list will be wrong within months. Treat the axes as durable and the numbers as perishable.
2. MedQA is a multiple-choice exam, and choosing a model by it is a proxy-metric error. A single headline score on a four-option MCQ says little about whether a model will structure a pathology report reliably. This is the same failure the wiki flags elsewhere: one number standing in for a task it does not resemble.
3. The runnable material is five Google Colab notebooks, not files in the repository. Nothing is pinned, nothing is version-controlled, and the links can rot or be edited silently. For something published in Nature Protocols, where the point is reproducibility, that is a real weakness — the repo itself contains no scripts despite the README saying it provides them.
4. It is a tutorial, not evidence. No new data, no benchmark, no comparison. Its authority is the authors’ standing, not a result — so it supports “here is a sensible way to proceed”, never “this approach was shown to work”.
5. How the tutorial handles validation is not visible from the README. The abstract mentions monitoring for fairness and bias, but whether it tells a reader how to measure whether their own LLM output is right is unknown. [unverified]

Why it is relevant here

It speaks directly to two open questions already recorded on wiki/methods/llm-report-quality-checking.md — whether model versions are captured in checker output, and whether prompts are version-controlled — and it is the first source in this wiki that treats LLM use as something with a method rather than a tool list. The trial-matching worked example is also structurally the same shape as report-to-criteria matching, which is what the group’s CAP/ICCR checkers do.

Derived from: the Nature Protocols abstract, and ncbi-nlp/LLM-Medicine-Primer README at commit f1baaa8 (2026-07-26), both read 2026-07-26.