How to use international guidelines and AI in methodological clinical research audits

Feeding large language models with guidelines such as PRISMA and CONSORT can transform AI into a tool for preliminary auditing of statistical analysis plans

Juliana Protásio

Close-up of a computer keyboard with a glowing blue AI key.

Mapping out a study’s design and statistical analysis plan is often one of the main bottlenecks in graduate research and clinical studies. The challenge has become even more complex with the growing use of generative artificial intelligence (AI) in the research process: the technology requires carefully crafted prompts to avoid fabricated responses (known as “hallucinations”) or biased analyses.

One approach to mitigating these risks was presented by João Brainer, a neurologist and clinical researcher at Einstein Hospital Israelita, during a virtual event hosted by Science Arena. He says that the key strategy is to feed large language models with the leading international reporting guidelines in the field—such as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) or CONSORT (Consolidated Standards of Reporting Trials) for randomized clinical trials—before beginning any analysis.

By comparing the requirements of these guidelines with a study’s preliminary data, AI can serve as a tool for conducting a preliminary methodological audit, with the aim of helping researchers refine elements such as sample size, outcomes, and subgroup analyses before submitting their work for peer review.

Recommended prompt for auditing a statistical analysis plan

This strategy can be implemented using a structured prompt that instructs the AI to compare a study’s data with the criteria outlined in the selected guideline and identify gaps in the analysis plan. The prompt template recommended by João Brainer is as follows:

Suggested prompt: “Analyze the preliminary data from my study, including the sample size and planned subgroups [insert study data or a summary of the study design]. Then compare this information against the checklist criteria from [insert or paste the PRISMA, CONSORT, or equivalent guideline]. Identify any required criteria missing from my statistical analysis plan and suggest specific refinements to minimize selection bias.”

Elements of a prompt for a methodological audit

1. Study data: Describe the sample size, planned subgroups, and a summary of the study design. These elements are essential for helping the AI understand the scope of the research.

2. Reporting guideline: Insert or paste the reporting guideline appropriate for the study type—for example, PRISMA for systematic reviews or CONSORT for randomized clinical trials.

3. AI instruction: Ask the AI to identify any required criteria missing from the statistical analysis plan and to suggest refinements to help minimize selection bias.

AI as an auditing tool, not a decision-maker

The goal is not to delegate responsibility for a study’s methodological decisions to AI. Rather, it is to use the technology to help verify the soundness of the methodology before the research is submitted for peer review.

Validating each recommendation still requires the researcher’s expertise and critical judgement. However, combining AI with established reporting guidelines can make the process more efficient while improving the transparency and traceability of methodological decisions.

The session with João Brainer offers additional practical guidance on integrating artificial intelligence into academic research while maintaining rigorous ethical standards. Watch the full video.

* This article may be republished online under the CC-BY-NC-ND Creative Commons license.
The text must not be edited and the author(s) and source (Science Arena) must be credited.