Jaime Yan — Research & Writing

LLM Agents in Clinical Trials: What's Real Beyond the Demos

Sun, 30 Aug 2026 00:00:00 GMT

How to tell demo-ware from deployable agentic AI in clinical trials: determinism, audit trails, 21 CFR Part 11, constrained architectures, and vendor questions.

AI Coding Assistants for Clinical Programmers: What Works in GxP

Sun, 30 Aug 2026 00:00:00 GMT

Where ChatGPT, Claude, and Copilot genuinely save time in SAS and R clinical programming, where they fail, and how to use them inside auditable GxP workflows.

CDISC CORE Explained: Open-Source Validation for SDTM/ADaM Programmers

Sun, 30 Aug 2026 00:00:00 GMT

CDISC CORE is a free, open-source rule engine for SDTM and ADaM validation. How its YAML rules work, how to run it, and where rule-based checking stops.

Temperature 0 Doesn't Buy You Reproducibility

Sun, 30 Aug 2026 00:00:00 GMT

Setting temperature to zero feels like determinism. It isn't — and in GxP work the difference will find you during an audit, not during development.

SAS to R Migration in Clinical Programming: A Practical Field Guide

Sun, 30 Aug 2026 00:00:00 GMT

Why pharma is moving from SAS to R, what breaks in practice (procedures, XPT files, QC), and how to migrate a validated macro library without a rewrite.

One Number That Explains Where Rule-Based Validation Stops

Sun, 30 Aug 2026 00:00:00 GMT

69.7% of oncology CORE rules port cleanly to a graph constraint language. The 30% that don't are exactly where the dangerous errors live.

LLMs in Clinical Statistical Programming: Proven vs Hype (2026)

Sun, 30 Aug 2026 00:00:00 GMT

A tiered survey of LLM evidence in clinical trial statistical programming: benchmarked results, promising single-team studies, vendor hype, and open gaps for 2026–2027.

The Contradictions Your Validator Can't See

Thu, 27 Aug 2026 00:00:00 GMT

Rule-based SDTM validators structurally miss cross-domain contradictions. SHACL-SPARQL graph constraints plus a deterministic agent layer catch all 20 archetypes.

Double Programming Without the Duplication: AI-Generated QC Code

Tue, 25 Aug 2026 00:00:00 GMT

Independent QC re-programming costs 30–50% of clinical programming effort. An AI framework matched 97.1–100% of variables while keeping independence intact.

Benchmarking RAG for Clinical TLF Templates: 1,999 Experiments

Sat, 22 Aug 2026 00:00:00 GMT

Five LLM generation methods benchmarked across 1,999 bootstrap experiments on ICH E3-conformant TLF templates: hybrid RAG with reranking beats direct prompting.

Why LLM Agents Fail at Regulated Programming — the Process-DAG Fix

Thu, 20 Aug 2026 00:00:00 GMT

Free-form agent loops break down in GxP clinical programming. Structuring the work as a typed process DAG makes LLM agents reliable, replayable, and auditable.

Five Years of Automation Evidence: Real Gains, Mostly Low Grades

Wed, 19 Aug 2026 00:00:00 GMT

A GRADE-rated review of 2020–2025 automation evidence in statistical programming: real gains, mostly Low to Very Low quality evidence.

Modernizing a 558-Macro SAS Library Without Touching Validated Code

Sun, 16 Aug 2026 00:00:00 GMT

A bridge map, typed IR, and orchestrator wrap a legacy SAS TFL library unchanged — AI-ready JSON on day one, 80%+ cell-level parity, optional 92% code cut.

CAVE-Onc: Graph-Constrained Agentic Validation for Cross-Domain Contradictions in CDISC Oncology Submissions

Fri, 14 Aug 2026 00:00:00 GMT

PLOS One — Regulatory submissions in oncology must remain consistent across interdependent CDISC SDTM domains, yet cross-domain contradictions routinely survive conventional validation. CAVE-Onc introduces a graph-constrained agentic validation framework that models SDTM submission datasets as an RDF knowledge graph, combining declarative SHACL-SPARQL graph constraints with a deterministic LangGraph-based agent layer to detect cross-domain contradictions that domain-scoped rule-based validators cannot express. In a pre-registered evaluation, CAVE-Onc detected all 20 clinician-reviewed injected contradiction archetypes (vs 8/20 for the CDISC CORE engine and 6/20 for the Pinnacle 21 FDA engine, both 0/10 on cross-domain RECIST contradictions) — a construction validation of expressiveness — and stayed specific (0.06–0.09 flags/subject) on two real Project Data Sphere oncology trials mapped to SDTM, detecting 10/11 and 16/18 of applicable injected archetypes. CAVE-Onc writes all validation traces to a Merkle-chained, tamper-evident audit store, providing a foundation for 21 CFR Part 11 compliance.

Thin MCP, Thick Skills: Five Layers for Clinical Programming Agents

Thu, 13 Aug 2026 00:00:00 GMT

ClinAgent splits clinical programming capability into five layers, keeping MCP tools stateless and packaging domain expertise as testable skills.

Synthetic ADaM That Survives a Join: Knowledge Graphs, LLMs, Faker

Mon, 10 Aug 2026 00:00:00 GMT

Schema-only synthetic ADaM generation plateaus at 0.45 overall quality; enriching schemas from protocol/SAP/CRF knowledge graphs plus templates reaches 0.70.

Fine-Tuning Llama 3.1 8B for admiral R Code: From 0.36 to 0.82

Fri, 07 Aug 2026 00:00:00 GMT

Base Llama 3.1 8B scores 0.36 on admiral code generation. LoRA fine-tuning plus knowledge-graph validation gets it to 0.82 — without sending data to an API.

A Non-Destructive Methodological Framework for Modernizing Legacy Clinical Reporting Systems for AI-Driven Pharmacoinformatics: A SAS Case Study

Wed, 13 May 2026 00:00:00 GMT

arXiv — Legacy SAS macro libraries resist AI integration because they emit opaque RTF with no machine-readable layer, yet source-level change triggers re-validation. This preprint presents a non-destructive framework — a bridge map, typed Intermediate Representation, and orchestrator that wrap the legacy library unchanged — applied to a 558-component industrial SAS library. Coexistence mode delivers AI-ready JSON output on day one; optional consolidation achieved a 92% SAS code reduction. Parity validation reached at least 80% cell-level parity on 11 of 14 reports from an internal Phase III study and 100% parity (4,764 cells) on the public CDISCPilot01 benchmark, and LLM experiments demonstrated IR-based table summarization, anomaly detection, and configuration generation.

ClinAgent: A Five-Layer Architecture for Autonomous Clinical Trial Statistical Programming

Fri, 16 Jan 2026 00:00:00 GMT

medRxiv — This preprint presents the five-layer ClinAgent architecture — A2UI rendering, skill router, thick skills, thin MCP tools, and compliance infrastructure — under a "Thin MCP, Thick Skills" design that augments any MCP-compatible AI coding agent (Augment Code, Claude Code, Cline, Cursor) with clinical programming capabilities rather than acting as an autonomous agent itself. Skills are independently testable, and the infrastructure layer provides audit logging and data masking for GxP compliance. Nine skills (SK-001–SK-009) were validated on a production Phase 2 cardiovascular study (11 ADaM domains, 93,239 synthetic observations): log analysis achieved 100% precision (1 error, 7 warnings over 10 logs), all 56 ADSL variables were matched, and specification generation reached 72.1% derivation accuracy overall and above 96% on simple domains.

ClinAgent: AI-Assisted Methodology for Clinical Trial Data Processing and Statistical Programming

Thu, 01 Jan 2026 00:00:00 GMT

Biology Methods and Protocols — Clinical trial statistical programming remains labor-intensive: ADaM dataset derivation, TLF generation, and QC programming consume substantial effort per submission. ClinAgent is a skill-and-tool layer that augments any MCP-compatible AI coding agent with clinical-programming capabilities: nine skills (SK-001 Study Setup through SK-009 eSub Packaging) package prompts, deterministic rule engines, and decision trees, while thin MCP tools provide stateless I/O for SAS datasets, Excel specifications, and log files. All nine skills passed functional validation on artifacts from a single production Phase 2 study — deterministic components matched all 56 ADSL variables, and prompt-based specification generation reached 72.1% derivation accuracy (>96% on simple domains, <55% on complex domains, with wide confidence intervals making these point estimates upper bounds) — while end-to-end productivity gains were not measured and remain future work. Human review gates and deterministic audit trails preserve the traceability and reproducibility required for regulatory submissions.

Eliminating QC Programming Duplication Through Claude AI-Assisted Independent Code Generation: A Practical Framework for Regulatory-Compliant Validation

Thu, 01 Jan 2026 00:00:00 GMT

PharmaSUG 2026 — Independent QC programming — recreating production programs from specifications alone — accounts for an estimated 30–50% of total clinical programming effort. This paper presents a Claude AI-assisted workflow that generates independent QC code in Python directly from ADaM specifications, combining a QC Trace Tree, a Decision Router, Agent Skills, and an automated code review engine to reduce duplication between production and QC streams while preserving operational independence through isolated AI instances, with the human QC programmer as the genuinely independent review layer. On the CDISCPilot01 benchmark, the framework achieved 97.1%–100% variable-level match across five ADaM domains and passed all 13 assertions. Companion code is publicly available.

Schema-Preserving Generation of Clinical TLF Templates and Executable R Code via Iterative LLM-Guided Debugging

Thu, 01 Jan 2026 00:00:00 GMT

PharmaSUG 2026 — Manual authoring of CSR TLF templates is resource-intensive, and naive LLM prompting suffers schema drift and regulatory inconsistencies. This paper compares five LLM generation methods for producing ICH E3/CDISC-conformant TLF templates across 1,999 instance-matched bootstrap experiments on three LLM providers, using JSON Patch (RFC 6902) to preserve schema fidelity, and separately evaluates iterative LLM-guided debugging for translating the templates into executable R code. A hybrid RAG approach with reranking significantly outperformed direct prompting (mean quality score 85.7 vs 81.7, p < 0.05, consistent across providers and therapeutic areas), and iterative LLM-guided debugging raised R-code execution success from a low zero-shot rate to 70% within 3–5 rounds, with higher-fidelity templates needing fewer iterations.

GxP-Agent: Process-DAG Topology for Reliable Clinical Trial Programming with LLM Agents

Thu, 01 Jan 2026 00:00:00 GMT

arXiv — LLM agents fail in regulated programming work when their execution is unstructured. GxP-Agent organizes clinical trial programming as a process DAG — typed nodes for data ingestion, derivation, validation, metadata application, and export — so that agent behavior is constrained, replayable, and auditable. The topology yields measurably higher task reliability than free-form agent loops and produces the evidence trails GxP review requires.

Evidence Behind the Automation of Clinical Trial Statistical Programming: A Scoping Review of Technology Adoption, Validation Frameworks, and AI/ML Integration (2020–2025)

Mon, 29 Dec 2025 00:00:00 GMT

medRxiv — This scoping review (reported per PRISMA-ScR) maps the 2020–2025 evidence base on automating clinical trial statistical programming — from macro-based tooling to LLM-driven approaches — characterizing study types, claimed efficiency gains, evaluation rigor, and open gaps. From 1,247 records, 262 studies were included; reported gains include 15–25% development-time reduction for pharmaverse TLF tools, 30–50% effort reduction for risk-based validation with CI/CD, and 75–85% SDTM conversion-time reduction for REDCap2SDTM, alongside 88–93% F1 for domain-specific LLMs on clinical NLP versus 60–85% code-generation accuracy for general models. Evidence quality is predominantly Low to Very Low: only 12 of 527 validation papers (2.3%) report quantitative outcomes, and no RCTs comparing validation approaches exist, defining the critical research priorities for the field.

An End-to-End Approach to Fine-Tune Small LLMs for Generating Admiral R Code in Statistical Programming

Wed, 01 Jan 2025 00:00:00 GMT

PhUSE US Connect 2025 — Base small LLMs perform poorly on Admiral code generation (OPS 0.36), and even strong general-purpose LLMs offer no built-in correctness verification for ADaM compliance. This paper presents an end-to-end pipeline for fine-tuning small, locally deployable LLMs on curated Admiral examples — fine-tuning LLaMA 3.1 8B with LoRA and validating output through a knowledge-graph system built from Admiral documentation. The pipeline covers data preparation, training, and evaluation, achieving reliable ADaM code generation without sending proprietary data to external APIs.

Automating SAS and R Code Interpretation and Debugging: A Practical Pipeline for Statistical Programmers

Wed, 01 Jan 2025 00:00:00 GMT

PhUSE US Connect 2025 — Statistical programmers spend substantial effort writing and debugging SAS and R code. This paper describes an LLM-based pipeline that translates natural-language queries into executable SAS, R, or Python code, runs it in a Jupyter kernel, captures runtime errors from the kernel, and feeds them back to the LLM for iterative repair until the code succeeds or a retry limit is reached, exposed through a Gradio-based user interface. Evaluated on a factorial corpus of 2,700 runs across Python, R, and SAS with injected syntax, runtime, and logical/statistical errors, the system achieved success rates above 96% across languages, resolving about 95% of syntax and 90% of runtime errors versus about 88% and 87% for logical and statistical errors.

Enhancing Clinical Trial Data Queries with LLMs and Neo4j: A Flexible Framework for ADaM Dataset Management

Wed, 01 Jan 2025 00:00:00 GMT

PhUSE US Connect 2025 — This paper models CDISC ADaM datasets in a Neo4j graph database via a reusable graph schema derived from ADaM specification files, and uses LLMs with LangChain to translate natural-language questions into both Cypher and SQL queries, making data checking more flexible, particularly for ad hoc tasks. Benchmarked head-to-head on 90 protocol-derived questions over synthetic ADaM datasets (ADSL, ADAE, ADCM, ADLB, ADVS) held in both PostgreSQL and Neo4j, LLM-generated Cypher achieved higher accuracy than SQL (73.33% vs 66.67% overall), with the advantage holding across difficulty tiers.

A Novel Pipeline for Generating Realistic Synthetic CDISC ADaM Datasets Using Large Language Models and Knowledge Graphs

Wed, 01 Jan 2025 00:00:00 GMT

PhUSE US Connect 2025 — Creating synthetic ADaM data that faithfully represents real trial characteristics is challenging. This paper combines knowledge graphs built from clinical trial documentation (protocols, SAPs, CRFs) with Faker-based generation, using LLMs to enrich and reorganize the JSON schemas that drive data generation. A three-way comparison of direct JSON, LLM-enhanced JSON, and template-based generation shows the template-based variant achieving the highest overall quality score (0.70 vs 0.45 for direct JSON-schema generation), with improved structural integrity and cross-dataset relationships.

GenAI Assisted Code Conversion: From SAS to R Standard ADaM Templates

Wed, 01 Jan 2025 00:00:00 GMT

PharmaSUG 2025 — As clinical trial Analysis & Reporting shifts toward open-source languages, our organization needed to convert its SAS-based ADaM standards library to R. This paper presents a GenAI-assisted conversion methodology using a structured workflow of source pre-processing, logical code segmentation, and iterative prompt engineering that accumulates conversion instructions from previously converted templates, followed by human validation comparing converted R outputs against the validated SAS-generated ADaM datasets. Using ChatGPT-4o and Claude 3.5 Sonnet, the effort converted about 78% of the SAS ADaM templates, with about 66% of the converted code used without modification, though macro-heavy templates still required significant human intervention.

Comparing SQL and Graph Database Query Methods for Answering Clinical Trial Questions with LLM-Powered Pipelines

Wed, 01 Jan 2025 00:00:00 GMT

PharmaSUG 2025 — Should clinical data exploration use SQL or graph databases? This paper benchmarks both query paradigms behind LLM-powered natural-language pipelines on 150 representative clinical trial questions over synthetic ADaM data, comparing accuracy, latency, and the classes of questions each paradigm handles well. The proposed RagQL-Nav framework — query decomposition, intelligent routing, and dual-query validation — reached 91% accuracy on complex queries, roughly 12 points above single-system approaches, offering practical guidance on when graph approaches earn their complexity.

AI-Enhanced Chatbot for Streamlined Clinical Trials Analysis and Document Management

Mon, 01 Jan 2024 00:00:00 GMT

PhUSE US Connect 2024 — Statistical programming teams lose hours searching specifications, protocols, and analysis documentation. This paper describes a retrieval-augmented chatbot that indexes clinical trial documents and analysis artifacts and answers natural-language questions against the retrieved content, with a local-deployment option that keeps confidential documents and data in-house. A fine-tuned LLM also maps mock table shells to SAS macro calls for end-to-end TLF generation, with a SAS kernel executing the generated code.

LLM-Enhanced Training Agent for Statistical Programming

Mon, 01 Jan 2024 00:00:00 GMT

PharmaSUG 2024 — This paper presents an LLM-enhanced training agent for statistical programming, implemented as a custom GPT-4 GPT built via prompt engineering with uploaded reference documents (SDTMIG, ADaMIG, ICH E9, and internal SAS guidance) and a slash-command interface. The agent guides learners through personalized curricula, lessons, and real-world simulation projects, tests their knowledge with quizzes and coding exercises, and gives immediate, personalized feedback, with positive feedback from a pilot testing phase.

Methodology for Automating TOC Extraction from Word Documents to Excel

Mon, 01 Jan 2024 00:00:00 GMT

PharmaSUG 2024 — Programmers tracking clinical study deliverables often transcribe the table of contents of SAP mock-shell Word documents into Excel trackers by hand. This paper presents an Excel-based VBA tool — chosen because company policy bars external tools — that extracts the TOC from Word mock-shell documents into an Excel tracker at the click of a button, locating the TOC field programmatically and stripping page numbers from entries. The resulting trace system supports monitoring each study deliverable’s progression and validation status.

JSON Data Generation: Linking Statistical Analysis with Large Language Models

Mon, 01 Jan 2024 00:00:00 GMT

PHUSE/FDA Computational Science Symposium (CSS) 2024 — This poster presents a framework that extracts schema from structured clinical datasets (e.g., CDISC ADaM) into JSON, uses it to design a graph-based database (Neo4j) with graph embeddings stored in a vector database, and enables LLMs to answer natural-language questions by retrieving relevant variables and generating SQL or Cypher queries — automating clinical data analysis workflows.

A Macro Utility for CDISC Datasets Cross Checking

Sun, 01 Jan 2023 00:00:00 GMT

PharmaSUG 2023 — CDISC validation tools such as Pinnacle 21 check standards compliance but not dataset content, and A&R programmers need study- and therapy-area-specific checks throughout the analysis lifecycle. This paper presents a configurable SAS macro utility that systematizes cross checking across SDTM and ADaM datasets; its modular sub-macro framework lets users plug in custom checking rules, extensible beyond CDISC datasets. The macro generates a formatted Excel discrepancy report with a hyperlinked summary tab and one sheet per issue category, supporting faster data review cycles.

Tips to Read In and Output Excel Spreadsheets in SAS

Sun, 01 Jan 2023 00:00:00 GMT

PharmaSUG 2023 — Excel is widely used in clinical trials to save and transfer data, and analysis & reporting programmers routinely receive spreadsheets and produce customized Excel reports for other departments. This paper collects practical techniques for reading different Excel formats (xls, xlsx, xlsm, xlsb, csv) into SAS via LIBNAME engines and PROC IMPORT — including how to choose between them by environment and format — and for writing well-formatted outputs with PROC EXPORT and ODS EXCEL. It offers practical decision tips for choosing import/export methods and output formats.

A Framework for Interactive Ad-hoc Request Handling: Empowering Clinical Insights through Interactive Plots

Sun, 01 Jan 2023 00:00:00 GMT

PHUSE/FDA Computational Science Symposium (CSS) 2023 — This poster presents a framework for producing interactive plots to handle ad-hoc analysis requests in the clinical domain. It standardizes the end-to-end process — Anaconda-based environment setup, package version control via renv/pipenv lock files, Jupyter Lab as the IDE, interactive visualization with Plotly (Python) and ggplot2 plus Plotly/htmlwidgets (R), and portable self-contained HTML output — with results validated by crosschecking between SAS, R, and Python. Example outputs include swimmer, waterfall, and Kaplan-Meier plots with drop-downs and range sliders.