P-Values & Clinical SAS Tests

Picture the interview desk: the Statistical Analysis Plan, or SAP, names Wilcoxon rank-sum for the primary endpoint, and a candidate who can merge datasets cleanly still freezes. That moment is not a statistics exam. It tests whether you can read the SAP, recognise what the statistician chose and roughly why, then quality control, or QC, the table that comes out of it. As programmers, we do not choose the tests; we explain them, program them, and prove the outputs are right. Over this course we walk the route: the p-value and its confidence interval, or CI, pairing, then each test family in turn — Wilcoxon rank-sum, Fisher's exact, mixed model for repeated measures, or MMRM, log-rank, and Cochran-Mantel-Haenszel, or CMH, row mean scores — and finally multiplicity and the QC craft behind every cell.

Let's make the p-value honest and plain. A p-value is the probability of seeing data at least as extreme as what was observed, assuming a model where the null hypothesis holds. So when the p-value is small, the data sit awkwardly with that null; when it is large, they do not. That is the whole claim, and we should not stretch it. Please remember what a p-value is not: it is not the probability the null is true, not the probability the result came from chance, and not a measure of effect size or importance. Your programmer duty is practical: run the right test on the right records with the right N, then transcribe the result into the table without damage.

An estimate on its own only tells you where the signal points. The confidence interval, or CI, tells you how precisely that estimate points, and it is a separate axis from the signal itself. At the level stated in the statistical analysis plan, the SAP, usually 95 percent, the CI gives the range of parameter values compatible with the observed data under the model. That matters because a tiny p-value can sit on an interval so wide that no clinically meaningful conclusion survives it, and the estimate-plus-CI pair is what lets a reviewer see that. In quality control, or QC, you verify the interval together with the estimate: same population, same sample size, N, and the interval built around the exact estimate printed beside it.

On this slide, we are choosing between the t-test and the Wilcoxon rank-sum test. The t-test compares two group means, and it assumes the data behave well enough for that comparison to be meaningful. The Wilcoxon rank-sum test compares two distributions using ranks, so it asks far less of the data. Your Statistical Analysis Plan, or SAP, switches to Wilcoxon when the endpoint is skewed, like lab deltas or pain scores, when it is ordinal, or when samples are small enough that no theory rescues the mean. In an interview, give two parts: name what each test compares, then name the condition that made the statistician switch; no one wants a formula. In SAS, which stands for Statistical Analysis System, use PROC TTEST for the t-test and PROC NPAR1WAY, the nonparametric one-way procedure, with the WILCOXON option for Wilcoxon.

Now you are looking at the choice between a chi-square test and Fisher's exact test, and the thing to watch is the expected cell counts. Chi-square approximates the null distribution of the discrepancy between observed and expected counts, and that approximation degrades when counts get small. The working rule is simple: when any expected cell count falls below about five, use Fisher's exact test, which computes the p-value directly instead of approximating it. In PROC FREQ, the CHISQ option gives the asymptotic tests, while the EXACT statement requests exact computation, and that cost grows as the sample size climbs. Because the output contains both tests, your duty is to confirm the table reports the one your statistical analysis plan, or SAP, names. And do not skip the checkpoint: verify that expected cells were checked before you trust the result.

Your Statistical Analysis Plan, or SAP, pre-specifies how to handle multiplicity. Every test at the 0.05 level carries a five percent chance of a false alarm under the null. Run a family of tests, and those chances compound into a real probability of at least one spurious win. So the SAP responds before the fact with a testing hierarchy, an alpha split across endpoints or interim looks, and adjusted p-values and confidence intervals. Your job is transcription with fidelity: reproduce the adjusted values the SAP names, footnote the method the SAP names, and adjust nothing on your own initiative. If an interviewer asks what multiplicity means, those two sentences about false alarms and compounding are the whole answer.

ANOVA, analysis of variance, compares means across more than two groups at one time point, with no covariate adjustment. ANCOVA, analysis of covariance, adds covariates to ANOVA, classically the baseline value of the endpoint, which trims noise and corrects baseline imbalance. The older longitudinal approach, LOCF, last observation carried forward, imputed missing visits and analyzed the filled-in data, but that manufactures flat trajectories and biases results when dropout is informative. MMRM, the mixed model for repeated measures, analyzes all observed values under a missing-at-random assumption, MAR, with no imputation, which is why longitudinal statistical analysis plans, SAPs, moved to it. In SAS, the Statistical Analysis System, MMRM is PROC MIXED with a repeated effect and an unstructured covariance; the denominator degrees of freedom in the output confirm the model was fit as specified.

Now we connect the log-rank test straight to the ADTTE, the time-to-event analysis dataset, which compares survival experience between groups across the whole follow-up time. It uses each event time and respects censoring. In ADTTE, AVAL carries the durations, and the censor variable, CNSR, tells the procedure which records are events: zero means event, one means censored. In SAS, you run PROC LIFETEST with a STRATA statement, and the Kaplan-Meier, KM, plot pairs with the test. The quality control, QC, tie is exact: the log-rank event count must equal the number of records where CNSR equals zero in the analysis population, and the censored count must reconcile the same way. If the log-rank table disagrees with its own ADTTE, it is wrong before any statistics get checked.

A shift table crosses baseline category against post-baseline category, tracking each subject's movement and asking whether those shifts differ by treatment arm. The Cochran-Mantel-Haenszel (CMH) row mean scores statistic treats ordinal categories as scores, then tests whether the average score shifts across arms, stratified by site or analysis pool. In PROC FREQ, you enable this by listing the strata variables first in the TABLES request and setting the CMH options, so the row mean scores section appears in output. For quality control, confirm the table prints the specific CMH statistic the shell specifies, and confirm the score assignment follows the Statistical Analysis Plan (SAP) category ordering. These tables are built from Basic Data Structure (BDS) outputs such as the Laboratory Analysis Dataset (ADLB) and Vital Signs Analysis Dataset (ADVS).

This scene is a hands-on quality control drill, so you run it in the browser at jaimeyan.com/learn rather than watching it in the video. The skill you practice is the hand-verify-one-cell method on a responder comparison table from Study XYZ: you build it in PROC FREQ, the SAS Statistical Analysis System frequency procedure, using the chi-square test the plan expects and Fisher's exact test, cross-foot the N by summing the intent-to-treat flag ITTFL in the subject-level analysis dataset ADSL against the distinct count of the unique subject identifier USUBJID, and confirm the test statistic and degrees of freedom against the procedure's Output Delivery System ODS output instead of the rendered table. The drill also plants near-neighbor traps, such as a t-test where the statistical analysis plan, SAP, says Wilcoxon rank-sum, or an asymptotic chi-square where the SAP says exact, so try it right after the video and see whether you catch them.

This is the final knowledge check: explain, program, and prove each p-value and clinical Statistical Analysis System (SAS) test decision. When a pre-specified Fisher exact test shows a p-value of 0.049 under a Statistical Analysis Plan (SAP)-declared two-sided alpha of 0.05, the correct interpretation is B: assuming no association exists, the probability of a table as extreme as or more extreme than the observed table is 0.049, because a p-value is a conditional probability under the null hypothesis, not the probability that the null is true. After SAP-defined multiplicity control, the necessary quality control (QC) statements before releasing a table row are A and B: a row may be marked significant only if the endpoint passed its pre-specified hierarchical gate, and the estimate, confidence interval, and p-value must come from the same SAP-specified statistical model and analysis population. For a 2x2 binary table with one expected cell count below 5, when the SAP says use chi-square only if all expected counts are at least 5 and otherwise use Fisher's exact, the correct match is B: run PROC FREQ with CHISQ and EXACT FISHER, then report the Fisher exact p-value, because the chi-square approximation is unreliable with small expected counts. For a continuous repeated-measures primary endpoint, when the SAP chooses mixed model for repeated measures (MMRM) rather than last observation carried forward (LOCF), the correct description is A: MMRM estimates treatment-group mean differences over time from all available postbaseline data and models within-subject correlation, so it does not impute missing values, unlike LOCF. For the log-rank time-to-event comparison, the model answer is to reconcile the analysis dataset for time-to-event (ADTTE) event count and denominator against the SAP and clinical database, confirm censoring and risk sets, and verify the log-rank p-value against the SAS output from PROC LIFETEST or the Output Delivery System (ODS) table before release.

For each test, say what it compares, when the Statistical Analysis Plan (SAP) switches to it, and which procedure runs it. Confidence intervals (CIs) ride beside estimates because precision is separate from signal; verify the pair uses one population and one N. Longitudinal SAPs moved to Mixed Model for Repeated Measures (MMRM) because Last Observation Carried Forward (LOCF) manufactured flat data under informative dropout; check the denominator degrees of freedom in PROC MIXED. Quality Control (QC): hand-verify one cell per block, tie every N to the Subject-Level Analysis Dataset (ADSL) flag, and check rounding against the shell before trusting a p-value. Pair reading: the p-values and tests article, the Time-to-Event Analysis Dataset (ADTTE) companion for log-rank, Part 17 for LOCF, and Part 3 for shells. This lesson used only canonical code snippets; no participant rows, so no fabricated patient numbers.