Statistical reasoning with generative tools

Rao and colleagues’ descriptive case study examines Rtutor.AI, which translates natural-language requests into R operations. Five undergraduates, each with one introductory statistics course and no programming experience, completed an approximately hour-long, in-person interview. The task concerned choosing a school using study hours and ACT scores for 18 students across three schools; these were task data, not the research sample.

Participants first sketched a graph, then used the tool to construct a multivariable scatter plot, and finally compared the output with their sketch and explained a school recommendation. This sequence makes the relationship between an intended analysis and its interpretation visible (§§2.2–2.4).

The contrast between Bennu and Bo is instructive. Bennu specified the graph type and both axis variables. Bo began with a broad graph request; the tool supplied axis variables and a school legend without explicit instructions. Yet Bo later struggled to explain the connection between ACT scores and the school context (§§3.4–3.5).

The authors analysed planning, monitoring and evaluating alongside contextual, statistical and software thinking. Tables 1–2 distinguish choosing an axis range from expressing that choice in a prompt. Two researchers independently coded each interview and reconciled differences (§§2.1, 2.5).

The educational implication is to preserve decisions that automation can complete silently. The authors recommend teaching statistical planning and interpretation before independent AI analysis, alongside step-by-step instruction in tool use (§4.2). A curriculum response is to ask learners to identify variables, justify graphical choices and explain what a generated display means in context. These are design proposals, not an intervention evaluated here.

This five-person qualitative study describes reasoning during one graphing task. It provides no independent non-AI comparison or delayed assessment of unaided learning. Its authors also caution that interview engagement may differ from homework behaviour. The findings cannot establish a causal learning gain or an AXIOVUM outcome (§§2.3–2.4; limitations).

Can learners justify the same variable mappings and contextual conclusions on a new task after the tool is removed? That remains a question for subsequent research; successful graph generation alone does not answer it.