Learning the best pathway from data

Where the theory-driven model encodes expert rules, the data-driven model estimates the optimal course-taking pathway directly from a large longitudinal study, using causal machine learning to ask: for a student like this, which sequence of courses would most improve the final outcome?

Two goals, two tracks

Every recommendation aims at one goal, along one of two tracks.

• Mathematics achievement aims for the highest weighted math GPA. For the Grade 9 only plan this is the school-adjusted Grade 9 math GPA; for the four-year plan it is the math GPA at the end of high school. College enrollment aims to raise the chance of enrolling in a four-year degree-seeking college.
• A plan for Grade 9 only recommends the single best starting course. A plan for all four high school years recommends a Grade 9 to 12 sequence that takes each year’s results into account.

The two do not combine freely. The Grade 9 only track is fitted for mathematics achievement, while the four-year track is fitted for both goals, so you can compare the two pathways side by side.

Optimal dynamic treatment regimes

The method comes from precision medicine, where an optimal dynamic treatment regime (ODTR) is a series of decision rules (one per stage) that maximizes a final outcome given a patient's evolving history. Course-taking maps onto this naturally: each grade is a decision stage, and the “treatment” is which math course to take next. The model searches for the rule at each grade that maximizes end-of-high-school math achievement (or college enrollment).

How it is estimated

• Targeted Maximum Likelihood Estimation (TMLE) with ensemble machine learning. We chose it for its double robustness and its flexible, automatic fitting.
• Trained on the High School Longitudinal Study of 2009 (HSLS:09), a nationally representative dataset following students across high school.
• Two versions, mirroring the theory-driven model: a plan for Grade 9 only and a plan for all four high school years (the single-stage and multi-stage models in the paper).

Feasibility comes first

An unconstrained model can chase higher estimated outcomes by recommending pathways that no real school would offer. This model instead imposes feasibility constraints (based on domain knowledge and propensity-score thresholds) so the recommendations are realistic and implementable, with stronger empirical and expert-informed support.

From a fitted model to a tool a school can use

There is a real gap between fitting the model and putting it in front of a counselor. The fitting process runs on research computers: an ensemble (a team of machine-learning models whose predictions are combined) estimates how each course option would change each student's outcome, and a screening step removes options that would be unrealistic in practice. That machinery is hundreds of megabytes, needs statistical software to run, and reads dozens of variables, some of which only exist in the research dataset. None of it can run in a school office or a web browser.

Distillation closes the gap by keeping only the finished decisions. For every student in the training data, the full pipeline settles on one feasibility-screened recommendation, the course left standing after unrealistic options are removed. We then train a small decision tree (in effect, a short checklist of yes-or-no questions) to reproduce those recommendations. The tree reads a small set of moderators much like the original model's, such as prior achievement, math identity, confidence, and class size, and the answer it learns to give is the original model's screened recommendation itself. It is a surrogate: it copies the model's final decisions rather than its internal effect estimates, and it comes out as a few kilobytes of logic that run entirely in the browser.

We also simplify the inputs. Rather than ask for an exact test score, the questionnaire asks whether a student is below, around, or above average, which is closer to how a counselor actually knows a student. The tree reads those bands. This keeps the tool usable in a real school and lets it run entirely on the device, with no student data sent anywhere.

How closely the tool follows the full model

Because the tree is a simplification, it does not always agree with the full model:

• Plan for Grade 9 only. Agreement with the full model is about 85% under flexible placement, 73% with some flexibility, and 73% under common placements.
• Plan for all four years (Math Achievement). Agreement by grade: about 56% (Grade 9), 86% (Grade 10), 78% (Grade 11), 57% (Grade 12).
• Plan for all four years (College Enrollment). About 59% (Grade 9), 86% (Grade 10), 79% (Grade 11), 49% (Grade 12).

The four-year plan is therefore presented as illustrative, with the entry and final years the least certain. Recommendations are driven mainly by prior achievement, with math identity, confidence, class size, and each year's grades refining the path. The underlying fitted model is a black box, so no per-student explanation is shown yet; post-hoc interpretation tools (for example TreeSHAP, partial dependence, and ALE plots) are planned.

What the placement-flexibility question controls

The first question in the Grade 9 plan ("Does your school require students to follow its regular math course sequence strictly?") controls a screening threshold. Before recommending, the model checks how often students with a similar background actually take each course (the propensity score) and drops courses below a cutoff: 0.01 when plans can be tailored freely, 0.03 with some exceptions, 0.05 when the regular sequence is followed strictly. A stricter cutoff keeps recommendations to well-trodden pathways; a looser one allows rarer course jumps that a flexible school could support.

The courses

Colors encode structure: the core spine deepens in blue, the alternative course is teal, breadth electives are ochre, advanced and capstone courses are indigo and green, and the AP credential is plum. The option set differs by grade: Grade 9 offers No Math through Algebra II plus Advanced Math; Grade 10 offers the same core six plus AP/IB Math, Pre-Calculus, and Calculus; Grades 11 and 12 offer all twelve below.