Experience builds pattern recognition. It also builds overconfidence in the patterns you have learned to recognize — including the ones that are wrong.
Daniel Kahneman spent a career documenting the systematic ways in which human judgment fails — not because people are unintelligent, but because the cognitive shortcuts that make decision making efficient also introduce predictable errors. His framework distinguishes between System 1 thinking (fast, automatic, intuitive) and System 2 thinking (slow, deliberate, analytical). Most leadership decisions happen in System 1 and are then rationalized by System 2 after the fact.
In higher education, this matters in a specific and consequential way: the leaders making decisions about student success programs are almost never the students those decisions affect. They are experienced professionals who developed their intuitions about student behavior in contexts that may bear little resemblance to the students they currently serve.
A VP who graduated in 1994, rose through the ranks at predominantly white residential institutions, and has never experienced food insecurity carries a set of pattern-recognition tools that will systematically misread the experience of the current community college population. This is not a character flaw. It is a cognitive architecture problem — and it has a structural solution.
"The institution that says 'we know our students' and cannot show you the data to prove it is operating on institutional mythology. Mythology feels like wisdom. It produces the same outcomes every time: confident decisions that keep missing the same students."
The institutions that retain the most students are not the ones with the most sophisticated data systems. They are the ones that have built cultures where advisors have time to notice, capacity to act, and frameworks to guide their response...
This Content is for Members
Subscribe to The Praxis Brief for full access to every issue, the complete archive, practitioner toolkits, and monthly deep-dive PDFs.
- 1Name the assumption before the decisionEvery institutional decision rests on assumptions about how people behave. Write the assumption down before acting on it. If you cannot articulate it, you cannot test it.
- 2Identify the counter-evidenceBefore finalizing any significant student success decision, assign one person the role of presenting the best evidence against the proposed course of action. This is a structural defense against confirmation bias, which is the most pervasive cognitive error in institutional leadership.
- 3Disaggregate the data before concludingAggregate metrics hide the populations your programs are failing. A 78% first year retention rate that looks stable may contain a 65% rate for a specific demographic group that has been improving for everyone else. Always disaggregate before concluding that an intervention is working.
- 4Build in pre-mortemsBefore launching a new initiative, run a pre-mortem: "Assume this program fails in 18 months. What caused it?" This exercise reliably surfaces implementation risks and stakeholder misalignments that optimism bias suppresses in conventional planning.
- 1Assumption surfacingPaste a proposed initiative into Claude or GPT-4o with this prompt: "What assumptions does this plan require to be true in order to succeed? What is the evidence for and against each assumption?" The model will surface blind spots your internal team has normalized.
- 2Literature cross-checkBefore claiming that an intervention is evidence-based, use AI to conduct a rapid literature review: "What does the research say about this intervention type in this population context? What are the conditions under which it works and where has it failed?"
- 3Disaggregation promptingWhen reviewing outcome data, use AI to generate disaggregation scenarios: "This program shows a 12% improvement in first year persistence. Generate 5 ways this aggregate number could mask disparate outcomes for specific student subgroups." Then check those scenarios against your actual data.
Score 6 to 8: evidence informed culture. Score 4 to 5: structural gaps present. Below 4: operating primarily on institutional mythology.
- We can name at least 3 institutional assumptions about student behavior we have tested with data in the past 12 months
- Our leadership team includes people with formal training in research methodology or data analysis
- When we launch a new initiative, we define success metrics before we begin — not after we need to report results
- We routinely disaggregate all student outcome data by race, first generation status, Pell eligibility, and age
- We have discontinued at least one program in the past 3 years because evidence showed it was not working
- Staff and faculty have formal pathways to surface evidence that contradicts leadership assumptions without professional risk
- Our strategic planning process includes explicit review of what has not worked and why
- We can describe, in specific terms, what the research says about at least one of our core student success strategies
That wraps the first six issues of The Praxis Brief. If these ideas have been useful, share this with one person in your network who is trying to solve these problems in their institution. That is how this grows.