Making a test prep app that learns how you study

The CPA exam isn't one test. It's four: Financial Accounting and Reporting (FAR), Auditing (AUD), Regulation (REG), and Business Environment and Concepts (BEP). Each covers different material, runs a different length, and has its own difficulty curve. FAR is content-heavy and wide. REG is dense with tax code. AUD requires judgment calls that don't reduce to formulas. A static question bank treats them all the same, which is the first mistake most prep apps make.

I built CPA Exam Prep because I kept hearing from accountants that their study tools felt like they were fighting the material instead of learning it. They'd grind through 200 questions on a topic they already knew, then run out of energy before touching the topics they were weak on. The app needed to figure out what each user didn't know and push them there.

How the adaptive algorithm works

The system tracks three signals per user per topic. Correctness -- straightforward, but tracked at the subtopic level, not the section level. Getting governmental accounting right doesn't mean you know lease accounting. Time spent per question. Someone who gets it right in 30 seconds knows it differently than someone who stares at it for four minutes and picks the same answer. And confidence patterns. If a user frequently changes their selected answer before submitting, that's a signal. They're uncertain even when they end up correct.

After each session, the algorithm reweights the next session's question selection toward weak areas. It's not purely remedial -- you still see some questions from strong topics to maintain retention -- but the balance shifts. A user who's solid on AUD ethics but shaky on audit sampling will see more sampling questions without being told why. The goal is to make the practice feel naturally focused rather than punitive.

Bootstrapping difficulty scores

CPA questions don't come with official difficulty ratings. The AICPA doesn't publish that data. So when I launched, every question started at a neutral difficulty score and I let actual user performance calibrate them. Questions that 90% of users answered correctly within a minute drifted toward easy. Questions where most users got stuck or answered wrong moved toward hard. Took about 2,000 completed sessions before the scores stabilized into something useful. Early users were essentially beta testing the difficulty curve without realizing it.

Scores still update, just slowly now. New questions added by content reviewers start at a neutral baseline and settle within a few weeks.

Simulating the real thing

Timed sections matter more than people expect. I've watched users who score 85% in untimed practice drop to 60% when a countdown clock appears. The psychological weight of a timer changes how people read questions, how carefully they eliminate wrong answers, how often they second-guess themselves. The app's practice exam mode replicates real timing constraints per section. It also tracks answer changes -- not in scoring, but in the analytics. If you're changing answers frequently under time pressure, the app flags that pattern and suggests strategies for committing to your first instinct when you're not sure.

The 20-minute finding

This one surprised me. I pulled study pattern data after about six months and segmented users by their eventual pass/fail outcome on the actual exam. Users who studied in sessions of roughly 20 minutes, five days a week, passed at meaningfully higher rates than users who did two-hour marathon sessions on weekends. Total study time was similar between the groups. Frequency and consistency mattered more than volume.

I added a nudge based on this. After onboarding, the app suggests a daily 20-minute goal and sends reminders. I was careful not to frame it as a rule -- just a "here's what worked for other users" note. Retention improved after adding it, though I can't fully separate that from other changes shipped around the same time.

The analytics dashboard

The dashboard shows weak topics per section, recent accuracy trends, and a predicted readiness score. That last one was controversial in my own head. I built a predicted pass probability based on historical correlations between in-app performance and reported exam outcomes. It works reasonably well. But showing someone "38% likely to pass FAR" could make them give up instead of study harder.

I kept it in, with guardrails. The probability only appears after a minimum number of completed sessions, so it's not reacting to a small sample. And it's framed alongside specific recommendations -- "your governmental accounting accuracy is 45%, improving this topic would have the biggest impact on your FAR readiness." A bare number would be demoralizing. Paired with what to work on next, it seems to motivate people. At least, that's what the retention data says so far.

Keeping the content accurate

CPA exam content changes. Tax law updates, new FASB standards, revised AICPA blueprints. Every question in the app goes through a review pipeline with CPA-licensed accountants who verify accuracy against current standards. I contract with three reviewers who rotate sections. When standards change, affected questions get flagged, reviewed, and either updated or retired. Least glamorous part of the app and probably the most important. An adaptive algorithm that efficiently teaches you wrong information is worse than a static question bank with correct answers.

-- Daniel