Thesis Workbench

How it works

From a question to a thesis you can defend

The easiest mistake in forming an opinion is to go straight from hearing an influential voice to adopting their conclusion. The workbench puts five stages and one habit between the two.

Open the workbench

Five stages, one question each

You move left to right. Each stage has a short checklist that tells you when you have done enough to move on, so you never have to guess whether you are finished.

Before you startWrite your gut take and how sure you are. You will compare it with where you end up.

    A review date, a ticked signpost, or a resolved prediction sends you back to stage 1 with a better starting view.

    Each stage answers one question, guards against one trap, and hands a concrete result to the next.

    One habit: three lines per session

    Every time you read, watch, or test something, leave with one observation, one explanation, and one prediction. The observation keeps you honest about what you saw. The explanation forces you to name a mechanism. The prediction makes the explanation checkable.

    • Written during the session
    • Happens later, when the date arrives
    The first three steps take a few minutes. The last three are what turn a pile of notes into a track record.
    A worked session
    Observation
    Coding agents can complete some well-specified programming tasks but struggle with certain complex, long-running workflows.
    Explanation
    Success depends on context, feedback quality, tooling, and the ability to detect and recover from mistakes.
    Prediction
    Agents will become reliable at a broader range of development tasks as their verification and planning systems improve, though full project autonomy may remain uneven. 70% · check in 9 months

    Confidence is a number, with a word attached

    "Likely" means 55% to one reader and 90% to another, and a word cannot be scored afterwards. So you state a number, and the workbench attaches the word that the US intelligence community uses for that range.

      The bands are uneven on purpose. "Likely" covers 25 points, "roughly even odds" only 10. Keep the exact number: in a study of 888,328 forecasts, rounding numbers into two or three verbal buckets made them less accurate.

      Keeping score

      Every prediction gets a probability and a date. When the date arrives you mark it as happened or not, and the Predictions page compares how often things happened with how likely you said they were.

      An illustration, not your data. Dots on the line mean stated odds matched reality. Here, predictions made at 90% came true 71% of the time, so this forecaster is overconfident at the high end. A dot above the line would mean the opposite.

      The Brier score is the average of (what you said minus what happened), squared.

      You saidItScore
      70%happened(0.7 − 1)² = 0.09
      70%did not happen(0.7 − 0)² = 0.49
      50%either0.25

      Zero is perfect. Always saying 50% scores 0.25, so anything below that means your numbers carry information.

      Read the chart as a rough sketch until you have a few dozen resolved predictions. With ten predictions in a group, luck alone can move the hit rate by nearly 30 points.

      Where the methods come from

      The stages borrow from people whose job is to commit to a view and be held to it. Every prompt in the workbench traces back to one of their habits.

      Each cell names the habit borrowed from that tradition for that stage. Details and sources follow.

      What the evidence says

      Structured thinking tools are easier to praise than to prove. These four findings shaped how much structure the workbench asks of you.

      Making it yours

      This version is deliberately generic. Everything a niche version would change sits in one file, js/framework.js:

      A version for judging AI research claims, for instance, might swap the source kinds for papers, benchmarks, demos and vendor announcements, and add a reproduction checklist to the Experiment stage.

      Open the workbench