I have had the good fortune of sitting in on numerous design reviews, conducting design interviews, and working with brilliant engineers throughout my career. I have learned a lot during this process. One of the most important lessons is developing the art of "engineering judgement".
The engineers I'd call genuinely cracked have a nose for smelling a bad design fairly quickly. Their process is different from others: they notice cues, figure out which constraints are more important than others, realise when common patterns no longer apply, etc.
You can't learn this skill with just diagrams and in an ordinary design review. One technique that has served me well over the years in building good engineering judgement is the "Shadowbox" technique.
A common consensus among engineering circles on X/HN/Reddit is to develop "taste" in software engineering with coding being rapidly automated (or already has, idk) by AI. My strong belief is that the first step in building taste is developing excellent engineering judgement. This is true for senior engineers and junior engineers alike. Even more so for junior ones because they haven't had the experience or opportunity to work on large-scale and complex engineering systems.
With AI becoming more and more accessible with each passing day and models becoming smarter, we can combine it with the Shadowbox technique to become cracked engineers ourselves.
What is the ShadowBox technique?
In this, we take a real situation and divide it into many decision checkpoints. At each checkpoint, we make a decision about the situation and rank the possible actions. We note down the following:
- the cues or signals
- assumptions made
- acceptable failures
Once we do this, we reveal the expert answer corresponding to that checkpoint. From there, we match the cues we made and the cues the expert made.
Then we change one condition and rank again. We keep doing this until we come up with a conditional rule: under condition X, choose approach Y, because the mechanism satisfied requirement Z, and reconsider it when signal W changes.
The original ShadowBox evals showed that just a short training with this technique improved alignment with expert decisions by 28%. But the number is not the point. The point is that you commit before you see the answer. Doing this over and over helps you develop a real mental model. Let's now talk about concrete steps on how we apply this technique in system design.
Step 1 - Start From a Production Case
First, we choose a source about actual architecture design, failures, postmortems, etc. A source can be:
- an engineering blog explaining a migration
- an Architecture Design Record (ADR)
- a DB- or infra-related paper
- a production postmortem from a real system
- a conference talk that includes constraints and outcomes
We should not start with something that only mentions "what worked". A real case gives us information about what cues the team saw, what they missed, why they missed that, what happened next, how they make sure this doesn't happen again, etc.
Remember, not every case is a good source. Expert intuition is trustworthy only when the environment has regularities, and the expert has had a chance to learn them. So, I prefer sources where there are many expert voices, disagreements, or proof that the decision was made under conditions someone could actually learn from.
You can add these constraints to your prompt and ask AI to only get sources that follow them.
Step 2 - Ask AI to Keep the Decision Hidden
We can now give a chosen source to AI and ask it to extract the following from it:
- business/technical context
- constraints at each checkpoint
- important decisions
- credible options
- what was chosen
- what happened afterwards
Then, we ask AI to provide only what was known before the first decision and hide everything else from us, including the decision made and the outcome.
For example:
The notification service receives large traffic bursts. One provider has unpredictable latency, duplicate delivery is harmful, and the team has limited operational capacity. Choose how work should move from the API to provider workers.
This is the situation you are dealing with. You don't have the answer yet.
Step 3 - Rank the Options Before Looking At the Answer
At this point, you ask AI to give you three options. For example, three options in the above situation may be:
- Call providers synchronously
- Store pending work in the database
- Use a managed queue
Now, you rank these options as per your understanding and make a note of five things:
- Which cues matter
- What assumptions you have made
- What are your expectations for each option - what it can cause
- Which failures are acceptable to you
- Your confidence in the ranking
By doing these steps, you have submitted your mental model about the problem. It may not be correct yet, but it's a start.
This is why it is the most important step. If you miss this, you are just reading a paper or going through a case study.
Step 4 - Compare Your Cues With Experts' Cues
This is the point where AI shows us what the engineers chose and why. This is strictly done using the source. Here, we need to compare two things: choices and, most importantly, cues.
Compare along the lines of:
- Which information did they treat as important?
- Which information did you overvalue?
- Which consequence did they anticipate that you missed?
- Did you reach the same choice through weaker reasoning?
- Did your different choice fit constraints that weren't there?
Matching the final architecture is barely sufficient. Sometimes we match the final answer with reasoning that would fail in the next case. That's why cues are important: they strengthen your architectural judgement.
I also ask AI to categorise every rationale: is it mentioned in the source, or is it a hypothetical that AI itself generated? At any point, we should not let AI "invent" an expert answer. If an option or rationale is missing from the source, the AI should mark it as inferred.
Step 5 - Change a Cue, and Rank Again
AI now changes one condition. For our notification service example above, it can be:
- Traffic becomes steady
- Duplicate delivery is allowed
- The engineering team knows Kafka well
- Provider latency becomes predictable
- Messages must be delivered within two seconds
We now need to rank the options again and explain exactly why the ranking changed. This is important because it identifies the boundary of the decision. For example:
A managed queue is preferable when bursts exceed provider capacity and delayed completion is acceptable. Predictable low traffic may make direct delivery simpler.
By doing this, we are learning when a component actually belongs in a design.
A warning: in real prod systems, cues are usually coupled. Changing traffic can also change failure modes, load, and cost. So, swapping only one cue can make you learn the wrong boundary.
Therefore, sometimes we need to change a group of related constraints and figure out what changed the decision. This is definitely harder, but this is how a real architecture design works.
Step 6 - Failures in a system are good decision points
Failures in a system teach us perhaps the most. Even more than happy paths. Therefore, postmortems of a public outage are gold for building intuition on architecture.
For a postmortem, AI stops just before the incident becomes unavoidable. We then choose what to look into or change. Then, we ask AI to show the next event from the source document.
This is error management training.
We make choices, see consequences, and come up with strategies for unfamiliar cases. This is what we want: identifying the signals for the next incident instead of memorising the one we are studying.
Step 7 - Extract a conditional rule
After each case, write:
Under these conditions, choose this approach, because this mechanism satisfied this requirement. It introduced this cost, and we need to reconsider it when this signal changes.
If you do this for enough problems, you will have a collection of conditional decisions. This is more valuable than a collection of architectural diagrams.
Step 8 - Test the rule on an unrelated system
We can ask AI to give us a different problem with the same constraints. If the first problem was related to notification delivery, a different problem can be related to image processing, payment reconciliation, webhook delivery, etc.
The goal is to build the muscle for recognising a shared structure without being told the topic.
And this is how intuition develops: repeated exposure to identify important cues, the interactions between them, and the limits of a decision.
What AI should and should not be
In this learning technique, AI should be treated as the dumb partner to do the heavy lifting and doing the grunt work. It can be someone who:
- extracts cases
- reveals information slowly
- generates controlled counterfactuals
- compares your reasoning and the source's reasoning
AI is not the expert, so it should not invent the expert answer. It should be clear when it is certain and when it is guessing. We should treat AI as a scaffold for attention, not an oracle for judgement.
The habit that changed my judgement
I have personally felt a change in my approach after applying this technique. When I come out of an architecture meeting, I have two takeaways: how I defended my design, and the cues I missed.
Defending your design can be the final step, but shadowing real decisions is how we build architectural judgement.
If you genuinely want to become a cracked engineer, you need to stop asking AI to praise or attack your design. You need to ask it to hide the correct answer and make you commit first.
I originally published this on X as well, but included the complete article here so that the discussion can remain on Reddit.