Interview rubric
How ML System Design Interviews Are Scored: Rubric by Level (2026)
A ml system design interview is scored on 6 dimensions: problem framing (20%), data and features (20%), modelling choices (15%), evaluation (20%), serving and operations (10%), communication (15%). Each is rated 1 to 4 against anchors that rise with the level. At mid level, an outstanding score on problem framing means: "Compares two framings, such as ranking against classification, and picks one with reasons."
The dimensions and their weights
| Dimension | Weight | Solid (3) at mid level |
|---|---|---|
| Problem framing | 20% | Translates the business goal into an ML objective and labels, and confirms ML is needed. |
| Data and features | 20% | Covers sources, labelling and features, and raises leakage or skew when relevant. |
| Evaluation | 20% | Aligns offline metrics to the goal and plans an online A/B test. |
| Modelling choices | 15% | Starts with a baseline and chooses a model with stated trade-offs. |
| Communication | 15% | Follows a clear structure from framing to serving and manages time. |
| Serving and operations | 10% | Covers latency, a feature store or cache, drift monitoring and retraining when asked. |
What changes between levels
On problem framing, the heaviest dimension, an outstanding mid-level answer "compares two framings, such as ranking against classification, and picks one with reasons", while an outstanding senior answer "frames the objective, names proxy-label risks and how they could mislead the business". The same answer can score 4 at mid level and 2 at senior: senior candidates are expected to raise things unprompted that mid-level candidates may be asked about.
Every score, at every level
Problem framing (20%)
| Level | 1 · Poor | 2 · Borderline | 3 · Solid | 4 · Outstanding |
|---|---|---|---|---|
| Intern / new grad | Jumps to a model without saying what it should predict. | States the prediction target only after the interviewer suggests it. | States a sensible prediction target and label when guided. | States the target, the label and whether it is classification or ranking unprompted. |
| Junior | Jumps to a model without defining what it predicts. | Defines a target only after prompting and does not link it to the goal. | Defines the ML objective and labels and links them to the stated goal. | Defines objective and labels and asks how predictions will be used in the product. |
| Mid-level | Jumps to a model without defining what it predicts. | Defines an objective that does not match the business goal. | Translates the business goal into an ML objective and labels, and confirms ML is needed. | Compares two framings, such as ranking against classification, and picks one with reasons. |
| Senior | Jumps to a model without defining what it predicts. | Frames the objective correctly but only after prompting on the business goal. | Frames the objective unprompted, questions whether ML is needed and names a simple baseline. | Frames the objective, names proxy-label risks and how they could mislead the business. |
| Staff | Jumps to a model without defining what it predicts. | Frames the objective well but ignores the wider product and organisation. | Frames the objective against business strategy and the teams that consume the output. | Frames the problem including responsible-AI risks and which teams own each decision. |
Data and features (20%)
| Level | 1 · Poor | 2 · Borderline | 3 · Solid | 4 · Outstanding |
|---|---|---|---|---|
| Intern / new grad | Cannot say what data the model would learn from. | Names a data source only with heavy hints. | Names data sources and a few sensible features when guided. | Names data sources and features and explains how labels are collected. |
| Junior | Cannot say what data the model would learn from. | Names data sources but no usable features. | Names data sources, features and how labels are collected. | Also raises one data risk such as class imbalance unprompted. |
| Mid-level | Cannot say what data the model would learn from. | Lists features but ignores labelling and data quality. | Covers sources, labelling and features, and raises leakage or skew when relevant. | Raises leakage and training-serving skew unprompted with concrete safeguards. |
| Senior | Cannot say what data the model would learn from. | Covers features but raises leakage or skew only when asked. | Covers labelling, leakage, skew and imbalance unprompted with concrete fixes. | Designs a feature pipeline that prevents skew and explains how labels are kept fresh. |
| Staff | Cannot say what data the model would learn from. | Covers data well for one model but not shared data needs. | Covers data quality and shared feature infrastructure across teams. | Covers data governance, consent and bias in the data, and platform reuse. |
Modelling choices (15%)
| Level | 1 · Poor | 2 · Borderline | 3 · Solid | 4 · Outstanding |
|---|---|---|---|---|
| Intern / new grad | Cannot name any model that fits the problem. | Names a model but cannot say why it fits. | Names a simple model that fits and explains how it learns, at a basic level. | Starts with a simple baseline and names when a more complex model would help. |
| Junior | Names a model that does not fit the problem type. | Names a fitting model but no baseline or reason. | Starts with a baseline and names a stronger model with a basic reason. | Starts with a baseline and compares two models on accuracy and cost. |
| Mid-level | Names a model that does not fit the problem type. | Jumps to a complex model with no baseline. | Starts with a baseline and chooses a model with stated trade-offs. | Compares model families on accuracy, latency and interpretability and picks with reasons. |
| Senior | Names a model that does not fit the problem type. | Picks a reasonable model but cannot defend it against alternatives. | Defends the model choice against alternatives given latency, data size and interpretability. | Plans the path from baseline to advanced model with the evidence needed at each step. |
| Staff | Names a model that does not fit the problem type. | Defends one model but ignores platform and cost. | Chooses models with platform, cost and maintenance in view. | Chooses models with cost, maintenance and fairness risks weighed for the organisation. |
Evaluation (20%)
| Level | 1 · Poor | 2 · Borderline | 3 · Solid | 4 · Outstanding |
|---|---|---|---|---|
| Intern / new grad | Cannot say how to tell whether the model works. | Names accuracy only, even when it does not fit. | Names a fitting offline metric such as precision or recall when guided. | Names a fitting offline metric and explains why it suits the problem. |
| Junior | Cannot say how to tell whether the model works. | Names a metric that does not match the goal. | Names a fitting offline metric and a held-out test set. | Names a fitting offline metric and suggests an online test. |
| Mid-level | Cannot say how to tell whether the model works. | Names offline metrics only, not tied to the business goal. | Aligns offline metrics to the goal and plans an online A/B test. | Also adds guardrail metrics and explains when offline and online results disagree. |
| Senior | Cannot say how to tell whether the model works. | Plans offline and online tests but no guardrails. | Plans aligned offline metrics, an A/B test and guardrails unprompted. | Plans evaluation by segment and explains how to catch regressions for minority groups. |
| Staff | Cannot say how to tell whether the model works. | Plans sound tests but ignores long-term and fairness effects. | Covers long-term effects, fairness checks and shared evaluation standards. | Sets an evaluation framework other teams can adopt, including responsible-AI review. |
Serving and operations (10%)
| Level | 1 · Poor | 2 · Borderline | 3 · Solid | 4 · Outstanding |
|---|---|---|---|---|
| Intern / new grad | Cannot say how the model would be used after training. | Says the model goes into an app but cannot say how. | Describes serving predictions through an API when guided. | Describes serving through an API and retraining when data changes. |
| Junior | Cannot say how the model would be used after training. | Mentions serving but not latency or retraining. | Covers batch or online serving and basic retraining. | Covers serving latency and monitoring for drift unprompted. |
| Mid-level | Ignores serving and monitoring even when asked. | Covers serving only when asked, with vague answers. | Covers latency, a feature store or cache, drift monitoring and retraining when asked. | Covers serving, drift monitoring and retraining triggers unprompted. |
| Senior | Ignores serving and monitoring even when asked. | Covers serving but raises drift only when asked. | Covers latency budgets, drift monitoring and retraining unprompted. | Covers safe rollout, fallback models and alerting on prediction quality. |
| Staff | Ignores serving and monitoring even when asked. | Covers serving for one model only. | Covers shared serving platform, cost and on-call ownership. | Covers platform reuse, cost and incident response for model failures across teams. |
Communication (15%)
| Level | 1 · Poor | 2 · Borderline | 3 · Solid | 4 · Outstanding |
|---|---|---|---|---|
| Intern / new grad | Stays silent or cannot explain the approach when asked. | Explains only when asked and loses track of the plan. | Explains the approach step by step when guided by the interviewer. | Explains the approach in a clear order and checks in with the interviewer. |
| Junior | Stays silent or explains the approach incoherently. | Follows the interviewer's lead but cannot structure the answer. | Explains clearly while the interviewer sets the agenda. | Explains clearly and proposes the next part of the system to discuss. |
| Mid-level | Stays silent for long stretches or jumps between data, model and serving with no thread. | Needs the interviewer to set every step of the agenda. | Follows a clear structure from framing to serving and manages time. | Drives with a stated agenda and adapts to questions without losing the thread. |
| Senior | Gives a disorganised walkthrough that the interviewer has to structure for them. | Structures the answer but the interviewer has to pull out the depth. | Drives the session, manages time and explains trade-offs as they arise. | Drives confidently, handles pushback with evidence and summarises risks. |
| Staff | Cannot present the design coherently, even with the interviewer setting the agenda. | Drives the structure but runs out of time before depth. | Runs the session like a design review, with most time on depth. | Runs a crisp review, invites challenge and summarises risks and next steps. |
Get scored against this rubric
Thita's AI interviewer runs a live ml system design round and scores you against exactly this rubric, dimension by dimension, for the level you choose.
Practise an ML system design roundWhere this rubric comes from
These rubrics are researched: they follow the scoring methods big-tech companies describe publicly (behaviourally anchored ratings, bar raisers, leadership principles, named interview signals) and how their rounds are commonly run. No company publishes its actual scoring sheet, so the level-by-level wording is our own synthesis. It is the same rubric our AI interviewer scores every mock against.
Questions, answered
- How is a ml system design interview scored?
- A ml system design interview is scored on 6 dimensions: problem framing (20%), data and features (20%), modelling choices (15%), evaluation (20%), serving and operations (10%), communication (15%). Each is rated 1 to 4 against anchors that rise with the level. At mid level, an outstanding score on problem framing means: "Compares two framings, such as ranking against classification, and picks one with reasons."
- What is the difference between a mid-level and a senior ml system design answer?
- On problem framing, the heaviest dimension, an outstanding mid-level answer "compares two framings, such as ranking against classification, and picks one with reasons", while an outstanding senior answer "frames the objective, names proxy-label risks and how they could mislead the business". The same answer can score 4 at mid level and 2 at senior: senior candidates are expected to raise things unprompted that mid-level candidates may be asked about.
- What does a score of 3 mean?
- Scores are relative to the level you are interviewing for. A 3 means solid for that level: the bar is met without help on the key points. A 4 means the candidate also raised the important points unprompted. A 2 usually means the interviewer had to lead the candidate to them.
- Is this the official rubric used by Google, Meta or Amazon?
- These rubrics are researched: they follow the scoring methods big-tech companies describe publicly (behaviourally anchored ratings, bar raisers, leadership principles, named interview signals) and how their rounds are commonly run. No company publishes its actual scoring sheet, so the level-by-level wording is our own synthesis. It is the same rubric our AI interviewer scores every mock against.