Data Scientist Questions d'entretien & Réponses

Exemple de réponse

Our marketing team was spending $200K monthly on user acquisition across 5 channels but had no clear picture of ROI by channel. I built a multi-touch attribution model using Markov chains, analyzing 6 months of user journey data. The analysis revealed that one channel driving 30% of spend was contributing only 8% of conversions, while organic search was being undervalued by 3x. I presented the findings to the CMO with clear visualizations. They reallocated $60K monthly from the underperforming channel, which increased overall conversion rate by 22% within the next quarter.

Exemple de réponse

I built a customer segmentation model that recommended discontinuing a loyalty program tier. The VP of Customer Success pushed back hard — that tier had their most vocal advocates. Instead of defending the model abstractly, I dug deeper into the data and found the VP was partly right: those customers had high NPS but low revenue contribution. I revised the analysis to include lifetime value projections and advocacy-driven referral revenue. The updated model showed the tier was worth keeping but needed restructuring. We reduced the program cost by 40% while retaining the high-advocacy segment. The key lesson: models capture what you measure, and sometimes the stakeholder knows what you're not measuring.

Exemple de réponse

I deployed a demand forecasting model for an e-commerce company that performed well in backtesting but degraded badly within 3 weeks of launch. The root cause was data drift — the training data covered stable periods, but we launched right before a competitor's major price change that shifted buying patterns. I implemented a monitoring pipeline that tracked input feature distributions and model prediction distributions in real-time. When drift exceeds a threshold, the model automatically retrains on recent data. I also added a fallback to simple heuristics when confidence drops below a threshold. The experience taught me that model deployment is only half the work — monitoring and graceful degradation are the other half.

Exemple de réponse

The executive team wanted to understand why our recommendation engine sometimes suggested seemingly random products. I had 15 minutes in the board meeting. Instead of explaining collaborative filtering mathematically, I used an analogy: 'Imagine a bookstore clerk who remembers what every customer bought. When you walk in, they think about customers similar to you and recommend what those similar people loved.' Then I showed 3 real user examples where the recommendations made perfect sense once you saw the similar-user logic. I also showed 2 failure cases and explained we were addressing them with content-based filtering to complement the approach. The board approved additional budget for the recommendation team based on that presentation.

Exemple de réponse

It depends on the problem context and the cost asymmetry of errors. For a fraud detection model where positive cases are 0.1% of the data, I'd first choose the right evaluation metric — accuracy is meaningless here, so I'd use precision-recall AUC, F1, or a custom cost function that weights false negatives by their business cost. On the data side, I'd try SMOTE for synthetic oversampling, random undersampling with ensemble methods (like EasyEnsemble), or stratified sampling. On the model side, I'd use class weights to penalize misclassification of the minority class. Algorithms like XGBoost handle imbalance well with scale_pos_weight. I'd also consider anomaly detection approaches — if the minority class is rare enough, framing it as anomaly detection rather than classification can work better. The key is evaluating on a hold-out set that reflects real-world class distribution.

Exemple de réponse

Bias is the error from overly simplistic assumptions — a linear model trying to fit a quadratic relationship will always be wrong regardless of training data. Variance is the error from sensitivity to training data fluctuations — a high-degree polynomial fits training data perfectly but fails on new data. The tradeoff: reducing bias typically increases variance and vice versa. In practice, I navigate this by starting simple (high bias, low variance) and increasing complexity only when validation metrics justify it. Regularization techniques (L1, L2, dropout, early stopping) let you increase model capacity while controlling variance. Cross-validation is essential for estimating where you sit on the bias-variance spectrum. For ensembles: bagging reduces variance (Random Forest), while boosting reduces bias (XGBoost). I choose based on whether my baseline model underfits or overfits.

Exemple de réponse

First, I define the hypothesis and primary metric. For a new checkout flow, the hypothesis might be 'the new flow increases purchase completion rate.' The primary metric is conversion rate, with guardrail metrics like revenue per session and page load time. Next, I calculate sample size using a power analysis — for a 2% absolute lift from a 10% baseline with 80% power and 95% confidence, I need roughly 15K users per group. I'd randomize at the user level (not session) to avoid inconsistent experiences. I run the test for at least one full business cycle to capture day-of-week effects. For analysis, I use a two-proportion z-test for the primary metric and check for novelty effects by examining the metric trajectory over time. I also segment results by key user cohorts — the new flow might help new users but hurt power users. Finally, I consider multiple comparison corrections if testing multiple metrics simultaneously.

Exemple de réponse

L1 (Lasso) adds the absolute value of weights to the loss function, while L2 (Ridge) adds the squared weights. The key practical difference: L1 drives weights to exactly zero, performing automatic feature selection. L2 shrinks weights toward zero but never reaches it, keeping all features with reduced influence. I use L1 when I suspect many features are irrelevant and I want a sparse, interpretable model — common in high-dimensional datasets like genomics or text. I use L2 when most features contribute some signal and I want to prevent any single feature from dominating — typical in well-curated feature sets. Elastic Net combines both and is my default when I'm unsure: it gets L1's sparsity with L2's stability for correlated features. The regularization strength (lambda) is always tuned via cross-validation.

Exemple de réponse

First, I'd resist the urge to start modeling. I'd spend the first 2-3 days on exploratory data analysis: profiling every column for missing rates, distributions, outliers, and inconsistencies. I'd document what I find and present it to the data owner — often, they can explain anomalies that would otherwise waste weeks of investigation. For missing values, my approach depends on the mechanism: if missing completely at random, imputation (median for numeric, mode for categorical, or model-based imputation) works. If missing not at random, the missingness itself is informative and I'd encode it as a feature. I'd set up data validation checks (Great Expectations or similar) to catch future quality issues at ingestion time. Only after establishing a clean, understood dataset would I start modeling — and I'd keep the first model simple to establish a baseline before adding complexity.

Exemple de réponse

I wouldn't just say 'no' or silently compromise quality. I'd break the work into layers of value. By Friday, I could deploy a simple collaborative filtering model using user-item interactions — it won't be perfect, but it'll outperform the current random suggestions. I'd present this as Phase 1 with clear limitations documented. Phase 2 (week 2-3) would add content-based features and handle the cold-start problem for new users. I'd outline what performance improvement they can expect from each phase with estimated metrics. This approach delivers real value immediately while setting expectations for the full solution. I'd also flag that rushing the full model into Friday's deadline would mean skipping offline evaluation and A/B testing — which means shipping with no idea if it actually helps users.

Exemple de réponse

I'd flag this immediately — not after deployment, not in a retrospective. I'd document the concern with evidence showing the proxy correlation (e.g., zip code to demographic data mapping) and present it to both the technical lead and a business stakeholder. Then I'd test the model's performance with and without the feature. Often, removing the proxy feature has minimal impact on overall accuracy but significantly reduces disparate impact. If the feature is genuinely necessary for performance, I'd explore fairness-aware modeling techniques: equalized odds post-processing, adversarial debiasing, or calibration across protected groups. I'd also recommend implementing fairness metrics as part of the model's evaluation pipeline — not just accuracy, but demographic parity and equalized opportunity. The business risk of deploying a discriminatory model (legal, reputational, ethical) far outweighs the marginal accuracy gain.

Exemple de réponse

I take 'doesn't feel right' seriously — domain experts often catch issues that metrics miss. First, I'd ask for specific examples of predictions that felt wrong and look for patterns. Common causes: the model optimizes for the wrong metric (high accuracy but poor calibration), the test set doesn't reflect real-world distribution, or the model captures statistical patterns that violate business logic. I'd examine the model's predictions on their specific examples using SHAP or LIME to explain individual predictions. If the model is technically correct but violates domain expectations, I might need to add business rule constraints or adjust the loss function to penalize certain types of errors more heavily. I'd also check for data leakage — a suspiciously high test score combined with business skepticism is a classic leakage signal.