Predict pass/fail yield from semiconductor fab sensor data, with explanations a process engineer can act on. An end-to-end ML application built around the two things that actually make this problem hard: severe class imbalance (6.6% of wafers fail) and explainability (which sensors drove a yield excursion).
The full chain is here — reproducible data pipeline → leakage-safe preprocessing → imbalance-aware model selection → honest evaluation → SHAP → FastAPI service → Streamlit dashboard — framed in fab/process terms throughout.
Headline (held-out test set): the model catches 18 of 21 defective wafers (86% recall on FAIL) at ROC-AUC 0.76 / PR-AUC 0.23, at the cost of re-testing some good wafers. Accuracy is deliberately not the headline: a model that calls everything "pass" scores 93% and catches zero defects.
A fab runs each wafer through hundreds of process steps, logging ~590 in-line sensor/process measurements (temperatures, pressures, flows, etc.). At the end, in-house testing labels the wafer pass or fail. Predicting that label early from the sensor trace lets engineers pull suspect wafers for re-test and, more importantly, trace a yield excursion back to the responsible measurement.
What makes it hard isn't raw accuracy:
- Fails are rare (~6.6%). Accuracy is meaningless — see below.
- The two mistakes cost very differently. A missed defective wafer (false negative) compounds scrap/rework cost downstream; a false alarm (false positive) just means re-testing a good wafer. The model should favour catching fails.
- A flag is only useful if it's explainable. "Wafer 1183 is high-risk" is noise; "high-risk, driven by sensors S60 and S104" is a place to start looking.
SECOM: 1567 wafers × 590
sensors, binary pass/fail labels, 104 fails / 1463 pass. It is genuinely
messy: ~4.5% of cells missing, 116 constant sensors (+122 quasi-constant) that
read the same for every wafer, heavy-tailed ranges, and a label encoded as
-1 = pass / 1 = fail (remapped to 0/1 so the rare FAIL class is positive).
Sensors are anonymised — there is no published mapping to physical quantities —
so they're referred to as S1…S590 and treated as process measurements at some
step of the line. Fetch it with:
python -m scripts.download_data # writes data/raw/secom.data + secom_labels.data![]() |
![]() |
With a 6.6% fail rate, predict-always-pass scores 93.4% accuracy while catching nothing. So accuracy is reported only to be dismissed; the model is selected and evaluated on metrics that respect the rare class: PR-AUC (primary), recall/precision on FAIL, balanced accuracy, and G-mean.
Three imbalance strategies are compared head to head — cost-sensitive class
weights, SMOTE oversampling, and random undersampling — each applied
inside the cross-validation folds (via an imbalanced-learn pipeline) so the
validation fold always keeps the real fail rate. There is no resampling leakage.
590 features against 1567 wafers (~⅓ the sample size) is a strong overfitting
risk, so dimensionality reduction is essential. The pipeline (src/preprocessing.py)
learns every parameter on the training fold only:
- Drop high-missingness sensors (offline most of a run → little signal)
- Median-impute (robust to excursion spikes; "assume nominal")
- Drop zero-variance + quasi-constant sensors (no discriminative power)
- Drop near-duplicate sensors (|corr| > 0.95 — fab sensor banks are redundant)
- Standardize (sensors span very different units/scales)
- Select the top 40 sensors by univariate ANOVA F
Selection over PCA — on purpose. PCA would reduce dimensionality just as well, but its components aren't physical sensors, which would break the SHAP root-cause story. Keeping real sensor IDs is what lets an explanation say "look at S60". The exact same pipeline object is reused in training and serving, so there is one definition of "clean a wafer" and no train/serve skew.
Stratified 5-fold CV, ranked by PR-AUC (mean ± std across folds):
| Model | Imbalance strategy | PR-AUC | ROC-AUC |
|---|---|---|---|
| Random forest | class weights | 0.195 ± 0.065 | 0.718 |
| XGBoost | class weights | 0.183 ± 0.029 | 0.706 |
| XGBoost | undersample | 0.182 ± 0.065 | 0.676 |
| XGBoost | SMOTE | 0.181 ± 0.041 | 0.706 |
| Random forest | SMOTE | 0.178 ± 0.065 | 0.692 |
| Random forest | undersample | 0.172 ± 0.045 | 0.691 |
| Logistic regression | SMOTE | 0.164 ± 0.050 | 0.653 |
| Logistic regression | class weights | 0.163 ± 0.049 | 0.654 |
| Logistic regression | undersample | 0.161 ± 0.078 | 0.658 |
PR-AUC sits around 0.16–0.19 for everything and ROC-AUC around 0.65–0.72 — SECOM is a hard, noisy signal, so the value of this project is doing the rare-event problem correctly, not chasing a leaderboard number. The top rows are within one fold-std of each other (effectively a tie); the random forest with class weights is taken as the production model and tuned with a randomized search.
| Metric | Value | Reading |
|---|---|---|
| Recall (FAIL) | 0.86 | catches 18 of 21 defective wafers |
| Precision (FAIL) | 0.13 | most flags are precautionary re-tests |
| ROC-AUC | 0.76 | ranks fail-risk well above chance |
| PR-AUC | 0.23 | ~3.5× the 0.066 always-fail baseline |
| Balanced accuracy | 0.72 | |
| G-mean | 0.70 | sensitivity × specificity balance |
| Accuracy | 0.60 | (shown only for contrast — see below) |
![]() |
![]() |
The cost trade-off, made explicit. The decision threshold is tuned on
out-of-fold training predictions to favour recall on FAIL (F-β, β=2), landing
at fail probability ≥ 0.05. At that operating point the model behaves as a
screening tool: it flags ~45% of wafers for re-test to make sure it catches
nearly all the real defects. That's the right bias when a missed defect costs far
more than a re-test — and the threshold is a single knob that can be re-tuned to a
fab's actual cost ratio. Accuracy drops to 0.60 precisely because the model is
willing to be wrong on cheap false positives to avoid expensive false negatives.
Because real sensor IDs survive preprocessing, SHAP attributes risk to physical measurements an engineer can investigate.
- Global — mean |SHAP| ranks the line's top yield-driving sensors (where to look first for an excursion).
- Per-wafer — every prediction comes with the sensors that pushed that wafer toward fail or pass.
A FastAPI service owns the model; the Streamlit dashboard is a thin client
that calls it — prediction logic lives in exactly one place (src/inference.py).
API (POST /predict):
curl -s localhost:8000/predict -H 'content-type: application/json' \
-d '{"sensors": {"S60": 8.9, "S104": -0.008, "S511": 74.3}}'Other endpoints: GET /health, /model/info, /sensors, /examples, /insights/global.
Dashboard — a Wafer risk assessment tab (pick an example wafer or dial in key sensors → live prediction + per-wafer SHAP) and a Line insights tab (model performance and the global sensor ranking), with language aimed at a process engineer rather than a data scientist.
Docker (API + dashboard together):
docker compose up --build
# API -> http://localhost:8000/docs
# UI -> http://localhost:8501Local:
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python -m scripts.run_pipeline # download -> train -> save model + figures
uvicorn api.main:app --port 8000 # terminal 1
streamlit run dashboard/app.py # terminal 2 (API_URL defaults to :8000)The trained model is committed, so the API/dashboard run without retraining;
run_pipeline reproduces everything (model, metrics, all figures) from scratch.
python -m scripts.run_pipeline # end-to-end: download -> preprocess -> train -> evaluate -> save
pytest -q # inference returns valid output; API endpoint shapesdata/ download script + raw cache (data not committed)
notebooks/ 01_eda · 02_preprocessing · 03_modeling · 04_evaluation_shap
src/ reusable logic, imported by notebooks, API, and dashboard
data.py · preprocessing.py · modeling.py · evaluate.py · explain.py
inference.py · train.py · eda.py · config.py
api/ FastAPI service (predict + explanation)
dashboard/ Streamlit app (thin client over the API)
models/ saved model bundle + metrics + comparison JSON
reports/ evaluation & SHAP figures
tests/ inference + API tests
- The signal is weak. SECOM is a hard dataset; PR-AUC ≈ 0.23 means precision is low at high recall. This is an honest screening aid, not an auto-reject gate.
- Anonymised sensors. Without the physical mapping, SHAP localises which sensor, not what it measures — a real fab would close that gap instantly.
- Static snapshot. One line, ~3 months of 2008 data; no temporal drift handling, tool-to-tool effects, or recipe context.
- Class weights shift the SHAP base rate, so a borderline-nominal wafer can sit just over the (deliberately low) threshold — expected behaviour for a recall-first screen, surfaced honestly in the UI.
- Streaming ingestion. Swap the CSV loader for a connector to the MES / sensor historian; score wafers as they come off the line instead of in batch.
- Drift monitoring. Track sensor distributions and PR-AUC over time; alert when a tool drifts and trigger retraining (the timestamps are already loaded for this).
- Cost-explicit thresholding. Replace the F-β threshold with the fab's real re-test vs scrap costs to set the operating point by dollars, not a proxy.
- Per-tool / per-recipe models. Condition on the process route rather than one global model, so SHAP attributions point at a specific chamber.
- Calibrated probabilities + human-in-the-loop, feeding confirmed root causes back as labels to close the loop.







{ "prediction": "fail", "fail_probability": 0.37, "threshold": 0.05, "explanation": [ {"sensor": "S60", "value": 8.9, "shap_value": 0.071, "direction": "raises fail risk"}, ... ] }