Batches
Each batch is its own independent project — its own blank library, training dataset, and models. Nothing pools across batches.
New batch
All batches
Blank library
Upload Solvent / Method / Carryover / Reagent blanks for the working batch — these form the pool used for blank subtraction.
Column mapping
If your instrument software exports different column names than the default (e.g. "Peak Name" instead of "Hit Name"), set that up here once — applies to every GC-MS upload from here on. Most companies never need to touch this.
Upload a blank
What entered the blank pool from this file — expand "Uploaded blanks" below for past uploads (compound counts only; re-download needs the original file since raw peaks aren't stored).
Pool composition
Combined counts per blank type for the working batch.
Uploaded blanks
Training data
Upload a raw sample — cleaned automatically against the working batch's blank pool, then tag its known Class.
Add a sample
RT Tolerance is how close (in minutes) a sample peak's retention time must be to a blank peak to count as a match. Ratio-based mode: if sample area ÷ blank area is below the threshold, the peak is purged as background — above it, it's flagged "retained (high area)" for your review. Strict mode: any name+RT match against the blank pool is removed outright, regardless of area — the Area Ratio Threshold is ignored.
Preview — every peak and what happened to it. Rows are colour-coded per the legend above; only non-purged rows get saved to the training dataset.
Training dataset — working batch
Predict sample
Uses the working batch's active model + diagnostic ratio. Cleaned the same way as training samples.
Column mapping
Loading current mapping…
This is the same company-wide mapping set on the Blank Library page — editing it here updates it everywhere.
Upload a sample
Model visualization
Prediction history — working batch
Models
Manually triggered. Re-selects biomarkers, retrains all 5 algorithms, and re-derives the diagnostic ratio from the working batch's training dataset.
Data preprocessing
Non-halal keyword(s): comma-separated text matched against Class names (case-insensitive) — decides which samples count as "non-halal" for the diagnostic ratio and for the Predict-sample halal/suspicious/non-halal call. Set this to whatever your Class labels actually use (e.g. "pork", "pig, swine", "haram").
Applied in this order — normalize samples (rows) → transform values → scale variables (columns) — to every PCA/biplot/PLS-DA/statistical-report/univariate-plot computation below. Model training keeps its own separate scaling per cross-validation fold, unaffected by this. α = 0.05 is the common food-industry default; halal-specific work sometimes uses a stricter 0.01.
Preview preprocessing effect
Before vs after the pipeline above — overall density curve + grouped box-and-whisker, so you can see the effect immediately without retraining.
Before
After
Before
After
Univariate plots — outlier detection
Box-and-whisker, all variables grouped in one plot (IQR rule: points beyond 1.5×IQR from Q1/Q3 flagged as outliers), with descriptive statistics below.
Full PCA statistical report
XLSTAT-style output — descriptive statistics, correlation structure, and the full eigen-decomposition behind the biplot below.
Summary statistics (quantitative data)
Correlation matrix — Pearson (n-1)
Eigenvalues
Eigenvectors
Factor loadings
Correlations between variables and factors
Identical to factor loadings above — both come from the same eigen-decomposition. With Standardize (n-1) selected, these are also literal Pearson correlations; with Pareto scaling, they are loadings in the pareto-scaled space.
KMO — measure of sampling adequacy per variable
Outlier detection (Hotelling's T²) iAutomated version of manually spotting and removing a stray point from a PCA plot — flags samples that sit unusually far from the rest in PCA score-space. Optional; nothing is deleted without your say-so.
Retrain
Biomarker candidates
Scree plot
Eigenvalue / % variance explained per component — look for the "elbow" to judge how many components are worth keeping.
Training dataset overview
PCA biplot (3D) — sample clustering with the compounds driving the separation overlaid as vectors.
PCA biplot (2D, PC1 vs PC2) — with a circle around each class's cluster.
PLS-DA observation plot
OPLS-DA (one-class-vs-rest)
Separates variation into ONE predictive component (correlated with the target class) and orthogonal components (removed) — cleaner separation than plain PLS-DA when you care about one specific class vs everything else.
Score plot — one point per SAMPLE, shows whether the groups separate.
S-plot — one point per COMPOUND. X-axis = reliability (correlation), Y-axis = effect size (covariance). Trustworthy candidate biomarkers sit in the far corners — high on both axes — not just one.
Model versions
Nothing auto-activates — pick the best-performing version and activate it yourself. Click "View" on any row to see how that specific model works.
⚠️ No model is active yet — Predict Sample won't work until you click "Activate" on one below.
Model visualization
Diagnostic ratio versions
Generate on demand — not part of Retrain, so you can regenerate anytime without retraining models.
How this works
VIP score (Variable Importance in Projection)
For each compound, VIP measures how much it contributes to the PLS-DA model's ability to separate classes, combining how strongly the compound loads onto each PLS component with how much variance that component explains. VIP ≥ 1 is the standard rule-of-thumb cutoff — it's a candidate filter, not proof of biological relevance on its own.
ANOVA p-value & the "validated" flag
A compound is marked validated: true only if VIP ≥ 1 and a one-way ANOVA across your Class groups gives p < 0.05 — both the multivariate model and a simple univariate test agree it separates classes. With small sample counts, ANOVA has low power, so treat the validated list as a shortlist, not a final answer.
The 5 ML algorithms
SVM (RBF kernel) maximizes the margin between classes — good with clear separation, less interpretable.
Random Forest averages many decision trees — robust to noise, gives feature importances for free.
k-NN (k=5) votes among the 5 nearest neighbours — simple, sensitive to feature scaling and irrelevant features.
Naive Bayes assumes features are independent and normally distributed per class — fast, works with small n, assumption often imperfect for correlated lipids.
ANN (2-layer MLP) learns a non-linear boundary via backpropagation — most flexible, needs more data to avoid overfitting.
All 5 use the same cross-validation split, so their metrics are directly comparable.
Diagnostic lipid ratio (ROC + Youden index)
Every pair among the validated biomarkers is tried as a ratio, scored by ROC-AUC at separating "contains lard" from everything else. The best pair's cutoff is the Youden-optimal point (max of true-positive rate − false-positive rate) — the threshold that best balances catching real lard against false alarms.
Why are there so many versions, and what do they mean?
Every retrain trains all 5 algorithms — one retrain = 5 new version rows, sharing the same version_label (a timestamp). Compare rows with the same label to see which algorithm did best on that dataset; compare across labels to see whether more training data helped. Nothing auto-activates — you choose which version is "live."
Which metric should I look at to pick a version?
Accuracy — overall % correct; misleading if classes are imbalanced.
Sensitivity/Recall — of true non-halal samples, how many were caught. Matters most here: missing real lard is the costlier mistake.
Specificity — of true halal samples, how many were correctly cleared. High specificity = fewer false alarms.
Precision — of samples called non-halal, how many actually were.
F1 — balance point between precision and recall.
ROC-AUC — ranking quality independent of any cutoff; 1.0 perfect, 0.5 random.
What actually happens when I "promote" a prediction?
The predicted sample's cleaned compound data is copied into the training dataset with the Class you confirm (tagged source: promoted). It does not retrain automatically — click Retrain now again for it to take effect.
Bartlett's test of sphericity
Tests whether your correlation matrix is significantly different from an identity matrix (i.e. whether the variables are correlated at all). A significant result (p < 0.05) means PCA/factor analysis has something real to work with — variables share enough correlation to be worth reducing to fewer factors. A non-significant result means the variables are close to uncorrelated, and PCA on them won't find meaningful structure.
KMO — Kaiser-Meyer-Olkin measure of sampling adequacy
Ranges 0 to 1. Compares the correlations between variables to their partial correlations (correlation left over after removing the influence of every other variable) — high KMO means a variable's correlation with others is mostly "real" shared variance, not noise. Rule of thumb: >0.8 great, 0.6–0.8 acceptable, <0.5 poor (PCA results may not be reliable). Per-variable MSA below the summary works the same way but one variable at a time — low-MSA variables are candidates to drop before a factor analysis.
Eigenvalues, eigenvectors, and factor loadings
Eigenvalues are how much total variance each factor (principal component) captures — an eigenvalue of 1.0 means that factor explains as much variance as a single original variable would. Eigenvectors are the raw direction each factor points in variable-space (unit length, no particular scale). Factor loadings rescale each eigenvector by √(eigenvalue) — because PCA here runs on the standardized (correlation) matrix, this rescaled value is exactly the Pearson correlation between that original variable and the factor, which is why "factor loadings" and "correlations between variables and factors" show identical numbers in this report.
Blacklist keywords
Keywords that mark a compound as an artifact — applied automatically during cleaning, on both blanks and samples, across every batch.
Add keyword
FTIR training data
Already-prepared spectra — no cleaning step, no batches. Every row in the CSV is a sample.
Add training samples (bulk)
CSV with SampleID, Class, and wavenumber columns.
Couldn't automatically find a Class column. Pick which columns are which from your file:
Training dataset
FTIR predict sample
Uses the active FTIR model + diagnostic ratio.
Predict samples (bulk)
CSV with SampleID and wavenumber columns (Class optional/ignored). Every row is predicted.
Model visualization
Prediction history
FTIR models
Manually triggered — same pipeline as GC-MS, run on wavenumber features instead of compounds.
Data preprocessing
Non-halal keyword(s): comma-separated text matched against Class names (case-insensitive) — set this to whatever your Class labels actually use (e.g. "pork", "pig, swine", "haram").
Applied in this order — normalize samples (rows) → transform values → scale variables (columns) — to every PCA/biplot/PLS-DA/statistical-report/univariate-plot computation below. Model training keeps its own separate scaling per cross-validation fold, unaffected by this. α = 0.05 is the common food-industry default; halal-specific work sometimes uses a stricter 0.01.
Preview preprocessing effect
Before vs after the pipeline above — overall density curve + grouped box-and-whisker.
Before
After
Before
After
Univariate plots — outlier detection
Box-and-whisker, all variables grouped in one plot (IQR rule), with descriptive statistics below.
Full PCA statistical report
XLSTAT-style output — descriptive statistics, correlation structure, and the full eigen-decomposition behind the biplot below.
Summary statistics (quantitative data)
Correlation matrix — Pearson (n-1)
Eigenvalues
Eigenvectors
Factor loadings
Correlations between variables and factors
Identical to factor loadings above.
KMO — measure of sampling adequacy per variable
Outlier detection (Hotelling's T²) iAutomated version of manually spotting and removing a stray point from a PCA plot — flags samples that sit unusually far from the rest in PCA score-space. Optional; nothing is deleted without your say-so.
Retrain
Biomarker candidates
Scree plot iEigenvalue / % variance explained per component.
Training dataset overview
PCA biplot (3D).
PCA biplot (2D, PC1 vs PC2) — with a circle around each class's cluster.
PLS-DA observation plot
OPLS-DA (one-class-vs-rest)
Separates variation into ONE predictive component and orthogonal components (removed) — cleaner separation than plain PLS-DA when you care about one specific class vs everything else.
Score plot — one point per SAMPLE, shows whether the groups separate.
S-plot — one point per COMPOUND. X-axis = reliability (correlation), Y-axis = effect size (covariance). Trustworthy candidate biomarkers sit in the far corners — high on both axes — not just one.
Model versions
Click "View" on any row to see how that specific model works.
⚠️ No model is active yet — Predict Sample won't work until you click "Activate" on one below.
Model visualization
Diagnostic ratio versions
Generate on demand — not part of Retrain, so you can regenerate anytime without retraining models.
Training data iAny CSV with a SampleID column and numeric variable columns. Class is optional — include it for classification, leave it out for pure exploratory PCA.
Upload and tag the samples your models will learn from
Add samples (bulk)
CSV: SampleID (or first column), optional Class, and 2+ numeric columns.
Training dataset
Models iExplore with PCA any time (no Class needed). Train a classifier only once your dataset has Class labels with 2+ groups.
Explore the data, pick biomarkers, and train classifiers
Suggest preprocessing iLooks at your actual uploaded data (skew, scale differences between variables, how much sample totals vary) and suggests a starting pipeline — not a mandate. You can always change any setting below yourself.
Data preprocessing
- None
- No row adjustment.
- By sum
- Each sample divided by its own total — corrects overall concentration/dilution differences between samples.
- By median
- Each sample divided by its own median — more resistant to a few very large peaks than "by sum".
- Quantile
- Forces every sample to the same value distribution. Best for very wide datasets (1000+ variables).
- None
- No element-wise change.
- Log10 / Log2
- Compresses large values, pulls in right-skew — the standard choice for concentration-like data spanning orders of magnitude.
- Square root / Cube root
- Milder than log — stabilizes variance without compressing as aggressively.
- Variance stabilizing
- Behaves like log for large values and linear near zero — handles data that includes zeros better than a plain log.
- Arcsin
- The standard transform for proportion/fraction data (e.g. relative abundances) — stabilizes variance near 0 and 1.
- Box-Cox
- Finds the power transform that makes a column closest to normally distributed. Needs positive values (shifted automatically if needed).
- Winsorize
- Caps (doesn't remove) the most extreme low/high values per column at a chosen percentile — tempers outliers without deleting the sample.
- Johnson
- Fits a flexible Johnson-family curve per column to push it toward normality — more aggressive than Box-Cox, works on data Box-Cox can't (including negative values).
- None
- No column adjustment.
- Center
- Subtracts each column's mean — centers on zero, no change to spread.
- Standardize (n–1)
- Center, then divide by the sample standard deviation — the common default ("auto-scaling"). Every variable ends up equally weighted regardless of its original units.
- Standardize (n)
- Same, using the population standard deviation (divide by n instead of n–1) — nearly identical for larger sample counts.
- / Std. dev. (n–1 or n)
- Divides by the standard deviation only — no centering, so a variable's original scale relative to zero is kept.
- Pareto scaling
- Divide by the square root of the standard deviation — a middle ground between no scaling and full standardizing; keeps large-signal variables somewhat more influential.
- Range scaling
- Center, then divide by (max − min).
- Rescale 0–1 / 0–100
- Plain min-max rescaling into a fixed range — easiest to interpret, but sensitive to outliers (an extreme value stretches the whole scale).
- Binarize (0/1)
- Every nonzero value becomes 1, zero stays 0 — reduces to simple presence/absence.
- Sign (−1/0/1)
- Keeps only the direction of each value, not its size.
- ON (default)
- The normalization, transformation and scaling above are used to pick the biomarkers and to train every model. Inside cross-validation they are re-learned from each fold's training samples only (nothing leaks from the held-out samples). Each saved model keeps what it learned, so unknown samples get exactly the same treatment. Normalization is computed over ALL variables first, then the biomarkers are picked, so the prediction file must contain every training variable.
- OFF
- The old behaviour: models and biomarker selection work on the raw areas, with each model only standardizing its own columns. Use it to compare.
- What actually changes results
- SVM, kNN, ANN and PLS-DA standardize their inputs internally anyway, so the Scaling choice barely moves them. Normalization and transformation (for example log10) are what really change the models. Scaling matters mostly for Random Forest and Naive Bayes.
Switch it off to reproduce the old behaviour. Scaling has little effect on SVM / kNN / ANN / PLS-DA (they standardize internally); normalization and transformation are what change them.
Applied in this order — normalize samples (rows) → transform values → scale variables (columns) — to every PCA/biplot/PLS-DA/statistical-report/univariate-plot computation below. With the switch above ON, the same settings are also used for biomarker selection (Step 2) and model training (Step 3).
Before
After
Before
After
Univariate plots iBox-and-whisker, one variable at a time (IQR rule), with descriptive statistics below. This looks at each variable SEPARATELY — a sample can look fine here but still be flagged below by Hotelling's T², which looks at all variables TOGETHER. The two are complementary, not duplicates — neither replaces the other. Works with or without Class labels.
Full PCA statistical report iXLSTAT-style output. Works with or without Class labels.
Summary statistics (quantitative data)
Correlation matrix — Pearson (n-1)
Eigenvalues
Eigenvectors
Factor loadings
Correlations between variables and factors
Identical to factor loadings above.
KMO — measure of sampling adequacy per variable
Multivariate outlier detection (Hotelling's T²) iDifferent from the Univariate plots above, which check one variable at a time — this checks ALL variables TOGETHER (via PCA), flagging a sample that's unusual in its overall combined pattern even if no single variable looks extreme on its own. Automated version of manually spotting and removing a stray point from a PCA plot. Optional; nothing is deleted without your say-so.
Shortcut: run Step 1 → 2 → 3 in one go
Optional. Runs the three steps below back to back, using whatever settings are currently set, and the results land in each step's own card below (exactly as if you'd clicked each button yourself) — nothing new or separate is created. Skip this and use Step 1 below if you'd rather stop partway or check each result before continuing.
Step 1 — PCA + PLS-DA iStop here if this is all you need — the next steps (biomarker selection, model training) are separate and optional.
Scree plot iEigenvalue / % variance explained per component.
Training dataset overview
PCA biplot (2D, PC1 vs PC2) — with a circle around each class's cluster (if any).
PLS-DA observation plot
Step 2 — Select biomarkers iVIP (from PLS-DA) + ANOVA significance, on the Step 1 result. Re-run Step 1 first if you've changed the preprocessing settings above. VIP ≥ 1 is the common default (a compound contributes at least an "average" amount), but it's just a convention — lower it to cast a wider net, or raise it to be stricter.
Full candidate table
Validated = Yes means the compound cleared BOTH checks: VIP score above your chosen threshold (meaningfully contributes to separating the classes) AND ANOVA p-value below your significance level above (the difference between classes isn't just random noise). Validated = No means it failed at least one — still shown for transparency, but not recommended as a primary biomarker.
Diagnostic ratio (optional, fast alternative to Step 3) iSearches every pair among Step 2's validated biomarkers for the ratio that best separates samples matching the target keyword below from everything else, with a statistically optimal cutoff. Not halal-specific — works for any classification problem: type any class name or keyword from your own data (it doesn't need to match the Predict page's halal keyword at all). A single number vs a cutoff is often enough for routine QC — no need to run a full ML model every time. Generate on demand, independent of Step 3; more than one ratio can be active at once. The highlighted ★ Recommended pair has the best separation in the latest run: highest ROC-AUC, then Youden J, then the widest margin between its cutoff and the nearest unflagged sample. Check its sensitivity and specificity — a high AUC can still mean some false flags.
Step 3 — Train models
- Both (default)
- Trains every algorithm twice — once on the validated biomarkers from Step 2, once on all variables — so you can see whether the biomarker selection actually helps. Takes about twice as long.
- Validated biomarkers only
- Smaller, easier-to-explain models. Needs Step 2 to have validated at least one biomarker. Note: biomarkers were chosen using all samples, so their cross-validation accuracy is somewhat optimistic.
- All variables
- No selection step, so no selection bias — a fair baseline. Doesn't need Step 2. The prediction file must then contain every variable used in training.
Model versions iNothing auto-activates — pick the best-performing version and activate it yourself. Click "View" on any row to see how that specific model works. The highlighted ★ Recommended row is the highest cross-validated accuracy in the latest run (ties: F1, then ROC-AUC, then fewer features). Older runs aren't ranked against it. For biomarker models that accuracy is somewhat optimistic, since the biomarkers were chosen using all samples.
⚠️ No model is active yet — Predict Sample won't work until you click "Activate" on one below.
Model visualization
Predict sample iUses the active model(s) for this dataset — only available once you've trained and activated at least one with Class labels.
Classify new, unknown samples with your active model(s)
Halal / Non-Halal determination iThe model predicts a class name (e.g. "Lard", "Beef Tallow") — this turns that into a Halal/Non-Halal verdict: if the predicted class name contains any of these keywords, the sample is marked Non-Halal. Comma-separated, case-insensitive. Change this any time without retraining — it's a labeling rule applied AFTER prediction, not part of the model itself.
Predict samples (bulk)
CSV with SampleID and the same numeric columns used for training.
- Unknowns on PCA / PLS-DA
- A second opinion that doesn't use the trained models: your unknown samples are placed into the PCA / PLS-DA built from the training data, to see which class cloud they fall in — or whether they fall outside every class.
- Same preprocessing, applied automatically
- The reference saved at Step 1 holds the preprocessing settings AND the values learned from the training samples (means, spreads, etc.). Unknowns are transformed with those frozen values — never re-fitted on themselves — so a sample lands in the same place whatever else is in the batch.
- Inside / overlap / outside
- A class cloud is the region within the 95th-percentile distance of that class's own training samples from its centre. Inside one cloud = fits that class. Inside several = the clouds overlap there. Inside none = doesn't resemble any training class.
- PLS-DA caution
- PLS-DA is supervised, so training samples look tidily separated by design; the informative part is where the unknowns land.
Model visualization
Prediction history
Sample dataset
Ready-made dummy data covering every feature in this platform — download, upload, and try things without needing your own data first.
Smoke test
Quick health check — confirms the Python/ML engine and every API endpoint are actually working, without touching your real data. Run this after any platform update.
GC-MS — Batch 1: "Hexane Extraction"
The full walkthrough. 2 blanks, 16 training samples (4 classes × 4 replicates: Olive Oil, Palm Oil, Beef Tallow, Lard — covering halal-plant, halal-animal, and non-halal), and 5 unknowns for Predict (one per class + one deliberately ambiguous blend to trigger "Suspicious").
Built with realistic mess on purpose: some peaks have quality <80 (get filtered), some match the blank pool at low area (get PURGED), one per class is deliberately high-area at a shared background peak (flagged RETAINED_HIGH_AREA) — so you'll actually see the cleaning stages do something, not just pass everything through.
How to use: create a batch (e.g. "Hexane Extraction") → Blank library: upload the 2 blanks → Training data: upload the 16 training files, tagging Class from the filename → Models: Retrain, then Activate the best version → Predict sample: upload the 5 "Unknown-*" files.
GC-MS — Batch 2: "Chloroform Extraction"
A second, independent batch — 2 blanks, 8 training samples (Beef Tallow vs Lard only), 2 unknowns. Different solvent-background compounds than Batch 1.
Purpose: prove batch isolation. Train and activate a model here, then switch back to Batch 1 — confirm Batch 1's active model is untouched, and vice versa. This is the scenario that exposed the "shared model" bug earlier in the build.
FTIR
One training CSV (24 samples, 4 classes × 6 replicates, ~150 wavenumber columns) and one external test CSV (5 samples incl. one Olive+Lard blend) — matches the format the original "3rd FTIR Halal Authentication" app used, now also usable in this platform's FTIR section.
PCA shows realistic partial overlap (FTIR spectra are inherently more correlated than discrete GC-MS compounds) but PLS-DA/LOOCV still classifies at ~83% accuracy — a good stress test for the "imperfect but working" case.
How to use: FTIR → Training data: upload the training CSV, Save all → Models: Retrain, Activate → Predict sample: upload the predict CSV.
General dataset
Two CSVs for the two General Dataset use-cases:
With Class — edible oil chemistry profile (Iodine Value, Saponification Value, Peroxide Value, Free Fatty Acid %, Refractive Index) across 4 oil types, 5 replicates each. Tests the full classification path on data that isn't spectra at all.
Without Class — a fast-food nutrition panel (10 items, 5 numeric columns, no Class). Tests "Explore (PCA only)" — pure exploratory analysis with no classifier involved.
How to use: General dataset → Training data: upload either CSV, Save all → for the "with Class" one, Models → Retrain works normally; for "without Class," use Models → Explore (PCA only) instead.
Learn
How the platform actually works, from raw data to a halal/suspicious/non-halal call — with a few playgrounds you can poke at.
1. The big picture
Click each stage to see what happens there.
2. ROC curve & Youden index — playground
This is exactly how the diagnostic ratio's cutoff gets chosen. Drag the slider and watch sensitivity/specificity trade off against each other on a toy dataset (24 samples, 8 "contains lard").
3. The 5 ML algorithms
Same cross-validation split, five different ways of drawing the decision boundary.
What matters most to you?
4. Halal decision simulator
The exact rule the platform uses on every "Predict sample" run. Move the sliders and watch the status change.