How a 30-cycle rolling mean beat a deep LSTM at turbofan prognostics

Two unglamorous decisions, a grouped split and a physics-sized feature window, took tabular XGBoost past a widely cited sequence model on NASA’s FD001 benchmark.

In 2017, a deep LSTM set a widely cited benchmark on NASA’s turbofan degradation dataset: 16.14 RMSE predicting how many flight cycles an engine has left. My XGBoost model reaches 15.85 on the same official test set with no neural network anywhere in the pipeline. This entry is about the two decisions that did the work, because neither of them is the model.

The problem, briefly

NASA’s C-MAPSS FD001 dataset simulates 100 turbofan engines running to failure, 20,631 training cycles in all, each cycle a row of 21 sensor readings: temperatures, pressures, shaft speeds, fuel flow. The task is remaining useful life: given an engine’s sensor history, predict how many cycles remain. You train on complete run-to-failure trajectories and predict on engines cut off at an arbitrary point, scored against the official held-back answers.

Seven of the 21 sensors barely move in FD001, so they carry nothing. That leaves 14 informative channels, and one convention worth knowing: remaining life is capped at 125 cycles, because a healthy engine’s sensors look identical whether it has 130 or 300 cycles left. Predicting “125 or more” is the honest ceiling.

Decision one: split by engine, never randomly

The most common way to get a flattering RUL number is a random train/validation split. Rows from the same engine land on both sides, the model memorizes each engine’s personal drift, and validation error collapses. It isn’t prediction, it’s recognition.

Every split in this project is grouped by engine: an engine’s entire trajectory is either training or validation, never both. The validation score got worse the day I did this, which is exactly the point. The number stopped lying.

If you take one thing from this entry: on any dataset where rows belong to units that degrade, group your splits by unit and expect the honest number to be uglier.

Decision two: size the window to the physics

The failure mode in FD001 is compressor fouling. It is slow. Cycle to cycle, the sensor drift is smaller than the noise; over tens of cycles, it is unmistakable. So the features are 30-cycle rolling means and standard deviations of each sensor, and 30 wasn’t tuned by grid search: it approximates one fouling cycle, the timescale on which the degradation physically accumulates.

This is the part I’d defend in any review: the window is where the domain knowledge lives. An LSTM is asked to discover that timescale from data. A rolling mean is told the timescale by someone who knows the machine. On a dataset this size, telling beats discovering:

Model	RMSE, FD001 official test
CNN (Babu et al., 2016)	18.45
Ridge regression, my baseline	17.47
Deep LSTM (Zheng et al., 2017)	16.14
XGBoost on rolling features, this project	15.85

Worth saying plainly: transformer and hybrid models published since have pushed FD001 below all of these. 15.85 is a strong classical result, not the state of the art. What makes it interesting is the ratio of result to machinery.

The model agrees with the thermodynamics

The top SHAP feature for most predictions is the 30-cycle mean of sensor 3, the HPC outlet temperature. That is the textbook fouling signature: as the compressor fouls, efficiency drops and outlet temperature climbs. The next features in the ranking are the upstream and downstream sensors that respond to the same fault. Nobody told the model any thermodynamics; checking that its explanations recover the known physics is the closest thing tabular ML has to a sanity proof.

What I’d redo

The prediction ships with a ±15 cycle band, and that band is a heuristic, labeled as such in the API schema itself. It is not a calibrated interval, and pretending otherwise would be worse than the crude band. The fix is known (conformal prediction or quantile regression) and it is the top of the V2 list, along with extending past FD001’s single operating condition, where this feature set would need regime clustering before any rolling statistics make sense.

Try it against the official answers

The live demo at turbofan.alvinalias.com does something I wish more ML demos did: one click loads a real engine from NASA’s held-back test set, the model commits to an estimate, and then the official answer prints next to it, error included. Some runs miss. That’s what 15.85 RMSE means, and watching it miss honestly tells you more about the model than any metric table. Code, notebooks, and the model card are in the repo.