Working notes from building data science systems, mistakes included.
Type All types Notes Entries References
Topic All topics
2026-07-10 entry I stopped asking an LLM to write every RAG answer A three-route RAG design for exact lookups, cited interpretation, and ambiguous questions, plus the benchmark that kept Groq in production.
2026-06-23 entry Why 0.775 is the right threshold when a miss costs $50K Accuracy is a decoy on imbalanced failure data. Charge each error its real price, sweep the threshold, and let the minimum tell you where to act.
2026-06-19 entry Regex confidence is a poor escalation signal My hybrid extractor cut LLM calls by 56% and kept far less than 56% of the accuracy gain. The autopsy is more useful than the result.
2026-06-16 entry Scoring equipment health when nobody labeled the failures Per-unit baselines, a second detector as witness, and why the strongest signal turned out to be COP volatility, not COP level.
2026-06-12 entry How I graded a RAG system before trusting it Separate report cards for retrieval and generation, refusal traps where declining is the right answer, and why a fluent answer is not evidence of anything.
2026-05-31 reference Point-in-time features for transaction data How to compute customer history features without leaking the future, why random splits lie on this kind of data, and how to check yourself with a rolling backtest.
2026-05-28 entry How a 30-cycle rolling mean beat a deep LSTM at turbofan prognostics Two unglamorous decisions, a grouped split and a physics-sized feature window, took tabular XGBoost past a widely cited sequence model on NASA’s FD001 benchmark.