Working notes from building data science systems, mistakes included.
Type All types Notes Entries References
Topic All topics
2026-07-10 entry I stopped asking an LLM to write every RAG answer A three-route RAG design for exact lookups, cited interpretation, and ambiguous questions, plus the benchmark that kept Groq in production.
2026-06-23 entry What a test-set threshold sweep can and cannot tell you Cost assumptions move an operating threshold, but selecting and reporting it on the same labels makes the result exploratory.
2026-06-19 entry Regex confidence is a poor escalation signal My hybrid extractor cut LLM calls by 56% and kept far less than 56% of the accuracy gain. The autopsy is more useful than the result.
2026-06-16 entry Ranking unusual building operation without failure labels What meter, weather, and metadata proxies can support, what detector agreement means, and why this is not equipment fault detection.
2026-06-12 entry How I graded a RAG system before trusting it Separate report cards for retrieval and generation, refusal traps where declining is the right answer, and why a fluent answer is not evidence of anything.
2026-05-31 audit A temporal split did not save my transaction model An audit of future-history and cancellation-sign leakage, why chronological evaluation still failed, and the checks I now require before trusting a transaction model.
2026-05-28 entry How rolling features improved my CMAPSS baseline A grouped split and a chosen 30-cycle smoothing window moved XGBoost past my local Ridge baseline. Published model scores remain unaudited context.