Research Papers
Academic papers, pre-prints, and topics I'm currently exploring.
* Papers curated from the hurshd0/must-read-papers-for-ml repository.
Model Evaluation, Model Selection, and Algorithm Selection
Sebastian Raschka's comprehensive guide to the different techniques for evaluating machine learning models and selecting the best one.
XGBoost: A Scalable Tree Boosting System
The original paper describing the system that dominates Kaggle competitions, detailing its algorithmic optimizations and sparsity-aware split finding.
Peeking Inside the Black Box (ICE Plots)
Introduces Individual Conditional Expectation (ICE) plots to visualize the relationship between features and predictions in black-box models.
A Tutorial on Principal Component Analysis
An excellent, highly readable tutorial on PCA that builds the intuition behind the math from the ground up.
Data preprocessing - Tidy data
A foundational paper by Hadley Wickham on organizing data to facilitate analysis, defining the standard for data tidying.
Statistical Modeling: The Two Cultures
Leo Breiman's classic discussion on the difference between data modeling (statistics) and algorithmic modeling (machine learning).
Frequentism and Bayesianism: A Python-driven Primer
Jake VanderPlas provides an intuitive, code-first approach to understanding the philosophical and practical differences between frequentist and Bayesian approaches.
The Netflix Recommender System
An industry case study examining the algorithms, business value, and innovation behind one of the most famous recommender systems.
Attention Is All You Need
The seminal paper introducing the Transformer architecture, which dispensed with recurrence and convolutions entirely and paved the way for modern LLMs.
Deep Residual Learning for Image Recognition
Introduces ResNet and residual connections, solving the vanishing gradient problem and allowing the training of extremely deep neural networks.
Adam: A Method for Stochastic Optimization
Presents Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, which is now a standard default optimizer.