Practical Statistical Learning (PSL)

(Often humorously dubbed “Pumpkin Spice Latte” by our online students.)

These materials have been curated from a course in statistical learning, developed by Professors John Marden (jimarden AT illinois DOT edu) and Feng Liang (liangf AT illinois DOT edu) at the University of Illinois Urbana-Champaign (UIUC). The course predominantly draws upon the seminal text The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman.

Note

This project is under active development.

• 1. Introduction
  • 1.1. Introduction to statistical learning
  • 1.2. Least squares vs. nearest neighbors
• 2. Linear Regression
  • 2.1. Multiple linear regression
  • 2.2. Geometric interpretation
  • 2.3. Practical issues
• 3. Variable Selection and Regularization
  • 3.1. Subset Selection
  • 3.2. Regularization
  • 3.3. Ridge Regression
  • 3.4. Lasso Regression
  • 3.5. Discussion
• 4. Regression Trees and Ensemble
  • 4.1. Regression Trees
  • 4.2. Random Forest
  • 4.3. GBM
• 5. Nonlinear Regression
  • 5.1. Polynomial Regression
  • 5.2. Cubic Splines
  • 5.3. Regression Splines
  • 5.4. Smoothing Splines
  • 5.5. Local Regression
• 6. Clustering Analysis
  • 6.1. Distance Measures
  • 6.2. K-means and K-medoids
  • 6.3. Choice of K
  • 6.4. Hierarchical Clustering
  • 6.5. R/Python Code
• 7. Latent Structure Models
  • 7.1. Model-based Clustering
  • 7.2. Mixture Models
  • 7.3. The EM Algorithm
  • 7.4. Latent Dirichlet Allocation Model
  • 7.5. Hidden Markov Models
• 8. TBA
• 9. Discriminant Analysis
  • 9.1. Introduction to Classification
  • 9.2. Discriminant Analysis
  • 9.3. Quadratic Discriminant Analysis
  • 9.4. Linear Discriminant Analysis
  • 9.5. Fisher Discriminant Analysis
  • 9.6. Naive Bayes Classifiers
  • 9.7. Summary
• 10. Logistic Regression
  • 10.1. Setup
  • 10.2. MLE
  • 10.3. Seperable Data
  • 10.4. R/Python Code
  • 10.5. Retrospective Sampling Data
• 11. Support Vector Machine
  • 11.1. Introduction
  • 11.2. The Separable case
  • 11.3. The Non-separable case
  • 11.4. Nonlinear SVMs
  • 11.5. Appendix
• 12. Classification Trees and Boosting
  • 12.1. Introduction
  • 12.2. Impurity Measures
  • 12.3. Misclassification Rate vs. Entropy
  • 12.4. AdaBoosting
  • 12.5. Forward Stagewise Additive Modeling
• 13. Recommender System
  • 13.1. Introduction
  • 13.2. Content-Based Methods
  • 13.3. Collaborative Filtering
  • 13.4. UBCF and IBCF
  • 13.5. Latent Factor Model
  • 13.6. Challenges and Strategies
  • 13.7. Deep Recommender Systems