Computer Science & Engineering Department

• Email: bebis@unr.edu
• Office: 411 WPEB
• Office Hours: Monday/Wednesday 2:15pm - 3:00pm or by appointment

• Email: aminul.huq@nevada.unr.edu
• Office: 414 WPEB
• Office Hours: Tuesday/Thursday 12:30pm - 1:30pm

Required Text:

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd edition, Wiley-Interscience. ISBN 0-471-05669-3 Errata

Optional Texts:

T. Hastie et al., The Elements of Statistical Learning, Springer-Verlag, 2001.
K. Murphy, Machine Learning: A probabilistic Perspective, MIT Press, 2012.
K. Murphy, Probabilistic Machine Learning: An Introduction, MIT Press, 2022.

IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)
Pattern Recognition (PR)
Machine Learning (ML)
International Journal of Pattern Recognition and Artificial Intelligence (IJPRAI)

IEEE Computer Vision and Pattern Recognition (CVPR)
International Conference of Pattern Recognition (ICPR)

Math Cheat Sheet (lots of useful formulas)
Numerical Recipes in C
Applied Statistics

Important Resources

UCI Machine Learning Repository
Review Papers on Statistical Pattern Recognition, Neural Networks and Learning

Useful software:

Statistical Pattern Recognition Toolbox
Tooldiag
Feature Selection Toolbox (C++)
VLFeat open source library
R Project

Source Code for Reading/Writing Images

Course Prerequisites

Exams and Assignments

Grading will be based on 7 quizzes (lowest quiz grade will be dropped), two exams, and 4 programming assignments (to be completed on an individual basis). Graduate students will be required to present a paper to the rest of the class.

Course Policies

Lecture slides, assignments, and other useful information will be posted on this web page. Discussion of the of your work is allowed and encouraged. However, each student should do his/her own work. Assignments which are too similar will receive a zero and you might also receive an "F" in the class. No late work will be accepted unless there is an extreme emergency. If you are unable to hand in an assignment by the deadline, you must discuss it with me before the deadline. Quizzes and exams will be closed books, closed notes. If you are unable to take a quiz or exam you must inform me in advance. No incomplete grades (INC) will be given in this course and a missed exam may be made up only if it was missed due to an extreme emergency. Regular attendance is highly recommended. If you miss a class, you are responsible for all material covered or assigned in class. You should carefully read the section on Academic Dishonesty found in the UNR Student Handbook. Your continued enrollment in this course implies that you have read it, and that you subscribe to the principles stated therein.

Handouts

• Syllabus

Sample Exams

• Midterm Exam
• Final Exam (disregard problem #3)

Lectures

• Course Overview
  • What is Pattern Recognition? A Gentle Introduction
    
  • Jain, A.K.; Duin, P.W.; Jianchang Mao, Statistical pattern recognition: a review, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 22, no 1, pp. 4-37, Jan. 2000.
  
  
• Introduction to Pattern Recognition
  
  
• Review of Linear Algebra (see Appendix A.2)
  • Matrix Algebra (StatLect)
  • Linear Algebra Review and Reference
  • Linear Algebra (Khan Academy)
  • Linear Algebra (online book - UC-Davis)
  
  
• Review of Probability (see Appendices A.4 and A.5)
  • Probability and Statistics Refresher (from Univ of Florida)
  • Probabilities and Statistics refresher (from Stanford Univ)
  • Statistics and Probability (Khan Academy)
  • Probability and Statistics (online book - Univ of Toronto)
  • Introduction to Probability (online book - Harvard and Stanford Universities)
    
  • Multivariate normal distribution
  
  
• Bayesian Decision Theory
  
  
• Case Studies
  • A. Madabhushi and J. Aggarwal, A bayesian approach to human activity recognition, 2nd International Workshop on Visual Surveillance, pp. 25-30, June 1999.
  • J. Yang and A. Waibel, A Real-time Face Tracker, Proceedings of WACV'96, 1996.
  
  
• Maximum Likelihood (ML) Estimation
  • In Jae Myung, Tutorial on Maximul Likelihood Estimation, Journal of Mathematical Psychology, vol. 47, pp. 90-100, 2003.
  
  
• Bayesian Estimation

• Midterm Exam Review
  
  
• Dimensionality Reduction (with case studies)
  • M. Turk and A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
    
  • Daniel L. Swets, John (Juyang) Weng Using Discriminant Eigenfeatures for Image Retrieval , IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 18, No 8 , pp. 831-836, 1996.
    
  • A. Martinez and A. Kak PCA versus LDA, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 23, No 2, pp. 228 -233, 2001.
    
  • C. Burges, Dimension Reduction: A Guided Tour, Microsoft Research Tech Report MSR-TR-2009-2013.

• Feature Selection (with case studies)
  • Feature Selection Algorithms - A Brief Guide
    
  • Jain, A.K.; Duin, P.W.; Jianchang Mao, Statistical pattern recognition: a review, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 22, no 1, pp. 4-37, Jan. 2000.
    
  • H. Liu et al., Feature Selection: An Ever Evolving Frontier in Data Mining, Fourth Workshop on Feature Selection in Data Mining, 2010.
    
  • Z. Sun, G. Bebis, X. Yuan, and S. Louis, Genetic Feature Subset Selection for Gender Classification: A Comparison Study, IEEE Workshop on Applications of Computer Vision, pp. 165-170, Orlando, December 2002.
    
  • Z. Sun, G. Bebis, and R. Miller, Object Detection Using Feature Subset Selection, Pattern Recognition, vol. 37, pp. 2165-2176, 2004.
    
  • G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, Face Recognition by Fusing Thermal Infrared and Visible Imagery, Image and Vision Computing, vol. 24, no. 7, pp. 727-742, 2006.

• Linear Discriminant Functions

• Support Vector Machines
  • C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, Kluwer Academic Publishers, 1998.
    
  • A. Mojan, C. Papageorgiou and T. Poggio, Example-based object detection in images by components, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 23, no. 4, pp. 349-361, 2001.
    
  • B. Moghaddam and M. Yang, Gender Classification with SVM, IEEE Transactions on PAMI, vol. 24, no. 5, 2002.
    
  • M. Pontil and A. Verri, Support Vector Machines for 3D Object Recogition, IEEE Transactions on PAMI, vol. 20, no. 6, 1998.
  
  
• Bayesian Networks
  • E. Charniak, Bayesian Networks Without Tears, AI Magazine, 1991.
    
  • d-separation Without Tears
    
  • Adnan Darwiche, Bayesian Networks, Communications of the ACM, 2010.
    
  • A Brief Introduction to Graphical Models and Bayesian Networks
  
  
• Final Exam Review

Programming Assignments

Submission Instructions

Assignment 1 (due on 3/11/2024 at 11:59 pm)
• How to generate Gaussian samples
  Click here for code and documentation

Assignment 2 (due on 4/1/2024 at 11:59pm)
• Data

Assignment 3 (due on: 4/22/2024 at 11:59pm)
• ppt file
• Face images (please note that the header of the PGM images provided has all the values in the same line, e.g., P5 48 60 255)
  
• Eigen-decomposition example
  
• jacobi.c and jacobi.h (from Numerical Recipes in C)
  
• Phillips, J. and Moon, H. and Risvi, S. and Rauss, J., The FERET evaluation methodology for face recognition algorithms, vol. 22, no. 10, pp. 1090-1104, 2000.
  
• The Face Recognition Homepage

Assignment 4 (due on: 5/7/2024)
Gender Data
A practical guide to SVM classification
LIBSVM: A Library for Support Vector Machines
Download LibSVM

Paper Presentation (Graduate Students Only)

Presentation Topics

Presentation Guidelines

1. Presentations should be professional as if it was presented in a formal conference (i.e., powerpoint slides/projector).
2. Your goal is to educate and inform your audience. Make sure your presentation follows a logical sequence. Help the audience understand how successive definitions and results are related to each other and to the big picture.
3. You should have your remarks prepared and somewhat memorized. Reading from your notes excessively should be avoided.
4. Anticipate Questions: think of some likely questions and plan out your answer. Understand the Question: paraphrase it if necessary; repeat it if needed. Do Not Digress. Be Honest: if you can't answer the question, say so.
5. Each student's material is different but 15 minutes each should be enough time for your presentation.
6. Meet the eyes of your audience from time to time.
7. Vary the tone of your voice and be careful to speak clearly and not talk too quickly.