On Machine Learning

The discipline of Machine Learning has inherited a major portion of it's theory from classical statistics. Today, the area is quite vast and it consists of many topics including classification, regression, clustering, reinforcement learning etc. We often see interview candidates with machine learning in resume. In academic circle, typically a researcher in machine learning is expected to publish in ICML, NIPS, COLT community.
Application research can be published in various different conferences such as KDD, ICDM, SDM, CVPR, ECCV, ICPR, SIGIR and even in WWW. However, the question arises that what knowledge is really required or valuable for software engineering positions. I feel that someone claiming some practical and academic knowledge in machine learning should be able to answer some of the questions such as follows:

1. What are sample complexity and model complexity?
2. What are the characteristics of feature selection algorithms?
3. How one handle sparsity in features?
4. What are various loss functions used for SVM, clustering, decision trees?
5. What are the desirable characteristics of loss functions and why?
6. What is regularization?
7. Why generalized linear models are called "generalized"?
8. What are exponential family distributions? What is their significance in machine learning?
9. Can one make a classifier using regression model?
10. How do one compare various models with same target and with different targets in regression?
11. Can one apply k-means algorithms for a data set with Zipf distribution? What modification is required?
12. What is Gibb's sampling?
13. What is the difference between HMM and CRF?
14. What is the complexity of inference in Graphical model?

The above questions are a few of the sample questions. I believe a good reading of selected chapters of Duda and Hurt, Bishop and Hastie's book (3 books total) can provide one a basic background in the subject. Boyd's convex optimization book and Feller's volume one in statistics can be used as references for knowledge in convex optimization and statistics.