Saturday, September 26, 2009

Quicksort

Quicksort is one of the most used Algorithm in computer science. It is based on the idea of recursive partitioning. The partition scheme is always linear but the expected problem sizes at each iteration can be data dependent. Hence the expected O(nlogn) time comes. However, its a very useful algorithm because the time complexity is fairly tight and that's the reason it can achieve close of nlogn speed for most of the cases.
Implentation of partition in place is not trivial and history proved that with many buggy variants of partition in 60sHowever, A functional (haskel) version of Quicksort can be wrtten in a very precise and short manner:

qsort [] = []
qsort (x:xs) = qsort less ++ [x] ++ qsort more
where less = filter () xs
more = filter (>=x) xs


Tuesday, September 22, 2009

sparse data, complex model and netflix prize

The bellcore team won the Netflix prize. They have spent about an year to work on this problem and then developed a complex model with millions of features for the Netflix data recommendation. Quite a few interesting papers are published. One million dollar is roughly equivalent to the cost of hiring 5/6 Engineers for an year. This is quite a gain for Netflix and it's a gain for research community as well. Yehuda's KDD paper got a best paper award and it is a very interesting paper. The use of temporal modeling for similar problems for web applications or predicting CTR for advertisements is not new. But, it is interesting to identify how closely we can model the intuition and mood to understand human behavior even from a sparse set of data.
This is a more than million dollar information. An open question could be however if the model necessarily need to be complex or if there is a simpler mathematics that can explain it.