Milestone 4 Progress Evaluation

Personalized Web Search

Erdal Tuleu
email: etuleu@fit.edu

Faculty sponsor:
Dr. Philip Chan
email: pkc@cs.fit.edu


Milestone 4 (Feb 23, 2009):

• Study other clustering algorithms for dividing the pages visited into logical sessions
• k-means
• hierarchical agglomerative clustering
• hierarchical divisive clustering
• Find a way to combine all the features used for finding similar pages into one similarity function
• Evaluate different algorithms for clustering and similarity functions against the ground truth using the rand index similarity measure

Progress of current milestone

Task	Completion (%)	Comments and To do
1. Study other clustering algorithms	100%	Read the chapter on clustering from the book "Introduction to Data Mining" and other online sources
2. Find a way to combine all the features used for finding similar pages into one similarity function	85%	Combined the 3 features (page content, link structure, url similarity) into one distance function. There is still room for improvement by trying out different combinations and maybe adding new features.
3. Evaluate different algorithms for clustering and similarity functions against the ground truth using the rand index similarity measure	90%	Evaluated k-means, agglomerative and divisive clustering and stayed with agglomerative because it also helps in finding the appropriate number of clusters. Implemented the L-method for finding the knee of a graph (used in finding the best number of clusters). Got good rand index results, but they could further be improved by trying different cluster distance measures and stopping criteria (such as a statistic test).

Plan for milestone 5 (March 23, 2009):

• Build the user profiles using the DHC algorithm as described in http://cs.fit.edu/~pkc/apweb/ 
• Score search results using the profile
• Evaluate the results
• Improve the algorithm for building profiles and the scoring function for re-ranking search results
• Poster for senior design showcase

Sponsor feedback on each task:

1.


2.


3.



Signature: _______________________________ Date: ________

---

Sponsor Evaluation
Sponsor: detach and return this page to Dr. Chan (EC 242)

Scores: circle a score (or circle two adjacent scores for .5 or write down a real/float number between 0 and 10) for each member:

1.  

   • Erdal Tuleu

     0

     1

     2

     3

     4

     5

     6

     7

     8

     9

     10

Edit this page (if you have permission) |

Google Docs -- Web word processing, presentations and spreadsheets.