Segaran Toby. Programming Collective Intelligence: Building Smart Web 2.0 Applications

O’Reilly Media, 2007. — 362 p. — ISBN: 978-0-596-52932-1.The increasing number of people contributing to the Internet, either deliberately or incidentally, has created a huge set of data that gives us millions of potential insights into user experience, marketing, personal tastes, and human behavior in general.This book provides an introduction to the emerging field of collective intelligence. It covers ways to get hold of interesting datasets from many web sites you’ve probably heard of, ideas on how to collect data from users of your own applications, and many different ways to analyze and understand the data once you’ve found it.
This book’s goal is to take you beyond simple database-backed applications and teach you how to write smarter programs to take advantage of the information you and others collect every day.The code examples in this book are written in Python, and familiarity with Python programming will help, but I provide explanations of all the algorithms so that programmers of other languages can follow. The Python code will be particularly easy to follow for those who know high-level languages like Ruby or Perl. This book is not intended as a guide for learning programming, so it’s important that you’ve done enough coding to be familiar with the basic concepts. If you have a good understanding of recursion and some basic functional programming, you’ll find the material even easier.Want to tap the power behind search rankings, product recommendations, social bookmarking, and online matchmaking? This fascinating book demonstrates how you can build Web 2.0 applications to mine the enormous amount of data created by people on the Internet. With the sophisticated algorithms in this book, you can write smart programs to access interesting datasets from other web sites, collect data from users of your own applications, and analyze and understand the data once you’ve found it.Programming Collective Intelligence takes you into the world of machine learning and statistics, and explains how to draw conclusions about user experience, marketing, personal tastes, and human behavior in general — all from information that you and others collect every day. Each algorithm is described clearly and concisely with code that can immediately be used on your web site, blog, Wiki, or specialized application.This book explains:
Collaborative filtering techniques that enable online retailers to recommend products or media
Methods of clustering to detect groups of similar items in a large dataset;
Search engine features — crawlers, indexers, query engines, and the PageRank algorithm;
Optimization algorithms that search millions of possible solutions to a problem and choose the best one;
Bayesian filtering, used in spam filters for classifying documents based on word types and other features;
Using decision trees not only to make predictions, but to model the way decisions are made;
Predicting numerical values rather than classifications to build price models;
Support vector machines to match people in online dating sites;
Non-negative matrix factorization to find the independent features in a dataset;
Evolving intelligence for problem solving — how a computer develops its skill by improving its own code the more it plays a game.
Each chapter includes exercises for extending the algorithms to make them more powerful. Go beyond simple database-backed applications and put the wealth of Internet data to work for you.Introduction to Collective Intelligence.
Making Recommendations.
Discovering Groups.
Searching and Ranking.
Optimization.
Document Filtering.
Modeling with Decision Trees.
Building Price Models.
Advanced Classification: Kernel Methods and SVMs.
Finding Independent Features.
Evolving Intelligence
Algorithm Summary.
Appendix. Third-Party Libraries.