Comcast Chooses Solr For New Entertainment Research Platform
Introduction
Comcast Company is among the largest suppliers of enjoyment, details and communication services and services, with in excess of 24 million cable people, 15 million high-speed The web people, and six.five million telephone people. Its Comcast Interactive Media (CIM) division is chartered to develop and mature the company’s The web establishments. CIM’s Fancast.coman broad on-line video collection of tv shows, motion pictures, trailers and clipsgets in excess of ten million original users per month. Users can browse and lookup across the site’s 4M+ written content objects to uncover the enjoyment they need.
Requirements/Challenges
Meet functionality goal of 20ms per query at peak load, and scale to 1 million original users per day
Provide easy lookup interface, whereas retaining deep customizability
Low fixed & operational costs
Deliver complete functional lookup features
Comcast Company is among the largest suppliers of enjoyment, details and communication services and services, with in excess of 24 million cable people, 15 million high-speed The web people, and six.five million telephone people. Its Comcast Interactive Media (CIM) division is chartered to develop and mature the company’s The web establishments. CIM’s Fancast.coman broad on-line video collection of tv shows, motion pictures, trailers and clipsgets in excess of ten million original users per month. Users can browse and lookup across the site’s 4M+ written content objects to uncover the enjoyment they need.
Difficulties
Lookup is critical to Fancast’s business objectives — getting users to all the media written content they need, as quickly and intuitively as possible. The lookup implementation had to meet three key issues:
1. Provide a easy lookup interface, ideally a person easy box without sacrificing deep customizability, to constantly meet and exceed user needs without exposing them directly to written content complexity
2. Handle massive written content scale literally all TV and enjoyment written content – at scales responsive to mass market traffic and reach.
3. Achieve very low fixed and operational costs in terms of dedicated development and support staff, and minimal additional hardware.
Functional and cyndi Overall performance Standards
Fancast uses metadata from many different 3rd party sources such as IMDB.com (the The web Movie Database) and Tribune Media Service. Each of these 3rd party sources has its own specific format, as well as differing written content refresh schedules, and none includes a comprehensive metadata store with consistent data and descriptions. For example, the official Hollywood hntrends.com Spider-Man movie titles from Marvell Entertainment use two hyphenated words, but most users enter them as a person word, with no hyphen.
The ability to present an authoritative index was not only essential to the user experience, but also a key differentiator for the best lookup experience. Users searching Jessica Simpson probably don’t want to end up with Homer Simpson.
In terms of functionality, the goal was to mature from 50,000 to 1 million peak original visitors per day in excess of 16 months. To ensure candidate lookup technologies could meet this goal, CIM defined a clear scaling metric, with lookup query response under 20ms/query at peak load, at the same order of magnitude as for website interactions. Scaling and capacity targets were also set at the application server level so that a single physical application server could host multiple server instances, each with a similar scaling profile. This also simplified sizing conditions for the operations team for calculating how many servers would be needed for a given number of users.
Testing & Evaluation
CIM shortlisted two lookup alternatives: Solr, the Lucene lookup server; and a large well-known commercial lookup product. To pick the finalist, they created a test-bed with indexes of both two million and four million documents deployed on each within the Sun x64 servers running Red Hat Linux. To review the results and optimize the Solr Lucene lookup infrastructure, CIM hired Lucid Imagination. Consultants from the commercial vendor did the same with their solution. The CIM team benchmarked query response rates at different load levels, ranging from 100 to 1500 requests per second, as well as stress tests at failure envelope points.
The result: Solr outperformed the commercial alternative lookup solution both in terms of response rates as well as failure-handling characteristics. There was no question that adaptsol Solr could meet the targets set for functionality.
CIM also compiled a list of 180 functional features for comparison. In addition to its superior functionality, Solr also came out ahead on functions and cost of ownership to meet CIM’s business objectives.
The Choice For Solr
Solr made the final cut based on:
Overall performance and scalability advantages
Required lookup features
Organizational fit
Total Cost of Ownership
Active Solr/Lucene open source development community
Other large organizations that “bet the company” successfully on Solr (CNET, Netflix, MySpace, Orbitz)
In addition to the availability of community and commercial support, CIM benefited from the deep expertise in lookup offered by Lucid Imagination to configure their Solr implementation in accordance with best practices, and to optimize scalability.
“Hiring Lucid Imagination took a superior potential platform that our people liked, and turned it into a reliable, high-performance platform that really satisfied our business leadership.” Ranga Muvavarirwa, Director Product Planning, Comcast Interactive Media
No comments yet.