The Little Engine(s) That Could: Scaling Online Social Networks

Posted by: Admin  :  Category: Web Hosting Comparison

Linux Cpanel shared hosting: 600 GB disk space, 6 TB bandwidth, free domain, unlimited databases and ftp accounts, web hosting cheap and pro at Hostony

Linux Cpanel shared hosting: 600 GB disk space, 6 TB bandwidth, free domain, unlimited databases and ftp accounts, web hosting cheap and pro at Hostony

Google Tech Talk (see below) June 17, 2010 Presented by Josep M. Pujol. ABSTRACT The difficulty of partitioning social graphs has introduced new system design challenges for scaling of Online Social Networks (OSNs). Vertical scaling by resorting to full replication can be a costly proposition. Scaling horizontally by partitioning and distributing data among multiple servers using, for eg, key-value stores using DHTs, can suffer from expensive inter-server communication and other performance issues. Such challenges have often led to costly re-architecting efforts for popular OSNs like Twitter and Facebook. We design, implement, and evaluate SPAR, a Social Partitioning and Replication middle-ware that mediates between the application and the database layer of an OSN. SPAR exploits the underlying social graph structure to partition user data and selectively replicate users to ensure that users have their neighbors’ data co-located on their machine. The gains from this are multi-fold: application developers can assume local semantics, ie, develop as they would for a single machine; scalability is achieved by adding commodity machines with low memory and network I/O requirements; and N+K redundancy is achieved at a fraction of the cost. We provide a complete system design, extensive evaluation based on datasets from Twitter, Orkut, and Facebook, and a working implementation. We show that SPAR performs well in terms of reducing the overhead, and dealing with high dynamics
Video Rating: 4 / 5

Linux Cpanel shared hosting: 600 GB disk space, 6 TB bandwidth, free domain, unlimited databases and ftp accounts, web hosting cheap and pro at Hostony

6 Responses to “The Little Engine(s) That Could: Scaling Online Social Networks”

  1. olegryaboy Says:

    Very impressive. Is there a publicly available implementation (open source or not)?

  2. galistube Says:

    Is there any software to manage data distribution via torrent or otherwise? How do you manage non public data when you are using distributed clients for storage?

  3. galistube Says:

    What happens at recovery time? When 40%, 60% or more of your subscribers go offline in one day, due to data access issues etc?

  4. TheMrIco Says:

    Interesting topic but like yeah you know like BORING!!!

  5. austincm18 Says:

    If he says like one time I’m going to explode :S

  6. DEVILSNAKELDS Says:

    Long Videos FTW :D

Leave a Reply

*