Social Media Research Blog

List Crazy Diggers

[UPDATE 08/22/2008, 12:10 AM]

I wonder why diggers are just list crazy?

At the time of writing this post (yesterday), here are the 'lists' from the top 100 diggs on the digg front page -- almost 16 of them! What this means is that on every page there is at least one 'list' type of an item. (And I have noticed this trend for some time now)

1. 5 Reasons to Write Down Everything You Eat For a Week

2. 6 Worst Movie Posters Of 2008

3. 15 Fugly Faces of George W Bush: Commander in Grief [PICS]

4. 10 Movie Trailers That Were Better Than The Actual Movie

5. 10 Coolest Chuck Norris T Shirts Ever!

6. 7 Principles Of Clean And Optimized CSS Code

7. 15 Images You Won't Believe Aren't Photoshopped

8. 10 Great Webcomics You Should NOT Share With Your Kids

9. The Top 10 Rookies That Will Impact the NFL This Season

10. The 5 Most Baffling Sex Scenes in the History of Fanfiction

11. The Top Five Weirdest Olympic Events!

12. 9 Linux Myth Debunked

13. Stephen Colbert's Top 7 Drug Moments

14. The Top 7 Girl Gamer Stereotypes

15. Top 10 Songs Over 10 Minutes Long

16. The 10 Skimpiest Superheroine Costumes

Lists can be entertaining and fun. But I wonder why we see lists enjoying such high popularity in digg? From a cursory glance, the comment activity on 'lists' is comparable to any other item. So my guess was simply that the 'Top Diggers' have a greater inclination towards lists. 

[UPDATE 08/22/2008, 12:10 AM]

To understand what was going on here, I turned to the expert, Dr. Kristina Lerman, who has extensively studied the Digg Community. Following is an excerpt from her response to my question about the popularity of lists on Digg:

"......did you notice how the amount of communicated information is shrinking? First we had Twitter, that advertised 128 chars or less in a message. Rather than being an inconvenience, other services followed the trend: "videos in 5s or less", or other bite-sized chunks of information. Lists are part of that trend, but not only do they communicate information in a convenient bite-sized chunk, but they also rank them - another passion of the current age. It seems that people are really averse to expressing thoughts in long paragraphs." [Text highlighted by me]

Digg is essentially a news site, but the audience probably loves the quick, bite-size and neatly ranked information that such posts provide. Even on traditional news reports and TV shows the "top 10"   whatever-odd-ball-potpourri land up being a part of the reporting.

Here are links to some of Dr. Kristina Lerman's papers on digg and a video of her talk (@Google) on Social Information Processing.

Kristina Lerman (2007), User Participation in Social Media: Digg Study , in Proceedings of the International Workshop on Social Media Analysis (SMA07), Conference on Web Intelligence and Intelligent Agent Technology [pdf] [bibtex]

Kristina Lerman (2007), Social Networks and Social Information Filtering on Digg, in Proceedings of Int. Conf. on Weblogs and Social Media, Boulder, CO,USA (poster). Extended version available on arXiv (cs.HC/0612046)

Thanks Kristina for your sharing your thoughts on this new trend!

FriendFeed Vintage Posts

So last month I had run a bunch of FriendFeed crawls. Due to various other higher priority tasks, I haven't yet worked on having the crawl run 24X7. Still the data I have continues to bring forth interesting nuggets. In this post I would like to share the distribution of the friendfeed posts per year.

Year    Number of Posts
2008    439949
2007    35920
2006    7996
2005    1782
2004    147
2003    3
2002    2
2001    1

I had fetched the most recent updated entries for each user. Upon examining the entry dates, I found that some posts dated back to even before friendfeed existed! This was basically due to the fact that FriendFeed is just an aggregator and when a user adds her del.icio.us feed old posts from del.icio.us, amazon and other sources are integrated into FriendFeed. Ofcourse, pretty much all of the recent entries for more prolific users are dated 2008. In case you are wondering, here are the links to the oldest 5 entries. (There were a few entries dated 1989 -- but I think maybe those might be date errors).

1. March 26, 2001 del.icio.us entry by Wilfred 
2. May 10, 2002 Amazon wishlist by Amber
3. Nov 3, 2002 Amazon wishlist by Jumagun
4. Dec 8, 2003 Stumbleupon entry by Michael
5. Dec 12, 2003 del.icio.us entry by Mihir

This is but a snapshot of FriendFeed data and is perhaps only representative of a few days worth of crawls. But what it shows is that at any point in time there are still some "vintage posts" in the FriendFeed cellar for us to enjoy and share.

Maybe there is some joy in reminiscing over old entries with your friends. Perhaps it is sort of the same feeling that one has when sitting down with an old shoebox full of pictures and sharing it with your family. Or... maybe those posts reminds us of a time few years back when we were all excited about del.icio.us, blogs and wikis -- or whatever else was the latest, coolest next big thing at that time!

So my question for you is -- when was the last time you looked for the oldest piece of post/entry you (or your friend) shared online and what do you think about it now? Is it funny now? perhaps even embarassing? :-)

Sphere of Influence

When we talk of Influence, the general problem setting is that of identifying a set of nodes in a social network that can be considered to be "Opinion Leaders": Individuals whose opinions can cause others around them to change their viewpoint or adopt a technology or product. 

The paper by Kempe et al. presents a technique for maximizing the spread of Influence in a social network. The algorithm presented in this paper is a greedy algorithm for selecting N best nodes which if populated with some information could spread it to a large number of nodes in the network through a simple diffusion process. Although in real world social networks it is often unclear how the diffusion process really takes place or how to determine the correct values of the thresholds wherein a node adopts an idea if enough of it's neighbors have also adopted the idea. Jure et al have presented a different approach and a solution for selecting nodes to detect outbreaks in the networks through the analysis of information cascade.

Most recently, the paper "Influentials, Networks and Public Opinion Formation" by Watts and Dodds presents a strategic shift in the way we think about influence and they help casts the problem in a different manner. Their radical shift in approach is best summarized by the following excerpt from the abstract:

..We find that large cascades of influence are driven not by influentials but by a critical mass of easily influenced individuals.

Through simulation studies they validate this intuition. According to me the major importance of this result is in highlighting the fact that a) it is perhaps the followers that are more important in the analysis and b) we need to focus on what these followers are influenced by? No opinion leader or influential nodes is arbitrarily influential just about everything and for everyone. Dailykos may be influential about politics in a particular community and Gizmodo is influential in the geek community. While Dailykos might have some influence in the political community when it comes to technology, it is quite unlikely to have a significant impact on the geek bloggers about a new device. Additionally, Popularity, Influence and Authority are talked about in a similar fashion. This can be quite problematic and I remember having a great discussion with Matt Hurst while we were at KDD last year.

Recently, I had been think about the idea that every individual in the social network has some sort of a sphere of influence around them. The sphere of Influence of a node can be described as the region around that node that is in some way affected by the node.

One way to describe it would be to identify a region/local neighborhood of the node that has more links within the neighborhood than outside of it.

The approach I was thinking of turned out to have quite a bit of similarity with the paper by Aaron Clauset on "Finding a local community structure in networks". This paper presents a greedy heuristic that starts with the initial node and grows the region around the node by adding one node at a time. Each new node to be added is selected based on a local version of a modularity score (Originally proposed by Girvan and Newman). The local modularity score is calculated as

R = I/T

Where I is the current number of edges that point within the group of nodes discovered by the algorithm thus far (or the local community). T is the set of edges that point to as yet undiscovered edges plus those edges that point within the local community. I ran the above algorithm on the classic Karate club dataset. Following graph shows the local community structures or what I call the "Spheres of Influence" around a node.

The lines in red outline the sphere of influence around the node 4 and the lines in blue represent the sphere of influence around the node 32. In order to determine the boundary or the cut-off for the nodes, I use another measure called conductance of the graph. Conductance measure of the graph is related to random walks in the network. Given a partition of the graph, a low conductance would indicate that the graph is well knit and thus the probability that a random walk starting inside the partition terminates outside it is low. Thus, as the nodes are added we can compute the conductance scores. The following graph shows the conductance values of the subgraph induced by the sphere of influence around a given node. The local minimas correspond to the two levels (one in red and other in darker shade, around node 4).

This is still a relatively simple example and it would be worth exploring the Spheres of Influence in real world social networks like the subscription graph of FriendFeed or the followers in Twitter. Some interesting questions to ponder about:

• How do topics change as you move further away from the influence of the root node.
• Is the probability of receiving "Likes" and "Comments" higher within the sphere of Influence?
• What is the role of connectors and information bridges in such spheres of influence?
• Given the timing of each post, do most cascades occur within the spheres?

Information Filters and Subscription Predictability

I read a superb blog post "Who is your Information Filter?". I think Hutch makes an important point here (text in bold is highlighted by me):

In an equal world, information shared by any of your friends will merit click-throughs and discussion. But the practical reality is that some people will be more “equal” than others in terms of driving the discussion agenda. There are two highly correlated components to that:

• Number of subscribers
• Reputation for identifying what is interesting

The sheer number of subscribers make some people Information Filters. The big power users on Twitter: Leo Laporte, Dave Winer, Robert Scoble, Jason Calacanis, etc. These guys really drive discussions around ideas, opinions and news. If you subscribe, you can’t help but be overwhelmed by the discussions they can kick off.

Subscribing to somebody's feed is like a commitment. On social network sites, your actions often speak for themselves. The posts one comments on or likes are quite predictive of who you subscribe to. This is what I found from the analysis of the "likes" in FriendFeed. The following graph shows the probability that a user A subscribes to a user B, given that User A "likes" k or more posts by user B. Typically, if a user has 'liked; more than 3 posts of another users there is over 80% probability of a subscription.

On one hand, this is because users are more exposed to posts from those whom they subscribe to. On the other hand, as the number of such actions between two users increases, so does the probability that there will be a continued commitment in terms of a subscription. Similar trends are observable from the "comments" graph.

The way I see it, an important implication of this is ensuring that the social network makes it possible for users to find and "try out" feeds they might "like". Findability and explorability of social networks allows users to initially explore posts from others and then decide which of these users they would like to pick as their "Information Filters".

Locality of Reference in FriendFeed Interactions

While I was having a discussion about my previous FriendFeed posts, I had mentioned that it would be interesting to see the split up of likes w.r.t the directed, shortest path distance. Following graph shows the number of likes that are received from people you directly subscribe to, or subscribe to 2 hops away, 3 hops away and so on...

Both from this graph and the earlier post, we can see that almost 45% of the likes are from people you directly subscribe to. But what is really interesting is that almost 90% of the remaining likes are accounted by friends who are just two or three steps away. FriendFeeders sure are a close bunch!

This type of locality of reference is also related to what sociologists have called the triangle closing. Partly, this is also due to the fact that FriendFeed tries to make it easier to attain "closing of triangles" by displaying the updates from friends who are just one step away. It would be worth looking into how this network's evolution takes place by recrawling FriendFeed subscription graph and observing how it changes. I suspect that through the likes and comments users find new friends and subscribe to their feeds: Your Friend's Friends who like you may be quite likable themselves!

I believe that models of Web graphs (small world, preferential attachment) are getting even more sophisticated in such social settings. The act of linking is not just creating a new hyperlink anymore -- but it is fragmented into actions such as "I subscribe to someone", "I comment on someones posts", "I like someones post". Such actions are not limited to FriendFeed alone. You can draw similar analogies in Youtube (watching, rating, commenting...), Flickr etc. As we explore these datasets through our empirical analysis, it might bring in a better understanding and intuition to build newer models for Social Graphs.

This data was generated using the 200,000 likes and the graph consisting of subscription information of over 75,000 users. The actual shortest path calculation was done using the Boost Graph Libraries (BGL). This is an amazing and extensive tool for graph analysis and quite frankly, I am a bit surprised that I hadn't used it before! I had used the MATLAB bridge but for very large graphs it is best to run the algorithms using BGL. But, more on that later!

Others Like You Too

Continuing with the FriendFeed comment and likes analysis. Here is a graph showing the percentage of self comments vs. comments from those you subscribe to vs. comments from others. This is based on roughly 700,000 comments and 200,000 Likes

Again, as noted by others the number of self comments are skewed by the fact that notes in del.icio.us and stumbleupon etc are treated by FriendFeed as individual comments. I have not compensated the data for this effect.

It gets interesting when we look at the Likes. I would have expected that the most likes would come from your friends. But likes from people you do not subscribe to slightly outnumber the likes from your friends. See.. others like you too!

That is the beauty of FriendFeed: discovering interesting memes from random people. Ofcourse it would be even more cool to see how many hops away most these people are. I suspect that they would actually be kind of close in the social graph.

Services That Attract Most Likes on FriendFeed

Over at FriendFeed, I have been having a superb conversation about my previous posts. John Duff asked me if I could run a similar analysis on the "Likes". Here it is, the most liked services on FriendFeed. Yep! FriendFeed is the most loved  raking in 50% of the likes. Closely followed by Twitter and Blogs.

Sharing good things is good. And FriendFeed makes it downright simple.

Services That Attract Most Comments on FriendFeed

Indexing about 400,000 posts and over 650,000 comments from about 70,000 users on FriendFeed gives some interesting data. There are about 45 different services that FriendFeed connects to at this point of time. First, I wanted to see what were the absolute volume of posts were from each of these services:

Next, I pulled up the number of comments that every service draws during the time period of my index (which was built by fetching the most recent posts and comments from the API).

Note that this is not normalized for multiple updates by the same user. A few interesting points:

• Large volumes of updates are from URL sharing and bookmarking tools like del.icio.us, Google Reader and StumbleUpon
• Most Comments are attracted by posts that were made directly via FriendFeed.
• del.icio.us does attract an equally large volume of comments.
• While Twitter is not far behind and receives a significant share.

Finally, for the services that attract the most comments, I have calculated the average number of comments per post:

The above chart says something important. On an average, original content like what one would find on blogs, still attracts the most comments. Followed by conversation focussed services like FriendFeed and Twitter. Finally, images posted on Flickr get a lot of attention too. Bookmark sharing sites are now lower in the ladder. See some trend here?

The attention and appeal one receives in social networks is a function of the original content you produce and the value, insights you bring to the table when you share new stuff.

DERI/NUI boards.ie SIOC Data Competition

Following Announcement is Via Dr. John Breslin spotted on W3C's Semantic Web Mailing List:

The Digital Enterprise Research Institute (DERI) at NUI Galway is running a unique competition from 1st August to 30th September 2008 in conjunction with
boards.ie, Ireland’s largest discussion forum site. The competition is an open contest in which entrants can win over €4000 in Amazon.com vouchers by submitting an interesting creation based on an RDF data set of discussion posts from boards.ie over the past ten years:

* The first prize is an Amazon voucher for $4000 (~€2500)
* The second prize is a voucher for $2000 (~€1250)
* The third prize is a voucher for $1000 (~€625)

Read the rules and find out more information on the contest at:

http://data.sioc-project.org/

The data set (approximately 9 million documents) has been represented in the Semantically-Interlinked Online Communities (SIOC) RDF format developed by DERI, NUI Galway for expressing the information contained in social websites (forums, mailing lists, blogs, etc.). Entrants may create whatever they feel is interesting based on this data: it could be a novel web application that makes use of the data set, a report on analyses performed on the data, a tool that allows one to visualise or browse the semantic structure, or whatever else the imagination can come up with!

This event is duly noted in the Social Media Events Calendar.

Six Degrees of Bret Taylor

I have been collecting a crawl starting from one of FriendFeed's founders Bret Taylor's subscriptions. In some of the previous posts, I have shared an initial analysis of this crawl. After the crawl came to a completion, I have found about 70,535 users who are either directly or indirectly connected to Bret. This leads me to believe that the true user population of FriendFeed might be around 75-80,000 range (also similar to the numbers found by crawling the public users). I think FriendFeed is still mostly an early adopter tool and hasn't gone as main stream yet.

Well, if you are curious what your bret number (six degrees of Bret Taylor) is, check out this zoho sheet and search for your name by clicking the Auto Filter.