Social Media Research Blog

ICWSM 2009 CFP

Alright... we are on! Get ready for another exciting event! The CFP for ICWSM is below and the deadlines are duly noted in the Social Media Events calendar as well. (rdfs:seeAlso Matt's blog post)

icwsmCFP - Upload a Document to Scribd

Read this document on Scribd: icwsmCFP

Notes from Brad Fitzpatrick's Invited Talk at ICWSM

Brad Fitzpatrick was over at ICWSM and gave a keynote on "Decentralization and Social Graphs". Following are some notes from the talk. Please note these are from rush transcription during the talk and from my recollection of what was presented. Please feel free to comment if I have missed something or if you feel something is incorrect.

Brad talks about Social graph and highlights some of the problems with traditional forms of communication. Email for example, lacks sender authentication or verification which lead to spam. Phone traditionally did not have any sender authentication and later these services were bolted on to the original infrastructure. Instant Messenger services don't always offer interoperability. Brad says that the  future of IM is Jabber/XMPP, which offers a decentralized system protocol. Gtalk, Livejournal both use Jabber and this makes it easily interoperable. In general the his talk highlights the importance of decentralization. Other things that are decentralized are email, wikis, ethernet, bittorrent etc.

Blogs are pretty much decentralized and interoperate using ATOM/RSS, but the difficult thing is commenting. Social Network sites are still "walled gardens" and basically need to re-invent everything -- sign up process, profile etc.... just to be able to provide one "cool feature". So as of now many of them dont work with each other. This is bound to change.

Brad says that the "glimmers of hope" are the following:

• XMPP Gtalk uses XMPP and in fact I have played around with it a bit and it is really a simple and neat way to communicate with instant Messengers.

• RSS/ATOM Ofcourse blog readers have relied on RSS/ATOM. From David Sifry's talk I recollect that ATOM is the preferred form of syndication for Technorati, since it avoids having to crawl the entire HTML content to be sure that partially syndicated feeds are handled correctly.

• OpenID This eases signups/logins. Already many of the providers and sites have adopted the OpenID standard.

• OAuth This provides an easy way for authorization... Often we are faced with a situation where some new random site says "hey why dont you gimme your password?". However, can you trust it with your password? OAuth is an alternative way for one website to request action on another.

• XFN/Microformat/FOAF Machine Readable format of the social graph. While there is a good amount of FOAF data online, it is hardly enough to cover the entire social graph. Apparently XFN is more popular and is also used by Twitter.

• OpenSocial This is a standard way for containers (social network sites) to host social apps. Same app works on Orkut, Myspace, Hi5 and .... now also Yahoo. This is a big leap towards interoperability.

• XRDS (Yardis) provides a discovery mechanism on a URL... suppose a site needs to ask another "where is your OpenID server" ... "Where is your calendar" etc.

• Social Graph API This is Brad's very own pet project. It finds and parses XFN, FOAF, me links. Provides an API for the social graph. A nice way to find missing friends. This solves the problem of not knowing all the alternative userids for my friends.

So to summarize the talk : use OpenID to log in to the site..... Site reuses public profile..... it can ask for pvt stuff using OAuth..... container app uses OpenSocial and Social Graph finds friends. And we all live happily ever after in our nice little online worlds :-) and all sites talk to each other and we are all F.R.I.E.N.D.S! :-)

The talk also highlighted some missing pieces:

    - messaging URLs

    - URL - email interop

    - XMPP/JID binding

    - Reputation server

    - promoting or documenting.

Some startups that are currently using open protocols twitter, dopplr, plaxo, pownce.

Brad says that users expect things to interoperate ... and decentralized is more robust. Social networks are broken today but are getting better.

Overall, it was a very interesting talk but I felt that it was kind of short and it would have also been fun to see more details.

Crowdvine: The Facebook for conferences

I just got back from ICWSM yesterday. I am still absorbing all the wonderful paper presentations, posters, ideas and discussions we had at this conference. There were some amazing invited speakers, paper presentations and engaging audience.  Kudos to the organizers (  Matthew Hurst,   Eytan Adar,   Natalie Glance,   Nicolas Nicolov, Belle Tseng and Tim Finin) for the fabulous arrangements and for bringing us together.

I have been thinking of what made this conference so different from others that I have attended in the past.  Firstly, this is a crowd that totally "gets" social media and loves to be a part of this exciting new research area. But as importantly, we are also people who "eat our own dog food", so to say.  One tool that really brought us together  even before the conference actually started was  crowdvine. It is what I would like to describe as the "Facebook for conferences". I think ICWSM would surely have made a record of some sort for the "fewest business cards exchanged". Not that there was any less networking -- its just that people already had a connection established before they met. I had found some of my ICWSM friends on Twitter, Facebook and Linkedin already, so there wasnt any real need to request for a business card.

Secondly, the conversations started before the event and they continued, even after it ended. For example, before the event, I had already exchanged a few references on my blog and via email. With some of the members I had met at the previous event in Boulder, I had been sharing feeds, Twitter messages and Facebook wall-to-wall posts. Unlike in other conferences where we meet people after a year with little context for discussion, I felt that I already knew some of my colleagues as online friends.  (When I met William Cohen, we talked about his cool blues video, for example). In words of Lisa Reichelt such connections are all about ambient intimacy and this made the people I interacted with at ICWSM closer than those at other venues.

After the conference the discussions moved to Twitter. A funny meme was around my absent-mindedness in forgetting my laptop after an amazing scifi experience, just hours before my flight. Fortunately, I did manage to recover it. :-)

I found Crowdvine to be a great tool and think that it should be used in every event. A couple of things that would really improve it though:
a) A way for me to add my Crowdvine friends to Twitter, Facebook, Flickr etc. I know there is a way to "import" but most often i meet new people at a conference so what I would like is an "export" functionality.
b) Conferences are about sharing ideas, academic papers and research. What Crowdvine currently lacks is a way to engage its members in discussions around the main focus of the conference -- the papers and presentations themselves.

I think that there is a great value proposition in Crowdvine and it has started with implementing a few things but doing it really really well. As more conferences start using it, I am certain that it is only going to get even better.

 

ICWSM Invited Talk: David Sifry

Scaling Innovation in a Network Effect World

Following is a liveblogging from David Sifry's invited talk at ICWSM. I hope that this (sort of rushed) transcription would be interesting for those who missed this talk and cant wait till they get hold of the video.

David is talking about "what makes a company scale?" Say you have the next cool idea and now want to build a product or company out of it. Now how do you scale it and make sure that you are not going to have a the whole thing blow up in your face.

Sifry says that it boils down to scaling not just the algorithms but also scaling your team and knowing that you are doing it right -- which is, how do you measure things right? Talking about some of the failed companies and comparing them with successful tools like Facebook and Gmail, Sifry's caveat is that the points he highlights (in this talk) are not the commandments but things he has learned from his experience over the years building and scaling companies.

Architecture of Innovation

Sifry talks about creating not just a product but an environment that flourishes innovation. Well, back to basics now. Most social network algorithms are O(N^2), for most part. So the question is how to take advantage of the hardware and RAM and work with approximations that do not deviate too far from the theoretical results, but would be efficient to scale. Also RAM is getting cheaper so in scaling things, we really need to move to caching mechanisms.

Flexibility

"The right way of doing things". Sifry says that there are two types of engineers

a. Worst is better: Doesnt have to be perfect, good is the enemy of the best, just get something out there!

b. Crystaline purity advocates: Says most people coming from academic background face this problem. It is a big block that hinders people from starting to scale something. Argues, its better to iterate fast and learn from what users do.

Combine Flexibility and Performance

Resist temptation of overoptimizing things that are not really important right now. Get quick things out and set some standards EARLY. Its OK to build different subsystems in different languages so long as they agree on the common standard.. Sifry suggest "DO ONE THING GREAT. THEN EXPAND". The goal is to pick one great idea and do that reallly realllly well! Trying to get too many features right is probably not going to let you be flexible.

Seams

a) Geography
There is a gravitational pull for people to often interact with people who are closer to each other geographically. However, things like friendster never optimized for it -- however facebook and secondlife are scaling well because they took this into consideration. Optimizing for cross geography can be done later.

b) Interests
We can optimize for social interests, topics, groups, clubs. Scale by interests, groups and even subgroups.

c) Tribes and Structures
People are often interacting with families relatives, universities, companies depts, etc. These are natural structures that are clearly dileniated and this would allow us to scale the algorithms quite naturally. For example Facebook started by limiting the campuses initially thus being able to optimize and scale slowly.

Scaling Teams

How do we set up teams? Sifry talks about using "super agile teams" using 2-3 engineer teams with 2 weeks to production including QA. This forces us to ask how to really break down the problem in a way that it can be completed in 2 weeks. Keeping it small also minimizes the meetings. Cant argue with that! Ambient meetings: Keep Skype turned on ALL THE TIME! Most of the time nothing happens but then if you have a question -- you can just ASK, as if your colleague is just sitting there in the same room next to you. Sifry says that it increases the efficiency... and ability to move really FAST! and reduce communication delays.

Big Systems
Try to set it up so that it is scripted out of the smaller systems that are existing in your smaller systems. This lets you reuse the existing components and tells you where to optimize.

Measurement
Know the key pieces of your components that need to be watched. Make it simple and look for high volumes and response times. Dont over optimize but look for smaller pieces that slow you down.

Growing Teams
Biggest transition is going from a one person team to a two person team. :-)
2-8 team sizes seem to be the optimal size of teams. Once we get to be about 25 teams start to become unmanageable.  Most great products come out of small teams (up to 8 people). Sifry calls 20-60 size the "Death Zone": Communication costs are higher, meetings run random, middle management is formed and for many startups this is where they get killed. This has also been supported by some research in social sciences. In order to get through the death zone... Sifry suggests that startups need to really justify their revenue model and every new hire in this phase. Suggests that we should ask the question: "What if someone got hit by a bus?".  At the point where it is 70-100 members one starts reaching the Dunbar's number. At a point when the organization grows that large -- it is usually subdivided into two smaller teams.

Unscalability as a Feature
Dont make it open - betas are a good way to limit the users and test if your system is going to scale. Give the users an opportunity to invite people and become evangelists for your tools. More IMPORTANTLY - Watch users who DONT send out recommendations... talk to them and understand why they dont send out recommendations.

What Went Wrong?
Places like friendster started out trying to build for flexibility and did not listen to their users (Fakesters) and didnt slow adoption to scale better.

What went right?
Both facebook and gmail understood the power of exclusivity.... but not for so long that the competition would come in and beat you at your game.

Define the Culture
Have small teams, A Players, be of service. Understand that there are different kinds of people and take an advantage of the crystal impurities. Some people are starters and some people are finishers -- take advantages of the right skills of the right people.

Q&A
- Build a great product and be obsessed so much about influencers. Dont worry about top 100/ A-List/B-list... be of service to as many people as possible. Reach out to people and those customers of yours are going to become your advocates and influencers. At the same time listen to what the issues of your users are. But dont end up optimizing for the most vocal users - know that you have to make a tradeoffs there.
- Scalability vs. Open source: Sifry advocates open source and says that Technorati works hard to contribute back to the community. Originally they started with MySQL, A million blog and full text search!!! It works upto a million blogs and indexing in RAM. Next thing they did was to shard the database into multiple partitions. This allowed Technorati to scale. Technorati has hired some core Lucene contributers, and who have made modifications to create small indexes as silos and later crush them into a bigger index. This has also been contributed back to the community. (If someone knows a pointer please let me know).
- Next point was about full feed and non rss content. Full atom feed is gauranteed to be indexed -- but RSS feeds are a problem. So for RSS Technorati indexes the full HTML in case of RSS feeds. Sifry says that this is one of the competitive differentiators compared to other early startups.
- Comments are HARD! Somehow there isnt a good standardized ways to deal with them.

Thanks David for a great talk!

ICWSM Kickoff

WWE workshops from which ICWSM originated had a participation of 50 people and this year just looking at the number of people at the conference room today it seems like it has easy gone to 200. This is a significant jump -- it shows both the significance as well as the popularity of this field. Also check out the number of people who are already interacting and connecting with attendees at crowdwine. Kudos to the organizers and the sponsors and AAAI who have supported the conference. The amazing thing is that the sponsors are not just the big companies but a bunch of neat startups who are also here with some great demos.

Yesterday, the there were two awesome tutorials: Jan Wiebe tutorial covered "Subjectivity and Sentiment Analysis" was and Mary McGlohon who covered "Graph Mining Techniques for Social Media Analysis". I shall try to put out a summary of the two talks over the course of the conference. Right now... Keynote by Brad Fitzpatrick.