Crunchberry Project

Spam! by grumbler %_|

Spam.

Ingredients: Ham, pork, sugar, salt, water, potato starch, and sodium nitrate (to keep its color).

Delicious.

Yet what was once a hearty start to my boy scout troop’s mornings has become a major annoyance to web communities across the Internet. And no, we’re not talking about canned meat here.

We’re talking about electronic spam - evil comments left by programs intended to flood the Internet with nonsense, links to viruses and oftentimes, free marketing.

Email spam, forum spam, message boards spam, chat room spam, spam and eggs, spam bacon spam and eggs… you get the point.

So in poking around the administrative end of our Web site and searching for something to write about this morning, I deleted almost 50 spam comments left since last week. The platform we use for crunchberry.org is Wordpress - blogging software offered by Automattic. With over five million Wordpress blogs, there are ample opportunities to spam. To mitigate this, we use Askimet, a plugin offered by Automattic for our blog.

By using this spam filter, comments like “uteijh3bibw36g2q” from our friend Jeanne Dominguez @ dctru.com on one of our recent posts are avoided. As interesting and insightful as her opinions are, I don’t think many people would understand what she is talking about.

Anyway, clearing out our cache of spam led me to ask one of our developers, Brian Boyer, about the integration of a spam filter for our commenting structures. Mostly because time is of the essence, we will not be looking into a spam filter. Also, a programmer would have to spend some serious time learning about the intricacies of our commenting system to formulate a spamming program. And because it isn’t widely used, it would more or less be pointless.

So while we will look into safeguards such as captcha, a spam filter will likely be in the recommendations section of our final report so we can make the most of our time.

Carpe diem!

Worlds Fair 1893

After four weeks deep in designing and developing new ways to approach commenting and conversation around news, we have an interesting challenge. How do we show off our work? As we have discussed in previous posts, our goal for this project was to look at ways to improve the quality of conversation on news Web sites. We’ve developed three ways to facilitate conversation - Question & Answer style comments, short format, tweet-like comments, and Letters to the Editor. And we have built all of these systems to integrate with Facebook. Over the next couple weeks we will focus on polish, and more importantly, developing a way to demonstrate what we’ve built.

There are a few ways that we can show off. We could build a news Web site, host content and attract viewers to participate in our community. We could try to export our app by making it embeddable into other websites. We could release the code and let others use it in existing sites.

We will definitely release the source code, but we can’t rely on others to show off our stuff. So in light of our time constraints, we have decided to build a simple news website. Something with local Cedar Rapids news that will allow interested people interact and discuss the news with other readers and Facebook friends.

So what should our local news website look like? How should we apply our commenting systems? Short-format twitter comments on entertainment and sport stories? Letters to the Editor for hot button issues? Questions and Answers on investigative stories? Some combination of the three on all stories?

These are some of the questions we would like to answer this week.

In part 2, I explained how we’re breaking down our work into tight, four-day-long iterations, but I didn’t get into what everyone is doing during the cycle.

It goes a little something like this: The design team sketches designs of the software that the development team will build during the following iteration, which will be tested by the consumer insights team during the iteration following that one.

The design team applies insights from testing to new and redesigned sketches, which will be implemented by the development team in a later iteration, which will be tested by the consumer insights team… you get the picture.

Design, develop, test, repeat!

In action it looks like this (slightly modified) schedule of our past three weeks:

IMG_3605.JPG by nautical2k

The benefits of a small vessel

By working in a tight cycle like this, we’ve been able to very quickly create testable, demoable work each week, while remaining flexible to changes in our feature list.

For example, this past week we’ve learned more about the requirements and terms-of-use for Facebook Connect applications.  If we were working with set-in-stone-up-front designs, we’d be scrambling to correct our course.  Instead, we’re just tacking as the winds change.

flickr/darth4life83

..without me telling you what I want to know.   One of the challenges in testing the comment structures we have been designing and developing is getting valuable and unbiased feedback from our test panel.  Since we are not in Cedar Rapids and cannot conduct the tests in person, we have been using CoPilot, a program that allows us to see the desktop of the person using our demo.   It is helpful to be able to see how the tester is interacting with our site and to see any glitches that arise for them.   It is not perfect, but it is better than being blind to what the tester is experiencing. 

We have been arranging times to chat with the people from our test panel and observe how they use the site.  Once they have poked around, we ask them questions about their experience.  We have improved our technique since the first week of testing, but it is still tricky to ask questions that are not leading.  We do have specific questions we want answered, but we would are also looking for ideas that we may not have considered during design or development – we want them to tell us whatever comes to their mind while using the site.  We have gotten some good feedbackand have been incorporating suggestions into the revised versions of our demos.

Another lesson we have learned is that it is best to somehow indicate to the test panel that we (the people on the phone with them) did not develop the site they are testing.  We don’t want them to hold back on telling us things they don’t like or  to be reluctant to give us their honest opinion.  As much as we appreciate positive feedback, it is really helpful to know what they don’t like or understand so we can try to make it better while we still have time.

We didn’t want to reinvent the wheel. Luckily, we didn’t have to. 

There are many sites out there that let users create profiles—but how many social networks and profiles can one person maintain? We decided not to do what someone else is already doing well, and instead utilize the social networking power that is Facebook, or rather, the ability to relate news to your social network using Facebook Connect.

From the Facebook developers blog: “Facebook Connect is the next iteration of Facebook Platform that allows users to “connect” their Facebook identity, friends and privacy to any site. This will now enable third party websites to implement and offer even more features of Facebook Platform off of Facebook – similar to features available to third party applications today on Facebook.”

This week I am busy designing a system for how comments a user makes on our site will appear on their Facebook profile. Again, not reinventing the wheel here. The Facebook developers wiki makes it pretty clear what the accepted norms are. 

The system I am designing for The Gazette Online this week is very similar to the entertainment celebrity news and gossip site, The Insider. The Insider was one of the first sites to integrate with the Facebook Connect system. Users can choose to add comments they make on The Insider to their Facebook profile wall feed in either a one line, short or full format. 

The good news is that if Facebook approves our implementation of Facebook Connect, the Gazette would likely be one of the first news sites to use the system. But that is a big IF. Facebook has said Connect will roll out November 30, but it’s anyone’s guess if they will hit that deadline, who they will approve and how it will be decided. All we can do is wait, design, develop and hope that the brains behind Facebook Connect like what they see.

We’ve been writing this blog since the project started.  For the first few weeks we wrote about the theory of the project, since that was what we were struggling with - the major decisions of which task to tackle, which features to include, and which to cut.  

Those were the heady days of philosophical discussions about Journalism and Conversations and Democracy and there was much to blog.  

Then came the era of the research, wherein the consumer insights team wrote about their findings, and the industry researchers dug through the host of products already in existence, and the team gleaned many new answers that provided course corrections for the project.  Since then we’ve blogged about some of our industry research discoveries.

The act of blogging served as a catharsis.  It was the cleansing act of articulating the problems we were combating.  It helped us organize, provided objectivity and some of the input we’ve received from comments has been invaluable.

Happy programmers write happy code. by Stuart Tiffen

But after much hand-wringing and painful decisions suddenly, overnight we were in full-on development mode.  Since now our day-to-day grind consists of tackling innumerable design and development obstacles, with some research and now final report and presentation preparation thrown in, such pursuits do not always make for interesting blog posts.

And so, I ask you, our loyal reader(s), what is it that you would like to know about the riddle, wrapped in a mystery, inside an enigma that we call the Crunchberry project?

xkcd Webcomic

One of the challenges in managing the Crunchberry project is understanding what the development team spends everyday working on. It’s a fundamental problem anywhere developers and non-developers work together: programming is looked at as a black box, something people avoid if possible. As a small group students we each fill a variety of roles and this makes it even more important to communicate how development time is spent.

On Friday after our end-of-the week meeting, I took a half hour to explain the basic concepts behind how Web servers and databases work and what frameworks like Django are and what they do. I tried to keep the explanation as simple as possible, explaining key concepts like what a server is, what web and database servers are and how they fit together, what the difference is between a programming language and a software framework.

I won’t take time here to explain these thing when others have done a better job.

My brief tutorial was helpful to the team. People asked questions and generally understood some of the basics of what the development team does.

Django and its Model, View, Template methodology is relatively simple to explain on a high level. For those who are not familiar - this methodology breaks out logically separate parts of a web application. Models represent data in the database, Views handle the request from a visitor, and Templates format and display the data. I showed off my development environment and some code, trying to dispel some of the magic by walking everyone through a trail of code a visitor would experience when using the Web site.

Programming has a high learning curve compared to many things, but as tools get better and people spend more time using computer systems, that barrier to entry gets smaller. The walls that have been built around technology people need to be torn down. Developers should be able to explain themselves with little or no jargon to non-developers and non-developers should show an interest in understanding.

Ask questions. Challenge the developers to explain things plainly. This helps developers to better understand the problem, and non-developers better understand the tools and constraints the developer works with.

So for the past seven weeks or so we’ve been talking about fostering community conversation, civic engagement, and enabling people to discuss the news that affects them. And we have come up with some pretty cool ways that undoubtedly encourage interactivity and will potentially help form an online community of engaged individuals who are motivated to have their voices heard.

But that’s what we need to get at - the people who comprise these communities. As much as the structure of what we create matters (ease of use, clarity, etc.), the fundamental piece to this puzzle isn’t the applications we are creating; It’s the people using it. Passionate people with genuine fervor for the content they are reading are the consumers targeted on the Web.

An understatement to say the least, the members of ESPN.com fit this profile perfectly. I sat down for a phone interview with Patrick Stiegman, vice president and executive editor and producer for ESPN.com to find out more about their Web properties, and the people who traffic them.

“The passion that sports fans have is the fuel that drives our community,” said Stiegman, who oversees production of an estimated 20 ESPN Web properties. “Sports is somewhat tribal - people tend to aggregate around a team, school, or state. With the fan profiles on our site, it’s like a virtual discussion at a bar,” said Stiegman.

The “fan profile” Stiegman refers to is the nexus of ESPN’s social community. With the fan profile, users can post a picture and information about themselves, message one another, see who has recently viewed their profile, add widgets, create a friend list, blog, post photos, join groups, and get updates about their ESPN fantasy sports teams. This virtual water cooler is essentially Facebook for sports fanatics – complete with a comment wall.

In addition to the fan profiles, users are able to personalize the news they receive from ESPN.com via their “MyESPN” page. Modeled after iGoogle, users can keep tabs on news from their favorite teams, check scoreboards, mark their calendars for TV listings, and check progress on their fantasy teams all on this one page. This personalization acts as a gateway to the communities the fans interact with, but according to Stiegman, it is a feature that is not used heavily.

“There is a lot of value around personalization in the news business, but people are hesitant to do it because they are worried that access could be shut down to it,” said Stiegman. “Ten to 15 percent of people personalize. The vast majority of people don’t want to put the time into personalization because they don’t want to miss something else going on in the sports world.”

While personalization (ideally) facilitates interactivity, it appears that many people are not willing to spend the time to customize the content they receive.

So to you, using your RSS reader, iGoogle page, and desktop widgets for news, (and of course, those of you who don’t), how vital is personalization to your Web experience?

A major component of the crunchberry project is researching Web sites and communities that have had some success connecting and engaging their users.  I am intrigued by TED.com, which has been referred to as the “Youtube for intellectuals.”

Ted.com is centered on video content, specifically talks and speeches by “world’s smartest thinkers, greatest visionaries and most-inspiring teachers.” With the aim of making speeches from the TED convention available to anyone who is interested.  I was specifically hoping to learn more about the built-in social network that allows people with similar interests to connect.  There seems to be a higher level of discourse on this site and I would like to know what the people behind TEd attribute this to — is it a reflection on the audience, the content, TED’s design, a combination of all these ideas and more?

The user profiles can be basic or detailed — users can talk about their work, save videos, and include information about their expertise and credentials.  Upon registration users are given the option to specify: Career Information, Organizations, Current Role, language, associations, universities, area of expertise, website links, gender, create a bio, and complete the statements “I am passionate about,” “Talk to me about,” “An idea worth spreading,” “People don’t know that I’m good at,”  and “My TED story.”  I especially like “Talk to me about” because it seems welcoming and provides an opening to engage other users with similar interests.

They are also given the option to complete the statement “I am”  with up to 10 attributes provided (e.g. Activist, Agnostic, Architect, Artist, Atheist, Athlete, Blogger, Brainstormer, Buddhist, Business adviser, Business leader, Concerned citizen, etc.)  Ted.com is giving users several ways to give personality to their profiles – selecting between a list of already specified attributes or by giving them a prompt and then asking them to write about themselves.  Users can flag their favorite content and other users as another method of defining thier interests.

The site provides several pathways for making connections between users — their associations with an organization, common interests or common friends (favorite users).  I am also intrigued by the credential part of the profiles that addresses one of the barriers we have encountered, readers do not perceive comments as believable.  It gives users a sense of who they are communicating with and a clue about whether the commenter is knowledgeable about the topic.  This is also addressed by including a history of comments by the user on individual profiles.

The users can interact with the contact is several ways as well.  They can rate videos or tag them as “beautiful” or “informative” by checking boxes or they can add their comments which appear in a standard scroll-down-to-view comment list.  

TED users can find the most discussed or most watched content so they know what is popular with other users.  The left side of the screen allows you to search videos by the most emailed, most discussed most favorited, most jaw-dropping, etc; users can also find talks by topic: technology, entertainment, design, etc.

TED is a great example of a social network that gives the users a lot of options to find content and to interact with both the content and other users.   I am hoping to be able to speak with someone behind the scenes at TED to learn more about their experiences designing and maintaining TED.com.

Follow this link to read David Pogue’s blog posts about TED in the technology section of NYTimes.com

We Americans like to judge things. We vote for a new leader every fourth year. Pick favorite teams in baseball. And befriend only those deemed popular in middle school. So it was bound to happen. After years of judging each other, we took the game online, and ratings were born. 

There’s binary – the Ebert model – a quick thumbs up/thumbs down. Ratings done by assigning points or stars. Or there’s the more complex. Like the Topix model where users can judge comments as “brilliant,” “nuts,” “clueless,” “racy,” and a whole host of other descriptions. 

Topix

In our weekly design team planning meeting we decided we couldn’t build commenting structures without addressing ratings. But the tricky part is building a system that will not just be a copy of existing models, but rather, be uniquely targeted at each commenting structure we have built. This means we must have a separate model for the question and answer structure, short format and this week’s letters to the editor format. 

Josh and I have made some progress on a rating system for the question and answer commenting structure we designed a few weeks ago. 

We didn’t like the idea of rating questions, at least in the traditional sense. As we are told often throughout life, there are no bad questions and we don’t want to discourage question asking by judging people for their inquiries. We are, however, designing a rating system for questions where the most popular questions would be highlighted. After a question is asked users with the same question can click a box that says, “I have this question too!” This will give priority to the questions readers most want answered, and possibly motivate the reporter to further investigate the story until the answer is found. 

For ratings on answers, we are combining the Topix model with traditional binary commenting. Users click either a thumbs up or thumbs down icon to vote an answer as good or bad. When users click the thumbs up icon, they are given a drop down menu of choices for how to positively rate the comment including “interesting,” “agree,” “helpful,” “insightful,” “informative,” etc. Conversely, when users click the thumbs down icon they are given a drop down menu of choices for how they can negatively rate the comment, including “disagree,” “offensive,” “off-topic,” “incorrect,” “rude,” etc. 

This system is not entirely brand new, it’s not off the wall, but it just might work. It maintains the speed and ease of use of binary ratings, while providing a little more insight into the content of the comments so the user can decide which comments are worth their time. 

While designs for that rating system move forward, Stuart and I are still busy at work designing a rating system for the letters to the editor model. We are struggling with a way to make ratings innovative and uniquely tied to this commenting structure, without adding too many bells and whistles.

What do you think? How would you want to rate letters to the editor? We’re all ears – ready to judge your suggestions and determine the winner.