Twitter, Friendfeed, and Facebook have seemingly converged on what has become the major reason to be connected to others on a social network: sharing short updates, links, photos, etc. A concept for mozilla's Firefox also looks similar, and lifts ideas from iTunes to help organize things. The designs share some major elements:

1. Publisher an area to enter some text, a url, or other media, to publish it out to friends or the public.
2. Items an area where items are listed, either most popular or latest items, or some subset of items
3. Sets an area where the set of items to show is chosen; it can be all items, items from certain friends or other sources, or user-created sets
4. Notifications two of the designs have an area to surface notifications, recommendations, alerts, or otherwise push to the user stuff that might be interesting

I like this design convergence, if only because establishing a vernacular for these kinds of sharable media apps will lead to more familiarity with the interface as more people start to use them, and form the basis for the next leap towards an interface that supports more sophisticated forms of sharing and publishing.

Douglas Bowman had to quit Google, and Valleywag explains it all for you (to hell with Owen!). I had a similar experience at Yahoo, so I'm only surprised Douglas lasted this long. The comments on Valleywag are really sad though; a palpable hostility towards "precious," "childish," "short-sighted" designers (you can look for yourself, I'm not linkin'). A lot of product design is really bad, sometimes the designers get a chance to do something really good with a job, but not often.

Jared Spool, an Extremely Important Person, once told me over Pad Thai that "visual designers are just failed artists." I took that personally, being a failed artist (heh), but didn't understand why the "visual" distinction was necessary... I guess he would have to be a failed artist as well if he just said "designers"? Or he has to get the frustration of just speaking at conferences out somehow.

Facebook's redesign inspires widespread unhappiness and derision. On Techcrunch, incredible bile is thrown at the designers. I can't say I like it, but why does anyone think that Facebook is anything other than an ongoing experiment? Facebook users are not "customers," they are collaborators in inventing new ways of being connected, and much is required of them sometimes. The new Facebook stuff is not very good, but at least they haven't given up like Irene Au and the crew at Google.

I have attempted to be useful as a designer, and had enough failures and successes to know a good deal of humility. There's no research method, process, innovation technique, conference presentation, or even extra-talented designer that magically makes good stuff.

UPDATE: Another comment thread at an article about designers quitting Google, filled with ignorant stuff. It really does seem that there is a cultural lack of understanding about design and what it is. I suppose the only real solution is to increase the overall cutlure's understanding and ability to parse visual and experiential elements; then (and probably only then) will people want a specialist to make the choices about those things instead...

• Breast Cancer Society
  
• Cancer Fund of America
  
• Children's Cancer Fund Of America
  
• Children's Charitable Foundation
• Detectives Benevolent Association
• Disabled Veterans Services
  
• Firefighters Assistance Fund
  
• Foundation For American Veterans
  
• Law Enforcement Alliance of America
  
• National Children's Leukemia Foundation
• United States Navy Veterans Association
  
  

Google has created a slew of innovative products, born from original thinking and supporting experimentation and new ideas, and Marissa Mayer has been a large part of the company's success. Google's success was not built on design however; it was built on the humbling (if you're a designer) fact that the thinking and engineering was so good that design was almost irrelevant. I think that's generally a good wake-up call to designers, and I've tried to reinvent what I do for myself around a deeper definition of design, one that tries to encompass engineering. This is a pretty typical story for a Silicon Valley tech company design decision:

A designer, Jamie Divine, had picked out a blue that everyone on his team liked. But a product manager tested a different color with users and found they were more likely to click on the toolbar if it was painted a greener shade. [...] Mr. Divine's team resisted the greener hue, so Ms. Mayer split the difference by choosing a shade halfway between those of the two camps. Her decision was diplomatic, but it also amounted to relying on her gut rather than research. Since then, she said, she has asked her team to test the 41 gradations between the competing blues to see which ones consumers might prefer.

If Google took design as seriously as they do engineering, they would not focus on details, but remaking interaction design and visual sensibility. Marissa Mayer wouldn't make a comment about grey text, she would be wondering how Google could give users better interfaces to information than an empty box. Google should swing for the fences again with new thinking, not imagine that because they are successful they do everything right.

It seems like there is a fairly straightforward deal possible to save the business of putting out newspapers (the news is fine, doesn't need to change!). Make it a much cheaper version of the cable business, where subscribers buy into a much-enhanced version of something they get a basic version of for free. Major ISPs like AT&T, Comcast, etc. could create a open news consortium that users could buy into by adding $2 to their monthly bill (this has nothing to do with network neutrality, by the way, just creating the same mechanism that supports free pop music radio).

Assuming that only 5% of broadband customers of the top 5 U.S. ISPs agree to that, that's $120 million each year. If even just newspapers banded together for this, ISPs would have a strong business incentive to offer the surcharge to their subscribers. Any content provider with a certain level of traffic could offer their content only to subscribers of the consortium, splitting that dollar 50/50. This money would be paid out to content providers on a strict traffic basis. Providing content this way would be much more efficient than via paper, and the writers, editors, and photographers would be responsible to their audiences first, as it should be (with advertising revenue on top of that). And they could continue to provide news summaries and headlines to news aggregators like Google News.

It wouldn't be the 40% margin of years-ago, but it would be a going concern. All that it would require would be placing the needs of the business as a whole above the fantasy that there is something basically wrong with journalism, Web sites, any particular newspaper, or an attachment to paper as media. And also the willingness to take action instead of letting things slide further towards... nothing.

Walter Kirn's review of David Denby's book Snark is pretty fun reading:

He wants to correct and restrain, using scholarship and logic, perhaps the keenest, most reflexive, prehistoric and anarchic of simple human pleasures, short of eating or achieving orgasm. The act of laughter, this would be. Or, for Denby, the act of low, illicit laughter -- laughter enjoyed for the wrong reasons and provoked by the wrong lines. Whether laughter for the right reasons is even possible, given humor's subversive, corrosive history, is a difficult philosophical question, of course, but Denby feels that it is. This follows from his belief that the impulses to giggle, grin and cackle (and the various means for stimulating these impulses) can be, and ought to be, consciously aligned with civic virtues and literary standards, lest our society laugh for no just cause, at jokes that aren't witty enough to laugh at and that may even be plain stupid and malicious.

I like the quote from John See's on the cover of the plan, start with the applications, not the technology:

Typically, technology committees go before school boards asking for a computer lab, or computers for classrooms. The first question board members will ask is, "Why do you need them?" Why not answer that question in the plan? It may be better to go to a school board saying, "This is what we want our students to be able to do"

• reading, researching, exploring
• writing, editing, taking pictures, collecting research, making docs
• sharing, messaging, commenting, evaluating
• planning, calendaring, collaborating (via the above applications)

• A campus-wide wireless network, 54Mbps or so.
• An internal and external Web server.
• Any computer that can run Mozilla Firefox 3
• A bunch of new Web sites that enable the above applications (could be built on top of an application server like Drupal)
• A set of policies for learning contexts that govern what data and work can go from internal network out to the public Web, and vice versa

• A lot of fancy new computers are not needed (just more older 'commodity' computers that can run a good Web browser well)
• No further work on the internal ethernet network is needed (better to replace ethernet with more wifi hubs or extenders)
• The real time, resources, and energy should be spent on the Web-based applications that will be directly part of the curriculum, instead of equipment

• Standardize on free, bulletproof software instead of a single hardware platform: a robust Web browser that can use Web-based applications like Firefox 3.
• A 1 year-old refurbished Dell laptop with Ubuntu costs $400, new MacBook costs $949, but they have identical performance running Firefox.
• Computers can be more easily maintained by standardizing on one simple configuration for everyone. If a computer is running badly, it can be wiped and re-imaged (instead of troubleshooting software installations).
• Software can be limited to what's free and available on all operating systems (Mac, Win, Linux): a Web browser like Firefox 3, a text editor, and media management applications like Songbird & Picasa.

Recently I've found myself having very unpleasant conversations with people about work. Often, both of us are in a bad situation, and there is no easy way to make the project better, only 'least bad' answers. It's a negative situation. Dealing with the disappointment and upset in this situation often leads people to act badly, myself included. I push too hard, questioning people's conclusions too much, and generally am a nuisance. One might even say an "asshole." I am not mean, but probably irritating.

When I went to school and worked my first few jobs, demanding bosses were the norm. Animated discussions, arguments, and emotion were part of caring about the work. I had a boss who extracted good things out of bad situations on a regular basis. I hated him at first, but eventually grew to admire him and respect him. Bewilderingly, I no longer have passionate arguments with people about work. Instead, discussions and meetings are meant to reinforce decisions already made. Negativity is to be avoided, and any criticism is almost offensive (even if just in tone).

I certainly can understand the desire to make work fun, lighthearted and focus on the positive at work, but I think something was lost. Progress and good work doesn't come easily. In fact, it's really easy to do mediocre work when everyone is afraid of failing (and getting laid off). There are great people who can avoid that and still be full of sweetness and light, but those people are few and far between, and the emotions involved are not going to last as long as good work. Abusive, mean behavior is wrong no matter what. But I find myself wishing for a couple of assholes at work, people who would shake things up, force the issues, and push past the usual solutions. But for now, I'll just try to stay positive :)

Newspapers and magazines are losing money very quickly. News content remains very popular, but most of it is available for free. News aggregator sites serve newspaper content to large audiences for free, and Craigslist has killed the major source of ad revenue that newspapers need, and people are used to getting magazine content for free online. Newsweek is exiting the mass-market, and it appears that many large newspapers will as well. Many people agree that journalism is an important part of a functioning political culture and society, but no alternative to the present course has emerged.

Henry Blodget points out that even if it stopped printing on paper, the New York Times could not pay for it's news room full of reporters and writers. Walter Issacson proposes a "pay by the slice" model, like iTunes for music. Amazon introduces the Kindle 2 as an analogous device for newspapers. Chris Anderson says "free" is the only business model that works, and it's okay if that doesn't support a lot of journalism. Andrew Keen says "mass-amateurization" is destroying our culture and must be fought. Michael Kinsley says that it's okay if we have a much smaller number of newspapers. Dave Winer says kill all the elitist journalists, we can make a new journalism out of bloggers and participatory media.

The conclusion I come to is that the sea change that has happened is the end of 'passive' culture, not journalism. It should be possible to re-invent journalism as a richer, participatory medium, with experts and professionals directly collaborating with (and directly responsible to) the enthusiasm and amazing creations of bloggers, makers, photographers, people of all kinds. The more sharing and creating of things that's happening online, the more engagement and value there will be there, and it will become something that can pay the bills for those who do it well.

Large media companies and newspapers could protect their futures best stopping the attempt to hold on to existing ad rates and formats, and instead moving their best advertisers quickly into performance-based online properties and formats. No one yet knows what will work, but they can do themselves a huge favor by trying a dozen experiments instead of moaning. And the bloggers & geeks can help invent new things, instead of imagining that this is a revolution where it will be a victory if a valuable part of society goes away.

The feet of one of three Palestinian siblings from the Al-samoni family, killed by an Israeli tank shell, are seen in the mortuary of Al-Shifa hospital, on January 5, 2009 in Gaza City. Seven members from the Al-samoni family were killed including the mother, three children and a baby, when an Israeli shell struck their house south of Gaza city. (Abid Katib/Getty Images, boston.com)

Marc Ambinder writes

Many a friend has asked me what I think of the Israeli invasion. I have some private thoughts on the subject, but they're not particularly interesting. I've studied enough, prayed enough, spent enough time in Israel to get the hang of why the conflict appears so tragic and intractable...

...nothing works for very long in the Middle East. Gaza is where dreams of reconciliation go to die. Gaza is where the dream of Palestinian statehood goes to die; Gaza is where the Zionist dream might yet die. [...] My paralysis isn't an analytical paralysis. It's the paralysis that comes from thinking that maybe there's no way out. Not out of Gaza, out of the whole thing."

Here, journalists on the ground in Gaza talk to a Current Vanguard reporter:

What has happened in Gaza is/was, as far as I can tell, cruel, pointless, and another example of how little we know as human beings about how not to totally fuck everything up. It is nauseating to watch the father lamenting the death of his daughter, alive just two hours before. I feel that I have to respond, but in the face of this overwhelming suffering and with such overwhelming problems, is that the right way to be thinking? The above smart people who have actually been there, etc., sound like they are stating the truth of the situation to me; for my own actions, that is where I would leave it (with thanks that U.S. role in the region will probably be a lot different with Obama). But I have been prompted by other people to do more to protest the obscene amounts of innocent death in Gaza, through small gestures like groups on Facebook or marching in protests with signs. It feels wrong and weirdly beside the point to me, but a lot of people feel strongly about it, so here goes.

As best I can tell, whatever solutions can be found to change, even in small ways, the situation will come not from protests or activism. Everyone who is at all directly connected to the conflict is desperately aware what everyone thinks, and has their own idea of justice worked out. Protests have become background noise, even at a large scale; a hundred million Europeans protesting couldn't stop Bush from invading Iraq. Politics has changed a great deal, and needs new tactics. (I have some ideas about that, but most people seem to be protesters, cynics, or oblivious, so I have some issues with finding someone who gives a shit.) Protest may be better than doing nothing, but that is about all it is. Like Marc, I am aware of how beside the point my own judgments and needs for action are, yet keep trying to create some activity, find something to do to push away the horror as it unfolds.

I follow a blog called "The Online Photographer" by Mike Johnston, an experienced photographer and writer who was the editor of Photo Techniques magazine for about ten years. It's a good blog, and he knows what he's talking about. I was sort of surprised to read this in his post (he's referring to a discussion of 'bokeh' at another site, photo.net):

It's a bit disorienting for me now when I post at other sites; despite the fact that my name was referenced several times in the thread before I commented, no one paid the least bit of attention to anything I said. Not that it was so important...it's just that, around here, I tend to get listened to. A lovely luxury, and thanks for that.

I've been taking pictures for a long time (far too long to have learned as little as I have!). When I was a teenager, I developed and printed black and white film, at the Salt Lake Art Center (now I can admit that I should have been paying for the darkroom time, but I simply walked in and used the equipment and chemicals, for months). In those days, they taught the Zone System. When I managed to get it right, it worked well, but it was tricky. Years later, I have a digital camera that produces decent exposure ranges straight out of the box, no thought by me required. Nonetheless, I thought I would look around for information on how to more directly control exposure.

The Zone System is (and here, people who know more will cringe) a way to plan how a picture's lights and darks will be captured and printed. Spot meters that older cameras had (like my Minolta SRT-303) just measured the exposure off the part of the image in the middle of the frame, so if there was another part of the picture that was much darker or lighter, that part would be way too light or dark.

For example, if I took a picture of a person in a car in bright sunlight, the spot meter would tell me to expose for the sun reflected off the car, say 1/500 of a second at f/16 (with ISO 400 film). If I did that, everything but the hood of the car, including the person, would be completely black. Following the Zone System, I decide that the highlights on the car would be the brightest thing in the picture, and the person would be in the middle range. So, I increased the exposure 3 stops, to f/5.6, and the person's face becomes visible (while the highlights on the car become pure white). It took a lot of practice to make these decisions, however.

Now, meters in cameras are using the entire image to decide the exposure, basically building in the Zone System into the camera's exposure calculation (this is the "evaluative" mode my camera has). If you are interested in more direct exposure control, then, it seems that you are left with:

• using the camera's meter in spot mode, manual exposure, and using the traditional zone system
• using the camera's meter in evaluative mode, automatic exposure, and let the camera own the exposure

Neither one of these is particularly satisfying to me; the evaluative mode is better than I am at exposure calculation even if I set the exposure manually (since I still use the meter), and I really have no desire to go back to spot metering. I found a third option:

• Underexpose everything, shoot in RAW mode, and adjust the values of the picture after it's copied to the computer

This has produced the most satisfying results so far; without any special software or too much worry about exposure, I can still use my Zone System knowledge to peg parts of a picture to particular values of light and dark I choose.

• I underexpose (I set the EV compensation to minus 1/2 stop) because digital cameras don't capture as much information about the bright areas of the picture as film does, so I want to be sure I get whatever detail is in the highlights
• I use RAW mode because it lets me change the exposure of the picture after it's shot without losing any information
• I adjust the exposure on the computer because I still want to decide what the most important part of the picture is (and what should get the 'middle range' in the photo). Almost all the time, this is as simple as adjusting is the white, grey, and black points (something every single program for handling digital photos can do; Picasa, iPhoto, etc.).

Sometimes I will set two or three other points to use a transfer curve in the RAW conversion, but not so much anymore. The photo above was taken in this way (I show it in black and white because I think it has a great middle dynamic range that illustrates the idea -- the pretentiousness of black and white is just a bonus). Next, I am going to have to figure out how to get a handle on the weird white balance problems I have...

I loved watching Lawrence Lessig on the Colbert Report, he was so articulate and impatient for things to make sense after so much stupidity. I liked his idea of the hybrid economy -- "read/write culture" and commercial culture in a symbiotic relationship, and wanted more. I have yet to receive the book, so I don't know where he goes with the "hybrid economy" idea. But, I want the tools of this read/write culture to be connected not just to commercial culture, but to the rest of the economy, so they don't go away in the tsunami that's sweeping away a great deal of the economy right now (man, I can only pray for the New York Times).

I don't think it's overstating the case to say that anything valuable to someone can be better if people have better information and more relationships to the people who make their living from it. If I work for, buy from, live around, or share a town with the workers in any industry, small company, or even just a gas station, that situation is better with more transparency, communication, and relationships between the people than what we have today.

A pharmaceutical company should be giving unmoderated web tools to patients who are dealing with a disease, so they can connect to each other and negotiate the completely dysfunctional health care system we have. A car company should design and build cars with the public, through open beta car designs, candid information about what the strengths and weaknesses of the company are, and total access to decisions executives make. Even the gas station should be posting the information it gets from its contacts about why the price is what it is, and what to expect next week. Government should be making the millions of decisions and policies it produces an ongoing, distributed dialogue with citizens, where the strong local opinions and experiences get rolled-up into the scorecards for massive programs.

Free-flow of information makes the people who have to buy things more powerful, and it makes those who want to sell something make better things. And it makes it much easier to see exploitation, avarice, and abuse. Ultimately, that's good stuff for everyone who shares a city, or a town, or a street. The wonderful sites that form the bases for these tools (Wordpress, Wikipedia, Facebook, Digg, Youtube, etc.) should be right in the middle of these real human needs, not just off to the side for entertainment (also, I don't know how they'll survive otherwise).

And the nice thing about all of this is that it's actually less money than what companies flush down the toilet today, in a process known as marketing. There's just not that much point in a game of using media to associate good feelings with a product when everyone can find out much more about it directly from real people. "Brand" is just not as valuable either; the products are it. Enough with the promises that a company is part of your family or people "love" a brand. It's corrosive and wrong to try to hijack our actual, human experiences and feelings to sell products. We don't have to do that anymore.

We do all have to make a living, pay for food, hopefully do work that's valuable to someone. There's nothing inherently sleazy in that, it's just practicality. These social web tools can make work, products, and business more honest and open. More socialist, in fact, and I am very comfortable with that.

Photowalking, originally uploaded by George.

George Oates (who I know some and whose work I like tremendously) was laid off in the latest round at Yahoo. Also, many extremely talented people from Brickhouse (Samantha Tripodi, Chis Martin, Jeannie Yang, Ben Ward, Ken Thornhill, Premshree Pillai, and Kevin Thornback, none of whom I knew well, I just saw and was awed by their work). Like Tom Coates ("Still reeling from the last few weeks"), I am still trying to get my head around the entire thing. I still don't understand how these people got cut. I can't imagine Flickr without a key part of its DNA. I have respect for Kakul, Heather, and the rest of the crew, but I think this shows that the decision making there has gone off the rails. I started using Flickr in 2005, and have loved it and the culture that has developed around it. But, I am going to move my photos off Flickr and find a new home for them, with many regrets.

In the larger context of Yahoo, it is doing yet more damage to itself. It simply won't survive as a commodity site or development platform alone. Mail, a front page, and news (all indications are that every other part of the company is flat or declining) are not enough to sustain the company at its current size, and focusing on advertising innovation while the entire structure goes through a recession seems wildly off-base. What they should do is do what other smart people did during the last bust: invent something really good. That's when people started Flickr, and Google, and Blogger, and on and on. Despite many assets and a lot of value, Yahoo's management would do better to improve the company's prospects by laying themselves off.

update: a couple of people have said that they don't understand the 'protest' I'm making; I should have been clearer. It's not a protest at all, Flickr is still a great thing and the people there are good, etc. I'm quitting it just because I had more of an emotional relationship with it than a practical one, and that's been changed. It's a purely personal thing, left over from the time when sites like Flickr were labors of love, built mostly from enthusiasm. That's rare these days, but that's why I do what I do, through booms and busts.

<?php
$contact = array("name" => "Ben");
$contact_encoded = json_encode($contact);
header('Content-type: text/html');
echo $contact_encoded;
?>

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<script type="text/javascript" src="http://code.jquery.com/jquery-latest.pack.js"></script>
<script type="text/javascript" charset="utf-8">
$(document).ready(function() {
	$.getJSON('http://servejson.php?callback=?', 
	function(data){
		$("#display").text(data.name);
	});
});
</script>
</head>
<body>
<div id="display"></div>
</body>
</html>

echo $_GET['callback'] . "(" . $contact_encoded . ")";

This is a good time to think about the identity of Barack Obama. Right now he's a blank screen that many different groups are projecting their wishes on to. He's not from any of their constituencies really, though it's clear he has his proclivities. When he came to Chicago, he was more "mutt" than anything else, searching for a sense of belonging. He pretty much made a choice to join black culture, to inhabit the identity and the role, just like he chose to go to an elite school and become a community organizer. But he did all that while staying removed from the visceral nature of some of the old crappy American fights (race, class, culture), a remove that let him see those fights more clearly (as his speech on race in Philadelphia attests).
So he's pretty much free of attachment to the old clashes, though he understands them. He can choose more intellectually what he wants to do. That could be dangerous (Robert McNamara was surely one of the smartest, most methodical people in the U.S. while he dragged the country into Viet Nam), but right now it is such a relief to have a president that doesn't seem trapped in any one corner.

Recently many commentators have blamed the current financial crisis on a law enacted under the Carter and Clinton administrations, the Community Reinvestment Act. They say that the law forced banks to issue loans to lower income an minority applicants that could not afford them. Usually there is some grain of truth to these things, but far as I can tell, not this time:

• The Community Reinvestment Act applies only to depository banks.
• 67% of 'subprime' mortgages were issued by non-depository investment banks or other firms that were not regulated by the CRA, not depository banks, which were.
• Of those issued by depository banks, only 54% were 'subprime.' That means that, according to a CATO study, only 17.4% of 'subprime' loans, or 2.3% of all mortgages issued in the U.S.
• Collateralized Debt securities (the drop in value of which caused the collapse of Merrill Lynch, Lehman Brothers, Bear Stearns, AIG, etc. and the present uncertainty about credit) were created by combining regular and 'subprime' mortgages. These securities were issued only by investment banks. Doing the math, that means that about 89% of all the securities issued had no CRA-covered loans in them.

The CRA was a very minor player in the financial crisis, issuing a small percentage of all loans. It did not apply to any of the banks that issued about 89% of the risky mortgages in the U.S. The CRA was irrelevant to the investment banks and other firms that issued risky loans.

"There has been a tendency to conflate the current problems in the subprime market with CRA-motivated lending, or with lending to low-income families in general. I believe it is very important to make a distinction between the two. Most of the loans made by depository institutions examined under the CRA have not been higher-priced loans,16 and studies have shown that the CRA has increased the volume of responsible lending to low- and moderate-income households." -- Janet L. Yellen* President and CEO, Federal Reserve Bank of San Francisco, March 31, 2008.

The markets that collapsed had almost no regulation. In fact, the Commodity Futures Modernization Act of 2000 explicitly barred their regulation; the law was written by Republicans and signed by Clinton -- there is plenty of blame to go around. One of the very few completely unregulated markets in the world stopped "functioning properly" (in Bush's words). I can't see why anyone would look further for the cause than that uncontroversial statement.

UPDATE: My brother Michael Clemens, a Harvard economics Ph.D, says no major mistakes! Whew. He also says to keep in mind that the "very simple fact is that no one completely understands the roots of the financial crisis, because it is a complex, emergent, chaotic phenomenon" (hope it was okay to quote you Michael). That is for sure.

*Some have said that as a Clinton appointee, Yellen is biased. She was appointed after his first choices were rejected by the Republican congress; Yellen was cited as a nominee the Senate Finance Committee could support, and they ratified her appointment unanimously.

There are lots of bits and pieces of information around that seem to tell you about what's going on in the world, but even the best of these are, finally, someone's subjective judgement call, what they think is happening. They may be professionals, etc., but I want to get a feel for myself of what's happening, not just accept the opinions of others-- for example, there is usually a pretty serious gap between the politics of Berkeley, California and the rest of the U.S. There's no easy way to get a sense of how big the gulf is, no easy road to figure out how anyone could conceivably want to have someone like Sarah Palin as Vice President.

Of course, I can immerse myself in firehoses of data of all kinds: populate my RSS reader with Republicans, watch 500 channels, visit 10,000 folksy web sites. But I really just want broad themes, changes, and trends. I want an engine that analyzes and summarizes lots of raw data into a few groups of things that are similar.

And no, this isn't some smirky internet-age bullshit! Based on Ben Fry's recommendation in his book, I got a copy of Exploratory Data Analysis via inter-library loan. It was published in 1977, before most people knew what a PC was, and waaay before anything like the Internet. In the very first chapter, John Tukey starts with talking about making simple "stem and leaf" graphs with paper and pencil of the megawatts generated by hydroelectric dams in the U.S. Why? He wants to show you how to "write down a bunch of numbers in such a way as to give a general feel of 'what they are like.'" What a great thing! I don't care too much about the numbers, I just want to know if there's any rough pattern there, and there is: most of these dams either generate about 30 kilowatts, or around 1,100, two groups that tell me where to start if I want to know more about that (I don't).

In information retrieval land, this is known as clustering (one of my hobby horses that I usually talk about with anyone I know for longer than 30 minutes). Instead of a laundry list of thousands of things, I get a set of groups of the words that are used together often. I can get a feel for what's happening in the data without sorting through it myself.

For example: I vaguely know who Kenzaburō Ōe is, but I don't really know what his books are about. I just want a feel for that, with some idea of what the themes of his books were. I can either read a page of Google search results, or I could see derived clusters for those documents. To me, seeing the groups is a much better interface to start exploring if I want to know about him (and I do).

The motivation to get a sense of what's out there in the world is an old one-- it goes back farther than 1977, even. Now that more and more of the stuff of everyday life (cats, politics, hobbies, cancer, errands, religion, etc.) is in the form of data online, I can imagine doing that. I should heavily note that clustering techniques applied to social data still produce very uneven results, and a lot more work is needed. But it's becoming possible to think of a better interface to information than a search box.