David Foster Wallace, giving the commencement address at Kenyon College in 2005, among many other things, said:

I submit that this is what the real, no-bullshit value of your liberal-arts education is supposed to be about: How to keep from going through your comfortable, prosperous, respectable adult life dead, unconscious, a slave to your head and to your natural default-setting of being uniquely, completely, imperially alone, day in and day out.

The really important kind of freedom involves attention, and awareness, and discipline, and effort, and being able truly to care about other people and to sacrifice for them, over and over, in myriad petty little unsexy ways, every day. The alternative is unconsciousness, the default-setting, the "rat race" -- the constant gnawing sense of having had and lost some infinite thing.

The capital-T Truth is about life before death. It is about making it to 30, or maybe 50, without wanting to shoot yourself in the head. It is about simple awareness -- awareness of what is so real and essential, so hidden in plain sight all around us...

There's a good deal of evidence from an excerpt from his unfinished book, The Pale King (about a man that works for the I.R.S.) that he was thinking along these lines:

Lane Dean, Jr., with his green rubber pinkie finger, sat at his Tingle table in his chalk's row in the rotes group's wiggle room and did two more returns, then another one, then flexed his buttocks and held to a count of ten and imagined a warm pretty beach with mellow surf, as instructed in orientation the previous month. Then he did two more returns, checked the clock real quick, then two more, then bore down and did three in a row, then flexed and visualized and bore way down and did four without looking up once, except to put the completed files and memos in the two Out trays side by side up in the top tier of trays, where the cart boys could get them when they came by.

Bliss -- a second-by- second joy and gratitude at the gift of being alive, conscious -- lies on the other side of crushing, crushing boredom. Pay close attention to the most tedious thing you can find (Tax Returns, Televised Golf) and, in waves, a boredom like you've never known will wash over you and just about kill you. Ride these out, and it's like stepping from black and white into color. Like water after days in the desert. Instant bliss in every atom.

Of course, Wallace calls such awareness and compassion 'unimaginably difficult' in his commencement address, and when he says he is not claiming any special knowledge or authority on how to do it, he's not being modest. He is talking about his own struggle, a struggle that few are conscious or smart enough to try, and almost no one (especially not these smug Zen monks) would be able to express in such clear and heartbreaking terms.

Those who practice Zen, I think, would say they strive to achieve a "mind as the mind that faces life like a small child, full of curiosity and wonder and amazement. 'I wonder what this is? I wonder what that is? I wonder what this means?' Without approaching things with a fixed point of view or a prior judgement, just asking 'what is it?'" Undoubtedly, this is a wonderful way to exist, and I would myself very much like to live that way.

But imagine Mr. Wallace, bristling with talent, supremely gifted with words, aching to wrestle with a good chunk of what people have done with literature. Could he actually do any of that while detaching himself in that way? It would be like cutting off an arm, or at least a finger. Buddhism seems to offer one track of fulfillment, but it doesn't seem like it encompasses enough for someone like him. Maybe he needed two tracks, one for his life and one for writing, each with different rules. I wish he was still around, so I could see what he would undoubtedly have figured out.

Posted by ben at 12:56 AM | Permalink | Comments ()

New methods for managing people making Web sites are big these days, with the most popular being Agile. Basically it boils down to making a small team of builders completely responsible for a project, and enforcing constant communication as people work on it in well-defined chunks. There are many engineer/programmer proponents for this, but few designers seem to have adopted the approach (though there some, but others are skeptical or downright hostile to it).

The difference is another example of Aristotle vs. Plato, the scientific, evolutionary approach vs. the aspirational large vision. Design wants to create a beautiful vision up-front for what a new world would be, but divorced from the deeper and more meaningful design that engineers do.

Without going into the problems and tensions that have been common in these projects, I think it's safe to say that no one has developed a great 'best practice' for how it can actually work (and I've worked within dozens of Agile teams and read quite a bit about it, so anyone who claims to have it solved I can say is probably pulling your leg). So what could work better, at least?

Just Start

Often there is the assumption that designers need time to develop separate deliverables and have their ideas specified out before they can talk to builders. This is a breeding-ground for mistrust, and more importantly the designers shouldn't be working without an intimate knowledge of the technical design--what information is gathered, how it moves around, and what's done to it. Designers should get into the project at the very start, talk through the project in detail with the whole team, and make decisions with the team. They should be making sketches or prototypes the first day.

Visual and Interaction Design Comes Last

Many sprints I've worked on have tried to have design done a sprint ahead of implementation, or a week, etc. This is sadistic. When engineers are iterating and improving as they go but designers get one shot at it before they've seen any working code, everyone is set up to fail. Rough interaction design and pages should be done collaboratively with engineers as they work, and detailed interaction design and visual design should come last, as a refinement of that.

Actual Alignment

Much of the attempts at collaboration between designers and engineers in Agile projects have put the designer in a rough spot, between the product owner and their team. The designer makes mocks/comps/boards to should how the thing could look, and the product person signs off on that before the sprint starts. The documents for what will be built, however (e.g. the stories, acceptance criteria, etc.), doesn't include the designs. Inevitably, the team makes changes to the designs as they implement, and the designer is either out of the loop (working on a different part, or not part of the team), or has to redesign on the fly in mid-sprint. Designers and engineers should put aside time for 'alignment' tasks, where they discuss and agree on what they will build, either as part of the pre-sprint tasks or the sprint itself. This is similar to 'engineering-only' sprints, which focus on the 'plumbing' of a project and not what users see.

The alternative to having these kinds of deep integration of designers into Agile teams, I believe, is mediocre work, or a lot of additional work to fix the inevitable breakdowns that happen when talented people work together. This is actually much more simple human nature stuff that's been going on in offices everywhere, for a long time, probably since cave-people first called a meeting.

Posted by ben at 3:15 PM | Permalink | Comments ()

Posted by ben at 9:44 PM | Permalink | Comments ()

Palin posted her beliefs about the health care bill. The only one that's available is the House bill HR 3200, so that's what she must be referring to. Below is her text's assertions (minus the characterizations and other material that doesn't deal with facts) and the facts as best I can glean them from reading the bill.

The Democrats promise that a government health care system will reduce the cost of health care...

...but as the economist Thomas Sowell has pointed out, government health care will not reduce the cost; it will simply refuse to pay the cost.

And who will suffer the most when they ration care? The sick, the elderly, and the disabled, of course.

The America I know and love is not one in which my parents or my baby with Down Syndrome will have to stand in front of Obama's "death panel" so his bureaucrats can decide, based on a subjective judgment of their "level of productivity in society," whether they are worthy of health care.

Through researching other blogs and sites, it seems that this is referring to section 1233, "ADVANCE CARE PLANNING CONSULTATION" (it starts on page 425 of the bill). This section sets standards for providing information on what options people have to create legal instruments concerning life and death concerns in their health care, specifically a "living wiil," health care proxy, and other advance directives for individuals to express their decisions about their care if they are unable to themselves. The section establishes a panel of health care 'practitioners' to create a set of national standards for how information about these options is communicated.

The section only mentions providing anyone eligible for Social Security information on these legal instruments, and the panel mentioned has no power to make any decisions about any persons' health care, let alone end-of-life options. The phrase "level of productivity in society" does not appear anywhere in the bill. The section is designed, so far as I can tell, so that people can set down their preferences to choose how they will die themselves, rather than having others make those decisions.

Nationalizing our health care system is a point of no return for government interference in the lives of its citizens.

Posted by ben at 9:40 PM | Permalink | Comments ()

I believe that there are two kinds of ideas in the world: those that divide things into two types, and those that don't... and then there's a third, which tries to wriggle out of either. This is one of those.

The basic idea: a better interface to data would be to turn the data itself into the interface, as a flow between an overview and actual experience.

What is an 'overview'?

• visual language of overview: parallel


• movie trailer, menu, signage


• graphs, piles, sorts, maps, matrices


• optimizes attention


• optimizes action


• browsing, analysis, editing, outlining, listing


• powerpoint, excel

What is 'experience'?

• visual language of experience: serial


• long, slow changes


• reading content


• still photos


• based on important details


• narrative


• based on sustained attention


• email & twitter

This is false duality of course; the actual value of either references the other, and both are necessary and interesting and have been around since the dawn of time. The reason this is an interesting thing to revisit is that there is a huge advantage in new media and social contexts: you can jump between them quickly and continuously, so that they start to merge into a 'flow.' This flow is an interface, perhaps a good one.

In basic terms, the flow interface resembles a lot of existing interfaces:

• a list of emails > reading an email


• table of contents > page


• map > walking

• use content itself in the overview, not a label/symbol/sign


• browse and refine both the overview and the experience


• interact with one through the other


• interaction with an overview element shows an experience


• interaction with an experience element references an overview

Now speed things up, so that the interface can offer:

• "you can sort this stuff into piles three ways, which one works the best?"


• "we have these three prototypes that show how the product could work"


• "have a taste of 12 dishes before you pick your meal, then change your mind halfway through"

And of course, this can benefit from the language of data visualization, but we need a framework for overviews and a framework for experiences, not just one, and the ability to pick the framework itself should be content-based and usable through familiar conventions:

• show proportional distances instead of vertices or measurements


• make maps


• show size relation, piles


• show a tree graph


• show nodes and edges

And here is where I sketch some actual designs for the very vague concepts I'm throwing around, but of course I haven't got there yet. But I think it would be easy to show new interfaces for twitter, netflix, and digg that work this way.

Posted by ben at 11:31 PM | Permalink | Comments ()

The below is my attempt to remove the frothy and breathless tone from "Generation M manifesto" by Umair Haque, because I liked it in many ways. It is definitely more boring, but I hope more real as well. I don't believe any manifesto can express the right amount of humility towards these questions, but it can emphasize belief in the possibility for something better, so I focused on that.

Dear gradualists, ideologues, and partisans,

We are in a time of large differences between groups, young and old, east and west, rich and poor, but one where many of the traditional ideologies seem to have been scrambled both by a global economy and crisis and fundamental changes in how information is shared through technology.

Everyday, we see the costs of doing the same things. It looks like some big, new, and huge problems are looming, but the solutions that are talked about are old, timeworn, and plain unambitious.

Old ideas of generational shift and left/right politics no longer seem to work. We can't use simple terms in this new, hypercomplex and interdependent world. We need a new way of seeing and strengthening the relationships we have, not a manifesto of ideas.

These times demand not single solutions, but systems of solutions, involving less large-scale business and more individual opportunity. Less ideology, and more practicality.

Businesses and governments must get connected to and become responsive to a public that is comfortable using social tools to express themselves in massive ways. The hyper-connected "sea of green" in Tehran is the model for a new, speeded-up politics.

Much of this new world no longer requires massive capital or leverage to work, and banks should play a smaller and more supporting role. A smaller role for finance means less focus on lucrative return.

The huge accumulation of risk and the massive gaming of global markets resulted in crisis. This should drive a lot of wealth away from financial instruments and towards tangible, collective works and accomplishments that everyone can benefit from.

Growth as a goal incentivizes distortion. We should prize flexibility and agility, so that no matter which way the markets go, business can prosper and act to benefit everyone.

Rather than nurturing a few elites (or even oligarchs), the new economy should be a huge number of distributed markets. It wouldn't be entirely controllable, and those that would want to profit from it will have to compete for influence just like everyone else.

We've seen the consequences of short-term thinking in spending and debt and felt the pain; now we should start working on ideas that are built to last a generation, not 5 years.

Our sense of ourselves has moved too far towards what we can do as individuals; it's time to nurture some shared beliefs, projects, and experiences.

Our culture should connect us to our shared past, and remind us that when it comes to the most meaningful things for human beings, there's usually nothing new under the sun.

In order to provide some label for what's needed, let's call it Generation "M."

This is not a movement in the traditional sense (our society is too distributed one manifesto, one protest, one set of ideas). It's more the recognition that a new set of norms is needed for a new time, the recognition of a shift. It's the belief that we can come up with practical ways to live and work together that do a better job at caring for each other.

Ideologies and manifestos will always run up against their own logical extremes. Gen M is the belief that innovative ideas married with historical consciousness and brutal practicality can be vastly more powerful, and meaningful.

Big changes will be necessary. The institutions and norms that we've lived within for a long time are too fragile to pass on to our children.

Since the end of the Cold War, we've lived with cheap, easy, expensive lifestyle, but one that was empty of meaning and for which we have little to show. Every age has a large responsibility, and this, I think, is ours: to foot the bill for yesterday's profligacy -- and to create, instead, an authentically, sustainably shared prosperity.

Anyone -- young or old -- can answer it. Generation M is more about what you do and who you are than when you were born. So the question is this: do you want to build the new relationships, businesses, and systems we need? Or do you want to keep repeating the same old ideologies, marching in protests, or clinging to dying institutions?

Posted by ben at 10:44 AM | Permalink | Comments ()

It seems like there is a fairly straightforward deal possible to save the business of putting out newspapers (the news is fine, doesn't need to change!). Make it a much cheaper version of the cable business, where subscribers buy into a much-enhanced version of something they get a basic version of for free. Major ISPs like AT&T, Comcast, etc. could create a open news consortium that users could buy into by adding $2 to their monthly bill (this has nothing to do with network neutrality, by the way, just creating the same mechanism that supports free pop music radio).

Assuming that only 5% of broadband customers of the top 5 U.S. ISPs agree to that, that's $120 million each year. If even just newspapers banded together for this, ISPs would have a strong business incentive to offer the surcharge to their subscribers. Any content provider with a certain level of traffic could offer their content only to subscribers of the consortium, splitting that dollar 50/50. This money would be paid out to content providers on a strict traffic basis. Providing content this way would be much more efficient than via paper, and the writers, editors, and photographers would be responsible to their audiences first, as it should be (with advertising revenue on top of that). And they could continue to provide news summaries and headlines to news aggregators like Google News.

It wouldn't be the 40% margin of years-ago, but it would be a going concern. All that it would require would be placing the needs of the business as a whole above the fantasy that there is something basically wrong with journalism, Web sites, any particular newspaper, or an attachment to paper as media. And also the willingness to take action instead of letting things slide further towards... nothing.

Posted by ben at 11:52 AM | Permalink | Comments ()

Walter Kirn's review of David Denby's book Snark is pretty fun reading:

He wants to correct and restrain, using scholarship and logic, perhaps the keenest, most reflexive, prehistoric and anarchic of simple human pleasures, short of eating or achieving orgasm. The act of laughter, this would be. Or, for Denby, the act of low, illicit laughter -- laughter enjoyed for the wrong reasons and provoked by the wrong lines. Whether laughter for the right reasons is even possible, given humor's subversive, corrosive history, is a difficult philosophical question, of course, but Denby feels that it is. This follows from his belief that the impulses to giggle, grin and cackle (and the various means for stimulating these impulses) can be, and ought to be, consciously aligned with civic virtues and literary standards, lest our society laugh for no just cause, at jokes that aren't witty enough to laugh at and that may even be plain stupid and malicious.

Posted by ben at 11:24 AM | Permalink | Comments ()

I like the quote from John See's on the cover of the plan, start with the applications, not the technology:

Typically, technology committees go before school boards asking for a computer lab, or computers for classrooms. The first question board members will ask is, "Why do you need them?" Why not answer that question in the plan? It may be better to go to a school board saying, "This is what we want our students to be able to do"

• reading, researching, exploring
• writing, editing, taking pictures, collecting research, making docs
• sharing, messaging, commenting, evaluating
• planning, calendaring, collaborating (via the above applications)

• A campus-wide wireless network, 54Mbps or so.
• An internal and external Web server.
• Any computer that can run Mozilla Firefox 3
• A bunch of new Web sites that enable the above applications (could be built on top of an application server like Drupal)
• A set of policies for learning contexts that govern what data and work can go from internal network out to the public Web, and vice versa

• A lot of fancy new computers are not needed (just more older 'commodity' computers that can run a good Web browser well)
• No further work on the internal ethernet network is needed (better to replace ethernet with more wifi hubs or extenders)
• The real time, resources, and energy should be spent on the Web-based applications that will be directly part of the curriculum, instead of equipment

• Standardize on free, bulletproof software instead of a single hardware platform: a robust Web browser that can use Web-based applications like Firefox 3.
• A 1 year-old refurbished Dell laptop with Ubuntu costs $400, new MacBook costs $949, but they have identical performance running Firefox.
• Computers can be more easily maintained by standardizing on one simple configuration for everyone. If a computer is running badly, it can be wiped and re-imaged (instead of troubleshooting software installations).
• Software can be limited to what's free and available on all operating systems (Mac, Win, Linux): a Web browser like Firefox 3, a text editor, and media management applications like Songbird & Picasa.

Posted by ben at 11:45 PM | Permalink | Comments ()

Posted by ben at 4:57 PM | Permalink | Comments ()

Posted by ben at 12:28 AM | Permalink | Comments ()

I loved watching Lawrence Lessig on the Colbert Report, he was so articulate and impatient for things to make sense after so much stupidity. I liked his idea of the hybrid economy -- "read/write culture" and commercial culture in a symbiotic relationship, and wanted more. I have yet to receive the book, so I don't know where he goes with the "hybrid economy" idea. But, I want the tools of this read/write culture to be connected not just to commercial culture, but to the rest of the economy, so they don't go away in the tsunami that's sweeping away a great deal of the economy right now (man, I can only pray for the New York Times).

I don't think it's overstating the case to say that anything valuable to someone can be better if people have better information and more relationships to the people who make their living from it. If I work for, buy from, live around, or share a town with the workers in any industry, small company, or even just a gas station, that situation is better with more transparency, communication, and relationships between the people than what we have today.

A pharmaceutical company should be giving unmoderated web tools to patients who are dealing with a disease, so they can connect to each other and negotiate the completely dysfunctional health care system we have. A car company should design and build cars with the public, through open beta car designs, candid information about what the strengths and weaknesses of the company are, and total access to decisions executives make. Even the gas station should be posting the information it gets from its contacts about why the price is what it is, and what to expect next week. Government should be making the millions of decisions and policies it produces an ongoing, distributed dialogue with citizens, where the strong local opinions and experiences get rolled-up into the scorecards for massive programs.

Free-flow of information makes the people who have to buy things more powerful, and it makes those who want to sell something make better things. And it makes it much easier to see exploitation, avarice, and abuse. Ultimately, that's good stuff for everyone who shares a city, or a town, or a street. The wonderful sites that form the bases for these tools (Wordpress, Wikipedia, Facebook, Digg, Youtube, etc.) should be right in the middle of these real human needs, not just off to the side for entertainment (also, I don't know how they'll survive otherwise).

And the nice thing about all of this is that it's actually less money than what companies flush down the toilet today, in a process known as marketing. There's just not that much point in a game of using media to associate good feelings with a product when everyone can find out much more about it directly from real people. "Brand" is just not as valuable either; the products are it. Enough with the promises that a company is part of your family or people "love" a brand. It's corrosive and wrong to try to hijack our actual, human experiences and feelings to sell products. We don't have to do that anymore.

We do all have to make a living, pay for food, hopefully do work that's valuable to someone. There's nothing inherently sleazy in that, it's just practicality. These social web tools can make work, products, and business more honest and open. More socialist, in fact, and I am very comfortable with that.

Posted by ben at 3:00 AM | Permalink | Comments ()

Recently many commentators have blamed the current financial crisis on a law enacted under the Carter and Clinton administrations, the Community Reinvestment Act. They say that the law forced banks to issue loans to lower income an minority applicants that could not afford them. Usually there is some grain of truth to these things, but far as I can tell, not this time:

• The Community Reinvestment Act applies only to depository banks.
• 67% of 'subprime' mortgages were issued by non-depository investment banks or other firms that were not regulated by the CRA, not depository banks, which were.
• Of those issued by depository banks, only 54% were 'subprime.' That means that, according to a CATO study, only 17.4% of 'subprime' loans, or 2.3% of all mortgages issued in the U.S.
• Collateralized Debt securities (the drop in value of which caused the collapse of Merrill Lynch, Lehman Brothers, Bear Stearns, AIG, etc. and the present uncertainty about credit) were created by combining regular and 'subprime' mortgages. These securities were issued only by investment banks. Doing the math, that means that about 89% of all the securities issued had no CRA-covered loans in them.

The CRA was a very minor player in the financial crisis, issuing a small percentage of all loans. It did not apply to any of the banks that issued about 89% of the risky mortgages in the U.S. The CRA was irrelevant to the investment banks and other firms that issued risky loans.

"There has been a tendency to conflate the current problems in the subprime market with CRA-motivated lending, or with lending to low-income families in general. I believe it is very important to make a distinction between the two. Most of the loans made by depository institutions examined under the CRA have not been higher-priced loans,16 and studies have shown that the CRA has increased the volume of responsible lending to low- and moderate-income households." -- Janet L. Yellen* President and CEO, Federal Reserve Bank of San Francisco, March 31, 2008.

The markets that collapsed had almost no regulation. In fact, the Commodity Futures Modernization Act of 2000 explicitly barred their regulation; the law was written by Republicans and signed by Clinton -- there is plenty of blame to go around. One of the very few completely unregulated markets in the world stopped "functioning properly" (in Bush's words). I can't see why anyone would look further for the cause than that uncontroversial statement.

UPDATE: My brother Michael Clemens, a Harvard economics Ph.D, says no major mistakes! Whew. He also says to keep in mind that the "very simple fact is that no one completely understands the roots of the financial crisis, because it is a complex, emergent, chaotic phenomenon" (hope it was okay to quote you Michael). That is for sure.

*Some have said that as a Clinton appointee, Yellen is biased. She was appointed after his first choices were rejected by the Republican congress; Yellen was cited as a nominee the Senate Finance Committee could support, and they ratified her appointment unanimously.

Posted by ben at 4:52 PM | Permalink | Comments ()

There are lots of bits and pieces of information around that seem to tell you about what's going on in the world, but even the best of these are, finally, someone's subjective judgement call, what they think is happening. They may be professionals, etc., but I want to get a feel for myself of what's happening, not just accept the opinions of others-- for example, there is usually a pretty serious gap between the politics of Berkeley, California and the rest of the U.S. There's no easy way to get a sense of how big the gulf is, no easy road to figure out how anyone could conceivably want to have someone like Sarah Palin as Vice President.

Of course, I can immerse myself in firehoses of data of all kinds: populate my RSS reader with Republicans, watch 500 channels, visit 10,000 folksy web sites. But I really just want broad themes, changes, and trends. I want an engine that analyzes and summarizes lots of raw data into a few groups of things that are similar.

And no, this isn't some smirky internet-age bullshit! Based on Ben Fry's recommendation in his book, I got a copy of Exploratory Data Analysis via inter-library loan. It was published in 1977, before most people knew what a PC was, and waaay before anything like the Internet. In the very first chapter, John Tukey starts with talking about making simple "stem and leaf" graphs with paper and pencil of the megawatts generated by hydroelectric dams in the U.S. Why? He wants to show you how to "write down a bunch of numbers in such a way as to give a general feel of 'what they are like.'" What a great thing! I don't care too much about the numbers, I just want to know if there's any rough pattern there, and there is: most of these dams either generate about 30 kilowatts, or around 1,100, two groups that tell me where to start if I want to know more about that (I don't).

In information retrieval land, this is known as clustering (one of my hobby horses that I usually talk about with anyone I know for longer than 30 minutes). Instead of a laundry list of thousands of things, I get a set of groups of the words that are used together often. I can get a feel for what's happening in the data without sorting through it myself.

For example: I vaguely know who Kenzaburō Ōe is, but I don't really know what his books are about. I just want a feel for that, with some idea of what the themes of his books were. I can either read a page of Google search results, or I could see derived clusters for those documents. To me, seeing the groups is a much better interface to start exploring if I want to know about him (and I do).

The motivation to get a sense of what's out there in the world is an old one-- it goes back farther than 1977, even. Now that more and more of the stuff of everyday life (cats, politics, hobbies, cancer, errands, religion, etc.) is in the form of data online, I can imagine doing that. I should heavily note that clustering techniques applied to social data still produce very uneven results, and a lot more work is needed. But it's becoming possible to think of a better interface to information than a search box.

Posted by ben at 10:32 PM | Permalink | Comments ()

Ok, I hate tag clouds. With a passion. Never again. But, since I was sorta happy with the term frequency normalization thing I was working on, I figured what the heck (the road to heck is paved with half-assed text analysis). And another thing, aren't there like, 2,810,000 (and one) twitter clouds out there?

But here it is anyway, twittercloud. It's slightly different than other clouds, in that you can generate a cloud from a query term (not just whatever is hot or an individual's tweets), and you can see what tweets are generating the term right in page. It auto-refereshes with ajax to give you that clean feeling. Also, I happen to think frequency normalization gives a nice weighting that makes more interesting words pop up. Anyway, it was fun!

UPDATE: Now includes Flickr photos (a tag search based on the query term and cloud word) and much better twitterbot detection.

Posted by ben at 11:09 PM | Permalink | Comments ()

Assuming that Ubiquity incorporates features to: a) add annotation elements to a page, b) stores annotations locally, c) lets me organize & collect annotated pages, and d) above all provides an easy way to share annotations socially, it seems like it could become my new bicycle! It would be:

• a standard XHTML container that works everywhere, on/offline
• a place to stash pages rather than remembering links
• a notebook with bits and pieces of other pages
• host an annotated page instead of drawing on a screenshot
• add video/images/media to a page, make snarky comments on it, and send it around
• make 2 or 3 pages into one page
• make a page into a wiki, an exquisite corpse, etc. etc.

And some cumbersome scenarios become really easy:

• I can email a web page that has a message, a highlighted portion, and a link to the original page, as an attachment that needs no internet connection.
• I can email a map that has text pointing to a place on the map
• I can choose and send 12 books for preschoolers from amazon (with titles, authors, thumbnail, price, and link) via an email
• I can turn the NL east division part of the MLB standings into a widget that appears on my blog, with the erratic Mets highlighted and have it update as they surge, then collapse again.

(I know there are ways to do all these things now via other services and tools. But most of them require a good chunk of time, an account on something, etc. The thing that will make web pages into a medium for sharing stuff (instead of just a url) is making it easy, three-four clicks, simple, etc.)

I love all the other features of Ubiquity (I'm an old Quicksilver user). And there are plenty of no so great aspects of this. But I think this part of Ubiquity has the most potential to make sharing things on the web a whole order of magnitude easier, in a way that no bookmarklet, plug-in, greasemonkey script, or single-purpose site could.

Posted by ben at 11:40 PM | Permalink | Comments ()

Deep in the depths of the dot-com bust, when I was lucky enough to have work of any kind (in my case it was commuting 200 miles each day contracting for Cisco), the people I worked with were fatalistic about the future of the work we did. Many people thought the money had been drained out of the exploratory culture of the boom, and that we were in a very conservative time.

The truth was the opposite in fact; much of the boom was spent doing variations on e-commerce ventures and content sites, with the idea that each vertical or market segment would need it's own one of those. Counter also to the conventional wisdom, these sites were not created because their entrepreneurs thought that business logic had been suspended, but instead because the tiny community of venture capitalists were funding only specific things. The 'bust' came not out of the sites' lack of income (it was no secret that very few ever had enough business to survive), but in the withdrawal of venture funds.

After the bust, many people left the business for school, or other jobs. A few people (who couldn't do anything else) kept creating new things however, and many of them were more creative than anything that came before (and set the stage for the current boom). Google, Blogger, Napster, and Friendster (a non-semantic search engine, easy blogging tool, media sharing tool, and social network, the hallmarks of today's Web) were all created in 1999 or 2000 in the shade of obscurity. The examplars of the successful social media Web were born in 2001 and 2002, when little venture capital flowed: Wikipedia, Flickr, del.icio.us, and Facebook.

In retrospect, it's easy to see that it's more possible (though harder of course) to create something new when no one is watching and no one cares, out of shear love of an idea. But it's not something that many seem to remember these days. Instead, it feels like 1999 all over again, with companies launching sites with the same functionality and ideas, venture-funded start-ups doing variations on social media ventures and aggregation sites, with the idea that each vertical or market segment will need it's own one of them. In fact, I can't think of a single company that's launched since 2002 that isn't a derivative of one of the bust's labors of love.

The complete lack of creativity and new ideas is typical today. I guess we all have something to protect: the market funding, the high-paying jobs, the advertising revenue. The more to protect, the more conservative it gets. I hope that someone, or some company, somewhere, will think back to the first boom and remember what happened. The business cycle will happen, the money will evaporate, and only those that have made something truly new will survive that.

Posted by ben at 9:00 AM | Permalink | Comments ()

China is going through huge growth, about 10% a year. While there remains a huge part of the country to lift out of poverty, one would think that this amount of growth would result in plenty of opportunity for everyone to make a good deal of money with less competitive pressure than in 'mature' markets. But apparently in some areas, the opposite is true; since the plants are partly owned by the government (and the managers not entirely responsible for keeping costs below income), they will sell products below costs in order to compete with other plants. This is, according to the article I read ("Ownership Distortion, Low Level Technology, and Excessive Competition," by Jie Ma and Weiying Zhang), the economic state of "over-competition," where too many competitors end up compromising the market and their own viability.

This rings a bell, because in a venture-capital fueled, social-media start-up frenzy much the same condition exists. 35 life streaming apps. 4,324 photo/video sharing sites. 321 twitter clones. Countless business-vertical mashups (102 travel business mashups alone). Why will any of these gain enough traction to garner general use (and "change the world" as their entrepreneurs like to say) if the market is in this crowded state and the owners are not risking their own money? Is this lottery-like state really an ok but brutal winnowing process, or more like the airline industry's race to the bottom?

For example, two competing commenting platforms in the same market causes some upset as one tries to market to the others' users. When so many blogs are being added all the time and the phenomenon is going mainstream, there should be plenty of room for at least two products. Instead, there are an overwhelming number of choices (who has time to sort through 14 options?). Decision theory says that "more choices may lead to a poorer decision or a failure to make a decision at all", losing situation for all these creative, hardworking start-ups.

So what's the alternative? I think the collections of start-ups in overlapping spaces should, as quickly as feasible, abandon the idea that they can create the be-all, end-all (e.g.) social shopping user base and content silo that will crush competing sites. Instead, they can open-source the basic containers and objects (much as Open Social and OAuth have done in one discreet area). That way, the personal data is by definition portable and owned by no company, and start-ups that aren't really, functionally different will have a shorter life (and I will never have to upload my profile picture again, ugh), and the viable sites will have a clearer shot at being viable businesses. What is the likelihood of this happening? Probably not so great, but these are the same geeks who advocate the Linux model for operating systems (where an app can be easily recompiled in the different distributions, and each distribution can use parts of others). There's really no reason other than a kind of protectionism not to extend this to Web applications.

Posted by ben at 10:57 AM | Permalink | Comments ()

In a recent post on Hitwise by Heather Hopkins, "Content Aggregation is King?," the bind that existing news media is in is highlighted again:

"Aggregators are taking a larger piece of the pie but the size of the pie is growing with visits to content creators and all News and Media websites growing. The trouble is - creating all that content is expensive. It's tough to justify the cost of content creation if those that sift and sort are gaining on those that create."

The deal between the a consortium of newspapers and Yahoo to outsource listings functions to Yahoo in return for a cut of revenue was predicated on the idea that there is a way to do some basic arbitrage on this situation, but even that doesn't work over the longer term. News media simply can't justify the cost of their content creation staffs, even if they get rid of the printing presses and costs of classifieds businesses. And they have yet to really deal with this (probably because they are still much larger than the aggregator sites even now).

Maybe, as the news media starts seriously shrinking the content pool, the aggregators will start to strike deals like Yahoo did, but more expansive and lucrative. Done right, such deals could raise all boats. For example, rather than aggregating traffic on one domain, Yahoo (or Google, etc.) could drive traffic to news media sites that federated their news with aggregated content from all over the Internet. A network of these sites, served by a single ad engine, could broaden the ad inventory far beyond what Yahoo would ever be able to support on its own and save some amount of a business model for content creators.

It may take a risk by someone on the scale of the 2001 AOL-Google deal, which similarly tied a content network to advertising engine. Marissa Meyer talked about it as "a very big bet, a revenue guarantee" to AOL:

"It caused a huge amount of controversy at the time because by some of the models that we had run, the deal was going to bankrupt Google. Like Jonathan Rosenberg actually got up on the table and jumped up and down about how much we shouldn't do this deal because Google was going to go bankrupt. We had models, one said that we were going to go bankrupt, one which said we might break even... and one year into the deal what we saw was that by signing AOL and broadening the reach of our advertising network we attracted so many more advertisers, and RPMs (revenue per thousand pageviews) went up across the network and we outperformed our expectations by a factor of two, maybe even three times."

Posted by ben at 2:24 PM | Permalink | Comments ()

Many of the good ideas below came in talking about personal information with Doug Fritz, the bad parts are my own ideas.

When Friendster and Flickr started getting traction (2002 and 2004 respectively), I wasn't a huge fan of the functionality, but the social aspect was totally addictive. I got seriously into each (and even got comfortable with the photostream concept eventually, though the fact that I can't easily control the order of the photos without using the organizr still bothers me). These sites ability to connect me with other people blew away all other shortcomings. At the time, the fact that I was putting so much personal information into a public space didn't bother me at all; I assumed a that the Web made old ideas of privacy kind of obsolete, and we would all eventually adapt and make new social norms to cope.

But now that social networks are taking on large chunks of people, I am not so sure. Facebook with a dozen real friends is a much different thing than Facebook with 140 "friends." The information I post there is, to me, the things that define me personally as I choose to represent myself. In the social network context however, the sites ultimately have more control over how I get represented than I do.

I had an experience with that when my story about a stolen cellphone and pictures from the thief unintentionally uploaded to Flickr briefly became news. There were some wrinkles in the story that caused people to be suspicious that I was doing marketing, or lying about the phone being stolen (as opposed to lost). I knew the facts of the matter and was not looking for any large amount of attention in posting the story. But once there was that attention, the people who read the story simply did not believe it and couldn't be convinced otherwise. They used personal information about me that was publicly available (on Flickr, my blog, and other sites) to make the case that I was perpetrating a hoax (for example the fact that I worked at Yahoo was used to say that I was marketing the Flickr service). Even the Reuters reporter that interviewed me asked "how can I know you're not deceiving me?" I didn't have an answer for her. The effortless flow of personal information (flow that I started) did not lead to anything like the truth, and caused some people to think that I was not a real person at all. Privacy as we knew may be gone, but the idea that everyone is now a public figure (with none of the protections of public figures) strikes me as wrong.

We can say that in the context of computer networks "information wants to be free," and I support the idea that copyright is an outmoded framework for intellectual property. But when it comes to representing ourselves, how can it be that personal information should flow everywhere and be used by anyone however they want? Open Social and the Data Portability initiatives are good starts for independent mechanisms, but I still have to give these systems a truckload of personal information, with no way to take it back after it's out there.

An alternate solution would be to allow people to own their personal information store, and choose to allow social network sites access to this store. Sites that behaved badly could be banned. This is much like OpenID and Oauth in concept, where one's identity is tied to a DNS-like way of creating a single namespace for unique user identifiers. It could take the form of a fancier version of an "Attention Profile Markup Language" file; a "Social Profile Markup Language" file, say. It would be stored on my own web server and under my direct control. If I wanted to share with Friendfeed or mybloglog (for example) what sites I've been posting to, saving, liking, or reading, I could allow them to access my SPML file under the condition that it be removed if I decided not to use the application any longer. (This is a geeky solution, but that's usually where these things start.) There should be a better solution to the new portability of social data than exists today, or my own understanding of my personal information will mean less and less.

Posted by ben at 12:47 PM | Permalink | Comments ()

In an article about the politics and economics of consumer choice in cable entertainment ("Bland Menu if Cable Goes à la Carte") in the New York Times, the author talks about how it's actually a good thing that the costs of producing niche programming are borne by all cable subscribers, as otherwise such programming would be very expensive and possibly not attract enough buyers to exist at all (even the most popular channel, ESPN, would rise from $3/month to $12/month). This makes sense, and could be a useful way of thinking about two other media businesses that are in some trouble: the Web and newspapers, both of which are in that bad à la Carte downward spiral.

In the case of the Web, despite the large amount of investment and growth in Web users, the economics still do not support niche content in any serious way. Small publishers have not made a living off of their content, depending on VCs or large distribution networks for investment to stay afloat, or publishing despite the lack of money. And even the most popular blogs on the Internet make a tiny amount of money compared to any other publishing medium like books, magazines, TV, etc. For example, Boing Boing is estimated to bring in $50,000/month in advertising revenue, which sounds like a lot until you consider that a single full-page ad in a local magazine like New York generates the same amount of revenue. Traditional media companies like AOL and Yahoo! have tried to aggregate many small publishers and sell ads across all of them, but despite getting a lot of traffic for this content, the rates for ads remain low (and at flat growth rates). The money remains in search text ads. Jaron Lanier pointed out in an Op-ed that this model doesn't work for content authors.

In the case of newspapers, their audience is falling steadily and won't sustain the costs of keeping the staffs of reporters and editors working (let alone the costs of publishing in print). They have had some success working with internet networks like Yahoo!, but this is likely not sustainable nor will it replace enough revenue to keep things going at current spending levels (magazines haven't had the same problem, but that's another topic).

So given that the cable model supports a number (not a huge number, but a number) of niche content players, and given that the Internet functions well as a big lab for new ideas but not for building content businesses, maybe what's needed is a sort of content consortium, or at least association of creators, with thresholds for membership and the ability to bargain collectively for better compensation. This would be different from a union, more like the Author's Guild or Screen Actors Guild, where plenty of work is done outside their auspices, but productions that generate a lot of revenue must conform to standards of pay. The cable model shows that if companies seeking to exploit content have to buy in to a collective pool of content, content costs are lower while allowing even niche content to thrive. This model doesn't need the cable networks to work for the author's benefit.

In a rough environment for content creators, where the economics are against them (but no one wants them to stop creating), the authors and makers should hang together more than they are. Some kind of association would re-balance a business that is out of kilter, draining the money out of a very valuable part of the culture we live in.

Posted by ben at 9:57 AM | Permalink | Comments ()

[see updates, below]

My brother, Michael A. Clemens, is an expert on issues of economic development and was recently asked to do a book review for the pre-eminent journal of thinking on international politics, Foreign Policy. I am not able to appreciate his work fully, but I know he's very smart and worked very hard to get to the point where his ideas can get the influence they deserve, and I am very proud of him and happy to share his name! In his honor, I want to rip off one of his frameworks and use it for my own purposes; pretty bad behavior, but that's family for you.

He divides thinkers on economic development for poorer countries into two camps: "planners" (who want to create grand visions for the future and fund large projects) and "searchers" (who want a more incremental and experimental approach to finding what works). If I butcher his arguments completely, I could say that he concludes that neither camp adequately and seriously takes on the complexities of actually improving the economic situation of developing countries at all, however. Development is hard to do, and there are no short-cuts.

Far away from the moral value of his work and in my petty world of Internet bullshit, there is an analogous divide, between "storytellers" and "searchers." The storytellers are usually the big thinker types and designers, who believe in their ability to imagine a solution, build excitement, tell the story of how it will make things better, and make it happen. The searchers don't believe in stories anymore, they trust hard facts and cold reality, eschewing fanciful stories for careful scientific principles and evolutionary methods.

In engineering, the culture is heavily weighted toward the searchers. In design and marketing, the culture is weighted toward the storytellers. For example, today visual artists were described to me as interfering with good design decisions in Web sites, "failed artists" who shouldn't been taken as seriously as "an architect, say." This is clearly a searcher perspective, heaping scorn on the emotional side in favor of 'more serious' approaches. Conversely, the storytellers will deride the 'incrementalism' and 'lack of ambition' of the searchers, wishing for the 'next game changing idea' to emerge out of some yeasty marathon whiteboard brainstorm.

Recently this article about the weaknesses of "democratized design" looks disapprovingly at the new "hack culture" of participatory innovation, saying that genuinely new things ("like the iPod" --ugh) can't be produced "by committee"; the clear implication is that a single person's great idea (storyteller) will beat a mob of tinkerers (searchers). I'm not sure about that either; seems too grandiose.

You can even find parallels to this in ancient philosophy, where Plato exalted the unseen ideal (storyteller) and Aristotle wanted to ban plays and poems because they distorted clear thinking (searcher). Being a designer, I constantly find myself on the wrong side of whatever group I'm in. Being able to draw a picture of something does not count nearly as much as code, but I am addicted to the power of stories. I have attempted to bridge this gap by becoming a designer who builds, but most people I know still sort themselves firmly into one of the two groups.

In fact, the abstract nature of Web products allows people to work purely in one camp or the other, without a hard need (other than the success or failure of their projects) to cross the divide in their thinking. This is apparent all the time in the whining of designers and their fantasies of becoming dictators to engineers bound to execute whatever they say, and the resentment engineers have toward the promiscuous, irresponsible, and arbitrary ideas of airhead designers. This game is stacked against a synthesis right now, and we're all worse off.

Ultimately, I come down on the side of the storytellers, but try to make a practice of humility about my ideas, checking them relentlessly against reality. The best practice is like life-drawing from a model: make a bold line and decide how you want to show a subject, but constantly look back and forth from your sketch to the person in front of you, making sure your drawing still has a likeness.

Update: This article on egotistical architects bemoans the masturbatory elitism of top architects, and feints briefly with democratic design ideas before coming down on the side of the elitists:

[Democratic design advocate Bruce] Nussbaum is dreaming if he thinks democracy and design are seriously compatible. Truth is, they're not even love muffins.

This is partly because specialism - as in honed, polished expertise - is the core of what we call civilization. Designing your own may bring spiritual satisfaction, and homegrown design may be less ill-advised than homegrown, say, brain surgery. But be it blog, bog or village, it still has that unmistakable backyard look.

"Design democracy" is a feelgood idea, and that's about the only quality it offers. As the Sun Microsystems co-founder Bill Joy told last year's Aspen Ideas Festival: "If I was competing with the US, I would love to have the students … spending their time on this kind of crap. To be a great designer is very hard. It's not about your friends [liking] something you did."

It's hard because humans are hierarchical primates. Only the few can be great at design or anything else. To be a great architect - a Brunelleschi, say - may require a self-belief so vast as to be limitless, but it also requires more than a Botoxed self-portrait as proof.

Meh, seems like they are trying to have it both ways. I think it's all about tension, not one extreme or the other. You could create a simplified two-card Oblique Strategies deck, with one card reading "let the usage and ideas of ordinary people be your guide" and the other reading "trust only your instincts and ruthlessly pursue what you think is best," and switch strategies randomly!

Update 2: An Economist blogger flagged Michael's review, and highlighted an idea that was implied in the review my brother wrote, that the only development strategy that made sense was allowing freer emigration to functioning economies (rather than trying to pursue 'planner' or 'searcher' strategies in countries that are so screwed up that no strategy is implementable. This made me wonder what the analogue to my world might be, and I think it's something along the lines of opening up companies and products so that their data and ideas are more portable and free. If products are 100% interoperable and your data is your data (not trapped or siloed) but completely portable, we could have a faster, easier evolution of better networked products and services. As it is, people satisfice; sticking with some things just because it's hard to move data around and nothing works easily with anything else.

Posted by ben at 11:21 PM | Permalink | Comments ()

In a meeting last week, I said I wanted to try to involve some other designers in a project, but just got a blank look and the question "what can design do?" -- designers can't really help with the actual making of the product, they meant. Ugh. It's true: Silicon Valley-type engineers and other very smart people have put designers to shame by creating products and services that no one imagined but that many can't live without, and for which "design" (the role, as it is commonly understood) is just not critical.

Rather than working within people's existing needs and expectations (the traditional approach of practical-minded designers), the work of engineers has changed people's habits, thinking, and behavior. Designers are relegated to optimizing a use-case, working out the complexities, or putting a nice shiny coat of varnish on top (and rounding the corners). This approach, and the "undesigned" look of Google, Craigslist, MySpace, etc. has led people to talk about design more as marketing (and to be avoided) than as part of the product. The real work is in the engineering design, and invention happens there first.

I've tried the fancy idea of using "design thinking" as a way of making product decisions, to be more strategic (and less production-oriented). It works for IDEO, but not so well for me -- it feels more like business development than inventing. Heroically, Bruce Nussbaum has tried to save us all by equating design and innovation, as the secret sauce that will enable the West to compete with China, etc., but I don't think it's gonna stick.

Ok, I'm starting to finally get it: this may be a time when the myriad possibilties of creating new-to-the-planet things means engineering (exploring the deeper design of systems). But I am not ready to give up all the fun to engineers! I would rather undergo some wrenching adjustments to what it means to "design" and be relevant again. I want to be able to say that 'design creates new things to make people happy,' not 'design makes it look pretty or more usable.'

Like an architect would, I am going to have to know a lot more of what an engineer does, and be able to work directly with a good part of the code if I really want to play in the deep end of the pool. I am going to start designing in code, maybe bullshit, arty code, but code nonetheless. This is a little rough and later in life than I would have liked, but what the fuck, bring it on!

technorati tags:design, designers, engineers, web

Posted by ben at 3:40 AM | Permalink | Comments ()

This screengrab is the list of tags for a Flickr photo I ran across randomly. It feels more like micro (or nano?) blogging than tagging, in fact it's anti-social!

I love casual personal revelations. It feels like there is a great deal of rich meaning in fragments of thoughts and ideas set down semiprivately (in a search box, or a tagging input) that is lost in the ultra-self-conscious blog post. Also, making a disorganized list of thoughts is more personal than a diary entry. It feels like there are several epiphanies somewhere in gathering my private thought fragments and using social media and networks to connect them to other people's fragments.

technorati tags:tags, terms, private, thoughts

Posted by ben at 10:41 AM | Permalink | Comments ()

Posted by ben at 9:36 PM | Permalink | Comments ()

At my job (and in Silicon Valley in general I think), Agile development is supposed to be the best way to do things. Specifically, the Scrum method encourages an evolutionary approach, where an entire team works together in short cycles and creates a product iteratively (first a barely-functional demo, then a rough prototype, then a more finished version, etc.), without an overall plan.

Scrum works very well for implementation, since it empowers a team to do its own planning, breaks a large project down into brief chunks and reduces the amount of churn and decision-making that can cause delays and rework. It is an approach with an evolutionary philosophy: no one can know at the outset what the right solution is, so let's just start working and refine as we go.

Coming from the perspective of design, there are some built-in tensions to this approach, however. Design as a discipline has as an article of faith that people can sit down and create a drawing or visualization of an idea that will make things better, and that this idea can help push the expansion of what's currently possible to engineer (this is not to say that only designers can do design, just defining the activity, everyone does design all the time).

Scrum allows no time for the messy task of conceptualization (other than beforehand), and divides up all the aspects of a project into small, unintegrated tasks, making it hard to see the 'big picture.' The team-focused environment encourages strong collaboration, but does not ensure that the value of a product is realized by itself (and the proponents of the method don't claim it does, either). In fact, the method ensures that engineers, designers, and product people spend less time thinking about whether the product is the right product overall.

What's needed is less of a focus on methodology, and more of a focus on ideas and people's talents. Scrum is a fine method to do work, but doesn't actually solve the problem of making good products. Rather than talking about intelligent design vs. evolution and assuming that one way of doing things explains the entire world, we should be valuing time spent on hard thinking and idea generation just as much as we value the time spent implementing things. There are certainly limits to emphasizing creativity, but there is a sameness to Web products these days I think. We need more really new ideas.

Update

People are fundamentally spiritual beings, with big needs for meaning and a sense of themselves in relation to the world (whether they believe in God or not). Scientific thinking is always needed and important. But what we most need now is that messy old-time religion: creative thinking, design thinking, the faith that we can know the right thing to do, to make the world better.

Posted by ben at 8:00 AM | Permalink | Comments ()

There's nothing quite like taking care of my daughter to blast away selfishness, bitch-slap my inner child, napalm my therapist, etc. And I'm doing just great. Right about the time I'm congratulating myself on how far I've come is great timing for another reminder, though.

Just the other day, we had a blow out. I had fed her (well, she's pretty good at feeding herself, I was just there to make sure she didn't decide to throw the bowl) and I was eating my own food, which she wanted some of. I refused to give her any, and she put her head right down on the cold floor and wailed and cried. She kept it up, no matter how I try to distract her, and trying to hold her just made it worse. Even after a half-hour of this I couldn't comfort her, and only my wife was able to hold her and calm her down. Over my wife's shoulder she looked at me with the burning eyes of someone safe from the enemy.

We make up of course, she's fine (she's always great). But damn it all if I wasn't left with hurt feelings. In some ridiculous way I want taking care of her to be personally fulfilling. What a crock of shit! It's one of those times when I have to look at my psyche and just shake my head, sadly. (I'm not alone though. Every month we get Parents magazine for some reason -- we didn't order it and never renew it -- and in every issue there is an article about how to get the most fulfillment out of raising a child.) The latest version of my personal mantra is to take care of her:

1. ...without being a rigid asshole
2. ...while staying open and intimate with her
3. ...without being wishy-washy, negotiating with her, or becoming pedantic
4. ...while staying in sync with my wife's style and expectations of her

and above all:

5. ...with the ability to let go of any desires for personal gratification I might want for what I do for her.

Technorati Tags: baby, parent, psyche

Posted by ben at 8:08 AM | Permalink | Comments ()