practicalist

sharable media design convergence

2009-04-21T16:38:25Z

Twitter, Friendfeed, and Facebook have seemingly converged on what has become the major reason to be connected to others on a social network: sharing short updates, links, photos, etc. A concept for mozilla's Firefox also looks similar, and lifts ideas from iTunes to help organize things. The designs share some major elements:

Publisher an area to enter some text, a url, or other media, to publish it out to friends or the public.
Items an area where items are listed, either most popular or latest items, or some subset of items
Sets an area where the set of items to show is chosen; it can be all items, items from certain friends or other sources, or user-created sets
Notifications two of the designs have an area to surface notifications, recommendations, alerts, or otherwise push to the user stuff that might be interesting

I like this design convergence, if only because establishing a vernacular for these kinds of sharable media apps will lead to more familiarity with the interface as more people start to use them, and form the basis for the next leap towards an interface that supports more sophisticated forms of sharing and publishing.

don't hate the designers

2009-03-22T03:17:25Z

Douglas Bowman had to quit Google, and Valleywag explains it all for you (to hell with Owen!). I had a similar experience at Yahoo, so I'm only surprised Douglas lasted this long. The comments on Valleywag are really sad though; a palpable hostility towards "precious," "childish," "short-sighted" designers (you can look for yourself, I'm not linkin'). A lot of product design is really bad, sometimes the designers get a chance to do something really good with a job, but not often.

Jared Spool, an Extremely Important Person, once told me over Pad Thai that "visual designers are just failed artists." I took that personally, being a failed artist (heh), but didn't understand why the "visual" distinction was necessary... I guess he would have to be a failed artist as well if he just said "designers"? Or he has to get the frustration of just speaking at conferences out somehow.

Facebook's redesign inspires widespread unhappiness and derision. On Techcrunch, incredible bile is thrown at the designers. I can't say I like it, but why does anyone think that Facebook is anything other than an ongoing experiment? Facebook users are not "customers," they are collaborators in inventing new ways of being connected, and much is required of them sometimes. The new Facebook stuff is not very good, but at least they haven't given up like Irene Au and the crew at Google.

I have attempted to be useful as a designer, and had enough failures and successes to know a good deal of humility. There's no research method, process, innovation technique, conference presentation, or even extra-talented designer that magically makes good stuff.

UPDATE: Another comment thread at an article about designers quitting Google, filled with ignorant stuff. It really does seem that there is a cultural lack of understanding about design and what it is. I suppose the only real solution is to increase the overall cutlure's understanding and ability to parse visual and experiential elements; then (and probably only then) will people want a specialist to make the choices about those things instead...

charity fraud

2009-03-16T01:20:16Z

Our house gets calls at least once a week on behalf of several different charities, each with familiar-sounding names:

Breast Cancer Society
Cancer Fund of America
Children's Cancer Fund Of America
Children's Charitable Foundation
Detectives Benevolent Association
Disabled Veterans Services
Firefighters Assistance Fund
Foundation For American Veterans
Law Enforcement Alliance of America
National Children's Leukemia Foundation
United States Navy Veterans Association

It turns out that the charities are very bad at what they do, handing out little money and paying a lot for fund raising to a telemarketing company called Associated Community Services. This means that only a small portion of the money donated goes to help anyone -- ACS keeps the rest. For just the State of Washington, ACS raised $1,152,000, but was only able to pass $353,000 of that to its 14 client 'charities'; the "Breast Cancer Society," operating in several states, manages to devote just 3% of the money it raises to actual services, "Cancer Fund of America" manages 9%, and "Firefighters Assistance Fund" manages to spend just 5% of the money it raises on assistance. ACS has even harassed people while soliciting donations, it seems. While all of this is sounds like it should be illegal, it isn't. The most that Attorneys General in Kentucky, Iowa, Conneticut, and Michigan have been able to do is make public warnings about the fundrasing.

Many sites have noted the suspicious nature of ACS or the charities, and amazingly representatives from ACS seem to be posting rebuttals and misinformation on some of them to try to obfuscate what they are doing. There's a special circle of hell reserved for this company and its ilk.

Marissa Mayer is a gigantic success, but she does not know anything about design

2009-03-02T03:47:58Z

Google has created a slew of innovative products, born from original thinking and supporting experimentation and new ideas, and Marissa Mayer has been a large part of the company's success. Google's success was not built on design however; it was built on the humbling (if you're a designer) fact that the thinking and engineering was so good that design was almost irrelevant. I think that's generally a good wake-up call to designers, and I've tried to reinvent what I do for myself around a deeper definition of design, one that tries to encompass engineering. This is a pretty typical story for a Silicon Valley tech company design decision:

A designer, Jamie Divine, had picked out a blue that everyone on his team liked. But a product manager tested a different color with users and found they were more likely to click on the toolbar if it was painted a greener shade. [...] Mr. Divine's team resisted the greener hue, so Ms. Mayer split the difference by choosing a shade halfway between those of the two camps. Her decision was diplomatic, but it also amounted to relying on her gut rather than research. Since then, she said, she has asked her team to test the 41 gradations between the competing blues to see which ones consumers might prefer.

So far, the usual. But the idea put forward by the rest of the article, however, that Marissa Mayer has a "keen sense of style and design" is false, and ridiculous. With a few exceptions, business executives almost never have a way of talking about design; it takes a lot of experience and training to do that. So, they will seize on a small detail or color preference as a way of shaping a design, or they rely on research on one small aspect. At an engineering-driven company, these kinds of details will often be the extent of the entire design discussion, with the personal pet-peeves of the executives and the vagaries of how alternatives are tested producing incoherent design direction. That dynamic is very old (probably dates back to cave-paintings), but there's absolutely no way it represents Marissa Mayer doing a good job for Google, or helping Google products to succeed.

If Google took design as seriously as they do engineering, they would not focus on details, but remaking interaction design and visual sensibility. Marissa Mayer wouldn't make a comment about grey text, she would be wondering how Google could give users better interfaces to information than an empty box. Google should swing for the fences again with new thinking, not imagine that because they are successful they do everything right.

make news like the cable tv business, please

2009-02-26T19:52:00Z

It seems like there is a fairly straightforward deal possible to save the business of putting out newspapers (the news is fine, doesn't need to change!). Make it a much cheaper version of the cable business, where subscribers buy into a much-enhanced version of something they get a basic version of for free. Major ISPs like AT&T, Comcast, etc. could create a open news consortium that users could buy into by adding $2 to their monthly bill (this has nothing to do with network neutrality, by the way, just creating the same mechanism that supports free pop music radio).

Assuming that only 5% of broadband customers of the top 5 U.S. ISPs agree to that, that's $120 million each year. If even just newspapers banded together for this, ISPs would have a strong business incentive to offer the surcharge to their subscribers. Any content provider with a certain level of traffic could offer their content only to subscribers of the consortium, splitting that dollar 50/50. This money would be paid out to content providers on a strict traffic basis. Providing content this way would be much more efficient than via paper, and the writers, editors, and photographers would be responsible to their audiences first, as it should be (with advertising revenue on top of that). And they could continue to provide news summaries and headlines to news aggregators like Google News.

It wouldn't be the 40% margin of years-ago, but it would be a going concern. All that it would require would be placing the needs of the business as a whole above the fantasy that there is something basically wrong with journalism, Web sites, any particular newspaper, or an attachment to paper as media. And also the willingness to take action instead of letting things slide further towards... nothing.

snark: too big to fail?

2009-02-22T19:24:49Z

Walter Kirn's review of David Denby's book Snark is pretty fun reading:

He wants to correct and restrain, using scholarship and logic, perhaps the keenest, most reflexive, prehistoric and anarchic of simple human pleasures, short of eating or achieving orgasm. The act of laughter, this would be. Or, for Denby, the act of low, illicit laughter -- laughter enjoyed for the wrong reasons and provoked by the wrong lines. Whether laughter for the right reasons is even possible, given humor's subversive, corrosive history, is a difficult philosophical question, of course, but Denby feels that it is. This follows from his belief that the impulses to giggle, grin and cackle (and the various means for stimulating these impulses) can be, and ought to be, consciously aligned with civic virtues and literary standards, lest our society laugh for no just cause, at jokes that aren't witty enough to laugh at and that may even be plain stupid and malicious.

Yet, I think there is something missing in the book (which I haven't read, of course) and the review. Snark is a good shorthand for humor that works on a sophisticated level, as sort of an end in itself. I think snarkiness is the Credit Default Swap or Mortgage-Backed Security of the cultural world-- an instrument so complex and disconnected from anything valuable that it creates dangerously inflated markets for worthless exchange. Like a book Snark and the snarky review of the book Snark (and definitely a blog post about both!). So, from now on (much like the painfully earnest site SnarkMarket), I'm only going to use the term snark ironically.

some feedback for a K-8 school's technology plan

2009-02-19T07:45:32Z

A schooI's technology plan I read is mostly about computers for students and teachers, other equipment, and goals for integrating equipment use into the classrooms and professional development plans, without describing what students would do with the computers. Some feedback I gave:

I like the quote from John See's on the cover of the plan, start with the applications, not the technology:

Typically, technology committees go before school boards asking for a computer lab, or computers for classrooms. The first question board members will ask is, "Why do you need them?" Why not answer that question in the plan? It may be better to go to a school board saying, "This is what we want our students to be able to do"

I would imagine the applications kids use to be various combination of:

reading, researching, exploring
writing, editing, taking pictures, collecting research, making docs
sharing, messaging, commenting, evaluating
planning, calendaring, collaborating (via the above applications)

All of these are best done in a networked context, where a kid or teacher has their own space but can easily get or move information around. Basically, a Web site, or sites. And none of these are particularly well served by specialized software like Word or PowerPoint (in fact, the software is usually a distraction to the task I'd imagine). So, what's really needed for the applications is:

A campus-wide wireless network, 54Mbps or so.
An internal and external Web server.
Any computer that can run Mozilla Firefox 3
A bunch of new Web sites that enable the above applications (could be built on top of an application server like Drupal)
A set of policies for learning contexts that govern what data and work can go from internal network out to the public Web, and vice versa

This is different in that:

A lot of fancy new computers are not needed (just more older 'commodity' computers that can run a good Web browser well)
No further work on the internal ethernet network is needed (better to replace ethernet with more wifi hubs or extenders)
The real time, resources, and energy should be spent on the Web-based applications that will be directly part of the curriculum, instead of equipment

And there are great alternatives to expensive computers:

Standardize on free, bulletproof software instead of a single hardware platform: a robust Web browser that can use Web-based applications like Firefox 3.
A 1 year-old refurbished Dell laptop with Ubuntu costs $400, new MacBook costs $949, but they have identical performance running Firefox.
Computers can be more easily maintained by standardizing on one simple configuration for everyone. If a computer is running badly, it can be wiped and re-imaged (instead of troubleshooting software installations).
Software can be limited to what's free and available on all operating systems (Mac, Win, Linux): a Web browser like Firefox 3, a text editor, and media management applications like Songbird & Picasa.

in praise of assholes

2009-02-19T04:19:21Z

Recently I've found myself having very unpleasant conversations with people about work. Often, both of us are in a bad situation, and there is no easy way to make the project better, only 'least bad' answers. It's a negative situation. Dealing with the disappointment and upset in this situation often leads people to act badly, myself included. I push too hard, questioning people's conclusions too much, and generally am a nuisance. One might even say an "asshole." I am not mean, but probably irritating.

When I went to school and worked my first few jobs, demanding bosses were the norm. Animated discussions, arguments, and emotion were part of caring about the work. I had a boss who extracted good things out of bad situations on a regular basis. I hated him at first, but eventually grew to admire him and respect him. Bewilderingly, I no longer have passionate arguments with people about work. Instead, discussions and meetings are meant to reinforce decisions already made. Negativity is to be avoided, and any criticism is almost offensive (even if just in tone).

I certainly can understand the desire to make work fun, lighthearted and focus on the positive at work, but I think something was lost. Progress and good work doesn't come easily. In fact, it's really easy to do mediocre work when everyone is afraid of failing (and getting laid off). There are great people who can avoid that and still be full of sweetness and light, but those people are few and far between, and the emotions involved are not going to last as long as good work. Abusive, mean behavior is wrong no matter what. But I find myself wishing for a couple of assholes at work, people who would shake things up, force the issues, and push past the usual solutions. But for now, I'll just try to stay positive :)

singletasking

2009-02-12T00:57:51Z

I liked this post from Caterina Fake so much that I made myself a small leaflet version with these and other singletasking axioms to post by my desk (download PDF). I am kicking the interrupt-driven lifestyle!

saving journalism cluster-frak

2009-02-10T21:20:08Z

Newspapers and magazines are losing money very quickly. News content remains very popular, but most of it is available for free. News aggregator sites serve newspaper content to large audiences for free, and Craigslist has killed the major source of ad revenue that newspapers need, and people are used to getting magazine content for free online. Newsweek is exiting the mass-market, and it appears that many large newspapers will as well. Many people agree that journalism is an important part of a functioning political culture and society, but no alternative to the present course has emerged.

Henry Blodget points out that even if it stopped printing on paper, the New York Times could not pay for it's news room full of reporters and writers. Walter Issacson proposes a "pay by the slice" model, like iTunes for music. Amazon introduces the Kindle 2 as an analogous device for newspapers. Chris Anderson says "free" is the only business model that works, and it's okay if that doesn't support a lot of journalism. Andrew Keen says "mass-amateurization" is destroying our culture and must be fought. Michael Kinsley says that it's okay if we have a much smaller number of newspapers. Dave Winer says kill all the elitist journalists, we can make a new journalism out of bloggers and participatory media.

The conclusion I come to is that the sea change that has happened is the end of 'passive' culture, not journalism. It should be possible to re-invent journalism as a richer, participatory medium, with experts and professionals directly collaborating with (and directly responsible to) the enthusiasm and amazing creations of bloggers, makers, photographers, people of all kinds. The more sharing and creating of things that's happening online, the more engagement and value there will be there, and it will become something that can pay the bills for those who do it well.

Large media companies and newspapers could protect their futures best stopping the attempt to hold on to existing ad rates and formats, and instead moving their best advertisers quickly into performance-based online properties and formats. No one yet knows what will work, but they can do themselves a huge favor by trying a dozen experiments instead of moaning. And the bloggers & geeks can help invent new things, instead of imagining that this is a revolution where it will be a victory if a valuable part of society goes away.

protest and gaza

2009-01-25T08:00:20Z

The feet of one of three Palestinian siblings from the Al-samoni family, killed by an Israeli tank shell, are seen in the mortuary of Al-Shifa hospital, on January 5, 2009 in Gaza City. Seven members from the Al-samoni family were killed including the mother, three children and a baby, when an Israeli shell struck their house south of Gaza city. (Abid Katib/Getty Images, boston.com)

Marc Ambinder writes

Many a friend has asked me what I think of the Israeli invasion. I have some private thoughts on the subject, but they're not particularly interesting. I've studied enough, prayed enough, spent enough time in Israel to get the hang of why the conflict appears so tragic and intractable...

and tries to recruit some thoughts from Jeff Goldberg, an Israeli reporter. Jeff writes that

...nothing works for very long in the Middle East. Gaza is where dreams of reconciliation go to die. Gaza is where the dream of Palestinian statehood goes to die; Gaza is where the Zionist dream might yet die. [...] My paralysis isn't an analytical paralysis. It's the paralysis that comes from thinking that maybe there's no way out. Not out of Gaza, out of the whole thing."

Here, journalists on the ground in Gaza talk to a Current Vanguard reporter:

What has happened in Gaza is/was, as far as I can tell, cruel, pointless, and another example of how little we know as human beings about how not to totally fuck everything up. It is nauseating to watch the father lamenting the death of his daughter, alive just two hours before. I feel that I have to respond, but in the face of this overwhelming suffering and with such overwhelming problems, is that the right way to be thinking? The above smart people who have actually been there, etc., sound like they are stating the truth of the situation to me; for my own actions, that is where I would leave it (with thanks that U.S. role in the region will probably be a lot different with Obama). But I have been prompted by other people to do more to protest the obscene amounts of innocent death in Gaza, through small gestures like groups on Facebook or marching in protests with signs. It feels wrong and weirdly beside the point to me, but a lot of people feel strongly about it, so here goes.

As best I can tell, whatever solutions can be found to change, even in small ways, the situation will come not from protests or activism. Everyone who is at all directly connected to the conflict is desperately aware what everyone thinks, and has their own idea of justice worked out. Protests have become background noise, even at a large scale; a hundred million Europeans protesting couldn't stop Bush from invading Iraq. Politics has changed a great deal, and needs new tactics. (I have some ideas about that, but most people seem to be protesters, cynics, or oblivious, so I have some issues with finding someone who gives a shit.) Protest may be better than doing nothing, but that is about all it is. Like Marc, I am aware of how beside the point my own judgments and needs for action are, yet keep trying to create some activity, find something to do to push away the horror as it unfolds.

at last

2009-01-21T22:00:59Z

what comes after search?

2009-01-20T08:28:56Z

Reading about the epic battle against Google that Microsoft, TimeWarner, and Yahoo continue to lose, I have to wonder if it's really such a world-beating thing to own search. Right now it is, since the search box is the interface to much of the Internet for people. But isn't that a sad, pinched state of affairs? There's a lot more valuable information in Twitter and Facebook than in Google. Won't something that lets me tap into that be much more valuable, and soon?

the attention economy: huh?

2009-01-13T06:54:35Z

I follow a blog called "The Online Photographer" by Mike Johnston, an experienced photographer and writer who was the editor of Photo Techniques magazine for about ten years. It's a good blog, and he knows what he's talking about. I was sort of surprised to read this in his post (he's referring to a discussion of 'bokeh' at another site, photo.net):

It's a bit disorienting for me now when I post at other sites; despite the fact that my name was referenced several times in the thread before I commented, no one paid the least bit of attention to anything I said. Not that it was so important...it's just that, around here, I tend to get listened to. A lovely luxury, and thanks for that.

In this case, he commented on a discussion about a photography term that he invented. Of course it's nothing new; people loud enough to get the attention of a mob, and especially on the internet, are probably full of shit. Actual knowledge and valuable work comes from the quiet folks, etc. But still good to be reminded of that...!

my camera, the zone system, and twenty years

2009-01-12T09:26:34Z

I've been taking pictures for a long time (far too long to have learned as little as I have!). When I was a teenager, I developed and printed black and white film, at the Salt Lake Art Center (now I can admit that I should have been paying for the darkroom time, but I simply walked in and used the equipment and chemicals, for months). In those days, they taught the Zone System. When I managed to get it right, it worked well, but it was tricky. Years later, I have a digital camera that produces decent exposure ranges straight out of the box, no thought by me required. Nonetheless, I thought I would look around for information on how to more directly control exposure.

The Zone System is (and here, people who know more will cringe) a way to plan how a picture's lights and darks will be captured and printed. Spot meters that older cameras had (like my Minolta SRT-303) just measured the exposure off the part of the image in the middle of the frame, so if there was another part of the picture that was much darker or lighter, that part would be way too light or dark.

For example, if I took a picture of a person in a car in bright sunlight, the spot meter would tell me to expose for the sun reflected off the car, say 1/500 of a second at f/16 (with ISO 400 film). If I did that, everything but the hood of the car, including the person, would be completely black. Following the Zone System, I decide that the highlights on the car would be the brightest thing in the picture, and the person would be in the middle range. So, I increased the exposure 3 stops, to f/5.6, and the person's face becomes visible (while the highlights on the car become pure white). It took a lot of practice to make these decisions, however.

Now, meters in cameras are using the entire image to decide the exposure, basically building in the Zone System into the camera's exposure calculation (this is the "evaluative" mode my camera has). If you are interested in more direct exposure control, then, it seems that you are left with:

using the camera's meter in spot mode, manual exposure, and using the traditional zone system
using the camera's meter in evaluative mode, automatic exposure, and let the camera own the exposure

Neither one of these is particularly satisfying to me; the evaluative mode is better than I am at exposure calculation even if I set the exposure manually (since I still use the meter), and I really have no desire to go back to spot metering. I found a third option:

Underexpose everything, shoot in RAW mode, and adjust the values of the picture after it's copied to the computer

This has produced the most satisfying results so far; without any special software or too much worry about exposure, I can still use my Zone System knowledge to peg parts of a picture to particular values of light and dark I choose.

I underexpose (I set the EV compensation to minus 1/2 stop) because digital cameras don't capture as much information about the bright areas of the picture as film does, so I want to be sure I get whatever detail is in the highlights
I use RAW mode because it lets me change the exposure of the picture after it's shot without losing any information
I adjust the exposure on the computer because I still want to decide what the most important part of the picture is (and what should get the 'middle range' in the photo). Almost all the time, this is as simple as adjusting is the white, grey, and black points (something every single program for handling digital photos can do; Picasa, iPhoto, etc.).

Sometimes I will set two or three other points to use a transfer curve in the RAW conversion, but not so much anymore. The photo above was taken in this way (I show it in black and white because I think it has a great middle dynamic range that illustrates the idea -- the pretentiousness of black and white is just a bonus). Next, I am going to have to figure out how to get a handle on the weird white balance problems I have...

"hybrid" economy: socialism, web tools, and the end of bullshit commerce

2009-01-10T11:00:43Z

Lawrence Lessig on the Colbert Report

I loved watching Lawrence Lessig on the Colbert Report, he was so articulate and impatient for things to make sense after so much stupidity. I liked his idea of the hybrid economy -- "read/write culture" and commercial culture in a symbiotic relationship, and wanted more. I have yet to receive the book, so I don't know where he goes with the "hybrid economy" idea. But, I want the tools of this read/write culture to be connected not just to commercial culture, but to the rest of the economy, so they don't go away in the tsunami that's sweeping away a great deal of the economy right now (man, I can only pray for the New York Times).

I don't think it's overstating the case to say that anything valuable to someone can be better if people have better information and more relationships to the people who make their living from it. If I work for, buy from, live around, or share a town with the workers in any industry, small company, or even just a gas station, that situation is better with more transparency, communication, and relationships between the people than what we have today.

A pharmaceutical company should be giving unmoderated web tools to patients who are dealing with a disease, so they can connect to each other and negotiate the completely dysfunctional health care system we have. A car company should design and build cars with the public, through open beta car designs, candid information about what the strengths and weaknesses of the company are, and total access to decisions executives make. Even the gas station should be posting the information it gets from its contacts about why the price is what it is, and what to expect next week. Government should be making the millions of decisions and policies it produces an ongoing, distributed dialogue with citizens, where the strong local opinions and experiences get rolled-up into the scorecards for massive programs.

Free-flow of information makes the people who have to buy things more powerful, and it makes those who want to sell something make better things. And it makes it much easier to see exploitation, avarice, and abuse. Ultimately, that's good stuff for everyone who shares a city, or a town, or a street. The wonderful sites that form the bases for these tools (Wordpress, Wikipedia, Facebook, Digg, Youtube, etc.) should be right in the middle of these real human needs, not just off to the side for entertainment (also, I don't know how they'll survive otherwise).

And the nice thing about all of this is that it's actually less money than what companies flush down the toilet today, in a process known as marketing. There's just not that much point in a game of using media to associate good feelings with a product when everyone can find out much more about it directly from real people. "Brand" is just not as valuable either; the products are it. Enough with the promises that a company is part of your family or people "love" a brand. It's corrosive and wrong to try to hijack our actual, human experiences and feelings to sell products. We don't have to do that anymore.

We do all have to make a living, pay for food, hopefully do work that's valuable to someone. There's nothing inherently sleazy in that, it's just practicality. These social web tools can make work, products, and business more honest and open. More socialist, in fact, and I am very comfortable with that.

leaving flickr

2008-12-17T17:45:08Z

Photowalking, originally uploaded by George.

George Oates (who I know some and whose work I like tremendously) was laid off in the latest round at Yahoo. Also, many extremely talented people from Brickhouse (Samantha Tripodi, Chis Martin, Jeannie Yang, Ben Ward, Ken Thornhill, Premshree Pillai, and Kevin Thornback, none of whom I knew well, I just saw and was awed by their work). Like Tom Coates ("Still reeling from the last few weeks"), I am still trying to get my head around the entire thing. I still don't understand how these people got cut. I can't imagine Flickr without a key part of its DNA. I have respect for Kakul, Heather, and the rest of the crew, but I think this shows that the decision making there has gone off the rails. I started using Flickr in 2005, and have loved it and the culture that has developed around it. But, I am going to move my photos off Flickr and find a new home for them, with many regrets.

In the larger context of Yahoo, it is doing yet more damage to itself. It simply won't survive as a commodity site or development platform alone. Mail, a front page, and news (all indications are that every other part of the company is flat or declining) are not enough to sustain the company at its current size, and focusing on advertising innovation while the entire structure goes through a recession seems wildly off-base. What they should do is do what other smart people did during the last bust: invent something really good. That's when people started Flickr, and Google, and Blogger, and on and on. Despite many assets and a lot of value, Yahoo's management would do better to improve the company's prospects by laying themselves off.

update: a couple of people have said that they don't understand the 'protest' I'm making; I should have been clearer. It's not a protest at all, Flickr is still a great thing and the people there are good, etc. I'm quitting it just because I had more of an emotional relationship with it than a practical one, and that's been changed. It's a purely personal thing, left over from the time when sites like Flickr were labors of love, built mostly from enthusiasm. That's rare these days, but that's why I do what I do, through booms and busts.

serving JSON to jquery: a head-slap

2008-12-15T05:53:06Z

I hope in some way that this post will help those who made the same stupid mistake I did and wasted way too much time figuring out something that in retrospect is obvious. Sigh. Hypothetically speaking, if you wanted to make a servejson.php page to serve data in JSON format, it might look something like this:

 "Ben");
$contact_encoded = json_encode($contact);
header('Content-type: text/html');
echo $contact_encoded;
?>

and your showjson.html page to get and show the data might look like this:

So far so good, the code follows the examples you've looked at to the letter. But it doesn't work of course. And it won't. And it will fail silently, giving you no idea what's going on. You'll do some searches and read some forums, and will find nothing on it. Eventually, while reading another person's frustrated blog entry, it will come to you. And then you will feel dumb for having copied and pasted so much example code without thinking about what you're doing, and the hot shame of obviousness will cause your hand to hit your head before you're even aware what's happening. Your consciousness will swim in the ocean of nothingness and flux. And you will be that much closer to the next life, and you will want the time back. So please, think. There's a callback key on the querystring in the jquery code up there. Jquery is going to generate a function name, send it, and want to get it back, etc., etc. DUH. So make your php code:

echo $_GET['callback'] . "(" . $contact_encoded . ")";

...and go back to obsessing about much more important things. Bless us all.

obama's identity

2008-12-09T09:07:45Z

This is a good time to think about the identity of Barack Obama. Right now he's a blank screen that many different groups are projecting their wishes on to. He's not from any of their constituencies really, though it's clear he has his proclivities. When he came to Chicago, he was more "mutt" than anything else, searching for a sense of belonging. He pretty much made a choice to join black culture, to inhabit the identity and the role, just like he chose to go to an elite school and become a community organizer. But he did all that while staying removed from the visceral nature of some of the old crappy American fights (race, class, culture), a remove that let him see those fights more clearly (as his speech on race in Philadelphia attests).
So he's pretty much free of attachment to the old clashes, though he understands them. He can choose more intellectually what he wants to do. That could be dangerous (Robert McNamara was surely one of the smartest, most methodical people in the U.S. while he dragged the country into Viet Nam), but right now it is such a relief to have a president that doesn't seem trapped in any one corner.

mumbai

2008-11-29T01:02:22Z

Look at the pictures on Boston.com

New York Times Election Results: County by County

2008-11-06T00:24:04Z

Beautiful maps from the New York Times (Times Digital) team!

A great, great day.

2008-11-05T18:18:08Z

...and a better world for my daughter.

sarah is four!

2008-10-23T23:14:08Z

Sarah is becoming a big girl at a torrid rate! When I think of how fast it's all happened I get into a weird emotional state of bliss and regret that I can't hold on more tightly to her days as a baby. We're so, so lucky to have her. I never knew what a wonderful person was waiting for us.

She is four.

The myth of the community reinvestment act as the cause of the current financial crisis

2008-10-12T23:52:33Z

Recently many commentators have blamed the current financial crisis on a law enacted under the Carter and Clinton administrations, the Community Reinvestment Act. They say that the law forced banks to issue loans to lower income an minority applicants that could not afford them. Usually there is some grain of truth to these things, but far as I can tell, not this time:

The Community Reinvestment Act applies only to depository banks.
67% of 'subprime' mortgages were issued by non-depository investment banks or other firms that were not regulated by the CRA, not depository banks, which were.
Of those issued by depository banks, only 54% were 'subprime.' That means that, according to a CATO study, only 17.4% of 'subprime' loans, or 2.3% of all mortgages issued in the U.S.
Collateralized Debt securities (the drop in value of which caused the collapse of Merrill Lynch, Lehman Brothers, Bear Stearns, AIG, etc. and the present uncertainty about credit) were created by combining regular and 'subprime' mortgages. These securities were issued only by investment banks. Doing the math, that means that about 89% of all the securities issued had no CRA-covered loans in them.

The CRA was a very minor player in the financial crisis, issuing a small percentage of all loans. It did not apply to any of the banks that issued about 89% of the risky mortgages in the U.S. The CRA was irrelevant to the investment banks and other firms that issued risky loans.

"There has been a tendency to conflate the current problems in the subprime market with CRA-motivated lending, or with lending to low-income families in general. I believe it is very important to make a distinction between the two. Most of the loans made by depository institutions examined under the CRA have not been higher-priced loans,16 and studies have shown that the CRA has increased the volume of responsible lending to low- and moderate-income households." -- Janet L. Yellen* President and CEO, Federal Reserve Bank of San Francisco, March 31, 2008.

The markets that collapsed had almost no regulation. In fact, the Commodity Futures Modernization Act of 2000 explicitly barred their regulation; the law was written by Republicans and signed by Clinton -- there is plenty of blame to go around. One of the very few completely unregulated markets in the world stopped "functioning properly" (in Bush's words). I can't see why anyone would look further for the cause than that uncontroversial statement.

UPDATE: My brother Michael Clemens, a Harvard economics Ph.D, says no major mistakes! Whew. He also says to keep in mind that the "very simple fact is that no one completely understands the roots of the financial crisis, because it is a complex, emergent, chaotic phenomenon" (hope it was okay to quote you Michael). That is for sure.

*Some have said that as a Clinton appointee, Yellen is biased. She was appointed after his first choices were rejected by the Republican congress; Yellen was cited as a nominee the Senate Finance Committee could support, and they ratified her appointment unanimously.

getting a feel for what's out there in the world

2008-10-09T05:32:37Z

There are lots of bits and pieces of information around that seem to tell you about what's going on in the world, but even the best of these are, finally, someone's subjective judgement call, what they think is happening. They may be professionals, etc., but I want to get a feel for myself of what's happening, not just accept the opinions of others-- for example, there is usually a pretty serious gap between the politics of Berkeley, California and the rest of the U.S. There's no easy way to get a sense of how big the gulf is, no easy road to figure out how anyone could conceivably want to have someone like Sarah Palin as Vice President.

Of course, I can immerse myself in firehoses of data of all kinds: populate my RSS reader with Republicans, watch 500 channels, visit 10,000 folksy web sites. But I really just want broad themes, changes, and trends. I want an engine that analyzes and summarizes lots of raw data into a few groups of things that are similar.

And no, this isn't some smirky internet-age bullshit! Based on Ben Fry's recommendation in his book, I got a copy of Exploratory Data Analysis via inter-library loan. It was published in 1977, before most people knew what a PC was, and waaay before anything like the Internet. In the very first chapter, John Tukey starts with talking about making simple "stem and leaf" graphs with paper and pencil of the megawatts generated by hydroelectric dams in the U.S. Why? He wants to show you how to "write down a bunch of numbers in such a way as to give a general feel of 'what they are like.'" What a great thing! I don't care too much about the numbers, I just want to know if there's any rough pattern there, and there is: most of these dams either generate about 30 kilowatts, or around 1,100, two groups that tell me where to start if I want to know more about that (I don't).

In information retrieval land, this is known as clustering (one of my hobby horses that I usually talk about with anyone I know for longer than 30 minutes). Instead of a laundry list of thousands of things, I get a set of groups of the words that are used together often. I can get a feel for what's happening in the data without sorting through it myself.

For example: I vaguely know who Kenzaburō Ōe is, but I don't really know what his books are about. I just want a feel for that, with some idea of what the themes of his books were. I can either read a page of Google search results, or I could see derived clusters for those documents. To me, seeing the groups is a much better interface to start exploring if I want to know about him (and I do).

The motivation to get a sense of what's out there in the world is an old one-- it goes back farther than 1977, even. Now that more and more of the stuff of everyday life (cats, politics, hobbies, cancer, errands, religion, etc.) is in the form of data online, I can imagine doing that. I should heavily note that clustering techniques applied to social data still produce very uneven results, and a lot more work is needed. But it's becoming possible to think of a better interface to information than a search box.

twittercloud, a new ersatz startup!

2008-09-26T06:09:53Z

Ok, I hate tag clouds. With a passion. Never again. But, since I was sorta happy with the term frequency normalization thing I was working on, I figured what the heck (the road to heck is paved with half-assed text analysis). And another thing, aren't there like, 2,810,000 (and one) twitter clouds out there?

But here it is anyway, twittercloud. It's slightly different than other clouds, in that you can generate a cloud from a query term (not just whatever is hot or an individual's tweets), and you can see what tweets are generating the term right in page. It auto-refereshes with ajax to give you that clean feeling. Also, I happen to think frequency normalization gives a nice weighting that makes more interesting words pop up. Anyway, it was fun!

UPDATE: Now includes Flickr photos (a tag search based on the query term and cloud word) and much better twitterbot detection.

man I wish I could go to this

2008-09-08T04:34:51Z

It's worth it just for the illustration, a neat summary of collective intelligence approaches.

The course has 2 phases: Network Structure and Network Dynamics. The first phase focuses on networks as static entities. It is based on the Graph Theory and concerned with the structure. The second phase focuses on the processes taking place in the networks. It is concerned with time, interaction, and multiple characteristics of the network elements.

See the full syllabus.

mozilla ubiquity could be the ultimate social media tool

2008-08-29T06:40:12Z

One of the most exciting things (to me) about Mozilla's Ubiquity is the ability to annotate pages directly in the browser. Really. There is not much there in the 0.1 release to make the feature useful or justify my enthusiasm, but the fact that it saves all my annotations in a browser storage area (at least temporarily) and includes sharing functions already makes it dangerously close to a new variant of social media.

Assuming that Ubiquity incorporates features to: a) add annotation elements to a page, b) stores annotations locally, c) lets me organize & collect annotated pages, and d) above all provides an easy way to share annotations socially, it seems like it could become my new bicycle! It would be:

a standard XHTML container that works everywhere, on/offline
a place to stash pages rather than remembering links
a notebook with bits and pieces of other pages
host an annotated page instead of drawing on a screenshot
add video/images/media to a page, make snarky comments on it, and send it around
make 2 or 3 pages into one page
make a page into a wiki, an exquisite corpse, etc. etc.

And some cumbersome scenarios become really easy:

I can email a web page that has a message, a highlighted portion, and a link to the original page, as an attachment that needs no internet connection.
I can email a map that has text pointing to a place on the map
I can choose and send 12 books for preschoolers from amazon (with titles, authors, thumbnail, price, and link) via an email
I can turn the NL east division part of the MLB standings into a widget that appears on my blog, with the erratic Mets highlighted and have it update as they surge, then collapse again.

(I know there are ways to do all these things now via other services and tools. But most of them require a good chunk of time, an account on something, etc. The thing that will make web pages into a medium for sharing stuff (instead of just a url) is making it easy, three-four clicks, simple, etc.)

I love all the other features of Ubiquity (I'm an old Quicksilver user). And there are plenty of no so great aspects of this. But I think this part of Ubiquity has the most potential to make sharing things on the web a whole order of magnitude easier, in a way that no bookmarklet, plug-in, greasemonkey script, or single-purpose site could.

sticker spaces

2008-08-14T03:07:48Z

fireeagle figured out!

2008-08-13T06:17:53Z

UPDATE Belay that. FireEagle not figured out :(

Damn, that was hard (for me). I finally have my tiny scratch of a twitter & location Web app ("twiphlo") updating not only twitter's profile location, but also fire eagle. It's still hacky, but it works. If you are one of my 12 4 regular users and that interests you, send me mail and I will enable it. It really is the Charlie Brown Christmas tree of fire eagle apps tho.

the problem with having something to protect

2008-08-11T16:00:34Z

Deep in the depths of the dot-com bust, when I was lucky enough to have work of any kind (in my case it was commuting 200 miles each day contracting for Cisco), the people I worked with were fatalistic about the future of the work we did. Many people thought the money had been drained out of the exploratory culture of the boom, and that we were in a very conservative time.

The truth was the opposite in fact; much of the boom was spent doing variations on e-commerce ventures and content sites, with the idea that each vertical or market segment would need it's own one of those. Counter also to the conventional wisdom, these sites were not created because their entrepreneurs thought that business logic had been suspended, but instead because the tiny community of venture capitalists were funding only specific things. The 'bust' came not out of the sites' lack of income (it was no secret that very few ever had enough business to survive), but in the withdrawal of venture funds.

After the bust, many people left the business for school, or other jobs. A few people (who couldn't do anything else) kept creating new things however, and many of them were more creative than anything that came before (and set the stage for the current boom). Google, Blogger, Napster, and Friendster (a non-semantic search engine, easy blogging tool, media sharing tool, and social network, the hallmarks of today's Web) were all created in 1999 or 2000 in the shade of obscurity. The examplars of the successful social media Web were born in 2001 and 2002, when little venture capital flowed: Wikipedia, Flickr, del.icio.us, and Facebook.

In retrospect, it's easy to see that it's more possible (though harder of course) to create something new when no one is watching and no one cares, out of shear love of an idea. But it's not something that many seem to remember these days. Instead, it feels like 1999 all over again, with companies launching sites with the same functionality and ideas, venture-funded start-ups doing variations on social media ventures and aggregation sites, with the idea that each vertical or market segment will need it's own one of them. In fact, I can't think of a single company that's launched since 2002 that isn't a derivative of one of the bust's labors of love.

The complete lack of creativity and new ideas is typical today. I guess we all have something to protect: the market funding, the high-paying jobs, the advertising revenue. The more to protect, the more conservative it gets. I hope that someone, or some company, somewhere, will think back to the first boom and remember what happened. The business cycle will happen, the money will evaporate, and only those that have made something truly new will survive that.

excessive competition, china, and social media glut

2008-08-08T17:57:26Z

China is going through huge growth, about 10% a year. While there remains a huge part of the country to lift out of poverty, one would think that this amount of growth would result in plenty of opportunity for everyone to make a good deal of money with less competitive pressure than in 'mature' markets. But apparently in some areas, the opposite is true; since the plants are partly owned by the government (and the managers not entirely responsible for keeping costs below income), they will sell products below costs in order to compete with other plants. This is, according to the article I read ("Ownership Distortion, Low Level Technology, and Excessive Competition," by Jie Ma and Weiying Zhang), the economic state of "over-competition," where too many competitors end up compromising the market and their own viability.

This rings a bell, because in a venture-capital fueled, social-media start-up frenzy much the same condition exists. 35 life streaming apps. 4,324 photo/video sharing sites. 321 twitter clones. Countless business-vertical mashups (102 travel business mashups alone). Why will any of these gain enough traction to garner general use (and "change the world" as their entrepreneurs like to say) if the market is in this crowded state and the owners are not risking their own money? Is this lottery-like state really an ok but brutal winnowing process, or more like the airline industry's race to the bottom?

For example, two competing commenting platforms in the same market causes some upset as one tries to market to the others' users. When so many blogs are being added all the time and the phenomenon is going mainstream, there should be plenty of room for at least two products. Instead, there are an overwhelming number of choices (who has time to sort through 14 options?). Decision theory says that "more choices may lead to a poorer decision or a failure to make a decision at all", losing situation for all these creative, hardworking start-ups.

So what's the alternative? I think the collections of start-ups in overlapping spaces should, as quickly as feasible, abandon the idea that they can create the be-all, end-all (e.g.) social shopping user base and content silo that will crush competing sites. Instead, they can open-source the basic containers and objects (much as Open Social and OAuth have done in one discreet area). That way, the personal data is by definition portable and owned by no company, and start-ups that aren't really, functionally different will have a shorter life (and I will never have to upload my profile picture again, ugh), and the viable sites will have a clearer shot at being viable businesses. What is the likelihood of this happening? Probably not so great, but these are the same geeks who advocate the Linux model for operating systems (where an app can be easily recompiled in the different distributions, and each distribution can use parts of others). There's really no reason other than a kind of protectionism not to extend this to Web applications.

twitter for place: /

2008-08-08T09:14:22Z

The great geek solution for adding a generic tag in twitter is "#word," a "hashtag", but some are using it to indicate place as well. Seems like a lost opportunity to integrate place into twitter in a first class way! The only reference I can find to an alternative is someone proposing "L:" -- horsey. Since directory structures are the geekiest kind of places, I propose the humble slash, as in "/90210/" or "/3rd & Bryant/". And no, updating the profile location is not a great substitute :)

twiphlo: another fake startup!

2008-07-31T19:12:25Z

"Twiphlo" (yeah, twitter iphone location... can't have a fake startup without a cutesy name) is my tiny contribution to making one particular activity easier: combining twitter with location updating, as an iPhone web app (yeah, I know web apps are dead on iPhone). The idea is that you can replicate some of the functions of brightkite and myloki without dealing with multiple services by just updating Twitter's location field in your profile. And, when having a history of your twitters with location information is interesting, you can see your past locations and tweets on a map. It also generates a geoRSS feed and a KML feed (Google Earth).

Of course, I only made this for myself, but it's there to try and to use if anyone finds it useful:

http://practicalist.com/twiphlo

It's functional in a web browser, but it works best in Safari or on an iPhone.

Disclaimers & Details: This is a test app. It gathers personal information. Use at your own risk. You may lose data, it may not work, Twitter may be down, etc. This application saves your Twitter password to a cookie on your local device only, mildly encrypted. Your username, locations, and tweets are stored on my secure database, to draw the map. If you are uncomfortable with this, bookmark this link and the app will not store any information locally or remotely. I will never have access to your Twitter account, and all the information stored is public already (unless you've set your tweets to private by default).

semantics, piles, and clusters

2008-07-23T18:02:02Z

As I approach the singularity (doing the whole spectral clustering thing on my own, rather than relying on genius kids for the heavy lifting), this caught my eye as a great rationale for doing the semantics-free work:

An interesting property of corpus-based theories of cognition (such as Latent Semantic Analysis) is that they cannot be tested independently of the corpus. Imagine that we collect a corpus, run and Single Value Decomposition on it, and use the resulting space to predict human similarity judgments between certain words. Imagine that the model does not explain the data very well. Is it that the model's processes are unrealistic, or is it that the corpus is not very representative? In this situation, those two factors are confounded. A possible solution is to test the same model with different corpora and different tasks. If the models explains the judgments' variance across different situations, we have more convincing evidence of the psychological reality of the model. ("Creating Your Own LSA Space," Jose Quesada, Carnegie Mellon University 2002.)

The limitations of trying to work with the basic themes inherent in text are large; the complexity of the arbitrary patterns of using language don't seem to lend themselves to having computers learn meanings. The Semantic Web cult assumed that a perfect set of taxonomies and folksonomies could be created so that a bunch of marked-up text could 'know' what it was about, and communicate that through a retrieval system. This has largely been a failure.

Chris Anderson's article "The End of Theory" takes this to a overwrought extreme:

Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required. That's why Google can translate languages without actually "knowing" them (given equal corpus data, Google can translate Klingon into Farsi as easily as it can translate French into German). And why it can match ads to content without any knowledge or assumptions about the ads or the content.

This is true insofar as the data is good and the systems work, but those conditions are rare, and despite all that data Google and other systems that analyze behavior patterns are still not very good (and the translations are really bad). And Google still renders its results in a long scroll. Whatever the intelligence behind it, there is still a person at the other end, doing most of the work to find the right item in a long, unorganized list.

Clustering as an interface, by contrast, doesn't care about semantics, and doesn't even try for a strict ranking. Groups and rough hierarchy fit human models of organization much better than a long list (much as piles remain the usual way people organize). Loose piles don't have to be semantically understood, a set of items is easier to take in and choose from (with two dimensions -- item and group -- rather than one). And when the algorithm is based on purely on user activity analysis, a better interface for presenting results, solving the interface problem (and thus engendering and capturing more user interaction) is really solving the whole problem of giving people information in ways they can understand it and use it.

UPDATE: Taking this further, it's been seen for a long time that changes in behavior often happen when a few people that are part of a small group cause that entire group to adopt the change (like buying a kind of shoe, or phone, etc.). This is called the "cluster effect":

"The cluster effect is similar to (but not the same as) the network effect. It is similar in the sense that the price-independent preferences of both the market and its participants are based on each ones perception of the other rather than the market simply being the sum of all its participants actions as is usually the case. Thus, by being an effect greater than the sum of its causes, and as it occurs spontaneously, the cluster effect is a usually cited example of emergence."

What better way to engender cluster effects and the large amount of significant social effects they have than to show people the clusters of activity around their interests?

HOWTO: Make a Mac development environment with Eclipse, Subversion, Apache, MySQL+phpmyadmin, and PHP5

2008-06-29T18:49:24Z

At my job they use a more elaborate setup for development than I've been used to (the Very Special engineer that I am). It has some advantages over the quick 'n dirty approach I've used, but it has proved to be extremely tricky to set up. So I thought I would share the set of solutions I gathered that make a working setup, in case it helps anyone else. (In advance I should say that these aren't the über geek ways of doing these things; for that, learn to love the Linux command-line and tip forums. They are stripped down and simplified.) I do mainly design, so I don't want to spend that much time getting things perfect, just workable. I run Mac OS X Tiger and Leopard. I've been doing a lot of one-person development with Textmate, Fetch, and Terminal. If that sounds similar to your speed, this setup might be interesting.

The idea is that you run a local Web server, manage code revisions in a shared repository, and set up a virtual host for each project. This has the advantages of:

Giving you a fast, private, local environment to see and test your work
Having code that can will work on a public server with no changes
Letting you roll back to a previous version before you screwed everything up
Collaborating with others or working on more than one machine, effortlessly
Use a bloated-but-pretty-good development application with idiot-proof debugging, code hints, docs, variable name lists, and function tracking

It has a disadvantages of:

Being complex to maintain
Requiring more steps to do simple things
Being wrong for small projects

So, anyway...

Eclipse is a coding tool, a completely free Textmate or Coda replacement that has 18,423,521 features. It takes a lot of getting used-to, but in the end I do like it better than simpler tools. Projects can get complex really fast, and it's much better for that than any tools I've used. Plus it has a lot users, plugins, and developers.

1. Eclipse

Eclipse is mainly for Java development, so it needs a plugin to function well for PHP coding. The easiest way to get that is to download a free, pre-configured version of Eclipse 3.3.2 that's maintained by the PDT project.

Download the PDT "all in one" build (Mac DMG file, gzip)
Double click on the file to uncompress it, then copy the folder to Applications. Open the 'Eclipse' application inside the folder
Make a 'workspace' folder when the application starts up that is not in a system folder (like Library or Applications). I use Macintosh HD/dev/workspace in all the below steps

2. Subversion

Subversion is a version management tool that lets you go back to previous versions of files and collaborate with others really easily. Happily, Mac OS X ships with it pre-installed.

First, set up a repository (where your code will be tracked)
Open the Terminal (Applications » Utilities) and enter:
sudo mkdir /dev/svn sudo mkdir /dev/svn/repository sudo svnadmin create /dev/svn/repository sudo chown -R www:www /dev/svn/repository
Then, if you want to get the most out of Subversion, continue following the instructions in the excellent documentation, setting up a trunk and branch for each project. Or, you can just create a new repository for each project like I do, and continue to the next part right now.

The simplest way to use Subversion is an Eclipse plugin called Subclipse. To get it, open Eclipse, then:

Go to the menu Help » Software Updates » Find and Install
Select "Search for New Features to Install"
Click "New Remote Site" and enter a name for the site (e.g. "Subclipse")
Enter http://subclipse.tigris.org/update_1.2.x for the url and click 'Ok'
On the 'Update Sites to Visit' screen, make sure the checkbox next to 'Subclipse' is checked, then click 'Next'
On the Updates screen, expand the 'Subclipse' item and uncheck the 'Integrations' item
Click next and accept all the following dialog's defaults
Create a new project (File » New » Other... » SVN Folder » Checkout Projects from SVN)
Choose 'Enter a new Repository' and enter http://localhost/dev/svn/repository/, select the item created, and click 'Finish'
On the next screen, set up the project as PHP by finding the PHP folder, opening it, and choosing PHP Project. Then give it a name, and code away
Be sure to read about how to use Subversion, and Eclipse either at their sites or from within Eclipse Help.

3. Apache

Both Mac OS X 10.4 Tiger and 10.5 Leopard come with Apache already installed. Turn it on in the System Preferences » Sharing preference pane. To configure it for this setup:

Open the Terminal application (in the Applications->Utilities folder) and:
for Tiger enter sudo nano /etc/httpd/httpd.conf, then your password
for Leopard enter sudo nano /etc/apache2/httpd.conf then your password
Scroll down (cntrl-v) until you find the line:
# Virtual hosts # Include /private/etc/[apache2 or httpd]/extra/httpd-vhosts.conf and change it to:
# Virtual hosts Include /private/etc/[apache2 or httpd]/extra/httpd-vhosts.conf
Then add a 'Directory' block, entering these lines below the lines above (use what ever folder path you made for your workspace in the first line):
Order allow,deny Allow from all
Next, set up Subversion for Apache. Enter in the same document, after the above lines:
LoadModule dav_svn_module /usr/libexec/apache2/mod_dav_svn.so DAV svn SVNParentPath /dev/svn
Save the file (cntrl-o) and exit (cntrl-x).

Next, set up a virtual host so that you can type in the url for your domain and see your project almost exactly as on the final server. Open the Terminal application (in the Applications->Utilities folder) and:

for Tiger enter sudo nano /etc/httpd/extra/httpd-vhosts.conf, then your password
for Leopard enter sudo nano /etc/apache2/extra/httpd-vhosts.conf, then your password
Delete all the lines in the file that don't have a '#' at the beginning
Enter, at the end (replace www.domain.com with your server domain):
NameVirtualHost 127.0.0.1 DocumentRoot "/dev/workspace/www.domain.com" ServerName www.dev.domain.com

4. php5

On Mac OS X 10.5, you can use the existing PHP5 installation:

Open Terminal and enter sudo nano /etc/apache2/httpd.conf
Find the line #LoadModule php5_module libexec/apache2/libphp5.so
Remove the # and save the file
Restart Apache (System Preferences » Sharing).

On Mac OS X 10.4, the existing PHP install doesn't seem to work reliably. Easiest is just to use Marc Liyanage's install packages with Apache 1.3 that comes installed:

Download the installer (gzip)
Follow the fantastic instructions

5. MYSQL

On Mac OS X 10.4:

Marc Liyanage's great post has the best instructions for setup

On Mac OS X 10.5:

Download MySQL 5 (dmg)
Run Installers for MySQL, the startup item and preference pane
Make sure the preference pane (in System Preferences) indicates it's running
Open the Terminal application and enter /usr/local/mysql/bin/mysql -u root
Remove a big security hole by adding a MySQL root account password. Enter:
update mysql.user SET password=PASSWORD('yourpasswordhere');
Enter EXIT to leave MySQL

6. phpmyadmin

Phpmyadmin lets you use a Web-based interface to MySQL instead of the command-line. For most tasks, this is drastically slower but far easier than the command-line.

Download and uncompress phpmyadmin
Copy to /Library/WebServer/Documents and rename the folder "phpmyadmin"
Open Terminal and enter sudo cp /private/etc/php.ini.default php.ini
Open the php.ini file and change the lines
mysql.default_socket = and mysqli.default_socket =
to:
mysql.default_socket = /private/tmp/mysql.sock mysqli.default_socket = /private/tmp/mysql.sock
In your Web browser, go to localhost/phpmyadmin/scripts/setup.php and click 'add' in server section, then:

Leave all defaults as they are, except for...
set config user as root
set config password as the one you set above in MySQL section
Click the 'Save' button

Go to the folder Macintosh HD/Library/WebServer/phpmyadmin/config/config.inc.php
move config.inc.php up one directory level and delete the 'config' directory
Go to http://localhost/phpmyadmin to use your database

You're done! Whew. Now, as long as you don't code any absolute paths, you can develop in Eclipse, save as you go, and see your site in its final form by going to www.dev.domainname.com. When it's time to deploy, you can move your files as they are to the production server and avoid all the last-minute path-fixing (at the very least).

(Thanks hugely to Jeff Reiecke and Yixu Lin for various pieces of help with this.)

funniest thing I've read in a long time

2008-06-19T17:04:38Z

NY Times magazine on flickr: fail

2008-04-29T18:12:33Z

In an article about Flickr by Virginia Heffernan ("Sepia No More" in the New York Times Magazine), she bemoans what she sees as the dominant aesthetic on Flickr:

As art-school photographers continue to shoot on film, embrace chiaroscuro and resist prettiness, a competing style of picture has been steadily refined online: the Flickr photograph. ...the most distinctive offerings, admired by the site's members and talent scouts alike, are digital images that "pop" with the signature tulip colors of Canon digital cameras.

She then ends with disappointment in the site because "...none of it looks like Diane Arbus or Henri Cartier-Bresson, the photographer many critics still consider the greatest of all time."

To me, the site's most popular photos are as good as or better than most magazine photography. Flickr members have created large amounts of quality work from the willingness to love and critique each other's work and newly available decent equipment. This is work that would otherwise not exist if not for the site, a strong contribution to popular culture.

It's hard to believe that Ms. Heffernan would confuse popular culture with incandescent art in the offline world; why does she try to mix the two up on the Web?

News Media vs. Web aggregators: what deal can stop the race to the bottom?

2008-04-18T21:24:10Z

In a recent post on Hitwise by Heather Hopkins, "Content Aggregation is King?," the bind that existing news media is in is highlighted again:

"Aggregators are taking a larger piece of the pie but the size of the pie is growing with visits to content creators and all News and Media websites growing. The trouble is - creating all that content is expensive. It's tough to justify the cost of content creation if those that sift and sort are gaining on those that create."

If the current situation continues, both content creators and aggregators will lose out. The pool of content will shrink, and the aggregators will lose overall pages to place ads on (there will be an expansion of participatory media, but it will not replace the clear demand for general news displayed by the numbers of people visiting Yahoo News, for example). This will be a race to the bottom. What the post doesn't do, and few people have, is to try and figure out what kind of business deal can avoid this result.

The deal between the a consortium of newspapers and Yahoo to outsource listings functions to Yahoo in return for a cut of revenue was predicated on the idea that there is a way to do some basic arbitrage on this situation, but even that doesn't work over the longer term. News media simply can't justify the cost of their content creation staffs, even if they get rid of the printing presses and costs of classifieds businesses. And they have yet to really deal with this (probably because they are still much larger than the aggregator sites even now).

Maybe, as the news media starts seriously shrinking the content pool, the aggregators will start to strike deals like Yahoo did, but more expansive and lucrative. Done right, such deals could raise all boats. For example, rather than aggregating traffic on one domain, Yahoo (or Google, etc.) could drive traffic to news media sites that federated their news with aggregated content from all over the Internet. A network of these sites, served by a single ad engine, could broaden the ad inventory far beyond what Yahoo would ever be able to support on its own and save some amount of a business model for content creators.

It may take a risk by someone on the scale of the 2001 AOL-Google deal, which similarly tied a content network to advertising engine. Marissa Meyer talked about it as "a very big bet, a revenue guarantee" to AOL:

"It caused a huge amount of controversy at the time because by some of the models that we had run, the deal was going to bankrupt Google. Like Jonathan Rosenberg actually got up on the table and jumped up and down about how much we shouldn't do this deal because Google was going to go bankrupt. We had models, one said that we were going to go bankrupt, one which said we might break even... and one year into the deal what we saw was that by signing AOL and broadening the reach of our advertising network we attracted so many more advertisers, and RPMs (revenue per thousand pageviews) went up across the network and we outperformed our expectations by a factor of two, maybe even three times."

The rest is history of course. Something like this may be a fantasy, but following the same course we're on is going to be pretty grim for everyone.

Time for a different solution for personal data on social networks

2008-04-09T19:47:07Z

Many of the good ideas below came in talking about personal information with Doug Fritz, the bad parts are my own ideas.

When Friendster and Flickr started getting traction (2002 and 2004 respectively), I wasn't a huge fan of the functionality, but the social aspect was totally addictive. I got seriously into each (and even got comfortable with the photostream concept eventually, though the fact that I can't easily control the order of the photos without using the organizr still bothers me). These sites ability to connect me with other people blew away all other shortcomings. At the time, the fact that I was putting so much personal information into a public space didn't bother me at all; I assumed a that the Web made old ideas of privacy kind of obsolete, and we would all eventually adapt and make new social norms to cope.

But now that social networks are taking on large chunks of people, I am not so sure. Facebook with a dozen real friends is a much different thing than Facebook with 140 "friends." The information I post there is, to me, the things that define me personally as I choose to represent myself. In the social network context however, the sites ultimately have more control over how I get represented than I do.

I had an experience with that when my story about a stolen cellphone and pictures from the thief unintentionally uploaded to Flickr briefly became news. There were some wrinkles in the story that caused people to be suspicious that I was doing marketing, or lying about the phone being stolen (as opposed to lost). I knew the facts of the matter and was not looking for any large amount of attention in posting the story. But once there was that attention, the people who read the story simply did not believe it and couldn't be convinced otherwise. They used personal information about me that was publicly available (on Flickr, my blog, and other sites) to make the case that I was perpetrating a hoax (for example the fact that I worked at Yahoo was used to say that I was marketing the Flickr service). Even the Reuters reporter that interviewed me asked "how can I know you're not deceiving me?" I didn't have an answer for her. The effortless flow of personal information (flow that I started) did not lead to anything like the truth, and caused some people to think that I was not a real person at all. Privacy as we knew may be gone, but the idea that everyone is now a public figure (with none of the protections of public figures) strikes me as wrong.

We can say that in the context of computer networks "information wants to be free," and I support the idea that copyright is an outmoded framework for intellectual property. But when it comes to representing ourselves, how can it be that personal information should flow everywhere and be used by anyone however they want? Open Social and the Data Portability initiatives are good starts for independent mechanisms, but I still have to give these systems a truckload of personal information, with no way to take it back after it's out there.

An alternate solution would be to allow people to own their personal information store, and choose to allow social network sites access to this store. Sites that behaved badly could be banned. This is much like OpenID and Oauth in concept, where one's identity is tied to a DNS-like way of creating a single namespace for unique user identifiers. It could take the form of a fancier version of an "Attention Profile Markup Language" file; a "Social Profile Markup Language" file, say. It would be stored on my own web server and under my direct control. If I wanted to share with Friendfeed or mybloglog (for example) what sites I've been posting to, saving, liking, or reading, I could allow them to access my SPML file under the condition that it be removed if I decided not to use the application any longer. (This is a geeky solution, but that's usually where these things start.) There should be a better solution to the new portability of social data than exists today, or my own understanding of my personal information will mean less and less.

my fake startup out of stealth mode

2008-04-05T09:38:37Z

Update: I really wanted to switch to Evernote, the built in OCR is amazing, but its lack of easy integration with all my existing stores of things that are spread all over was a deal-breaker ultimately. Still hoping for the new, real thing that's better...
Update 2: Looks like we have a winner: Buru. If it had a badge and a couple other bells and whistles, I'd switch...

My development skills are pretty limited, so when I actually do something I have to make a big deal about it. I hacked together a tool so that I could more easily save links and publish my links feed. I wanted to save some to del.icio.us, some to my blog, some to an email list, and some to twitter, but these were all separate bookmarklets and copy and paste. So the project comprises a new bookmarklet and a bad copy of a Tumblr-like linkblog, which can easily live as a badge on my main blog (there it is to the right). So then I had to come up with a bunch of fake stuff (with that crap Web 2.0 shininess) to make the accomplishment more than it is of course. In the five minutes before Tumblr or someone else makes this thin idea obsolete, it features:

- Extraction of thumbnails, descriptions, and tags for Flickr images, Amazon products, and Vimeo videos.
- Extraction of media embeds for YouTube, Current TV, and Vimeo videos.
- Tagging and collections for all saved items
- Up to 10 email lists of 10 recipients each

So where can you use it? You can't unfortunately; I only have a tiny bit of a server, and I have no idea how to support an actual application. Sorry! I hope someone actually builds something like this so I can use it. For the time being it will be my basement hobby...

Not "out of print" at all, just out to lunch

2008-03-31T17:56:22Z

Eric Alterman's article ("Out of Print, The death and life of the American newspaper" in The New Yorker) about journalism is wrongheaded and off-base. As someone who really loves newspapers wants them to survive, I was surprised by how inadequate to the circumstances it was. His example of an alternative, the Huffington Post, is a great site but is not journalism as we value it nor a business model to follow. Despite its traffic, the site does not support a newsroom of reporters, and every post is heavily opinionated. How is this site (or the Drudge report, or any similar site) relevant to the troubles facing the journalism business? They are just less valuable versions of existing media. Actual new forms of media like Wikipedia, Flickr, Newsvine, and other participatory content sites have the potential to increase our connectedness and make our media culture more human. In setting up a conflict between these sites as populist and journalism as elitist, the article is comparing apples to oranges. As the article points out, social news sites base a lot of their opinion on the work of journalists, and it could be vice versa. The better answer to the choice between the two is "yes." Smarter thinking is needed to create a model where both can exist -- they need each other.

a single-use tool

2008-03-29T04:46:03Z

When I was a teenager I was lucky enough to own a Radio Shack TRS-80 Model 100, with a 300 baud modem. It allowed me to program in BASIC, compose documents in a Wordstar-compatible text editor, and even access CompuServe (sorry about those hourly rates Dad!). It started instantly exactly where I left off, no need to save at all (until you had to change the batteries, then it was time to break out the cassette tape recorder drive). It's eight-line by 40 character LCD screen used little power, so I could use it for 30 hours or so on one set of batteries. I doubt anyone has ever looked so completely scary nerd while believing that they were the hottest shit as I did with that thing. I wish there was something like it now, something smaller, lighter, and simpler of course. I would just like to be able to write wherever I am, without carrying or pulling out a laptop, and I would like not to worry about saving, battery life, weight, expensive equipment, etc. Something like the Flip for words.

Specifications 2GB RAM (enough for 8000 pages of text or about 5 million words, surely enough for anyone), full keyboard, non-backlit LCD display, low-power Bluetooth transceiver (file access only), 1 mini-USB port for data transfer and charging. 22 hours of use per charge. RTF/HTML format text editor. 8 x 8 x 0.3 inches, 1 pound.

Update: Rob Mcartney points me to the Alphasmart, a little bulky but maybe just the right thing! Ordered one... thanks Rob.

magenta

2008-03-27T12:31:52Z

magenta from benjaminclemens on Vimeo. Another fragment, with the very patient Elan Freydenson and Lily Chai.

clear thinking about social media

2008-03-27T11:08:47Z

Given the petabytes (exabytes?) of words that have been expended on social media, user generated content, participatory media, etc. etc. etc., it is miraculous to read a book that lays out clearly and simply the why and how of the phenomenon. That book is Here Comes Everybody, by Clay Shirky. I like reading a lot of the clever and snarky commentary about the blogosphere, and geek out on the details of one start-up's idea or another. But the clearing away of superfluous material and the deceptively simple and straightforward presentation of ideas is a masterpiece of editing. Making social media tools and products has the potential to make our connectedness to each other and our culture better and more human. But the huge amount of half-assed opinion there is among all the people trying to figure this stuff out could swamp that project in bullshit. Shirky has done the world a favor.

See also a video of Shirky's lecture-presentation on the book, and his blog around the book's themes.

hush

2008-02-04T06:57:56Z

hush from benjaminclemens on Vimeo.

Five design processes that don't work

2008-01-10T22:30:30Z

See update at bottom.

Having a "design process" for Web projects is an appealing idea. It codifies principles that designers like into rationales for involving design in major decisions. Since design is imagining, a lot has been written about doing innovation through design. Since I'm getting really old now I have seen many different attempts at making an innovative design process. I can say right up front that the only thing that actually works is working with a designer on a small team that is driven, wants to create something new, and has the courage to go where that leads. As a former advocate and practitioner (for years) of these design processes, it's my experience that they contribute little of value to the success or failure of a design.

1. User and market research doesn't work. As Douglas Rushkoff sez:

"...because you can only research the market's past, not its future, consumer research doesn't ever lead to true innovation. It only helps companies to sort out some of last year's trends in order to create an illusion of sales predictability." -- pg 235, Get Back in the Box (Thanks to Bob Baxley for this quote!)

Users (i.e. people) simply will not tell you what new thing they actually want. In focus groups or other artificial situations, people will reject unfamiliar things. Lab studies and interviews can only evaluate performance of elements of a product, not the whole. Live A-B testing has a homogenizing effect on products, reducing them to whatever is immediately familiar to users. Innovation is ultimately about changing user behavior, not hewing to it.

2. Brainstorming doesn't work. Creation of many ideas and scenarios divorced from reality is good, but limiting this to a two-hour meeting every so often insures that ideation is not part of major decisions. Occasional brainstorming with project teams is often used as a team-building exercise, so everyone is happy when the ideas they produce are close-in, not new or disruptive directions. Make brainstorming part of making the product, not a separate meeting.

3. "Wizard of Oz" videos don't work. Fleshing-out an idea in a speculative (i.e. faked) animated scenario is a tactic used by some designers to sell an idea that has no existing engineering or business basis. Without constraints, real innovation does not happen, just out-of-control "idearrhea." Actual innovation is build-able in some way, anything else should be taken as suspect.

4. Personae don't work. Defining users as archetypes and attempting to see the product through their eyes has some use for avoiding massive disaster, especially for non-designers. But they are of limited use for designers making hard decisions. Personae should not be over-sold as the answer to making the product user-centric, that has to happen in other ways.

5. "Agile" methodology doesn't work. Scrum is a huge success in helping large companies redefine engineering to give ownership of products to teams and to deliver better products. But the process fails when applied to design and ideation. A two or three-week sprint is simply not enough time to try a few design directions and prototype working code (and I have tried on at least a dozen occasions). Any kind of exploration has to go out the window. A very experienced Scrum master and advocate I've worked with agrees that Scrum is best for delivering against a well-defined specification, and even then produces huge amounts of stress on designers (who have to stay somewhat ahead of the other team members in their deliverables).

A good objection to all of the above would be that IDEO and other great companies have been very successful at innovating through design and have many solid methods they use that they detail in books, etc. I would point out that most of the examples people use for success are physical products (like, as is mentioned again and again, the iPod), not Web sites. I think this is because it is much, much easier to create a prototype of a physical product and see how it feels to use it that to do the equivalent with Web work, where application pieces, data structures, and lots of other very abstract design work needs to be done before you can see how something feels in use.

Another good objection would be that for utilitarian, task-oriented projects, the goals are clear enough that all of these techniques are valuable, and that's true enough. But these kinds of commodity projects (e.g. building a new shopping cart or a content management system) are not what most designers can contribute to in valuable ways.

Creating products is not a rational process; much depends on tiny aspects, timing, and emotional dynamics between people. Designers should embrace this and stop trying to define their value through processes. Designers should ideate through building, directly with engineers on small teams.

Update: Thanks to Kai Turner who made several valuable points, I've edited the post to not sound old and cynical quite as much. I should also make the disclaimer that this is a polemic and deliberately foamy-at-the-mouth.

further

2008-01-06T00:11:14Z

further from benjaminclemens on Vimeo.

A ten second thought in pictures, inspired by and made with Jill.

Saving newspapers and books, writers of all kinds

2007-11-28T17:57:55Z

In an article about the politics and economics of consumer choice in cable entertainment ("Bland Menu if Cable Goes à la Carte") in the New York Times, the author talks about how it's actually a good thing that the costs of producing niche programming are borne by all cable subscribers, as otherwise such programming would be very expensive and possibly not attract enough buyers to exist at all (even the most popular channel, ESPN, would rise from $3/month to $12/month). This makes sense, and could be a useful way of thinking about two other media businesses that are in some trouble: the Web and newspapers, both of which are in that bad à la Carte downward spiral.

In the case of the Web, despite the large amount of investment and growth in Web users, the economics still do not support niche content in any serious way. Small publishers have not made a living off of their content, depending on VCs or large distribution networks for investment to stay afloat, or publishing despite the lack of money. And even the most popular blogs on the Internet make a tiny amount of money compared to any other publishing medium like books, magazines, TV, etc. For example, Boing Boing is estimated to bring in $50,000/month in advertising revenue, which sounds like a lot until you consider that a single full-page ad in a local magazine like New York generates the same amount of revenue. Traditional media companies like AOL and Yahoo! have tried to aggregate many small publishers and sell ads across all of them, but despite getting a lot of traffic for this content, the rates for ads remain low (and at flat growth rates). The money remains in search text ads. Jaron Lanier pointed out in an Op-ed that this model doesn't work for content authors.

In the case of newspapers, their audience is falling steadily and won't sustain the costs of keeping the staffs of reporters and editors working (let alone the costs of publishing in print). They have had some success working with internet networks like Yahoo!, but this is likely not sustainable nor will it replace enough revenue to keep things going at current spending levels (magazines haven't had the same problem, but that's another topic).

So given that the cable model supports a number (not a huge number, but a number) of niche content players, and given that the Internet functions well as a big lab for new ideas but not for building content businesses, maybe what's needed is a sort of content consortium, or at least association of creators, with thresholds for membership and the ability to bargain collectively for better compensation. This would be different from a union, more like the Author's Guild or Screen Actors Guild, where plenty of work is done outside their auspices, but productions that generate a lot of revenue must conform to standards of pay. The cable model shows that if companies seeking to exploit content have to buy in to a collective pool of content, content costs are lower while allowing even niche content to thrive. This model doesn't need the cable networks to work for the author's benefit.

In a rough environment for content creators, where the economics are against them (but no one wants them to stop creating), the authors and makers should hang together more than they are. Some kind of association would re-balance a business that is out of kilter, draining the money out of a very valuable part of the culture we live in.

Paul ford is my hero

2007-11-18T14:39:23Z

$5 Chocolate Bar "the urge to say something stupid or fucked rises up like habit, but I fight it. I'm hungry for one genuine moment without insult, wit, or smoke."