getluky.net

Yup, this is my goodbye post. My last day is tomorrow, Friday the 14th. How did I get here?

Eleven years ago, when I first got to USC, I sidled up to a gadget-lugging fellow headed in the same direction I was, and asked if he was also going to the honors engineering retreat. That’s how I met Leonard Lin, who still amazes me with flights of technological fancy that i’m sure to hear about two years from whenever he finds something interesting.

Then, about four years ago, Leonard Lin introduced me to Andy Baio. Andy was looking for a “perl programmer” to come to work for him at a mutual fund company in Santa Monica. Not just any perl programmer — one who would be able to wear a suit daily. Although I had another option that was seemed far more lucrative (talk about finding a local maxima!), Andy seemed to be growing a really groundbreaking and nerdy team, especially for a mutual fund company. Since one of my great priorities in young life was to get to know the financial world better, and also because I liked the looks of Andy’s team, I accepted that position. As it turns out, that’s one of the best decisions I’ve ever had the chance to make.

After getting called out by Jon Udell when he was at infoworld, Andy was insistent that he’d have Jon’s suggested changes incorporated into Upcoming (then Upcoming.org), his pet people-powered events calendar project, within a week. At that point, I had been practicing development of quick web apps and open source libraries for over a year, and we’d built a great working relationship. I offered to help integrate Freetag into Upcoming, and assist with the rest of the work as well. That’s how I got started working on Upcoming, and ever since then, i’ve had the fortune of working for one of the greatest online communities on the web.

So it’s with a tinge of sadness that I decided to leave Yahoo! a few weeks ago. It’s been a great run, and I can’t begin to enumerate the ways in which I’ve become a better developer, leader, and person through this experience. It’s just the right time for me to move on.

What about Upcoming?

If you actually know that I work on Upcoming (a small crowd indeed), you’ll also know that i’m the kind of person who likes to be responsible in my work. I’m happy to say that I’ve been passing on as much of my knowledge as possible, and that the new generation of Upcoming is looking strong. I have faith that they’ll continue to make decisions with respect for the existing community, and most likely will push out features faster than we had a chance to in the past year or so.

What about BravoNation?

This is a trickier one. It’s certainly an experimental project that I’m thrilled to have been able to take from Hack Day to private beta within the context of Brickhouse. It’s also an experiment whose future has not yet been completely decided. I leave a well-documented and fairly mature platform in Brickhouse’s hands, and I hope that those who have used it were intrigued by the idea of combining peer-to-peer recognition with an open network award platform for integration. The possibilities behind the core BravoNation idea are really nothing original; it’s simply the combination of concepts from the video gaming world and the social media web that arose from my experiences investigating the gaming side of SXSW 2007.

I’ve been working on something internal lately which should help see my work on BravoNation live on in a very helpful way. With a little luck, it will be a great legacy. If it sees the light of day, i’ll see if I can score an interview about it.

What’s next?

Well, first, I think I need a little break. I think i’ve never had a serious break - the last time I was out of work for a serious bit of time, I spent it building a collaboration tool for designers, whose needs I got to know intimately in a previous job. Actually, that whole tool was the entire reason why Andy decided to hire me in the first place.

So, it’s safe to say that i’ll be working on personal projects like that old one once again. I just need some time to refresh myself, and get out of my current mindset and get used to being on my own. In actuality, the freedom to work on new things is what I’m most excited about. I’ll probably be developing some toys, some tools, some more artistic abilities, and maybe even a few waffles. Maybe I will even blog more. In any case, if you’re ever in the Los Angeles area, look me up and i’ll be happy to show you around town.

I’d like to thank all the Yahoos who made my time at the big Y! enjoyable, productive, and fruitful. Stewart and Caterina, for introducing us to Y! Local. Paul Levine, for believing in us and giving us the freedom we needed. Vince Maniago, Neil Kandalgaonkar, Kelsey Parker, and Shawn Shen, for being a powerful but small team that helped us get stuff done. The new Upcoming generation, for carrying on the torch! Kevin Cheng, Ernie Hsiung, Nikhil Bobb, Ray McClure, Jeffery Bennett, Salim Ismail, and the rest of the Brickhouse team, for putting up with my neurotics that got BravoNation out the door. Edward Ho, for all the Mario Kart, car talk, and inspiration. Kevin Krawez, for being my navigator in the scary world of ops. Bradley Horowitz, for creating the environment inside Yahoo! that made us feel comfortable in the first place. Anand, Ronny, Van, Peng, Eric, Don, Ganesh, and all the Local folks for not killing us over noise problems. Chad Dickerson, for creating the Hack program, often imitated, never recreated. Tara Kirchner, for changing my mind about PR people. Eric Wu, for caring so much about Y!, and welcoming me to the valley. Legal and the paranoids, for saving us from ourselves. Daniel Raffel, for seeding knowledge I used to build BN. Jay Janssen, for all the crazy MySQL knowledge. A certain team inside Y! for powering so much of Upcoming and being my favorite technology. And, of course, Andy and Leonard, for being two of the most unique and interesting personalities i’ve had the fortune to know so well. Apologies if i’ve forgotten anyone, it’s just that the full list runs a mile long, and I fear that it’ll turn into the entire content of this blog post. Be certain that I won’t forget the way you helped make my experience at Yahoo! better.

Peter Bright at ArsTechnica posted a story this morning about Microsoft’s about-face to make standards-compliant rendering the “default” mode of the new Internet Explorer 8. This is in contrast to its earlier position of attempting to render in a IE7-compliant mode by default, switching to “standards” mode only if the website opts-in.

While I don’t have anything really interesting to say about the technical decision, other than “hooray,” I did find that the latter half of the article tried to explain the change as a hive mind reacting protectively to an external political influence. My experience at a large company leads me to believe that this was not the case.

Microsoft is citing its new interoperability initiative as the impetus behind the change. This move, designed primarily to stave off further EU intervention, emphasizes support and promotion of open standards in a way that the company hasn’t previously done. This move should also help to fend off Opera’s antitrust complaint, which argues that the EU should force IE into better standards compliance.[...]

If the company honestly believed that its approach was, from a technical perspective, the best one—and the software giant certainly put quite some effort into designing and defending it—then it should be of some concern that politics should have caused it to switch. Don’t get me wrong—I’m glad that they’re going to make “standards” mode standard. I just wish they were doing so for the right reasons.

To me, this is a prejudice rooted in the way that the outside world prefers to think of large companies — giant, monolithic entities of a single mind. This occurs time and time again in journalism. The preference to characterize corporate behavior as if the organization were a single, albeit giant, individual is almost always inaccurate. It also does the reader a disservice, as people who follow tech then follow the journalist’s lead, and this becomes the common way to think about and understand what large companies do.

First of all, all large companies are comprised of individual actors. Each of them has their own goals, preferences, and opinions about the company’s decisions. Within Microsoft, undoubtedly there are proponents of interoperability and web standards. There are certainly prominent ones in the public eye, but I’d bet that there are many pockets of culture that live, breathe, and prefer open standards. For example: the engineering staff building IE8. These people must make their case for use of open standards to their management, and must consider the company and team’s objectives and stated goals when constructing arguments for spending dev time on open standards.

Here’s my guess as to what happened at Microsoft.

Imagine that you are an influential lead in a team of developers working on the world’s incumbent majority web browser, and you know that the work that you do impacts an enormous amount of people. However, the company that you work for has a history of prioritizing backwards compatibility, especially with prior work that involves the company’s own products and closed standards. In this environment, it is difficult to make the case for prioritizing the adherence to open standards over compliance with earlier products (in this case, IE7-style rendering). You understand that, and so you consent, perhaps against your better judgement or personal feelings, to go with backwards compatibility.

Suddenly, there’s a cultural shift that comes down from the top, stating that the company is now prioritizing interoperability and open standards. Undoubtedly, this is a strategic shift, and one can imagine that it developed as a result of some parts politics, some parts market environment, and some parts executive staff composition. Since you’re on the IE8 team, and you’ve always bought into making open standards the default mode for your browser for the good of the Internet, now’s your chance to really push your case.

You can suddenly frame your argument in the context of new, important, shiny corporate objectives. Your powerpoint decks make their way up the chain, as managers above consider how they might use the opportunity to prove their “leadership” in following new corporate objectives. This is how the management chain can now evaluate the decision in a completely different internal political light. All of them agree that the decision makes sense. The go-ahead is given.

The GM can then post a blog announcement on the IE8 blog that explains the technical details without a modicum of hype or fuss — but it is likely that many people within that team feel very validated at the moment.

So that’s my theory of what might have caused this. As you can see, I do think that there were most likely politics involved in the decision, but only partially in the way that Peter supposed. Most corporate decisions or changes have their start as external market forces bearing down on the executive staff, which then cause a shift in stated objectives or behavior. Whether these things are communicated to the employees internally (memos) or externally (through PR), the message is sent. Especially in a tech company, the individual workers and engineers now tend to rethink and reframe their decisions in light of the new information. So, although the chain of events frequently may have their basis in external political events and relationships, decisions eventually get made through internal politics and relationships.

That’s my understanding, and if you don’t believe me, feel free to go work at a large company for a couple years, and see how things really happen for yourself.

From Andy Baio’s latest entry:

Very few weblogs do any kind of original research on a daily basis. Most either spend their time repurposing (or just linking to) original research from mainstream media or other sources, or they do commentary and analysis. Their most important role is as information filters, distilling everything going on in the world relevant to their audience and presenting only the good stuff. That’s definitely valuable, but at the end of the day, have they created anything new?

Andy’s going to have a hard time doing original net journalism with so many competitors already in this space. I mean, daily blogging is good, but what you’d really want is at least two or three authors to keep the churn up. It’s going to be difficult for his little Oregon-based reporting startup to get his series A, especially with only a pure advertising business model.

I think Digg has shown that original content is not important; the most important thing is having a big network of friends all pointing to the same stuff as you, all day long. Sure, it means that we’re all little but glorified filters, but that’s how you make the big bucks nowadays.

Good luck on that, buddy! You’ll need it.

Just a HUGE shout-out to Jeffery Bennett for helping us out and creating the cool Bravo Book. You can go get your own if you’re a Bravoteer. I’ll spend some time soon on a blog redesign so I can incorporate this in the base template.

In response to some tongue-in-cheek criticism of polluting the “alpha/beta” project status namespace with yet another term, I’d like to explain.

My experience is that outside the valley, alpha/beta never makes sense to non-developers. One could even argue that it’s been misused, as many products have been released as “beta” even though they are hardly feature-complete. I believe that the original permanent beta tags on Flickr were somewhat of an in-joke (which the step up to gamma sort of shows). On the other hand, pretty much everyone in the states has gone through phased writing exercises, where a rough draft might touch on all the fundamentals of an argument in an unpolished, rough, and sometimes erroneous way. The later drafts reflect a more polished work, but final drafts of high quality imply a high level of editing.

This is much more close to what we are trying to experiment with at BravoNation than alpha/beta. We are also targeting not only developers or people from the Bay Area, so we are also more comfortable with reinventing the idiosyncrasies of the Valley when they don’t work for us. Our goal with the Rough Draft label is to build the expectation among non-technical AND technical audiences that we will be editing, modifying, and re-architecting the entire thing according to the usage that we see. This is the type of project which could gain strong momentum early on in its life, but needs to have a chance to grow quickly while making small mistakes before hitting a huge, mainstream usership.

Anyway, it’s important to note that this was, in major ways, our project. Yahoo! believes in our team to the point that we were given a high degree of latitude in the way we’ve developed, launched, and branded BravoNation, while supporting us and getting us prepared for the types of issues that we’ll need to deal with as a Yahoo! project. So when we call it a Rough Draft, that’s the 4 of us in the main project team, not the mothership, making that call.

Earlier this year, I presented a team prototype hack at one of Yahoo!’s internal Hack Days called “World of Y!Craft”, which was an attempt at creating a platform for incentive systems for various Yahoo! properties. For the purposes of making it a hack, the idea was to make a lightweight and easy-to-integrate platform, and limit it just to Yahoo! sites.

After the Hack Day was over, however, I was approached by Ash Patel, Chief Product Officer (and a Hack Day judge) about opening up that small prototype hack, and making it so that any website could give out awards, not just Yahoo! sites.

So, a handful of people and I worked in free time, met in spare moments, and developed that initial rough prototype into a vision that took our original concept much further than we imagined. We didn’t want to just open up the ability to send awards to websites, but we also wanted to open that up to everyday people, too.

Brickhouse, the internal group at Yahoo! on the lookout for great bottom-up concepts, took a liking to the project from the start, and encouraged our investigation of the concept. Once we got the official go-ahead and backfill for me sorted out, the real fun began.

So, over the past four months, if I’ve been unresponsive, difficult to get a hold of, stressed out, or any of those things - BravoNation is why. I’ve got so much to say about the project: insights about getting things done inside a big corporate culture, motivations for crossing disciplinary interests, and BravoNation itself. In the hopes of not rambling too far in a single blog post, i’ll split it up piece by piece. Of course, the last time I promised to blog a post every day, it was during my series of SXSW writeups (especially my Understanding Avatars triplet) that formed the genesis of the BravoNation concept. I basically had to break my post-a-day promise in order to spend my time building the damn thing, so I think this is probably pretty fair that I resume with some real blogging time now that I’ve more or less completed the original thoughts with an actual product.

See Andy’s coverage for a really great overview and screenshots, or my writeup on next.yahoo.com about the site itself, or us getting ripped on at Metafilter for some good, old-fashioned fun with critics.

Big ups to the BravoNation team, too. Kevin Cheng with his mad interaction design sense, Ernie Hsiung for his community advice and frontend layout-fu, Niki Bobb for jumping into the javascript fray super fast (and getting our coffee, haha), and Ray McClure for his work getting our Builder environment set up - and all the other folks who contributed to the project within Yahoo!. There’s no way we could have gotten so far so quickly (and in such style) without everyone’s personalities and contributions. Thanks Bravoteers! And yet, there’s still so much to do…

In case you haven’t heard, I’ll be starting to work remotely from the Los Angeles area in only another week. I’ll be continuing to work for Yahoo!, as there are still plenty of projects that I want to work on. Unfortunately for me, this happens to be an extraordinarily busy time. However, i’ve got to follow my priorities, and it’s time for me to move closer to my long-distance girlfriend and try out close-distance for a while. It’s going to be hard to stay on top of everything, but knowing myself, a good challenge and probably a beneficial change of scenery.

I’ll be back pretty frequently, so hope to see all my Bay Area people all the time. Zankou Chicken, here I come!

Do you ever have posts sitting around in wordpress for months at a time, delayed for one reason or another? This is one of them, and after re-reading it, I think I’ll go ahead and post it, but remember that it’s kind of a warp back in time to October 2006.

Yahoo! Open Hack Day was a massive, massive success, and i’m glad to have been a part of it. Now that i’ve had a few days to rest and reflect upon my experiences, I want to discuss an observation of Bradley Horowitz’s that has stuck in my mind.

Bradley’s one of the foremost advocates for social search development here at Yahoo. He’s one of the brightest minds around, and always makes my head spin a little bit when I talk with him. You can check out his Keynote presentation here (warning, this was 4GB to download!). Around the end of minute five, Bradley says some really interesting stuff. First, he showed the famous grainy video clip of a monkey trained to perform martial arts kicks in the context of what the worst-case scenario behind user-filtered content could produce. Then he went on to show some beautiful photographs from Flickr’s Interestingness, as a way to demonstrate the better side of what can be efficiently extracted from collaborative participation. His point that these photos bubbled to the top because of implicit user activity is key; as he mentions, the aggregate human cost of photo moderation borne by the user community on Flickr dwarfs anything possible by simply paying employees to review and rate them.

Ze Frank, seen in this video speaking at TED, a design conference, seems to also think hard about the new culture of participation on the Internet. Ze often invites his viewership to participate with him on various flights of fancy, including making silly faces, creating short video clips, playing with flash toys and drawing tools, etc. During his TED presentation, and also at various times on The Show, Ze talked about the hold that various groups have on the perception of art, and how many people are able to participate and create in a new culture without being ostracized by an established hierarchy. He seems to hold that the “ugliness” which seems to permeate MySpace is, in fact, a manifestation of participation outside of the boundaries of hierarchical editorial control. Thus, his position seems to be that the silliness and ugliness of the huge amount of web “design” on myspace depends heavily on perspective. At the minimum, he seemed to believe that participation culture removes barriers to experimentation that could lead to an overthrow of traditional design aesthetics.

These perspectives seem to be at odds. On one side, Bradley appears to be advocating the harvesting of social participation to come to results that select traditionally valuable content. In other words, using New Media platforms to efficiently perform the job of the Old Media publishing empires (Kung Fu Monkeys should be buried!). On the other side is Ze, who seems to be advocating not only a disruption of Old Media distribution through mass publication, but also seems to be leading a charge to disrupt traditional aesthetic values (Kung Fu Monkeys are beautiful, and should be encouraged!).

I think it’s an interesting contrast, and I worry that i’m mischaracterizing the arguments of each.

My personal viewpoint is a bit more nuanced. I believe that one day, web platforms will also be able to efficiently cluster their users based upon interests or tastes, similar to how Flickr can cluster tags to disambiguate meaning. These clusters will probably be designed not around user surveys or self-reported demographics, but instead will most likely be extracted through efficient methods of recording implicit participation information over the long term. There may well be a cluster (which I would belong to!) of folks that do enjoy Kung Fu monkeys, and there is almost definitely a cluster that find it degrading and offensive. The difference here between traditional preference filtering and clustered audiences is similar - one requires a great deal of potentially inaccurate user feedback about their preferences, whereas the latter acts more on implicit activity, and is thus more likely to produce the desired effects.

Not only would such a model be able to try and target clusters of preferences among users, but it would also allow for users to participate in cultures in which they feel welcome from the beginning.

This is going to be the first in a series of woolgathering posts that i’ll make with my random observations about programming. I don’t expect to be really serious in them, but I’d like to get some stupid ideas out in the open, and see if any catch.

Stupid ideas are actually pretty good to aim for, especially if you follow a less structured opinion of creativity. Ideas are things that appear unformed and unstressed by the rigor of actual work, and many programmers fall into the trap of belief that ideas are somehow an end unto themselves.

Any good idea i’ve had has only truly become good after going through the process of turning it into a reality. The constraints of the real world invariably force a programmer to embrace their own fluency in programming expressiveness, as well as the realities of operating systems and system architecture.

Typically, those ideas that survive the realities of implementation tend to be those that seem stupid at first glance. Growth tends to challenge a status quo; to propose ideas that challenge a system guarantees that objections will be raised both internally in one’s mental dialogue, and externally.

If you’ve ever woken from a dream and rushed to pen and quill to write down some grand master plan, novel plotline, or software design, you’ve had one of the stupid ideas that i’m talking about. Chances are, you look at them the next morning wondering what the hell you were thinking. However, it might be worth it to take one of them and see how far you can get - that’s the type of craft that I think makes for the genesis of really crazy good stuff. If the idiotic idea survives the process of making it happen, you might have something great on your hands.

Because i’m a lazy programmer, I like things that save me time. I don’t like having to keep phpdocumentor/doxygen style code up to date with the actual code inside a function. Also, including a documentation compilation step in a build process is annoying, and phpdocumentor/doxygen don’t result in a clean data source of functionality metadata that I can produce my own documentation interfaces from.

So, I guess you could say that there are two main problems I see with a phpdocumentor/doxygen style approach. First, it’s duplication of information. Aside from the DRY (Don’t Repeat Yourself) principle, this is a step worse because you end up repeating the information at least twice, both in code and also in some separate documentation format. That’s annoying, and it will be easily skipped by programmers in search of quick fixes. Secondly, there’s no live source of trustworthy metadata data where one can get at it easily in a programmatic way, without a compilation step.

A good solution presents both a structured data source of documentation, and also avoids needless repetition of information between the code and the docs.

So, since documentation about functions and their parameters is simply metadata about the function, why not store the documentation in data structures within the function itself?

The way I started solving this problem by taking a page from a subset of a hardcore approach to programming called Design by Contract. I’d heard about it before just as a general concept while doing my time in an undergrad CECS program, but always figured that it was too serious to get into. It was really fun to recall this old foggy concept while I was deriving a pretty useful solution to the problem described above.

The entire goal is to convert function docs from these specialized comment markup formats into a machine-readable data structure inside of each function. One will need to define a standard data structure in which to store metadata about a function’s parameters. This will vary depending on the programming language’s embedded types and parameter semantics.

Once a structure is defined, then one can standardize a method of passing in a documentation request struct reference to each method, and when the method notices it, it can simply fill it with the doc metadata and skip the normal control flow for the function. It’s essentially the same as asking a function to describe itself.

Secondly, one can immediately go about enhancing the parameter data structure to include detailed information about each parameter’s type, expected inputs, etc. This information can doubly be taken advantage of by writing generalized validation routines that use this metadata about parameters. If you were going to write validation routines anyway, this saves you having to needlessly repeat that metadata within code.

Thirdly, one now not only has quick, programmatic (and realtime, not compiled) access to function definition metadata, but it’s also more or less in exact sync with the actual parameters it expects, and it also in a standardized data structure. So one can dynamically generate an entire interface for say, spelunking through an API, out of the simple presence of functions that know how to document themselves.

You do pay the price of some realtime overhead, but the very real benefits in maintainability and elegance far outweight that, in my opinion. The implementation of this general derivative concept is pretty academic if you understand all the nerdery above. The key thing to do is to just take a page from Design by Contract, and have your program know about itself, rather than hiding its metadata within invisible comments.