iCalendar

IF IF IF IF IF IF IF IF IF IF IF IF IF IF IF

Update: Sorry to all those who became really confused by this post. I was working on something seemingly not too hard – adding floating times to the iCalendar output from Upcoming, and I ran into severe troubles with schema incompatibilities while playing with both RFC2445 and Sunbird’s implementation of iCal. I got really upset about the super-conditional nature of the RFC and also by how many little tidbits seem to be implemented inconsistently because the spec is so complex.

Just another reminder that nothing is easy.

Bacon Tournament!!!

Okay, so there isn’t really a Bacon Tournament. But if there was, i’m pretty sure that Hormel’s Black Label Bacon would be a contender. I have yet to try the thick style (thick bacon is pretty common to eat in Rome), but I had four strips of the Black Label this morning with eggs, and the extra meat was definitely much appreciated.

Update: The second after I posted this, I began to think about how awesome an actual Bacon Tournament would be. If anyone in the Bay Area is interested, please post a comment and I’ll host a Bacon Tournament if there is enough interest.

Roma, Italia

I had a quick trip to Rome right after SXSW. It was a whirlwind! I think four days is a pretty good amount of time to spend touring the most famous areas of Rome. I’ll try to write up a more cohesive post about it later, but here are some of the photos I took and spent the past two evenings remastering. My favorites are the Colloseum photos and the photos of the statues above St. Peter’s Basilica. Such easy photography subjects…

http://flickr.com/photos/getluky/sets/72057594088612200/

SXSW Wrap-up

I believe I wrote significantly less about the panels at SXSW, but overall, I felt it was a much better conference than I expected. I possibly wrote less because I was much more busy enjoying the crowds and meeting the various people I ran into there. The panels played a much more background role to the vast amount of networking going on during the evenings. It seemed like Upcoming.org became the authoritative resource for all of the gatherings and parties going on during the conference, and I spent a great deal of time using it myself with my Blackberry 8700c, picking up addresses and what-not. It gave me a lot of ideas about how I’d best interface with a mobile experience when on the go.

We (Yahoo) threw the Flickr/Upcoming.org/Delicious party while we were there, and that was a blast. Thanks to everyone who attended. We finally got some schwag out there, and I’m still on the lookout for my first shirt sighting “in the wild.”

For someone like me who doesn’t really meet web people too often, the experience was really valuable. There was a lot of good feedback that I personally heard while at Etech and SXSW, and the amazing thing to me is that our small team was able to adapt certain things to work better live, while attending these conferences ourselves.

Etech Wrapup

The final day of etech was substantially scaled down in terms of size of presentation rooms, but the audience just got more and more packed. It seemed almost ridiculous, to the point that Bradley Horowitz half-jokingly asked the Yahoos to leave and free up valuable oxygen. In any case, here’s what I tried to cover in this last post:

There was definitely some gee-whiz factor present at the morning AJAX sessions. The first presentation was more theoretical than anything – it was Carsten Bormann presenting the beginnings of a disconnection-tolerant AJAX library, which would assist people in creating apps that survive network detachments and outages. Beyond the obvious storage of cached deltas, he went a bit into hinting about technical gotchas, including error handling on communication errors themselves (due to some weird firefox error capturing that goes on), and he also suggested the approach of storing all changes as cookies in the browser, regardless of connection state. Clearly, this presents some inefficiency problems, but it seems like a reasonable approach (to me) to make sure state changes consistently get communicated to the browser on each request. Then, the server takes over the responsibility to delete each cookie transaction confirmed with Set-Cookie. This is definitely confined by the regular cookie limitations, but it’s an interesting approach. He said that a release is available at http://prj.tzi.org.

Then, following up on the AJAX braindumps was the venerable Steve Yen, creator of fascinating NumSum and NextAction, which are AJAX-style web apps that offer the intriguing feature of saving state on Save Page As…, PLUS the ability to upload local changes when a connection is available. His apps show some serious promise, and i’ve been really impressed with his work so far, especially with the crazy development he’s done with NumSum. Apparently, many modern browsers support saving the modified DOM tree on File->Save As…, which allows him to do his magic in the applications. With a Flash 8 storage technique, you get around the missing browsers, but Flash 8 has its own issues… supposedly one of the Safari guys from Apple was IN the room, and I think this may bode well for support in Safari soon.

A lot of the things that Steve’s apps do are really slick. He went into a small demo of the current NumSum and showed layered graphs on the web excel-style spreadsheets that actually looked like they operate more snazzily than the real thing. They also support inline view and edit(!) source. He was able to go into some of the technical implementation details in pretty heavy detail, as he does everything in terms of small deltas stored in single records, which are stored locally and later synchronized with the server. To support that, he wrote a SQL-like TQL emulator in javascript to mirror the similar statements that are run on the server’s MySQL store. He also went over some more tech specs of his TrimPath Junction framework written for these apps, including a small discussion of the power of the with method to dynamically scope methods in code. Pretty mind-bending stuff, and he’s been able to go far with these basic concepts.

After that, I had a tough time deciding between Liz Turner’s presentation on a visualization strategy she built for a huge Harper’s Weekly data set, and the Pervasive Electronic Games presentation. I chose Liz Turner’s presentation.

She started off with a discussion of the existing browseable archive that she operated upon, a time-sorted list of articles that was browsable by keyword hierarchy. The taxonomy itself was nothing to sneeze at, and provided for an interesting basis for the visualization research. The browsing app itself was fascinating, and for the purpose of discovering intersections between various topics covered, it was definitely an impressive tool. Many of the concepts she covered were her general design approach behind visualizing the data set effectively, her use of an iconography or “picture plane” to give viewers an anchor through which they could differentiate different connected layers of data. She also expressed the philosophy that the map was built to reflect and allow manipulation on the query itself. I felt that some of the confusion and difficulty the audience felt when trying to interpret it was because it needed some way to differentiate between the chosen keyword/icon and all of the keyword/icons that were attempting to intersect the main data set.

Although I felt the usefulness of this example was limited due to the bias of the content, I felt that the approach was sound, and would love to see a mostly automated solution based upon automatic tagging and filtering of content, resulting in 2-level tag hierarchy on any time series of text-based content. That would be something astounding, as you would be able to feed in huge text data sets (Usenet/the Well archives, anyone?) and get this interesting visualization to find clusters and pieces of intersected data in a pretty smooth way. I’d probably hook together a new exclusion dictionary ability of Freetag along with an autotagging parser and hierarchy generator, and then hope that the graphical browser was open source and operated upon an open schema.

Bradley Horowitz‘s presentation was a packed room, and yes, I absolutely felt guilty for not leaving. However, I felt like I needed some inspiration, and his presentation was a whirlwind overview of the worthwhile goals that Yahoo! is pursuing as a whole. It ended with an awesome demo of Checkmates over a bluetooth phone-to-J2ME profile connection that enabled Ed to demo the device and have the progress reflected live on the projection.

After that, I was able to make it to the Videogame controller presentation by Tom Armitage, which was a good point about slowness of innovation in game controller design.

Then, I stuck around for the presentation from some EFF guys about the next lawsuits to expect in the near future. It was definitely good to hear these guys talk about what they’re trying to do to protect the innovations that conventions like Etech try to encourage.

In conclusion, I would say that what I got out of Etech had a great deal to do with the fact that I haven’t been hanging out with all these people that much at all. I’ve been barbequing in Santa Monica and working on interesting commercial problems in relative isolation for a few years. Perhaps the reason for the general dissatisfaction with Etech on the part of the blogerati has more to do with the simple fact that they know exactly who to follow and who to look to in the forefront of technical innovation. However, it is hard to find these people if you’re not one of them. This tiny community of sophisticated developers and designers are really a small portion of the huge industry of disseminating technical knowledge an information. Therefore, I believe that conferences like Etech provides a highly curated level of access to no-bullshit technical innovators, to whom access to each other is no big deal. The difficulty of the conference is to attract enough high-level people and technically creative and influential presentations, while keeping the content accessible to those of us without the magical access card to web 2.0.

Another Interesting Day at Etech

Well, the second session-oriented day at etech is now over, and it’s time for me to write some of my notes.

I spend a good part of this morning’s presentations fiddling with setting up a socks proxy on my laptop (see http://www.randomfoo.net/blog/id/3908 for instructions) while listening to some keynotes. Brian Dear’s presentation of Demand was interesting, and a good feature concept. Overall, the approach it seems they took towards populating the set of Performers is to go all the way with user-entered data. Compared with the rest of the site, which is heavily influenced by data feeds, this is a bit of a different approach. I don’t doubt that the trickiness behind building an accurate “artist” term extractor is potentially a good reason to do so. Interesting in that regard was last.fm, who has to deal with matching all sorts of named mp3′s and mp3 id3 tags to a “real” performer in their database. It will be fascinating to see whether users adopt the feature and find it useful enough to justify the additional data entry.

My other interest in their feature is mostly to see whether industries adopt it as an accurate indicator in market research. Large promoters (such as SFX) are likely to be cautious with the data, and small promoters are not as likely to even care about market research. I do like the fact that it’s a grassroots-oriented feature. The big question on my mind is whether MySpace looks at it and integrates a Demand feature on their artist myspace pages before eventful becomes big enough to actually become an accurate sample of real-life local interest in performers. I attended some afternoon sessions of note, including Tom Coates’ presentation, during which I spent a great deal of time attempting to win Buzzword Bingo. No dice though, as Tom is pretty good about never mentioning “Monetization.” His presentation had useful advice about the importance of cleanness of URL’s and other good practices.

I stayed in the same room for Alex Russell’s presentation on push technologies for AJAX, which is used by a lot of “new” web 2.0 apps, even though technically it’s “old” technology. This talk was pretty heavy for me, as i’m quite new to this stuff, but I did pick up some good knowledge about when to know whether push tech is useful, and also some tips about what to look for in building a separate event-driven server platform. After the talk, there was some discussion about a joint effort between Yahoo and MS to build JSONRequest tech that passes data-only across maintained connections, so that should be interesting to watch.

I was at the data & charting presentation, which was good but ran long, and I would have liked to see the entire Technorati presentation that included some smart techniques about detecting splogs.

Sam Ruby’s talk about Neurotransmitters was a pretty whirlwind overview of ideas that Sam had about analogs between human knowledge of biology and human knowledge of web and programming idioms. Of regard was a mention of a possible GPL v3 that would require source disclosure of inclusive services, and a lot of interesting thought experiments which I won’t enumerate here for brevity, and also because I have to go to dinner soon. I really enjoyed his presentation and have a new mental model about networked applications.

More Etech

Had to take a series of breaks in order to re-do news on Upcoming.org, and now that that’s done, I can return to concentrating on the stuff going on at Etech.

Some of the morning presentations were interesting, including Ray Ozzie’s bridging prototype that enabled copy and paste of structured data through autodetecting microformats and allowing suction into and annotation within the operating system’s clipboard. I’m of the mind that this will provide bountiful functionality for MS to harvest into MS Office by running a clipboard enhancer by default in conjunction with IE7. I think that in order to compete, other PIM apps would also need enhancers running in your taskbar, which seems like a gateway into more taskbar hell. Good idea, but i’m worried about the implications.

The multitouch demo was pretty awesome. I especially liked the multiple video feeds being manipulated in realtime in an unlimited desktop.

Last.fm is doing some pretty awesome stuff, and I think we got a hint into some pretty complex deduplication and validation technology and know-how that these guys have in the area of music. Imagine all the different ways for the public to title and share their music… to be able to say that you’ve got about 8 million “clean” titles is a pretty impressive accomplishment.

I’m sitting in on the playsh – the Playful Shell presentation currently and will write more as time goes on.

Cal’s morning talk, continued

I zoned out over the development process part, to work on some Upcoming stuff. Sorry about that.

Now we’re on Unicode 101. Short discussion of charset vs. encoding, a few examples of ASCII, UCS2/UTF-16, and UTF-8. “charset” in HTTP headers and meta’s is a misnomer, should be “encoding”. Rationale discussion using substr… my preferred example is in truncation of db strings. Truncating a string that’s in a different language will completely bust a non-unicode implementation. Some of the probs with MySQL are on older (3.0 and 4.0 versions), so those aren’t huge issues for us. Javascript mostly handles UTF-8 successfully, except for the escape() function, so we have to implement our own UTF-8 escape() function.

UTF-8 and email. Content type headers only apply to content blocks. Headers need inline encoding, ex: “=?utf-8?Q?…” Defined in RFC 1342. “Q” is similar to quoted-printable, B is base 664. This allows non-ASCII in the subject line.

Any junk that you receive, you assume is Latin-1 that is misidentified.

Filtering done only at the outside, possibly except for “signed data.” I don’t really understand his example here, so i’ll ask later. You never want chars below 0×20. Apart from normalized carriage returns. Carriage returns mess up XML attributes, though.

He has a Filtering PCRE sample, but there’s a mistake, and he recommends using iconv to convert from UTF-8 to UTF-8.

Discusses HTML and javascript filtering. Some common XSS hacks. Promotes lib_filter. :)

Dealing with email – receiving email is useful, very handy to support mobile blogging, support tracking. Discusses uses of pipes from /etc/aliases.

Mime in a nutshell – defines some content types, multipart mail can contain sub parts. For mail with attachments, main part is in text, rest is in binary.

Mail::mimeDecode – what Flickr mail parser is based on. Not too broken for their use, heh. application/ms-tnef – that’s MS’s winmail.dat. Only used by Outlook (transport neutral encapsulation format). A packed list of files and metadata. The spec was buried on the MS site, and is fairly easy to unpack if you know the spec. Some code somewhere may document or handle it.

Incoming email isn’t necessarily latin-1 or utf-8. Forcing a character set is kind of lame. Can find out the intended charset from email’s content-type header. Spec states they must state the content-type unless they’re latin1. Fortunately, iconv does the heavy lifting.

Wireless messages suck, because they do special casing but then also append crap at the end often. Attachments as images but text/plain mime type. Wireless carriers also append additional images which are their logo and extra spacer gifs. The worst offenders send links to images instead of actual images. Sometimes slashes are doubled up, so they break automated grabs. Try to capture weird emails and add them to a test suite, along with expected results.

The desired system is a closed test system with easily repeatable regression tests.

That completes the pre-lunch session, so we’re off to lunch.

Arrived at Etech

So I just got in to San Diego yesterday, and i’ll be here til Friday. I’ve brought along the ol’ Powerbook so I can get some Freetag hacking on in the evenings. I’m currently attending Cal’s Flickr scaling presentation, which is pretty detailed, and the audience seems to be pretty engaged.

Interesting points so far:

  • A nice, quick overview of available software load balancers.
  • Not much of a discussion so far of the dynamic structure of Flickr’s API, but some hints that the approach lies between monolithic functions and full OOP (a difficult balance to achieve).
  • Short discussion of usefulness of Layer 7 caching. Separation of hashed content in caches and ACLs would probably allieviate some of the performance issues that we regularly see on Upcoming. Luckily, we are at the point where we are starting to optimize some of our process, and so far, the one major point i’d make is to avoid left joins whenever possible.
Especially droll was this quote about mixing of Presentation and Databases: “That would be like putting HTML directly in your database and serving it directly. That would be insane.”

One nagging question I have: Where can I sample a trifle in the US, Northern California, specifically?

Save House of Cosby’s!

Yesterday, Andy received a Cease and Desist from Bill Cosby’s lawyers that ordered him to remove a parody of Bill Cosby called “House of Cosby’s,” which was created by amateurs. As this is pretty clearly a parody of a public figure, the legal threat is tantamount to legal bullying on the behalf of Bill Cosby. Any comedian should understand that parody and satire are protected under free speech, and those who would attempt to silence that are simply hurting themselves.

To show my support, i’ve created a “Save House of Cosby’s” banner that you can grab from here and post on your website. I’m not a graphic designer, but hopefully it does make you curious. Link it to: http://www.waxy.org/archive/2006/03/03/litigati.shtml Hopefully it will help spread awareness about the stand that Andy is taking on this issue. Save House of Cosby's!