I’m done with microformats. From now on, i’m either building separate developer tools and relationship, or i’m not. I say that having been through the cycle of adopting hCalendar and hCard a few times, not just as an industry commentator. My reasons are threefold. First, in the real world, publishing microformats requires you to rewrite DOM structures, publish extraneous invisible elements, adopt new schemas, and adopt data publishing-like structures on frontend pages intended for the browser. Second, the relationship between publisher and developer is not significantly improved by microformats, and would be better served by a separate pathway for developers. Third, its proclaimed benefits as a standard are extraneous because hCard, hCal, etc. are valiantly attempted, but incomplete redefinitions of existing standards like vCard and iCalendar in XHTML-like formats.

The idea of microformats sounds swell, inexpensive, and easy. Take your existing data, and surround the data-ish bits with tags that separate it into parts with semantic meaning. Unfortunately, saying that something is easy does not make it so. I don’t mean to disparage the work that microformats mavens have done, but my experience with being a microformat publisher has shown that things are exponentially more complex than they let on in the “sales pitch.” I don’t think they realized what they were getting into when they started on the process of actually getting publishers to conform to this stuff.

Surrounding your existing markup sounds simple enough, right? But consider how you’ll have to nest things together. Should you publish microformats on both listings and detail pages? Are there required fields that aren’t present in the content you already have on the page? Do you just publish lots of invisible XHTML content to the page to fill in the missing stuff? Do you dare to deal with recursions? What if your data is split up in separate divs? What happens when your data model does not fit with the standard? What if you are not a very good HTML contortionist? What happens when your presentation is different based on varying pieces of available data? Is hCard publishing even useful at all on public pages without private, uniquely identifiable information? Oops, there are no microformat validators, either. This is especially difficult for something with as many ominous implications as a stepchild of iCalendar.

Much like an oral agreement, publishing microformats is an informal agreement between you and (hopefully) a developer community that sets up a relationship with plenty of vagueness, inertial resistance to change, and potential landmines to step on. Would you create a real developer API without a TOS, agreement, or at the very least, guidelines? Are you prepared to deal with objections if, when cutting costs, you rev a frontend design and lose some important aspect of microformat structure on the page (or, god forbid, you just don’t bring microformats over at all). Alternatively, are you prepared to announce all frontend markup changes? Does publishing a microformat without a special agreement mean that you are implicitly allowing comprehensive scraping of your web data? If you spend an hour seriously considering the costs of treating your frontend interface as a programming API for the sake of your relationship with developers, would you then rather spend those costs there, or on proper versioning, documentation, and communication with a developer community over a real publishing protocol or API? Publishing microformats while not having formal consumer support is a commonly what happens, but it is a poor midway point to leave yourself at.

The only place I can still imagine microformats surviving a cost/benefit analysis is in the case of preparing for search engine crawlers. In theory, if everyone publishes their websites according to a few semantic standards, the Big SE’s can embed structured data in their search results and act as aggregators of real data. There are a couple practical issues that you’d have to ignore in order to go for this pipe dream, though. You’d have to hope that the structure of microformats is fault-tolerant enough to survive the endless mangling of random developers trying to publish junk data, and that whatever was clean enough to make it through would be parseable, high quality, not eliminated in dupe checks, and relevant to a big enough segment of searches to justify the cost for the search engines to link to it. Second, you’d have to accept the fact that you probably wouldn’t get any permanent special treatment in search results, lest microformats become a new meta tag for SEO. Third, you’d have to assume that the big SE’s wouldn’t take these handful of high-order data entities and send it to their big topical datastores for aggregation and republishing themselves. I realize the benefit of a “standard.” However, why wouldn’t big aggregators crawl the web for content that conforms to existing, more mature standards that microformats are based on? Especially when these have often been passed through working, real world consumers of vCard/iCalendar before making it to production.

Anyway, here’s the question I want to put into the reader’s mind: should one spend time and effort making a frontend into an informal API through microformats, or to instead spend it on building a fully supported API or data publishing system that exists and operates separately? I think my stance is clear – i’m not against the theory of microformats, but i’m certainly going to differ with anyone who thinks it’s practical. If you can really think all that through, and still think microformats are a good thing to spend your resources on, then by all means, give it a shot. Just don’t say you weren’t warned.

Quick tip. Since the dev only built a windows binary for version 0.2.1 of uTidyLib, this bugfix for locating the dynamic libtidy on macosx is necessary.

Just edit tidy/lib.py, re-run the setup install script, and it suddenly can find libtidy.

As some of my friends know, i’ve had a love/hate relationship with the Woot-sold Infocus DLP 61md10 HDTV. I had one bulb fail nearly 10 months in, and I lucked out and got a warranty replacement. After moving this past weekend, I had a weird green colorband floating up the screen very slowly on both component inputs. At the time, I didn’t realize that it might have been related, but I had a very loud hum when putting the audio leads directly to the TV to power the stereo.

I did some research about color weirdness with DLP’s, as well as specific problems associated with the Infocus / RCA model that I have, I was worried that the issue might be the colorwheel on the lamp assembly, or (horror of horrors) possibly the entire “light engine.” I blew out the colorwheel with some compressed air, but didn’t notice any ball bearing noise at all, so I didn’t think that was the issue.

The next day, I was able to hook up an HDMI input and verify that the problem didn’t exist digitally (phew). This post referring to 60hz power line interference made me curious about the cable line coming in from Charter to the cable box. I routed that cable line through my cheesy Monster Cable surge protector (which has 3 coax in/out thingamajigs), and that cleared up the issue – no more weird green color band, and no more loud hum. I am assuming that there’s some sort of voltage regulator or noise filter on those coax passthrus, but this is the first time i’ve been glad that Woot gave away those surge protector kits along with the TV.

Now, hopefully I can enjoy another couple thousand hours of DLP television without spending $500 bucks on a new lamp.

RPX, a new startup, has come onto the scene offering to aggregate a patent portfolio, and offer a covenant not to sue to companies who license their entire portfolio in aggregate. They are funded by venture capital and also claim to have IBM and Cisco as initial clients.

Dogg, that reminds me of some cards.

Much like Roast Beef’s greeting cards, a technology patent originator can fathomably come up with an infinite pool of them by just applying the same rubric to a new set of circumstances (in the patent trolls’ case, by appending 14 pages of a definition of a computer to tie a business process to a tangible machine — le voila!). This means, as a creator of software technology, you are potentially on the hook for limitless amounts of patent infringement, as nearly everything you can do on a computer has had a patent filed on it in the past 20 years. The mere fact that one entity is aggregating a portfolio of patents has no bearing on the fact that there are potentially unlimited amounts of patents that can be aggregated by others, much as readers of Achewood would probably be able to assist Roast Beef in coming up with limitless amounts of new greeting cards.

It looks like TechCrunch is perhaps appropriately skeptical. Their business model revolves around subscription fees instead of enforcing patents through offensive litigation. If i’m understanding these terms correctly, if you don’t subscribe to their patent portfolio, then you are potentially subject to offensive litigation (which is the practice they are claiming to fight against). Correct me if i’m wrong, but doesn’t this seem to you to just be an extension of existing patent trolling practices (i.e. adding on subscription revenues to existing litigation + settlement proceeds) that is made feasible by the aggregation of large amounts of patents with a large amount of venture capital, instead of an attack on the NPE cottage industry? If so, wouldn’t it seem disingenuous to frame oneself as such?

It is not possible for one entity to aggregate all broad technology patents that pose a threat to oneself. Therefore, the subscription of a company to one particular NPE’s portfolio does not preclude the possibility of being sued for infringement by others. If a subscription model proves to be a more profitable / reliable revenue stream, you can expect other NPEs to adopt it too. Who knows, perhaps portions of the subscription model will be patented by RPX, and they’ll force other NPEs to license their patent as well if they want in to the game. It would be like patenting some aspects of the business process of patent trolling, which would be a fun irony, but still a sad one.

Some days, it seems like buying rights gets you a better return than buying property, but mainly if you’re an asshole.

I’ve been working on my own PHP-based project for a month or so, and have rewritten some basic generic libraries from scratch in PHP. I’m pondering whether I should open source them, and would appreciate it if you would comment on this post if you’re interested in seeing me put in the effort to write docs, standardize, then toss specific portions or all of it online. I learned my lesson through Freetag; it takes a lot of energy to keep something maintained. Here’s some of what I have:

• A database wrapper singleton around PDO
  • Simplifies code for working with “prepared statements” DBI-style. PDO prepared statements work great for getting around SQL injection, but I found them cumbersome by nature.
  • Consolidated error and exception handling with custom callback support.
  • I’ll eventually add fancier stuff in here, such as handle splitting for replicated architectures in here, and some read-after-write consistency code.
• A simple nonce library for protecting against XSRF/CSRF
• A simple library for handling file and image upload/resize
• A simple code profiling utility
• Some validation routines (I would like to add in some self-documenting API neatness that I wrote about before, when I get some time)
• Some other random handy stuff, like arg signing, link building, human interval descriptions.

It’s all pretty consistent with my philosophy of abstracting away website-related functionality instead of architecture. With this set of limited libraries, i’m building at a pretty fast rate, and I can almost always figure out what’s going on by just looking at one or two files. I’m trying to get a nice foundation that I can build fairly serious stuff on top of without getting completely confused.

Oh, and it all depends on php/filter being enabled with a default filter of “special_chars”. Come to think of it, I might write a separate post about using that, because it’s handy.

These days, i’m generally a packaged framework-for-the-web curmudgeon. I prefer simple libraries corresponding to simple, common, generic problem sets. It’s one thing to adopt a general-purpose scripting language, or even a general-purpose “hard” programming language, but it’s another thing entirely to download or buy a prepackaged framework on the basis of marketing material or buzz about the shiny new ur-framework for all development. My opinion was arrived at mostly through personal experience, and undoubtedly people will have different opinions based on their own experiences working with such software.

My hypothesis is that frameworks tend to occur naturally out of the needs of specific applications of technology. If you’re using a programming language to build a listing-based website, and you’re building things from scratch, patterns will evolve in your code. If you’re being rigorous about refactoring, you’ll be acutely aware of patterns that may be abstracted away, and it will be very natural to reduce them in some form or another. Eventually, if you add more developers, the lead developer will probably generalize an approach to building features that looks awfully like a framework. It could be lightweight, generator based, MVC based, queue based, or whatever — but this evolved framework, at its essence, is a reduction of noticed patterns about listing data systems to practice. However, why not avoid repeating all this work, and instead skip it by picking up a third party off-the-shelf framework and starting with that off the bat?

A heavily evolved framework is a codification of patterns that work for a specific problem set. This is fundamentally what newcomers to programming languages do not understand when they go reading neato tutorials for the framework du jour. It is also what many framework zealots (and sometimes, evangelists) do not understand when they are aggressively hunting down misguided naysayers who misapply frameworks and complain about it in public.

The risk of having programmers begin their work on top of established frameworks is that they typically do not have the experience necessary to understand what patterns have been adopted, why they have been adopted, and how those patterns would apply to their task at hand. The only people who understand this are usually the most experienced developers of the framework.

Unfortunately, the type of developer most attracted to the marketing claims of frameworks are inexperienced ones. And even more unfortunately, most frameworks’ greatest advocates are experienced developers of the frameworks, whose subtle and nuanced defenses about the proper use of their framework go completely over the heads of most beginners, who only stare googly-eyed at fancy baked demos.

This leads to large, bizarre codebases built on hacks, on top of hacks, on top of frameworks. It also leads to completely unrealistic expectations from businesspeople about the sustainability of development pace for fast, new, framework-based prototypes. It’s par for the course for our industry, and it’ll probably continue for a very long time as the trade of programming grows up in the new century.

Is there a reasonable argument that some “general-purpose” frameworks are good enough for nearly all “straightforward” web applications? Maybe — but this is a tricky subject. Many web applications share the same fundamental characteristics. If you have a general-purpose framework, it may work especially well with the characteristics of web applications. The thing that most web applications have in common is that most of them are dropped far before they get to a very serious type of problem that i’m referring to. So, in the sense that I don’t believe most web applications will get very far, it’s probably fine to use a framework for experimentation or prototyping. But one should be prepared to do the real work at some point, if the project is to be taken seriously.

There is also the annoying fact that when frameworks are popular, they tend to be a big hindrance to prototypes that never got rewritten. The point at which the misapplication of technology becomes a problem too often coincides with surges in popularity of a service. This leaves you open to competition and puts you in a very difficult place, especially if you’ve worked hard to open up a new market.

My advice for anyone starting to get a bad feeling about frameworks: work on real world projects, and start from scratch at least once. Real world projects jolt you out of the 5-minute screencast armchair Internet expert mentality. They have messy requirements, will suddenly strike you with landscape-changing revelations about how you’ve been going about things wrong, and will challenge you to grow more than you thought you ever could. Invest in your frustration up front, and the reward? When you’ve built a mature project, maybe you can generalize it and turn it into a framework… ;P

I’m so proud of my country tonight. Barack Obama has become the next President-Elect in an historic Presidential race that saw the best and the worst emerge from everyday Americans. His acceptance speech was moving; not only did he emphasize the importance of individuals to his campaign, he also brought up the topic of humility and addressing the country’s problems together. We can indeed find common ground to work on the most threatening problems to our country.

I’ve had a lot of difficulty even listening to the Republican point of view over the past eight years. I don’t doubt that the neoconservatives in the Bush Administration used and abused the differences of opinions between people in America to drive a wedge between us all – public, house, senate, and media. However, Obama’s campaign has gone beyond just earning my vote; it has also made me reconsider my emotionally charged reactions towards GOP talking points.

The first was Joe Biden’s anecdote about motives during the Vice Presidential debate:

I have been able to work across the aisle on some of the most controversial issues and change my party’s mind, as well as Republicans’, because I learned a lesson from Mike Mansfield. Mike Mansfield, a former leader of the Senate, said to me one day — he — I made a criticism of Jesse Helms. He said, “What would you do if I told you Jesse Helms and Dot Helms had adopted a child who had braces and was in real need?” I said, “I’d feel like a jerk.” He said, “Joe, understand one thing. Everyone’s sent here for a reason, because there’s something in them that their folks like. Don’t question their motive.”

Joe’s point is that although we may oppose other peoples’ political views, we should not succumb to prejudice about their character. Barack Obama’s campaign has shown that you can address the American public with maturity, dignity, and calm assertiveness, and still win the highest office in the nation. We can’t mistrust an entire segment of Americans simply for identifying with some policy views of the Republican party.

We can and should argue strongly with logic and passion when we disagree, but we cannot forget that our opponents are Americans too. I believe that both Obama’s acceptance speech and McCain’s concession speech strummed on this chord, and I sincerely hope that the political bases of both parties can be mature enough to rise to a more respectful level of discourse, and heal the wounds of divisiveness that have proven so profitable for the special interests in this country. Now let’s see what we can accomplish in the next four years.

As much as i’ve hated hearing GOP commentators and Sarah Palin herself talk about the “Real America” that supports McCain/Palin, I don’t personally feel that it’s an expression of racism. Rather, this is a sign of denial on the part of party leaders that they have lost the support of the majority of the American people.

This rationalization for losing is actually a pretty common thing to experience growing up in middle America. Lost the grade-school soccer game to a team that fouled yours a few times? Don’t worry, you were the “real winners.” The characterization of losses as non-”real” is a defensive compensatory mechanism. Subscribing to one party in our political system is often shown to limit your ability to think critically about your own stances.

So, although I think it’s despicable to actually believe that anyone who doesn’t support your own political party is un-(or anti-)American, I don’t honestly think that’s what’s going through the minds of the people giving this explanation for the lack of support for McCain/Palin. It’s just a obvious sign that the faithful holdouts are in denial about the unsustainability of the methods and practices of the modern Republican party.

I stumbled upon this short while researching John C. Reilly, and absolutely loved it.

I apologize in advance for the following post, because, firstly, I am an apologist by nature, and secondly, it is a bit less well thought out than most posts, but the topic compels me to write about anyway.

I have a theory that what underlies and led up to the current credit crisis is really a crisis in trust. Trust is the fundamental thing that allows us to enter a transaction with another person (or with a corporation), and if we do not trust our counterparties, then there is no way that capitalism may work. What the financial industry has done in the name of innovation over the past couple decades is to bet far more than they should have on the premise that trust is unnecessary; basically, if you don’t trust someone, you just charge more interest, and extract more of a profit from a larger segment of people to make up for those who default. As long as we were in a boom economy, the relative numbers of parties that would be in default were low, and real, honest-to-goodness counterparty integrity and trust (like getting a loan from a banker you knew in high school) was quietly replaced by complex financial modeling.

What has occurred in the lending market (probably best described by This American Life recently) may the effect of the sudden failure of decades of financial modeling to compensate for the gradual deterioration of real trust between individuals on Wall St. Trust was just a limitation on the levels of profit that could be gained through financial innovation; the real geniuses found out that you could give a loan to basically anyone as long as you could include their probability of default as a number in your financial calculation. The lack of information or faith about the balance sheets of counterparties seemed to become a problem so suddenly because the innovations obfuscated the real cost of the debts that were being incurred (think option ARMs or balloon payments), until finally it all came due and nobody could be counted on to fulfill the terms of the mortgages (or, for that matter, credit default swaps) that they had agreed to. The boom market can not be described as the result of complete trust in all counterparties; it was the hubris of believing that trust could be outsourced to third parties (the ratings agencies), and as such, not being particularly important, instead of being an absolutely essential component of deciding to do business with someone else.

It seems like I’m being a bit hysterical about the importance of integrity and trust in a free market system where capitalists are incentivized only to maximize short term profit, long term implications be damned. To trust recent corporate philosophy, preserving your real trustworthiness is a matter best left to those in branding or P.R., and that your actual actions can be painted over or hidden by name changes or evasive legal maneuvering. However, the culture of a company or even the reputation of a business leader with integrity can visibly affect the way that all people interact with your company. To make my point, i’ll invoke the image of Warren Buffett, whose every word the American public seems to look to as if messages from a mystic oracle.

It may be true the reason the credit market seized up is solely because of the sudden extreme value of cash on hand to protect against strategic competitive moves, such as that alleged by some against J.P. Morgan. It also may be true that trust has no place in a free market; that all decisions should be based upon rational evaluation of the creditor’s ability to pay. However, in this type of an economic storm, I am going to make a prediction that the value of integrity and ability to honor promises, regardless of the cost, is going to be a determining factor in which businesses are able to survive and make it through the recession/depression. Those who are able to cultivate a supernatural track record and reputation for trustworthiness and integrity will survive where the sharp business of the past will fail without trusting counterparties. In fact, I believe it will likely be a successful tactic for small businesses to overtake their failing large competitors caught up in this mess over the next decade. Although I sincerely hope that the country recovers from this crisis quickly, it could very well prove to be a good time to be in business and change the way that business is done in America.