For your learning enjoyment, a nice explanation of the mathematical difference between Surely and Almost Surely.

Nerd alert: If you don’t use or care about Drupal, feel free to ignore this.

Drupal is a popular PHP framework/CMS for constructing small to medium-scale websites, and it has an extremely extensive (and complicated) set of hooks available in a modular architecture. One of the framework-y things that Drupal does is to create and maintain a one-to-one relationship between URLs and content. If you only maintain sites where each section is completely different, and content can be segregated cleanly, this isn’t an issue. However, this can’t always be done. In some cases, it’s a requirement to have content duplicated in multiple sections, and to maintain breadcrumbs, menu locations, etc – in general, contextual stability.

One of my friends had such a project, and he asked me if I’d be willing to work on a module that would fit into Drupal’s framework and allow for a single piece of content to be published to multiple places on a site. The first thing I did was to look around for an existing module that might fit his client’s needs. Node aliases were already part of the drupal core semantics, but those were more of a URL alias. Searching through the modules section, I found something pretty close – the Node Symlinks module from VojtÄ›ch Kusý. Unfortunately, it was basically a proof of concept with several major flaws, and Vojta, the maintainer, was busy completing his university degree. It was a good start, though, so I began taking a look at what this module did. In order to meet deadlines, I ended up writing a significant portion of new code, redoing the user interface, and rewriting a large portion of old code as well.

It’s never straightforward whether to fork or to patch and existing open source project, so i’ve basically forked the module at this point. I’m willing to try and re-merge the work into Node Symlinks, but the user interface and operation is quite different now, and the maintainer is busy, so i’m not sure how that will work. I’m going to hold off attempting to do anything on drupal.org’s module repository until we figure that out. Anyway, for the time being, i’m just offering a download of this code from my blog until we get it sorted out. This works with Drupal 6.12, requires ahah_helper, and the following patches applied: #435826 on ahah_helper, and #336517 on teaser.js. Update: It looks like some of the AHAH / AHAH helper stuff is breaking the Delete button on the node edit form, so i’ve made a quick and dirty workaround update to help with that. The link has been updated below.

http://getluky.net/downloads/nodemultiparent-6.X-1.3-dev.tar.gz

How it works

Node Multi-Parenting uses a similar approach to Node Symlinks to generate alternative links to content nodes that maintain contextual state within the menu system. It does this by inserting and maintaining menu_links entries within Drupal manually. Instead of /node/node_id, these links look like /node/node_id/dup/menu_link_id, and the menu hook is registered to handle these. However, since this is essentially denormalized data within Drupal, it gets wiped out every time a module is enabled or disabled, and is relatively unstable within the architecture. Thus, i’ve added in a new table to the drupal schema in which serialized data is stored and maintained so that re-enabling the module can re-populate the menu_links table.

The UI has been modified to change language to be more intuitive – instead of “Symlinks”, which nobody outside of UNIX programmers will know, i’ve changed to “Locations”. Users of drupal can then think about these as alternative locations in which to show content.

There’s an administrative menu now available to show all of these alternative locations on one page. It doesn’t have pagination, so it’ll be slightly cumbersome if you put this to heavy, heavy use.

The node edit form is modified to add a collapsible menu which now works via AHAH and ahah_helper, instead of multi-step forms in the original Node Symlinks (it should degrade gracefully if JS is not enabled, though). This was kind of challenging to get working, but it seems to do fine now. I’ve edited the instructions somewhat for clarity. It should auto-expand and collapse depending on whether there are any locations currently available for that node.

As in the original Node Symlinks, a META ROBOTS FOLLOW, NOINDEX tag has been added to the header on each of these duplicate pages. This should avoid the #1 objection to such duplicated content schemes, which is the potential duplicate content penalty that google can levy if you don’t include specific robots instructions to avoid it.

Controversy and a Caveat

If you use this module, you should be aware of two things:

• There are those within the Drupal community that are vehemently opposed to doing this, because of the philosophical decision to make content uniquely addressable via one URL. However, this does not permit multiple contexts as was a specific requirement of my friend’s client. I am sure that this blog post will catch some flak about it. I’ve done my best to try and make this work for Drupal 6.X without getting in the way, and without relying too heavily on the denormalized menu_links table. In commercial programming, you know the rules so that you can break them safely, so please be aware that the use of this module does diverge from the Drupal philosophy.
• Because this goes a bit against the grain, I can offer no guarantees that this module will work in future versions of drupal, or that some upgrades may change the functionality of Drupal in ways that might break this module. I believe that since data is stored in a separate table, you should always be able to program or hire a programmer to get you out of any binds, but be forewarned that this module should be considered slightly fragile in upgrade scenarios, especially as I haven’t ported or tested it on 7.X
• I performed this work under sponsorship, and although i’m happy to contribute back to the Drupal community, I don’t have the time to really dedicate a huge chunk of time to answering questions or maintaining this on a volunteer basis.
• I can’t count. Sorry!

With that out of the way, give it a shot and let me know what you think. Feedback on code quality or approach (especially the AHAH stuff, which was kind of mind-bending) is appreciated.

About two years ago, the popular “health quiz” website RealAge.com was acquired by the Hearst Corporation, a media conglomerate, for an estimated $60-70 million. It’s a website that dresses up as a fun, friendly, and harmless quiz, but if you look beyond the facade, it’s a drug marketer’s wet dream. I wrote about the clear transfer of private health information to the Hearst Corporation when it happened, because I was dismayed that the mainstream media completely ignored this aspect of the acquisition.

Well, apparently the mainstream media has woken up two years later (better late than never), and has started to comment on the targeted marketing programs RealAge is offering to big pharmaceutical companies. From the NY Times article last week:

RealAge allows drug companies to send e-mail messages based on those test results. It acts as a clearinghouse for drug companies, including Pfizer, Novartis and GlaxoSmithKline, allowing them to use almost any combination of answers from the test to find people to market to, including whether someone is taking antidepressants, how sexually active they are and even if their marriage is happy.

This also got coverage on Foxnews.com and on a Healthcare IT news site, where I got the trackback that alerted me to the new stories. I’ve done a little more digging, and I’d like to offer another post summarizing what I believe to be the issues that the mainstream press needs to cover more thoroughly here.

RealAge is selling targeted marketing to specific segments of this population based on private health information that the consumer has “trusted” to RealAge by answering their online quiz and creating an account. It sells to big pharmaceutical companies, who would like to get drug advertising to consumers, especially if it matches up with their own personal health data. While the technique of targeted marketing isn’t technically selling anyone’s private information directly (as Fox News claims in its article), it does provide an extremely accurate method of reaching consumers that has tended to rile privacy advocates in the past.

The actual way that targeted marketing works is by collecting user behavior information, aggregating user profiles by their behavior, and then creating marketing channels against which advertising can be sold by the marketer. If you’re a drug company, you can sell depression drugs to people who have indicated problems with depression, without actually having their email address or personal info. You just have to find a company that is willing to sell you ad space in email or on the web to a specific set of users who have indicated such a problem.

This is the exact kind of behavior that internet giants Google, Yahoo, and Microsoft got a Congressional letter about last year, but it’s being done with one of the most sensitive topics imaginable – personal health data.

However, doesn’t this exact wording in the RealAge privacy policy today seem to go beyond targeted marketing?

Except as otherwise stated in this policy, we do not sell, trade, or rent the Personal Data collected from our services to third parties. However, we will share your Personal Data with third parties to fulfill the services that you have asked us to provide to you, including but not limited to sending you free newsletters and promotional e-mails. These third parties are required not to use your Personal Data other than to provide the services requested by RealAge. You expressly consent to the sharing of your Personal Data with our contractors and third-party service providers for the sole purpose of providing services to you.

Hmm. Let’s see how they define Personal Data, shall we?

We collect Personal Data from you, such as your name, e-mail address, country of residence, zip code or mail code, gender, and/or birth date when you register for a membership with us or when you choose to use the various services we provide on our site (e.g., Tip of the Day, Health Assessments, etc.). We also collect other types of Personal Data that you provide to us voluntarily, such as your health-related information when you respond to surveys, health assessments, and questionnaires.

Does this strike anyone else as giving them wiggle room to go beyond targeted marketing, and provide Personal Data to third parties? This section certainly seems bizarrely worded to me. Here’s the version of the Privacy Policy that archive.org grabbed in October of 2007 (immediately following the acquisition):

Except as otherwise stated in this policy, we do not sell, trade, or rent the Personal Data collected from our services to third parties. However, we will share your Personal Data with third parties to charge your credit card (if authorized by you), fill your order, and deliver promotional e-mails. These third parties are required not to use your Personal Data other than to provide the services requested by RealAge. You expressly consent to the sharing of your Personal Data with our contractors and third-party service providers for the sole purpose of providing services to you.

The change is highlighted in bold. While I wouldn’t go so far as to say that something malicious happened, I believe that the newer version does grant them more abilities in the disclosure of Personal Data to third parties, while restricting them to those that “fulfill the services that you have asked us to provide to you”. That still seems like an awfully large loophole to me, but I think the main thing that allows for scary possibilities is the lumping together of fairly generic personal info and private health information both into the moniker of Personal Data. If the data they could share were limited to address, zip code, etc., then I’d be more clear that private medical data was not being given out to third parties at all, while allowing them to perform targeted marketing. It would still be a cause for concern for people who don’t want to be targeted for health info, but it would presumably be easier to swallow than direct sharing of personal health data. Also, the user is being told that the third parties are required not to use the Personal Data for other purposes, but that seems like an awfully large leap of faith.

The second thing I think any RealAge consumer should be aware of is the network of companies that RealAge belongs to, namely, the Hearst Corporation. Hearst seems to have successfully “leveraged” its various properties and relationships to grow the user base from 8 million as of its acquisition, to a reported 27 million today. One of those marketing channels is a spokesperson by the name of Dr. Mehmet Oz.

If you haven’t heard of Dr. Oz, he’s a popular visitor to Oprah’s widely-viewed talk show, and the Times attributes a notable portion of RealAge’s growth in the past two years to his frequent endorsements as a paid RealAge spokesperson and adviser. It might raise your eyebrow further to learn that the Hearst Corporation, who owns RealAge (along with three other medical marketing companies, First DataBank, Zynx Health, and Medscape – are they considered third parties in the privacy policy?), also owns O, The Oprah Magazine. It may be an innocent relationship, but this is a giant media conglomerate, and as I wrote in my old post, when Hearst acquired this company, the Hearst Company acquired the Personal Data of all of its users along with its technology.

Altogether, these are the issues that I wish that the mainstream media would focus on now, as 27 million people and counting are almost certainly not all aware of what’s being done with the answers to their surveys that they have submitted. First of all (and perhaps most importantly), the wording of the privacy policy should be changed to separate personal information like name, email address, zip code, etc., from private health information that RealAge collects from their web quizzes. Public awareness about RealAge’s targeted marketing practices should be raised through press coverage. Finally, the relationship between the Hearst Company, O, The Oprah Magazine, RealAge, and Dr. Oz should be disclosed.

As is my usual yearly tradition, here are a few collected thoughts inspired by the scheduled aspects of SXSW 2009. I hope you enjoy them!

From Educating Kids Through Gaming

Although this “Core Conversation” was a bit rambling and confrontational at times, there were common threads in the anecdotal examples which the participants discussed. The easiest way to inject an educational experience into a game is to cause the player to actively reflect on their time, through writing or discussion. With regards to games, it seemed like the most interesting approach to causing learning through games was to have a teacher guide their students through reflective thinking and writing about their experience. I remember book reports being rather useless and often cheated on by extreme use of Cliff Notes. Why not start having Game Reports instead? Many modern games do carry a good narrative, and there’s arguably much more incentive to experience the game world firsthand. It could create a good habit of reflection and critical thinking over a medium that the current generation is more likely to actively expose themselves to instead of books.

Also, it seemed that entertaining formats are natural incentives towards developing the basic skills that are needed to access them – skills which many children do not yet have. Arithmetic games need basic arithmetic knowledge. Adventure games typically need reading skills. Would it be so bad to provide “pure” adventure games to illiterate kids to encourage them to read?

In either case, I think the way forward will be for teachers to develop skills as shepherds that assist in causing learners to actively seek out value from their experiences. There are certainly risks associated with blindly tossing kids games to play, but with a guide present, children can be encouraged to make the most of their experiences, regardless of whether they are pure entertainment or traditional “edutainment.”

From What Can We Learn From Games

In this panel, I especially enjoyed the dialog between a questioner, Warren Spector, and Henry Jenkins near the end of their talk. The questioner wanted to know how to save the world through games, and WS and HJ had some unexpectedly blunt responses. Warren Spector responded that people should give up the idea that they are somehow going to change the world through content creation. He believed that the best thing you could do would be to give someone a world complex enough that they start developing system-level thinking. Henry Jenkins said that we should not expect games to produce social activists any more than we can point to games as the origin of psycho killers. It was a nice dose of reality just when I needed it.

From Journey to the Center of Design

Now available to view via Youtube! [Part 1, Part 2]

Jared Spool’s talk was one of the most entertaining this year, and it was also one of the most iconoclastic and thought-provoking. So much good stuff to talk about in this one, but i’ll try to keep it to two paragraphs.

He’s been doing research on the things that companies and groups do that produce great products, and he ended up splitting the “things that companies and groups do” into five groups – namely, tricks, techniques, process, methodology, and dogma, with each one reflecting more and more unwavering belief in formality, until you end up with a quasi-religious viewpoint. Surprisingly, the groups that did the best relied almost solely on tricks and techniques, with little tolerance for the rest. This fits with my experience, and to me, feels pretty validating, so I can’t be counted upon to provide much criticism for this finding.

He took this schema and then proceeded to knock on the door of User-Centered Design dogma, telling us that there’s never been any actual research proving that these practices actually work. Instead, Spool gave us some real-world practices that he’s found do actually work in providing a good user experience. Now, keep in mind that although these don’t seem like design activities, if they have an impact on the end result, then we should be aware of them. The first was having a shared group vision. Every team member should be able to independently describe what the user experience will be like in five years.The second was a feedback loop. Jared’s pH test for the presence of a feedback loop was this question: In the last six weeks, have you spent at least two hours watching someone use yours or a competitor’s design? The third was a good culture. The question for this was interesting, too: In the last 6 weeks, have you rewarded a team member for creating a major design failure? Jared’s talk was all about converting from blind faith in a design dogma to a results-based set of techniques with which to inform design.

From other stuff…

I enjoyed several other panels, and took some limited notes, but these were the main themes that I felt ran through my experience. The discipline and practice of working on the web is advancing, and although it is still young, the worlds of business and learning are cautiously reaching out to learn lessons from the work that people are creating on the web. We’re an industry that is growing more quickly than other industries of the past, less formalized and more communal in nature, but still an industry that is exploring what its practical reach and applications are. The developers and builders that go first are trying many things, making mistakes, and learning, and as a whole, we all benefit from that. It’s a promising time to be working on the web, and SXSW has again proven to be a useful reflection for the web community to glance at.

Man, it sure seems like Jon Stewart pulled Jim Cramer apart. It’s sad that Comedy Central is the only place I feel like I could have seen an interview this awkward and effective at informing the public. I’m concerned about the surprise to which Cramer professed about being lied to by CEO’s, though. Why would he be surprised about getting lied to, when he talked about manipulating prices through effective tomfoolery himself?

Seems like when you’re an insider with access, it’s okay to lie to people as long as they’re outsiders, but how can you expect such shifty people to tell the truth to each other?

I just helped a friend get through a devious issue with his PHP installation that I thought i’d blog about. There are other posts discussing problems related to having a $_POST array be completely empty although reading POST data directly via php://input wrappers works.

I used a similar test file to try and determine what was happening in the first place:

<?php
print "CONTENT_TYPE: " . $_SERVER['CONTENT_TYPE'] . "<BR />";
$data = file_get_contents('php://input');
print "DATA: <pre>";
var_dump($data);
var_dump($_POST);
print "</pre>";
?>
<form method="post">

    <input type="text" name="name" value="ok" />
    <input type="submit" name="submit" value="submit" />

</form>

If you submit the form and you see this as the result:

CONTENT_TYPE: application/x-www-form-urlencoded
DATA:

string(21) "name=ok&submit=submit"
array(0) {
}

… then that means that your browser is correctly submitting the right CONTENT_TYPE, and is also sending the browser POST data correctly. PHP is also seeing the right raw post data via the raw POST input wrapper, but the $_POST superglobal is totally empty.

What this eventually got tracked down to was an update in the php.ini that changed post_max_size from “8M” to “10MB”. Did you catch that? The use of “MB” instead of “M” was invalid, but instead of throwing an error on startup, PHP internally interpreted this as a “0″ since it was invalid. Since the post_max_size was effectively zero, nothing made it into the $_POST array.

Hope this helps anyone else debugging a similar issue! Oh, and hopefully it goes without saying, make sure you delete this test file as soon as you’re done, but it’s a giant XSS hole waiting to happen.

It just occurred to me that RRDs might be really well-suited to some general purpose roles as counters, statistics trackers, etc. for use in webapps. There are statistical analysis tools built in, so I could potentially see it as a way to (locally) rate limit API requests, track certain types of errors, etc. It’s also very fast, writes back to disk, and already plays very nicely with a variety of graphing and monitoring tools.

I’ll do some experimentation soon and report back.

I was shopping online for a replacement for my bluetooth wireless mouse for mac, and I read that there was an alternative mouse driver that might let me tweak the acceleration settings to something more acceptable.

I ended up downloading and trying Steer Mouse, and it’s actually pretty good at doing that. You can bring down the “Tracking” and bump up the “Sensitivity”, and it won’t be as wonky. Qualitatively, there’s also less noticeable lag between mouse movement and actual input (something that’s been driving me crazy as well). It may hold up better in heavy application use as well, time will tell.

However, one thing I wasn’t willing to live with was the default remapping of middle click to “Move to Close Button”. To change it to work with Firefox new tabs again, you can remap it to the Click action with a apple-key modifier. Seems like that works again. I’ll let it go through the trial period and see if I still like it after some more use.

My earlier post about the costs of microformats led to some interesting comments that I’d like to respond to at the top level here.

In this comment, André Luís brings up his personal experience with ease of publishing hAtom, and also points out that it may have just been my publishing platform which gave me difficulty. He also mentions that i’ve overlooked added value for power users.

Shira believes that many of the problems I listed will go away with maturity of the microformats project, and are present because of its work-in-progress status.

Yes, it’s true that there will be some cases where publishing microformats will be a technically straightforward matter (especially with say, hAtom or hCard). I don’t wish to say that it’s always technically difficult, but what I’d really love to get across is that there are not only potential technical landmines, but also policy and relationship complications that can result from such a move. In the case of my publishing platform, which was, in my case, in-house written code, there were extreme difficulties in attempting to contort our markup to something that would follow microformat structure expectations in hCalendar and hCard. Additionally, publishing microformats resulted in ongoing maintenance, policy, and backwards-compatibility costs that were not considered at the get-go. No matter what method you use to reach out to developers or power users, you will need to pay the price to keep those relationships healthy, and your data strong. Microformats don’t make your job any easier, because you’ve just intermingled the methods you use to serve your general user base and your developer base. Haphazardly publishing microformats while keeping your developer API users happy is going to cause grief for some users, and will likely be interpreted as sloppy behavior.

I certainly have avoided discussing any of the potential value of publishing microformats, as the title of my previous post may let on. I find that nearly all of the information available solely discusses the benefits without spending due time on the costs. Furthermore, the benefits of microformats (as well as the potentially reduced technical costs that Shira refers to) are, from a pragmatic standpoint, hypothetical. As a writer, it’s my hope that by spreading more complete information, people can make more educated decisions, and I feel that what’s missing in the hype about microformats is the discussion of costs. So, I apologize if I don’t spend much time on the benefits, but I really feel you can get more of that information elsewhere, as long as you remember that those benefits are typically projected onto a vision of the future where microformats are omnipresent. I’ll just say that I don’t see many benefits that aren’t already available to publishers who conform to the existing standards that microformats are based on.

So, what is the likelihood of a world in which microformats are an accepted standard in XHTML publishing? What are the chances that your real, tangible costs will eventually be paid back by real, tangible benefits? How would you begin to evaluate or consider that likelihood?

Typically, standards succeed when there is a massive market need that has not yet been met. However, there already exist viable, well-known methods of serving the needs of both browser users and developers, so I don’t see why these large publishers should invest in serving the “power users” market in between those two, when these power users needs are so close to that of developers. The power users are often served by the standard data formats underlying microformats anyway. Also, the developers often take developer platforms and build products specifically targeted towards power users! So, I don’t see a large, unmet need. To me, it just seems like an product aimed at serving a niche user base that is already being served pretty well.

Also, there is the fundamental matter of data quality that drives the growth of a standard. If your data is questionable, nobody will want it. Or, they’ll collect it and throw it away. If your data is valuable, consumers will go to great lengths to take in and use as much as your policy allows. This natural magnetism that high-value publishers have predates microformats, and explains well why there are dominant platform APIs that don’t seem to play by the rules of standardization. Sure, there is a high technical barrier to entry, but typically, these producers invest heavily on data quality, policy, AND their developer relationships. Relationships between publishers and consumers around valuable data is what drives your average successful Web 2.0 platform, not whether the publishing format is standardized. So, if you want to really predict adoption of microformats, pay attention to the data quality of published microformat data.

If you want to consider the potential of momentum to bring about a microformat-based future, you’ll pay attention to the spread of adoption of microformats, and the hype surrounding them. As mentioned above, I would encourage you to also frame your perspective according to the data quality of data made available via microformats, but that requires a lot of time and energy to really evaluate. The vision here is pretty blurry; I think that’s just the nature of hype. It’s easy to get excited about something before the costs come into the picture.

Essentially, by publishing microformats now, you are becoming an early adopter. These standards are not mature, which Shira is essentially pointing out. The costs may one day be lower, but by getting into the game now, you are not only placing a bet that many others will follow, but also they are subsidizing the costs of publishing for latecomers. I don’t think that the current tangible costs outweighs the current tangible benefits plus the hypothetical benefits multiplied by the likelihood of them coming to pass. But don’t take my word for it. I’d just encourage you to at least make an educated decision, and consider all of the costs I listed in my previous post before taking the plunge.