getluky.net

I really enjoyed this Dadhacker post. I got confused by all the other comments so far, so I’m going to ignore them and just mention the point that I think resonates with me.

Whenever I have some really hard problems in my mental heap, i probably spend a ratio of 4:1 subconscious processing to conscious processing. That’s kinda how I justify all the time I spend running around the web, doodling, woolgathering, drawing out ideas on paper, etc. Most days when I was forced to do this, I would pick one or two small bugs to fix so i’d at least have something to talk about at the daily meeting. It’s more or less impossible to tell everyone what my subconscious has been working on the last day. At the least, I always felt somewhat intellectually dishonest when trying to report activities that have their roots in some portion of creativity as well as mechanical work.

The main benefit of talking regularly is available if um, you talk regularly, but I think the main appeal of Scrumjax is to have stuff to report up a chain in a medium to large sized group of programmers that don’t really talk with each other naturally.

I’ve known my share of sysadmins who held some measure of belief that the more secrets they kept to themselves, the more secure their job was. It’s a belief held amongst a good portion of bad sysadmins, and unfortunately for them, it usually restricts their career more than it ever helps. Clinging desperately to power, a bad sysadmin like this will hoard information like Gollum to the Ring, and attempt to undermine anyone knowledgeable who comes near.

A disgruntled city computer engineer has virtually commandeered San Francisco’s new multimillion-dollar computer network, altering it to deny access to top administrators even as he sits in jail on $5 million bail, authorities said Monday. [...] Childs created a password that granted him exclusive access to the system, authorities said. He initially gave pass codes to police, but they didn’t work. When pressed, Childs refused to divulge the real code even when threatened with arrest, they said.

- SFGate

This guy has taken his particular brand of sysadmin paranoia all the way to jail. If I were a betting man, I’d say that there’s no way this kind of person was intelligent enough to truly do enough to lock everyone else out, and someone with some real skills will actually resolve the predicament for the city in no time.

How does someone like this get so far into a position of power?

Typically, it’s by becoming a false prophet of IT, filling the ears of superiors and colleagues with bullshit simply because they don’t know any better. If a guy like this gets in a company at the ground floor, how is a nontechnical person going to realize that the sysadmin who seems so smart is really stunting the growth of the firm? I’ve even seen cases where sysadmins just flat-out lie about their work and spray a fusillade of jargon anytime they’re questioned about it. This tends to fool the below-average CTO and technical boss as well.

The unfortunate truth is that once someone like this gets power, it’s essentially poison for the entire technical side of the company. To make things worse, the kind of person that would hire someone like this is your average non-technical entrepreneur, who is impressed by jargon and confidence and might not know the difference. So i’ll go over my personal spotting guide to good and bad sysadmins below.

In my consulting and work history, i’ve come across the Bad Sysadmin personality type more times than I’d like. The typical signs are refusal to document work, an excessive amount of jargon, hiding for large amounts of time in obscure projects, and a tendency to look at people as either allies or enemies. A huge indicator is a refusal to verify backups (usually it’s because they lied about making them). These people tend to cost more to a company than they’re worth for the rudimentary technical skills they can employ. Many of them are completely incompetent, and they use these techniques as a smokescreen to hide their deficiencies. Generally, the Bad Sysadmin will be incomprehensible to the average person, mostly because they have no true interest in sharing knowledge. Instead, they prefer to wield their limited knowledge as a weapon in order to appear infallible, instead of using their actual work to justify their existence.

A Good Sysadmin, on the other hand, will be happy to verify or give an update on their work for you. In fact, they’ll be thrilled that anyone’s actually interested in what they do at all. They’ll have strong attention to detail, and a desire to keep things well-organized and documented for their own reference and for others. They’ll typically try to avoid jargon, and will try to explain things in layman’s terms. They might be very proud of things that nobody else understands, but if someone genuinely is interested, they will make the effort to translate. If they don’t know how to do something, they’ll go Google it instead of trying to bullshit you. They won’t be overtly political, instead hoping for someone above to help them with their career. Generally, these sysadmins tend to be deeply involved with their work, and unfortunately are usually less visible in an organization than the noisy, political type. It’s not uncommon to find one Good Sysadmin quietly doing all the work in a group of Bad or just plain Incompetent Sysadmins.

I’ve seen too many of the bad ones, and too few of the good ones in my career. If you’re in a position of technical responsibility, please make sure to cultivate your own sniff test for Bad Sysadmins. It could save you from a long period of IT hell.

It’s a fairly frequent experience for everyone to come across an existing application of an idea they’ve had on the backburner for a while. However, it’s pretty rare to find one done so thoroughly and well that it’s just completely unnecessary to go and do it yourself.

FlashcardExchange does a lot of cool stuff that I wanted to build myself in a flashcard application. Among them are shortcut keys for navigating through decks of cards quickly, special self-testing systems such as incorrectly-answered only and even the Leitner file technique, and most importantly, the ability to share and search through existing sets of cards.

Really, the only thing in my vision that isn’t present on this or the other flashcard sites is the strong subdivision of flashcard sets by college or school class. However, I can see how making the site appeal more to generic collections of knowledge is particularly helpful in making it useful to the general public. Kudos to Culley Harrelson and FlashcardExchange for a job well done!

Except for the ridiculous domain name, everything else about The Fall was incredible. Thanks to Neil for the recommendation. I had an inkling that i’d love it when I saw the trailer; i’m a sucker for heavily stylized dream sequences. I kept getting more and more engrossed in the film as I was watching it, and I came away extremely impressed with the realization of this plot.

Trying not to spoil the movie is difficult; the basic premise is that a young girl with a broken arm befriends an injured man in a hospital, and he recounts an epic tale to her about several travelers throughout the film. Her imagination is rendered vividly with gorgeous, thrilling dream sequences, and reality and imagination blend in the way that reminds the audience of the vastly different world we inhabited as children.

The dream story is designed so colorfully and beautifully that it’s hard to describe. The most strikingly rendered details were those which were re-imagined from the viewpoint of the imagination of a child only partially familiar with the world of adults and the English language. In one scene, a group of secondary characters is said to have been found tortured and hung (oh yeah, this movie can be pretty dark). In the dream sequence, they are found roped together attached to the ceiling, hanging from the waist down as part of a macabre chandelier.

The darkness of this movie does not belie its inspiring effect on the audience that makes it through. This is a movie that makes you wonder if the creators have gone too far; placing a child in a role where she must deal with darkness in a way that is completely out of bounds for the moral preferences of modern America. Although this tack may have been responsible for its lack of widespread distribution, I believe the underlying message is surprising; that the imagination and love of a child is stronger than the darkness of growing up. And with that, I believe that I should give you my recommendation to go see this movie in the theaters before it is gone.

I’ve been having very odd intermittent problems with the Lift-Master garage door opener where I live. On the occasional hot afternoon, it has refused to close. It would click several times, and sometimes pop down and up. However, the problem invariably resolved itself after a half hour or so, which led me to believe that yet another deity of mischief had found his way to my dwelling. At first, I thought it was an overheating issue. I also found some clues in a forum, but they all sounded so complex. I read in the FAQ that the behavior is normal IF an obstruction is detected in the path of the sensors.

I looked into them, and had some trouble seeing one of the LED’s, because it was so bright, but sure enough, it looked fine. There were some spiderwebs near the sensors, so I swept those up, but no dice. Hmm… intermittent problem, behavior is similar to a sensor detecting an obstruction, and it only happened so far on hot summer afternoons.

A head-slapping realization came to me quickly, and I turned around to confirm the hunch. That LED that I couldn’t see because it was so bright happened to be in the direct path of the fiendishly aligned afternoon sun. At this precise time of day, that LED was just as bathed in direct sunlight as the sensor was. To test my theory, I stood outside the garage, blocked the sunlight with my silhouette, and hit the remote. The door closed with satisfaction. Or, more likely, I imagine that the door closed with contempt, as I walked away in satisfaction. It was kind of an Indiana Jones moment, realizing that fate has conspired to make an unusual alignment of cosmic rays interfere in your life in a very surreal way.

This post is a question addressed to the Rails folks who read this blog:

In the event that you were asked for help to work on a legacy Rails project from a while back (say, the pre-1.2 days), how would you even begin to find out how to port the thing to a modern version? What are the compelling reasons to do so? I’m not really a Rails or Ruby developer by trade, so basic advice is still appreciated.

If the decision is made to just stay with the legacy version to keep it quick, I guess there’s no real way to get any documentation from a specific version? If so, that seems to be a real drag on maintainability of real-world projects that get built and left alone for a while.

Disclaimer: I really know nothing about marketing. This is some simple woolgathering by a curious tyro.

For a while now, this post by Scott Ruthfield has had me pondering how a company makes it customers feel like its products are living, adapting, and growing things. Scott observed something in friends’ attitudes about the iPhone which he describes as an "implied upgrade". He and his friends who bought iPhones believed and expected that they would be upgraded as time went on, despite all experience to the contrary with other cell phones bought from other companies.

When I talked to others, they were sure of it too. They were waiting for the announcements of this or that new feature, and we all assumed that our phones would get it. Why did we all think our phones would be upgraded? Something “told” us that. Maybe it was the unconscious reaction to the iPod firmware update process inside iTunes; maybe it was a belief that Apple “got it”; maybe since this was more like a computer and computers get free upgrades… but they don’t. For some reason, we believed in an upgrade.

In light of his interesting observation, i’ve been trying to understand how a company like Apple can release a product without having to promise future upgrades, or detail all of its features exhaustively in advance, yet we expect that when we purchase the product, we’ll get everything we’ve seen in the commercials, and more. I would definitely characterize Apple’s products as somewhat mythical, in that its customers seem to propagate its perception in a way that goes beyond what the company actually states. I think that this is sometimes referred to as the “reality distortion field” around Apple.

Now, the reason I dredge this up is because i’m personally interested in seeing whether there is a viable alternative to focusing on promotional channels when considering how to market a new product. I’ve seen marketing plans that list conference blitzes, promotional clothing and accessories, mainstream media promotion, etc., all in attempt to show to upper management that someone, somewhere is thinking about marketing. I guess the rationale behind that is that it actually seems like marketing effort, but I always get the ugly feeling that concentrating on distribution of a message without worrying about the message is a giant waste of time.

Maybe it’s my lack of experience with this sort of thing, but i’d like to see marketing plans that spend all their time talking about the way in which you mention, discuss, present, or show off a product that treat it more like a mythological creature than a piece of static hardware or software. A good part of the equation is possibly to create well-designed, well-implemented products, and continually improve and upgrade them in a timely manner, and sometimes for free. But in addition to that, there may be a special way to treat product development and marketing in a company that asks consumers to plunk down a big wad of cash without worrying too much about the next model down the line.

This was annoying, but understandable. The tutorial “patch” to the Sun J2EE-1.4 tutorial, distributed by JBoss.org, is difficult to find [it's a zip inside a zip, here], and also needs some additions to its classpath to get a correct compile.

For the good of the internet, here’s what to do:

• Unzip the startguide40.zip package.
• Move jbossj2ee-src.zip into the j2eetutorial directory.
• Chdir into that dir.
• Unzip the jbossj2ee-src.zip.
• After it’s unzipped, edit jboss-build.properties, and put the location of your jboss install in jboss.home
• Go to examples/bank/.
• Edit jboss-build.xml, and modify the build.classpath to appear as follows:

   
    <path id="build.classpath">
        <path refid="client.classpath"/>
  <fileset dir="${jboss.server}/lib/">
            <include name="javax.servlet*.jar"/>
            <include name="jboss-j2ee.jar"/>
            <include name="servlet-api.jar"/>
            <include name="jsp-api.jar"/>
        </fileset>
    </path>

• Edit src/com/jboss/ebank/WSClient.java, and look for the import lines re: ServiceFactoryImpl and ServiceImpl. Modify them to look as follows:

    import org.jboss.ws.core.jaxrpc.client.ServiceFactoryImpl;
    import org.jboss.ws.core.jaxrpc.client.ServiceImpl;

After that, hopefully the instructions will work. My understanding is that versions of JBoss-4.2X are backportings of popular 5.X features, and they’ve probably ceased updating the Getting Started guides while work is prioritized on the 5.X branch. No harm done, I suppose.

Alley Insider does some back-of-the-napkin math and guesses that Google’s food budget is approximately $7.5k yearly per Googler. This is an interesting calculation to me, even if it’s a wild guess.

Google is one of the only places i’ve eaten at where the cafeteria at dinnertime literally feels busy. The difference in atmosphere between there and its competitors is astonishing. If you have a stint working as a programmer in Silicon Valley, do your best to visit other campuses and see how the culture and environments feel different from company to company. I do not believe that I have ever seen a dinner gathering anywhere near the magnitude of Google’s daily Mountain View cafeterias within a large tech company.

From anecdotal accounts i’ve received, it is not just that Googlers eat, then leave for home. Frequently, they’re eating as a short respite from a long workday, and go back to work, sometimes after meeting with family for dinner. I would fathom a guess that no other established, large tech company gets away with such long work hours from any of its employees as Google.

I don’t even think you’d need to invest in free meals around the clock to get this kind of behavior, either. You could still charge for lunch, but offer a free early (pre-8 or 9am) and late (post-7pm) dinner service in order to encourage people to stay past the regular hours, and still defray some of your lunchtime costs. If you even got a 15% boost in working, productive hours from your employees, I surely think that would make up for the handful of thousands of dollars worth of food you’re sending their way.

After the reaction of a few friends to my last post, I wanted to put up a separate post specifically about my current opinions regarding the ethics of taking public data, doing a lot of cleanup, then potentially charging for commercial use or download of the new (transformed) dataset. My planned project working with EPA data may not be the last time I do something related to data transformation, so i’m trying to understand the issues here.

Daniel Raffel pointed me to some pertinent info about WestLaw, one of the most well-known providers of information that originates in the public domain. WestLaw takes public legal information, incorporates it into its datastores, then charges a fee for legal professionals to use it to perform legal research. It provides proprietary interfaces, and also made several features that have apparently become indispensable to the legal profession, including a proprietary key-oriented classification of legal data.

Recently, Carl Malamud, who works for Resource.Public.Org, began a project with the aim of making all the primary sources of that legal data available on the Internet. It’s clear from his letter that he believes that making primary source data publicly available does not compete directly with the services and tools that WestLaw provides. However, it appears that WestLaw’s summary publication literature, such as the Federal Reporter, may be the only available published information from the primary public domain data.

Carl is essentially saying that since these summary documents may constitute information derived from data in the public domain, he will be attempting to extract the public domain data from the documents commercially produced on behalf of the government by WestLaw. I am not an information expert about this sort of thing, but it seems to me that a reasonable person would believe that if the original public domain data is ONLY made available in any useful form to a commercial vendor who then transforms it into literature, then the data which is public domain should not be covered under copyright for that commercial vendor. How you go about extracting that data is more of a fuzzy area, but presumably if an effort can be shown that bounds were respected, I think a reasonable person would say that reverse engineering is okay. Carl’s letter to WestLaw carries this type of reasoning down its natural path, and even suggests to them that they save everyone some time and just release the entire text of their publications, free to download.

Anyone who spends any time around primary source data knows that not all data is created equal. If your intent is to provide tools and services around data, there is a good amount of time and effort that must go into transforming primary source data into a useful format for some specific purpose. I believe that an appropriate action on West Publishing’s part is to go ahead and publish all historical cases in text format (not the Federal Reporter, etc., itself), and let anyone else who wishes to transform that data into a useful format go ahead with their project. This wouldn’t exactly satisfy Carl’s request, but it would meet the standard suggested earlier that the primary source data be available in at least one form.

It’s important to note that I’m not arguing that the final products (the Federal Reporter, etc.) need to be put into the public domain by WestLaw. By the standard suggested above, if the data’s available for the public domain, I don’t believe there is a strong ethical basis for compelling a commercial venture to take a risk and completely release all of their products for free. This would preserve some of the economies of scale of data manipulation, while righting the “wrong” that public domain source data is not available at all to the public. Put more simply, if a competitor wished to create a similar research product to WestLaw, the cost of transforming the data into a useful information repository with competitive features would still remain. Any proprietary content inside its publications still remains non-free, and presumably anyone who would buy them for their convenience would still do so. Carl’s threat is that he does have a very strong point if the only option to get the original primary data is through extraction from their commercial resource. If the decision makers at WestLaw decide to completely oppose his reverse-engineering, I think it would be a very politically difficult decision to defend, and could cause the government to step in and make the boundaries between public domain and private very clear. As it’s to WestLaw’s advantage to keep those boundaries murky, a compromise of providing just the case data in a text format seems the best solution for their interests.

Now, as to how this pertains to the situation i’ll be going into with regards to the EPA emissions datasets, all of that data is available via their website as more-or-less large CSV downloads. Each year has a somewhat different format. From what I recall from my earlier work on it, it’s kind of a pain to go through and clean up that source data, and it requires some knowledge about automotive industry emissions standards nationwide. Still, the original information is visible, even if it’s in a format that needs some work.

What i’m in the middle of doing is the standard drill - analyze the datasets, design a fairly acceptable standard schema to use as a blueprint for importing the data, then go set by set, programming transformations from the yearly data into the database. Then, an interface to perform queries can be created, as well as a set of useful services to offer on top of the transformed data. Long after all available data is imported, a maintainer might write new transformations yearly in order to keep the data current. This activity of doing work to transform data from a public domain resource into a different format is original work, and does take a lot of time and effort. Nearly all researchers deal with this sort of work on a regular basis.

What i’d like to suggest is that if data that is already in the public domain, and available on the internet is transformed into a version that is more useful to commercial ventures or professionals, that it is perfectly fine to charge a fee for access to tools or regular “dumps” of those transformed dataset. For one, the data’s already available, and it is not the original primary source data that is being offered for sale. The business product would be the combination of transforming the original data into a more useful form, and then offering either the transformed data directly, or simply services and tools on top of that data.

If anyone wanted to do that work, then re-open it up completely to the public domain, I believe that would be a gracious gesture, but I’m of the current opinion that it’s not ethically or morally necessary. Plenty of goodwill could be achieved by offering scholars, nonprofits, or individuals free access, and anyone who thought the cost is too high could attempt to achieve a lower cost by taking the original public domain sources and doing the work themselves.

That’s my current opinion about all this, but it does seem like there’s a lot of strong opinion out there, maybe not as long winded as me. Feel free to use the comments to let me know what you think.