Potential Differences

Inheritable Attributes in Rails

During a debugging session the other day, I found myself digging into how Rails implements inheritable attributes. These are an extension to the Class class and deeply intertwined in the inner workings of Rails. They’re part of ActiveSupport. I haven’t seen much on the net discussing them, and I don’t know whether that’s because few people have to deal with them directly, they’re mundane, or my search skills are failing. In any case, I decided to jot down my discoveries.

Inheritable attributes are kind of like class variables, except they’re copy-on-subclass. The subclass gets a copy of these from the superclass, but the subclass can further modify them independently. One major use of them in Rails is for STI (Single Table Inheritance) subclasses to inherit the superclass’ associations. The actual association declarations in the superclass (such as has_many, belongs_to, etc.) create an object stored in an inheritable attribute called reflections. Then once a subclass is created it gets a copy of this array of reflections and can further refine or extend them. Cool stuff.

But there’s a dark side. I mentioned above that this was found in a debugging session, not just a random exploration through the Rails source. The way these work depends on the Class#inherited method, which is called on a superclass whenever a subclass is defined and the subclass is passed as a parameter. This is where the actual copy-on-subclass logic occurs. If another monkey-patcher messes with the inherited method, weird things can happen, like a STI subclass losing access to the superclass’ associations.

However, once they’re understood, it’s not so mysterious. Not only that, but they’re a useful item to keep in your bag of tricks. There are analogs to attr_accessor and similar methods called class_inheritable_accessor as well as special methods for array and hash versions for those occasions when you want a subclass to have copy-on-subclass class variables.

Joined Relevance

I’ve made a career change and joined the great folks at Relevance. After doing a JRuby on Rails project at my last job, I knew that going back to Java would be like putting on a straitjacket. I’m spending this week at Relevance World Domination Headquarters learning how to shave yaks, and I expect to get back into blogging since I’ll be working on interesting technical challenges again.

On Craftsmanship

OK. I haven’t blogged in quite some time (for both of you that care), but I ran into something that really struck me today. Some of you may know that I have a hobby of woodworking, and not surprisingly, the idea of software as craftsmanship resonates with me. I’ve also found myself in an Architect role at work, and there’s some amount of cognitive dissonance I work to resolve. I found the text of a book from 1914 (now public domain) entitled Carpentry for Boys (I read “boys” as “apprentices”) and I was attracted to the chapter called “The Carpenter and the Architect.” Go ahead and read it. It’s not long, but full of metaphor to software craftsmanship and architecture.

I love the concluding career advice the author offers the apprentices: Know for yourself the reasons for things.

Eunuch Test Antipattern

I ran across a horrible anti-pattern today — JUnit test class methods called “doNotTest…” I think I’ll call them Eunuch tests. It’s pretty clear that either a) the team is not doing TDD because they can’t figure out how to test the code they already wrote or b) some “refactoring” wasn’t a refactoring but instead a code word for “dangerous unknown swath of changes to code”.

I don’t know whether to laugh or cry, but it’s Friday, so I choose to laugh today.

The Meaning of Agility in 20 Words or Less

Thanks to Alan Francis for this insight.

Agile methods are a response to Waterfall methods so that verification of decisions can happen sooner.

That’s brilliant, and quite useful when I find myself in a position to advocate agility to executives and business people.

Laws are Childish

Ah, serendipity. I’ll share the steps that brought me to the point of writing this entry. Last Friday, I noticed that two people I knew through entirely different channels and have worked with started getting into a blog debate. Now Jef (with one ‘f’) Newsom has outed me near the end of this entry. That, plus another point below, got me to write this up. On the other side of the debating table is Jeff (with two ‘f’s) Schneider. I wasn’t aware Jef (with one ‘f’) had a blog until Jeff (with two ‘f’s) had linked to it. I thought it was a good excuse to get back in touch with each of them and comment on what a small world it is.

Now I find out that Jeff (with two ‘f’s) has brought the bible into the discussion. Well, I just happened to hear something in this Sunday’s sermon that’s quite relevant. Ah, but before I explore that, for those who may not have followed the above links, I should mention just what it is that Jeff and Jef are ~~arguing~~ discussing with gusto: Service Oriented Architectures, their definition, rules, tenets, precepts, best practices, etc. (pick your term). Well, the sermon quote that struck me was “laws are childish”.

What does that mean? Well, that laws/rules are for those who are lacking knowledge and maturity, ie children. Those who have matured and internalized the fundamentals of a subject do not need them. This applies not only in the religious sense of how to live your life, but also to the technical sense of SOA. If you have an internalized understanding of good software design and distributed systems, you’re bound to come up with a SOA.

I mean no offense to those who are trying to nail down the fundamentals of what makes an architecture “service oriented”. For some it is a very handy way to organize their own knowledge. And there’s certainly a need to have laws for any developers on a project who have not reached the required maturity and experience levels, but in my opinion these should be more project specific rather than broad architectural category laws. Frankly, I’m not sure if my post adds anything to either side of the debate; I haven’t digested the details of the debate enough to really know.

I do like the slogan “laws are childish”. I’ll have to keep testing that to see where else it applies. I wonder if it explains why the people most suited to run a country are the ones least likely to want to run for office.

Alan Kay’s Turing Lecture

I didn’t take extensive notes during the talk, but here’s a transcription and elaboration of what I did take while it’s fresh on my mind. The title was First Courses in Computing Should be Child’s Play. Some of the notes I took down weren’t necessarily major points of his, but something that personally struck me.

He explored the dichotomy of the terms “Computer Science” and “Software Engineering” as well as the times the terms were invented. He discussed the relative youth of the computing field compared to other endeavors. Compared the builidng of the Egyptian pyramids, to Medieval cathedrals, to the Empire State Building. He also brought up the humorous sidebar of “physics envy”: topics that want to be taken seriously so they put ‘science’ in their name (eg Political Science, Library Science) — even though none of physics, chemistry, and biology have ‘science’ in their names.

Choice quote: “Our field is the next great 500 year goal after the printing press.”

He discussed personality types. I’m not sure where his terminology came from, but I mentally mapped it to Myers-Briggs style, since that’s what I’m familiar with. It takes about two-thirds of the “normal people” (sensing extroverts) to be on the cusp of an idea for it to be truly accepted. It explains everything from wearing baseball caps backwards to bare midriffs. He demonstrated this sort of meme propogation with a simulation comparable to forest fires. There’s about 30 years for good ideas to be accepted by the mainstream — gave unix as an example. He hopes the good Xerox Parc technologies will soon be widespread since their 30 years are almost up. He did say that MS Windows is a terrible caricature of Parc technologies. He compared the Windows UI to a nuclear power plant control panel. That wasn’t the only disparaging remark about MS, but the most blatant. Other digs were against C++ and Java.

He see’s a major problem in CS being our delight in complexity, and most is unnecessary. We need more of a joy of simplicity. A first college English course is not aimed at people who will become professional writers. Why is CS different?

He went into his inspiration as a grad student when learning about Ivan Sutherland’s Sketchpad application. He showed a video of Sketchpad, and discussed the hardware it ran on and how advanced it was for it’s time. Not only could Sketchpad do engineering drawings, but with its constraints system it could even simulate bridges, surpassing Ivan’s expectations. He named Ivan Sutherland the Isaac Newton of CS — first graphic UI, first object oriented system, and a two handed UI (as they should all be). One of Ivan Sutherland’s points in his dissertation was that he hopes the system is surpassed in the future.

Around the same time he learned of Simula. From these influences, and his background in molecular biology, the key insight for him was that messages are the central abstraction of OOP (which he pronounced like ‘oops’ minus the ’s’). From there he discussed a point made by one of his advisors (long name I couldn’t write quick enough) about learning styles. If you look at a two dimensional graph with Abilities along the X and Challenges along the Y, you find a state of “Flow” along the diagonal. We enjoy learning when the challenges we face are close to our abilities to overcome. Above and left of this diagonal is Anxiety. Below and to the right is Boredom. Safety features such as undo in UIs expand that flow stream into the anxiety area. Attention features such as an easy to use and engaging UI expand the flow stream down into the boredom area.

No surprise, but his entire presentation was running out of a Squeak image. At this point he went into more of a demo of Squeak, focusing on how to engage students in that flow stream with it. He drew a car, animated it with scripts, then tied the script to a drawn steering wheel. He mentioned squeakland.org for more information on Squeak in education. From that he segued into a demo of Croquet. In there he built a bridge, then relaxed the spring constant and added wind gusts to simulate the Tacoma Narrows bridge. That then morphed into a Canadian flag for the climax of the talk.

The bridge simulation tying back to Sketchpad was not lost on me, but was not a point he explicitly made. His final point was our responsibility to get kids interested in computing to further the development of our field.

It ended with a standing ovation — a great talk, well orchestrated, expertly delivered. I’m privileged to have gotten to see it in person. It definitely got me thinking about Squeak as a way to introduce my 4 year old homeschooled daughter to computing.

OOPSLA Day 3

No, you didn’t miss the day 1 and 2 entries. I didn’t write any. Rather than trying to go through each particular session I attend, I’d rather give a general overview at nearly the halfway point. It’s subject to revision by the end of the conference though.

It’s been my observation that OOPSLA is a great place to come to get a feel for where software development is going on a 5+ year horizon. I recall back at the 2000 OOPSLA in Minneapolis getting a hallway demo of AspectJ from Gregor Kiczales after a BoF. This year, AOP is much more prevalent and has progressed well beyond the typical logging example. I attended a tutorial by Ron Bodkin and Nick Lesiecki that focused on real enterprise applicability of Aspects. They had examples of using it to transition a system from EJB Entity Beans to Hibernate. I was intrigued by an example of adding JMX management to a system via Aspects. I do believe AOP is in the mainstream’s future.

Another topic I’m seeing in various places, tutorials, practitioner reports, etc., is an analysis of what Architecture really is. One tongue in cheek definition given by Douglas Schmidt at a tutorial about the Forgotten Craft of Software Architecture is that architects are those people whose development skills are too oudated to still be developers, but don’t look good enough in suits to be managers. I’m following the thought that architects act as a bit of an O/R mapping layer. Their main job is to handle the impedence mismatch between customers and developers. They work at the highest layer of abstraction, and delegate much of the intra-component design to experienced developers.

That brings up another point — I’ve heard the best definition of the difference between a component and an object here. A component can have more than one interface, while an object has only one. I like that.

More later. Alan Kay’s Turing lecture is this evening. There’s also a BoF about Michael Feather’s Working Effectively with Legacy Code book that I plan to attend. Then there’s also a “party” celebrating the 10th anniversary of the GoF Design Patterns book.

Organic Software Metaphors

It’s become somewhat fashionable to use organic metaphors in imagining what software will be like in the future. I most lately ran into it at Brian Marick’s Your body is a gross kludge post. It’s a thought provoking post, but ultimately I just can’t get behind organic arguments about software. Sure, I’ll admit that in the distant future software might become like living organisms, but that’s too far in the future to be useful. I can’t imagine how to get there from here. There has to be an intermediate stage.

The most striking difference in my mind is that software does not reproduce. The closest thing we have are ~~CASE~~ Model Driven Architecture (MDA) tools (aside: I wonder why someone picked an abbreviation that matches the Muscular Dystrophy Association). And I’m highly skeptical of those. Plus, they don’t truly reproduce. They don’t write other improved MDA tools. To be able do that independently, they’d be closely approaching strong AI. From reading Douglas Hofstadter and Roger Penrose, I highly doubt that strong AI is even possible. Suspending my disbelief for a moment, I do think the kludgy designs Brian talks about would be a remarkable achievement for organic software — after all that is how most human written software is!

To be fair though, I do think there are some features of organic systems we would do well to better incorporate into software, such as resiliency. Recent interests have gotten me thinking about instrumentation and monitoring. Monitoring allows us to know what’s going on inside our systems, and instrumentation allows us to tweak parameters on running systems. The real power comes when you programmatically tie the monitoring back into the instrumentation to create a feedback loop for the system to modify itself in response to the situation at hand. I did this in one particular instance a year ago to great effect, and I’m looking into more general practices. However, contrary to the kludgy designs of organic systems, mine was a well encapsulated, reused, JMX component, configured and installed independently of the business system.

Anachronism

I’ve got their weirdest sense of vuja de — the feeling that nothing like this has ever happened to anyone ever before. I’m surfing the Internet looking for information about a current power outage. I’m doing it via the battery of my Powerbook. Oh, yeah, and I decided to blog about it. Am I the first to blog to the light of a kerosene lamp? :-)

The Dallas area got some nasty thunderstorms this evening. We saw lightning and heard thunder simultaneously (never a good sign) and then the power went out. Five minutes later we heard fire engine sirens. There’s a fire station about 5 blocks away, but it sounded like they were coming into the area. I hope everyone is alright.

Embarrassing Cost of Poor Design

Design of everyday things complaint of the day: within a building, the men and women’s restrooms should be in the same relative position (i.e. men always on the right, or always on the left). I suspect most people will agree with this on priciple, but do you want to know how bad it bit me?

The restooms nearest me (with men on the right) were closed by the janitor for cleaning, so I went to another set on the same floor and went in the right hand door. The restrooms in my office building are divided with the sinks by the door and the stalls in the back behind a divider. Luckily I just had to wash some ink off my hands, which I was doing when I heard a flush and then a woman came around the divider to the sink next to me. She gave me a funny look and it took me about a half a second to figure out what had happened, then I profusely apologized and left.

Finished laughing yet? :-)

Please consider the everyday folks who are going to have to live with the design decisions you make in software. Please, please, be consistent.

Source Control Tension

After reading Glenn’s blog on Six of One, a Half Dozen of the Other I started thinking about source control systems as another example of these opposing tensions. As Glenn notes, this is tied into Martin Fowler’s directing vs. enabling attitudes.

On the one hand you’ve got the more traditional style of source control such as PVCS that uses an exclusive check-out model. Only one person is allowed to modify a file at once. It’s a safe pessimistic locking model (directing). On the other hand there’s systems such as CVS that allow multiple users to edit the same file at the same time. It’s an optimistic locking model (enabling). Yes, there can be merge problems. That alone keeps some people from using them.

I’ve used both forms of source control. Merge errors can and do happen with optimistic locking, but they’re not usually a problem. You’re notified right away and the fix is usually obvious. On the other hand, when I’ve used pessimistic locking systems, I’m often having to go talk to other developers to ask when they can check a file in because I also need to make an edit to it. In the meantime my flow is broken and I have a choice to work on something else until they can check in, or work around the source control system to make edits on a local version. Then I’ll have to remember what I changed and re-create it later when I can check the file out.

This draws a nice parallel to the perennial favorite blogging topic of static vs. dynamic typing. Static typing is like an exclusive locking source control system. It is safer, but it can get in your way — how frequently is open to debate, but I say it often does. Dynamic typing plus unit tests is like an optimistic locking source control system. It doesn’t get in your way and you can go faster, but it will sometimes have problems that your unit tests should catch right away and allow you to fix.

This brings into question Glenn’s whole premise that neither side is really right — it’s just about tradeoffs. I feel much more comfortable saying that optimistic locking in a source control system is categorically better than pessimistic locking than I am saying that dynamic typing is categorically better than static typing, but why? Is it a limit in the metaphor or in the industry’s experience with dynamic typing? Also, is there a corelation between fans of static typing and fans of pessimistic locking?

Automate Plist Verification

Mac OS X can have some hard to diagnose problems if preference files become corrupted. Based on an initial idea on Macintouch, I read a couple of man pages and came up with an /etc/weekly.local file that looks like this.

#!/bin/sh

echo "Checking preference plist integrity"
find /Users/gvaughn/Library -name '*.plist' -print0 | xargs -0 /usr/bin/plutil -s

In detail, we’re finding all the *.plist files under your home directory’s Library subdirectory (substitute your username for ‘gvaughn’). The 0 on the print and xargs uses a NULL separator so we can handle files with spaces in their name. Then all the found files are passed to plutil for syntax checking. The -s option says to be quiet unless there’s a problem.

I hope someone finds this useful. If you find ways to beef it up, please let me know.

Re: Best Practice vs. Useful Practice

Gee, it’s been a long time since my last entry. For both of my readers, I guess now you know I haven’t fallen off the end of the earth. I’ve been buried at work and with personal projects, and I’m afraid writing has fallen by the wayside for the time being. I hope things are slowing down enough that I can get back into it. I’ve got a backlog of ideas for posts. I had some time and was really struck by some synchronicity today.

Esther Derby’s great observation on Best Practice vs. Useful Practice reminds me of an experience I had back in grad school. I was studying physics and taking a course in Solid State physics. There was also an Electrical Enginnering student taking the class as an elective. He was in our study group. I think it was a homework assignment we were discussing and he asked for help. I began to discuss the physics principles involved, but he quickly interrupted me and said that he didn’t care about that, he just wanted to know what formula to use. My jaw dropped as I wondered why in the world he was taking this physics course that wasn’t a requirement.

In light of Esther’s discussion, he wanted the “best” practice. I was offering a useful one. I wonder if there’s a correlation between an engineering and scientific mindset in approaching programming.

Naked Objects Exposed

I think I now have a handle on what makes Naked Objects interesting. I had previously perused the site and came to the preliminary conclusion that it’s just a generic UI framework. Last night at the DFWPP (Dallas-Ft.Worth Pragmatic Practitioners) meeting I got to hear Dave Thomas talk about Naked Objects (and believe me, it is a topic rich with double entendres). It helps me to think of Naked Objects in a parallel manner to TDD (Test Driven Development).

On the surface TDD looks like it is primarily a means of testing. The ‘aha’ moment happens when you invert your thinking. TDD is primarily a design tool, and secondarily a testing method. It ends up that testable code is more cohesive and loosely coupled, and that’s the major benefit. My ‘aha’ moment with Naked Objects happened when I began to see it as primarily a design and requirements gathering tool, and secondarily a UI framework. Coding in Naked Objects style promotes minimal temporal coupling (the GUI is modeless), complete business rules in the business objects (none of it hanging in the GUI layer), and explicit relationships between business objects (so the GUI knows what can be dragged and dropped onto what).

So, just as TDD taught us that testable code is well designed code, so now Naked Objects are teaching us that naked (or maybe strippable?) code is well designed code. Also in a parallel vein, we know that the unit tests from TDD are rarely going to be a sufficient amount of testing, so also the GUI of Naked Objects is rarely going to be a sufficient GUI. Just as the more traditional functional, system, and user acceptance tests can be added to a TDD-built system, so too can a more traditional menued, or script-driven GUI be added to a Naked Objects-built system.

Dave will be giving that Naked Objects talk (and also one on Coupling! I warned you about the double entendres) at the Atlanta Java Software Symposium this coming weekend. Come if you can! <shameless-plug>You can even hear me speak on JMX, IO Performance Tuning, and Expressive Code there too.</shameless-plug>

Note: This is crossposted to my java.net weblog.

NoFluffJustStuff

I had a great time at last weekend’s Dallas NoFluffJustStuff symposium. I was so busy putting together the presentations, I didn’t have a chance to mention it ahead of time here. I’m also slated for the Atlanta and Denver instances. The three talks in my current set are on Expressive Code, a JMX case study, and I/O performance tuning.

One of the best things about these conferences are the hallway discussions. All the speakers are very accessible both in and out of the sessions. I almost gave away Stuart Halloway’s classloading talk (good thing he shut me up quickly :-) and Ted Neward had some great input during the QA of my JMX talk when a question was asked about some overlap between the things JMX and Aspects are good at.

Kirk Pepperdine has talked me into preparing an article for an upcoming issue of the Java Performance Tuning newsletter based on a performance hack I talked about in that talk. I’ll put a note here once it’s published, but you could always go get yourself subscribed in the meantime.

Another Software Development Analogy

There’s much discussion of different analogies for software development. Some like to think of it as engineering such as building a bridge. Others like to compare it to building construction. The Pragmatic Programmers compare it to gardening. But none of those really resonate with me.

James Robertson compares it to movie production. Now that’s an interesting concept. We always hear about movies missing time/budget projections. Producers are the financial sponsors. The director has creative control. Various tradespeople (craftsmen?) do the acting, camera, sound, editing, special effects, etc. Most of those are organized around unions or guilds. Each movie is unique, but may be formulaic, and the cost of copies is miniscule compared to the cost of the development in the first place. I think this analogy has legs.

It gets me thinking about a guild or union structure for developers. Could it work? A few years ago during the dot-com boom it certainly wouldn’t, but I wonder if it would these days. Fred Brooks talked about a 20X difference in productivity between developers in The Mythical Man Month and Pete McBreen talked about compensating developers based upon that in Software Craftsmanship. You see an even broader range among actors, yet they are all members of the Screen Actors Guild. It sure would be nice to be a Mel Gibson equivalent in a Software Developers Guild :-)

Generalist vs. Specialist

I’m playing some blog catchup right now, but there’s another entry of Brian Marrick’s I want to comment on: Technology-facing Product Critiques . In it he explores the need for specialists even in an Agile project. I’m not certain I can completely agree.

I submit myself as a counter-example to his argument. I’m a developer, and Agile-minded. I’m interested in security, UI design, and even testing (why I started reading his blog in the first place :-) I guess that makes me a jack of all trades and a master of none. While I may not feel qualified to be the sole person in charge of security, UI design or testing on a project, I do think that with a group of developers that are as much of a generalist as I am, we could collectively get the job done.

I suppose pragmatically there’s the question of how many projects have a group of generalists this broad. Maybe I’m just a singular oddity. I’ve been accused of that before.

Hardest Part of Software Development

Over time I’ve come to realization that the hardest part about software development is human psychology and sociology. That was the farthest thing from my mind when I was a bright-eyed and bushy-tailed newbie professional programmer (mumble) years ago. Maybe I’m just getting old, or maybe this is specific to software development inside enterprises, where the majority of my experience is.

In so many cases it seems the most frustrating parts are getting people to care — mostly to get developers to care about honing their craft and their own skills. But I could take it in many other directions — different sets of people, different things they should care about. Anyway, this puts me squarely behind Brian Marick’s position paper to the NSF on the science of design. I agree with his stance completely.

I hope this isn’t pushing me toward the path of … management !

JMX and Jini

About a month ago, I wrote about using JMX to manage JavaSpaces and Jini services. Now I’ve found an article by Frank Jennings doing it inside out. He’s discovering a JMX server via Jini. I’m glad to see people combining Jini with J2EE.

I wonder…. if you combined both approaches, would that create a tear in the fabric of spacetime?

So, What Are Web Services?

Werner Vogels tells us that Web Services are not Distributed Objects. I was going to comment on his blog, but as I started writing, I began a spiral of asking more and more fundamental questions and thought it would be more appropriate here.

I’m still trying to figure out what web services are. OK, so they’re not distributed objects in general, but how are they different from a well-designed, stateless, message-based distributed object system that uses XML as a serialization mechanism? Are they just a means of talking about good distributed object architecture in a different context?

To address some of Werner’s points:

Just because distributed objects enable stateful interactions, doesn’t mean that they are inseparable concepts. I prefer stateless architectures whenever possible, and the experience people have with web applications have taught them how to work around a stateless protocol (HTTP) to add state where needed (and even where it’s not needed in many cases).
Werner’s insistence on the transport agnostic aspect reminds me of Java servlets. They are not technically tied to HTTP either. I’ve worked on a servlet system based around SMTP. However, for all practical purposes servlets and HTTP are inseparable in 99.9% of techologists’ minds. Will it be different for web services?
In my mind the service nature is at the architectural level. Distributed objects will be the predominant implementation technology. But OO folks have been doing distributed service based technologies (Jini) and message oriented systems for years. What sort of productivity gains can web services supply above these?
I am glad to hear about the reduction in status of XML-RPC. It’s just a buzzword-induced performance throttle. Even sticking to the message model, how are web services fundamentally different from a stateless distributed object with a single processDocument method that accepts a parameter that is a rich data structure?
It’s still not clear to me that web services are anything other than a subset of functionality that distributed objects offer, assuming the same architectural considerations. Their only distinction is that via XML they get cheap language interoperability at the expense of performance, which is not the appropriate tradeoff in all situations.

I’m not trying to be harsh. I really do want to understand what sort of problems web services help me solve that I can’t already do as easily with existing tools.

Breaking Down Test Driven Development

Brian Marick analyzes the different uses of unit tests. It’s something I’ve been thinking about too. On one project when I was refactoring TDD written code, I found myself deleting quite a few unit tests. Keeping them all up to date was a source of inertia that was slowing down progress after we had higher, component-level, tests in place.

Brian’s using a number of terms that may not be familiar, but it’s a valuable exercise. The term ‘test’ has so many overloaded usages, especially when testers and developers try to communicate. He’s calling unit tests ‘checked examples’ and ‘change detectors’ to elucidate the two major purposes of TDD. Unit tests that help drive design and development are ‘checked examples’. The built up test suite that serve a regression function he’s calling ‘change detectors’.

This distinction has helped me to understand my instinct in my situation. The unit tests had already served their function as ‘checked examples’ that drove the design, and their function as ‘change detectors’ were redundant. Now I can feel less guilty about hitting that delete key.

Fundamental Problem with Extreme Programming

Matt Stephens gives some strong arguments against Extreme Programming. It’s a long article, but worth a read if there’s just something about XP that doesn’t feel right to you. Now, I think he does go off into rant mode in places (and he’s also trying to sell you something), but the article was helpful to me to really put my finger on the fundamental problem with XP.

XP requires too much of an investment from the business. Even aside from the obvious on-site customer that XP requires (and Mr. Stephens makes a good case that the best you’ll get is a junior person) there’s the regular unfinished (and unpolished?) versions of the software they’re expected to use and give feedback about in the Planning Game every few weeks. There’s also the ideal of the business users working to develop the acceptance tests.

It’s like watching sausage being made. Business users are not techies, and generally don’t want to become techies. They want the software, and they want it with a minimum of their effort. The overall development efficiency that XP is built around actually decreases the business users’ efficiency in doing their own job during the development. They find it tiring to think like a techie and answer all the detailed questions about how various requirements interact and handling all sorts of ‘what if’ scenarios.

Please, don’t misunderstand me, gentle reader. I do respect XP. It has served as the spark that got people (me included) thinking more about Agile methods in general (though the ideas go back much further than that). I’ve written in more detail about why I choose to associate myself with the Agile movement rather than XP before. The gist of which is best explained by comparing XP to Free Software (FS) and Agility to Open Source Software (OSS). XP/FS adhere to uncompromising purity of their ideal. Agility/OSS agree with those ideals, but realize compromises must be made in the name of practicality. XP/FS have actually become special cases of the broader Agility/OSS categories.

I guess I’m just a practical sort of guy.

Doomsday Algorithm

Ever wanted to calculate the day of the week of any date in your head? Me neither, but it’s fun to read about this memorizable algorithm, albeit with a rather gloomy name.

The “do do” Guy

Dave Thomas graciously accepted my invitation to a book discussion of The Pragmatic Programmer today. I hope we at least provided some entertainment for him. At one point there was a discussion about whether a peculiar behavior was truly a bug. Dave asked “and the business people say…” with the same cadence as the Lou Reed song, “Walk on the Wild Side”. Mike Schaefer recognized this and immediately answered with “doo do doo, doo, doot do doo…” Everyone in the room nearly doubled over in laughter. I think Dave will forever remember him as “the do do guy” :-)

The Ideal Programmer

Bruce Eckel has some great stuff about programming. I suppose I think it’s great because it mirrors some similar thoughts I’ve had for a while. It’s nice to be validated by someone well known and respected :-)

Programming is about communication between humans.

This mirrors the opening quotes for an Expressive Code talk I have. I gave it at the local Java Users Group back in February and I’ll be taking it on the road for a few dates of the NoFluffJustStuff conferences in a couple of months.

Programming is not about racing to throw together a solution. It’s about changeable systems.

This I would love to agree completely with, but I can’t. To draw in the concept of craftsmanship, let’s think about a wood furniture craftsman. He lovingly makes heirloom quality furniture by hand. The furniture is beautiful, resilient, and adaptable to changes in decor over the years. You expect to hand it down to your grandchildren one day. Contrast that with particle board furniture sold by the local discount warehouse. It’s much less expensive, available right now, and functional for a while — until the particle boards start sagging, and the adhesive veneer starts peeling, etc.

If you’re a college student who expects furniture to receive abuse, only expects it to last only a few years, needs it right now, and has limited income, then that particle board bookshelf is probably your best bet. However, it also makes sense to you if you know next to nothing about furniture, you’re caught up by the pretty pictures of it in a catalog, you’re cheap, and you’re not prone to thinking in the long term (oh, and you’re an executive at a large company).

Bruce Eckel is thinking of that second case by saying that programming is not a sprint to finish in that the “executive” needs an eduction, but I think that sometimes the “college student” case applies, when a quick solution thrown together best fits the situation. That being said, I’d personally rather be crafting than hacking (in the bad sense), but we need to recognize that it’s not always appropriate. Programming in the real world is not always craftsmanship.

The “Love It or Leave It” section reminds me of an experiment I’ve been running in the self-selecting nature of programmers of like mind being drawn together. I guess the results are conclusive enough that I can let the cat out of the bag now. In my first weblogs.java.net post I talked about people needing to take some pride in their craft and keep themselves informed of what going on, even for things that aren’t directly relevant to their task at hand. In it I referenced an incident here at work where some people could use a bit more broad of an exposure to Java technologies. That weblog is much higher profile than mine here, so I was wondering if any of those people would comment to me about it. They haven’t.

Distinctions of an Agile Developer

Warning: risk of stereotypes and psychobabble ahead

I’ve been experiencing resistance to Agile methods lately and I’m trying to figure out why. A few other bloggers have begun discussing this and I hope I can add something useful to the conversation. Brian Marick touches on several issues relating personality type/thinking style to testers and Agile methods in general. He references Bret Pettichord’s “Testers and Developers Think Differently” which initially made me wonder if I’d fit better as a tester than as a developer. Brian also ties it in with Alistair Cockburn’s investigation of what sort of personality a business coach needs.

Now, I’ve long been fascinated with the Myers-Briggs style of personality assessments since first being exposed in college (interestingly, it was a Physics class, not Psychology). I’m INTJ with INTP leanings (now you can all pigeonhole me :-) This personality type (or really the more general ‘NT’ — the middle two initials) is a small percentage of the general community, but a large percentage of the software development community, so I believe that the differences between Agile and un-Agile (for lack of a better, non-perjorative term) developers must be a second order effect.

These ‘NT’ (Rational) types learn to trust in their intelligence, often basing their own self-image around it, which is usually compounded by their awkwardness in social situations and the corresponding belittling during adolescence. Though we don’t like to admit it, we ‘rationals’ do carry emotional baggage as Chrisitan Sepulveda explores. As a friend, Paul Holser, put it “we all want to be Spock, but are really McCoy underneath all that.”

Agile development requires a large amount of humility. We have to trust that practices such as TDD (Test Driven Development) might lead to better software than what we could develop purely via our own creative processes. And if it doesn’t then the problem might be us rather than the method. To someone who’s self-image is largely centered on their own intelligence, this hits mighty close to home and evokes emotional defenses. Ironic in such rationally centered people. My working model is that it takes maturity to reach humility, but I’m not particularly satisfied with that because I can envision the argument “you don’t agree with me because you’re immature and I’m not, so nyah.” I also wonder if there’s a faith aspect too because it can cause one to re-center their self image on something beyond themself thereby avoiding that emotional threat, but that’s not something we ‘rationals’ talk about much, so it’s tough to tell.

One of the keys of the Agile Manifesto is valuing “individuals and interactions over processes and tools”. It’s people oriented. That’s what has been leading me down this path of trying to understand people and their resistance. The manifesto makes it sound so common-sensical, so adaptable, and so applicable in all circumstances, but I’m starting to discover that valuing individuals includes knowing when to stop pushing. So at what point to I stop trying to convince people that Agile development would improve their situation and simply look for a new situation for myself? Is Agile Methodology just a “birds of a feather flock together” sort of mindset? Anybody know of some healthy flocks?

A New Name for TDD

Brian Marick writes Things Agilists Want to be True. It’s a great start, but I have the gut feeling there’s more to it. I can’t point to any specifics right now, but the synergy is quite interesting. I just purchased the Scrum book he refers to last week, and I also just told a group of testers this morning that I predict that TDD will be renamed within the next year. I don’t know how many times I’ve had to explain to people that the validation aspect of TDD is secondary to the way it drives the design of the code (including this morning right after reading Brian’s blog).

So maybe it’s time to start brainstorming other terms. I’ve had a few come to mind:

Usage Driven Development
Client Driven Development
Outside-In Development

I think Client Driven is my preference at this point, but that may be a result of my affection for plays on words. I’m not sure how common of a usage it is for a client of a class to be another class that calls it. It also brings up the issue of it being a human client of the software who defines the requirements that get refined into unit tests.

I’m also going to be observing the team dynamics as our department takes on more of an agile philosphy. Without much authority behind me this morning, I was trying to explain to the testers that developers are encroaching on their ‘territory’, and they should be responding in kind by expanding their test automation and styles of testing.

Software Engineering Certification

Cem Kaner writes about Software Engineering’s Body of Knowledge. Martin Fowler has some good commentary. The basic point is that these practices could be codified into a certification program in which certified software engineers could be sued for malpractice for not following them. The thing that is a problem for Agile thinkers is that SWEBOK is very waterfall-ish. However, the document is open for public review. So feel free to give it a bad one.

Software engineering certification will come one day, so we should take some time to think about how to make it work. There are good arguments in favor of certification and even the sort of document-heavy, big-design-up-front style SWEBOK current promotes. The best explanation I’ve read was from Alistair Cockburn about the failure modes of a piece of software. In the extreme case when software failure could result in loss of life, a heavy process with all of its checks and balances makes good sense.

If this is the sort of process that software engineering certification requires, then we must make sure that the market is free to hire non-certified software developers depending on their needs. For truly critical software, certified engineers can be hired, but most software doesn’t need this level of rigor. For these projects companies should be free to give up the ability to sue for malpractice and hire non-certified developers. Agile developers will be cheaper and/or faster in this sort of market.

This might even be the key to publicizing the fact that different software projects require different approaches. One size does not fit all. Sometimes you’ve got to make it come down to dollars and cents for business to take notice.

Props to JavaMUG

The Dallas area Java users group, of which I’m a member, was named one of the top 25 in the world. One of the upsides is that it means that we’ll have access to alpha and beta versions of Sun products such as Rave. I don’t know details, and I’m not even sure they’ve been ironed out yet. If it’s going to be allowable, I’ll write up my experiences here in upcoming months.

More Rsync Fun

As I wrote before, I’ve been using rsync to update my blog entries between my laptop and web host. I finally got around to debugging a problem I had with my previous shell script that allowed me to download a referrer.log at the same time I uploaded my blog entries. It ended up that on second and subsequent uses, the symlink to referrer.log on the webhost got overwritten with the actual file that was on my laptop. So I added an exclusion of *.log going from my laptop to the webhost. Here’s the updated script:


#! /bin/sh
rsync -Cavz -e ssh --delete --exclude="*.log" \
/Users/gvaughn/Sites/gigavolt.net/blosxom/ \
userid@webhost.com:/WebSite/gigavolt/blosxom/

rsync -urptgovLz -e ssh --copy-unsafe-links \
userid@webhost.com:/WebSite/gigavolt/blosxom/ \
/Users/gvaughn/Sites/gigavolt.net/blosxom/

In general, if you’re not working directly on your web server, you owe it to yourself to learn about rsync. It’s so much better than ftp for those sorts of synchronizations you’ll want to do on a regular basis.

Raising the Bar?

I started asking myself some new questions today. I’m going to try to capture the thought process that led me to it here, so bear with me. I’ll get to the point eventually.

In response to a question today, I said that once someone has truly embraced unit testing on a decent sized project, they’d be forced to either re-invent mock objects if they hadn’t been exposed to the concept, or give up unit testing as too difficult. I’ve found myself using mock objects regularly in my unit tests. On the first project I used unit tests, I’d independantly decided to use lightweight inner classes within my test classes for the collaborators of the test class. I’m enough of a researcher that I soon learned and read about mock objects. I’ve heard similar stories from others. The point being that mock objects are a natural consequence of TDD (Test Driven Development) or even unit testing after writing the subject code.

The reason mock objects are a natural consequence is that using unit testing as a design tool leads us to loosely coupled code, just to make unit testing bearable. This has led me to rely less on static methods including the Singleton pattern. To make up for this, I find myself using JNDI and JMX more, as they provide a way to find collaborators without being tightly coupled. I’ve even written trivial partial implementations of each for use during unit testing. For actual deployment I’ve been using JBoss lately.

This week I’ve written a small system using JMX, MessageDrivenBeans, and utility classes stored in the JNDI tree. Each component has a good suite of unit tests and I have high confidence in the actual code. However, the next step of assembling these components into a useful application has been troublesome.

It was a good thing I was already writing a JMX MBean, because that’s apparently the only way to bind objects into the JNDI tree. Then it seems that the objects bound must be Serializable, even in the ‘java’ context. Then there were deployment order issues about whether I had to manually create an application specific sub-context, or whether the DataSource config file ran first and created the sub-context for me. Then I had to make sure I had all the JNDI mappings correct. The MessageDrivenBean looks up simple names, that are in a context specific to the bean, and the deployment descriptor maps those to where the real object is bound — this causes more order dependencies.

Now to my question. Is this really making our lives any simpler as developers? Are we truly raising the bar? I feel that I’m having to learn an additional programming language that is a mixture of Java, XML, properties, and general app server understanding in order to assemble my components. I’m worried that even though unit tests gave me confidence in each component in isolation, I have little confidence is the entire application assembly process.

The J2EE docs speak of those different roles of bean developer, deployer, assembler, administrator, etc. but in practice I’ve never seen those roles taken by different people. The developers are called upon to do them all. I’ve wondered for a while whether software development would reach the point where there’s a distinction between developers that write components, and people (developers, technical business analysts, users) who wire the components together into applications. Is the widespread use of unit testing going to bring us closer to this? And then the biggie: is it a desirable outcome?

A New Job for John Cleese

The Ministry of Silly Walks is my favorite Monty Python sketch. I would try to lookup a link to a transcript, but it’s such visual humor that the transcript would be useless to anyone who hasn’t seen it. Well, as an example of life imitating humor, the Pentagon is researching how to identify people by analyzing the way they walk. Note the irony of finding this story about the US government at a British news source.

Well, maybe if John Cleese did get involved, Total Information Awareness could have some entertainment value.

Forgiveness or Permission

Dan Steinberg got me thinking along the lines of the old saying “it’s easier to ask for forgiveness than permission” with a side remark in his article on iCal. What he actually wrote was “With exceptions and strong typing, Java makes you say ‘please’ while Perl makes you say ‘sorry’” A static typed language makes you ask ‘please’ ahead of time, while a dynamic language makes you say ‘sorry’ by waiting until runtime to find some errors. But, of course, a good unit test suite reduces that significantly.

My last entry on this topic discussed the idea of using the ‘safe’ aspects of static typing for writing robust system facilities, while using the ‘easy’ aspects of dynamic typing for business logic that will probably need to change over time. Ted Neward recently wrote about using static and dynamic languages for these different purposes.

In my opinion, one of the generating functions for the whole Agile movement is that requirements from non-technical people are unavoidably vague. These are the people we get the business logic requirements from. However, it is typically other developers that define what the system facilities must do. Recently I wrote a JMX MBean that had to handle retrying communication with an external web service that occasionally goes down. It basically moves JMS Messages to and from a temporary queue with the use of a timer. The MessageDrivenBean that actually performs the communication and business logic just has to rollback the transaction that includes accepting receipt of the message when it can’t communicate with the external service. The MBean system facility takes care of calling it again later.

In writing this MBean, the static typing of Java helped make it robust. The checked exceptions forced me to seriously consider different failure modes and how to handle them without losing any messages from the queue. This is a system facility whose requirements were ultimately driven by developers. Once the requirements were understood they never changed. I propose this is the common behavior for this sort of system facility, and that it is precisely because there is not the typical impedance mismatch of communication between technical and non-technical folks. But for business logic, this mismatch exists, and dynamic typing makes the changes much less painful

I’m on a roll here, so I’ll bring up one more related topic that has come up in recent conversations with coworkers. We’ve been discussing exception handling. Most the developers have been doing Java lately, but there’s a new project being written in .NET, which only has unchecked exceptions. Java gives you the choice of either checked or unchecked exceptions. My position is that checked exceptions go hand in hand with strong typing, and unchecked exceptions go with dynamic typing. Unchecked exceptions help make the code more fungible around business logic, which checked exceptions help make the code more robust around system facilities. Java gives us that choice in the exception arena, but not more broadly for type safety. (Oh, and .NET is really mismatched by having static typing, but unchecked exceptions.)

I forget where I read it lately, but someone proposed a language that would allow you that choice in how types were enforced. If this could be done on a class by class, or component by component level, it would let us avoid the complexity of using two implementation languages for an application, which would happen if we go with a static typed language for system facilities and a dynamically typed language for business logic. Perhaps there already is a non-mainstream language that allows that. Let me know if you’re aware of any.

Educational Tests

Brian Marrick comments on Mike Clark’s blog entry Staying Fit With Learning Tests . I’m quite interested since I noticed some of my own backsliding behavior recently when I was writing code against a new API. I was writing my first JMX MBean, and had no one experienced with JMX around for guidance. I didn’t write it test first :-O

I did, however, go back and give it a full unit test suite after-the-fact, and it has been rock solid in production. It did take some refactoring to get it in shape to be easy to unit test, so the late testing step did improve the design. Anyway, back on topic: I attributed my backsliding to the fact that I was exploring/learning a new API.

I think there’s a well-known compromise between Brian and Mike’s approaches, and I’m waiting for a Smalltalker to go into more details — the Inspector. My first development job was for the Apple Newton PDA, which used a custom language called NewtonScript that had features from a number of languages (I’ll try really hard not to bore everyone with all the details, but suffice it to say that it was really a great language to program in, but I was too inexperienced to really realize it at the time).

One of the features the NewtonScript IDE copied from Smalltalk was the Inspector. I’m sure I never used it to its full potential due to being young and dumb at the time, but it allowed the sort of interactive exploration Brian is mentioning, and stored these “lessons learned” in a file that could easily be reopened and re-executed at any time. Mike’s learning tests don’t sound like they’re going to be included in the regular source tree and build process. The only reason they would be saved is to review the lesson and copy/paste some of it into the real system.

Eclipse has the concept of scrapbook pages, which are essentially renamed Inspector pages. It’s not too surprising considering Eclipse’s Visual Age roots, however, I was coding in IntelliJ IDEA so I didn’t think of a scrapbook until I read Brian’s blog. The advantage of an inspector/scrapbook page over Mike’s practice is that you don’t need the overhead of a full compilation unit. You don’t have to have a class and method in which to place executable lines of code. The ‘run’ function of the scrapbook handles that for you.

I suppose I need to explore some more nooks and crannies of IntelliJ to see if scrapbooks are available, but I had looked briefly when I first switched from Eclipse and did not find them. Perhaps it is time to file a feature request.

iLoo

I first saw an AP story on this, but I thought it had to be a hoax until I found this on Microsoft’s own website. They’re creating an Internet connected port-a-potty!

Must…….resist……..toilet…….humor……. but it’s such fertile ground.

Hackers and Painters

I know I gave Paul Graham a bit of a hard time earlier, but I have nothing negative to say about his latest essay, Hackers and Painters. He weighs into the dynamic vs. static languages with this great characterization:

… a programming language should, above all, be malleable. A programming language is for thinking of programs, not for expressing programs you’ve already thought of. It should be a pencil, not a pen. Static typing would be a fine idea if people actually did write programs the way they taught me to in college. But that’s not how any of the hackers I know write programs. We need a language that lets us scribble and smudge and smear, not a language where you have to sit with a teacup of types balanced on your knee and make polite conversation with a strict old aunt of a compiler.

I’m seeing writings all about dynamic vs. static languages all over blogspace. In the meta-sense, it’s really interesting to see a common topic over such a wide variety of individual writers. Good writers, must first be good readers.

I also was quite struck by this insightful, but disheartening observation:

Big companies want to decrease the standard deviation of design outcomes because they want to avoid disasters. But when you damp oscillations, you lose the high points as well as the low. This is not a problem for big companies, because they don’t win by making great products. Big companies win by sucking less than other big companies.

The only slightly contrary point I’d make about the whole essay is that experimental scientific research is a lot more like debugging than he realizes.

Rsync Your Referrer log

I had thought about this the other day, but it didn’t immediately work as easily as I expected. On my webhost, I’ve added a symlink to my referrer log file in my blosxom directory. I also made a custom shell script that Blapp executes to synchronize my blog entries. That script does a two way rsync so not only does it publish my latest entries from the laptop to the webhost, but it also copies the referrer log from my webhost to my laptop. The script looks like this (with some mods to protect the innocent and a line continuation backslash for presentation):


#! /bin/sh
rsync -Cavz -e ssh --delete /Users/gvaughn/Sites/gigavolt.net/blosxom/ \
userid@webhost.com:/WebSite/gigavolt/blosxom/

rsync -urptgovLz -e ssh --copy-unsafe-links \
userid@webhost.com:/WebSite/gigavolt/blosxom/ \
/Users/gvaughn/Sites/gigavolt.net/blosxom/

As a bonus tip, that ‘C’ in the first rsync call respects entries in $HOME/.cvsignore. I added ‘.DS_Store’ in there to keep some of that extra junk from going across.

Dynamic vs. Static Languages

Bob Martin asks Are Dynamic Languages Going to Replace Static Languages. I replied:

A couple of months ago my mind went down this same path. I do believe that TDD compensates for many of the things static typing helps with. Over an IM conversation with Glenn Vanderburg, he came up with a good counter-point. The pointcut concept introduced by AOP is very powerful, and relies on static typing. I haven’t gotten my hands into AOP enough to be sure, but my impression is that it could be viewed as a compensation for the lack of meta-programming. Since weakly typed languages are more likely to offer meta facilities, this may be a moot point.
I’m glad to read the article to refresh this idea in my mind, because I think it may tie into the other direction my mind has been going lately (and I love those connections between ideas). I’ve recently began emphasizing the difference between the business logic classes, and the system infrastructure classes and how the different goals imply the different approaches we should use. Now I’m confronted with the question: Is system infrastructure best with static typing, and business logic with weak typing? I think I’ll start up a new background thread..

So, I’m not so sure dynamic languages will replace static ones, but I do think they’ll take some of the market where they are typically used. This also ties into my early entry about looking for languages that truly move the state of the art ahead. I will have the time to get more acquainted with Ruby by using it in a project in the next month or two. I’m excited.

Finding Bugs is Easy

I’m referring to a paper describing the utility FindBugs. It appears to be developed by a student of Bill Pugh, who is best known for discovering why double-checked locking is not completely safe in Java. I haven’t had the time to play with it yet, but judging from the paper, it looks very promising. It uses heuristics to find typical bug “patterns”. Yes, it’ll have false positives, but it could also prove invaluable in tracking down real bugs before they bite.