Posted by Anders
Thu, 09 Nov 2006 19:26:00 GMT
How can you truly tell if someone is a skilled programmer?
Years ago it appeared that knowing all the buzzwords from GoF’s “Design Patterns” was synonymous with being a good designer. But the times when people proudly carried around a copy of that book as a token of their status are long gone. These days most people you interview will have learned the GoF patterns early in their education.
Basic language skills?
If a person has years of Java development behind them there are some details about the language that they couldn’t have missed. But it’s easy to overestimate it. There are lots of good and productive programmers who don’t know some language details that others take for granted. The people who know details the best may even be those who have studied for useless certifications but know little else.
Long experience?
Some people can work for years without really gaining any experience. You come across people who last learned something when they were in University years ago.
Knowledge about current tools and methods?
It may only mean that the person is on top of the latest hype, so it’s not enough in itself. But it’s always good if people are up-to-date with what is happening. If they have also truly used the techniques, not just “tried JUnit a little1”, then we could be on to something good.
The proof?
I think nothing compares to showing real code. By looking at their work you can imagine just what this person will do with your code base if you hire them.
On one occasion I’ve asked applicants to bring their own code, just something, anything, that they’ve written. It didn’t work that well, mostly lots and lots of boring code, using JDBC, Struts and other APIs in the same way that everyone else uses them.
What seems to work better is specific tasks to solve. That should give you a much more limited and focused material to judge. I’ve solved one such task myself, but never had the opportunity to use them on others. It would be fun to try.
1 Everyone has tried JUnit a little.
Posted in Java, Programming | no comments
Posted by Anders
Wed, 18 Oct 2006 19:38:00 GMT
I believe in the theory of lazy optimization, that you shouldn’t optimize something unless you’ve truly seen the need for it. But we’ve run across a few cases now where it wasn’t that simple. Initially we saw the potential for a huge bottleneck. We did some experiments, made simple measurements, profiled our code and decided it wasn’t going to be a problem. But then we scaled it up and suddenly there was a big problem, as we initially suspected.
In some cases it’s been the case of not doing the necessary math early. In retrospect, knowing the measured throughput, amounts of data, latencies, etc. and adding them up showed that it wasn’t going to work. Many of the inputs to that calculation are unknown when you start out, but spending the time to find them out is probably worth the time.
In other cases it’s more a question of the test environment. When production data is lots and lots of gigabytes and takes ages even to copy, it’s really hard to set up a realistic test environment. Even a 75% size replica of the production environment may fail to display the problems. This may seem like a hard problem to solve, but hardware limitations are usually easier to solve than our human shortcomings.
Something related is ignoring a problem that’s right under your nose during development. You know things are broken or running too slow, but falsely believe it’s an artifact of the test environment. I’m not really sure how to solve this, except to question your assumptions and to create a test environment as close to the production environment as possible.
If you’re lucky the problem is simple to solve once you find out about it. I’ve had mixed blessings lately.
Posted in Java, Programming | 1 comment
Posted by Anders
Tue, 03 Oct 2006 06:03:00 GMT
I’ve been working on speeding up a couple of systems that were transferring data through a central database. We’re doing this by sending messages directly between the systems, which both relieves pressure from the database and gives us much quicker updates.
ActiveMQ Disappointments
We started out with ActiveMQ as the message broker, but quickly ran into some show-stoppers. ActiveMQ is capable when run as a single broker and probably when handling large queues. But publish-subscribe topics with durable subscribers quickly used up all memory. The network-of-brokers features also proved to be immature. We simply couldn’t use ActiveMQ.
Instead we switched to SwiftMQ, which costs money, but works flawlessly. It handles huge message backlogs without affecting performance1. The only things we had to change was configuration and some error handling2. Standard interfaces are a good thing.
XML Messages
We wanted to get away from earlier versioning and coupling troubles with Serialization, so we went with simple XML messages. It adds the benefit of easy inspection and even the ability to hand-craft messages for testing.
XML adds some overhead in creating the messages and a considerable overhead in parsing them. But as long as it’s not a bottleneck I believe it’s worth it, because of the good decoupling of systems we get.
There is also an overhead in message size, because of XML’s insane verbosity. While there are other plain-text syntaxes that are better suited for carrying data, like YAML, XML wins by simply being the “standard”. At least for the moment.
1 SwiftMQ appears to implement durable subscribers as queues internally.
2 The JMS specification is vague about exceptions thrown in message listeners.
Posted in Java, Programming | 3 comments
Posted by Anders
Tue, 05 Sep 2006 00:42:00 GMT
The talk about Joel Spolsky’s in-house language Wasabi reminds me of my experiences at a previous employer:
In some glorious past the company founder and some other wizards had created an object-oriented language of their own, more advanced than Objective C (for this was in the days before Java). Using this language they wrote a successful application which the company was built around.
When Java arrived, they created a Java-syntax pre-processor to their language, and machine converted their entire code-base to an almost1 Java syntax. But the compiler, runtime and libraries where still very much non-Java. Imagine not having the java.lang.* and java.util.* classes.
By the time I joined them, the founder and most of the wizards had left the company. The system had evolved into something very big and complex, parts of it essentially being an application server. The remaining programmers, very few with any experience, where left to develop this system. As a result there was very little maintenance done on the underlying language and platform, even though bugs and limitations in it was a constant nuisance. It simply had to be worked around.
For the programmers, it meant that their previous Java education and experience wasn’t as useful. Everyone had to be trained in-house and you couldn’t just go to the computer bookstore and buy titles about it. Neither could you share experiences with anyone outside the company, making the place a bit intellectually isolated. There may have been some documentation, but hardly comparable to what you’ll find on a mainstream, or even obscure, language and system. The ones who tried to move to other companies found that their lack of real Java experience was a problem2. Unsurprisingly morale wasn’t very high.
The company’s solution? Developing another application server of their own, in parallel, this time in pure Java. But that’s another story.
1 Differences in language type systems forced them to weakly typed workarounds, causing endless troubles.
2 For experienced software developers it may be less critical, but for the ones just out of a short education it was a real problem.
Posted in Ruby, Java, Programming | 1 comment
Posted by Anders
Sun, 27 Aug 2006 04:52:00 GMT
I was using a SAX parser in Java to parse some XML documents into objects. Since one of the benefits of XML is validation, I set the parser to be validating. I also had to add lots code of my own to make sure that numeric fields contained numbers etc, but I imagined that I was now reasonably sure that any document passing the parser was correct.
Later I had a different kind of XML document, which I accidentally fed to the parser. To my surprise it did a complete parse without complaining, spitting out a defective object.
What I had overlooked was that the SAX parser only validates that the document is valid against the DTD that the document itself claims to follow. Sure, the document is valid, but if it’s a valid fantasy novel instead of the valid product data I expect, it’s of little use.
Looking around
So, how to make sure that the document is valid against a specific DTD? You would have thought that someone would have a nice solution for this by now. It turns out that there is an obscure SAX2 property that you can set. But only if you’re using XML Schema, not DTD.
Most people seem to solve it by pre-processing the XML document, forcing it to declare the Doctype, before feeding it to the parser. Some do it by ad-hoc string manipulation and there’s even a SourceForge project for this very purpose. Not a very pretty solution—I figured I could do better.
Solution
I thought that it would be easy to just use one of the SAX events to check that the DTD was right. I will get a SAX event when it parses the DOCTYPE, right? No. There wasn’t an event that would give me the information I needed. Instead I had to go through the EntityResolver, which is the thing that finds the DTD. I subclassed it to intercept the resolveEntity() method, where I added a check. I also had to ensure that resolveEntity() was actually called during the parse, so that a DTD-less document couldn’t sneak through.
So far I was only verifying that it was the right DTD. But you can have several kinds of documents defined in one DTD. For the document to be correct it needs to have the right top-level element1.
<!DOCTYPE TheTopLevelElement PUBLIC "foo" "foo.dtd">
So I added a second check, on the startElement() event, to make sure that the first tag it comes across is the top-level tag I want.
Conclusion
Maybe I should have just gone with the string-manipulation. My solution wasn’t very clean either, despite the good intentions.
Or maybe I should just accept that XML validation is mostly useless, since I still have to add lots of more checks in my code. (XML Schema does this slightly better that DTD, but at the cost of more complexity).
1 Not available through SAX events either.
Update, August 31th:
Jörn Horstmann pointed out the LexicalHandler interface. This enabled me to remove the checks in the startElement and the EntityResolver, leaving only this:
LexicalHandler lexicalHandler = new DefaultHandler2() {
public void startDTD(String name, String publicId, String systemId) {
// Check name, publicId and systemId here
// Throw exception if they're wrong
}
};
parser.setProperty("http://xml.org/sax/properties/lexical-handler", lexicalHandler);
This should solve it all, in one place. The identifier ending in “2” and the wierd URL-as-property-name aren’t very pretty, but that goes for a lot of things in the strange world of XML.
Posted in Java, Computers, Programming | 2 comments
