Tuesday, February 12, 2013

Bndtools 2.0 released

Yesterday Neil Bartlett released bndtools 2.0, a major piece of work by Neil and others. He worked so hard on it that he even stopped harassing me in the morning! Kidding aside, even though I am part of the time I am often surprised to find very useful functions that I was totally unaware of. Now, if you looked at bndtools some time ago, take another look since it really matured. I am convinced that it is by far the best tool to develop OSGi bundles in the market. This new release adds a lot of new features, some of them I'd like to point out in this blog. Today I am starting with release management.

For me, by far the most interesting new feature is the release tool. I hate versions with a vengeance, they are not very user friendly. In a large build it is so easy to get versions wrong and the resulting problems are not easy to detect, to say the least. Automating is the solution of course; the original reason behind bnd. However, bnd works on the bundle and class path level and therefore has no concept of change, and versions are all about change. Semantic versioning provides a language to express the nature of the change and thereby gives us the mechanism to describe a conditional dependency that accepts certain changes but rejects other changes. As the white paper argues, this dependency differs on the role the requirer plays. Consumers are loosely coupled to an API since we go out of our way to be backward compatible but Providers of that API have a much stronger connection, almost any change in the API is a responsibility of the provider. These different roles and their influence on imports is reflected in their import version policies.

The theory is fine but it requires very minute maintenance of versions. For a lot of developers the idea to maintain package versions sounds horrific because it is already so difficult to manage their artifact versions, just imagine that you have to do that 20x more! In reality, bnd has always provided support for minimizing  that work. You rarely ever have to specify import versions since they're picked up from the classpath.

The story for exports is different. You have to specify the export versions by hand, and this is trickier because it requires a judgement of the changes that were made. For example, adding a method to an interface breaks all implementers of that interface. Depending if that interface was implemented by a Consumer or Provider, you have a major or minor change. Judging this change requires a comparison against a previous version and as I stated earlier, bnd had no concept of time. It had, however, a concept of (pluggable) repositories, and repositories can provide history. Though all parts were present, there was only rudimentary functionality that bound it all together before release 2.0.

There was a release tool in the previous release but it had some shortcomings. PK Søreide from CommActivity in Stockholm and I set out to improve this function for this release. I extended bndlib with an extensive API and resource diff tool and a baseline tool. The diff tool understands Java semantics and is capable of representing the changes between two JARs all the way up to the modifiers of a field or method. It also judges if a change is MAJOR, MINOR, or MICRO: the parts of our version. Since this judgement depends on the role that an interface or class plays, the diff tool supports the @ConsumerType and @ProviderType annotations. The diff tool also supports many other, sometimes subtle, rules about compatibility like adding methods to an interface or making a field protected.

$ bnd diff -am some-1.1.0.jar some-1.0.1.jar
ADDED      PACKAGE    baeline
 REMOVED    HEADER     Bundle-Version:1.0.1
 ADDED      HEADER     Bundle-Version:1.1.0
 CHANGED    HEADER     Export-Package
  ADDED      CLAUSE     baeline
 ADDED      HEADER     Provide-Capability:x-jpm-plugin; x-jpm-plugin=apache.felix.webconsole

The baseline tool then takes the diff tool's output and calculates the minimum required version bump for the bundle and the packages. For example, from the command line:

$ bnd baseline some-2.0.0.jar some-1.1.0.jar 
  Package                       Delta      New        Old        Suggest    If Prov.  
  aQute.library.remote          UNCHANGED  1.0.0      1.0.0      1.0.0      -         
  aQute.service.library         UNCHANGED  1.0.0      1.0.0      1.0.0      -         
  aQute.struct                  UNCHANGED  1.0.0      1.0.0      1.0.0      -         
  baseline                      MAJOR      2.0.0      1.0.0      2.0.0      -  

Now, command lines are my favorite tool but, although they seem to be gaining in popularity, most developers want buttons and lists. So PK developed a GUI tool that displays the aggregates of the changes with the possibility to drill down to the most minute details. It also provides a suggestion for the bundle version and each modified package version. With one press of a button, the build is updated and the artifacts are pushed tot the release repository.

There is also a global command for releasing all the bundles in a workspace in one go. With one press, you can now release all your bundles to the Release repository. Since we also added a new File Repository that is easy to connect to ftp, git or some other deploy tool you can automatically make it available to team members or the world. However, that is for a next blog.

 Peter Kriens

Thursday, February 7, 2013

Where Java Fails

Today a local software developer and I enjoyed one of those wonderful French lunches. We had met a few times over the phone because we have a shared interest in the local Internet fibre, paid for by the county, but that seems hardly used. Before the lunch I visited his office to have a chat. where it immediately became clear that Java was a 4-letter word in the office, despite being an IBM shop. The story they told me was quite tragic and I am afraid that thousands, maybe even hundreds of thousand shops can retell this same story at the detriment of the Java community.

At the beginning of this millenium Java was heavily promoted by IBM since this language could unify their amazing breadth in incompatible platforms. So they send their (COBOL and RPG) developers to IBM for training. After 4 weeks they returned, no longer being to walk with their heads up, and not having a clue where to start with a Websphere application. Then, when they finally got something to run it actually crawled. It did not take much time before they got so disgusted that they threw it all out and happily moved to PHP ...

Yes, Java is big at big corporations (a not unimportant market) but what does it say when developers run for the door when it is their own money? Actually, my lunch partner confirmed what I always felt: Enterprise Java is making simple things really hard and as a result we loose a large number of potential Java developers. Working on an enterprise like application for the last year I actually find Java quite perfect for the task. After working with Javascript and Coffeescript over the past year I know using Java is a lot less frustrating. Not that the language is superior, there are many nicer languages, it is the Eclipse JDT with Java combination that is far superior to anything else I know. Scala might be nicer as a language but last time I looked their plugin it was not even close. Loved Xtend as a language but using it sucked 0without the brilliance of JDT. Pretty sure Ceylon has superior language features but confident that their plugin does not come close to JDT without even having to look. The lack of support for refactoring makes using any language a pain in comparison and Java is quite lonely at the top due to JDT. 

My lunch partner never even heard of refactoring, he had disengaged long before those incredibly powerful tools became opportune. So where is the gap? Let us take a look at some of the issues.

Object/Relational mismatch. Much of what enterprise developers do is shuffling around information in databases so you would think this should be easy to do in an Enterprise language. However, since all we had was objects Java crashed right into relational database technology. It is almost awesome to see what lengths our industry is willing to go (think Hibernate, JPA) to cover this fundamental impedance mismatch. What you can do with a few lines of PHP takes XML, Java, Annotations, and more in our world. Looking at many frameworks one can only be awed for the amount of complexity we are willing to add for ease of use (and fail at it).

Develop Cycle. I f there is one thing I learned in the last year it is that I spent most of my time figuring out how things really work out there. Documentation is often lacking and the only way to figure out what works and what isn't is trying, looking at the result, and making corrections. Turnaround time in PHP and other dynamic languages is negligible since you just change the file on disk.  We have a fundamentally more lengthy edit/compile/debug cycle. Any second between saving your code and being able to view the result is just plain unpleasant. Even though we have in process code replacement nowadays, it is still not as fluid (although bndtools is nowadays as close as it gets).

Custodians mismatch. Over the last years I've worked with many system developers. Many of them really good people, bright, hard working, and sincere. However, few of them have actually developed real world applications, they invariably have worked on the software vendors site. More often than not hired from University. And still, these people are the trend setters, write the specifications, and are more or less the custodians of Java. Just something to ponder: how come there is no money type in Java? Enterprise software is very much about shuffling money; readers that ever had to write this kind of code know money calculations are not straightforward and map neither well to int, long, double, or float and calculating with BigDecimal is a daft. The absence of such a type illustrates the discrepancy between the custodians of the language and their users.

Modules. All popular web languages (Perl, PHP, Python, Ruby, etc) have a module system, CPAN being the father of them all. We utterly lack something in Java. Yes, Maven central is a great collection but using a maven artifact is for many reasons a lot more complicated than using PHP Pear or NPM. Not only should it be a lot easier to reuse components, it should also be really easy to modify them to suit our needs. Though there a multitude of open source projects, cloning is expensive since it is often not cohesive and usually quite large. There are very few highly cohesive uncoupled enterprise components ready for reuse out there.

Concluding, the reason my lunch partner had not discovered the joys of refactoring is because he could not start simple. I also learned this when I wanted to teach Java to my children or when I wanted to build a simple website. There is no Java alternative to Ruby on Rails or just plain old PHP. And actually this frustrates me to no end since I do believe we have by far the best technology. Despite this advantages, we seem to be unable to provide a simple entry point to our temple for the rest of the world. Sadly, in our quest for backward compatibility we actually tend to make it harder and harder for newcomers, something that does not bode well for our industry.

Peter Kriens