Wednesday, May 23, 2012

X-Rays for OSGi

One of my frustrations of my OSGi years is that while OSGi provides this tremendous wealth of dynamic data about the state of the system nobody has taken the time to really visualize it. By far the best tool I know is the Apache Felix Web Console because it provides comprehensive information about the operations. However, even with this tools I am well known to waste hours chasing problems that would have been obvious had I looked at the proper page. Though the Web Console is by far the best tool around to find information, it falls far short of a tool that can inform me when there are problems.

Our intelligence is closely related to our visual brain. We can spot problems in one glance, problems that would take ages to discover in raw data. With a 17" MacBook Pro and an extra 27" screen, I've so much screen space (and many developers have a similar setup) that I gladly devote the Pro's 17" to a continuous X-Ray of my running framework. Alas, such a view does not exist.

Well, it did not exist, past tense. Though not related my venture my frustration made me spent some time on creating such an X-Ray for OSGi. The excuse was that I need to learn Javascript anyway and this seemed an excellent opportunity to learn the intriguing d3 library. I had hoped to finish this last week but Richard S. Hall and Karl Pauls spent a worthwhile week here to discuss their plans and see how we can collaborate. This was much more (intense!) talking than I had figured so I had to postpone the finishing touches till this week.

Since I really like the Apache Felix Web Console I started from there. It turns out that it is trivial to do a plugin for the Web Console since there is a good base class that takes of most of the chores. The Web Console does not only look good, it is also well designed. Adding a plugin is as simple as registering a service. The Abstract Web Console Plugin they provide was an easy way to get quickly started.

So after a few hours I had a basic SVG window with bundles and services. Getting it look nice was the really hard part. I wanted to use the diagramming technique I always used, triangles for services and boxes with rounded corners for bundles. That was the easy part. Getting the wires to run without causing a visual mess was the hard part. The d3 library contains some layout managers but none was suitable for my purpose (though bundles and services look pretty cool in a force directed graph!). In the end I settled on a grid where bundles go vertical and services go horizontal. In this model you can always wire with a horizontal and vertical line that do not meet obstacles. This sounds simpler than it is because the diagramming technique for OSGi requires  registering bundles to connect at the sharp side of the triangle, getters at the flat side, and listeners at the angled side. Since wires can overlap you also need a visual cue to see what wires are joined. Getting this right was quite tricky and required me to dust off my trigonometry books. The result looks like the following picture:


What functions did I implement. In some random order:
  • Objects navigate to the page in the Web Console where they are detailed. Clicking on a bundle takes you to the page for that bundle, clicking on a service takes you to the services page.
  • The state of a bundle is indicated with color. An orange bundle is happy and active, a grey bundle is resolved, and a white bundle is installed only.
  • The information is polled from the system and automatically updated. The update is transitioned so uninstalling a bundle causes the bundle beneath it to smoothly crawl up, automatically adjusting any services if necessary. Useless from a functional point of view but pretty cool to see!
  • You can remove services by dragging them off the screen. Refreshing the window brings them back.
  • If a bundle has recent errors or warnings in the log then a small warning icon is displayed. Clicking on this icon takes you to the log, hovering over it gives you the log messages. The Log Service must obviously be present for this to work.
  • If the Service Component Runtime service is registered then the information about DS components is collected and displayed in the bundle. Each component is summarized with a LED that can be red (not satisfied), green (active).
  • Listener Hooks are used to find out what bundles are waiting for what service. Services that are not present but still are waited for are drawn with a dashed outline. Services that are only registered but not used are displayed white. Active services used by bundles are yellow.
The xray bundle only requires the Web Console, you can download it from github repo. Source code for this plugin is inside the JAR. If you just want to try it out quickly, you can:
$ sudo jpm install xray.demo.jar http://dl.dropbox.com/u/2590603/xray.demo.jar
Installed command [xray]
$ # start/stop bundles with the shell ...
$ xray
-> stop 10 ...
Oops, guess jpm is not ubiquitous yet ... So just download it from the URL and use it like:
$ java -jar xray.demo.jar
-> stop 10 ...
If you got it running, open your browser on http://localhost:8080/system/console/xray and have fun. Feedback very much appreciated! Since I am a committer at Apache Felix I will likely move the code over there once it gets a bit more mature.

Peter Kriens

12 comments:

  1. From the SSH console of a running Karaf instance, this works:

    bundles:install https://github.com/bnd/aQute.repo/blob/master/repo/aQute.xray.plugin/aQute.xray.plugin-1.0.0.jar?raw=true

    ReplyDelete
  2. Any chance to get it for Glassfish 3.1.2?

    ReplyDelete
    Replies
    1. Christian: it's just a bundle, and Glassfish runs on Felix, so you should be able to simply install it.

      Delete
  3. Cool stuff Peter I really like it…

    Do you think about to mark cycles e.g. red?
    For example bundle A provides the service Echo and bundle B has a reference to this service. Bundle B provides a service Monitor and bundle A use this service. So we have a cycle on the service layer, not on the module layer.

    ReplyDelete
  4. Cool!

    Would it be possible to embed the d3 library as a resource in the jar? For those occasions when a connection to the internet is not possible / allowed...

    ReplyDelete
  5. Saw this on the bnd-tools mailing list earlier and in just a few hours it has proved invaluable to me already! Fantastic. :)

    ReplyDelete
  6. I know that getting x ray services is very important and can really reveal what exactly is going on and you do not have to jump to surgery just to find out. Thanks for all the information and the great post too.

    ReplyDelete
  7. A big kudo and thanks to you Peter Kriens. Your links here do not work but i found one here http://repo.jpm4j.org/#!/p/sha/b9cd47a5ad2f3abd4e9d89dffa65772c3674c042//0.0.0, took its jar link and put in our iDempiere (complex) project https://experimental.idempiere.org/osgi/system/console/xray (SuperUser/System) . Too charming. I have credited you in our sites. And put a note here to show others it is still kicking. Least i can do to give back.

    ReplyDelete
  8. This is such a great resource that you are providing and you it away for free. I love seeing blog that understand the value.Digital x ray system in delhi

    ReplyDelete

  9. Excellent and decent post. I found this much informative, as to what I was exactly searching for. Thanks for such post and please keep it upDigital x ray system in delhi

    ReplyDelete