Wednesday, September 29, 2010

The Death of OpenSolaris: Choosing an OS for a Java Developer

A Bit of History: The Struggles of OpenSolaris

This is no news: you probably know all about it.

As a long time Solaris user, the recent years have been full of good news for me.

I remember starting with GNU/Linux at home to have "something similar" to the Solaris workstations I used at work. It was the time when software would most likely compile on Solaris rather than on Linux.

Years later I bought my first Sun Workstation: it was the time when trying to compile on Solaris packages that would supposedly compile on a POSIX system was a pain. Still, I continued to regard Solaris as a stable platform to keep on using it for my work duties, such as Java programming.

Then came Solaris 10 and all of its wonderful technologies such as ZFS, Zones and DTrace, just to cite a few. With it, there came the Solaris Express distributions which, at last, filled a long standing gap between Solaris and other operating systems, providing us a pretty modern desktop environment.

In late 2008 came the first OpenSolaris distribution. I installed it, played with it, but kept on using SXCE for my workstations. The main reason was compatibility with many Sun packages, such as the Java Enterprise System or the Sun Ray Server Software, that had more than one glitch on OpenSolaris.

When SXCE was discontinued, I waited for the 2010.xx OpenSolaris release to upgrade my systems. Unfortunately, that release will never see the light.

The Oracle Leaked Memo (the dignifying uppercase is a must given Oracle prolonged silence over the subject) shed a light over Oracle plans for Solaris proper and OpenSolaris. Part of the "good news" is that the Solaris Express program has been resurrected and the first binary distribution is expected later this year.

The bad news is that the code, at least the parts of it that will be released with an open source license, will be released after the releases of the full Solaris Operating Systems. Basically, our privileged observation point over the development of the operating system has been shut down.

Lots of ink has been been spilled since the Leaked Memo and plenty of information, discussions and wars are going on in the blogosphere. I'm not an authoritative source to speak about the subject and it's not even clear to me what I'm going to do, now.

Benefits of Solaris for a Java Developer

Solaris has been my operating system of choice since before I started working in the IT industry. As a student, I grew up with Solaris at the data center of my University and the Slackware I used at home seemed like a kid toy, compared to it. After graduating, I started working as a design engineer for a leading microprocessors producer. Needless to say, Solaris was the platform we ran our design software upon. Then, I moved to a consulting firm and started my career as a Java architect.

Solaris was and is the platform of choice for most of the clients I've been working for. Even today, the application servers, the cluster software, the database, and most of the infrastructure used by my clients run on Solaris. It always seemed a sound choice to me, then, developing software on the same platform that will run it in production.

IDEs, Tools and Runtimes

As a Java developer, I can run all of my tools I need on a supported platform. My favorite IDEs (NetBeans and JDeveloper), the application servers my clients use (WebLogic and WebSphere, mostly), the databases used by my applications (MySQL, Oracle RDBMS, PostgreSQL): all of them run and are supported on Solaris. Some of them are even bundled with it or readily available by Sun sponsored package repositories. The Eclipse platform, to cite a widely use IDE for Java, is available in the OpenSolaris IPS repository, too.

Solaris Technologies

Solaris 10 also integrates DTrace, a powerful, unobtrusive framework that allows you to observe and troubleshoot application and OS problem in real time, even in production systems with an almost null overhead. DTrace has allowed us to diagnose strange production quirks with no downtime: once you've tried DTrace and the D language, there's no going back to "just" a debugger, even in the development stages of your project.

Other kinds of problems does not show up in your debugger or are extremely hard to catch. It might be the case of network or file systems problems. That's where DTrace comes in handy: it lets you observe with an incredibly high detail what's going on in your application and in the kernel of the operating systems, if it's necessary to dig so deep.

Solaris Virtualization Technologies

Solaris is also an ideal host virtualization platform. Solaris can "virtualize itself" with Containers, Zones and Logical Domains: you can start a Zone in no time (and almost no space overhead), assign a resource cap to it and also build a virtualized network in a box to simulate a complex network environment.

One of the problems that I encountered during the development of big enterprise system is that the development environment, and sometimes even the integration environment, is very different than the production one. It surely is a methodology problem: nevertheless, developers have few weapons to counteract. For example, applications that appear to run fine on a single node may not run correctly on a server cluster, or scale badly.

The more you wait to catch a bug, the more impact will have a patch for it. That's why in my development team, for example, we use Solaris Zones to simulate a network cluster of IBM WebSphere Application Servers and a DB cluster. All of them run in completely isolated Zones in one physical machine and communicate on a virtual network with virtual etherstubs (sort of a network switch), VLANs and routing rules. This environment lets us simulate exactly how the software will behave in the production system. Without a flexible and lightweight virtualization technology it would have been much more difficult and expensive to prepare a similar environment.

And if you (still) need to run other platforms, you can use Xen or VirtualBox to run, for example, your favorite Linux distro, Windows, or *BSD.

Summarizing

Enumerating the advantages of Solaris is difficult in such a reduced space, however I'll try:
  • It's a commercially supported operating system: that's an option, since Solaris is free for development purpose. Nonetheless, it's an important point to take into account.
  • Is (very) well documented: there's plenty of official and unofficial documentation.
  • It's easy to administer: Solaris is pretty easy to administer, even if you're not a seasoned system administrator.
  • It's an UNIX system: with all of its bells and whistles.
  • It is a great virtualization platform.
  • It has some unique technologies that add much value to its offering, such as ZFS and DTrace.


If you're a Java developer and haven't given Solaris I try, I strongly suggest you do it. Maybe you'll start to benefit from other Solaris 10 technologies such as Zones and ZFS, even for running your home file or media server.

Complaints

I often hear complaints about Solaris coming from different sources and with the most imaginative arguments: proprietary, closed, old, difficult to use. I usually answer inviting users to try it and see for themselves before judging it (if that's the case). Most of the times I'm not surprised to discover that the complaining guy had minimal or null exposure to Solaris.

Also, I'd like to point out that no OS I tried is a swiss army knife. Solaris is a server-oriented OS with a good desktop but it's not comparable with other operating systems for such an use. So: no comparison with Linux, please. It would be so unjust as comparing Linux and Mac OS X for the average home user. ;)

Alternatives

Since Java "runs anywhere", there's plenty of choice for a Java developer.

Since I own a laptop with Mac OS X, I've built a small development environment with all of the tools I need. Mac OS X is a great operating systems that comes with many fancy features out of the box and, although it has some idiosyncrasy with Java (read: you have to use the JVM shipped by Apple), it's a good OS for a Java developer. Since the Mac OS X hype has begun, there's plenty of packages for it and a big ecosystem which is still growing. Still, many software packages run in the enterprise aren't supported on Mac OS X. Since I prefer to have an environment as close as possible as the production one, I think that OS X is not the best choice for the average Java EE architect.

I've also been an hardcore Slackware and Debian user for a long time. An enterprise Java developer would miss nothing in a modern GNU/Linux distribution, nowadays, and most of the software packages you'll find in the enterprise will run on your GNU/Linux distribution.

No need to talk about Windows, either.

So, why Solaris? Every OS has its own advantages and disadvantages. The point is to just recognize them. Mac OS X, in my opinion, is the best OS for a home user. I would change it for no Windows and no Linux. But as far as it concerns my developers' duties, every other OS just lacks the features and the stability that make Solaris great. ZFS, DTrace and Zones, for my use cases, are killer features.

What's Next?

You've decided to give Solaris a try, so: which is Your distribution? I don't know.

Solaris Express/Oracle Solaris

I strongly suspect that my wait will be prolonged and I will finally upgrade my machines as soon as Solaris Express has been released. Upgrading to Solaris 10 09/10 is not possible since I'm using some ZFS pools whose version is not yet supported by Solaris proper but it is a sound choice for a starter.

The advantage I see in using one of these versions is the availability of optional support and the good level of integration with the most commonly used software packages that Oracle is likely to guarantee.

OpenIndiana

You should also know that OpenSolaris sources have been (sort-of) forked and two new projects are born: Illumos and OpenIndiana. The project were started by Nexenta employees and volunteers of the OpenSolaris community. The first projects aims at maintaining the OpenSolaris code and the parts of the code that are closed or code that upstream might choose not to maintain. The OpenIndiana project aims at producing binary distribution of the operating system built upon the Illumos source code. OpenIndiana will provide a really open source, binary compatible alternative to Solaris and Solaris Express.

Sounds good and I'll willingly support it. In the meantime I've installed OpenIndiana in a couple of virtual machines and the first impressions are very good. I suppose it hasn't passed enough time yet for diverging changes to have emerged.

If you prefer a more modern desktop with a recent Gnome interface, drop Solaris 10 and go for OpenIndiana, if you don't feel like waiting for Solaris Express. In any case, switching between the two shouldn't pose any problems. What's clear to me is that I won't consider using both operating systems: I'll have to make a choice.

Support Availability

As an enterprise user and a Java developer, I've always been more concerned about OS support and support for the packages I use, rather than about eye candy. Even at the cost of running a proprietary platform.

In conclusion: I'll wait for Solaris Express to be released and only then will decide which one I'll use for my purposes between Oracle Solaris Express and  OpenIndiana. My heart is betting for OpenIndiana. My brain is betting for Oracle Solaris Express and Solaris proper. Only time will tell which one is right (for me.)

Follow-Up

A follow-up of this blog post is avaible at this address. In this post I'll try to summarize some use cases in which the technology we introduced in this post are effective and add real value to your development duties.

I hope you enjoy it.



9 comments:

juhoeemeli said...

Cheers,

And thanks for the informative commentary on the going-ons in (open)solaris front. As an enthusiastic opensolaris newbie myself, I was really disappointed by the Oracle leak. And now of course facing the decision between OpenIndiana (preffered) and forthcoming Solaris Express. I have a question though, in case you have time: I decided to give OpenIndiana a try and upgraded my OpenSolaris 2009.06 (svn_111b) to OpenIndiana as described in http://wiki.openindiana.org/oi/Installing+or+Upgrading. Works like a charm, except the globalSAN iSCSI initiator on my mac stopped working with the target on the server (it dies with some vague "Target hardware or software error"). A windows laptop however is happily churning away with it's scheduled backup as I write. It has the same target, can see both backup drives and has no problems at all. So I'm assuming the issue on the mac has something to do with globalSAN initiator. Any insights?

Grey said...

Hi juhooemeli.

Glad you liked this! ;)

Yes, it seems it's a problem on the Mac side. Let's start from the quickest check.

Sanity check: Have you recently upgraded the globalSAN iSCSI initiator? The only time I saw that error on my Mac was after such an upgrade without uninstalling the previous version with the uninstaller in its dmg. I had to uninstall both versions using the uninstaller in each dmg and then reinstall the latest version.

If that's not the case, I would start with a packet snoop and see what's going on.

However, you're using COMSTAR and not the legacy iSCSI, aren't you?

juhoeemeli said...

Thanks for quick reply

Yea, it's COMSTAR for me.

Nah, didn't update globalSAN 'till already had the problem. The previous BEs that have OpenSolaris 2009.06 (svn_111b and svn_134) still work fine with it anyways. It's only the OpenIndiana environment that has the problem. I think I found some suspicion before about globalSAN not working with some versions of COMSTAR, but that was way back when I was putting the server together. Never gave me any trouble though until now, if that is even the case here.

Really don't mean to take up all your time. Maybe I go moaning about this in the OpenIndiana forums now to see if anyone there had the same issue...

Anyways thanks, got yourself a new reader :)

Grey said...

Glad to hear it! ;) Hope it'll be always an interesting read.

I'm curious to try it and see what happens.

You can also check with the guy of [storage-discuss] at OpenSolaris.

Neil Matatall said...

I have used Solaris for 4 years now and I just don't see the point. In your section about the benefits of Solaris, how is this not true on any other *nix system (yum, apt, etc)?

What does Solaris really give you? Specifics? I did not develop on Solaris during school so I didn't reap that benefit.

I'm curious because on the argument that Solaris is not free/open is enough to end the debate for me.

Grey said...

Hi Neil.

As far as it concerns what you say about package managers, that was not really the point. The point was that Eclipse, unlike other Java IDEs, is not purely Java. In fact, it requires some native library recompilations and, since Eclipse was not officially available a Solaris binary, it usually was a pain having to recompile those library yourself. After SXCE and the OpenSolaris hype, plenty of packages are readily available using (more or less) ease to use package repositories (such as Blastwave or the OpenSolaris and OpenIndiana IPS repositories).

However, I've written a follow-up to detail some use cases in which Solaris technology might give real advantages to a developer:

http://thegreyblog.blogspot.com/2010/10/some-reasons-why-solaris-is-great-java.html

Bye.

Suhail said...

I believe that Oracle don't want OpenSolaris to compete against Solaris. In my opinion OpenSolaris should be a desktop centric distribution for students and web developers and Solaris should be a ServerOS. Instead of releasing a new version every six months (like linux) OpenSolaris should be release once in a year. Every major yearly release should be supported for at least 2 years with 9 updates on a quarterly basis.

El critico said...

You state in your blog:

> It's a commercially supported operating system: that's an option, since Solaris is free for development purpose. Nonetheless, it's an important point to take into account.

It isn't true. It is subject to the otn license. This license is very restrictive. It only allows to use one server with the program, only used by one person in the company.

Please read carefully the OTN license. It is complex to understand all its implications.

Grey said...

Thank you for your comment, El Critico.

At the time of writing we were all waiting for the next OpenSolaris release (that never came), Oracle Solaris 11 Express could not be seen yet.

It is true that it is now subject to the OTN license, a license I never liked, but I did not updated this post.

Despite the licensing issues, I'm still convinced that Solaris is one of the best Java development platforms out there. Unfortunately, unless OpenIndiana cuts its space and proves me wrong, the opportunity that Solaris could have in the developers' workstations shrank considerably after the Sun acquisition and the licensing changes.

Also, please note that I was speaking about an OS for a developer's workstation: in that use case I'm sure that OTN will not stop a developer from using it.

A server shared by more than one user is not "a developer's workstation", although I agree that in that case, OTN will stop you from doing what was permitted with the former license.

Moreover, chances are that a developer choosing Solaris as a development platform in a corporate environment does so because the corporate systems run Solaris. In that case, I'd expect licensed Solaris servers to be available when the "developer's deliverable" has to be tested outside of its workstation.

Grey