Christoph Jahn

When you are running Linux (or any other Unix-like OS) on a mobile computer, you may have a bit of a problem with mounting shares from a server. On a normal PC with a fixed connection to the server, you would simply add the appropriate entries to /etc/fstab and have them mounted automatically during startup. But since I am not “always on”, this was not an option. So I added the “noauto” option to the entries in /etc/fstab, which effectively means that they get ignored during system startup. Of course I could later call mount /my/mountpoint manually for each entry. But this is cumbersome. So I went and wrote a small script (download here). For security reasons the extension of the script is set to .sh.txt instead of .sh, so you will need to rename it before execution.

There are a few conditions for the script to work:

• Effectively the script will call “mount /my/mountpoint” for each relevant entry in /etc/fstab. All configuration details for the actual mount operation need to be provided in /etc/fstab. So before running the script, make sure those manual calls go through without problems.
• For CIFS mounts the credentials should be provided using the “credentials” option from mount.cifs (see respective man page for details). Those can then be protected properly. And should there be several people using the system, they can maintain their passwords by themselves.
• Currently there is connectivity check to the server(s). So you will get delays/errors when calling the script without a proper connection.

The script will detect whether a particular mount point is in use and skip the respective entry for mounting. The same goes for umount when the mount point is not used.

For CIFS mounts you will most likely need to call the script with sudo. At least on Fedora 12 I could not get mount.cifs to work for normal users, event with the SETUID bit set.

Here is a number of links to resources I found useful.

• There is a great (and free!) book available called Pro Git. At the very least you should read the chapters 2 (basic operation) and 3 (branching and merging). Although I had worked for almost a year with Git before coming across this book, it greatly improved my understanding. I especially liked the great visuals that make it pretty easy to grasp things quickly. An absolute must-read!
• Git tutorial: A nice reference for quickly looking up things
• Setting up Apache for Git push via HTTP(S)
• Installing Git from source on Debian
• Setting up SourceGear DiffMerge with Git
• Git casts
• Git community book

I just upgraded my OpenSolaris box from 2009.06 (snv_111b) to the latest development build (snv_131 from January 14th, 2010). The main reason was the broken CIFS server that comes with 2009.06. And although the next release is pretty much around the corner (2010.02), I needed the beast in a working state now.

The good thing about such upgrades is that thanks to ZFS and its snapshot functionality, a roll-back is trivial. The package manager will automatically create the snapshot and also create a new entry in the Grub menu for the upgraded OS. So if things go badly wrong, you can just select the old version and continue from there.

The upgrade itself is a three-step process

• Change repository URL from release to development builds
• Upgrade package manager
• Upgrade other packages

The URL change for the repository is done with the following command
$ pfexec pkg set-publisher -O http://pkg.opensolaris.org/dev opensolaris.org
The update of the package manager is done like that
$ pfexec pkg install SUNWipkg
To start the upgrade from a shell, run this command
$ pfexec pkg image-update
If you want the GUI version, please execute
$ gksu packagemanagerand press the “Update All” button.

Part of the upgrade (bug or feature?) is that the permissions of /dev/ptmx will be changed. The result is that shell access via an SSH login is not possible any more. Since my box is headless, SSH is the only way to work on it. So it’s a real problem. Fortunately, it is only the shell access that is blocked (and this not only via SSH but universally) and giving the error message:
PTY allocation request failed on channel 0

The execution of commands via SSH is still working. So you just have to issue the following command:
$ ssh -l user host "pfexec chmod 666 /dev/ptmx"

which will fix the problem. It seems that for people working directly on the machine, the solution is to blindly enter the command from above.

Interesting video from Google Tech Talks. It not only covers what you might anticipate from the title. Instead a good portion is actually about how to run an open source project in the first place.

If you want to host a Git repository there should normally be no shell access for all the people that need access to the repositories. So far many people have used gitosis to achieve this. Now there is a “new kid on the block”, called gitolite. I have not really used it in practice so far, but the added functionality looks promising and I also like the fact that it’s written in Perl. There is also a chapter about it in the Pro Git book.

The file system type for the mount command has been changed from smbfs to cifs. So using “-t smbfs” will not work any more. Instead you need to enter something like this:

mount -t cifs -o username=user[,uid=500,gid=500] //machine/share[/path] /mountpoint

There seems to be a documentation bug in the man pages, because they still mention smbfs.

A few weeks ago I have finally made the big step and built my own NAS with OpenSolaris using the ZFS file system. The main reason for this was the hassle of having multiple USB hard disks for redundancy purposes and also the relatively poor performance of USB 2.0. In the course of the decision process I had looked into the NAS systems that are available off-the-shelve. But they were either too slow or too expensive. If you want proper speed and resilience against hardware failure you pretty soon end up with systems that are 500+ Euros without any hard disk. And those are still somewhat inflexible. Of course they have the advantage of relatively low power consumption and special-purpose cases with mostly hot-swappable drives. In my case neither was relevant because I am the only person using it and the system is running only when my main machine is up.

When looking into the possible solutions I quickly realized that I want ZFS. It’s reliable, easy to set up, has fantastic features and does not require a hardware RAID controller (which alone saves you about half a grand). There is a ZFS module available for Linux but at the time people seemed to agree that it was in its infancy (relatively speaking). So it was down to FreeNAS and OpenSolaris. There were performance issues with FreeNAS (a known problem at the time), so I decided to go for the reference implementation, which of course meant Solaris. Interestingly, FreeNAS has recently announced to switch to Debian instead of FreeBSD and will therefore cease to support ZFS in the future.

There is bunch of great stuff on the net for exactly my use-case and I recommend you use your favourite search engine to find out more about the details. The one place that really stuck out was and still is Simon Breden’s blog. Make sure to check it out!

For those interested here is my setup:

• 5* 1 TB WD Green configured as RAID-Z2 (equiv. RAID-6) with 2.8 TB net capacity
• Asus M3A78-CM with AMD Athlon II X2 250
• 8 GB ECC RAM
• Intel® PRO/1000 GT Desktop Adapter (OpenSolaris did not work reliably with the built-in NIC)
• Case Tacens Prior M

I have not really done any performance tuning so far and get around 55 MB/sec using CIFS server (not Samba). Neither have I used FTP or NFS so far (solely for lack of time). For those that have only worked with Linux before, be advised that Solaris is sufficiently different to greatly increase the time needed to set up things. Nevertheless I think it was the right decision and gives me a pretty flexible system.

There is a bunch of folks out there that don’t like SOA (Service-Oriented Architecture) for various reasons. So I try to look at things without all the buzz and distill out a few aspects that in my view provide value. The goal is to provide an alternative perspective that is hopefully hype-free.

I want to split this in two parts: First (in this post) comes the conceptual side of things, which looks at what SOA in my view is about at its core. Second are the more practical benefits we get from the way SOA is approached in real life. This will be a separate post.

Concepts

As a “preface” let me point out that I look at things mostly from a technical perspective. So everything along the lines of “SOA is about business and not technology” gets pretty much ignored. There are two reasons for that: Firstly I am a technical person and more interested in the technical aspects. Secondly, the business argument is probably the one most resented by skeptics. So let’s get started …

The core thing about SOA is that all functionality is made available as a service (surprise, surprise). This is really trivial and hardly worth mentioning. However, it has far-reaching consequences once you dig a bit deeper. And it’s those secondary aspects that provide the advancements.

• The right level of slicing up things: Of course there were many other approaches and technologies before (e.g. OO and  CORBA). However, none of those has kept its promise. Admittedly it remains to be seen to what extent SOA can fulfill expectations. On the other hand, those expectations are still in flux as we all continue to improve our understanding. So there is a chance that expectations and reality will actually meet some time (where?). Also, the criticism I am aware of is not about this aspect. In fact it seems pretty much undisputed, at least I have never heard anything from customers or prospects.
• The service is the application: A direct consequence of the previous point is that the service gets elevated to the place that was formerly held by an entire application; at least from a user’s perspective. Whether the implementation reflects this or not is irrelevant to the consumer.
  For new development it is usually desirable that the internals match the services exposed to the outside. For “legacy” stuff that just gets a service interface, the wrapper logic takes this place. In either case, however, the exposed logic is much smaller than an entire application.
• State management: There has been a lot of talk about loose coupling. This principle can be applied at many levels, transport protocol and data format being the obvious ones. A slightly more subtle place is the handling of state, which pretty much depends on the aforementioned transport protocol and data format.
  The interface, at least of the initial operation, must be exposed in a way that it can just be called.  In other words it is kind of stateless. Of course from that point on everything could be state-full. There is no difference between SOA and traditional applications here. In reality, however, people pay much more attention to keeping things stateless when working in a SOA context.
• Life cycle management: This one is strongly related to the point “the service is the application”. The notion of a life cycle has formerly mostly been on the level of entire applications. The obvious disadvantage is that this huge scope almost demands big bang-approaches for new versions. With the effort and costs associated for bringing a new version into production, there are only very few such releases and much stuff gets crammed into it. The risks increase, business agility is close to zero and nobody is really happy. With a service as the “unit to be delivered”, the scope is drastically reduced and so are things like risk, complexity etc. Thus I think the concept will be brought to much more practical relevance than before.
• Standards: Although a real SOA can be built using an entirely proprietary technology, in reality a set of standards has established itself. Yes, there are probably to many WS-* things around that only work together in particular versions. So I would not recommend to jump onto too many things. But with the basic things you are certainly on the right track.

There is probably a whole lot more to say, but let’s keep it here for now. I am looking forward to any comments on the subject!

Subversion provides a property (svn:externals) to include references to other projects into a given location within your source code tree. This is pretty much the same as a symbolic link (symlink) in Unix/Linux. But while the usage of symlinks is good practice to de-couple things in the file system, it is just the other way around for svn:externals, at least in my opinion.

Interestingly enough, there is a number of sources that recommend its usage. I disagree here and strongly discourage people from making use of it for a number of reasons:

1. It creates a lock-in into Subversion, because many other Version Control Systems (VCS) do not have a comparable feature. And even if they had, an automated migration will most likely be cumbersome, to say the least. One would have to find and extract all svn:externals properties, build a dependency tree (hopefully without circular dependencies) and and process things in the appropriate order. This is far from trivial!
2. On a conceptual level svn:externals is not about version control but dependency management. This is even more important an argument than the lock-in effect.
   • Dependency management and version control are two entirely different things, which should not be mixed. Having a somewhat implicit mechanism to define the dependencies will make it easier for people to not have a clear understanding about this separation.
   • Dependencies are hidden and only show up during VCS operations. To find out a project’s dependencies, in theory one could dig through the repository with a special browser but this is not feasible for a large enough project.
   • Different dependencies can occur at different stages of an artifact’s life-cycle: compilation, unit testing, run-time etc. There is no way to reflect this requirement.
   • Other dependency management systems (e.g. Maven or Ivy for Ant) offer way more functionality and can be extended for additional requirements. Those customisations would have to go into hook scripts for Subversion (which, on top of things, would probably be OS-specific).
3. Quite often the dependencies will be about artifacts that are not source code at all (usually third-party libraries). You may not want to have compiled artifacts in your VCS.
4. Also, the dependency could be about source code that is maintained by an external organisation. If they are not using Subversion you could not link directly there but would have to set up a mirror internally. (Admittedly, you may want to do that anyway.)

I had used svn:externals when I started out with Subversion and have gone through quite some headaches since then because of that. Practically, most of them were around the lock-in effect. Nevertheless, I still think the conceptual argument is more important in the long run.

This is just to let you know that I am not completely gone yet. I am just extremely busy job-wise but will have time again to put something out at the end of the month. So stay tuned