Christoph Jahn

There is a great online book available for Solaris, called ‘The consolidated “Less known Solaris Features” tutorials‘. It is constantly being extended and at the time of this writing contains almost 400 (!) pages.

CollabNet are providing up-to-date binary packages of Subversion for many platforms. In my case this is CentOS 5, which by itself only has a rather dated version of Subversion. So I downloaded and installed the client, server and extras packages from CollabNet. The server package comes with a bundled Apache and a pretty nice installation script. However, I wanted to use my regular Apache for hosting the Subversion repositories, which means that I had to include the Apache modules from the CollabNet installation. So here are the respective lines from /etc/httpd/conf/httpd.confLoadModule dav_svn_module /opt/CollabNet_Subversion/modules/mod_dav_svn.so
LoadModule authz_svn_module /opt/CollabNet_Subversion/modules/mod_authz_svn.soThose modules require access to additional libraries from /opt/CollabNet_Subversion/lib, so Apache needs to be told to include this directory into the search path (LD_LIBRARY_PATH). The bold part in the below snippet from /etc/init.d/httpd shows what needs to be added:start() {
echo -n $"Starting $prog: "
check13 || exit 1
LANG=$HTTPD_LANG LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/CollabNet_Subversion/lib daemon --pidfile=${pidfile} $httpd $OPTIONS
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch ${lockfile}
return $RETVAL
}
Simply sourcing in LD_LIBRARY_PATH does not work, because the daemon function calls a separate Bash instance. The only way to feed environment variables into Apache, was by prepending them as shown above. This is also the approach to take for extending the PATH variable (which I needed to do for including /opt/CollabNet_Subversion/bin).

Sometimes people wonder why Oracle DB (both 10g and 11g) will not start after installation on Linux/Unix. In many cases the simple reason is the content of /etc/oratab. For each database it contains a line in the format $ORACLE_SID:$ORACLE_HOME:START_DB_FLAG:The last column is set to “N” by default. Just change it to “Y” and run $ORACLE_HOME/bin/dbstart again. Your database should come up now.

I have been using Ant for a lot of things in the past. Just recently, I decided to also leverage it for an important aspect of my approach to shell script programming. This had been accomplished by a combination of Perl and Bash in the past. The “old” approach is ok, except that it doesn’t integrate with Maven as well as Ant does. Also, I have the requirement to run the build process on various operating systems, incl. Windows and Unix flavours that don’t necessarily have Bash installed.

So what needs to happen during the build? A lot of things in shell script programming come up time and again. So I want to encapsulate them as functions and build a library. Sourcing libary files in at run-time is really hard to achieve, if you want to be portable (your favourite search engine has loads of information on the topic). So instead of throwing a lot of time on these sometimes very subtle problems, I decided to have include statements in the source code and pre-process the scripts during the build.

There is include as well as replace functionality in Ant in Maven. But nothing I found gave me the capability to easily retrieve the file name from the currently processed source file. Yes, it would have been possible to come up with something based on normal Ant tasks (perhaps with some support from ant-contrib), but the result would have been quite ugly. So, after some investigation, I decided to write my own Ant filter reader for that. To speed things up, I borrowed from two of the filters that come with Ant, ExpandProperties and ReplaceTokens. (Needless to say my stuff comes with the same licence terms as the rest of Ant.)

Here is how it works:

• Check the source content (in most cases coming from a file) for a regular expression. This has to match the format in which you have the include statements in your source code. So while I have set the default regex to fit my needs, you can configure it to whatever you need; just remember that Java regex handling is a bit different from e.g. Perl. This also means, that the thing can not only be used for shell scripting but any purpose that requires parametrised includes.
• Retrieve the file name from the regex match. This is done by using a regex group. By default the first group is taken, but you can also override this if needed.
• Read the include file line by line and insert them into the original content.
• Optionally, you can add prefixes and suffixes to every line. By default both are set to an empty string, which effectively switches things off. My personal use case was that I want to be able to include instructions from a file and have them marked as a comment (prefix = “# “). Probably this could have been achieved differently as well. But it was easy to implement on top of the rest and makes things more comfortable for me.

Now, let’s look at an example Ant build file:

<?xml version="1.0"?>
<project name="TestAntTools" basedir="." default="" >

<property name="base.dir" value="../../.."/>
<property name="lib.dir" value="${base.dir}/target"/>

<property name="test.file"
   value="${base.dir}/src/test/resources/main_file.txt"/>

<typedef resource="antlib.xml">
  <classpath>
    <pathelement location="${lib.dir}/ant-0.0.1.jar" />
  </classpath>
</typedef>

<!-- =================================
target: TestProcessIncludes
================================= -->
<target name="TestProcessIncludes" depends=""
    description="Test the filter processIncludes">
  <loadfile srcfile="${test.file}" property="test.file.processed">
    <filterchain>
      <processIncludes>
        <setting key="prefix" value=""/>
        <setting key="suffix" value=""/>
        <searchdir value="."/>
        <searchdir value="../resources"/>
      </processIncludes>
    </filterchain>
  </loadfile>
  <echo>${test.file.processed}</echo>
</target>

</project>

The main file has this content:

# !/bin.bash

# This is before include

##include "incl_file.txt"

# This is after include

And the included file looks like this:
# ===========================
#   This was included
# ===========================

The result from the Ant script above:
Buildfile: /home/XXXXX/workspace/AntTools/src/test/ant/build.xml
TestProcessIncludes:
[echo] # !/bin.bash
[echo]
[echo] # This is before include
[echo]
[echo] # ===========================
[echo] #   This was included
[echo] # ===========================
[echo]
[echo] # This is after include
BUILD SUCCESSFUL
Total time: 142 milliseconds

As you can see, nothing special here. To download the jar containing the filter, please click here. You will need to rename the file upon saving to end up with a proper extension (Wordpress does not allow to save attachments with .jar as extension). And for those intersted, here is the Eclipse project with source code etc.

When you are running Linux (or any other Unix-like OS) on a mobile computer, you may have a bit of a problem with mounting shares from a server. On a normal PC with a fixed connection to the server, you would simply add the appropriate entries to /etc/fstab and have them mounted automatically during startup. But since I am not “always on”, this was not an option. So I added the “noauto” option to the entries in /etc/fstab, which effectively means that they get ignored during system startup. Of course I could later call mount /my/mountpoint manually for each entry. But this is cumbersome. So I went and wrote a small script (download here). For security reasons the extension of the script is set to .sh.txt instead of .sh, so you will need to rename it before execution.

There are a few conditions for the script to work:

• Effectively the script will call “mount /my/mountpoint” for each relevant entry in /etc/fstab. All configuration details for the actual mount operation need to be provided in /etc/fstab. So before running the script, make sure those manual calls go through without problems.
• For CIFS mounts the credentials should be provided using the “credentials” option from mount.cifs (see respective man page for details). Those can then be protected properly. And should there be several people using the system, they can maintain their passwords by themselves.
• Currently there is connectivity check to the server(s). So you will get delays/errors when calling the script without a proper connection.

The script will detect whether a particular mount point is in use and skip the respective entry for mounting. The same goes for umount when the mount point is not used.

For CIFS mounts you will most likely need to call the script with sudo. At least on Fedora 12 I could not get mount.cifs to work for normal users, event with the SETUID bit set.

Here is a number of links to resources I found useful.

• There is a great (and free!) book available called Pro Git. At the very least you should read the chapters 2 (basic operation) and 3 (branching and merging). Although I had worked for almost a year with Git before coming across this book, it greatly improved my understanding. I especially liked the great visuals that make it pretty easy to grasp things quickly. An absolute must-read!
• Git tutorial: A nice reference for quickly looking up things
• Setting up Apache for Git push via HTTP(S)
• Installing Git from source on Debian
• Setting up SourceGear DiffMerge with Git
• Git casts
• Git community book

I just upgraded my OpenSolaris box from 2009.06 (snv_111b) to the latest development build (snv_131 from January 14th, 2010). The main reason was the broken CIFS server that comes with 2009.06. And although the next release is pretty much around the corner (2010.02), I needed the beast in a working state now.

The good thing about such upgrades is that thanks to ZFS and its snapshot functionality, a roll-back is trivial. The package manager will automatically create the snapshot and also create a new entry in the Grub menu for the upgraded OS. So if things go badly wrong, you can just select the old version and continue from there.

The upgrade itself is a three-step process

• Change repository URL from release to development builds
• Upgrade package manager
• Upgrade other packages

The URL change for the repository is done with the following command
$ pfexec pkg set-publisher -O http://pkg.opensolaris.org/dev opensolaris.org
The update of the package manager is done like that
$ pfexec pkg install SUNWipkg
To start the upgrade from a shell, run this command
$ pfexec pkg image-update
If you want the GUI version, please execute
$ gksu packagemanagerand press the “Update All” button.

Part of the upgrade (bug or feature?) is that the permissions of /dev/ptmx will be changed. The result is that shell access via an SSH login is not possible any more. Since my box is headless, SSH is the only way to work on it. So it’s a real problem. Fortunately, it is only the shell access that is blocked (and this not only via SSH but universally) and giving the error message:
PTY allocation request failed on channel 0

The execution of commands via SSH is still working. So you just have to issue the following command:
$ ssh -l user host "pfexec chmod 666 /dev/ptmx"

which will fix the problem. It seems that for people working directly on the machine, the solution is to blindly enter the command from above.

Interesting video from Google Tech Talks. It not only covers what you might anticipate from the title. Instead a good portion is actually about how to run an open source project in the first place.

If you want to host a Git repository there should normally be no shell access for all the people that need access to the repositories. So far many people have used gitosis to achieve this. Now there is a “new kid on the block”, called gitolite. I have not really used it in practice so far, but the added functionality looks promising and I also like the fact that it’s written in Perl. There is also a chapter about it in the Pro Git book.

The file system type for the mount command has been changed from smbfs to cifs. So using “-t smbfs” will not work any more. Instead you need to enter something like this:

mount -t cifs -o username=user[,uid=500,gid=500] //machine/share[/path] /mountpoint

There seems to be a documentation bug in the man pages, because they still mention smbfs.