Getting J to Run with Your Shared Libraries

This article is targeted primarily at Jugglers on Linux who have problems to get J running on their system because they lack one of the right shared libraries. SuSE Linux users need to know this information to get J running.

Nota bene: The material here has nothing to do with the 15!:0 stuff in J. Well, little.

Background

Up to version 4.01, we provided both so-called statically and dynamically linked executables of for J for most of the platforms.

"Statically linked" means that all the code for fundamental operations like opening a file or exponentiating a floating point number are copied from the libraries on our development systems into the executable J program we send out.

"Dynamically linked" means that such routines will be taken from the system libraries when J starts up. The executable knows the entry points it needs, but does contain the actual code. The big advantage of this system is that programs will be smaller. The savings are in particular significant if one considers this use of shared libraries in the context of the entire system. At the same time, a fix in a shared library is of immediate benefit to all programs using it, even without recompiling them all.

There are pros and cons for both approaches. The upshot is that

1. statically linked programs are larger then dynamically linked ones,
2. dynamically linked programs are smaller and benefit from, say, an improvement to the exponentiation routines in your libraries.
3. dynamically linked programs require that you have the right libs in place when you run the program.

Up to J 4.01, we used to make statically linked binaries because they are so foolproof. They might be big, but at least they are completely independent from your local setup.

In J 4.02, we made an experiment just to provide the dynamically linked J executables. There are two reasons behind it:

1. On some platforms, static linking prevents any dynamic loading at runtime (as required for 15!:0).
2. J needs only "fairly standard" shared libraries. (Whatever may be non-standard, is statically linked in.)

BUT SURPRISE!

There appears to be more variety in the Linux world than we thought. It turns out that Debian Linux (which is used for some of the Linux builds) does some things differently with respect to the GNU readline library J is using.

Problems and Fixes

Here is a list of typical problems with dynamically linked programs (such as J) and non-matching shared libraries (such as on your system ;-).

Before you do anything, get a picture what is actually happening and installed on your systems. It might be a bit tough the first time you do it, but it's worth the trouble in the long run.

ldconfig -p or some variation can show you how linkage requests will be resolved when there's a big mess of different lib versions in different directories.

Depending on the platform, the shared libs will be loaded by ld(1), ld.so(1), or rtld(1). READ THE FINE MANUAL in order to understand how YOUR system searches for shared libs.

Libraries commonly hide in the directorries /lib, /usr/lib, and /usr/local/lib and have starting with "lib". Common suffixes are ".a" for static libs and -- more important -- ".so" for shared libs. So a typical example would be /lib/libreadline.so.2.1. Platforms differ, though. For example, AIX uses ".a" for shared libs, too. If in doubt, file(1) is your friend. Also, watch out about symbolic links.

ldd /usr/local/bin/jint-402a will tell you shared libraries J would like to find on your system.

Version number discrepancies

Effect: When you try to fire up J, the system complains about a wrong shared lib version number. This may also happen in th for of: libreadline.so.3: file not found.

Fix 1: Check out the CD with your Linux distribution and see what's there. Check out external up-to-date archives.

On most platforms, shared libraries are provided with major and (optionally) minor version numbers. This can be recognized by the file name, such as /lib/libreadline.so.2.1 . The theory is that increments in the minor number are just compatible improvements.

Good shared library systems will do some intelligent search for the most appropriate shared lib to use. Fix 2: On some systems, setting an environment variable such as LD_SUPPRESS_WARNINGS or LD_NOWARN can shut up harmless warnings. (A good place to set it would be the "j" and "jrun" wrappers.) Again, nothing but RTFMing can help you.

Changes in the major number reflect bigger changes in the library API where it may or may not be possible to mix program and lib with different major versions. Fix 3: In our experience, it often is worth a try to throw a symbolic with "ln -s" and to pretend you have the requested version. After that, it may be necessary to rerun ldconfig(8). If you are successfull, you should still regard this just as a stop-gap measure.

For example, both with SuSE-6.4 and SuSE-7.0, the following link is all you need to get j402c runnning:

	ln -s /lib/libreadline.so.4.0 /usr/lib/libreadline.so.3

Problem: Static libs only present

Starting J on a current 6.x SuSE Linux distribution will just result in a complaint that /usr/lib/libreadline.so.2 cannot be found.

That's right, these systems just come with the static version of the gnu readline library, i.e. /usr/lib/libreadline.a . The proper first approach is (Fix 1, remember? to check out the SuSE CD for perhaps another readline package, an approach which will fail. OK, what now?

You are not lost. After all, you got the code in that static lib, it just cannot be linked to J upon startup. The next attempt now is to create a shared lib from your static one. Yes, that's possible. Fix 4: The proper commands are (as root):

cd /usr/lib
ld -shared --whole-archive libreadline.a -o libreadline.so.2
ldconfig

which creates the requested library. Well, perhaps it does, because, at this point you cannot really what major version of readline was hidden in libreadline.a.

Which promptly leads us to our next error message and exercise:

Unresolved symbols

You do the above, start J, and all what you get is:

/usr/lib/libreadline.so.2: undefined symbol: tputs

The Debian Linux system used to build the packages j402a-Linux-Debian.tgz and j402b-Linux.tgz was built with a readline library version which does not require the additional use of the "termcap", "term", or "curses" libraries. This differs across platforms and I'm unable at this point to say whether this clash between Debian against SuSE is Debian's fault, SuSE's fault, or mine.

I know for sure that these Derbain-based packages run fine on RedHat and (surprise!) Debian systems. More empirical data points are most welcome.

The remedy for the problem with SuSE board equipment is as follows. A man tputs will reveal (after reading through obscure hints...) that tputs might be defined in the "termcap" library. A nm(1) on the termcap library verifies this indeed and the solution Fix 5 is thusly:

cd /usr/lib
ld -shared --whole-archive libreadline.a libtermcap.so.2 -o libreadline.so.2
ldconfig

The ld command incorporates both the old libreadline.a and the (small) termcap library into a working libreadline.so.2. Voila! And even better, J will actually run fine with this beast.

At this point, you should make a note to yourself to Fix 1 grab a true readline.2.something package in either source or binary the next time you run across one. Install it on your system so that you know you have a proper library.

Summary

Whenerver you run into such problems getting J to run, let me know. This should not happen and needs to be fixed.