Sunday, August 14, 2011

Changing Java Library Path at Runtime

The java.library.path system property instructs the JVM where to search for native libraries. You have to specify it as a JVM argument using -Djava.library.path=/path/to/lib and then when you try to load a library using System.loadLibrary("foo"), the JVM will search the library path for the specified library. If it cannot be found you will get an exception which looks like:
Exception in thread "main" java.lang.UnsatisfiedLinkError: no foo in java.library.path
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
	at java.lang.Runtime.loadLibrary0(Runtime.java:823)
	at java.lang.System.loadLibrary(System.java:1028)
The java.library.path is read only once when the JVM starts up. If you change this property using System.setProperty, it won't make any difference.

Here is the code from ClassLoader.loadLibrary which shows how the path is initialised:

if (sys_paths == null) {
    usr_paths = initializePath("java.library.path");
    sys_paths = initializePath("sun.boot.library.path");
}
As you can see from the code above, the usr_paths variable is only initialised if sys_paths is null, which will only be once.

So, how can you modify the library path at runtime? There are a couple of ways to do this, both involving reflection. You should only do this if you really have to.

Option 1: Unset sys_paths
If you set sys_paths to null, the library path will be re-initialised when you try to load a library. The following code does this:

/**
 * Sets the java library path to the specified path
 *
 * @param path the new library path
 * @throws Exception
 */
public static void setLibraryPath(String path) throws Exception {
    System.setProperty("java.library.path", path);

    //set sys_paths to null
    final Field sysPathsField = ClassLoader.class.getDeclaredField("sys_paths");
    sysPathsField.setAccessible(true);
    sysPathsField.set(null, null);
}
Option 2: Add path to usr_paths
Instead of having to re-evaluate the entire java.library.path and sun.boot.library.path as in Option 1, you can instead append your path to the usr_paths array. This is shown in the following code:
/**
* Adds the specified path to the java library path
*
* @param pathToAdd the path to add
* @throws Exception
*/
public static void addLibraryPath(String pathToAdd) throws Exception{
    final Field usrPathsField = ClassLoader.class.getDeclaredField("usr_paths");
    usrPathsField.setAccessible(true);

    //get array of paths
    final String[] paths = (String[])usrPathsField.get(null);

    //check if the path to add is already present
    for(String path : paths) {
        if(path.equals(pathToAdd)) {
            return;
        }
    }

    //add the new path
    final String[] newPaths = Arrays.copyOf(paths, paths.length + 1);
    newPaths[newPaths.length-1] = pathToAdd;
    usrPathsField.set(null, newPaths);
}

14 comments:

  1. Thank you so much! You made my day!

    ReplyDelete
  2. Thanks a lot. This solved my big problem.

    Will follow your blog from now on.

    ReplyDelete
  3. Much appreciated, sir. I don't want all of my work buddies to have to edit their vm args.

    ReplyDelete
  4. I'm going to agree with everyone else and say that this is fantastic. Nobody else seems to have figured this one out. It seems to me like this should be a standard java feature rather than a clever hack.

    ReplyDelete
  5. Thank you, great post!!

    I'll send this post to the guy in charge of javacv, I was stuck setting the right path depending on the platform.

    ReplyDelete
  6. It is not working for me as I have dependent shared libraries also to be loaded. When I keep the shared libraries in a folder and give its path in PATH env-variable only then loading successfully.

    how to resolve this issue if shared libraries are also to be placed in the same folder as my immediate native library ?

    thanx in adv.

    ReplyDelete
    Replies
    1. I am the developer of SikuliX (sikulix.com) and also have this 2-fold problem.
      I am adding the needed path to system path at runtime using the JNA package com.nativelibs4java::bridj:0.6.2 (Maven) that allows to access the Win32 API directly (GetEnvironmentVariableW/SetEnvironmentVariableW). If interested contact me. What was missing was this usrPaths solution here, to let JIntellitype load it's dll via System.loadLibrary().

      Delete
    2. Hi Raimund,
      Can you publish the code of your bridj solution?
      Thanks in advance.

      Delete
  7. this almost worked except that there was a space in the path I tried to add...e.g. C:\program files\... and the space would mess things up even when I surrounded the string with quotes...kept getting this...

    ...
    java.lang.UnsatisfiedLinkError: no javaHeclib in java.library.path
    java library path=C:\Program%20Files\mpvt\mpvt\modules\lib\amd64;
    ...
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.loadLibrary0(Unknown Source)
    at java.lang.System.loadLibrary(Unknown Source)
    ...

    Any ideas?

    ReplyDelete
    Replies
    1. You should check your path string preparation. the %20 is the problem. this usually happens, if you intermediately work with URLs instead of plain path strings.
      In doubt just finally use path = path.replaceAll("%20", " ")

      Delete
  8. This piece of information has saved my day.Thanks a lot!

    ReplyDelete
  9. really great finding and well written. thanks

    ReplyDelete
  10. Thanks for this post, here is the scala function based on your work :
    http://www.crosson.org/2014/04/javalibrarypath-change-at-runtime.html

    ReplyDelete
  11. Nice post!! saved me a lot of work!!

    ReplyDelete

Note: Only a member of this blog may post a comment.