Java Ninja Chronicles By Norris Shelton

Things I learned in the pursuit of code

We’ve all been there. Hot deploys are so convenient, but they consume memory via ClassLoader leakage. You eventually run out of memory and your server dies.

java.lang.OutOfMemoryError: PermGen space

Tomcat leak prevention

Tomcat helped us a lot when it came out with it’s memory leak prevention. This has been our mainstay tool since Tomcat 6. At times, even this wasn’t enough to prevent dreaded permgen errors. http://wiki.apache.org/tomcat/MemoryLeakProtection

We have all seen a log message similar to the following, thanks to Tomcat:

02-Dec-2015 14:50:41.197 WARNING [RMI TCP Connection(6)-127.0.0.1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [bonus-services] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

Java 8 says goodbye to PermGen

Java 8 helped a lot by removing the limits of permgen by opening up all of the meta space. This wasn’t as much a fix as it was to make the memory container much larger. Even this has it’s limits.

java.lang.OutOfMemoryError: Metadata space

Mattias Jiderhamn’s ClassLoader Leak Prevention library

Enter the https://github.com/mjiderhamn/classloader-leak-prevention library. This does many of the things that Tomcat does along with some others. It’s uses an Apache 2 license. There is a Servlet Context Listener that listens for the context creation and context destruction. This allows it to perform it’s work, all for your benefit.

It is very easy to integrate into your webapp. The first step is to add the Maven dependency.

<dependency>
    <groupId>se.jiderhamn</groupId>
    <artifactId>classloader-leak-prevention</artifactId>
    <version>1.15.2</version>
</dependency>

Then you need to add the listener as the first listener in your web.xml.

<listener>
    <description>https://github.com/mjiderhamn/classloader-leak-prevention</description>
    <listener-class>se.jiderhamn.classloader.leak.prevention.ClassLoaderLeakPreventor</listener-class>
</listener>

When your webapp starts up, you will see the following as logging:

ClassLoaderLeakPreventor: Settings for se.jiderhamn.classloader.leak.prevention.ClassLoaderLeakPreventor (CL: 0xcb6c98f):
ClassLoaderLeakPreventor:   stopThreads = true
ClassLoaderLeakPreventor:   stopTimerThreads = true
ClassLoaderLeakPreventor:   executeShutdownHooks = true
ClassLoaderLeakPreventor:   threadWaitMs = 5000 ms
ClassLoaderLeakPreventor:   shutdownHookWaitMs = 10000 ms
ClassLoaderLeakPreventor: Initializing context by loading some known offenders with system classloader

When you hot deploy your webapp, you will see logging similar to the following:

ClassLoaderLeakPreventor: se.jiderhamn.classloader.leak.prevention.ClassLoaderLeakPreventor shutting down context by removing known leaks (CL: 0xcb6c98f)
ClassLoaderLeakPreventor: Looping 5 RMI Targets to find leaks
ClassLoaderLeakPreventor: Looping 5 RMI Targets to find leaks
ClassLoaderLeakPreventor: Internal registry of java.beans.PropertyEditorManager not found
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Prototype beans currently in creation with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transactional resources with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transaction synchronizations with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Prototype beans currently in creation with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transactional resources with value null will be made stale for later expunging from Thread[RMI TCP Connection(idle),5,RMI Runtime]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transaction synchronizations with value null will be made stale for later expunging from Thread[RMI TCP Connection(idle),5,RMI Runtime]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Prototype beans currently in creation with value null will be made stale for later expunging from Thread[RMI TCP Connection(idle),5,RMI Runtime]
ClassLoaderLeakPreventor: Since Java 1.6+ is used, we can call public static final void java.util.ResourceBundle.clearCache(java.lang.ClassLoader)
ClassLoaderLeakPreventor: Releasing web app classloader from Apache Commons Logging

Notice that it is using System.out to log because Logging libraries are common causes of leaked ClassLoaders.

JVM settings

I picked this up from Michael Barnes. You will need to specify the following to make the garbage collector work correctly.

-XX:+UseG1GC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
  • -XX:+UseG1GC – tells the JVM to use the Garbage First Garbage Collector. This is a new GC for Java 8.
  • -XX:+CMSClassUnloadingEnabled – Tells the GC that class unloading is enabled.
  • -XX:+CMSPermGenSweepingEnabled – Tells the GC that it should enable permanent generation sweeping. Note, this the JVM will give you a message stating that this is unnecessary. Ignore the message, because it will not work without it.

Note that two of the settings are for the Concurrent Mark and Sweep garbage collector. It isn’t documented, but these flags do indeed work.

December 3rd, 2015

Posted In: Java, java ninja, Javaninja

Tags: , , , , , , , , ,

Leave a Comment

I was loading a java resource via the classloader. I had the need to know where the file was on the file system.

    /**
     * Gets the absolute path of a file loaded via a classloader.
     * @param classpathFile file to be loaded from the classpath.
     * @return absolute file path of the file
     * @throws URISyntaxException
     */
    public static String getAbsoluteFilePath(String classpathFile) throws URISyntaxException {
        String absoluteFilePath = "";
        URL url = CommonUtil.class.getResource(classpathFile);
        if (url != null) {
            Path path = Paths.get(url.toURI());
            if (path != null) {
                absoluteFilePath = path.toString();
            }
        }
        return absoluteFilePath;
    }

February 14th, 2014

Posted In: Java, java ninja, Javaninja

Tags: , , , , , , , , ,

Leave a Comment

WP to LinkedIn Auto Publish Powered By : XYZScripts.com