Java Ninja Chronicles By Norris Shelton

Things I learned in the pursuit of code

We’ve all been there. Hot deploys are so convenient, but they consume memory via ClassLoader leakage. You eventually run out of memory and your server dies.

java.lang.OutOfMemoryError: PermGen space

Tomcat leak prevention

Tomcat helped us a lot when it came out with it’s memory leak prevention. This has been our mainstay tool since Tomcat 6. At times, even this wasn’t enough to prevent dreaded permgen errors. http://wiki.apache.org/tomcat/MemoryLeakProtection

We have all seen a log message similar to the following, thanks to Tomcat:

02-Dec-2015 14:50:41.197 WARNING [RMI TCP Connection(6)-127.0.0.1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [bonus-services] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

Java 8 says goodbye to PermGen

Java 8 helped a lot by removing the limits of permgen by opening up all of the meta space. This wasn’t as much a fix as it was to make the memory container much larger. Even this has it’s limits.

java.lang.OutOfMemoryError: Metadata space

Mattias Jiderhamn’s ClassLoader Leak Prevention library

Enter the https://github.com/mjiderhamn/classloader-leak-prevention library. This does many of the things that Tomcat does along with some others. It’s uses an Apache 2 license. There is a Servlet Context Listener that listens for the context creation and context destruction. This allows it to perform it’s work, all for your benefit.

It is very easy to integrate into your webapp. The first step is to add the Maven dependency.

<dependency>
    <groupId>se.jiderhamn</groupId>
    <artifactId>classloader-leak-prevention</artifactId>
    <version>1.15.2</version>
</dependency>

Then you need to add the listener as the first listener in your web.xml.

<listener>
    <description>https://github.com/mjiderhamn/classloader-leak-prevention</description>
    <listener-class>se.jiderhamn.classloader.leak.prevention.ClassLoaderLeakPreventor</listener-class>
</listener>

When your webapp starts up, you will see the following as logging:

ClassLoaderLeakPreventor: Settings for se.jiderhamn.classloader.leak.prevention.ClassLoaderLeakPreventor (CL: 0xcb6c98f):
ClassLoaderLeakPreventor:   stopThreads = true
ClassLoaderLeakPreventor:   stopTimerThreads = true
ClassLoaderLeakPreventor:   executeShutdownHooks = true
ClassLoaderLeakPreventor:   threadWaitMs = 5000 ms
ClassLoaderLeakPreventor:   shutdownHookWaitMs = 10000 ms
ClassLoaderLeakPreventor: Initializing context by loading some known offenders with system classloader

When you hot deploy your webapp, you will see logging similar to the following:

ClassLoaderLeakPreventor: se.jiderhamn.classloader.leak.prevention.ClassLoaderLeakPreventor shutting down context by removing known leaks (CL: 0xcb6c98f)
ClassLoaderLeakPreventor: Looping 5 RMI Targets to find leaks
ClassLoaderLeakPreventor: Looping 5 RMI Targets to find leaks
ClassLoaderLeakPreventor: Internal registry of java.beans.PropertyEditorManager not found
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Prototype beans currently in creation with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transactional resources with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transaction synchronizations with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Prototype beans currently in creation with value null will be made stale for later expunging from Thread[http-nio-8080-exec-1,5,main]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transactional resources with value null will be made stale for later expunging from Thread[RMI TCP Connection(idle),5,RMI Runtime]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Transaction synchronizations with value null will be made stale for later expunging from Thread[RMI TCP Connection(idle),5,RMI Runtime]
ClassLoaderLeakPreventor: Custom ThreadLocal of type org.springframework.core.NamedThreadLocal: Prototype beans currently in creation with value null will be made stale for later expunging from Thread[RMI TCP Connection(idle),5,RMI Runtime]
ClassLoaderLeakPreventor: Since Java 1.6+ is used, we can call public static final void java.util.ResourceBundle.clearCache(java.lang.ClassLoader)
ClassLoaderLeakPreventor: Releasing web app classloader from Apache Commons Logging

Notice that it is using System.out to log because Logging libraries are common causes of leaked ClassLoaders.

JVM settings

I picked this up from Michael Barnes. You will need to specify the following to make the garbage collector work correctly.

-XX:+UseG1GC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
  • -XX:+UseG1GC – tells the JVM to use the Garbage First Garbage Collector. This is a new GC for Java 8.
  • -XX:+CMSClassUnloadingEnabled – Tells the GC that class unloading is enabled.
  • -XX:+CMSPermGenSweepingEnabled – Tells the GC that it should enable permanent generation sweeping. Note, this the JVM will give you a message stating that this is unnecessary. Ignore the message, because it will not work without it.

Note that two of the settings are for the Concurrent Mark and Sweep garbage collector. It isn’t documented, but these flags do indeed work.

December 3rd, 2015

Posted In: Java, java ninja, Javaninja

Tags: , , , , , , , , ,

Leave a Comment

Streams, functional programming and Lambda expressions have opened up a whole new world of syntax for the Java language. The old forEach has new competition.

An example of a normal forEach of a list of objects was:

for (MyModel myModel : myModelList) {
    System.out.println(myModel);
}

This can be replaced with .forEach and the code gets much smaller.

myModelList.forEach(System.out::println);

One common thing to do with a forEach is to iterate over a set of data and pick and choose which objects should be added to another list.
An example of a forEach that filters a list is:

List<GameModel> externalGames = new LinkedList<>();
for (GameModel myModel : gameList) {
    if ("external".equals(myModel.getIdGameExternal())) {
        externalGames.add(myModel);
    }
}

This is when it gets fun. We can use .filter and .collect to transform this into the following. The .filter contains all of the logic that you would have had in an if statement. The .collect contains the processing necessary to convert the stream to a LinkedList.

externalGames = gameList.stream()
                        .filter(myModel -> "external".equals(myModel.getIdGameExternal()))
                        .collect(Collectors.toCollection(LinkedList::new));

Sometimes you need just one element from a list. Here is what a typical forEach loop looks like that filters a list down and then we get a reference to the first object in the list.

List<GameModel> externalGames = new LinkedList<>();
for (GameModel myModel : gameList) {
    if ("external".equals(myModel.getIdGameExternal())) {
        externalGames.add(myModel);
    }
}
gameModel1 = externalGames.get(0);

We can easily do the same thing using streams. In this case, I filter like before, then grab the first element. We pull out .findFirst to retrieve the first element of the stream, then use .get to make that object available.

gameModel1 = gameList.stream()
                     .filter(myModel -> "external".equals(myModel.getIdGameExternal()))
                     .findFirst()
                     .get();

Sometimes you have the case where you want to search through a collection of data and if an element contains a specific value, set something else. In this case, we use findFirst() and isPresent() to set a boolean value based upon an element in the collection having a specific value.

inBonus = bonusRedemptionModel.getBonusTransactions()
                              .stream()
                              .filter(b -> BonusOperationEnum.OPT_OUT.value().equals(b.getIdOperation()))
                              .findFirst()
                              .isPresent();

August 6th, 2015

Posted In: Collections, Java, java ninja, Javaninja

Tags: , , , , ,

Leave a Comment

We normally use @Transactional to declaratively control transactions. Sometimes, that doesn’t quite give us the flexibility we need. I had a case where I needed the ability to write data to a database within a transaction, but I couldn’t use an exception to indicate a rollback was necessary because there was information within the method that I would need. Specifically, I needed the information related to the failure. In order for this to work. I had to use a Springframework TransactionTemplate. Like most Spring Template objects, it’s fairly easy to use. Note that an exception will cause an automatic rollback, just like the normal transactional processing.

First, you need to declare the TransactionManager and the TransactionTemplate. In this case, my transaction manager gets it’s database connection from an EntityManagerFactory because I am using JPA.

    <bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"
          p:entityManagerFactory-ref="entityManagerFactory"/>

    <bean id="transactionTemplate" class="org.springframework.transaction.support.TransactionTemplate"
          p:transactionManager-ref="transactionManager"/>

Once the TransactionTemplate is declared, it can be injected like any other spring bean.

    @Autowired
    private TransactionTemplate transactionTemplate;

A TransactionTemplate exposes it’s functionality via a callback method. One implementation does not have a return type and the other has a return type that can be expressed as a generic.

Here is an example of using the callback method that doesn’t have a return type. This is implemented by implementing an anonymous inner-class of type TransactionCallbackWithoutResult. This is an abstract class with one method, doInTransactionWithoutResult. The TransactionStatus object is used to rollback the transaction. This is caused by calling setRollbackOnly().

transactionTemplate.execute(new TransactionCallbackWithoutResult() {
    @Override
    protected void doInTransactionWithoutResult(TransactionStatus status) {
        // do this
        // do that
        // if some sort of state exists
        status.setRollbackOnly();;
        // }
    }
});

If you need to return data from inside the callback method, then use the TransactionCallback. This works like the other callback method, but allows a value to be returned. Note that the Boolean return type is a generic and can be specified as whatever you need it to be.

boolean returnValue = transactionTemplate.execute(new TransactionCallback<Boolean>() {
    @Override
    public Boolean doInTransaction(TransactionStatus status) {
        // do this
        // do that
        // if some sort of state exists
        status.setRollbackOnly();
        // }
        return myObject.getMyBoolean();
    }
});

If you need a return value and you are on JDK8, you can use a Lambda expression to make the code even smaller.

boolean returnValue = transactionTemplate.execute(status -> {
    // do this
    // do that
    // if some sort of state exists
    status.setRollbackOnly();
    // }
    return null;
});

August 5th, 2015

Posted In: Java, java ninja, Javaninja, Spring

Tags: , , , , ,

Leave a Comment

WP to LinkedIn Auto Publish Powered By : XYZScripts.com