Java Ninja Chronicles By Norris Shelton

Things I learned in the pursuit of code

I am using a Springframework project and had a method that had been cached before, but stopped working after being refactored. The method cached application properties that were being read from a webdav repository. I noticed while debugging some other production code that the properties file was being read on every request.

    I verified the following:

  • The class was an annotated spring bean
    @Repository
  • The method was annotated as a cacheable method
    @Cacheable(“GlobalPropertiesDao.getProperties”)
  • The method was public
  • The package was being scanned for components
    )
  • The spring context was configured for annotated cache beans
  • Ehcache was configured for caching
        <bean id="ehcache" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean" p:shared="true"/>
        <bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheCacheManager" p:cacheManager-ref="ehcache"/>
    
  • Verified that there was an ehcache.xml that was being placed in the root of my classpath
  • Verified that the specified name of my cache was in the ehcache.xml
        <cache name="GlobalPropertiesDao.getProperties" maxElementsInMemory="1000" eternal="false" overflowToDisk="false"
               timeToIdleSeconds="300" timeToLiveSeconds="300"/>
    

Still the method was not being cached. While reading through the Spring documentation, I ran across the following paragraph.

Note
In proxy mode (which is the default), only external method calls coming in through the proxy
are intercepted. This means that self-invocation, in effect, a method within the target object
calling another method of the target object, will not lead to an actual caching at runtime even
if the invoked method is marked with @Cacheable – considering using the aspectj mode in
this case.

Doh!!!! That was it. I provide a method that looks for the property within the properties and it automatically looks for the property that is defined for this environment (property.env=…) and the property without an environment (property=…). The class was composed of 2 methods. One that looks for the property and one that reads the property file. I chose to cache the reading of the properties file so that the webdav would take the file read hit at most once every 5 minutes. Had I chosen to cache the individual properties, the webdav could take the file read hit once per property every 5 minutes.

I started with the following class:

package com.appriss.jxp.properties;

import com.appriss.justicexchange.commons.http.HttpUtils;
import com.appriss.justicexchange.commons.utility.Utility;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Repository;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.Properties;

import static com.appriss.core.Environment.*;


/**
 * JX Property Dao.
 * @author jmiller Jan 19, 2012 12:58:26 PM
 * @author Norris Shelton
 */
@Repository
public class PropertiesDao {

    private static final Logger log = LoggerFactory.getLogger(PropertiesDao.class);

    /**
     * Gets a JX Portal property.  It first tries to find the property with the environment as the suffix.  If not
     * found, it will look for the property without the environment.
     * @param key the key of the property
     * @return the value of the property
     * @throws Exception if the property is not found
     */
    public String getProperty(String key) throws IOException {
        String property = getProperties().getProperty(key + '.' + getEnvironment().toString());
        if (StringUtils.isBlank(property)) {
            property = getProperties().getProperty(key);
            if (StringUtils.isBlank(property)) {
                throw new IOException("Global property not found " + key + '.' + getEnvironment().toString() + " or " + key);
            }
        }
        return property;
    }

    /**
     * Reads the properties from the webdav for this environment.
     * @return Properties object
     */
    @Cacheable("GlobalPropertiesDao.getProperties")
    public Properties getProperties() throws IOException {
        Properties properties = new Properties();
        String response = HttpUtils.getResponse(Utility.getWebdavServer() + "/product/jxportal/config/jxportal.properties");
        if (StringUtils.isNotBlank(response)) {
            //noinspection ConstantConditions
            properties.load(new ByteArrayInputStream(response.getBytes()));
        }
        return properties;
    }
}

The getProperties method on line 54 was being called by the getProperty method on line 41. That was my problem. I ended up splitting up the code into 2 classes. Here are the 2 classes.

package com.appriss.jxp.properties;

import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.io.IOException;

import static com.appriss.core.Environment.*;


/**
 * JX Property Service class.
 * @author Norris Shelton
 */
@Component
public class PropertiesService {

    @Autowired
    private PropertiesDao propertiesDao;

    /**
     * Gets a JX Portal property.  It first tries to find the property with the environment as the suffix.  If not
     * found, it will look for the property without the environment.
     * @param key the key of the property
     * @return the value of the property
     * @throws IOException if the property is not found
     */
    public String getProperty(String key) throws IOException {
        String property = propertiesDao.getProperties().getProperty(key + '.' + getEnvironment().toString());
        if (StringUtils.isBlank(property)) {
            property = propertiesDao.getProperties().getProperty(key);
            if (StringUtils.isBlank(property)) {
                throw new IOException("Global property not found " + key + '.' + getEnvironment().toString() + " or " + key);
            }
        }
        return property;
    }
}
package com.appriss.jxp.properties;

import com.appriss.justicexchange.commons.http.HttpUtils;
import com.appriss.justicexchange.commons.utility.Utility;
import org.apache.commons.lang3.StringUtils;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Repository;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.Properties;

/**
 * Properties DAO cache object.
 * NOTE: This is the real cache.  It had to be made into another class because Spring will not proxy a proxied object.
 * Meaning that the cached method was being called by another method in the class and this prevented the properties
 * file from being cached.  It is better to cache the whole properties file instead of each property.  This leads to
 * one read every n minutes instead of x reads every n minutes.
 * <p>Note
 In proxy mode (which is the default), only external method calls coming in through the proxy
 are intercepted. This means that self-invocation, in effect, a method within the target object
 calling another method of the target object, will not lead to an actual caching at runtime even
 if the invoked method is marked with @Cacheable - considering using the aspectj mode in
 this case.</p>
 */
@Repository
public class PropertiesDao {

    /**
     * Reads the properties from the webdav for this environment.
     * @return Properties object
     */
    @Cacheable("GlobalPropertiesDao.getProperties")
    public Properties getProperties() throws IOException {
        Properties properties = new Properties();
        String response = HttpUtils.getResponse(Utility.getWebdavServer() +
                                                "/product/jxportal/config/jxportal.properties");
        if (StringUtils.isNotBlank(response)) {
            //noinspection ConstantConditions
            properties.load(new ByteArrayInputStream(response.getBytes()));
        }
        return properties;
    }

}

Finally, problem solved.

February 26th, 2013

Posted In: Ehcache, Java, Spring

Tags: , , , ,

Leave a Comment

We had an existing Springframework web application that needed to handle images being stored in AmazonWS. Here are the changes taht I needed to make to add the new functionality.

  1. Add the AmazonWS Maven dependency
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk</artifactId>
                <version>1.3.30</version>
            </dependency>
    
  2. Create the files that contain the AmazonWS credentials. Ours are stored in a webdav repository and we have different credentials per environment. To differentiate the files, I used the naming convention of amazon-.properties. The contents of the files were:
    # Fill in your AWS Access Key ID and Secret Access Key
    # http://aws.amazon.com/security-credentials
    accessKey=<nunya bidness>
    secretKey=<also nunya bidness>
    
  3. I created a Spring java configuration class.. Since our files are stored on a webdav and there is a webdav for each environment, I need to jump through some hoops in order to get the properties file.
    package com.appriss.jxp.amazon;
    
    import com.amazonaws.auth.PropertiesCredentials;
    import com.amazonaws.services.s3.AmazonS3Client;
    import com.appriss.justicexchange.commons.http.HttpUtils;
    import com.appriss.justicexchange.commons.utility.Environment;
    import com.appriss.justicexchange.commons.utility.Utility;
    import org.apache.commons.io.IOUtils;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    
    import java.io.IOException;
    
    /**
     * Spring java configuration class for AmazonWS.
     * @author nshelton
     */
    @Configuration
    public class AmazonConfiguration {
    
        /**
         * Creates an Amazon S3 Client.
         * @return AmazonS3Client
         */
        @Bean
        public AmazonS3Client amazonS3Client() throws IOException {
            PropertiesCredentials propertiesCredentials = new PropertiesCredentials(
                IOUtils.toInputStream(
                    HttpUtils.getResponse(
                        Utility.getWebdavServer() + "/product/jxportal/config/amazon-" + Environment.getValue() + ".properties")));
            return new AmazonS3Client(propertiesCredentials);
        }
    }
    
  4. The Junit test class for this is very simple. Did it create a spring bean of the desired type.
    package com.appriss.jxp.amazon;
    
    import com.amazonaws.services.s3.AmazonS3Client;
    import org.junit.Test;
    import org.junit.runner.RunWith;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.test.context.ContextConfiguration;
    import org.springframework.test.context.TestExecutionListeners;
    import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
    import org.springframework.test.context.support.DependencyInjectionTestExecutionListener;
    
    import static org.junit.Assert.*;
    
    /**
     * Tests the Amazon configuration class.
     * @author nshelton
     */
    @RunWith(SpringJUnit4ClassRunner.class)
    @TestExecutionListeners({DependencyInjectionTestExecutionListener.class})
    @ContextConfiguration
    public class TestAmazonConfiguration {
    
        @Autowired
        private AmazonS3Client amazonS3Client;
    
        @Test
        public void testAmazonS3Client() {
            assertNotNull(amazonS3Client);
        }
    }
    
  5. We are going to use AmazonWS to create temporary URLs that will expire after a set amount of time. The timeout value is stored in another file on the webdav and is fronted by a PropertiesDao class that caches the value for a set amount of time so it will not bombard the webdav with too many requests, yet allows it to be changed at runtime. I extracted this logic into a method in a utility class.
    package com.appriss.jxp.amazon;
    
    import com.amazonaws.HttpMethod;
    import com.amazonaws.services.s3.AmazonS3Client;
    import com.amazonaws.services.s3.model.GeneratePresignedUrlRequest;
    import com.appriss.jxp.properties.PropertiesDao;
    import org.joda.time.MutableDateTime;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.stereotype.Component;
    
    import java.io.IOException;
    
    /**
     * Amazon utility class.
     * @author nshelton
     */
    @Component
    public class AmazonUtility {
    
        @Autowired
        private PropertiesDao propertiesDao;
    
        @Autowired
        private AmazonS3Client amazonS3Client;
    
        /**
         * Gets an AmazonWS url for the given resource.
         * @param bucket the value of the AmazonWS bucket
         * @param siteId the siteId of this request
         * @param s3FileName the name of the S3 resource to generate an expiring URL for
         * @return the expiring AmazonWS URL for the requested resource
         * @throws IOException
         */
        public String getUrl(String bucket, String siteId, String s3FileName) throws IOException {
            GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(bucket + siteId,
                                                                                                      s3FileName,
                                                                                                      HttpMethod.GET);
            MutableDateTime mutableDateTime = new MutableDateTime();
            mutableDateTime.add(Long.parseLong(propertiesDao.getProperty("amazonws.timeout")));
            generatePresignedUrlRequest.setExpiration(mutableDateTime.toDate());
    
            return amazonS3Client.generatePresignedUrl(generatePresignedUrlRequest).toString();
        }
    }
    
  6. Just to make things fun, where I need to use this is legacy code and doesn’t know anything about Springframework. The desired logic is for it to use the AmazonWS S3 to create URLs if the image has been stored there and IF that URL resolution has been turned on. It had been requested to make it possible for the use of AmazonWS to be turned off if the service isn’t responsive enough.
        public String getFormattedSigImageURL() {
            String formattedSigImageURL = null;
            // if I have the information needed to construct an AmazonWS URL
            if (StringUtils.isNotBlank(s3SignatureBucket) &amp;&amp; StringUtils.isNotBlank(s3ImageName)) {
                PropertiesDao propertiesDao = CommonTools.getBean(PropertiesDao.class);
                try {
                    // if the AmazonWS service is turned on
                    if ("true".equals(propertiesDao.getProperty("amazonws.enabled"))) {
                        AmazonUtility amazonUtility = CommonTools.getBean(AmazonUtility.class);
                        formattedSigImageURL = amazonUtility.getUrl(s3SignatureBucket, pharmacySiteId, s3ImageName);
                    }
                } catch (IOException ignore) {}
            }
    
            // not found, try another way
            if (StringUtils.isBlank(formattedSigImageURL)) {
                try {
                    formattedSigImageURL = CommonTools.getImageURL(sigImageURL);
                } catch (NamingException ignore) {
                    formattedSigImageURL = sigImageURL;
                }
            }
    
            return formattedSigImageURL;
        }
    

February 15th, 2013

Posted In: Java, JUnit, Maven, Spring

Leave a Comment

All of my Springframework projects were configured via the context xml files. I know they support configuration by @Configuration annotated java classes. I never used it, because I never had a need. The context file is simple and easy to understand. It also gives you one place to go to track down classes.

I have a properties file that sits on a webdav. Unfortunately, there is one per environment and I have to run some java code to determine which environment I am in before I can retrieve the file.

  • @Configuration Indicates that a class declares one or more @Bean methods and may be processed by the Spring container to generate bean definitions and service requests for those beans at runtime
  • @Bean Indicates that a method produces a bean to be managed by the Spring container.
  • jxportal() The default strategy for determining the name of a bean is to use the name of the Bean method
package com.appriss.justicexchange.springextension.configuration;

import com.appriss.justicexchange.commons.http.HttpUtils;
import com.appriss.justicexchange.commons.utility.Utility;
import org.apache.commons.io.IOUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.io.IOException;
import java.util.Properties;

/**
 * Spring java configuration to instantiate webdav objects.
 * @author nshelton
 */
@Configuration
public class WebdavConfig {
    private static Logger log = LoggerFactory.getLogger(WebdavConfig.class);

    /**
     * Produces the jxportal.properties object from the webdav.
     * @return Properties object with jxportal.properties as it's contents
     */
    @Bean
    public Properties jxportal() {
        Properties webdav = new Properties();
        try {
            webdav.load(
                IOUtils.toInputStream(
                    HttpUtils.getResponse(Utility.getWebdavServer() + "/product/jxportal/config/jxportal.properties")));
        } catch (IOException e) {
            log.error("Unable to load /product/jxportal/config/jxportal.properties", e);
        }
        return webdav;
    }
}

February 12th, 2013

Posted In: Java, Spring

Leave a Comment

I’m going to use Java Generics to refactor a utility to make it more versatile. I wrote a utility method that used httpclient to retrieve a text file for me. Then I had a need to retrieve a binary file. I ended up with two methods that were very similar.

package com.appriss.justicexchange.commons.http;

import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;

/**
 * Http utilities.
 * @author nshelton
 * @since HttpClient 4.0
 */
public class HttpUtils {

    private static final Logger log = LoggerFactory.getLogger(HttpUtils.class);

    private HttpUtils() {}

    /**
     * Returns the http response as a string.
     * @param url the url to request
     * @return String representation of the response
     * @throws IOException
     */
    public static String getResponse(String url) throws IOException {
        log.debug("Reading: {}", url);
        String response = "";
        HttpClient httpclient = new DefaultHttpClient();
        try {
            HttpGet httpget = new HttpGet(url);
            ResponseHandler<String> responseHandler = new BasicResponseHandler();
            response = httpclient.execute(httpget, responseHandler);
        } catch (IOException e) {
            log.error("IOException getting http response", e);
            throw e;
        } finally {
            httpclient.getConnectionManager().shutdown();
        }
        return response;
    }

    /**
     * Returns the http response as a byte[].
     * @param url the url to request
     * @return byte[] representation of the response
     * @throws IOException
     */
    public static byte[] getResponseBytes(String url) throws IOException {
        log.debug("Reading {}", url);
        byte[] response;
        HttpClient httpclient = new DefaultHttpClient();
        try {
            HttpGet httpget = new HttpGet(url);
            ResponseHandler<byte[]> responseHandler = new ByteResponseHandler();
            response = httpclient.execute(httpget, responseHandler);
        } catch (IOException e) {
            log.error("IOException getting http response", e);
            throw e;
        } finally {
            httpclient.getConnectionManager().shutdown();
        }
        return response;
    }
}

I am a DRY advocate so I started thinking of how I could reduce the amount of code I needed. I could see that the code was very similar, but the types were not the same. I thought about generics and the answer was pretty easy. This was made easy because the ResponseHandler object is already a generic. I use generics a lot, but don’t have that many opportunities to write generic methods. Now that it is refactored, I can add another type with just a few lines.

package com.appriss.justicexchange.commons.http;

import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;

/**
 * Http utilities.
 * @author nshelton
 * @since HttpClient 4.0
 */
public class HttpUtils {

    private static final Logger log = LoggerFactory.getLogger(HttpUtils.class);

    private HttpUtils() {}

    /**
     * Returns the http response as a string.
     * @param url the url to request
     * @return String representation of the response
     * @throws IOException
     */
    public static String getResponse(String url) throws IOException {
        return execute(url, new BasicResponseHandler());
    }

    /**
     * Returns the http response as a byte[].
     * @param url the url to request
     * @return byte[] representation of the response
     * @throws IOException
     */
    public static byte[] getResponseBytes(String url) throws IOException {
        return execute(url, new ByteResponseHandler());
    }

    /**
     * Executes the http request and returns the desired response type.
     * @param url the url of the request
     * @param responseHandler the response handler
     * @param <T> the type of response
     * @return the http response as returned by the specified response handler
     * @throws IOException
     * @see ResponseHandler
     */
    private static <T> T execute(final String url, final ResponseHandler<? extends T> responseHandler)
    throws IOException {
        log.debug("Reading {}", url);
        T response;
        HttpClient httpclient = new DefaultHttpClient();
        try {
            HttpGet httpget = new HttpGet(url);
            response = httpclient.execute(httpget, responseHandler);
        } catch (IOException e) {
            log.error("IOException getting http response from {}", url, e);
            throw e;
        } finally {
            httpclient.getConnectionManager().shutdown();
        }
        return response;
    }
}

February 9th, 2013

Posted In: Java

Tags: , ,

Leave a Comment

Updated on 2013/08/22

I added the maven-source-plugin and the maven-javadoc-plugin to my Maven projects. Great idea, but building the source and javadoc jar every time was increasing my build times. The solution is to specify a phase, then it will only run when the specified goal is ran in that phase instead of any phase that has that goal.

            <plugin>
                <artifactId>maven-source-plugin</artifactId>
                <version>2.2.1</version>
                <executions>
                    <execution>
                        <id>attach-sources</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>jar-no-fork</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-javadoc-plugin</artifactId>
                <version>2.9.1</version>
                <executions>
                    <execution>
                        <id>attach-javadocs</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

February 8th, 2013

Posted In: Maven

One Comment

When my OS was upgraded, I had to start typing the FQDN for all of the servers I used for work. I knew it had something to do with /etc/hosts on other Linux machines.

The file that needs to be changed for OSX is /etc/resolv.conf. there is a line that begins with search. Add the domain names that you want appended and it will append them in order until if finds the host.

February 8th, 2013

Posted In: Mac, OS/X

Tags: , , ,

Leave a Comment

LinkedIn Auto Publish Powered By : XYZScripts.com