[Grails]ignore the excludePathPatterns parameter when using the Compress Plugin

In Grails environment, Compress Plugin ignore the excludePathPatterns option and filter doesn't work fine.

• References
  
  • Grails Compress Plugin
  • pjl Compression Filter
    
    
• Description
  The Compress Plugin ignores the excludePathPatterns parameter even though I set the excludePathPatterns parameter in grails-app/conf/Config.groovy as follows, the Compress filter tries to compress for the css or pdf files. the css and pdf files should be exclude and disable the compression process for these files.
  (If you need to use the excludePathPatterns, the value of includePathPattern has to be set to null or empty.)
  

  includePathPatterns = []
  excludePathPatterns = [".*\\.css",".*\\.gif",".*\\.ico",".*\\.jpg",".*\\.swf",".*\\.pdf"]
  

• Solution(Workaround)
  Change the plugin's source code as follows in CompressGrailsPlugin.groovy and recompile or rebuild the application. Please check the exclamation mark carefully.
  

  line and source code
  55     // excludePathPatterns configuration
  56     'init-param' {
  57          'param-name'("excludePathPatterns")
  58          //changed the source code from line:59 to line:60
  59          //if (compress && !compress["excludePathPatterns"] && compress["excludePathPatterns"].isEmpty()) {
  60          if (compress && compress["excludePathPatterns"] && !compress["excludePathPatterns"].isEmpty()) {
  61              'param-value'(compress["excludePathPatterns"].join(","))
  62          } else {
  63              'param-value'(".*\\.gif, .*\\.ico, .*\\.jpg, .*\\.swf")
  64          }
  65     }
  

  
  

[Grails]IOException occurs when calling redirect() in case of using Compress plugin

IOException occurs when calling redirect() in case of using Compress plugin in Grails environment.

• References
  
  • Grails Compress Plugin
  • pjl Compression Filter
    
  
  
• Description
  IOException occurs and outputs if you call redirect() in case of using Compress plugin(pjl Compression filter) under the Jetty as the servlet container. If you set the debug level of the log4j option to WARN or DEBUG for the org.mortbay java package in Config.groovy, the message "java.io.IOException: Closed" outputs into the stdout or debug file.
  (Maybe this problem will not occur if you use the Tomcat or other servlet container., but i don't know the detail of this in case of using the Tomcat or other servlet container.)
  
  
• Error Messages
  Following is a error stack when calling redirect() method.
  

  java.io.IOException: Closed
       at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:627)
       at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:577)
       at java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:91)
       at com.planetj.servlet.filter.compression.CompressingStreamFactory$GZIPCompressingStreamFactory$1.finish(CompressingStreamFactory.java:369)
       at com.planetj.servlet.filter.compression.ThresholdOutputStream.close(ThresholdOutputStream.java:131)
       at com.planetj.servlet.filter.compression.CompressingServletOutputStream.close(CompressingServletOutputStream.java:92)
       at com.opensymphony.module.sitemesh.filter.RoutableServletOutputStream.close(RoutableServletOutputStream.java:46)
       at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:343)
       at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126)
       at org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:293)
       at org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:269)
       at org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:261)
       at org.codehaus.groovy.grails.web.mapping.filter.UrlMappingsFilter.doFilterInternal(UrlMappingsFilter.java:181)
       at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      ......
  

• Cause
  This problem occures because the pjl Compression Filter tries to call the close() method for the OutputStream object which is already closed at the end of the response. When closing the output stream, flush() method is called before closing in org.mortbay.jetty.AbstractGenerator$Output.write object. When calling the org.mortbay.jetty.AbstractGenerator$Output.write method, System checks whether the Output object is already closed or not. If the object is already closed, JVM raises the IOException.
  
  
• Solution(workaround)
  Before calling the redirect() method, response.getWriter().write("") calls as follows.
  

   def test={
    response.getWriter().write("");
    return redirect(url:"http://example.com");
   }
  

[Google Map V3]How to use the OverlayView's fromDivPixelToLatLng and fromLatLngToDivPixel functions(how to convert the google.maps.Point to LatLng)

How to use the OverlayView's fromDivPixelToLatLng(converter method from pixel in div to LatLng object) and fromLatLngToDivPixel(converter method from LatLng object to pixel in div field) functions.
(how to convert the google.maps.Point to LatLng) on Google Map Version 3.
This is my sample program. This program works fine in my test environment.Please let me know if I'm misunderstanding.

References

• Reference Manual of the Google Map v3
• Reference Manual of the Google Map v2
• Discussion about "how to fromDivPixelToLatLng" on google map v3
• example code

Examples

 var map=null;
var projectionHelper=null;

function initialize(){
 var mapElem = document.getElementById('map');
 var gmapOpts={
  zoom:10
  ,center:new google.maps.LatLng(37.421843,-122.084026)
  ,mapTypeId:google.maps.MapTypeId.ROADMAP
  ,mapTypeControl:true
  ,scaleControl:true
  ,navigationControl:false
  ,draggable:true
  ,scrollwheel:false
 }

 //initialize the google map object
 map=new google.maps.Map(mapElem,gmapOpts);
 //initialize the OverlayView object
 projectionHelper=new ProjectionHelper(map);
}

//converter from google.maps.Point object to google.maps.LatLng object
function fromPixelToLatLng(point){
 if(point==null || !(point instanceof google.maps.Point)){
  return null;
 }
 return projectionHelper.getProjection().fromDivPixelToLatLng(point);
}

//converter from google.maps.LatLng object to google.maps.Point object
function fromLatLngToPixel(latLng){
 if(latLng==null || !(latLng instanceof google.maps.LatLng)){
  return null;
 }
 return projectionHelper.getProjection().fromLatLngToDivPixel(latLng);
}

//calculate the array of the polygon's path and draw polygon ogject into the google map
//
//***** Attention *****
//This method should be called from the ProjectionHelper.prototype.draw()
//because the fromLatLngToDivPixel and fromDivPixelToLatLng functions won't work fine.
//*********************
function drawPolygons(){
 //google hq's location
 var googleHqLatLng=new google.maps.LatLng(37.421843,-122.084026);
 //conversion the google.map.LatLng object to the google.map.Point object.
 var googleHqPoint=fromLatLngToPixel(googleHqLatLng);
 ....
 //culculates the pixel X-Y coordinates to draw the circle around the google headquarter.
 var radius=5000;
 var plotPoints=40;
 var plotPointsPath=new Array();
 for(var i=0;i < plotPoints+1;i++){
  //calculate the pixel X-Y coordinates on the google map object to draw the circle
  var theta=2.0*Math.PI/plotPoints*i;
  var x=radius*Math.cos(theta)+googleHqPoint.x;
  var y=radius*Math.sin(theta)+googleHqPoint.y;
  //after calculating the pixcel x and y, convert the google.maps.Point(x,y) to the LatLng object.
  var latLng=fromPixelToLatLng(new google.maps.Point(x,y));
  //set the LatLng object into the Array object
  plotPointsPath.push(latLng);
 }

 var polygonOpts={
  paths:paths
 }
 var polygon=new google.maps.Polygon(polygonOpts);
 polygon.setMap(map);
}

//initialize the OverlayView
//helper functions
function ProjectionHelper(overlayMap) {
  google.maps.OverlayView.call(this);
  this.set_map(overlayMap);
}

ProjectionHelper.prototype = new google.maps.OverlayView();
ProjectionHelper.prototype.draw = function () {
 if (!this.ready) {
  this.ready = true;
 
  //call the draw function here.
  drawPolygons();
 }
}

[Grails]check list when java.lang.OutOfMemoryError: Java heap space in grails application

check list when java.lang.OutOfMemoryError: Java heap space in grails application

References

• Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine
• Introduction to Hibernate Caching
• Batch processing in Hibernate
• Grails Memory Leak
• Java HotSpot VM Options
• Java SE HotSpot at a Glance
  

Tools

• GCViewer
• Memory Analyzer
• IBM HeapAnalyzer
  

Check list

• Check the setup for the second level hibernate cache
  Open grails-app/conf/DataSource.groovy and check the setup of the second-level and query cache. If you don't need to use the hibernate query cache, both cache.use_second_level_cache and cache.use_query_cache are set to false. And then, you need to comment out the cache.provider_class settings if you don't need to use second level cache.(In our case, if the cache provider setting is not commented out, grails caches the domain object into the EhCache internally.)
  

       hibernate {
     cache.use_second_level_cache=false
     cache.use_query_cache=false
     //cache.provider_class='org.hibernate.cache.EhCacheProvider'
  }
  

  If you use the Multiple Datasources plugin, please check hibernate cache settings for the other database sessions in the datasources file.
  
  
• Check the history of the java heap usage and Garbage Collector
  Check whether the memory setting for the java virtual machine is enough or not.
  
  1. run the grails application with followng java options.
     

     export JAVA_OPTS=-verbose:gc -XX:+PrintClassHistogram -XX:+PrintGCDetails -Xloggc:/tmp/jvm_loggc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp
     

  2. After appearing the OutOfMemory error, view the /tmp/jvm_loggc.log.
     If you use the GCViewer(http://www.tagtraum.com/gcviewer.html), you can check the history of running the garbage collector and usage of the java heap size easily.
     
     
• Check and analyze the heap dump file.
  After occuring the OutOfMemory Exception, Java VM outputs the HeapDump file under the HeapDumpPath directory, the file name is java_pidXXXXX.hprof. You can check and analyze the usage of the java heap for each java objects and view the description when the OutOfMemory error occures.
  If you use the Memory Analyzer(http://www.eclipse.org/mat/) tool, you can view the information about the Heap Memory Dump file visually and easily.
  Memory Analyzer reports the Leak Suspect Report, you can see the output of the "Leak Suspect Report" analyzed by the Memory Analyzer. Memory analyzer reports the overview of the memory usage and problem suspect.
  
  Check the memory leak in your program based on the output of the Memory Leak Suspect Report. If you cache the objects pointer in your program, garbage collector never clear and garbage the heap memory.
  

Problems

• Memory usage of the org.hibernate.engine.StatefulPersistenceContext" object
  If the Problem Suspect in Leak Suspect Report says that the memory usage of the org.hibernate.engine.StatefulPersistenceContext" object is accumulated, this object depends on the Hibernate's First-level cache, and maybe we can't turn off the hibernate first-level cache functionality in grails environment.(In standalone hibernate env, we can find the StatelessSession functionality, but I don't know how to use this in grails.)
  In case of this, I can find the 2 solutions to avoid increasing the memory usage of the hibernate first level cache.
  
  1. Set the session's AutoClear flag to true.
     When starting the application, this flag is set to true, cached objects in session are cleared after the end of the transaction. But the cached objects aren't always cleard out.
     

         Session session=sessionFactory.getCurrentSession();
       if((session instanceof SessionImpl)){
            ((SessionImpl)session).setAutoClear(true);
       }
     

  2. Call the clear() method of the session
     We often call the clear() method of the session object in our program and release the cached objects in first-level cache manually.
     

          Session session=sessionFactory.getCurrentSession();
        session.clear();
     

• 
  

[Java]how to use the gzipped http response by using the apache http-client ver.3.x

how to use the gzipped http response by using the apache http-client ver.3.x

• References
  
  • apache http-client 3.x home
  • example to use gzipped response in http client-4.x
    
• Example
  

  //set url
  String url="http://hc.apache.org/httpclient-3.x/";
  
  //initialize apache HttpClient object and HttpMethod
  HttpClient httpClient = new HttpClient();
  HttpMethod httpMethod = new GetMethod(url);
  //set followRedirects function to true if you need.
  httpMethod.setFollowRedirects(true);
  //set Accept-Encoding request header
  httpMethod.setRequestHeader("Accept-Encoding","gzip");
  
  try{
   //access to the url and get response status
   int status = httpClient.executeMethod(http_method);
  
   //check response status. if the value of response status is set to 200, get body stream
   if(status == 200){
    //check the response header "Content-Encoding". the value of Content-Encoding header contains the
    //"gzip" value, this means the response stream is gzipped.
    Header contentEncodingHeader=httpMethod.getResponseHeader("Content-Encoding");
    String contentEncoding = contentEncodingHeader!=null ? contentEncodingHeader.getValue() : "";
    String contentEncodingLowerCase=contentEncoding.toLowerCase();
    boolean isGzipped=(contentEncodingLowerCase.indexOf("gzip")>=0);
    
    //get response stream
    InputStream stream=httpMethod.getResponseBodyAsStream();
    
    InputStream bodyStream=null;
    ByteArrayOutputStream outStream=null;
    try{
     //if the response stream is gzipped,  derived stream is converted into the GZIPInputStream.
     //the response stream is not gzipped, set the stream without conversion
     bodyStream = isGzipped ? new GZIPInputStream(stream) : stream;
     
     //change the response from InputStream to Byte Array.
     outStream=new ByteArrayOutputStream();
     byte[] buffer = new byte[4096];
     int length;
     while((length=bodyStream.read(buffer))>0){
      outStream.write(buffer,0,length);
     }
     
     //get the response charset.
     String charset=httpMethod.getResponseCharSet();
     //convert the response byte array to the String object.
     String body=new String(outStream.toByteArray(),charset);
  
     //Instead of converting the InputStream to Byte array,
     //we can convert the inputstream to String by using the InputStreamReader and BufferedReader directly.
     //But we can't read the responsed InputStream twice.
     //for example.
     //InputStreamReader bodyReader=new InputStreamReader(bodyStream,charset);
     //BufferedReader bodyBufferedReader=new BufferedReader(bodyReader);
     //String line=bodyBufferedReader.readLine();
     //while(line!=null){
     // bodyBuffer.append(line);
     // line=bodyBufferedReader.readLine();
     //}
     //String body=bodyBuffer.toString();
    }catch(Exception e1){
     throw e1;
    }finally{
     //close ByteArrayOutputStream
     if(outStream!=null){
      try{
       outStream.close();
      }catch(Exception ignore){}
     }
     
     //close InputStream
     if(bodyStream!=null){
      try{
       bodyStream.close();
      }catch(Exception ignore){}
     }
     if(stream!=null){
      try{
       stream.close();
      }catch(Exception ignore){}
     }
    }
   }
  }catch(Exception e0){
   System.out.println("Error, "+e0);
  }finally{
   httpMethod.releaseConnection();
  }
  

  
  
  

[Java>Rome]Rome plugin module for twitter search RSS

Rome plugin module for twitter search RSS(Beta version 0.1)

• References
  
  • Twitter Search
    
  • Using the Rome Java API to access metadata fields
    
  
  
• Description
  the rss output of the twitter search is including the original elements like the twitter:source and twitter:lang tags. These tags are extended by the Twitter Search, so we need to use new plugin module to get the value of these tags by using the rome rss library.
  
• Setups
  
  • Dwonload Jar file
    http://groups.google.com/group/taapps-sourcecode-libraries/web/tskr-twitter-rss-0.1-b1.jar
    
  • Put the downloaded jar file into the directory
  • Restart the application
    
  
  
• Sample Program
  

  //import
  import com.sun.syndication.fetcher.FeedFetcher;
  import com.sun.syndication.fetcher.FetcherException;
  import com.sun.syndication.fetcher.impl.HttpURLFeedFetcher;
  import com.sun.syndication.feed.synd.SyndEntry;
  import com.sun.syndication.feed.synd.SyndFeed;
  import com.sun.syndication.io.FeedException;
  import java.io.IOException;
  import java.net.MalformedURLException;
  import java.net.URL;
  import java.util.List;
  
  //import rome module
  import com.sun.syndication.feed.module.Module;
  //import plugin new module
  import jp.tskr.feed.module.twitter.Twitter;
  
  public class TwSample{
    public static void main(String[] args){
       FeedFetcher feedFetcher=new HttpURLFeedFetcher();
       try{
            String urlStr="http://search.twitter.com/search.atom?q=iphone";
            URL feedUrl=new URL(urlStr);
            SyndFeed feedFetch=feedFetcher.retrieveFeed(feedUrl);
  
            for(SyndEntry entry : (List) feedFetch.getEntries()){
                 System.out.format("\tUpdate:[%s] URL:[%s] Title:[%s]\n",entry.getPublishedDate(),entry.getLink(),entry.getTitle());
  
                 //calling twitter search module
                 Module module=entry.getModule(Twitter.URI);
                 //checking the module object whether the object is instanceof Twitter class or not
                 if(module instanceof Twitter){
                      Twitter twModule=(Twitter)module;
                      //get the value in the twitter search rss field 
                      String source=twModule.getSource();
                      //get the value in the twitter search rss field 
                      String lang=twModule.getLang();
                 }
             }
       }catch(Exception e){
         //.....
       }
    }
  }
  

  
  

[rome-fetcher]how to get rss data from the basic authenticated web site

Steps to get rss data from the basic authenticated web site like the twitter.

1. Create a new class implemented the CredentialSupplier interface. this class stores the username and password for the basic authentication and returns the Credentials object of the apache http client. following code is a sample class implemented the CredentialSupplier interface
   

   import com.sun.syndication.fetcher.impl.HttpClientFeedFetcher.CredentialSupplier;
   import org.apache.commons.httpclient.UsernamePasswordCredentials;
   import org.apache.commons.httpclient.Credentials;
   public class AuthCredentialSupplier implements CredentialSupplier{
    private String username=null;
    public void setUsername(String username){
     this.username=username;
    }
    public String getUsername(){
     return this.username;
    }
   
    private String password=null;
    public void setPassword(String password){
     this.password=password;
    }
    public String getPassword(){
     return this.password;
    }
   
    public AuthCredentialSupplier(){
    }
    public AuthCredentialSupplier(String username,String password){
     setUsername(username);
     setPassword(password);
    }
    public Credentials getCredentials(String realm, String host){
     String username=getUsername();
     String password=getPassword();
     return new UsernamePasswordCredentials(username,password);
    }
   }
   

2. Access to the rss url and get rss data from the basic authenticated web site. When accessing to the rss url, we need to use the HttpClientFeedFetcher instead of HttpURLFeedFetcher.
   

    //sample program
    //rss url
    String url="http://twitter.com/statuses/friends_timeline.atom";
   
    //username and password
    String username="username";
    String password="password";
     
    try{
     //create and initialize CredentialSupplier Object
     AuthCredentialSupplier authCredentials=new AuthCredentialSupplier(username,password);
     //create HttpClientFeedFetcher object
     //(we can not use HttpURLFeedFetcher with basic authentication)
     FeedFetcher feedFetcher=new HttpClientFeedFetcher(null,authCredentials);
     List result=(feedFetcher).retrieveFeed(new URL(url)).getEntries();
     //get response.
     if(result!=null){
      .....
     }
     else{
      System.out.println("ERROR")
     }
    }catch(FetcherException e){
     int responseCode=e.getResponseCode();
     System.out.println("ERROR, response code="+responseCode+", error="+e);
    }catch(Exception e){
     System.out.println("Unexpected Exception, e="+e)
    }