Troubleshooting Java Libraries

2009年7月24日金曜日

[Grails]check list when java.lang.OutOfMemoryError: Java heap space in grails application

check list when java.lang.OutOfMemoryError: Java heap space in grails application

References

Tools

Check list

Check the setup for the second level hibernate cache
Open grails-app/conf/DataSource.groovy and check the setup of the second-level and query cache. If you don't need to use the hibernate query cache, both cache.use_second_level_cache and cache.use_query_cache are set to false. And then, you need to comment out the cache.provider_class settings if you don't need to use second level cache.(In our case, if the cache provider setting is not commented out, grails caches the domain object into the EhCache internally.)
```
     hibernate {
   cache.use_second_level_cache=false
   cache.use_query_cache=false
   //cache.provider_class='org.hibernate.cache.EhCacheProvider'
}
```
If you use the Multiple Datasources plugin, please check hibernate cache settings for the other database sessions in the datasources file.
Check the history of the java heap usage and Garbage Collector
Check whether the memory setting for the java virtual machine is enough or not.
1. run the grails application with followng java options.
```
export JAVA_OPTS=-verbose:gc -XX:+PrintClassHistogram -XX:+PrintGCDetails -Xloggc:/tmp/jvm_loggc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp
```
2. After appearing the OutOfMemory error, view the /tmp/jvm_loggc.log.
  If you use the GCViewer(http://www.tagtraum.com/gcviewer.html), you can check the history of running the garbage collector and usage of the java heap size easily.
Check and analyze the heap dump file.
After occuring the OutOfMemory Exception, Java VM outputs the HeapDump file under the HeapDumpPath directory, the file name is java_pidXXXXX.hprof. You can check and analyze the usage of the java heap for each java objects and view the description when the OutOfMemory error occures.
If you use the Memory Analyzer(http://www.eclipse.org/mat/) tool, you can view the information about the Heap Memory Dump file visually and easily.
Memory Analyzer reports the Leak Suspect Report, you can see the output of the "Leak Suspect Report" analyzed by the Memory Analyzer. Memory analyzer reports the overview of the memory usage and problem suspect.

Check the memory leak in your program based on the output of the Memory Leak Suspect Report. If you cache the objects pointer in your program, garbage collector never clear and garbage the heap memory.

Problems

Memory usage of the org.hibernate.engine.StatefulPersistenceContext" object
If the Problem Suspect in Leak Suspect Report says that the memory usage of the org.hibernate.engine.StatefulPersistenceContext" object is accumulated, this object depends on the Hibernate's First-level cache, and maybe we can't turn off the hibernate first-level cache functionality in grails environment.(In standalone hibernate env, we can find the StatelessSession functionality, but I don't know how to use this in grails.)
In case of this, I can find the 2 solutions to avoid increasing the memory usage of the hibernate first level cache.
1. Set the session's AutoClear flag to true.
  When starting the application, this flag is set to true, cached objects in session are cleared after the end of the transaction. But the cached objects aren't always cleard out.
```
    Session session=sessionFactory.getCurrentSession();
  if((session instanceof SessionImpl)){
       ((SessionImpl)session).setAutoClear(true);
  }
```
2. Call the clear() method of the session
  We often call the clear() method of the session object in our program and release the cached objects in first-level cache manually.
```
     Session session=sessionFactory.getCurrentSession();
   session.clear();
```

2009年7月3日金曜日

[Java]how to use the gzipped http response by using the apache http-client ver.3.x

how to use the gzipped http response by using the apache http-client ver.3.x

References
- apache http-client 3.x home
- example to use gzipped response in http client-4.x

Example

//set url
String url="http://hc.apache.org/httpclient-3.x/";

//initialize apache HttpClient object and HttpMethod
HttpClient httpClient = new HttpClient();
HttpMethod httpMethod = new GetMethod(url);
//set followRedirects function to true if you need.
httpMethod.setFollowRedirects(true);
//set Accept-Encoding request header
httpMethod.setRequestHeader("Accept-Encoding","gzip");

try{
 //access to the url and get response status
 int status = httpClient.executeMethod(http_method);

 //check response status. if the value of response status is set to 200, get body stream
 if(status == 200){
  //check the response header "Content-Encoding". the value of Content-Encoding header contains the
  //"gzip" value, this means the response stream is gzipped.
  Header contentEncodingHeader=httpMethod.getResponseHeader("Content-Encoding");
  String contentEncoding = contentEncodingHeader!=null ? contentEncodingHeader.getValue() : "";
  String contentEncodingLowerCase=contentEncoding.toLowerCase();
  boolean isGzipped=(contentEncodingLowerCase.indexOf("gzip")>=0);
  
  //get response stream
  InputStream stream=httpMethod.getResponseBodyAsStream();
  
  InputStream bodyStream=null;
  ByteArrayOutputStream outStream=null;
  try{
   //if the response stream is gzipped,  derived stream is converted into the GZIPInputStream.
   //the response stream is not gzipped, set the stream without conversion
   bodyStream = isGzipped ? new GZIPInputStream(stream) : stream;
   
   //change the response from InputStream to Byte Array.
   outStream=new ByteArrayOutputStream();
   byte[] buffer = new byte[4096];
   int length;
   while((length=bodyStream.read(buffer))>0){
    outStream.write(buffer,0,length);
   }
   
   //get the response charset.
   String charset=httpMethod.getResponseCharSet();
   //convert the response byte array to the String object.
   String body=new String(outStream.toByteArray(),charset);

   //Instead of converting the InputStream to Byte array,
   //we can convert the inputstream to String by using the InputStreamReader and BufferedReader directly.
   //But we can't read the responsed InputStream twice.
   //for example.
   //InputStreamReader bodyReader=new InputStreamReader(bodyStream,charset);
   //BufferedReader bodyBufferedReader=new BufferedReader(bodyReader);
   //String line=bodyBufferedReader.readLine();
   //while(line!=null){
   // bodyBuffer.append(line);
   // line=bodyBufferedReader.readLine();
   //}
   //String body=bodyBuffer.toString();
  }catch(Exception e1){
   throw e1;
  }finally{
   //close ByteArrayOutputStream
   if(outStream!=null){
    try{
     outStream.close();
    }catch(Exception ignore){}
   }
   
   //close InputStream
   if(bodyStream!=null){
    try{
     bodyStream.close();
    }catch(Exception ignore){}
   }
   if(stream!=null){
    try{
     stream.close();
    }catch(Exception ignore){}
   }
  }
 }
}catch(Exception e0){
 System.out.println("Error, "+e0);
}finally{
 httpMethod.releaseConnection();
}

2009年5月12日火曜日

[Java>Rome]Rome plugin module for twitter search RSS

Rome plugin module for twitter search RSS(Beta version 0.1)

References
- Twitter Search
- Using the Rome Java API to access metadata fields
Description
the rss output of the twitter search is including the original elements like the twitter:source and twitter:lang tags. These tags are extended by the Twitter Search, so we need to use new plugin module to get the value of these tags by using the rome rss library.
Setups
- Dwonload Jar file
  http://groups.google.com/group/taapps-sourcecode-libraries/web/tskr-twitter-rss-0.1-b1.jar
- Put the downloaded jar file into the directory
- Restart the application

Sample Program

//import
import com.sun.syndication.fetcher.FeedFetcher;
import com.sun.syndication.fetcher.FetcherException;
import com.sun.syndication.fetcher.impl.HttpURLFeedFetcher;
import com.sun.syndication.feed.synd.SyndEntry;
import com.sun.syndication.feed.synd.SyndFeed;
import com.sun.syndication.io.FeedException;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.List;

//import rome module
import com.sun.syndication.feed.module.Module;
//import plugin new module
import jp.tskr.feed.module.twitter.Twitter;

public class TwSample{
  public static void main(String[] args){
     FeedFetcher feedFetcher=new HttpURLFeedFetcher();
     try{
          String urlStr="http://search.twitter.com/search.atom?q=iphone";
          URL feedUrl=new URL(urlStr);
          SyndFeed feedFetch=feedFetcher.retrieveFeed(feedUrl);

          for(SyndEntry entry : (List) feedFetch.getEntries()){
               System.out.format("\tUpdate:[%s] URL:[%s] Title:[%s]\n",entry.getPublishedDate(),entry.getLink(),entry.getTitle());

               //calling twitter search module
               Module module=entry.getModule(Twitter.URI);
               //checking the module object whether the object is instanceof Twitter class or not
               if(module instanceof Twitter){
                    Twitter twModule=(Twitter)module;
                    //get the value in the twitter search rss field 
                    String source=twModule.getSource();
                    //get the value in the twitter search rss field 
                    String lang=twModule.getLang();
               }
           }
     }catch(Exception e){
       //.....
     }
  }
}

2009年4月28日火曜日

[rome-fetcher]how to get rss data from the basic authenticated web site

Steps to get rss data from the basic authenticated web site like the twitter.

Create a new class implemented the CredentialSupplier interface. this class stores the username and password for the basic authentication and returns the Credentials object of the apache http client. following code is a sample class implemented the CredentialSupplier interface

import com.sun.syndication.fetcher.impl.HttpClientFeedFetcher.CredentialSupplier;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.Credentials;
public class AuthCredentialSupplier implements CredentialSupplier{
 private String username=null;
 public void setUsername(String username){
  this.username=username;
 }
 public String getUsername(){
  return this.username;
 }

 private String password=null;
 public void setPassword(String password){
  this.password=password;
 }
 public String getPassword(){
  return this.password;
 }

 public AuthCredentialSupplier(){
 }
 public AuthCredentialSupplier(String username,String password){
  setUsername(username);
  setPassword(password);
 }
 public Credentials getCredentials(String realm, String host){
  String username=getUsername();
  String password=getPassword();
  return new UsernamePasswordCredentials(username,password);
 }
}

Access to the rss url and get rss data from the basic authenticated web site. When accessing to the rss url, we need to use the HttpClientFeedFetcher instead of HttpURLFeedFetcher.

 //sample program
 //rss url
 String url="http://twitter.com/statuses/friends_timeline.atom";

 //username and password
 String username="username";
 String password="password";
  
 try{
  //create and initialize CredentialSupplier Object
  AuthCredentialSupplier authCredentials=new AuthCredentialSupplier(username,password);
  //create HttpClientFeedFetcher object
  //(we can not use HttpURLFeedFetcher with basic authentication)
  FeedFetcher feedFetcher=new HttpClientFeedFetcher(null,authCredentials);
  List result=(feedFetcher).retrieveFeed(new URL(url)).getEntries();
  //get response.
  if(result!=null){
   .....
  }
  else{
   System.out.println("ERROR")
  }
 }catch(FetcherException e){
  int responseCode=e.getResponseCode();
  System.out.println("ERROR, response code="+responseCode+", error="+e);
 }catch(Exception e){
  System.out.println("Unexpected Exception, e="+e)
 }

2009年4月24日金曜日

[Grails]Grails1.1 Setup fo the Apache Log4j Logging

Sample Setup fo the Apache Log4j Logging on Grails 1.1, A new Log4j DSL is available on Grails 1.1.

References

Sample Setup in grails-app/conf/Config.goovy file
This sample setup is modified based on the document(http://d.hatena.ne.jp/nobeans/20090323/1237826907, this documentation is written in Japanese).

// log4 setup
log4j = {
 appenders {
  //override the setuf of the default console out
  console(name:'stdout', layout:pattern(conversionPattern: '%d{HH:mm:ss} [%p] (%c{2}) %m%n'))
  
  //override the setup of the default log
  rollingFile(name:'file', file:'logs/debug.log', maxFileSize:'10MB', maxBackupIndex:5, layout:pattern(conversionPattern: '%d{HH:mm:ss} [%p] (%c{2}) %m%n'))
  
  //override the setup of the default error stack
  rollingFile(name:'stacktrace', file:'logs/stacktrace.log', maxFileSize:'10MB', maxBackupIndex:5, layout:pattern(conversionPattern: '%d{yyyy-MM-dd HH:mm:ss} [%p] (%c{2}) %m%n'))
 
  //daily rolling log
  appender new org.apache.log4j.DailyRollingFileAppender(name:'dailyRollingFile', datePattern:"'.'yyyy-MM-dd",layout:pattern(conversionPattern: '%d{HH:mm:ss} [%p] (%c{2}) %m%n'), file:'logs/daily.log');
 }

 root {
  error 'stdout', 'file'
  additivity = false
 }

 //controller
 error 'org.codehaus.groovy.grails.web.servlet'
 //gsp
 error 'org.codehaus.groovy.grails.web.pages'
 //layouts
 error 'org.codehaus.groovy.grails.web.sitemesh'
 //url mapping filter
 error 'org.codehaus.groovy.grails."web.mapping.filter'
 //url mapping
 error 'org.codehaus.groovy.grails."web.mapping'
 //core, classloader
 error 'org.codehaus.groovy.grails.commons'
 //plugins
 error 'org.codehaus.groovy.grails.plugins'
 //hibernate integration
 error 'org.codehaus.groovy.grails.orm.hibernate'
 error 'org.springframework'
 //info  'org.springframework.security'
 //hibernate
 error 'org.hibernate'

 //jetty
 warn  'org.mortbay.log'

 //error stack
 error(
  additivity:false
  //,stdout:"StackTrace"
  ,stacktrace:"StackTrace"
 )

 //debug for my my app
 //info dailyRollingFile:"grails.app.controller.TestController"
 info(
  additivity:false
  //,stdout:"grails.app.controller"
  ,dailyRollingFile:"grails.app.controller"
 )
 info(
  additivity:false
  ,dailyRollingFile:"grails.app.service"
 )
 info(
  additivity:true
  ,dailyRollingFile:"grails.app.task"
 )
}

2009年4月17日金曜日

[Grails]Grails Upgrade Steps from 1.0.x to 1.1

Grails Upgrade Steps from 1.0.x to 1.1

References
- Upgrade Manual(Upgrading from Grails 1.0.x)
- Upgrade Steps(Japanese)
Upgrade Steps
- Backup all application files
- Add following statements into the grails-app/conf/Config.groovy file
```
   //upgrade
grails.views.enable.jsessionid=false
//grails.project.plugins.dir="./plugins"
```
- Create a file grails-app/conf/BuildConfig.groovy file if this file doesn't exist and add following statements into this file.
```
 //upgrade
//grails.views.enable.jsessionid=false
grails.project.plugins.dir="./plugins"
```
- if you use some plugins, you need to re-install plugins.
  Remove all plugins files and cleanup files stored under plugins directory
```
   mv plugins/* /tmp/
```
- Run the upgrade command as follows
  (After running this commands, grails will create a new plugin named hibernate-1.1 under the plugins directory automatically. we don't need to re-install this hibernate-1.1 plugin.)
```
grails upgrade
```
- re-install plugins by using "grails install-plugins" command
```
(for example)
grails install-plugin /tmp/grails-quartz-0.3.1.zip
```
TroubleShooting

Plugins TroubleShooting

quartz-0.3.1 plugin

Error Description
After starting the grails application, following error appears while starting up the quartz plugin

Error Stack

2009-04-14 16:12:39,907 [main] ERROR context.ContextLoader  - Context initialization failed
org.springframework.beans.factory.access.BootstrapException: Error executing bootstraps; nested exception is org.codehaus.groovy.runtime.InvokerInvocationException: groovy.lang.MissingPropertyException: No such property: startDelay for class: QuartzGrailsPlugin
 at org.codehaus.groovy.grails.web.context.GrailsContextLoader.createWebApplicationContext(GrailsContextLoader.java:74)
 at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:199)
 at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)

Cause
This error raises from following statements of the configureJobBeans closure in the QuartzGrailsPlugin.groovy file because of the "this" clause.

      // registering triggers
  jobClass.triggers.each {name, trigger ->
      "${name}Trigger"(trigger.clazz) {
          jobDetail = ref("${fullName}JobDetail")
          trigger.properties.findAll {it.key != 'clazz'}.each {
              this["${it.key}"] = it.value  /* here */
          }
       }
  }

Resolution
I changed this clause from "this" to "delegate", quartz plugin works fine.

2009年4月7日火曜日

[grails]upgraded the plugin for grails to execute native query

I created and upgraded the grails plugin named native-query(Version 0.2) to execute native query statement like ORACLE's query hint, mysql match-against statement on the grails framework.

Installation Steps

download zip file named grails-native-query-0.2.zip via http://groups.google.com/group/taapps-sourcecode-libraries/web/grails-native-query-0.2.zip
installing plugin into the grails application
grails install-plugin grails-native-query-0.2.zip

How to use

use DomainObject.executeJdbcQuery method injected into the Domain Class
- Description
  This executeJdbcQuery method is static method injected into the Domain Class object.
- Return Value
  This method returns the List object and each objects stored into the List are Domain Objects of the fetched rows.
- Arguments
  1. Required String type, where, order by, match against clauses(No need to enter the "select * from " statement)
  2. Optional String type, select option like the "SQL_CALC_FOUND_ROWS(mysql)" or hint text(ORACLE)
  3. List object or Map object stored the bind values
- Example
  Following examples are using the Domain Class named "Test", this is a test domain class.
  - use the standard select statement
    def result=Test.executeJdbcQueryMap("where id=160")
  - use positonal parameter
    def result=Test.executeJdbcQueryMap("where id=?",[160])
  - use named map parameter
    def result=Test.executeJdbcQueryMap("where id=:id",[id:160])
  - use the positonal parameter wit
    def result=Test.executeJdbcQueryMap("where id=?","SQL_CALC_FOUND_ROWS",[160])
  - use the List object as the member of the positional parameters
    def result=Test.executeJdbcQueryMap("where id in (?)","SQL_CALC_FOUND_ROWS",[[160,161,162,163,164]])
use NativeQueryUtil.executeJdbcQueryMap method to execute free sql statement(not need to be related to the Domain Class)
- Return Value
  This method returns the List object and each objects stored into the List are Map object stored the fetched column name and the value, map key is column name
- Arguments
  1. String type, Select statement
  2. List object or Map object stored the bind values
- Example
  - use the standard select statement
    def result=NativeQueryUtil.executeJdbcQueryMap("select * from test where id=103")
  - use positional map
    def result=NativeQueryUtil.executeJdbcQueryMap("select * from test where id=?",[103])
  - use namedmap parameter
    def result=NativeQueryUtil.executeJdbcQueryMap("select * from test where id=:id",[id:103])
  - use the List object as the member of the positional parameters
    def result=NativeQueryUtil.executeJdbcQueryMap("select * from test where id in (?)",[[103.104,105,106]])
use the NativeQueryUtil.executeNativeQuery method(execute sql statement via Hibernate's createQuery method)
- Description
  This method executes the select statement via Hibernate's session.createQuery method.
- Return Value
  Domain Object or List object stored the domain objects of the fetched rows.
- Arguments
  1. String Type, select statement
  2. Map type, mapping the table name and domain class
  3. List object or Map object stored the bind values
- Example
  - def result=NativeQueryUtil.executeNativeQuery("select /* query */ tbl.* from test tbl where id=?",[tbl:Test],[101])

登録: 投稿 (Atom)