[Java>Original]Useful Network Utility(Utility to check Netmask, Network address)

Useful Network Utility to calculate CIDR address and check network address.

References

• about CIDR
  http://h50146.www5.hp.com/products/software/oe/tru64unix/manual/v51a_ref/HTML/MAN/MAN7/0127____.HTM
• Sample Source code
  http://osdir.com/ml/jakarta.james.devel/2003-02/msg00469.html
  http://lenya.apache.org/apidocs/1.2/org/apache/lenya/net/InetAddressUtil.html
  

Original Source Code

• http://groups.google.com/group/taapps-sourcecode-libraries/web/Net.java
  

Check Utility to contain the ip address into the specified CIDR network address.

• Description
  Network address check utility
  Check whether the specified ip address is into the CIDR network address(format like "192.168.1.0/24") or not.
  This method has 2 arguments, one is String object contains CIDR network address, other is IP address you want to check.
  
• Sample
  

  //for example, my ip address is 192.168.1.3
  //check whether my ip address "192.168.1.3" is into the network 192.168.1.0/24 or not
  boolean compare=Net.containIp("192.168.1.0/24","192.168.1.3");
  if(compare){
  //contain the specified ip address "192.168.1.3" into network "192.168.1.0/24"
  }
  

Check Utility 2 to contain the ip address into the specified CIDR network list.

• Description
  Check whether the specified ip address is into the CIDR network address list or not.
  This method has 2 arguments, one is List object contains CIDR network addresses, other is IP address you want to check.
  
• Sample
  

  List networkList=new ArrayList();
  networkList.add("192.168.1.0/24");
  networkList.add("192.168.2.0/24");
  networkList.add("192.168.3.0/24");
  networkList.add("192.168.4.0/24");
  boolean compare2=Net.containIp(networkList,"192.168.2.200");
  if(compare){
  //contain the specified ip address "192.168.1.3" into network "192.168.1.0/24"
  }
  

[Java>Original]Useful Date Utility(Utility to Convert Date to String, Convert String to Date)

I created useful Useful Date Utility including some functions, utility to convert Date to String and convert String to Date easily.

Original Source Code

• http://taapps-sourcecode-libraries.googlegroups.com/web/Dat.java
  

Converter from Date object to String easily

• Description
  This function is a converter from Date object to String.This function works like ORACLE PL/SQL's to_char function.
  Dat.convertDateToStr method has 2 arguments, one is a Date format you want to convert, other is a Data format String object like "yyyy/MM/dd".
  
• Sample
  Output Date object to String changed format like "2007/12/31 12:00:00".
  This function works like ORACLE PL/SQL's to_date function.
  

     java.util.Date testDate=new java.util.Date();
  .......
  System.out.println("Date Output:"+Dat.convertDateToStr(testDate,"yyyy/MM/dd HH:mm:ss"));
  

Converter from String object to Date easily

• Description
  This function is a converter from String object to Date object.This function works like ORACLE PL/SQL's to_date function.
  Dat.convertStrToDate method has 2 arguments, one is a String object including date, other is a Date format String.
  
• Sample
  

     String testDateStr="2007/12/31";
   ....
   java.util.Date convertedDate=Dat.convertStrToDate(testDateStr,"yyyy/MM/dd");

Get the long value between 2 Date objects.

• Description
  You sometimes need to get long value between 2 Date objects(for example, you need to get execution time when debugging the program and checking performance).
  Dat.getDateDiff method has 3 arguments and returns the difference long value between 2 Date object. one argument is a first Date object, second argument is a Date object, third argument is a boolean type argument to convert the result long value to absolute value.
  
• Sample
  

     java.util.Date date1=Dat.convertStrToDate("2007/12/31","yyyy/MM/dd");
    ..........
    java.util.Date date2=Dat.convertStrToDate("1973/12/31","yyyy/MM/dd");
    ..........
    long dateDiff=Dat.getDateDiff(date1,date2,false);
  

  
  

[Java>Original]Useful Object Utility(resource close function to avoid resource leak, and null and empty checker)

I created useful Object Utility including some functions, resource close function to avoid resource leak, and null and empty checker.

Original Source Code.

• http://taapps-sourcecode-libraries.googlegroups.com/web/Obj.java

Object null and empty checker

• Description
  We sometimes need to check whether the java object is null before accessing the object to avoid occuring the NullPointerException, and then we check whether the object is empty if the object is not null as follows.
  

  String test="test";
   ....
  if(test!=null && !test.equals("")){
   ....
  }
  

  This is a long and mistakable, and Obj.isNull method returns the value of true if the first argument object is null (not initialized),Obj.isNotNull(obj,true) method returns the value of !Obj.isNull(obj,true) like this.
  

  String test="test";
   ....
  if(Obj.isNull(test,true)){
   ....
  }
  

  The second argument is used for checking the object value. The second argument is set to true, System checks whether the first argument object is empty or not if the first argument object is defined as String. If the first argument is defined as Object[], Map, or Collection, System checks the Object has any elements more than 1. If Object doesn't have any elements, Obj.isNull(obj,true) returns true.
• 
  
• Normal Object
  Obj.isNull(normalObject,false) means if(normalObject==null) return true
  
• Check String object
  Obj.isNull(stringObject, true) means if(stringObject==null || stringObject.equals("")) return true.
  
• Check Array
  Obj.isNull(objects[], true) means if(objects==null || objects.length<1)>
• Check Map Object
  Obj.isNull(mapObject,true) means if(mapObject==null || mapObject.size()<1)>
• Check Collection Object
  Obj.isNull(listObject,true) means if(listObject==null || listObject.size()<1)>

Resource close function to avoid resource leak

• Description
  For example, we have to close the opened stream or connection after we finished to use java.io.FileInputStream or java.sql.Connection like this.
  

  FileInputStream   fis=null;
  InputStreamReader isr=null;
  BufferedReader    br =null;
  try{
  fis=new FileInputStream("/tmp/test.txt");
   ....
  }catch(Exception e){
   ......
  }finally{
   if(br!=null){
     try{
        br.close();
     }catch(Exception ignore){}
   }
   if(isr!=null){
     try{
        isr.close();
     }catch(Exception ignore){}
   }
   if(fis!=null){
     try{
        br.close();
     }catch(Exception e){}
   }
  }
  

  Before closing the FileInputStream, InputStreamReader, BufferedReader in finally block, we usually check whether the object is null or not, and then we close the opened object in try-catch exception handler to catch the Exception when closing the opened object.
  These statements are troublesome and mistakable. We can change the statements described above into following. We don't need to check whether the object is null or not and catch the Exception occuring while closing the object.
  

  FileInputStream   fis=null;
  InputStreamReader isr=null;
  BufferedReader    br =null;
  try{
    fis=new FileInputStream("/tmp/test.txt");
    ....
  }catch(Exception e){
    ......
  }finally{
    Obj.close(fis);
    Obj.close(isr);
    Obj.close(br);
  }

• We can use this Obj.close function for these class.
  

1. java.io.Reader object
2. java.io.InputStream
3. java.io.OutputStream
4. javax.imageio.ImageOutputStream
5. javax.imageio.ImageInputStream
6. javax.imageio.ImageWriter
7. java.awt.Graphics
8. java.io.Writer
9. java.sql.Connection
10. java.sql.Statement
11. java.sql.ResultSet
12. java.net.HttpURLConnection
13. java.net.URLConnection

[Java>Norbert]Norbert Utility(robots.txt parser) sometimes fails to check robots.txt properly

References

1. Norbert Page(parser for robots.txt file)
   http://osjava.org/norbert/
2. About specification of robots.txt file
   http://www.robotstxt.org/wc/norobots-rfc.html
3. About robots.txt by Baidu(japanese)
   http://www.baidu.jp/search/robots.html
4. list of Crawlers
   http://www.robotstxt.org/wc/active/html/index.html
   

Description
Norbert is a parser and utility to find and check the robots.txt file in web page.
(If you need to get the norbert utility, please access to http://osjava.org/norbert/ described above.)
But Norbert utility sometimes doesn't check and parse the robots.txt file properly even though the robots.txt file is existed in top directory of the web site. Norbert checks the string case sensitive, but some web sites ignore the tag's case sensitive in robots.txt file(refer to http://www.robotstxt.org/wc/norobots-rfc.html) and Norbert utility failed to check the line and can't pick up the tags properly.

Solution
You can avoid this problem if you change parseTextForUserAgent function in org.osjava.norbert.NoRobotClient.java as follows.
Before checking the tags in robots.txt, System changes all of the line's text to lowercase, and then System uses the String object converted to lowercase.

Customization Example

   
  private RulesEngine parseTextForUserAgent(String txt, String userAgent) throws NoRobotException {
  .....
          try {
          while( (line = rdr.readLine()) != null ) {
              // trim whitespace from either side
              line = line.trim();

              //change the line's string to new lowercase string
              //add this line to store the text after exchanging all of the line's text to lowercase.
              String lineToLowerCase=line.toLowerCase();
              //end
              .....
              //replace the "line" object with lineToLowerCase string object when checking the line string.
              if(lineToLowerCase.startsWith("user-agent:")) {

              }
              else{
                      //replace using the line object with lineToLowerCase string object.
                      if(lineToLowerCase.startsWith("allow:")) {
                      .....
                      } else
                      //replace using the line object with lineToLowerCase string object.
                      if(lineToLowerCase.startsWith("disallow:")) {
                      .....
                      } else {
                      .....
                      }
              }
  .....
  }

[Java>Basic Authorization]Sample implementation of Basic Authorization Logic in JavaServlet manually.

References

1. About WWW-Authenticate, Authorization header
   http://www.tohoho-web.com/ex/http.htm#WWW-Authenticate
   http://www.tohoho-web.com/ex/http.htm#Authorization
2. Basic Authentificator Sample written in Java
   http://www.katch.ne.jp/~h-inoue/tips/javaservlet/0001.html
3. About Digest Authorization(Japanese)
   https://www.codeblog.org/blog/inoue/20060227.html
   http://shain.tomocreative.net/2007/03/phpdigest.html(for PHP)
4. MD5, Base64 useful Library
   ostermiller utility, http://ostermiller.org/
   

Customization Example

/*
     Created by Tatsuya Anno
     Sample implementation of Basic Authorization Logic in JavaServlet manually.
*/
     String authorizationMethod="Basic";
     String wwwAuthenticateValue="BASIC realm=\"Basic Auth Test\"";

     //check authorization header
     //Get Authorization Header information
     //Authorization header is encoded in Http Header based on following format.
     //"BASIC [Username]:[Password]", value of [Username]:[Password] is encoded by Base64.
     //Sample value of authorization header is "Basic YW50YXRzdTp0ZXN".
     String authorization=request.getHeader("Authorization");
     if(authorization==null||authorization.equals("")){
          //if there is no Authorization Header, return SC_UNAUTHORIZED status.
          response.setHeader("WWW-Authenticate",wwwAuthenticateValue);
          response.sendError(HttpServletResponse.SC_UNAUTHORIZED);
          return null;
     }
     else{
          if(!authorization.startsWith(authorizationMethod)){
               //If there is unexpected authorization header, return SC_UNAUTHORIZED status.
               //If we use basic auth logic, the vlaue of authorization header starts with "Basic".
               response.setHeader("WWW-Authenticate",wwwAuthenticateValue);
               return null;
          }
          
          //Remove "BASIC" characters from the value of Authorization Header
          //before decoding base64 value to Username:Password value.
          String encodedHeader=authorization.substring(6);
          
          //decode BASE64 characters
          //base64 decoder by using sun.misc.BASE64Decoder
          //but sun.misc.BASE64Decoder class is not public.
          //(Please refer to http://www.source-code.biz/snippets/java/2.htm)
          //sun.misc.BASE64Decoder decoder = new sun.misc.BASE64Decoder();
          //byte[] decodedBytes = decoder.decodeBuffer(encodedHeader);
          //----
          //I changed the base64 utility from sun.misc.BASE64Decoder to ostermiller utility.
          //decode base64 encoded characters by using Ostermiller Utility.
          //Please refer to the http://ostermiller.org/
          byte[] decodedBytes=com.Ostermiller.util.Base64.decodeToBytes(encodedHeader);
          if(decodedBytes==null){
               //If the decoded byte values is null, return UNAUTHORIZED status
               response.setHeader("WWW-Authenticate",wwwAuthenticateValue);
               response.sendError(HttpServletResponse.SC_UNAUTHORIZED);
               return null;
          }
          
          //create new string based on the byte values decoded from base64
          String decodedAuthHeader=new String(decodedBytes);
          if(decodedAuthHeader==null||decodedAuthHeader.equals("")){
               response.setHeader("WWW-Authenticate",wwwAuthenticateValue);
               response.sendError(HttpServletResponse.SC_UNAUTHORIZED);
               return null;
          }
          //get username and password from decoded string object.
          //split decoded string which is separated by ":".
          String[] authTokens=decodedAuthHeader.split(":");
          if(authTokens.length<2){
               response.setHeader("WWW-Authenticate",wwwAuthenticateValue);
               response.sendError(HttpServletResponse.SC_UNAUTHORIZED);
               return null;
          }
          
          //If system fails to Split string, return SC_UNAUTHORIZED status because of unexpected.
          if(authTokens[0]==null||authTokens[0].equals("")){
               response.setHeader("WWW-Authenticate",wwwAuthenticateValue);
               response.sendError(HttpServletResponse.SC_UNAUTHORIZED);
               return null;
          }

          //check the values of username and password, following is a sample logic.
          if(!authTokens[0].equalsIgnoreCase("auth_test")||!authTokens[1].equals("12345"))
          {
               //if fails to check username and password, return SC_UNAUTHORIZED status
               response.setHeader("WWW-Authenticate",wwwAuthenticateValue);
               response.sendError(HttpServletResponse.SC_UNAUTHORIZED);
               return null;
          }
     }
     //end of manual basic authorization

[Java>JavaMail] Sometimes "java.net.ConnectException: Connection refused" occures.

[Description]
I'm using Sendmail SMTP Server in our Web System.
Sometimes in our web systems, "java.net.ConnectException: Connection refused" were occured when accessing SMTP Server via Java Servlet Programs. I tried to reproducing this problem again and again, but I could not reproduced this problem manually. I tried to simulate the operations to reproduce this problem in parallel by using Apache JMeter, I could reproduce this problem .

[Error Details]
java.net.ConnectException: Connection refused -> final: xxxxx.jp(Domainname): javax.mail.MessagingException: Could not connect to SMTP host: localhost, port: ;

And I got the result of executing ps command as follows to check the sendmail process when this problem occured.
xxxxx 21249 1 0 12:22 ?
00:00:00 sendmail: rejecting connections on daemon MTA: load average: 13

[Cause]
In case of this, this problem occured only when the CPU load average was high.
We are using Sendmail and qpopper as SMTP system, Sendmail has a functionality to reject connecting to the SMTP system.

[Workaround]
We can control rejecting the user's connection by using following parameters when CPU load is high. We set these parameters into sendmail.mc, try recreating sendmail.cf and restart sendmail process.
define(`confQUEUE_LA',`30')dnl
define(`confDELAY_LA',`40')dnl
define(`confREFUSE_LA',`100')dnl

[JAVA>Velocity, Template]customization how to convert String to Template Object

References

1. Apache Velocity Home: http://velocity.apache.org/

Description
I added following customization logic which convert String java object to Velocity Template object because I sometimes needed template object dynamically without creating template file.
(Please let me know if you have any other good ideas.)

Customization Example Steps

1. get Velocity Source code from velocity.apache.org site.
   
2. open org.apache.velocity.Template.java file.
   
3. add java class import statements in Template.java
   

   import org.apache.velocity.runtime.RuntimeSingleton;
   import org.apache.velocity.runtime.RuntimeConstants;
   import java.io.StringReader;
   

4. add customized constructor function of Template object as follows.
   

   /*
          customized constructor which convert String Object to Template Object.
          @param pTemplateName is an template name
          @param pTemplateString is String object you want to render
          @return a velocity template object.
   */
          public Template(String pTemplateName,String pTemplateString)
                  throws Exception
          {
                  //Initialization
                  RuntimeSingleton.init();
   
                  setRuntimeServices(RuntimeSingleton.getRuntimeServices());
                  setName(pTemplateName);
                  //set encodeing if you need
                  //setEncoding(RuntimeConstants.ENCODING_DEFAULT);
   
                  data = null;
                  errorCondition = null;
                  StringReader   stringReader=null;
                  BufferedReader bufferedReader=null;
   
                  try{
                          stringReader=new StringReader(pTemplateString);
                          bufferedReader=new BufferedReader(stringReader);
   
                          data = rsvc.parse(bufferedReader,name);
                          initDocument();
                  }catch(Exception e){
                          errorCondition=new Exception("Error occured while initializaing String Template, customization");
                          throw errorCondition;
                  }finally{
                          if(stringReader!=null){
                                  stringReader.close();
                          }
                          if(bufferedReader!=null){
                                  bufferedReader.close();
                          }
                  }
          }
   /*
          End of Customization
   */
   

5. compile new java file.
   

how to use this customization

String testTemplate="Velocity Template Test";
Template template=new Template("TemplateName",testTemplate);

[JAVA>JML, MSN Messenger] When using JML(Java MSN Messenger Library), NullPointerException occured

References

1. Reference Site
   Java MSN Messenger Library (JML) Home: http://java-jml.sourceforge.net/

Problems
In case of using Java MSN Messenger Library(JML), NullPointerException occures when adding a friend into Friend List or removing a friend from Friend List, but this problem doesn't always occured.

Reason why this problem occures
System checks whether a friend is already added into the list or not before adding or removing a friend. When checking a friend, System accesses the user's contact list object and get contact information from the contact list object.
When a friend doesn't contain the user's contact list, system returns the null value. And then, system accesses the object even though the value of the contact object is null.

Solution Example
Change addFriend() and removeFriend() function int the net.sf.jml.impl.SimpleMessenger.java file as follows. You need to check whether the value of MsnContact object is nujll or not.

1. addFriend function
   

   
   private void addFriend(MsnList list, Email email, String friendlyName) {
          if (list == null || email == null || list == MsnList.RL
                  || list == MsnList.PL)
              return;
          MsnContact contact = getContactList().getContactByEmail(email);
   /*
   *      added by tatsuya anno to avoid NullPointerException
   *      because contact object is null when adding new user
   */
          if(contact!=null){
                  if (contact.isInList(list)){
                          return;
                  }
          }
   /*
   *      End of Bugfix
   */
   .......
   }
   

   
   
   
2. removeFriend function
   

   
   private void removeFriend(MsnList list, Email email, String id,
              String groupId) {
          if (list == null || list == MsnList.RL)
              return;
          if (list == MsnList.FL) {
              if (id == null)
                  return;
          } else if (email == null)
              return;
          MsnContact contact = getContactList().getContactByEmail(email);
   
   /*
   *      added by tatsuya anno to avoid NullPointerException
   *      because contact object is null when adding new user
   */
          if(contact!=null){
                  if (!contact.isInList(list)){
                          return;
                  }
          }
   /*
   *      End of Bugfix
   */
   ......
   }