Tuesday, March 31, 2009

Using Java I/O API to read and write integers as bytes to a binary file

Java I/O API is very vast. I have shown how to use BufferedInputStream and BufferedOutput stream to read and write integers as bytes to a file.My test case only involves 5 integers but the program is meant for a large number of integers.

My input stream is binary and not character. So I have used the stream type BufferedInputStream. I have wrapped the plain FieInputStream with the BufferedInputStream so that the input can be buffered. Then I have used the following method int read(byte[] b,int off,int len) to read up to len bytes of data from the stream into an array of bytes, thereby, improving performance.

I have used the BufferedOutputStream to write all the bytes at once instead of using the plain FileOutputStream.

Efficient way to read integers from a file of bytes

import java.io.*;

public class ReadFile {

private static final int numOfBytesInInt = 4;

* This method reads bytes from a file and converts them to integers.
* @param name filename
* @param numOfBytes Number of bytes to read
* @return Integer array
public static int[] getIntegersStoredAsBytesFromFile(String name, int numOfBytes) throws IOException {
byte[] b = readBytesFromFile(name,numOfBytes);
return convertBytesToInteger(b);

* This method actually reads bytes from a file in one shot(hopefully).
* It is more efficient than reading 1 integer or byte at a time in a file.
* @param name filename
* @param numOfBytes Number of bytes to read
public static byte[] readBytesFromFile(String name, int numOfBytes) throws IOException {
BufferedInputStream bis = null;
byte[] b = new byte[numOfBytes];
try {
bis = new BufferedInputStream(new FileInputStream(name));
return b;
} finally {

* This helper method actually converts each 4 bytes into an integer.
* There is also another way of doing it through bit manipulation.
* @param b The byte array to convert to int
* @return The array of ints
public static int[] convertBytesToInteger(byte[] b) throws IOException{
ByteArrayInputStream bai = new ByteArrayInputStream(b);
DataInputStream dis = new DataInputStream(bai);
int len = b.length;
int[] ints = new int[len/numOfBytesInInt]; //4 bytes = 1 int
for (int i = 0,j = 0; i < len; i += numOfBytesInInt, j++) {
ints[j] = dis.readInt();
return ints;



Efficient way to write integers as bytes to a file

import java.io.*;
import java.util.Random;

public class WriteFile {

private static final int numOfBytesInInt = 4;

* This method generates integers using random number
* generator, converts each integer to 4 bytes and
* writes it to a file
* @param name filename
* @param numOfBytes Number of bytes to write
public static void generateIntegersAndStoreInFileAsBytes(
String name, int numOfBytes) throws IOException {
//Generate a byte[] of integers
byte[] b = generateIntegersAsBytes(numOfBytes);
//Write them to the file

* This helper method actually generates the integers
* using random number generator and converts
* each integer to 4 bytes
* @param numOfBytes Number of bytes to generate
* @return Array of bytes
public static byte[] generateIntegersAsBytes(int numOfBytes) throws IOException {
byte[] b = new byte[numOfBytes];
int randomNumber = 0;
Random r = new Random();
for (int offset = 0; offset < b.length; offset += numOfBytesInInt) {
//Generate random int
randomNumber = r.nextInt();
//Just have it here to show what is written into the file
//is what we are reading later..
//convert int to bytes
intToByte(randomNumber, b, offset);
return b;

* This method writes all the bytes to a file in a one shot(hopefully without blocking).
* It is more efficient than writing 1 integer at a time in a file.
* @param name filename
* @param b Array of bytes to write
public static void writeBytesToFile(String name, byte[] b) throws IOException{
BufferedOutputStream bos = null;
try {
bos = new BufferedOutputStream(new FileOutputStream(name));
} finally {

* This helper method actually converts each integer to 4 bytes.
* See the next method for another way to do the same thing.
* @param integer Integer to be converted to bytes
* @param b The byte array into which the bytes have to be stored
* @param offset The start offset in b
public static void intToByte(int integer, byte[] b, int offset) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(bos);
System.arraycopy(bos.toByteArray(), 0, b, offset, numOfBytesInInt);

/* Instead of using the intToByte method, one could also manually convert
* the integer to an array of bytes using bit arithmetic.
* Here is a way to do it. To use this method, just replace the call
* to the method intToByte() with generateByte() above
* private static void generateByte(int integer, byte[] b, int offset) {
b[offset]=(byte)((integer & 0xff000000)>>>24);
b[offset+1]=(byte)((integer & 0x00ff0000)>>>16);
b[offset+2]=(byte)((integer & 0x0000ff00)>>>8);
b[offset+3]=(byte)((integer & 0x000000ff));



Test It

import java.io.*;
import static java.lang.System.out;

public class TestingFiles {

private static final String filename = "INPUTBYTES";

* I am just testing with 5 integers but you can practically
* use any number which is a multiple of 4.
private static final int numOfBytes = 20;

public static void main(String[] args) throws IOException{
out.println("Writing to File");
out.println("Reading from File");
int[] ints = ReadFile.getIntegersStoredAsBytesFromFile(filename,numOfBytes);
for (int i: ints) {


Here is the output:
Writing to File
Reading from File

Wednesday, March 25, 2009

Getting started with google app engine

Its pretty straight forward to create an application on Google App Engine. I am briefly listing the steps involved in creating and deploying applications along with some mistakes that one could potentially make.

1. After you download the SDK (I used a Mac), start the GoogleAppEngine Launcher. This opens up a nice UI which I henceforth refer to as 'Launcher UI'
NOTE: When the launcher comes up, allow it to create symlinks. Basically this creates links for some very useful scripts (like appcfg.py , dev_appserver.py etc) in /usr/local/bin. This is very important if you plan to use the appcfg.py or the dev_appserver.py scripts from command line instead of using the Launcher UI. if you do not create the symlinks, when you run the appcfg.py script using the full path to it, the script may terminate with errors like "......ImportError: import google not found....."

Launcher UI

2. Log in to the google app engine website and register an application.

3. Follow the steps 'Creating Simple Request Handler' and 'Creating the Configuration File' listed here.
NOTE: the 'application' label in app.xml should be same as the Application Identifier you use when you create an application. If they are not, when you deploy the application on the app engine you will get errors like:
appcfg.py:1235 An unexpected error occurred. Aborting.
Traceback (most recent call last):
File "/Users/annika/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appcfg.py", line 1213, in DoUpload...........
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)HTTPError: HTTP Error 403: Forbidden

4. Test the application using the script dev_appserver.py as listed in the above website under the section 'Testing the application' or you can use the Launcher UI. In the Launcher UI, use the File -> 'Add Existing Application' since you have already created the application in step 3. Clicking the run button will run it on the default app server on your local host. Clicking the browse button will launch the browser and you can view your application on your local host. Clicking the stop button will stop the app server.
NOTE: In the launcher UI if you choose File -> 'New Application', and provide it the directory of your existing application, it will create a brand new application directory within your existing application directory. It will automatically create default app.yaml and .py file in the new directory. This will be the application that will be deployed and not your existing application.

5. Upload the application using the appcfg.py script as listed here or click the deploy button on the launcher UI.

6. You can modify the application by editing the .py file and can give it a different version in app.yaml file. Then you can redeploy the app using step 5.
NOTE:If the version in app.yaml is different from the original version(which is considered default), and you wish to make the newly uploaded file the default version for your application, go to your application dashboard in the app engine website, click on versions and make the newly uploaded file your default version, otherwise you will not see the changes. Remember only the default version shows up as the application in the website.

Monday, March 23, 2009

Simple RMI example

Here is a very simple RMI example. The writeup about RMI can be found in my previous post here.

Remote Interface

import java.rmi.Remote;
import java.rmi.RemoteException;

public interface Product extends Remote{

public String getDescription() throws RemoteException;


Implementation of the Remote Interface:

import java.rmi.server.*;
import java.rmi.*;
import working.testing.myrmi.remoteinterface.*;

public class ProductImpl implements Product {

private String type;

public ProductImpl(String type)throws RemoteException{
this.type = type;

public String getDescription() throws RemoteException{
return ("The Product is " + type);

The Server

import java.rmi.*;
import java.rmi.server.*;
import java.rmi.registry.*;
import working.testing.myrmi.remoteinterface.*;

public class MyRMIServer {

public static void main(String[] args) {
String toaster = "toaster";
try {
/** If an RMI program does not install a
* security manager, RMI will not download
* classes (other than from the local class path)
* for objects received as arguments or
* return values of remote method invocations.
if (System.getSecurityManager() == null) {
(new RMISecurityManager());
Product product = new ProductImpl(toaster);
* Since the server dosent extend
* UnicastRemoteObject,
* First we need to export the
* object. Once the remote object has been
* successfully exported it is ready for
* invocations. However for clients to
* invoke the remote object, it needs to
* be bound to the registry with a name.
* If extending UnicastRemoteObject,
* We should do only the following
* Naming.rebind(toaster,product);
* because it automatically exports
* the remote object for us
Product stub = (Product)UnicastRemoteObject.
Registry registry = LocateRegistry.createRegistry(5555);
System.out.println("Waiting for clients");
} catch (Exception e) {
System.out.println("The Exception is " + e);



The client

import working.testing.myrmi.remoteinterface.*;
import java.rmi.*;
import java.rmi.registry.*;

public class MyRMIClient {

public static void main(String[] args) {
try {
/** If an RMI program does not install a
* security manager, RMI will not download
* classes (other than from the local class path)
* for objects received as arguments or
* return values of remote method invocations.
if (System.getSecurityManager() == null) {
(new RMISecurityManager());

* First we need to get a handle
* to the registry on the server machine and
* then we need to lookup for the remote
* object
* We can also do
* Product product =
* (Product)Naming.lookup
* ("rmi://host:1099/toaster");
* where host is the hostname where
* the server is running.
* host will be localhost if the server is
* running locally. Sometimes putting in
* localhost can cause
* some unnecessary exceptions, so just do
* lookup("rmi://:1099/toaster") if
* on localhost
* Also while getting registry if on localhost
* better to do just getRegistry() instead of
* getRegistry("localhost") to avoid similar
* exceptions as above.
Registry registry = LocateRegistry.
Product product = (Product)registry.
/*Product product = (Product)Naming.
} catch (Exception e) {
System.out.println("The exception is " + e);

Sunday, March 22, 2009


Here is a short summary about RMI. A simple example can be found in my post here.

A RMI based application has the following minimum pieces:

1. RemoteInterface (import java.rmi.*;)

- Defines the interface.
- The interface extends java.rmi.Remote and declares remote methods.
- Each remote method throws java.rmi.RemoteException.

public interface RemoteInterface extends Remote {
public String concat(String s1, String s2) throws RemoteException;

2. Implementation of the RemoteInterface (import java.rmi.*;)

- Implements all methods of remote interface.
- Has a constructor which throws RemoteException

public class RemoteInterfaceImpl implements RemoteInterface{
//Has constructor which throws RemoteException
public RemoteInterfaceImpl() throws RemoteException{
//Implements all methods of the RemoteInterface
public String concat(String s1, String s2) throws RemoteException{

3. The server (import java.rmi.registry.*; import java.rmi.*;)

- Creates and exports the remote objects.
- Gets handle to the RMI registry. (It runs at default port 1099.
if you start at a different port specify port when trying to get registry).
- Bind/register the remote object stubs with the registry.

public class Server {
public static void main(String[] args) {
try {
RemoteInterfaceImpl rii = new RemoteInterfaceImpl();
RemoteInterface stub = UnicastRemoteObject.exportObject(rii, 0);

Registry registry = LocateRegsitry.getRegistry();
registry.bind("concat", stub);

catch(RemoteException e) {

4. Client (import java.rmi.registry.*;)

- Do following:

1. Get the stub to the registry on the remote host.If registry running on different port provide it too.
Registry registry = LocateRegistry.getRegistry(host);

2. Lookup the remote object's stub using its name in the registry.
ri = (RemoteInterface)registry.lookup("concat");

3. Invoke the methods on the remote object using the stub.
String c = ri.concat("1","2");


concat is a method defined in the RemoteInterface
The object returned or method parameters must be Serializable (Remember: String class implements Seriableable)
RemoteException must be caught and do something valuable with it.

5.Running them

- Compile source files using javac.
- start the rmiregistry. (uses port 1099 by default).
- start the server.
- start the client.

More on RMI:

Here are a few high level things to remember about RMI:

Passing parameters in RMI

RMI deals with 3 types of parameters:

- Primitive paramters: Always passed by value. Copy is made and sent across.
- Object parameters: A copy of the object itself is made and sent across. Note, the reference to the object is not passed the object itself is passed. The object should be Serializable. If the object is composed of other objects, they are also sent across, so they must also be Serializable.
- Remote Object parameters: The remote object parameter is substituted with the remote proxy/stub and sent across.


Garbage collection of remote objects:

- It is done using Distributed Garbage Collection and reference counting.
- When the client creates a remote reference, it lets the server side know this by calling dirty() on the Distributed Garbage Collector(DGC) of the server.
- The client keeps renewing the lease periodically on the remote reference by calling dirty().
- Once the client is done with the remote reference(GCed on the client), the client calls clean() on th server-side DGC, indicating that the server-side DGC no longer needs to keep the remote object alive for this client
- The server-side DGC keeps track of all its remote references and does not GC them and if they are being used.


- RMIRegistry is purely for bootstraping purposes, so that clients can get a remote object stub. If the client can get a remote object stub through another mechanism then the rmiregistry is not needed.
- rmiregistry and server can run on different machines but sometimes the registry may give AccessException if you try to bind from non local host.I have never tried this, so cannot provide more insight on this.
- You can also create the registry from within your code using createRegistry() API.If you do this, you dont have to explicity start the rmiregistry, it is started when the server is started.
Registry registry = LocateRegistry.createRegistry(port);

Exporting Objects:

- UnicastRemoteObject.exportObject(), exports the remote object receive incoming remote invocation by making it listen on an anonymous TCP port in the machine.
- It returns a stub to the remote object which has the info on the port on which the remote object is listening.
- This remote object stub is returned to the client when it does a lookup on the rmiregistry. Now the client through the remote stub knows how to contact and call methods on the remote object.

RMI stubs:

- rmic is no longer required to generate stub post 1.5. stub classes are generated at runtime.
- When a stub's method is invoked, it does the following:

- initiates a connection with the remote JVM containing the remote object,
- marshals (writes into byte stream and transmits) the parameters to the remote JVM,
- waits for the result of the method invocation,
- unmarshals (reads) the return value or exception returned, and
- returns the value to the caller.


Wednesday, March 11, 2009

Nesting Java enums in an interface

Continuing on my post about enums from yesterday...

Just like a class can be nested inside an interface, an enum can also be nested inside an interface. Nested enums in an interface are implicitly static and public. Here is an example.

interface SoundInterface {
public void makeSound();

//nested enum
enum WesternStringInstruments implements SoundInterface {
PIANO("Piano"), GUITAR("Guitar"),

private String name;
name = null;

WesternStringInstruments(String name) {
this.name = name;

@Override public String toString() {
if (name != null) {
return name;
return this.name();
public void makeSound(){
System.out.println("String sound");


public class InterfaceWithEnum {

//Using the nested enum
public static void main(String[] args) {
for( SoundInterface.WesternStringInstruments st:
SoundInterface.WesternStringInstruments.values()) {

Here is the output:
String sound

Monday, March 9, 2009

Java enums

Listed below are a few highlights about Java enums. It is followed by an example.
- enum is a type in Java
- used to represent a fixed set of constants. so the enum constants are implicitly public static final.
- is Serializable
- implements Comparable
- is NOT Cloneable
- implicitly extends the class java.lang.Enum so they cannot extend any other class or enum. So is implicitly final.
- cannot be abstract
- can have the access modifiers public or default.
- can be defined outside the class or as a class member. They cannot be defined inside methods. Enums defind inside a class are implicitly static.
- can have one or more constructors but they cannot be called explicitly. So better declare them private.
- cannot be instantiated using new operator.
- can have methods including abstract methods. An enum constant can override any number of available methods. Abstract methods have to be implemented by each and every constant of that enum.
- name(), valueOf() and defualt implementation of the toString() methods is to return the name of the enum constant. The enum constants can override toString() method to return something else.
- can be used as an operand in the instanceof operator
- enum constants can be used in switch statements.
- '==' and equals yield the same results for enum constants.
- natural order of the enum constants is the way they are defined and is used by compareTo(), values(), EnumSet.range().
- ordinal() returns the natural ordering of the enum constants starting at the index at 0 for the first constant.
- cannot be generic

public class MyEnumType {

  //Simple enum
  public enum TrainType {
    STEAM,DIESEL //; is optional
  //Complex enum
  public enum ThomasAndFriends {
    //Enums can have constructors
    THOMAS(1,"blue"), EDWARD(2,"blue"),GORDON(4,"blue"),
      //Can override method
      @Override public TrainType getTrainType() {
        return TrainType.DIESEL;
    private int engineNumber;
    private String color;
    //Constructor's access can be private or default.
    ThomasAndFriends(int engineNumber, String color) {
      this.engineNumber = engineNumber;
      this.color = color;
    public int getEngineNumber(){
      return engineNumber;
    public String getColor(){
      return color;
    public TrainType getTrainType(){
      return TrainType.STEAM;
  public static void main(String[] args) {
    //Walk through the enumeration and print values
    for (ThomasAndFriends tf: ThomasAndFriends.values()) {
      //Enums have ordinals but do not use them. Code dosent 
      //scale, instead use constructors to specify values.
      System.out.println("Name: " + tf + 
              " EngineNumber: " + tf.getEngineNumber() +
              " Enum Ordinal: " + tf.ordinal() +
              " Color: " + tf.getColor() +
              " TrainType: " + tf.getTrainType());  


Name: THOMAS EngineNumber: 1 Enum Ordinal: 0 Color: blue TrainType: STEAM
Name: EDWARD EngineNumber: 2 Enum Ordinal: 1 Color: blue TrainType: STEAM
Name: GORDON EngineNumber: 4 Enum Ordinal: 2 Color: blue TrainType: STEAM
Name: DAISY EngineNumber: 9 Enum Ordinal: 3 Color: black TrainType: DIESEL
Name: JAMES EngineNumber: 5 Enum Ordinal: 4 Color: red TrainType: STEAM
Name: PERCY EngineNumber: 6 Enum Ordinal: 5 Color: green TrainType: STEAM

Saturday, March 7, 2009

Configuring MySQL for the Sun App Server

Last winter, I took a course on "Designing J2EE Enterprise Applications" at UC Berkeley Extension. While doing the course, though not required, I ended up installing MySQL server and configuring it for the Sun App server. Due to lack of documentation on the configuration, I went through some trial and error but finaly figured out the configuration. Here are the steps:

Installing database server and setting up
Install mysql server
Create a database
Create tables, users as suitable
Install j connector JDBC driver
Add the connector to your system classpath (computer->system->advanced->env variables)
Open admin console of server
Add the jdbc driver (jar fle) to the JVM settings->path settings.

Configuring connection resource in appserver
Open admin console of server
add a new connection pool under resources->connectionPool.
Select new.
Name: enter any name to identify this connection pool. Select javax.sql.DataSource
for resource type. Database Vendor see mysql and select it or select your vendor or leave blank if your vendor do not appear. click next.
Data Source Class Name: enter your class name i.e. Find your database driver name and enter here. For mysql 5.0, it is com.mysql.jdbc.jdbc2.optional.MysqlDataSource
Scroll down to add Properties. Add the following:
user, password, databaseName, port(3306:default), serverName(localhost)
Now connection pool is created for the database.

Now create the resource
In admin console go to resources->jdbc resources
Add a new jndi name for jdbc resources
Go back to connection pool and ping
Click new.
Enter the JNDI Name you will use to identify the connection pool in your application i.e ejb/customer or anything you want . Select the Pool Name you just configured from the list and click save.
Go back to Connection Pools navaigational link and select your database you configured.
Click ping. If you get errors that means it is not yet working. So try again.