07 December, 2006

MySQL and JDBC - Updates by One Connection Object Not Visible Through Another?

Stumbled upon this while developing a multi-threaded Java application that connects to MySQL via JDBC. Basically, i have one thread updating a database table through a Connection (java.sql.Connection) object (connection1), and another querying the same table through a different connection object (connection2). What i found was that, after connection2 does its first query against the table, subsequent similar queries (by connection2) always return the same result (as the very first one) even though in between, connection1 has executed several updates against that table.

i wrote out a simple single-threaded test application to examine this and to ensure that it was not due to some other application bugs or an oversight:

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;

public class TestMySqlDatabase
{
    private static final String CLASSNAME = "com.mysql.jdbc.Driver";

    // Database URL
    private static final String URL
            = "jdbc:mysql://192.168.1.1:3306/test";

    // Database User Name
    private static final String USERNAME = "root";

    // Database User Password
    private static final String PASSWORD = "password";
    
    public static final void main(String[] args)
            throws ClassNotFoundException, SQLException
    {
        // connection0 is used to CREATE and DROP
        // the test table
        Connection connection0 = null;
        
        // connection1 is used to update the test table
        Connection connection1 = null;
        
        // connection2 is used to query the test table
        Connection connection2 = null;
        
        Statement statement = null;
        ResultSet resultSet = null;
        
        try
        {
            Class.forName(CLASSNAME);
            
            // CREATE the test table
            try
            {
                connection0
                        = DriverManager.getConnection(
                                URL, USERNAME, PASSWORD);
                connection0.setAutoCommit(false);
                
                statement = connection0.createStatement();
                statement.executeUpdate(
                        "CREATE TABLE TestTable "
                                + "(TestColumn VARCHAR(1))");
                
                connection0.commit();
            }
            finally
            {
                if (statement != null)
                {
                    try
                    {
                        statement.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
                
                if (connection0 != null)
                {
                    try
                    {
                        connection0.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
            }
            
            connection1
                    = DriverManager.getConnection(
                            URL, USERNAME, PASSWORD);
            connection1.setAutoCommit(false);
            
            connection2
                    = DriverManager.getConnection(
                            URL, USERNAME, PASSWORD);
            connection2.setAutoCommit(false);
            
            // Add one row to the test table
            try
            {
                statement = connection1.createStatement();
                statement.executeUpdate(
                        "INSERT INTO TestTable "
                                + "(TestColumn) VALUES ('a')");
                
                connection1.commit();
            }
            finally
            {
                if (statement != null)
                {
                    try
                    {
                        statement.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
            }
            
            // Query the test table and print result(s)
            try
            {
                statement = connection2.createStatement();
                resultSet
                        = statement.executeQuery(
                                "SELECT * FROM testtable");
                
                System.out.println("Query 1:");
                while (resultSet.next())
                {
                    System.out.println(
                            resultSet.getString("TestColumn"));
                }
            }
            finally
            {
                if (resultSet != null)
                {
                    try
                    {
                        resultSet.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
                
                if (statement != null)
                {
                    try
                    {
                        statement.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
            }
            
            // Add a second row to the test table
            try
            {
                statement = connection1.createStatement();
                statement.executeUpdate(
                        "INSERT INTO TestTable "
                                + "(TestColumn) VALUES ('b')");
                
                connection1.commit();
            }
            finally
            {
                if (statement != null)
                {
                    try
                    {
                        statement.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
            }
            
            // Query the test table and print result(s) again
            try
            {
                statement = connection2.createStatement();
                resultSet
                        = statement.executeQuery(
                                "SELECT * FROM testtable");
                
                System.out.println("Query 2:");
                while (resultSet.next())
                {
                    System.out.println(
                            resultSet.getString("TestColumn"));
                }
            }
            finally
            {
                if (resultSet != null)
                {
                    try
                    {
                        resultSet.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
                
                if (statement != null)
                {
                    try
                    {
                        statement.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
            }
            
            // DROP the test table
            try
            {
                connection0
                        = DriverManager.getConnection(
                                URL, USERNAME, PASSWORD);
                connection0.setAutoCommit(false);
                
                statement = connection0.createStatement();
                statement.executeUpdate("DROP TABLE TestTable");
                
                connection0.commit();
            }
            finally
            {
                if (statement != null)
                {
                    try
                    {
                        statement.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
                
                if (connection0 != null)
                {
                    try
                    {
                        connection0.close();
                    }
                    catch (SQLException ignore)
                    {
                        //
                    }
                }
            }
        }
        finally
        {
            if (connection1 != null)
            {
                try
                {
                    connection1.close();
                }
                catch (SQLException ignore)
                {
                    //
                }
            }
            
            if (connection2 != null)
            {
                try
                {
                    connection2.close();
                }
                catch (SQLException ignore)
                {
                    //
                }
            }
        }
    }
}


The output of this test application turned out to be:

Query 1:
a
Query 2:
a


i.e. The results of the first and second queries were the same, even though a second record was inserted before the second query was performed.

This led me to do some online searching, and i found that the answer to this lies with the transaction isolation level setting. The default setting for MySQL is REPEATABLE READ, and this means that within a transaction, identical queries will return identical results. Because connection2 (above) had auto-commit set to false, and there was no commit (on connection2) executed in between its two query invocations, those two queries are considered to be within a single transaction. Hence, identical results for the two identical queries.

The few common transaction isolation level values are explained in detail in this article, so do give that a read as well.

Finally, the transaction isolation level setting for MySQL can be modified by changing the tx_isolation system variable using the MySQL Administrator.

28 November, 2006

How to Solve a Polynomial Number Sequence

Given a polynomial sequence (e.g. 1, 2, 4, 7, 11), how can we easily find the next number in that sequence?

First of all, each of the numbers in a polynomial sequence is a result of a polynomial function on n (where n = 1, 2, 3, ...). In other words, f(n) = amnm + am-1nm-1 + am-2nm-2 + ... + a2m2 + a1m + a0. The example above (1, 2, 4, 7, 11) can be represented by the polynomial function f(n) = 0.5n2 - 0.5n + 1.

Checking,
f(1) = 0.5(1)2 - 0.5(1) + 1 = 1
f(2) = 0.5(2)2 - 0.5(2) + 1 = 2
f(3) = 0.5(3)2 - 0.5(3) + 1 = 4
f(4) = 0.5(4)2 - 0.5(4) + 1 = 7
f(5) = 0.5(5)2 - 0.5(5) + 1 = 11

Hence, the next number in the sequence is
f(6) = 0.5(6)2 - 0.5(6) + 1 = 16

With this knowledge, how can we find the next number in a given polynomial sequence? Initially, i tried to find a0, a1, ..., am in the function f(n) = amnm + am-1nm-1 + am-2nm-2 + ... + a2m2 + a1m + a0 and work on from there. That would work, but for sure it was very tedious, especially if we have a high-degree polynomial function.

I then did a search for other methods to do this, and came across this page: http://www.everything2.com/index.pl?node_id=1026695. The first idea (by ariels) on that page is a simple and powerful method for solving polynomial sequences without having to solve for the coefficients of the function itself. You can read up on the method by visiting the link itself; my contribution here would be a Java programme which makes use of the algorithm described by that method:

import java.util.ArrayList;
import java.util.List;

public class Main
{
    public static final void main(String[] args)
    {
        int numInput = args.length;

        List<int[]> sequencesList = new ArrayList<int[]>(100);

        int currentSeqLen = numInput;
        int[] currentSequence = new int[currentSeqLen + 1];
        boolean constantFound = true;

        for (int i = 0; i < currentSeqLen; i++)
        {
            currentSequence[i] = Integer.parseInt(args[i]);

            if (constantFound
                    && (i > 0)
                    && (currentSequence[i] != currentSequence[i - 1]))
            {
                constantFound = false;
            }
        }

        sequencesList.add(currentSequence);

        while (!constantFound)
        {
            int[] prevSequence = currentSequence;

            currentSeqLen--;
            currentSequence = new int[currentSeqLen + 1];
            constantFound = true;

            for (int i = 0; i < currentSeqLen; i++)
            {
                currentSequence[i]
                        = prevSequence[i + 1] - prevSequence[i];

                if (constantFound
                        && (i > 0)
                        && (currentSequence[i] != currentSequence[i - 1]))
                {
                    constantFound = false;
                }
            }

            sequencesList.add(currentSequence);
        }

        for (int i = currentSeqLen; i < (currentSeqLen + 1); i++)
        {
            currentSequence[i] = currentSequence[i - 1];
        }

        for (int i = (sequencesList.size() - 2); i >= 0; i--)
        {
            int[] prevSequence = currentSequence;

            currentSeqLen++;
            currentSequence = sequencesList.get(i);

            for (int j = currentSeqLen; j < (currentSeqLen + 1); j++)
            {
                currentSequence[j]
                        = currentSequence[j - 1] + prevSequence[j - 1];
            }
        }

        System.out.println(currentSequence[numInput]);
    }
}


This programme takes in a sequence of numbers as arguments, and outputs the next number of that polynomial sequence.

E.g.
Input: java Main 1 2 4 7 11
Output: 16

14 November, 2006

Ubuntu 6.10 - How to Have a Minimal Graphical Desktop

The desktop installation of Ubuntu is touted as a complete Linux-based operating system, and includes many of the major applications. Indeed, Firefox, OpenOffice.org, Gaim, GIMP, and the Totem Movie Player are just some of the applications that come pre-installed if you used the desktop CD.

The downside of this is that if you have an older system with limited resources (memory, hard disk space, etc.), you may not want to have all of those applications pre-installed. One way around this is to manually un-install those applications that you do not want. However, that can be quite a pain if you intend to remove more than a few of them, and if you are really THAT low on hard disk space, you will not be able to get the desktop installation done in the first place.

The alternative is to start with a minimal graphical desktop and almost none of the applications installed. You then have the flexibility to add an application only as and when you need it. We will explore the two steps needed to achieve this.

1. Download the server install CD (instead of the desktop CD), and install from that.

One thing to note is that the server install CD functions differently from the desktop CD. The server install CD does not function as a live CD, so do not expect to boot to a graphical desktop to do the installation. The installation procedure is purely text-based, but otherwise, it is quite straightforward and should take a much shorter time than installing with the desktop CD (less packages to install).

2. Install the packages necessary for a graphical GNOME desktop.

When you boot into the system after you complete step one, all you will get is a text-based system. If you are used to the graphical desktop, this may be a little disconcerting, but do hang in there! The next step is to use apt-get to install the packages necessary for a graphical GNOME desktop.

The packages that you will need are:
  • gdm
  • gnome-applets
  • gnome-control-center
  • gnome-icon-theme
  • gnome-menus
  • gnome-panel
  • gnome-session
  • gnome-terminal
  • menu
  • metacity
  • nautilus
  • synaptic
  • x-window-system-core

The packages listed here are related to the X window system and the GNOME desktop environment, with the exception of nautilus, which is a graphical file manager, and synaptic, which is an application that makes it easier for you to install other applications and packages in future (sort of like a graphical front-end to apt-get).

In a nutshell, the two commands you will have to run from the command line (either as root, or using sudo) are

apt-get update

and

apt-get install gdm gnome-applets gnome-control-center gnome-icon-theme gnome-menus gnome-panel gnome-session gnome-terminal menu metacity nautilus synaptic x-window-system-core

Once that is done, reboot, and you should get a graphical log in prompt this time.

There you have it. A graphical desktop installation for Ubuntu without all the heavy applications.

Reference:
Ubuntu Web Forum: What Packages Needed for Basic Installation + GNOME

02 November, 2006

5 Pitfalls Involving Java Exceptions

After looking at a fair number of application source codes written by others, i have noticed that these are some of the most common pitfalls involving exceptions. i must admit that i have committed them at one time or another as well -- earlier, due to ignorance, and later on, due to convenience (but only in trivial, run-once-and-throw utilities). i am sure there are other common mistakes and bad habits in this area, but these few i have come across most often.

1. Catching an exception, then doing absolutely nothing.

try
{
    FileReader in = new FileReader("MyFile.txt");
}
catch (FileNotFoundException ex) {}


This, in my humble opinion, is the single worst thing to do with an exception. Should the exception occur, the offending application continues to run in an indeterminate state. At best, another exception (usually a NullPointerException) will terminate the application a few lines later. At worst, it just continues to inexplicably (to the programmer) churn out erroneous results.

There is a, well, exception to this though. When we clean up resources in finally blocks (e.g. closing connections, streams, etc.), We typically enclose those statements in try-catch blocks to allow the finally block to run to completion.

InputStream in = null;

try
{
    // Do Stuffs Here
}
finally
{
    if (in != null)
    {
        try
        {
            in.close();
        }
        catch (IOException ignore)
        {
            // Ignore
        }
    }
}


In such cases, it is "forgiveable" to ignore the exception. The failure to close a resource usually does not prevent an application from continuing to run correctly, and there is also little corrective action that can be done about it. However, it is still better to log the exception (perhaps with a lower priority), so that if indeed a resource leakage is detected later on (and resource leakages are usually not obvious right away), the cause of the leakage can be traced.

2. Allowing an exception to manifest itself to the user, and terminating the application.

For desktop applications, this means throwing exceptions out of the main method.

public static void main (String[] args)
        throws IOException, FileNotFoundException
{
    // Do Stuffs
}


This is bad because usually, the user cannot do anything about the exception that just showed up, and in most cases, the stack trace would not even make any sense to him. In the worst case, the user could be halfway through a critical piece of work when the application crashed on him.

3. Not ensuring that the finally block completes normally.

As mentioned above, an important practice is to make use of finally blocks to clean up resources before they are garbage-collected, and these cleaning-up statements are wrapped in try-catch blocks, but sometimes, it is not done correctly.

FileInputStream in = null;
OutputStream out = null;

try
{
    // Do Stuffs
}
finally
{
    try
    {
        in.close();
        out.close();
    }
    catch (IOException ignore)
    {
        // Log Exception
    }
}


The problem here is that, if the in.close() statement throws an exception, the out.close() statement will not even be attempted, and the output stream will be left open. In more serious cases, this could lead to resource leakages. The more correct way to do this, would be to wrap each clean-up statement individually in its own try-catch block.

FileInputStream in = null;
OutputStream out = null;

try
{
    // Do Stuffs Here
}
finally
{
    if (in != null)
    {
        try
        {
            in.close();
        }
        catch (IOException ignore)
        {
            // Log Exception
        }
    }

    if (out != null)
    {
        try
        {
            out.close();
        }
        catch (IOException ignore)
        {
            // Log Exception
        }
    }
}


4. Throwing ExceptionS.

Throwing exceptions in general is not a big issue, it is the throwing of an instance of the Exception class (as opposed to a specific sub-class) that sometimes causes a loss of clarity in the code.

private String readFromFile(String filename) throws Exception
{
    FileInputStream in = null;

    try
    {
        in = new FileInputStream(in);

        // Do Stuffs
    }
    finally
    {
        if (in != null)
        {
            try
            {
                in.close();
            }
            catch (IOException ignore)
            {
                // Log Exception
            }
        }
    }
}


In the readFromFile method above, there are possibly FileNotFoundExceptionS and IOExceptionS that need to be handled. One way to handle them would be to catch them within the method itself (and appropriately handling them). Declaring throws FileNotFoundException, IOException in the method header is fine too, but declaring an all-purpose throws Exception is not a good practice, and there are a couple of reasons to this.

Firstly, the throws clause in a method header is considered to be part of the interface contract. Following the principles of abstraction and encapsulation, declaring specific exceptions (and providing the corresponding Javadoc comment) would allow a programmer to understand how this method could potentially fail without having to delve into the implementation details. Declaring a generic throws Exception simply tells the programmer that, well, this method could potentially fail.

Secondly, declaring specific thrown exceptions would allow the calling method to handle each exception separately in different catch blocks. In the example above, the calling method would have to handle the exception in a generic manner. Should it be more appropriate for the calling method to throw the exception instead of handling it, it would also have to declare a generic throws Exception.

5. Throwing the wrong exception.

private void setValue(String s) throws ClassNotFoundException
{
    if (s == null)
    {
        throw new ClassNotFoundException();
    }

    this.value = s;
}


This is a rather contrived example. The next example is a more common sighting, but it is not much more better off either.

private void setValue(String s) throws Exception
{
    if (s == null)
    {
        throw new Exception();
    }

    this.value = s;
}


This sort of follows the argument in the previous point about the importance of the declared exception. Always throw the appropriate exception type. If an appropriate exception is not available, by all means create a new sub-class of Exception. In the examples above, the most appropriate exception to throw would perhaps be the IllegalArgumentException.

12 October, 2006

What's a Podcast?

Despite its recent popularity, a large majority still do not know what a podcast is. Hence, i thought that i could use a little space here to spread the word around a bit.

The Number One Myth: You need an iPod to be able to listen to a podcast.

That's a natural line of thought, and that was the misconception i had as well, when i first encountered the term.

So, what's a podcast?

According to Wikipedia (http://en.wikipedia.org/wiki/Podcast),

A podcast is a multimedia file distributed over the Internet using syndication feeds, for playback on mobile devices and personal computers.


That definition just about nicely sums it all up. If you still have difficulties grasping the concept, just think a radio programme or segment saved as an MP3 (or any other audio format), which you can listen to just as you would any other MP3 file, at any time.

The first advantage of a podcast is that anyone can have his/her own podcast. There is no need to own/hijack a radio station to produce a podcast. Hence, there is large number of podcasts out there, of virtually any genre.

Secondly, as a listener, you can listen to a podcast at any time, as opposed to having to tune in at a specific time to catch your favourite radio programme.

Some of the podcasts that i listen to frequently:


03 October, 2006

Stand Up Against DRM

Today, 3rd October, is the Day Against DRM. Take a stand today. Fight DRM.

October 3rd is the Day Against DRM

02 October, 2006

How to Have Flickr Badge Link Open in New Window

If you have a Flickr account, as well as a personal homepage or blog elsewhere, it is possible to add a "Flickr badge" to your homepage or blog. A "Flickr badge" is simply made up of one or more photos from your Flickr collection (either the most recent or just a random selection). To get the script fragment for adding this, sign in to your Flickr account, go to the site map, and follow the link for "add a Flickr badge to your website". After a bit of configuring (number of photos, layout style, etc.), you will be presented with the code that you can copy and paste elsewhere.

When a visitor to your website clicks on your Flickr badge, he/she will be taken to the displayed photo on the Flickr page itself. However, one downside of this is that the Flickr page will be loaded in the same window, which may not be what you desire. (I am not too sure about this, but if your website contains frames, the link may actually open in its enclosing frame, and that would not be ideal.)

As the code fragment for displaying the Flickr badge is solely Javascript, there is no HTML anchor element for you to add a target attribute to. Hence, to have the link open in a new window, a bit of a Javascript hack is needed.

The following is the code fragment for displaying the Flickr badge. Yours may differ according to the selections you made when generating it (and obviously, your user ID would be different from mine).

<script
    src='http://www.flickr.com/badge_code_v2.gne?count=1
        &display=random&size=t&layout=x
        &source=user&user=97681386%40N00'
    type='text/javascript'>
</script>


Just below that, add the following:

<script type='text/javascript'>
    var flickrBadgeImage1Div
        = document.getElementById('flickr_badge_image1');
    var a = flickrBadgeImage1Div.firstChild;
    a.setAttribute('target', '_new');
</script>


Basically, this additional Javascript code fragment looks for the correct anchor element, and adds the target attribute to it. How do you know that the div element that encloses the anchor element has an id of flickr_badge_image1? Simply open your website in Firefox, use of the DOM Inspector, and work your way down to the DOM element that contains the Flickr badge.

25 September, 2006

Tip on Event Handling in C# (Compact Framework)

Learnt something while doing some C# programming for the compact framework. There are times when we have code like this to handle the clicking of a button:

private Button btnDoSomething;

private void InitializeComponent()
{
    this.btnDoSomething = new Button();

    this.btnDoSomething.Click
            += new EventHandler(BtnDoSomething_Click);
}

private void BtnDoSomething_Click(Object sender, EventArgs e)
{
    this.btnDoSomething.Enabled = false;

    DoSomething();

    this.btnDoSomething.Enabled = true;
}

private void DoSomething()
{
    // Do Something Here
}


What we are trying to achieve here is to disable the button while the processing is being done, so that it cannot be clicked on for the time being.

However, one problem with this approach is that, while the button is disabled, if it is clicked on, the event will be queued, and fired when it is enabled again, as if the action is performed twice in quick succession.

Here is why. There is an event queue where such events (of clicking on a button) are queued, and an event handler thread that picks up events from the queue and handles them. While the event handler thread is working on one event, new events are queued, and not picked up by the event handler thread immediately. In the case above, when the "Do Something" button is first clicked, the event handler thread will pick up that event and handle it, which is to call the method BtnDoSomething_Click. While this is being done (typically the method DoSomething may take a bit of time), if the button is clicked again, this new event is queued. When the event handler thread finally gets to handle this queued event, the "Do Something" button will already be enabled again (by the last statement in the BtnDoSomething_Click method) and hence the clicking of the button is handled again accordingly.

To get around this, there is an easy way, and a right way. First the easy way:

private void BtnDoSomething_Click(Object sender, EventArgs e)
{
    this.btnDoSomething.Enabled = false;

    DoSomething();

    Application.DoEvents();

    this.btnDoSomething.Enabled = true;
}


Application.DoEvents will cause the event handler thread to handle all queued events at that point in time. This works because, at that point in time, the "Do Something" button is still disabled. So, forcing the click events to be handled at that point in time, it is as if we are simply clearing the events queue. However, the downside is that Application.DoEvents forces ALL queued events to be handled, which means, e.g., if you have other buttons on the form, queued click events from those buttons will also be handled at that time, and things may get messy (remember that while this is all taking place, we are still inside the BtnDoSomething_Click method).

The right way is to do the processing (which is triggered by the clicking of the "Do Something" button) on a seperate thread, so that the event handler thread is quickly freed up to handle new events (including clicking on the same button again while it is still disabled):

private void BtnDoSomething_Click(Object sender, EventArgs e)
{
    this.btnDoSomething.Enabled = false;

    Thread doSomethingThread
            = new Thread(new ThreadStart(DoSomething));
    doSomethingThread.Start();
}

private void DoSomething()
{
    // Do Something Here

    this.btnDoSomething.Enabled = true;
}