Insertion Sort Algorithm

1. Algorithm Description

Insertion Sort is one of the simplest, but less effective, sorting algorithms . It works very similar to the way (us humans) sort a hard of playing cards:

1. At first none of our cards are sorted . So we start with an “empty” space in our hand were we are going to insert cards one at a time .
2. We take a card from our hand and we put it in the “special place” were we planned to keep our cards sorted . We do this for every card in our initial hand, until we finish them off .
3. Finding the right position for the insertion is not hard . At this point we can apply two tactics :

  • We compare the card we are planning to insert with all the cards that are already in sorted state O(n);
  • We use binary search to determine the right position (O(logn)) .

In our case that “special place” were we are going to insert cards one at a time, is not an additional array, but the array itself . We know for sure that an array of size 1 is always sorted, so we consider that first sorted sub-array, is the block composed by the first element .

For example, we want to sort the following array (with bold, are the elements already sorted):

8 0 3 11 5 -1 14 10 1 1 -2 Array is unsorted .
8 0 3 11 5 -1 14 10 1 1 -2 The first sorted block is {8} .
0 8 3 11 5 -1 14 10 1 1 -2 We compare {0} with {8} we move {0} at the new position, we shift {8} .
0 3 8 11 5 -1 14 10 1 1 -2 We compare {3} with {8}, {0}, we move {3} at the new position, we shift {8} .
0 3 8 11 5 -1 14 10 1 1 -2 We compare {11} with {8}, {11} remains at the same position .
0 3 5 8 11 -1 14 10 1 1 -2 We compare {5} with {11}, {8} and {3}, we move {5} at the new position, we shift {8}, {11} .
-1 0 3 5 8 11 14 10 1 1 -2 We compare {-1} with {11}, {8}, …, {0}, we move {-1} at the new position, we shift {0}, {3}, … {11} .
-1 0 3 5 8 11 14 10 1 1 -2 We compare {14} with {11}, we move {11} at the new position .
-1 0 3 5 8 10 11 14 1 1 -2 We compare {10} with {14}, {11}, {8}, we move {10} at the new position, we shift {11}, {14}.
-1 0 1 3 5 8 10 11 14 1 -2 We compare {1} with {14}, {11}, …, {0}, we move {1} at the new position, we shift {3}, {5}, …, {14} .
-1 0 1 1 3 5 8 10 11 14 -2 We compare {1} with {14}, {11}, …, {1}, we move {1} at the new position, we shift {3}, {5}, …, {14} .
-2 -1 0 1 1 3 5 8 10 11 14 We compare {-2} with {14}, {11}, …, {-1}, we move {-2} at the new position, we shift {-1}, {0}, …, {14} .

The pseudo-code for the algorithm can be easily written as follows:

2. Algorithm Implemenation in Java

The Java implementation of the algorithm is pretty straight-forward (follow the comments): Read More

Bottom-up Merge Sort (non-recursive)

1. Algorithm Description

In the last article I’ve described a recursive version of the Merge Sort Algorithm . Of course every recursive algorithm can be written in an iterative manner . So today I am going to present the bottom-up version of the same algorithm, written in a non-recursive fashion .

The main idea of the bottom-up merge sort is to sort the array in a sequence of passes . During each pass the array is divided into smaller sub-arrays of a pseudo-fixed size (step) . Initially step is 1, but the value increases with every pass, as every adjacent sub-arrays are merged, thus step doubles .

Example.:

0. We consider an array with size < = 1 sorted .
1. The array that needs to be sorted is A = { 5, 2, 1, 12, 2, 10, 4, 13, 5} . At this point step = 1 .
2. At the first iteration array A is divided into blocks of size step = 1 . The resulting blocks (sub-arrays) are {5}, {2}, {1}, {12}, {2}, {10}, {4}, {13}, {5} .
3. step *= 2, thus step is now 2 . At this point we have a collection of sorted sub-arrays (because their size is = 1) . We will group the sub-arrays one-by-one and we will start merging them . After the merge, the resulting sub-arrays are: {2, 5}, {1,12}, {2,10}, {4, 13} and {5} . {5} remains unpaired as the array size is an odd number . We will take care of this block later .
4. step *= 2, thus step is now 4 . Now we have a collection of 4 blocks of size two and one block of size one . We will start to merge again the adjacent blocks, so the sub-arrays collection becomes: {1, 2, 5, 12}, {2, 4, 10, 13} and {5} .
5. step *= 2, thus step is now 8 . Now we have a collection of 2 blocks with size 4 and one block with size 1 . We will merge the adjacent blocks so the sub-arrays collection becomes {1, 2, 2, 4, 5, 10, 12, 13} and {5} .
6. We now have two blocks one of size 8 and one of size 1 . We will merge those blocks and obtain the resulting sorted array: {1, 2, 2, 4, 5, 5, 10, 12, 13} .

The pseudo code for the algorithm can be written as follows . We will start by writing the merge function . This is responsible with the merging of two already sorted blocks (sub-arrays) from a given array . The input parameters for this function are : the array itself, and the interval headers (stop, start) for the two already sorted blocks . The sentinel is a concept that simplifies the code . In our case we consider SENTINEL infinity (no value from the array is bigger or equal to the Sentinel) .

And the actual function responsible with the actual merge can be written as follows:

Where A is the unsorted-but-soon-to-be-sorted array, length represents the size of A, step is the current size of the block, startL and startR represent the starting indexes of the sorted blocks that are going to be merged .

2. Algorithm implementation in Java Read More

The merge sort algorithm (implementation in Java)

1. Description

In computer science many algorithms are recursive by nature and they usually follow an approach called divide-and-conquer . In this particular case dividing-and-conquering a problems means to break it into smaller identical sub-problems, resolve the smaller sub-problems recursively (by dividing the smaller sub-problems into even smaller problems), and eventually combine all their results to determine the solution for the initial problem .

The Merge Sort algorithm is based on this technique :

1. The merge sort algorithm works on arrays (or lists). If the array has its size < = 1, then we consider the array already sorted and no change is done . This is considered to be the limit condition of this recursive approach . [Limit Condition]
2. If size of the array (or list) > 1, then we divide the array into two equally-sized sub-arrays . [Divide] – O(1)
3. At this point we recursively sort both of the two arrays . [Conquer] 2*T*(n/2)
4. We merge the two sorted sub-arrays into one sorted array . [Merge] O(n)

We will start by defining the MERGE function . This is function is responsible with the merging of two sorted arrays. The resulting array (in our case portion of the initial array composed by the two merged sub-arrays) will be also sorted :

Where:

L The initial unsorted array .
llim The starting index of the left sub-array;
mlim The stopping index for right sub-array, and the starting index for the right sub-array;
rlim The stopping index for the right sub-array;
LEFTL, RIGHTL Additional helper arrays .
SENTINEL A value to simplify our code, there’s nothing “bigger” than the Sentinel .

After we define the MERGE function, the actual MERGE-SORT function can be written as:

2. Java implementation Read More

Converting infix to RPN (shunting-yard algorithm)

If you’ve tried to write your own calculator (something in the style of gcalctool) you’ve probably had to build a simple converter for your mathematical expressions from infix notation to  RPN (Reverse Polish Notation).

Inspired by this classical SPOJ challenge I wanted to write my own simplified version of an expression converter. If this topic is new for you, or you need to refresh your memory please don’t skip the following paragraph.

Infix notation is the common arithmetic and logical formula notation, in which operators are written infix-style between the operands they act on (e.g. 2 + 5). Unfortunately what seems natural for us, is not as simple to parse by computers as prefix or RPN notations.

RPN also known as the Reverse Polish Notation is mathematical notation wherein every operator (eg. + – * %) follows all of its operands. Examples:

Infix notation Reverse Polish Notation
A + B A B +
A ^ 2 + 2 * A * B + B ^ 2 A 2 ^ 2 A * B * + B 2 ^ +
( ( 1  + 2 ) / 3 ) ^ 4 1  2 + 3 / 4 ^
( 1 + 2 ) * ( 3 / 4 ) ^ ( 5 + 6 ) 1 2 + 3 4 / 5 6 + ^ *

In order to parse and convert a given infix mathematical expression to RPN we will use the shunting-yard algorithm . Just like the evaluation of RPN, the algorithm is stack-based . For the conversion we will use two buffers (one for input, and one for output). Additionally we will use a stack for operators that haven’t been yet added to the output.

A simplified version of the Shunting-yard algorithm (complete version)
  • For all the input tokens [S1]:
    • Read the next token [S2];
    • If token is an operator (x) [S3]:
      • While there is an operator (y) at the top of the operators stack and either (x) is
        left-associative and its precedence is less or equal to that of (y), or (x) is right-associative
        and its precedence is less than (y) [S4]:

        • Pop (y) from the stack [S5];
        • Add (y) output buffer [S6];
      • Push (x) on the stack [S7];
    • Else If token is left parenthesis, then push it on the stack [S8];
    • Else If token is a right parenthesis [S9]:
      • Until the top token (from the stack) is left parenthesis, pop from the stack to the output buffer [S10];
      • Also pop the left parenthesis but don’t include it in the output buffer [S11];
    • Else add token to output buffer [S12].
  • While there are still operator tokens in the stack, pop them to output [S13]

Note: [SN] Relate with code.

I) Java Implementation

The following implementation of the shunting-yard algorithm does not impose any validations. The input should be a valid mathematical expression or else the program may end abruptly or Read More