Andrei Ciobanu – andreINC

Writing an unified validation mechanism for REST APIs using Spring Boot and JSR-380

October 2, 2017November 2, 2017 Andrei Ciobanu 3 Comments

JSR-380 defines a metadata model and API for Java Bean validation. It can be used an “architectural-agnostic” way and it is particularly useful when it comes to validating the RESTful APIs (syntactic validation).

The default metadata source is @Annotations, with the ability to override and extend the metadata through the use of XML validation descriptors or custom code:

// source: http://www.baeldung.com/javax-validation
public class User {
 
    @NotNull(message = "Name cannot be null")
    private String name;
 
    @AssertTrue
    private boolean working;
 
    @Size(min = 10, max = 200, message = "About Me must be between 10 and 200 characters")
    private String aboutMe;
 
    @Min(value = 18, message = "Age should not be less than 18")
    @Max(value = 150, message = "Age should not be greater than 150")
    private int age;
 
    @Email(message = "Email should be valid")
    private String email;
 
    // standard setters and getters 
}

// source: http://www.baeldung.com/javax-validation

public class User {

@NotNull(message = "Name cannot be null")

private String name;

@AssertTrue

private boolean working;

@Size(min = 10, max = 200, message = "About Me must be between 10 and 200 characters")

private String aboutMe;

@Min(value = 18, message = "Age should not be less than 18")

@Max(value = 150, message = "Age should not be greater than 150")

private int age;

@Email(message = "Email should be valid")

private String email;

// standard setters and getters

}

JSR-380 was finished in August, 2017 and it’s now a Java EE 8 standard. It’s also included by default in Spring Boot through the only available implementation – Hibernate Validator.

By default, JSR-380 doesn’t standardise a big list of validating @Annotations. Hibernate Validator adds a few extra, and enhances the existing ones, but it’s still not enough. As a developer working for a real world application you will probably need to write custom constraints.

That’s why i’ve compiled myself a library of common annotations (that are not in the standard or in Hibernate Validator). The library and code can be found on github under the name JBVExt (Java Bean Validation Extensions).

The project setup

The “example” project is a simple Spring Boot project. The main dependencies are:

dependencies {
    compile ("org.springframework.boot:spring-boot-starter-web")
    compile 'net.andreinc.jbvext:jbvext:0.0.6'
    compileOnly('org.projectlombok:lombok')
}

dependencies {

compile ("org.springframework.boot:spring-boot-starter-web")

compile 'net.andreinc.jbvext:jbvext:0.0.6'

compileOnly('org.projectlombok:lombok')

}

By default the spring-boot-starter-web includes Hibernate Validator, so there’s no need to explicitly define it as a dependency.

Decorating the RESTful API

The API we are going to validate is composed by two REST web-services:

– POST /user/ : For creating an user;
– POST /post/ : For submitting the text;

The requestBodies are being mapped to the following two classes.

@Data
@AllArgsConstructor
@NoArgsConstructor
class CreateUserReq {
    // Needs to be alphanumeric (with no space)
    // Max Size = 32, Min Size = 6
    private String userName;

    // Needs to be a valid email address
    private String email;

    // Can have two values: "MALE" and "FEMALE"
    private String gender;

    // Needs to be in the format: "yyyy-MM-dd"
    private String dateOfBirth;

    // It's a non-null string
    // Starts with "A"
    // EndsWith "00"
    private String appCode;
}

@Data

@AllArgsConstructor

@NoArgsConstructor

class CreateUserReq {

// Needs to be alphanumeric (with no space)

// Max Size = 32, Min Size = 6

private String userName;

// Needs to be a valid email address

private String email;

// Can have two values: "MALE" and "FEMALE"

private String gender;

// Needs to be in the format: "yyyy-MM-dd"

private String dateOfBirth;

// It's a non-null string

// Starts with "A"

// EndsWith "00"

private String appCode;

}

The comments describe each of the field-level constraints.

For example in the above class the field appCode needs to always start with “A” and end in “00”.

@Data
@AllArgsConstructor
@NoArgsConstructor
class CreatePostReq {
    // Needs to be a valid email
    private String authorEmail;
   
    // Contains only alphanumeric characters and spaces
    // Max String size is 3, max size is 256
    private String title;

    // Non-null string in the format: "dd-MM-yyyy"
    private String publishDate;

    // Not null or empty
    // Min size is 0, max size is 1000
    // Can contain even non-ASCII characters
    private String text;

    // Each string in the list is alphanumeric with no spaces
    private List<String> tags;
}

@Data

@AllArgsConstructor

@NoArgsConstructor

class CreatePostReq {

// Needs to be a valid email

private String authorEmail;

// Contains only alphanumeric characters and spaces

// Max String size is 3, max size is 256

private String title;

// Non-null string in the format: "dd-MM-yyyy"

private String publishDate;

// Not null or empty

// Min size is 0, max size is 1000

// Can contain even non-ASCII characters

private String text;

// Each string in the list is alphanumeric with no spaces

private List<String> tags;

}

Note: The @Data, @AllArgsConstructor and @NoArgsConstructor are not related with JSR-380, but are part of Project Lombok.

In order to implement the constraints at the field level we would normally to write custom Java code. This type of code can become repetitive (and less readable) as soon the application grows.

JSR-380 proposes a new “decorative” approach. So instead to add the validation code at the controller level, we could just decorate the model layer with @Annotations.

In conclusion the CreateUserReq.java becomes:

@Data
@AllArgsConstructor
@NoArgsConstructor
class CreateUserReq {

    @Alphanumeric
    @Size(min = 6, max = 32)
    private String userName;

    @Email
    private String email;

    @OneOfStrings({"MALE", "FEMALE"})
    private String gender;

    @IsDate("yyyy-MM-dd")
    private String dateOfBirth;

    @StartsWith("A")
    @EndsWith("00")
    private String appCode;
}

@Data

@AllArgsConstructor

@NoArgsConstructor

class CreateUserReq {

@Alphanumeric

@Size(min = 6, max = 32)

private String userName;

@Email

private String email;

@OneOfStrings({"MALE", "FEMALE"})

private String gender;

@IsDate("yyyy-MM-dd")

private String dateOfBirth;

@StartsWith("A")

@EndsWith("00")

private String appCode;

}

And the CreatePostReq.java class becomes:

@Data
@AllArgsConstructor
@NoArgsConstructor
class CreatePostReq {

    @Email
    private String authorEmail;

    @AlphanumericSpace
    @Size(min = 3, max = 256)
    private String title;

    @IsDate("dd-MM-yyyy")
    private String publishDate;

    @NotNull
    @Size(min = 0, max = 10000)
    private String text;

    private List<@Alphanumeric String> tags;

}

@Data

@AllArgsConstructor

@NoArgsConstructor

class CreatePostReq {

@Email

private String authorEmail;

@AlphanumericSpace

@Size(min = 3, max = 256)

private String title;

@IsDate("dd-MM-yyyy")

private String publishDate;

@NotNull

@Size(min = 0, max = 10000)

private String text;

private List<@Alphanumeric String> tags;

}

Creating an unified way of intercepting validation errors.

The JBVExt library contains an util method SimpleValidation.validate(…) that will throw a BeanValidationException exception the first time a constraint is violated.

So we will make each controller of our API invoke this method before doing anything else:

@PostMapping("/user/")
public ResponseEntity<?> createUser(@RequestBody CreateUserReq request) {
        // If the TestBodyReq fails the validation a BeanValidationException will be thrown.
        SimpleValidation.validate(request);
        System.out.println("Request received: " + request);
        return status(OK).build();
}

@PostMapping("/user/")

public ResponseEntity<?> createUser(@RequestBody CreateUserReq request) {

// If the TestBodyReq fails the validation a BeanValidationException will be thrown.

SimpleValidation.validate(request);

System.out.println("Request received: " + request);

return status(OK).build();

}

@PostMapping("/post/")
public ResponseEntity<?> createPost(@RequestBody CreatePostReq request) {
        // If the TestBodyReq fails the validation a BeanValidationException will be thrown.
        SimpleValidation.validate(request);
        System.out.println("Request received: " + request);
        return status(OK).build();
}

@PostMapping("/post/")

public ResponseEntity<?> createPost(@RequestBody CreatePostReq request) {

// If the TestBodyReq fails the validation a BeanValidationException will be thrown.

SimpleValidation.validate(request);

System.out.println("Request received: " + request);

return status(OK).build();

}

The next step will be define a “Global” Exception Handler that will intercept any BeanValidationException thrown from our Controllers and treat them in the same manner.

The @ControllerAdvice and @ExceptionHandler annotations (part of Spring) will become handy:

@ControllerAdvice
public class BeanValidationExceptionCatch extends ResponseEntityExceptionHandler {

    /**
     * This method acts as a "Global" catch block for all the exceptions of type BeanValidationException.
     * We will use to identify all the "mal-formatted" requests that are made against our REST API and treat them
     * in a generic way.
     *
     * @param ex Can be cast to (BeanValidationException).
     * @param webRequest The webRequest.
     * @return A ResponseEntity that is a BAD_REQUEST and contains the error message supplied by the JSR-380 validation.
     */
    @ExceptionHandler(value = {BeanValidationException.class})
    public ResponseEntity<?> handleValidationException(RuntimeException ex, WebRequest webRequest) {
        BeanValidationException exception = (BeanValidationException) ex;
        return handleExceptionInternal(exception, exception.getMessage(), new HttpHeaders(), HttpStatus.BAD_REQUEST, webRequest);
    }

}

@ControllerAdvice

public class BeanValidationExceptionCatch extends ResponseEntityExceptionHandler {

/**

* This method acts as a "Global" catch block for all the exceptions of type BeanValidationException.

* We will use to identify all the "mal-formatted" requests that are made against our REST API and treat them

* in a generic way.

* @param ex Can be cast to (BeanValidationException).

* @param webRequest The webRequest.

* @return A ResponseEntity that is a BAD_REQUEST and contains the error message supplied by the JSR-380 validation.

@ExceptionHandler(value = {BeanValidationException.class})

public ResponseEntity<?> handleValidationException(RuntimeException ex, WebRequest webRequest) {

BeanValidationException exception = (BeanValidationException) ex;

return handleExceptionInternal(exception, exception.getMessage(), new HttpHeaders(), HttpStatus.BAD_REQUEST, webRequest);

}

So instead of having to repeat all the try/catch code in each controller, we can define this unified strategy in a separate class, that will threat each exception (of type BeanValidationException) in the same way.

Running the code

The code is available on git:

git clone https://github.com/nomemory/spring-boot-jbvext-example

1	git clone https://github.com/nomemory/spring-boot-jbvext-example

The Spring Boot application will run by default on port 8080.

If we try to POST http://localhost:8080 with an invalid bodyRequest (as described by the JSR380 validation):

Simple authentication with Spring Boot and JWT Tokens

September 10, 2016October 30, 2017 Andrei Ciobanu 4 Comments

In the following article I am going to prove how you can secure a REST API (developed with Spring Boot) with JWT tokens. For simplicity Spring Security will not be used.

It is assumed the reader is already familiar with JWT.

Our Rest API will contain 3 endpoints, 2 public and 1 private (that can only be accepted with JWT):

/api/public/hello/{name} : Public web service that prints hello.
/api/secure/hello/{name} : Private web service that prints hello. Can only be called if the JWT token exist on the header. Otherwise returns HTTP 403.
/api/public/auth/ : Authentication service. Based on user/pass credentials generates and valid JWT token.

All the code is available on github:

git clone https://github.com/nomemory/spring-boot-jwt-example

1	git clone https://github.com/nomemory/spring-boot-jwt-example

Project is bootstrapped using Spring Initialzr together with gradle.

The generated build.gradle file is:

buildscript {
	ext {
		springBootVersion = '1.4.1.RELEASE'
	}
	repositories {
		mavenCentral()
	}
	dependencies {
		classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
	}
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'spring-boot'

jar {
	baseName = 'simple-jwt'
	version = '0.0.1-SNAPSHOT'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8

repositories {
	mavenCentral()
}


dependencies {
	// tag::jetty[]
	compile("org.springframework.boot:spring-boot-starter-web") {
		exclude module: "spring-boot-starter-tomcat"
	}
	compile("org.springframework.boot:spring-boot-starter-jetty")
	// end::jetty[]
	// tag::actuator[]
	compile("org.springframework.boot:spring-boot-starter-actuator")
	// end::actuator[]
	compile("org.projectlombok:lombok:1.16.10")
	// The JWT library 
	compile group: 'io.jsonwebtoken', name: 'jjwt', version: '0.7.0'
}

buildscript {

ext {

springBootVersion = '1.4.1.RELEASE'

}

repositories {

mavenCentral()

}

dependencies {

classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")

}

apply plugin: 'java'

apply plugin: 'eclipse'

apply plugin: 'idea'

apply plugin: 'spring-boot'

jar {

baseName = 'simple-jwt'

version = '0.0.1-SNAPSHOT'

}

sourceCompatibility = 1.8

targetCompatibility = 1.8

repositories {

mavenCentral()

}

dependencies {

// tag::jetty[]

compile("org.springframework.boot:spring-boot-starter-web") {

exclude module: "spring-boot-starter-tomcat"

}

compile("org.springframework.boot:spring-boot-starter-jetty")

// end::jetty[]

// tag::actuator[]

compile("org.springframework.boot:spring-boot-starter-actuator")

// end::actuator[]

compile("org.projectlombok:lombok:1.16.10")

// The JWT library

compile group: 'io.jsonwebtoken', name: 'jjwt', version: '0.7.0'

}

Observations:

The library that coverts the JWT functionality is called jjwt.
I prefer to use project lombok in my projects. It’s an useful library that can generate getters, setters, constructors, etc. through @Annotations.

We will be starting the project by defining some of the constants. A good idea is to store them in the application.properties file, so we can easily inject them at runtime using @Value annotation.

How to use SeekableByteChannel interface for Random Access to Files (RAF)

December 9, 2013October 4, 2016 Andrei Ciobanu 1 Comment

RAFs, Random Access Files permit asynchronous (random) access to a file contents. To access a file randomly we open the file, seek a particular position, and then we read or write to that file.

Java NIO.2 introduces a new interface – SeekableByteChannel for working with Random Access Files. Also improves the well-known FileChannel class by implementing this interface.

Before we start to talk about about FileChannel and SeekableByteChannel it is advisable to first talk a little bit about ByteBuffers and Channels.

ByteBuffers

A byte-buffer is an in-memory array of bytes. It usually contains data that was recently read, or that will be written from/to a destination.

A buffer has three important properties:

The buffer’s capacity represents the “maximum amount of information” that can be stored in the buffer.
The buffer’s position represents how much data has been read or written. The position is an index in the buffer’s array, and cannot have a negative value or a value bigger than the buffer’s capacity.
The buffer’s limit is the difference between buffer’s capacity and the buffer’s position.

Channels

Channels are in a way similar with the classical I/O streams, the difference is that while streams are one-way directed (read or write), channels can support both operations in the same time. Also Channels allow you to write and read asynchronously.

Using SeekableByteChannel to write / read to / from files

The SeekableByteChannel interface has 6 methods:

position(): Returns the channels current position.
position(long): Sets the channel position to the specified value. The value needs to be a positive number.
truncate(long): Truncates the entity connected to the SeekableByteChannel to the specified value.
read(ByteBuffer): Reads into the buffer (from the channe).
write(ByteBuffer): Writes bytes from the buffer to the channel.
size(): Returns the current size of the entity to which the channel is connected.

To open a SeekableByteChannel we will need to use the the two methods from the java.nio.file.Files class:

public static SeekableByteChannel newByteChannel(Path path, OpenOption... options);

1	public static SeekableByteChannel newByteChannel(Path path, OpenOption... options);

Or:

public static SeekableByteChannel newByteChannel(Path path, Set<? extends OpenOption> options, 
FileAttribute<?>... attrs);

1 2	public static SeekableByteChannel newByteChannel(Path path, Set<? extends OpenOption> options, FileAttribute<?>... attrs);

As you can see to open a seekable channel you need to supply as input the Path you want to open (basically the file) and also open options, which are enum constants.

The possible enum options are as follow:

READ	The file is opened with READ access.
WRITE	The file is opened with WRITE access.
CREATE	Creates the file if the file does not already exist.
CREATE_NEW	Creates the file if the file does not already exist. If the file exists throw an exception.
APPPEND	Appends to the file. It is used in conjunction with CREATE and WRITE.
DELETE_ON_CLOSE	Deletes the file after the channel is closed. Use this when creating and editing / reading from temporary files.
TRUNCATE_EXISTING	Truncates the file to size 0. It is used in conjunction with WRITE and it’s useful when you want to clean the contents of a file.
SPARSE	Usually used in conjunction with CREATE or CREATE_NEW. On some file systems large files that have big “data gaps” are stored in a more efficient way.
SYNC	File content + metadata is synchronized with the underlying file system.
DSYNC	File content is synchronized with the underlying file system.

Reading a file using a SeekableByteChannel

Please follow the comments:

package mainpack;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.SeekableByteChannel;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.EnumSet;

import static java.nio.file.StandardOpenOption.READ;

public class HideAndSeek {
	public static void main(String... args) {
		// We create a Path out the file we are going to read
		Path file = Paths.get(System.getProperty("user.home"), "file.txt");

		// We open the file in order to read it ()
		try (SeekableByteChannel sbc = Files.newByteChannel(file,
				EnumSet.of(READ))) {
			
			// We use a ByteBuffer to read (2^5 size = 32)
			ByteBuffer buff = ByteBuffer.allocate(1<<5);
			// Position is set to 0
			buff.clear();
			
			// We use the current encoding to read 
			String encoding = System.getProperty("file.encoding");
			
			// While the number of bytes from the channel are > 0
			while(sbc.read(buff)>0) {
				
				// Prepare the data to be written
				buff.flip();
				
				// Usins the current enconding we decode the bytes read
				System.out.print(Charset.forName(encoding).decode(buff));
				
				// Prepare the buffer for a new read
				buff.clear();
			}
			
		} catch (IOException ioe) {
			ioe.printStackTrace();
		}
	}
}

package mainpack;

import java.io.IOException;

import java.nio.ByteBuffer;

import java.nio.channels.SeekableByteChannel;

import java.nio.charset.Charset;

import java.nio.file.Files;

import java.nio.file.Path;

import java.nio.file.Paths;

import java.util.EnumSet;

import static java.nio.file.StandardOpenOption.READ;

public class HideAndSeek {

public static void main(String... args) {

// We create a Path out the file we are going to read

Path file = Paths.get(System.getProperty("user.home"), "file.txt");

// We open the file in order to read it ()

try (SeekableByteChannel sbc = Files.newByteChannel(file,

EnumSet.of(READ))) {

// We use a ByteBuffer to read (2^5 size = 32)

ByteBuffer buff = ByteBuffer.allocate(1<<5);

// Position is set to 0

buff.clear();

// We use the current encoding to read

String encoding = System.getProperty("file.encoding");

// While the number of bytes from the channel are > 0

while(sbc.read(buff)>0) {

// Prepare the data to be written

buff.flip();

// Usins the current enconding we decode the bytes read

System.out.print(Charset.forName(encoding).decode(buff));

// Prepare the buffer for a new read

buff.clear();

}

} catch (IOException ioe) {

ioe.printStackTrace();

}

Output:

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer nec odio. Praesent libero. Sed cursus ante dapibus diam. Sed nisi. Nulla quis sem at nibh elementum imperdiet. Duis sagittis ipsum. Praesent mauris. Fusce nec tellus sed augue semper porta. Mauris massa. Vestibulum lacinia arcu eget nulla. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Curabitur sodales ligula in libero. Sed dignissim lacinia nunc. Curabitur tortor. Pellentesque nibh. Aenean quam. In scelerisque sem at dolor. Maecenas mattis. Sed convallis tristique sem. Proin ut ligula vel nunc egestas porttitor. Morbi lectus risus, iaculis vel, suscipit quis, luctu.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer nec odio. Praesent libero. Sed cursus ante dapibus diam. Sed nisi. Nulla quis sem at nibh elementum imperdiet. Duis sagittis ipsum. Praesent mauris. Fusce nec tellus sed augue semper porta. Mauris massa. Vestibulum lacinia arcu eget nulla. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Curabitur sodales ligula in libero. Sed dignissim lacinia nunc. Curabitur tortor. Pellentesque nibh. Aenean quam. In scelerisque sem at dolor. Maecenas mattis. Sed convallis tristique sem. Proin ut ligula vel nunc egestas porttitor. Morbi lectus risus, iaculis vel, suscipit quis, luctu.

Writing a file using SeekableByteChannel

Please follow the comments:

package mainpack;

import static java.nio.file.StandardOpenOption.APPEND;
import static java.nio.file.StandardOpenOption.WRITE;
import static java.nio.file.StandardOpenOption.CREATE;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.SeekableByteChannel;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.EnumSet;

public class HideAndSeek2 {

	public static void main(String... args) {
		// The path were we are going to write
		Path file = Paths.get(System.getProperty("user.home"), "file2.txt");

		// We open the file in APPEND mode with CREATE
		try (SeekableByteChannel sbc = Files.newByteChannel(file,
				EnumSet.of(CREATE, APPEND, WRITE))) {

			// We create the buffer with the text we are going to write
			ByteBuffer buff = ByteBuffer.wrap("Writing text\n".getBytes());
			
			// Write the byte buffer contents to the file and show
			// how many bytes were written
			System.out.println("Bytes written: " + sbc.write(buff));
			buff.clear();
			
		} catch (IOException ioe) {
			ioe.printStackTrace();
		}
	}
}

package mainpack;

import static java.nio.file.StandardOpenOption.APPEND;

import static java.nio.file.StandardOpenOption.WRITE;

import static java.nio.file.StandardOpenOption.CREATE;

import java.io.IOException;

import java.nio.ByteBuffer;

import java.nio.channels.SeekableByteChannel;

import java.nio.file.Files;

import java.nio.file.Path;

import java.nio.file.Paths;

import java.util.EnumSet;

public class HideAndSeek2 {

public static void main(String... args) {

// The path were we are going to write

Path file = Paths.get(System.getProperty("user.home"), "file2.txt");

// We open the file in APPEND mode with CREATE

try (SeekableByteChannel sbc = Files.newByteChannel(file,

EnumSet.of(CREATE, APPEND, WRITE))) {

// We create the buffer with the text we are going to write

ByteBuffer buff = ByteBuffer.wrap("Writing text\n".getBytes());

// Write the byte buffer contents to the file and show

// how many bytes were written

System.out.println("Bytes written: " + sbc.write(buff));

buff.clear();

} catch (IOException ioe) {

ioe.printStackTrace();

}

If we are going to open the file called “file2.txt” from our home folder the content will be:

Writing text

1	Writing text

If we run multiple times the program the content of the file will look like this (the text is appended):

Writing text
Writing text
...
Writing text

Writing text

...

Writing text

Using the SeekableByteChannel to read characters from different file locations

Given the following acrostic:

JANet was quite ill one day.
FEBrile trouble came her way.
MARtyr-like, she lay in bed;
APRoned nurses softly sped.
MAYbe, said the leech judicial
JUNket would be beneficial.
JULeps, too, though freely tried,
AUGured ill, for Janet died.
SEPulchre was sadly made.
OCTaves pealed and prayers were said.
NOVices with ma'y a tear
DECorated Janet's bier.

JANet was quite ill one day.

FEBrile trouble came her way.

MARtyr-like, she lay in bed;

APRoned nurses softly sped.

MAYbe, said the leech judicial

JUNket would be beneficial.

JULeps, too, though freely tried,

AUGured ill, for Janet died.

SEPulchre was sadly made.

OCTaves pealed and prayers were said.

NOVices with ma'y a tear

DECorated Janet's bier.

We want to read the first 3 letters at the beginning of every word and print on the standard output.

Code example:

package mainpack;

import static java.nio.file.StandardOpenOption.READ;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.SeekableByteChannel;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.EnumSet;
import java.util.LinkedList;
import java.util.List;

public class HideAndSeek3 {

	// This list contains indexes for every beginning of the
	// rows
	public static List<Long> indexes = new LinkedList<Long>();
	static {
		indexes.add(0L); // start index for row 1 - JAN
		indexes.add(30L); // start index for row 2 - FEB
		indexes.add(61L); // start index for row 3 - MAR
		indexes.add(91L); // start index for row 4 - APR
		indexes.add(120L); // start index for row 5 - MAY
		indexes.add(152L); // start index for row 6 - JUN
		indexes.add(181L); // start index for row 7 - JUL
		indexes.add(216L); // start index for row 8 - AUG
		indexes.add(246L); // start index for row 9 - SEP
		indexes.add(273L); // start index for row 10 - OCT
		indexes.add(312L); // start index for row 11 - NOV
		indexes.add(338L); // start index for row 12 - DEC
	}

	public static void main(String... args) {
		// The path we are going to open - the file containing the acrostic
		Path acrostic = Paths.get(System.getProperty("user.home"),
				"acrostic.txt");

		// The ByteBuffer is an array of 3 characters
		ByteBuffer buff = ByteBuffer.allocate(3);

		// Obtain encoding
		String encoding = System.getProperty("file.encoding");

		// We open the file in READ mode
		try (SeekableByteChannel sbc = Files.newByteChannel(acrostic,
				EnumSet.of(READ))) {

			// We jump on every index using position , we read the characters
			// in the buffer and we print them on the screen.
			for (Long idx : indexes) {
				sbc.position(idx);
				sbc.read(buff);
				buff.flip();
				System.out.print(Charset.forName(encoding).decode(buff)+" ");
				buff.rewind();
			}

			// Clear buffer
			buff.clear();
		} catch (IOException ioe) {
			ioe.printStackTrace();
		}
	}
}

package mainpack;

import static java.nio.file.StandardOpenOption.READ;

import java.io.IOException;

import java.nio.ByteBuffer;

import java.nio.channels.SeekableByteChannel;

import java.nio.charset.Charset;

import java.nio.file.Files;

import java.nio.file.Path;

import java.nio.file.Paths;

import java.util.EnumSet;

import java.util.LinkedList;

import java.util.List;

public class HideAndSeek3 {

// This list contains indexes for every beginning of the

// rows

public static List<Long> indexes = new LinkedList<Long>();

static {

indexes.add(0L); // start index for row 1 - JAN

indexes.add(30L); // start index for row 2 - FEB

indexes.add(61L); // start index for row 3 - MAR

indexes.add(91L); // start index for row 4 - APR

indexes.add(120L); // start index for row 5 - MAY

indexes.add(152L); // start index for row 6 - JUN

indexes.add(181L); // start index for row 7 - JUL

indexes.add(216L); // start index for row 8 - AUG

indexes.add(246L); // start index for row 9 - SEP

indexes.add(273L); // start index for row 10 - OCT

indexes.add(312L); // start index for row 11 - NOV

indexes.add(338L); // start index for row 12 - DEC

}

public static void main(String... args) {

// The path we are going to open - the file containing the acrostic

Path acrostic = Paths.get(System.getProperty("user.home"),

"acrostic.txt");

// The ByteBuffer is an array of 3 characters

ByteBuffer buff = ByteBuffer.allocate(3);

// Obtain encoding

String encoding = System.getProperty("file.encoding");

// We open the file in READ mode

try (SeekableByteChannel sbc = Files.newByteChannel(acrostic,

EnumSet.of(READ))) {

// We jump on every index using position , we read the characters

// in the buffer and we print them on the screen.

for (Long idx : indexes) {

sbc.position(idx);

sbc.read(buff);

buff.flip();

System.out.print(Charset.forName(encoding).decode(buff)+" ");

buff.rewind();

}

// Clear buffer

buff.clear();

} catch (IOException ioe) {

ioe.printStackTrace();

}

And the output of the code:

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

1	JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

Writing a simple file/folder monitor using the Watch Service API

December 6, 2013October 4, 2016 Andrei Ciobanu 4 Comments

The Watch Service was introduced in Java 7 as a “thread-safe” service responsible for watching objects for changes.

The most popular use of this API would be to use it as way to monitor a folder of files for changes such as: addition of new files, deletion of files, file renames, etc.

Implementing a WatchService

The WatchService is dependent on the FileSystem, the first step of creating a new Service is to obtain the underlying file system:

// Create a new Watch Service
WatchService watchService = FileSystems.getDefault().newWatchService();

1 2	// Create a new Watch Service WatchService watchService = FileSystems.getDefault().newWatchService();

Every object needs to be explicitly registered with the newly created service. In our particular case we will register a Path instance (a folder).

After we register the object, we also need to specific events the service need to watch.

Eg.:

StandardWatchEventKinds.ENTRY_CREATE: This event triggers when a folder entry is created, or a new entry is moved or renamed.
StandardWatchEventKinds..ENTRY_DELETE: This event is triggered when a folder/file is deleted, moved or renamed.
StandardWatchEventKinds.ENTRY_MODIFY: This event is pretty-platform dependent. Usually is triggered when the contents of a file is modified. But on some file systems it can also trigger when the attributes of that particular file are modified.
StandardWatchEventKinds.OVERFLOW: Indicates that an event has been lost.

The following code will register ENTRY_CREATE, ENTRY_DELETE and ENTRY_MODIFY to the home folder:

// Folder we are going to watch
Path folder = Paths.get(System.getProperty("user.home"));

// Create a new Watch Service
WatchService watchService = FileSystems.getDefault().newWatchService();

// Register events
folder.register(watchService, 
		StandardWatchEventKinds.ENTRY_CREATE,
		StandardWatchEventKinds.ENTRY_MODIFY,
		StandardWatchEventKinds.ENTRY_DELETE);
	
// Closes a watch service
watchService.close();

// Folder we are going to watch

Path folder = Paths.get(System.getProperty("user.home"));

// Create a new Watch Service

WatchService watchService = FileSystems.getDefault().newWatchService();

// Register events

folder.register(watchService,

StandardWatchEventKinds.ENTRY_CREATE,

StandardWatchEventKinds.ENTRY_MODIFY,

StandardWatchEventKinds.ENTRY_DELETE);

// Closes a watch service

watchService.close();

For every Path instance we are registering we will receive an WatchKey instance.

To wait for incoming events we will need to write an infinite loop:

for(;;) {}

1	for(;;) {}

while(true) {}

1	while(true) {}

In this loop we will be able to poll for obtaining WatchKey instances.

while(true) {
	// Obtaining watch keys
	final WatchKey key = watchService.poll();
	// key value can be null if no event was triggered
}

while(true) {

// Obtaining watch keys

final WatchKey key = watchService.poll();

// key value can be null if no event was triggered

}

We can also add a timing interval for polling (if we don’t need an instant feedback on the events):

while(true) {
	// Obtaining watch keys every 10 seconds
	final WatchKey key = watchService.poll(10, TimeUnit.SECONDS);
	// key value can be null if no event was triggered
}

while(true) {

// Obtaining watch keys every 10 seconds

final WatchKey key = watchService.poll(10, TimeUnit.SECONDS);

// key value can be null if no event was triggered

}

A WatchKey can have the following states:

Ready: The WatchKey is ready to accept events.
Signaled: In this state the WatchKey has at least one event that occurred and it was queued.
Invalid: In this state the key is considered to be no longer valid.

So the next step will be to retrieve the pending events from the WatchKey. There can be multiple events that were triggered. Those events are queued. The code to retrieve the pending events is:

for (WatchEvent<?> watchEvent : key.pollEvents()) {
	final Kind<?> kind = watchEvent.kind();
	// Overflow event
	if (StandardWatchEventKinds.OVERFLOW == kind) {
		continue; // loop
	}
}

for (WatchEvent<?> watchEvent : key.pollEvents()) {

final Kind<?> kind = watchEvent.kind();

// Overflow event

if (StandardWatchEventKinds.OVERFLOW == kind) {

continue; // loop

}

To obtain the Path from the watchEvent we will need to do something like this:

final WatchEvent<Path> wePath = ( WatchEvent<Path>) watchEvent;
final Path path = wePath.context();

1 2	final WatchEvent<Path> wePath = ( WatchEvent<Path>) watchEvent; final Path path = wePath.context();

The last step of the implementation is to put key back into it’s Ready step.

// Inside the loop
if(key.reset()) { break; }

1 2	// Inside the loop if(key.reset()) { break; }

Also you should also take in consideration that if the loops break (eg.: because of an error) you need to explicitly close the Watch Service:

watchService.close();

1	watchService.close();

Or, since Java 7, include the opening of the WatchService in the new try() block:

try (WatchService watchService = FileSystems.getDefault().newWatchService()) {
…
}

try (WatchService watchService = FileSystems.getDefault().newWatchService()) {

…

}

Putting all the code togheter

Please take in consideration that in our particular case we are only going to watch the Home folder, and not the whole sub-tree of folders. If you wish to watch the whole sub-tree for modifications you will need to register a watch service for every folder in the tree.

To obtain the list of sub-folders of a given folder, I recommend you to read my previous article called: Java 7 NIO.2 – Recursive folder walks

Also you will need to maintain the collection of watchers in the case you are creating/deleting new/existing sub-folders.

The Watch Service API is a low-level approach, so maybe it’s best for you if you write your own high-level mechanism, or use an already-existing solution.

package mainpack;

import static java.nio.file.LinkOption.NOFOLLOW_LINKS;
import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.OVERFLOW;

import java.io.IOException;
import java.nio.file.FileSystem;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.WatchEvent;
import java.nio.file.WatchEvent.Kind;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;

public class MainWatch {

	public static void watchDirectoryPath(Path path) {
		// Sanity check - Check if path is a folder
		try {
			Boolean isFolder = (Boolean) Files.getAttribute(path,
					"basic:isDirectory", NOFOLLOW_LINKS);
			if (!isFolder) {
				throw new IllegalArgumentException("Path: " + path + " is not a folder");
			}
		} catch (IOException ioe) {
			// Folder does not exists
			ioe.printStackTrace();
		}
		
		System.out.println("Watching path: " + path);
		
		// We obtain the file system of the Path
		FileSystem fs = path.getFileSystem ();
		
		// We create the new WatchService using the new try() block
		try(WatchService service = fs.newWatchService()) {
			
			// We register the path to the service
			// We watch for creation events
			path.register(service, ENTRY_CREATE);
			
			// Start the infinite polling loop
			WatchKey key = null;
			while(true) {
				key = service.take();
				
				// Dequeueing events
				Kind<?> kind = null;
				for(WatchEvent<?> watchEvent : key.pollEvents()) {
					// Get the type of the event
					kind = watchEvent.kind();
					if (OVERFLOW == kind) {
						continue; //loop
					} else if (ENTRY_CREATE == kind) {
						// A new Path was created 
						Path newPath = ((WatchEvent<Path>) watchEvent).context();
						// Output
						System.out.println("New path created: " + newPath);
					}
				}
				
				if(!key.reset()) {
					break; //loop
				}
			}
			
		} catch(IOException ioe) {
			ioe.printStackTrace();
		} catch(InterruptedException ie) {
			ie.printStackTrace();
		}
		
	}

	public static void main(String[] args) throws IOException,
			InterruptedException {
		// Folder we are going to watch
		Path folder = Paths.get(System.getProperty("user.home"));
		watchDirectoryPath(folder);
	}
}

package mainpack;

import static java.nio.file.LinkOption.NOFOLLOW_LINKS;

import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;

import static java.nio.file.StandardWatchEventKinds.OVERFLOW;

import java.io.IOException;

import java.nio.file.FileSystem;

import java.nio.file.Files;

import java.nio.file.Path;

import java.nio.file.Paths;

import java.nio.file.WatchEvent;

import java.nio.file.WatchEvent.Kind;

import java.nio.file.WatchKey;

import java.nio.file.WatchService;

public class MainWatch {

public static void watchDirectoryPath(Path path) {

// Sanity check - Check if path is a folder

try {

Boolean isFolder = (Boolean) Files.getAttribute(path,

"basic:isDirectory", NOFOLLOW_LINKS);

if (!isFolder) {

throw new IllegalArgumentException("Path: " + path + " is not a folder");

}

} catch (IOException ioe) {

// Folder does not exists

ioe.printStackTrace();

}

System.out.println("Watching path: " + path);

// We obtain the file system of the Path

FileSystem fs = path.getFileSystem ();

// We create the new WatchService using the new try() block

try(WatchService service = fs.newWatchService()) {

// We register the path to the service

// We watch for creation events

path.register(service, ENTRY_CREATE);

// Start the infinite polling loop

WatchKey key = null;

while(true) {

key = service.take();

// Dequeueing events

Kind<?> kind = null;

for(WatchEvent<?> watchEvent : key.pollEvents()) {

// Get the type of the event

kind = watchEvent.kind();

if (OVERFLOW == kind) {

continue; //loop

} else if (ENTRY_CREATE == kind) {

// A new Path was created

Path newPath = ((WatchEvent<Path>) watchEvent).context();

// Output

System.out.println("New path created: " + newPath);

}

if(!key.reset()) {

break; //loop

}

} catch(IOException ioe) {

ioe.printStackTrace();

} catch(InterruptedException ie) {

ie.printStackTrace();

}

public static void main(String[] args) throws IOException,

InterruptedException {

// Folder we are going to watch

Path folder = Paths.get(System.getProperty("user.home"));

watchDirectoryPath(folder);

}

I will create new files/folders inside my home directory the output will look like:

Watching path: C:\Users\IBM_ADMIN
New path created: New Text Document.txt
New path created: test1.txt
New path created: New Text Document.txt
New path created: test2.txt

Watching path: C:\Users\IBM_ADMIN

New path created: New Text Document.txt

New path created: test1.txt

New path created: New Text Document.txt

New path created: test2.txt

Recursive folder walks using NIO.2 API

December 6, 2013October 4, 2016 Andrei Ciobanu Leave a comment

FileVisitor Interface

The FileVisitor interface allows us to recursively traverse file structures – folders, sub-folders and files.

Every method of this interface can return 4 possible results (instances of the FileVisitResult enum):

FileVisitResult.CONTINUE: This means that the traversal process will continue.
FileVisitResult.SKIP_SIBLINGS: This means that the traversal process will continue without visiting the siblings (files or folders) of that particular Path
FileVisitResult.SKIP_SUBTREE: This means that the traversal process will continue without visiting the rest of the tree entries.
FileVisitResult.TERMINATE: This means that the traversal process should stop.

This FileVisitor interface has 4 methods:

visitFile():
The method is invoked for a file. The method should return a FileVisitResult.CONTINUE result or a FileVisitResult.TERMINATE result. The method receive a reference to the file (a Path object) and to the BasicFileAttributes object associated with the Path.
preVisitDirectory():
This method is invoked for a directory before visiting its children. The method returns FileVisitResult.CONTINUE if we want it’s children to be visited or FileVisitResult.SKIP_SUBTREE if we want the process to stop. If we want to skip visiting the siblings of the directory we need to return FileVisitResult.SKIP_SIBLINGS .
postVisitDirectory():
This method is invoked after we visit all the children of a directory (including other folders and their descendants).
visitFileFailed():
This method is invoked if a file (or folder) cannot be accessed.

In practice it is also possible to use the SimpleFileVisitor class if we want to traverse only the directories.

Once we have created the “recursive-walking-mechanism” by implementing FileVisitor interface or by extending the SimpleFileVisitor class we can start the recursive process by calling the walkFileTree() method .

Example: Writing an application that search for files bigger than a pre-defined size

In this example we are going to implement a FileVisitor that walks a folder and logs to output all files that are bigger than certain amount.

The first step is to write the FileVisitor:

/**
 * This FileVisitor searches for files bigger than 'size' .
 * 
 * If a file matching our criteria is found, we log the results in the stdout .
 * 
 * @author AndreICiobanu
 * 
 */
class FileSizeVisitor implements FileVisitor<Path> {

	private Long size;

	public FileSizeVisitor(Long size) {
		this.size = size;
	}

	/**
	 * This is triggered before visiting a directory.
	 */
	@Override
	public FileVisitResult preVisitDirectory(Path path,
			BasicFileAttributes attrs) throws IOException {
		return FileVisitResult.CONTINUE;
	}

	/**
	 * This is triggered when we visit a file.
	 */
	@Override
	public FileVisitResult visitFile(Path path, BasicFileAttributes attrs)
			throws IOException {

		long fileSize = attrs.size() / 1024;

		if (fileSize >= this.size) {
			System.out.println("File bigger than " + this.size + "KB  found: "
					+ path);
		}

		return FileVisitResult.CONTINUE;
	}

	/**
	 * This is triggered if we cannot visit a Path We log the fact we cannot
	 * visit a specified path .
	 */
	@Override
	public FileVisitResult visitFileFailed(Path path, IOException exc)
			throws IOException {
		// We print the error
		System.err.println("ERROR: Cannot visit path: " + path);
		// We continue the folder walk
		return FileVisitResult.CONTINUE;
	}

	/**
	 * This is triggered after we finish visiting a specified folder.
	 */
	@Override
	public FileVisitResult postVisitDirectory(Path path, IOException exc)
			throws IOException {
		// We continue the folder walk
		return FileVisitResult.CONTINUE;
	}

}

/**

* This FileVisitor searches for files bigger than 'size' .

* If a file matching our criteria is found, we log the results in the stdout .

* @author AndreICiobanu

class FileSizeVisitor implements FileVisitor<Path> {

private Long size;

public FileSizeVisitor(Long size) {

this.size = size;

}

/**

* This is triggered before visiting a directory.

@Override

public FileVisitResult preVisitDirectory(Path path,

BasicFileAttributes attrs) throws IOException {

return FileVisitResult.CONTINUE;

}

/**

* This is triggered when we visit a file.

@Override

public FileVisitResult visitFile(Path path, BasicFileAttributes attrs)

throws IOException {

long fileSize = attrs.size() / 1024;

if (fileSize >= this.size) {

System.out.println("File bigger than " + this.size + "KB found: "

+ path);

}

return FileVisitResult.CONTINUE;

}

/**

* This is triggered if we cannot visit a Path We log the fact we cannot

* visit a specified path .

@Override

public FileVisitResult visitFileFailed(Path path, IOException exc)

throws IOException {

// We print the error

System.err.println("ERROR: Cannot visit path: " + path);

// We continue the folder walk

return FileVisitResult.CONTINUE;

}

/**

* This is triggered after we finish visiting a specified folder.

@Override

public FileVisitResult postVisitDirectory(Path path, IOException exc)

throws IOException {

// We continue the folder walk

return FileVisitResult.CONTINUE;

}

The main method:

Path homeFolder = Paths.get("C:\\");
FileVisitor fileVisitor = new FileSizeVisitor(new Long(5000));
try {
	Files.walkFileTree(homeFolder, fileVisitor);
} catch (IOException e) {
	e.printStackTrace();
}

Path homeFolder = Paths.get("C:\\");

FileVisitor fileVisitor = new FileSizeVisitor(new Long(5000));

try {

Files.walkFileTree(homeFolder, fileVisitor);

} catch (IOException e) {

e.printStackTrace();

}

And some sample output from my machine:

...
File bigger than 5000  found: C:\Windows\System32\IME\imekr8\applets\mshwkorrIME.dll
File bigger than 5000  found: C:\Windows\System32\IME\IMETC10\applets\MSHWCHTRIME.dll
File bigger than 5000  found: C:\Windows\System32\korwbrkr.lex
ERROR: Cannot visit path: C:\Windows\System32\LogFiles\WMI\RtBackup
...

...

File bigger than 5000 found: C:\Windows\System32\IME\imekr8\applets\mshwkorrIME.dll

File bigger than 5000 found: C:\Windows\System32\IME\IMETC10\applets\MSHWCHTRIME.dll

File bigger than 5000 found: C:\Windows\System32\korwbrkr.lex

ERROR: Cannot visit path: C:\Windows\System32\LogFiles\WMI\RtBackup

...

Writing a file search application based on a criteria

We can extend the example from above and create a more general approach.

The idea is to write an abstract implementation of the FileVisitor interface, that contains an abstract method “criteria(Path, BasicFileAttributes)“.

Later we can use anonymous classes to define a new behavior of our visitors specifying only the criteria and avoiding to write the boiler-plate-code necessary to implement a FileVisitor.

We will name our FileVisitor implementation FileSearchByCriteriaVisitor:

abstract class FileSearchByCriteriaVisitor implements FileVisitor<Path>

1	abstract class FileSearchByCriteriaVisitor implements FileVisitor<Path>

This class will have two instance variables called results and failedVisits:

private List<Path> results = new LinkedList<Path>();
private List<Path> failedVisits = new LinkedList<Path>();
public List<Path> getResults() {
	return this.results;
}
public List<Path> getFailedVisits() {
	return this.failedVisits;
}

private List<Path> results = new LinkedList<Path>();

private List<Path> failedVisits = new LinkedList<Path>();

public List<Path> getResults() {

return this.results;

}

public List<Path> getFailedVisits() {

return this.failedVisits;

}

The “criteria(Path, BasicFileAttributes)” mentioned before will be used like this:

// This method will be later implemented
protected abstract Boolean criteria(Path path, BasicFileAttributes attrs);
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
		throws IOException {
	// Everytime we visit a file we check if that particular file matches a
	// criteria .
	if (criteria(file, attrs)) {
		// If the file matches the criteria we add it as a result
		results.add(file);
	}
	return FileVisitResult.CONTINUE;
}

// This method will be later implemented

protected abstract Boolean criteria(Path path, BasicFileAttributes attrs);

@Override

public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)

throws IOException {

// Everytime we visit a file we check if that particular file matches a

// criteria .

if (criteria(file, attrs)) {

// If the file matches the criteria we add it as a result

results.add(file);

}

return FileVisitResult.CONTINUE;

}

Now everytime we implement a new FileSearchByCriteriaVisitor we must supply an implementation for the abstract method defined before.

Example how to use the FileSearchByCriteriaVisitor:

// Search all the files bigger than size
final long size = 5000; // KB
// Defining a new visitor criteria
FileSearchByCriteriaVisitor sizeVisitor = new FileSearchByCriteriaVisitor() {
	@Override
	protected Boolean criteria(Path path, BasicFileAttributes attrs) {
		if (attrs.size() / 1024 >= size) {
			return true;
		}
		return false;
	}
};
// Walk don't run
Files.walkFileTree(Paths.get("C:\\"), sizeVisitor);
// Evaluate results
evaluate(sizeVisitor.getResults());

// Search all the files bigger than size

final long size = 5000; // KB

// Defining a new visitor criteria

FileSearchByCriteriaVisitor sizeVisitor = new FileSearchByCriteriaVisitor() {

@Override

protected Boolean criteria(Path path, BasicFileAttributes attrs) {

if (attrs.size() / 1024 >= size) {

return true;

}

return false;

}

};

// Walk don't run

Files.walkFileTree(Paths.get("C:\\"), sizeVisitor);

// Evaluate results

evaluate(sizeVisitor.getResults());

The results and the paths with errors are available in “sizeVisitor.getResults()” and “sizeVisitor.getFailedVisits()”.