Thursday, July 27, 2017

Logging in Java - Log4j2 over SLF4J

When it comes to logging in Java applications, it is common practice to go with one of many available logging frameworks. As frameworks evolve and address recurring logging needs of applications, it could be cumbersome to switch from an existing framework to a new one. For this reason, it is recommended to program to a common logging interface and change the underlying implementation as needed. SLF4J is a commonly used logging interface, and Log4j is a good example of the work involved in upgrading from one version to another.

In this blog, I will highlight a Log4j2 over SLF4J approach to application logging.

Logging Levels

Using the following log levels are recommended as best practice. Logging approach will adhere to SLF4J Interface and only use log levels available in SLF4J (see http://www.slf4j.org/api/org/apache/commons/logging/Log.html).

Trace
This level should be used when capturing very detailed information about the system. Logging every step of the execution and capturing state of the code. Trace is useful to detect obscure behavior. By default it should be OFF, as IO operations are expensive.

Debug
This level should be used to report detailed information about the behavior and state of the code. It can be used to diagnose specifics or record user/application behavior. By default it should be OFF, as IO operations are expensive.

Info
Use this level to capture useful information about the system, record diagnostics info,  and report normal behavior. It should be used to indicate that the system is correctly initialized and started, and that connections are made successfully. This level can be set to ON.

Warning
Use this level to report issues with the system, such as running low in resources [e.g.,db connections], accessing information that should not be accessed again, retry limits reached, normal timeouts, etc... This level should be used to log messages that need to be flagged and attended to by responsible parties. This level must be ON.

Error
Use this level to record exception and errors with the system. Examples are network problems, resource availability. Messages logged at this level should be handled and responsible parties should be alerted. This level must be ON.

Fatal
Most serious messages should be logged at this level. Information such as why system was stopped should be logged and report immediately. This level must be ON.

Working with Log4j2
Assuming maven is used to build the application, to start programming with Log4j2, include the required dependencies in your pom.xm

<properties>
    ...
    <org.slf4j.version>1.7.21</org.slf4j-version>
    <log4j2.version>2.6.2</log4j2.version>
    ...
</properties>
 
<dependencies>
    ...
    <!-- SLF4J Logging -->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>${org.slf4j.version}</version>
    </dependency>
 
    <!-- Log4j2 -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>${log4j2.version}</version>
    </dependency>
    <!-- Log4j2 Binding to SLF4J -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-slf4j-impl</artifactId>
        <version>${log4j2.version}</version>
    </dependency>
    <!-- you need the following jar files if planning to use Log4j2 Client GUI with jconsole  -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>${log4j2.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-jmx-gui</artifactId>
        <version>${log4j2.version}</version>
    </dependency>
    ...
</dependencies>

In your java code, import the following packages

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
And add the following to Java classes requiring logging

private static final Logger logger = LoggerFactory.getLogger(YourClassName.class);

Configuration
Typically log4j2 configuration file (log4j2.xml) can be placed on the classpath. Several points related to the location of the config file:
  • To make it available on classpath,  place it in src/main/resources
  • To load it as an external file, use -Dlog4j.configurationFile="/path/to/config/log4j2.xml"
  • To load it only for testing, place log4j2-test.xml in src/test/resources
  • To specify the location in a web application, use a servlet context parameter. See Set the Configuration Path.
Here is an example of log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <File name="MyAppLog" fileName="/opt/myapp/logs/myapp.log">
            <PatternLayout>
                <Pattern>%d [%p] %c{1.} : %m%n</Pattern>
            </PatternLayout>
        </File>
        <Console name="STDOUT" target="SYSTEM_OUT">
            <PatternLayout>
                <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
            </PatternLayout>
        </Console>
    </Appenders>

    <Loggers>
        <Root level="WARN">
            <AppenderRef ref="MyAppLog" level="WARN" />
            <AppenderRef ref="STDOUT" level="WARN" />
        </Root>
        <Logger name="org.springframework" level="WARN">
            <AppenderRef ref="STDOUT" level="WARN" />
        </Logger>
        <Logger name="org.apache.cxf" level="WARN">
            <AppenderRef ref="STDOUT" />
        </Logger>
    </Loggers>
</Configuration>

Notes
  • In general, writing messages to a log file is expensive, so use logging levels wisely.
  • In version 1 of Log4j, it was recommended to guard log operations with is[LogLeve]Enabled checks. Here is a great blog on how to overcome inefficiency of logging when log levels not requiring logging.
  • You can add 3rd party tools logging to your configuration
    • For CXF, add the following to src/main/resources/META-INF/org.apache.cxf.Logger
    • org.apache.cxf.common.logging.Slf4jLogger
  • You can configure log files periodically rotate based on certain criteria. Here is an example of rotating log files:
  • <?xml version="1.0" encoding="UTF-8"?>
    <Configuration>
            <Properties>
                    <property name="name">myapp-rolling</property>
                    <property name="pattern">%d{yyyy-MM-dd HH:mm:ss.SSS} | %-5.5p | %-10.10t | %-20.20C:%-5.5L | %msg%n</property>
    <!--
    It will look like that:
    2013-04-03 07:37:51.993 | WARN  | main       | myserver.Server:56    | My app is logging stuff
    -->
            </Properties>
            <Appenders>
                    <RollingFile name="RollingCdspLog" fileName="/path/to/my/logs/${name}.log"
                                 filePattern="/path/to/my/logs/$${date:yyyy-MM}/${name}-%d{yyyy-MM-dd}-%i.log.gz">
                            <PatternLayout>
                                    <pattern>${pattern}</pattern>
                            </PatternLayout>
                            <Policies>
                                    <TimeBasedTriggeringPolicy /><!-- Rotated everyday -->
                                    <SizeBasedTriggeringPolicy size="1 MB"/> <!-- Or every 100 MB -->
                            </Policies>
                    </RollingFile>
                    <File name="CDSPLog" fileName="/path/to/my/logs/override-myapp.log">
                            <PatternLayout>
                                    <Pattern>%d [%p] %c{1.} : %m%n</Pattern>
                            </PatternLayout>
                    </File>
                    <Console name="STDOUT" target="SYSTEM_OUT">
                            <PatternLayout>
                                    <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
                            </PatternLayout>
                    </Console>
            </Appenders>
    
            <Loggers>
                    <Root level="DEBUG">
                            <AppenderRef ref="CDSPLog" level="INFO" />
                            <AppenderRef ref="RollingCdspLog" level="DEBUG" />
                            <AppenderRef ref="STDOUT" level="WARN" />
                    </Root>
                    <Logger name="org.springframework" level="WARN">
                            <AppenderRef ref="STDOUT" level="WARN" />
                    </Logger>
                    <Logger name="org.apache.cxf" level="WARN">
                            <AppenderRef ref="STDOUT" />
                    </Logger>
            </Loggers>
    </Configuration>
Log4j2 JMX

Log4j 2 has built-in support for JMX. The StatusLogger, ContextSelector, and all LoggerContexts, LoggerConfigs and Appenders are instrumented with MBeans and can be remotely monitored and controlled.

To configure Log4j2 at runtime, a client GUI plugin can be used with jconsole. The following steps will demonstrate how to attach jconsole to a running Junit process. This Junit process will start logging messages.
  1. Download  log4j-core-2.6.2.jar, log4j-api-2.6.2.jar, log4j-jmx-gui-2.6.2.jar (see pom.xml)
  2. Run jconsole -pluginpath ./log4j-core-2.6.2.jar:./log4j-api-2.6.2.jar:./log4j-jmx-gui-2.6.2.jar
  3. Select a local process that is running your app. For example, if you are junit testing in eclipse, select: org.eclipse.jdt.internal.junit.runner.RemoteTestRunner from Local Process list, and press connect. Select Log4j2 tab.
  4. You can change the configuration and select the appropriate Reconfigure button

Saturday, August 27, 2016

On Kafka

This blog captures steps to install and setup Kafka.

For instructions on how to install Kafka, visit: Installation Steps

After downloading:
untar bundle to /opt/apache-kafka
ln -s kafka_2.11-0.9.0.1 current
cd /opt/apache-kafka/current

At this point you can start Zookeper:

bin/zookeeper-server-start.sh config/zookeeper.properties&

To run Kafka on the current server, modify config/server.properties:

Uncomment:

port=9092
host.name=localhost

To allow for removing topics, add the following to config/server.properties:

# Enable topic deletion - only for development
delete.topic.enable=true

Now you can start Kafka:

bin/kafka-server-start.sh config/server.properties&

Kafka REST Proxy
In order to publish messages to Kafka via REST calls, you can use Kafka REST Proxy. To install it, visit Confluent Platform Quickstart.

Untar the downloaded bundle to /opt/confluent:

ln -s confluent-2.0.1 current
cd current
vi etc/kafka-rest/kafka-rest.properties

Uncomment

id=kafka-rest-test-server
schema.registry.url=http://localhost:8081
zookeeper.connect=localhost:2181

To run Kafka REST Proxy:
bin/schema-registry-start etc/schema-registry/schema-registry.properties &
bin/kafka-rest-start etc/kafka-rest/kafka-rest.properties &

Some useful aliases to start/stop/view Kafka processes
alias zs="cd /opt/apache-kafka/current; nohup bin/zookeeper-server-start.sh config/zookeeper.properties 2>/dev/null &"
alias ks="cd /opt/apache-kafka/current; nohup bin/kafka-server-start.sh config/server.properties 2>/dev/null&"

alias krp1="cd /opt/confluent/current; nohup bin/schema-registry-start etc/schema-registry/schema-registry.properties 2>/dev/null&"
alias krp2="cd /opt/confluent/current; nohup bin/kafka-rest-start etc/kafka-rest/kafka-rest.properties 2>/dev/null&"

alias pk="ps -aef|grep kafka"
alias topics="/opt/apache-kafka/current/bin/kafka-topics.sh  --list --zookeeper localhost:2181"

To create topics:
cd/opt/apache-kafka/current; bin/kafka-topics.sh --create --topic test --zookeeper YOUR.IP.ADDRESS.HERE:2181 --partition 1 --replication-factor 1

To list topics:
cd /opt/apache-kafka/current; bin/kafka-topics.sh --list --zookeeper YOUR.IP.ADDRESS.HERE:2181

To consume messages from a topic:
/opt/apache-kafka/current/bin/kafka-console-consumer.sh --zookeeper YOUR.IP.ADDRESS.HERE:2181 --topic MY_TOPIC 


Tuesday, May 31, 2016

JAXB & XML Binding Using XJC

Recently I was given an XML file with multiple schemas defining the structure of the XML file. Given my comfort level with Java technologies, I decided to use JAXB to marshall/unmarshall XML files. This blog describes the steps I took to work with XML files.

How to auto-generate Java classes that represent XML schemas using XJC

Schemas were placed in multiple sub-directories:

~/myproject/src/main/schema/schema1/*.xsd
~/myproject/src/main/schema/schema2/*.xsd

Running xjc on files in schema2 directory worked fine and Java classes were generated.
However, files in schema1 failed to auto-generate with errors like:

[ERROR] Property "Lang" is already defined. Use <jaxb:property> to resolve this conflict.
line 302 of http://www.w3.org/2002/08/xhtml/xhtml1-strict.xsd

[ERROR] The following location is relevant to the above error
line 303 of http://www.w3.org/2002/08/xhtml/xhtml1-strict.xsd

One of the schemas had a dependency on http://www.w3.org/2002/08/xhtml/xhtml1-strict.xsd. I noticed attribute "lang" was used several times in http://www.w3.org/2002/08/xhtml/xhtml1-strict.xsd. So xjc was not able to generate unique Java classes to represent these repeating attributes. The good news is you can instruct xjc to rename any portion of the schema is having problems with, using a binding customization file.

To fix the above errors, I created a file called: xhtml1-strict.xjb

<bindings xmlns="http://java.sun.com/xml/ns/jaxb"
          xmlns:xsi="http://www.w3.org/2000/10/XMLSchema-instance"
          xmlns:xs="http://www.w3.org/2001/XMLSchema"
          version="2.1">
    <bindings schemaLocation="http://www.w3.org/2002/08/xhtml/xhtml1-strict.xsd" version="1.0">
        <!-- rename the value element -->
        <bindings node="//xs:attributeGroup[@name='i18n']">
            <bindings node=".//xs:attribute[@name='lang']">
                <property name="I18nLangAttribute"/>
            </bindings>
        </bindings>
        <bindings node="//xs:element[@name='bdo']//xs:complexType">
             <bindings node=".//xs:attribute[@name='lang']">
                 <property name="BdoLangAttribute"/>
             </bindings>
        </bindings>
    </bindings>
</bindings>


Run xjc again to auto-generate Java Binding classes:

cd ~noushin/workspace/ajs/src/main/schema
xjc -d ~noushin/workspace/ajs/src/main/java -b xhtml1-strict.xjb */*.xsd

After running the above commands, you should see Java packages and classes in src directory.

The following snippet shows you how to read a XML file with parent tag and parse it to get content of :

public MyDocument getMyDocument() {
    JAXBContext context;
    try {
        context = JAXBContext.newInstance(MyDocument.class);
        Unmarshaller um = context.createUnmarshaller();
        FileReader fileReader = new FileReader("/path/to/xmlfile/);
        MyDocument myDocument = (MyDocument) um.unmarshal(this.getClass().getClassLoader()
     .getResourceAsStream("mydocument.xml"));
        Metadata metadata = null;
        if (myDocument != null) {
               metadata = myDocument.getMetadata();
        }
    }
    catch (FileNotFoundException | JAXBException e) {
        // handle exception
        e.printStackTrace();
    }
}

You will get an exception during unmarshalling if your xml document parent tag is "":
javax.xml.bind.UnmarshalException: unexpected element (uri:"urn:hl7-org:docns", local:"myDocument"). Expected elements are ...

In this case, you need to add the following to your JAXB Class MyDocument:

@XmlRootElement(name = "myDocument")









Thursday, March 5, 2015

Writing RESTful web services using Apache CXF & Spring

In an earlier blog, I showed you how to develop APIs using REST and Spring RS. Since Spring does not fully implement JAX-RS, in this blog, I will show you how to use Apache CXF with Spring to write RESTful web services. pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.noushin</groupId>
    <artifactId>cxfws</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>
    <name>cxfws</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <cxf.version>3.0.2</cxf.version>
        <junit.version>4.10</junit.version>
        <log4j.version>1.2.17</log4j.version>
        <spring.version>4.1.1.RELEASE</spring.version>
        <spring.security.version>3.2.0.RELEASE</spring.security.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Logging -->
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>${log4j.version}</version>
        </dependency>

        <!-- Spring Test -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring.version}</version>
            <scope>test</scope>
        </dependency>

        <!-- Spring --> 
         <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>${spring.version}</version>
        </dependency>
 
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <!-- JSON Binding -->
        <dependency>
            <groupId>org.codehaus.jackson</groupId>
            <artifactId>jackson-jaxrs</artifactId>
            <version>1.5.7</version>
        </dependency>

        <!-- CXF -->
        <dependency>
            <groupId>org.apache.cxf</groupId>
            <artifactId>cxf-rt-frontend-jaxrs</artifactId>
            <version>${cxf.version}</version>
        </dependency>

    </dependencies>
    
    
    <build>
        <finalName>cxfws</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>
The first class you need to write is an interface to expose your web services:
package com.noushin.cxfws.user.service;

import javax.ws.rs.Consumes;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import com.noushin.cxfws.user.model.Status;
import com.noushin.cxfws.user.model.User;

@Path("/user")
public interface UserService {

    /*
     * curl  -i -v -X GET -H "Accept: application/json" http://localhost:8080/cxfws/services/user/123
     * return: {"id":"123","firstName":"John","lastName":"Smith"}
     */
    @GET
    @Consumes(MediaType.APPLICATION_FORM_URLENCODED)
    @Produces({ MediaType.APPLICATION_JSON })
    @Path("/{id}")
    public User getUser(@PathParam("id") String id);
    
    /*
     * curl  -i -v -X POST -H "Content-type: application/json" -H "Accept: application/json" http://localhost:8080/cxfws/services/user/ -d @param.json
     * return: {"id":"456","firstName":"Jane","lastName":"Doe"}
     */
    
    @POST
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces({ MediaType.APPLICATION_JSON })
    @Path("/")
    public User saveUser(User user);
    
    /*
     * curl  -i -v -X PUT -H "Accept: application/json" http://localhost:8080/cxfws/services/user/123/abcd
     * return: {"id":"123","firstName":"abcd","lastName":"old lastname"}
     */
    @PUT
    @Consumes(MediaType.APPLICATION_FORM_URLENCODED)
    @Produces({ MediaType.APPLICATION_JSON })
    @Path("/{id}/{fn}")
    public User updateUser(@PathParam("id") String id, @PathParam("fn") String firstName);
    
    
    /*
     * curl  -i -v -X DELETE -H "Accept: application/json" http://localhost:8080/cxfws/services/user/123
     * return: {"message":"User removed successfully.","code":0}
     */
    @DELETE
    @Consumes(MediaType.APPLICATION_FORM_URLENCODED)
    @Produces({ MediaType.APPLICATION_JSON })
    @Path("/{id}")
    public Status removeUser(@PathParam("id") String id);

}
Now, you have to write the implementation class:
package com.noushin.cxfws.user.service;

import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import com.noushin.cxfws.user.dao.UserDao;
import com.noushin.cxfws.user.model.Status;
import com.noushin.cxfws.user.model.User;
import com.noushin.cxfws.user.reference.StatusCodes;

@Service("userService")
public class UserServiceImpl implements UserService {

    private final static Logger logger = Logger.getLogger(UserServiceImpl.class);
    
    @Autowired
    UserDao userDao;
    
    public User getUser(String id) {
        String msg = "getting user with id: " + id + ".";
        logger.info(msg);
        return userDao.getUser(id);
    }

    public User saveUser(User user) {
        String msg = "saving user with id: " + user.getId() + ".";
        logger.info(msg);
        return userDao.saveUser(user);
    }

    public User updateUser(String id, String firstName) {
        String msg = "updating user with id: " + id + ".";
        logger.info(msg);
        User user = new User();
        user.setId(id);
        user.setFirstName(firstName);
        return userDao.saveUser(user);
    }

    public Status removeUser(String id) {
        String msg = "removing user with id: " + id + ".";
        logger.info(msg);
        Status status = null;
        if (userDao.removeUser(id))
            return new Status(StatusCodes.FAILED, "User removed successfully.");
        return  new Status(StatusCodes.FAILED, "User could not be removed.");
    }

}

The JSON objects that will be passed to or retuned from user web services, will be presented as simple POJOS. User JSON:
package com.noushin.cxfws.user.model;

public class User {
    
    private String id;
    private String firstName;
    private String lastName;
    
    public User() {
    }
    
    public User(String id, String firstName, String lastName) {
        super();
        this.id = id;
        this.firstName = firstName;
        this.lastName = lastName;
    }

    public String getId() {
        return id;
    }
    
    public void setId(String id) {
        this.id = id;
    }
    
    public String getFirstName() {
        return firstName;
    }
    
    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }
    
    public String getLastName() {
        return lastName;
    }
    
    public void setLastName(String lastName) {
        this.lastName = lastName;
    }
    
    public String toString() {
        return "User {id:" + id + ", firstName:" + firstName + ", lastName:" + lastName + "}";
    }
    
}

Status JSON:
package com.noushin.cxfws.user.model;

public class Status {
    private int code;
    private String message;
    
    
    public Status() {
    }

    public Status(int code, String message) {
        super();
        this.code = code;
        this.message = message;
    }
    
    public int getCode() {
        return code;
    }
    
    public void setCode(int code) {
        this.code = code;
    }
    
    public String getMessage() {
        return message;
    }
    
    public void setMessage(String message) {
        this.message = message;
    }
    
    
}


UserDao.java:
package com.noushin.cxfws.user.dao;

import org.apache.log4j.Logger;
import org.springframework.stereotype.Component;

import com.noushin.cxfws.user.model.User;

@Component
public class UserDao {
    
    private final static Logger logger = Logger.getLogger(UserDao.class);

    public User getUser(String id) {
        User user = new User(id, "John", "Smith");
        String msg = "getting user with id: " + id + ".";
        logger.info(msg);
        return user;
    }
    
    public User saveUser(User user) {
        String msg = "saving user with id: " + user.getId() + ".";
        logger.info(msg);
        return user;
    }
    
    public boolean removeUser(String id) {
        String msg = "removing user with id: " + id + ".";
        logger.info(msg);
        return true;
    }
    
}


Reference data:
package com.noushin.cxf.user.reference;

public class StatusCodes {
    public final static int SUCCESS = 1;
    public final static int FAILED = 0;
    
}

~cxfws/src/main/webapp/WEB-INF/web.xml
<?xml version="1.0" encoding="UTF-8"?>
<web-app version="3.1" xmlns="http://xmlns.jcp.org/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd">

    <display-name>Spring CXF WS Demo</display-name>
    <description>Spring CXF WS Demo</description>

    <servlet>
        <servlet-name>CXFServlet</servlet-name>
        <servlet-class>org.apache.cxf.transport.servlet.CXFServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>CXFServlet</servlet-name>
        <url-pattern>/services/*</url-pattern>
    </servlet-mapping>

    <context-param>
        <param-name>log4jConfigLocation</param-name>
        <param-value>/WEB-INF/classes/log4j.properties</param-value>
    </context-param>

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/cxfws-servlet.xml</param-value>
    </context-param>

    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>

</web-app>
Spring application context is in ~cxfws/src/main/webapp/WEB-INF/cxfws-servlet.xml
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jaxrs="http://cxf.apache.org/jaxrs"
    xmlns:cxf="http://cxf.apache.org/core"
    xmlns:http-conf="http://cxf.apache.org/transports/http/configuration"
    xsi:schemaLocation="
      http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
      http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
      http://cxf.apache.org/jaxrs http://cxf.apache.org/schemas/jaxrs.xsd 
      http://cxf.apache.org/transports/http/configuration http://cxf.apache.org/schemas/configuration/http-conf.xsd">

    <context:component-scan base-package="com.noushin.cxfws.user" />
    <context:annotation-config />

    <jaxrs:server id="userServer" address="/">
        <jaxrs:serviceBeans>
            <ref bean="userService" />
        </jaxrs:serviceBeans>
        <jaxrs:providers>
            <bean class="org.codehaus.jackson.jaxrs.JacksonJsonProvider" />
        </jaxrs:providers>
        <jaxrs:extensionMappings>
            <entry key="json" value="application/json" />
        </jaxrs:extensionMappings>
        <jaxrs:features>
            <cxf:logging/>
        </jaxrs:features>
    </jaxrs:server>

    <bean id="userServiceImpl" class="com.noushin.cxfws.user.service.UserServiceImpl" />

</beans>

Friday, February 27, 2015

Using SSH to transfer files in Java

In this example, I'll be using JSch library to copy a file from my local server to a remote host. If using Maven, add the following dependency to your pom.xml file:
        <!-- for ssh -->
        <dependency>
            <groupId>com.jcraft</groupId>
            <artifactId>jsch</artifactId>
            <version>${jsch.version}</version>
        </dependency>
Define an interface for your remote file operations:
public interface RemoteFileUtilities {
    public void copyFile(String filename, HostInfo hostInfo);
    public List<String> lsDir(String dir, HostInfo hostInfo);
}
SSH implementation class:
public class SshUtilities implements RemoteFileUtilities {

    @Override
    public void copyFile(String filename, HostInfo hostInfo) {
        JSch jsch = new JSch();
        Session session = null;
        ChannelSftp channel = null;
        try {
            session = jsch.getSession(hostInfo.getUsername(),
                    hostInfo.getHostname(), hostInfo.getPort());
            session.setPassword(hostInfo.getPassword());
            session.setConfig("StrictHostKeyChecking", "no");
            session.connect();
            channel = (ChannelSftp) session.openChannel("sftp");
            channel.connect();
            File localFile = new File(filename);
            // If you want you can change the directory using the following line.
            channel.cd(hostInfo.getPath());
            channel.put(new FileInputStream(localFile), localFile.getName());   
        } 
        catch (JSchException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } 
        catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } 
        catch (SftpException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        channel.disconnect();
        session.disconnect();
    }

    @Override
    public List<String> lsDir(String dir, HostInfo hostInfo) {
        JSch jsch = new JSch();
        Session session = null;
        ChannelSftp channel = null;
        try {
            session = jsch.getSession(hostInfo.getUsername(), hostInfo.getHostname(), hostInfo.getPort());
            session.setPassword(hostInfo.getPassword());
            session.setConfig("StrictHostKeyChecking", "no");
            session.connect();
            channel = (ChannelSftp) session.openChannel("sftp");
            channel.connect();
            return channel.ls(dir);  
        } 
        catch (JSchException e) {
            e.printStackTrace();
        } 
        catch (SftpException e) {
             e.printStackTrace();
        }
        channel.disconnect();
        session.disconnect();
        return null;
    }     
}

A supporting class to hold information related to your host:
public class HostInfo {
    
    private String hostname;
    private String username;
    private String password;
    private String path;
    private int port;
    
    
    public HostInfo(String hostname, String username, String password, String path, int port) {
        super();
        this.hostname = hostname;
        this.username = username;
        this.password = password;
        this.path = path;
        this.port = port;
    }
    
    public String getHostname() {
        return hostname;
    }
    public void setHostname(String hostname) {
        this.hostname = hostname;
    }
    public String getUsername() {
        return username;
    }
    public void setUsername(String username) {
        this.username = username;
    }
    public String getPassword() {
        return password;
    }
    public void setPassword(String password) {
        this.password = password;
    }
    public String getPath() {
        return path;
    }
    public void setPath(String path) {
        this.path = path;
    }
    public int getPort() {
        return port;
    }
    public void setPort(int port) {
        this.port = port;
    } 
}
And finally, a JUnit test class:
public class SshUtilitiesTest {

    @Test
    public void test() {
        HostInfo hostInfo = new HostInfo("dev-core-ccm-1", "noushin", "nbcg14g~", "/home/noushin/ssh-test", 22);
        SshUtilities sshUtils = new SshUtilities();
        sshUtils.copyFile("/tmp/log/error.log", hostInfo);
        List<String> files = sshUtils.lsDir(hostInfo.getPath(), hostInfo);
        if (files != null) {
            for (int i = 0; i < files.size(); i++)
                assert(files.get(i).contains("error.log"));
        }
    }
}

Sending and receiving files via RESTful web services in Java

In this blog, I'll show you how to pass the content of a file to a RESTful web service and have it stored on the server it was received. First write your CXF service and annotate it as follows:
@POST
@Path("saveFile")
@Produces({MediaType.APPLICATION_JSON})
@Consumes({MediaType.MULTIPART_FORM_DATA})
public UploadFile saveFile(MultipartBody body) {
     List<Attachment> attachments = body.getAllAttachments();
     Attachment att = attachments.get(0);
     File destinationFile = new File("/tmp/log/fileX.txt");

     try {
          att.transferTo(destinationFile);
     } 
     catch (IOException ioe) {
          ioe.printStackTrace();
     }
     UploadFile savedFile = new UploadFile();
     savedFile.setFilename("/tmp/log/fileX.txt");
     return savedFile;
}
To call this service, I am using Apache httpcomponents package. Add the following dependencies to your pom.xml:
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.0-beta2</version>
        </dependency>

        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpmime</artifactId>
            <version>4.0-beta2</version>
        </dependency>
And here is the test snippet to call the web service:
    @Test
    public void restSaveFile()  {
        String url = "http://localhost:8080/mywebapp/services/";
        ResponseEntity<AuthenticationToken> authToken = restClient.authenticate(url + "authenticate");
        String token = null;
        if (authToken != null)
            token = authToken.getBody().getToken();
        try {
            HttpClient httpclient = new DefaultHttpClient();
            HttpPost httpPost = new HttpPost(url + "saveFile");
            // Assuming ~mywebapp/src/test/resource/data/test-file.txt
            FileBody body = new FileBody(new File(Thread.currentThread().getContextClassLoader().getResource("data/test-file.txt").getFile()));
            StringBody comment = new StringBody("Test data for saving a file.");
            httpPost.addHeader("X-Auth-Token", token);
            httpPost.addHeader("Content-Type", MediaType.MULTIPART_FORM_DATA_VALUE);       
            MultipartEntity requestEntity = new MultipartEntity();
            requestEntity.addPart("file", body);
            requestEntity.addPart("comment", comment);
            httpPost.setEntity(requestEntity);        
            System.out.println("Executing request " + httpPost.getRequestLine());
            HttpResponse response = httpclient.execute(httpPost);
            HttpEntity responseEntity = response.getEntity();
            System.out.println("----------------------------------------------");
            System.out.println(response.getStatusLine());
            if (responseEntity != null) {
                System.out.println("Response content length: " + responseEntity.getContentLength());
                System.out.println("Response content type: " + responseEntity.getContentType().getValue());
            }
            if (responseEntity != null) {           
                responseEntity.consumeContent();
            }
            System.out.println("----------------------------------------------");
        } 
        catch (Exception ex) {
            ex.printStackTrace();
        }        
    }

JAXB & Xml Binding Using Annotations

Create an entity bean and annotate it for JAXB Xml binding:
package com.noushin.soaws.model;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlType;

@XmlRootElement(name="book")
@XmlType(propOrder={"name", "author", "isbn", "publisher"})
public class Book {

    private String name;
    private String author;
    private String isbn;
    private String publisher;
    
    public Book() {}
    
    public Book(String author) {
        this.author = author;        
    }
    
    public String getAuthor() {
        return author;
    }
    
    public void setAuthor(String author) {
        this.author = author;
    }

    @XmlElement(name="title")
    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getIsbn() {
        return isbn;
    }

    public void setIsbn(String isbn) {
        this.isbn = isbn;
    }

    public String getPublisher() {
        return publisher;
    }

    public void setPublisher(String publisher) {
        this.publisher = publisher;
    }

}

Create a class to manage the entity bean, read to and write from XML files:
package com.noushin.soaws.service;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;

import javax.jws.WebService;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;
import javax.xml.bind.Unmarshaller;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlElementWrapper;
import javax.xml.bind.annotation.XmlRootElement;

import com.noushin.soaws.model.Book;

@XmlRootElement(namespace="com.noushin.soaws")
public class BookService {

    @XmlElementWrapper(name="books")
    @XmlElement(name="book")
    ArrayList<Book> books;

    public void setBooks() {
        books = new ArrayList<Book>();

        // create books
        Book book1 = new Book();
        book1.setIsbn("978-0060554736");
        book1.setName("The Game");
        book1.setAuthor("Neil Strauss");
        book1.setPublisher("Harpercollins");
        books.add(book1);

        Book book2 = new Book();
        book2.setIsbn("978-3832180577");
        book2.setName("Feuchtgebiete");
        book2.setAuthor("Charlotte Roche");
        book2.setPublisher("Dumont Buchverlag");
        books.add(book2);

    }

    public ArrayList<Book> getBooks() {
        return books;
    }

    public Book getBook() {
        return new Book("Mark Twain");
    }

    public static void manage() {
        BookService bookService = new BookService();
        bookService.setBooks();
        
        // create JAXB context and instantiate marshaller
        JAXBContext context;
        try {
            context = JAXBContext.newInstance(BookService.class);

            Marshaller m = context.createMarshaller();
            m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);

            // Write to System.out
            m.marshal(bookService, System.out);

            // Write to File
            m.marshal(bookService, new File("/tmp/books.xml"));

            // get variables from our xml file, created before
            System.out.println();
            System.out.println("Output from our XML File: ");
            Unmarshaller um = context.createUnmarshaller();
            BookService bookService2 = (BookService) um.unmarshal(new FileReader("books.xml"));
            ArrayList<Book> list = bookService2.getBooks();
            for (Book book : list) {
                System.out.println("Book: " + book.getName() + " from " + book.getAuthor());
            }
        } 
        catch (JAXBException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } 
        catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
}


Using Junit, write a test case to test the functionality:
package com.noushin.soaws.service;

import static org.junit.Assert.*;

import org.junit.Test;

public class BookServiceTest {

    @Test
    public void test() {
        BookService bookService = new BookService();
        bookService.manage();
    }

}