SyntaxHighlighter JS

2015-09-29

How to build an offline Karaf server

What you will do:
  • Create an offline version of Apache Karaf
What you will learn:
  • Configure karaf-maven-plugin and maven-dependency-plugin
  • Override Karaf default behavior with Maven 
The default behavior of Apache Karaf is to download Java library dependencies as needed from Maven repositories such as http://repo1.maven.org/ . To perform this functionality, Karaf will use the Maven ~/.m2/settings.xml file and store dependencies in the ~/.m2/repository directory.

The confirm this behavior empirically
mv ~/.m2 ~/.m2_save

And disable internet access.
If you start a new version of Karaf and try to install any feature such as
feature:install jndi

you will get the error message
Error executing command: Can't install feature jndi/0.0.0: 
Error resolving artifact org.apache.xbean:xbean-naming:jar:3.18: Could not transfer artifact org.apache.xbean:xbean-naming:jar:3.18 from/to central (http://repo1.maven.org/maven2/): repo1.maven.org: nodename nor servname provided, or not known

If you turn internet access back on and rerun feature:install jndi, Karaf will create the directory ~/.m2/repository and put the library dependencies for Karaf jndi in that directory.

This may be undesirable behavior, especially in secured production, when servers typically cannot download artifacts freely from the internet.  From a developer perspective, it can be surprising that changing Maven settings.xml for another unrelated project can affect the runtime behavior of Karaf.

Follow the instructions below to create an offline version of Apache Karaf where the dependencies are preloaded at compile time and is not dependent on Maven.

This tutorial is based on Karaf 3.0.4 . The procedure for Karaf 2 and 4 are similar
Karaf 2: Coming Soon
Karaf 3: https://github.com/juttayaya/karaf/tree/master/karaf3/offline-karaf
Karaf 4: https://github.com/juttayaya/karaf/tree/master/karaf4/offline-karaf4

Step 1: Configure karaf-maven-plugin to generate an offline version of Karaf
Configure the setting ignoreDependencyFlag of karaf-maven-plugin to true. This configures the karaf-maven-plugin to download library dependencies into the directory ${karaf.home}/system
<plugin>
    <groupId>org.apache.karaf.tooling</groupId>
    <artifactId>karaf-maven-plugin</artifactId>
    <!-- Plugin requires at minimum 3.0.3 version for dependency=true bug fix
    https://issues.apache.org/jira/browse/KARAF-2596 -->
    <version>${karaf.plugin.version}</version>
    <extensions>true</extensions>
    <configuration>
      <karafVersion>${karaf.version}</karafVersion>
      <!-- ignoreDependencyFlag is true forces plugin to also
           download feature dependent libraries -->
      <ignoreDependencyFlag>true</ignoreDependencyFlag>
    </configuration>
</plugin>

Step 2: Add the Karaf features.xml you want to install as dependencies.
For example, to add camel to Karaf, add this dependency
<!-- https://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/2.15.2/apache-camel-2.15.2-features.xml -->
<dependency>
    <groupId>org.apache.camel.karaf</groupId>
    <artifactId>apache-camel</artifactId>
    <version>${camel.version}</version>
    <classifier>features</classifier>
    <type>xml</type>
    <scope>runtime</scope>
</dependency>

Step 3:  Add the feature as bootFeatures or installedFeatures in karaf-maven-plugin
BootFeatures automatically begins on Karaf startup. InstalledFeatures just installs the library dependencies in the ${karaf.home}/system directory. The user will have to start it up manually via the command feature:install

For example, to start up camel at boot time but manually start up camel-quartz2
<bootFeatures>
    <feature>camel</feature>
</bootFeatures>
<installedFeatures>
    <feature>camel-quartz2</feature>
</installedFeatures>


Step 4: Add Maven repository http://svn.apache.org/repos/asf/servicemix/m2-repo
The dependency org.eclipse.equinox:region:jar:1.0.0.v20110506 for the "region" Karaf feature is not uploaded to the normal repo1.maven.org. Add this servicemix repository so Karaf can download the correct file
<repositories>
    <repository>
      <id>servicemix</id>
      <name>ServiceMix Repo for Karaf</name>
      <url>http://svn.apache.org/repos/asf/servicemix/m2-repo</url>
    </repository>
</repositories>

Step 5: Configure Karaf not to use external Maven
Create file src/main/resources/etc/org.ops4j.pax.url.mvn.cfg and add the following lines
org.ops4j.pax.url.mvn.settings=${karaf.home}/etc/karaf_maven_settings.xml
org.ops4j.pax.url.mvn.repositories=

This configures Karaf to not use the default Maven ~/.m2/settings.xml and not to try to download from any external Maven repositories like repo1.maven.org

See https://github.com/juttayaya/karaf/blob/master/karaf3/offline-karaf/src/main/resources/etc/org.ops4j.pax.url.mvn.cfg for a full example.

Create file src/main/resources/etc/karaf_maven_settings.xml with no Maven settings
<settings 
    xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http:/maven.apache.org/SETTINGS/1.0.0 
        http://maven.apache.org/xsd/settings-1.0.0.xsd">
</settings>

See https://github.com/juttayaya/karaf/blob/master/karaf3/offline-karaf/src/main/resources/etc/karaf_maven_settings.xml for a full example

Step 6: Manually add any missing dependencies
Not all Karaf runtime dependencies are listed in features.xml (Probably a bug). We have to help offline Karaf by manually downloading the missing dependencies.

For example, https://repo1.maven.org/maven2/org/apache/karaf/features/standard/3.0.4/standard-3.0.4-features.xml is missing org.apache.karaf.jaas.boot.jar. To download it manually and put it in the ${karaf.home}/system directory, configure the maven-dependency-plugin
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
      <execution>
        <id>copy</id>
        <phase>generate-resources</phase>
        <goals>
         <goal>copy</goal>
        </goals>
        <configuration>
         <artifactItems>
          <artifactItem>
          <groupId>org.apache.karaf.jaas</groupId>
          <artifactId>org.apache.karaf.jaas.boot</artifactId>
          <version>${karaf.version}</version>
          <outputDirectory>
          target/assembly/system/org/apache/karaf/jaas/org.apache.karaf.jaas.boot
          </outputDirectory>
          </artifactItem>
         </artifactItems>
        </configuration>
      </execution>
    </executions>
</plugin>



2015-09-28

How to uncompress tar.gz with one command

Problem:
I have a file called "thefiles.tar.gz" and I want to uncompress and unarchive in one Unix command.

Solution:
Use the command

tar xzpvf thefiles.tar.gz


Problem:
I have a directory called "thefiles" and I want to compress and archive it into a file called "thefiles.tar.gz" in one command.

Solution:
Use the command

tar czpvf thefiles.tar.gz thefiles

2015-09-21

Change Apache Karaf default logging options

What you will do:
  • Change the location of the Karaf log file
  • Change the Karaf log file name, size, and retention rate
  • Write Karaf system output to a log file
What you will learn:
  • Overwrite default Karaf configuration files
  • Create new directories in custom Karaf server
  • Set Karaf environment variables
Please see the post "Create a custom Apache Karaf server" for the initial setup.

Apache Karaf has poor initial logging configurations for production.  Below are instructions on how to change it.

Step 1: Overwrite default Karaf logging configuration file
The default Karaf logging configuration file is etc/org.ops4j.pax.logging.cfg . The default Karaf log settings is to write to data/log/karaf.log , with a 1MB file size limit and a 10 file retention. To overwrite, create the file src/main/resources/etc/org.ops4j.pax.logging.cfg .  We are changing the log file to logs/dekantar.log, with a daily rotating log. 

The contents of etc/org.ops4j.pax.logging.cfg should look like
# Root logger
log4j.rootLogger=INFO, daily, osgi:*
log4j.throwableRenderer=org.apache.log4j.OsgiThrowableRenderer

# Daily File appender
log4j.appender.daily=org.apache.log4j.DailyRollingFileAppender
log4j.appender.daily.layout=org.apache.log4j.PatternLayout
log4j.appender.daily.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j.appender.daily.file=${karaf.home}/logs/dekantar.log
log4j.appender.daily.datePattern='.'yyyy-MM-dd


Step 2: Create the logs directory
Create the logs directory in /src/main/resources/logs.


Step 3: Create the Karaf system output logfile
In Karaf 3.0.4, the Karaf system output log file is defined in the file bin/setenv as the variable KARAF_REDIRECT . If the variable is not defined, then the system output goes to /dev/null . In previous versions of Karaf 3, the system output logfile was hardcoded as data/log/karaf.out . 

To change the system output logfile to logs/dekantar.out , create the file src/main/resources/bin/setenv 

The content of bin/setenv should look like
#!/bin/sh

ENVHOME=`dirname $0`
LOG_HOME=`cd $ENVHOME/..;pwd`
export KARAF_REDIRECT=${LOG_HOME}/logs/dekantar.out

2015-09-20

Migrating JAXB from WebLogic 11g to WebLogic 12c

From WebLogic 11g to WebLogic 12c, the default JAXB implementation changed from Glassfish RI to EclipseLink MOXy. (1)

EclipseLink MOXy has some fun surprises and bugs if you are not expecting it. The JAXB bindings and data population may not work as previously expected. Click the links below for some examples.


To revert to the previous version of Glassfish RI in WebLogic 12, try one of the solutions below.

Use Glassfish RI at the application level
1. Add the following section to the weblogic-application.xml of the EAR archive

<prefer-application-resources>
    <resource-name>
      META-INF/services/com.sun.xml.ws.spi.db.BindingContextFactory
    </resource-name>

    <resource-name>
      META-INF/services/javax.xml.bind.JAXBContext

    </resource-name>
</prefer-application-resources>

2. In the EAR archive, create the file
META-INF/services/com.sun.xml.ws.spi.db.BindingContextFactory

In this file, just add this one line

com.sun.xml.ws.db.glassfish.JAXBRIContextFactory

3. In the EAR archive, create the file
META-INF/services/javax.xml.bind.JAXBContext

In this file, just add this one line

com.sun.xml.bind.v2.ContextFactory

Use Glassfish RI at the Weblogic Server level
In the file user_projects/domain/mydomain/bin/setDomainEnv.sh, change the following

from
WL_HOME="/mypath/Oracle/Middleware/Oracle_Home/wlserver"
export WL_HOME

to
WL_HOME="/mypath/Oracle/Middleware/Oracle_Home/wlserver"
export WL_HOME

PRE_CLASSPATH="${WL_HOME}/modules/databinding.override_1.2.0.0.jar
export PRE_CLASSPATH

Use Glassfish RI with WebLogic client tools like jwsc
Add the Java system properties to the jwsc tool classpath

-Dcom.sun.xml.ws.spi.db.BindingContextFactory=
com.sun.xml.ws.db.glassfish.JAXBRIContextFactory,
javax.xml.bind.JAXBContext=
com.sun.xml.bind.v2.ContextFactory


Source
(1) http://docs.oracle.com/cd/E24329_01/web.1211/e24964/data_types.htm#WSGET345

2015-09-19

Change the username and password of Apache Karaf admin user

What you will do:
  • Change the username and password for Apache Karaf admin user
What you will learn:
  • Overwrite default Karaf configuration files
  • Use Maven filtered resources
Please see the post "Create a custom Apache Karaf server" for the initial setup.

By default, the admin user has the username/password of karaf/karaf.  For branding purposes, we want to rename the username/password of dekantar/dekantar .

Step 1: Define username/password
Define the username/password as Maven properties in the pom.xml

<properties>
    <dekantar.username>dekantar</dekantar.username>
    <dekantar.password>dekantar</dekantar.password>
</properties>

Step 2: Turn on Maven filtered resources
Filtered resource will substitute Maven variables in text files with the property values

<resources>
<resource>
    <directory>src/main/resources</directory>
    <filtering>false</filtering>
    <includes>
    <include>**/*</include>
    </includes>
</resource>
<resource>
    <directory>src/main/filtered-resources</directory>
    <filtering>true</filtering>
    <includes>
    <include>**/*</include>
    </includes>
</resource>
</resources>
The filtering is true property turns on variable substitution in the src/main/filtered-resources directory.


Step 3: Overwrite default Karaf security file
The default Karaf security file is etc/user.properties. To overwrite, create the file src/main/filtered-resources/etc/user.properties.

The content of etc/user.properties is

${dekantar.username} = ${dekantar.password},_g_:admingroup
_g_\:admingroup = group,admin,manager,viewer,webconsole

The variable substitution results in the output

dekantar = dekantar,_g_:admingroup
_g_\:admingroup = group,admin,manager,viewer,webconsole

Don't worry about the clear text password. Karaf will automatically encrypt it on first login.


Step 4: Test it
After building the Karaf server, start it via command

cd $KARAF_DIR/bin
./start

Then log in with the new username

ssh dekantar@localhost -p8101

Change name of Apache Karaf startup script

What you will do:
  • Change the name of the Apache Karaf startup script
What you will learn:
  • Configure maven-antrun-plugin to do post-processing
Please see the post "Create a custom Apache Karaf server" for the initial setup.

By default, the startup script is named "karaf". For branding purposes, we want to rename the karaf script to "dekantar".

Configure maven-antrun-plugin to rename the script name and do a "search-and-replace" in the script text.

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
 <executions>
    <execution>
        <id>rename_script</id>
        <phase>process-resources</phase>
        <goals>
        <goal>run</goal>
        </goals>
        <configuration>
        <target>
        <move file="target/assembly/bin/karaf"
              tofile="target/assembly/bin/dekantar"/>
        <move file="target/assembly/bin/karaf.bat"
              tofile="target/assembly/bin/dekantar.bat"/>
        <replace dir="target/assembly/bin"
                 token="bin/karaf "
                 value="bin/dekantar "/>
        <replace dir="target/assembly/bin"
                 token="karaf.out"
                 value="dekantar.out"/>
        <replace dir="target/assembly/bin"
                 token="karaf.bat"
                 value="dekantar.bat"/>
        <replace dir="target/assembly/bin"
                 token='KARAF_SCRIPT="karaf' 
                 value='KARAF_SCRIPT="dekantar'/>
        </target>
        </configuration>
    </execution>
    </executions>
</plugin>

The target/assembly directory is where the karaf-maven-plugin assembles all the files and dependencies for the Karaf server distribution.

See https://github.com/juttayaya/karaf/blob/master/karaf3/custom-karaf/custom-karaf-server/pom.xml for full example.

Moral of the story:
Use maven-antrun-plugin to execute custom commands in Maven builds

2015-09-17

XMLGregorianCalendar DatatypeFactory considered harmful

If you have ever worked with XML and JAXB, you will be familiar with XmlGregorianCalendar.  It is the default Java class that JAXB uses to represent XML date/time.  

A Java web service application I worked on experienced performance issues under load.  By simulating heavy SOAP request traffic with LoadUI and profiling with YourKit, I discovered the culprit was DatatypeFactory.


This particular SOAP web service response had at least fifty date/time fields. The code to construct the JAXB response naively called 


    DatatypeFactory.newInstance().newXMLGregorianCalendar();

for each date/time field.


The call DatatypeFactory.newInstance() turns out to be very expensive. As shown in the JavaDoc

the newInstance method is a four step process with reflection.

Each time DatatypeFactory.newInstance() is called, Java has to rerun the four step discovery and creation process.


The obvious solution is to cache and reuse DatatypeFactory.newInstance(). I needed to confirm that the DatatypeFactory is thread-safe.  By doing a simple DatatypeFactory.newInstance().getClass().getName(), I discovered that the concrete implementation is com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl

By looking at the source code at http://www.docjar.com/html/api/com/sun/org/apache/xerces/internal/jaxp/datatype/DatatypeFactoryImpl.java.html, I verified that newXMLGregorianCalendar() is thread-safe.


The implementation at line 201 is just

public XMLGregorianCalendar newXMLGregorianCalendar() {
    return new XMLGregorianCalendarImpl();
}

which is about as thread-safe as you can get.

Moral of Story: Do not call DatatypeFactory.newInstance() every time you need an XMLGregorianCalendar


Additional source

Create a custom Apache Karaf server

What you will do:
  • Create a custom Apache Karaf distribution using karaf-maven-plugin
  • Create a custom logo using maven-bundle-plugin
What you will learn:
  • Configure karaf-maven-plugin
  • Build a OSGi bundle using maven-bundle-plugin
  • Use maven-dependency-plugin for post-processing
This tutorial creates a custom Karaf 3 server called "Dekantar" using Maven. The procedure for Karaf 2 and 4 are similar.
Karaf 2: Coming soon
Karaf 3: https://github.com/juttayaya/karaf/tree/master/karaf3/custom-karaf
Karaf 4: Coming soon


Step 1: Import the Karaf 3 dependencies
Add the following entry in the dependencyManagement section of the pom.xml
<dependencyManagement>
<!-- Import Karaf POM to use the correct 
     version of the Karaf dependencies -->
<dependencies>
    <dependency>
        <groupId>org.apache.karaf</groupId>
        <artifactId>karaf</artifactId>
        <version>${karaf.version}</version>
        <type>pom</type>
        <scope>import</scope>
    </dependency>
</dependencies>
</dependencyManagement>

Doing this will automatically configure the custom Karaf Maven build to use the correct version of dependencies libraries.

See https://github.com/juttayaya/karaf/blob/master/karaf3/custom-karaf/pom.xml for full example.

Step 2: Configure the karaf-maven-plugin
Add the following entry to the pom.xml to configure karaf-maven-plugin to build Karaf
<dependencies>
<dependency>
<!-- Scope is compile so all features (there is only one) are installed
     into startup.properties and
     the feature repo itself is not added in
     etc/org.apache.karaf.features.cfg file
-->
    <groupId>org.apache.karaf.features</groupId>
    <artifactId>framework</artifactId>
    <type>kar</type>
    <scope>compile</scope>
</dependency>
<dependency>
<!-- Scope is runtime so the feature repo is listed
     in etc/org.apache.karaf.features.cfg file,
     and features will installed into the system directory.
     This imports the standard Karaf features described at   https://repo1.maven.org/maven2/org/apache/karaf/features/standard/3.0.4/standard-3.0.4-features.xml
     These features are installed in the karaf-maven-plugin configuration
-->
    <groupId>org.apache.karaf.features</groupId>
    <artifactId>standard</artifactId>
    <classifier>features</classifier>
    <type>xml</type>
    <scope>runtime</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
    <groupId>org.apache.karaf.tooling</groupId>
    <artifactId>karaf-maven-plugin</artifactId>
    <configuration>
    <karafVersion>${karaf.version}</karafVersion>
    <!-- bootFeatures starts up these features when Karaf first boots up.
         The feature names are obtained from the features.xml in the
         dependencies section
    -->
    <bootFeatures>
        <feature>standard</feature>
        <feature>config</feature>
        <feature>ssh</feature>
    </bootFeatures>
    <!-- installedFeatures only installs the libraries but the user will
         have to start it up manually via Karaf command line feature:install
    -->
    <installedFeatures>
        <feature>scheduler</feature>
        <feature>war</feature>
    </installedFeatures>
    </configuration>
</plugin>
</plugins>
</build>


Step 3: Create customized logo
You can customize the logo displayed when users first log in via command line

A.) Create the file src/main/resources/org/apache/karaf/branding/branding.properties
This file configures the welcome logo and command prompt.

B.) Configure maven-bundle-plugin to create an OSGi bundle with the branding.properties file.  Add the following entry in the pom.xml
<build>
<plugins>
<plugin>
    <groupId>org.apache.felix</groupId>
    <artifactId>maven-bundle-plugin</artifactId>
    <configuration>
    <instructions>
        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
        <Import-Package>*</Import-Package>
        <Private-Package>!*</Private-Package>
        <Export-Package>org.apache.karaf.branding</Export-Package>
        <Spring-Context>*;public-context:=false</Spring-Context>
    </instructions>
    </configuration>
</plugin>
</plugins>
</build>

The maven-bundle-plugin creates a JAR file with a META-INF/MANIFEST.MF and branding.properties .

See https://github.com/juttayaya/karaf/blob/master/karaf3/custom-karaf/custom-karaf-branding/pom.xml for full example.

C.) Configure maven-dependency-plugin to copy the branding JAR created in step B into the Karaf lib directory.  Add the following entry to the pom.xml
<build>
<plugins>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <inherited>false</inherited>
    <executions>
        <execution>
        <id>copy</id>
        <phase>generate-resources</phase>
        <goals>
        <goal>copy</goal>
        </goals>
        <configuration>
        <artifactItems>
        <artifactItem>
        <groupId>${project.groupId}</groupId>
        <artifactId>custom-karaf-branding</artifactId>
        <version>${project.version}</version>
        <!-- target/assembly is the directory karaf-maven-plugin
  assembles all the files of the Karaf server -->
        <outputDirectory>target/assembly/lib</outputDirectory>
        <destFileName>custom-karaf-branding.jar</destFileName>
        </artifactItem>
        </artifactItems>
        </configuration>
        </execution>
    </executions>
</plugin>
</plugins>
</build>

See https://github.com/juttayaya/karaf/blob/master/karaf3/custom-karaf/custom-karaf-server/pom.xml for full example.

Step 4: Build it
Run mvn clean install
The output file is target/custom-karaf-server.tar.gz and target/custom-karaf-server.zip