SyntaxHighlighter JS

2015-09-29

How to build an offline Karaf server

What you will do:
  • Create an offline version of Apache Karaf
What you will learn:
  • Configure karaf-maven-plugin and maven-dependency-plugin
  • Override Karaf default behavior with Maven 
The default behavior of Apache Karaf is to download Java library dependencies as needed from Maven repositories such as http://repo1.maven.org/ . To perform this functionality, Karaf will use the Maven ~/.m2/settings.xml file and store dependencies in the ~/.m2/repository directory.

The confirm this behavior empirically
mv ~/.m2 ~/.m2_save

And disable internet access.
If you start a new version of Karaf and try to install any feature such as
feature:install jndi

you will get the error message
Error executing command: Can't install feature jndi/0.0.0: 
Error resolving artifact org.apache.xbean:xbean-naming:jar:3.18: Could not transfer artifact org.apache.xbean:xbean-naming:jar:3.18 from/to central (http://repo1.maven.org/maven2/): repo1.maven.org: nodename nor servname provided, or not known

If you turn internet access back on and rerun feature:install jndi, Karaf will create the directory ~/.m2/repository and put the library dependencies for Karaf jndi in that directory.

This may be undesirable behavior, especially in secured production, when servers typically cannot download artifacts freely from the internet.  From a developer perspective, it can be surprising that changing Maven settings.xml for another unrelated project can affect the runtime behavior of Karaf.

Follow the instructions below to create an offline version of Apache Karaf where the dependencies are preloaded at compile time and is not dependent on Maven.

This tutorial is based on Karaf 3.0.4 . The procedure for Karaf 2 and 4 are similar
Karaf 2: Coming Soon
Karaf 3: https://github.com/juttayaya/karaf/tree/master/karaf3/offline-karaf
Karaf 4: https://github.com/juttayaya/karaf/tree/master/karaf4/offline-karaf4

Step 1: Configure karaf-maven-plugin to generate an offline version of Karaf
Configure the setting ignoreDependencyFlag of karaf-maven-plugin to true. This configures the karaf-maven-plugin to download library dependencies into the directory ${karaf.home}/system
<plugin>
    <groupId>org.apache.karaf.tooling</groupId>
    <artifactId>karaf-maven-plugin</artifactId>
    <!-- Plugin requires at minimum 3.0.3 version for dependency=true bug fix
    https://issues.apache.org/jira/browse/KARAF-2596 -->
    <version>${karaf.plugin.version}</version>
    <extensions>true</extensions>
    <configuration>
      <karafVersion>${karaf.version}</karafVersion>
      <!-- ignoreDependencyFlag is true forces plugin to also
           download feature dependent libraries -->
      <ignoreDependencyFlag>true</ignoreDependencyFlag>
    </configuration>
</plugin>

Step 2: Add the Karaf features.xml you want to install as dependencies.
For example, to add camel to Karaf, add this dependency
<!-- https://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/2.15.2/apache-camel-2.15.2-features.xml -->
<dependency>
    <groupId>org.apache.camel.karaf</groupId>
    <artifactId>apache-camel</artifactId>
    <version>${camel.version}</version>
    <classifier>features</classifier>
    <type>xml</type>
    <scope>runtime</scope>
</dependency>

Step 3:  Add the feature as bootFeatures or installedFeatures in karaf-maven-plugin
BootFeatures automatically begins on Karaf startup. InstalledFeatures just installs the library dependencies in the ${karaf.home}/system directory. The user will have to start it up manually via the command feature:install

For example, to start up camel at boot time but manually start up camel-quartz2
<bootFeatures>
    <feature>camel</feature>
</bootFeatures>
<installedFeatures>
    <feature>camel-quartz2</feature>
</installedFeatures>


Step 4: Add Maven repository http://svn.apache.org/repos/asf/servicemix/m2-repo
The dependency org.eclipse.equinox:region:jar:1.0.0.v20110506 for the "region" Karaf feature is not uploaded to the normal repo1.maven.org. Add this servicemix repository so Karaf can download the correct file
<repositories>
    <repository>
      <id>servicemix</id>
      <name>ServiceMix Repo for Karaf</name>
      <url>http://svn.apache.org/repos/asf/servicemix/m2-repo</url>
    </repository>
</repositories>

Step 5: Configure Karaf not to use external Maven
Create file src/main/resources/etc/org.ops4j.pax.url.mvn.cfg and add the following lines
org.ops4j.pax.url.mvn.settings=${karaf.home}/etc/karaf_maven_settings.xml
org.ops4j.pax.url.mvn.repositories=

This configures Karaf to not use the default Maven ~/.m2/settings.xml and not to try to download from any external Maven repositories like repo1.maven.org

See https://github.com/juttayaya/karaf/blob/master/karaf3/offline-karaf/src/main/resources/etc/org.ops4j.pax.url.mvn.cfg for a full example.

Create file src/main/resources/etc/karaf_maven_settings.xml with no Maven settings
<settings 
    xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http:/maven.apache.org/SETTINGS/1.0.0 
        http://maven.apache.org/xsd/settings-1.0.0.xsd">
</settings>

See https://github.com/juttayaya/karaf/blob/master/karaf3/offline-karaf/src/main/resources/etc/karaf_maven_settings.xml for a full example

Step 6: Manually add any missing dependencies
Not all Karaf runtime dependencies are listed in features.xml (Probably a bug). We have to help offline Karaf by manually downloading the missing dependencies.

For example, https://repo1.maven.org/maven2/org/apache/karaf/features/standard/3.0.4/standard-3.0.4-features.xml is missing org.apache.karaf.jaas.boot.jar. To download it manually and put it in the ${karaf.home}/system directory, configure the maven-dependency-plugin
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
      <execution>
        <id>copy</id>
        <phase>generate-resources</phase>
        <goals>
         <goal>copy</goal>
        </goals>
        <configuration>
         <artifactItems>
          <artifactItem>
          <groupId>org.apache.karaf.jaas</groupId>
          <artifactId>org.apache.karaf.jaas.boot</artifactId>
          <version>${karaf.version}</version>
          <outputDirectory>
          target/assembly/system/org/apache/karaf/jaas/org.apache.karaf.jaas.boot
          </outputDirectory>
          </artifactItem>
         </artifactItems>
        </configuration>
      </execution>
    </executions>
</plugin>



7 comments:

  1. Hi,

    Many Thanks for your great blog post and sample project.

    I was wondering if you would prepare another post for offline Karaf 4.0.1 server? I tried to modify your pom.xml for karaf 3.0.4 version simply by changing the karaf.version property with 4.0.1, but got quite some errors during mvn install... for e.g.,

    [ERROR] Failed to execute goal org.apache.karaf.tooling:karaf-maven-plugin:4.0.1:assembly (default-assembly) on project offline-karaf: Unable to build assembly: Could not find matching feature for standard -> [Help 1]

    ReplyDelete
    Replies
    1. I posted a working offline Karaf 4 build at
      https://github.com/juttayaya/karaf/tree/master/karaf4/offline-karaf4

      Karaf4 changed quite a bit from Karaf 3. If you compare
      https://repo1.maven.org/maven2/org/apache/karaf/features/standard/3.0.4/standard-3.0.4-features.xml
      with
      https://repo1.maven.org/maven2/org/apache/karaf/features/standard/4.0.1/standard-4.0.1-features.xml

      you will see that
      feature name="standard"
      is not in the 4.0.1 version.

      I had some issues with getting offline Camel to work, which I will work on tonight

      Delete
    2. Great thanks for your help! Hope you can resolve the issue soon

      I also recognize that standard feature is no longer available in 4.0.1 version, and try to add each individual features like the pom.xml in your github. One thing is that I added the org.ops4j.pax.url.mvn.cfg in your karaf3 folder to my karaf4 build, which commented all the external mvn repository so it become really "offline". Although I can complete the mvn install successfully, I always get different kinds of "Could not find artifact " exceptions during startup...

      e.g.
      2015-10-05 17:57:18,500 | WARN | pool-3-thread-1 | AetherBasedResolver | 1 - org.ops4j.pax.logging.pax-logging-api - 1.8.3 | Error resolving artifactorg.apache.aries:org.apache.aries.util:jar:1.1.0:Could not find artifact org.apache.aries:org.apache.aries.util:jar:1.1.0
      shaded.org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.apache.aries:org.apache.aries.util:jar:1.1.0

      it's okay to startup without error if I remove the comments in that mvn.cfg file... but I guess then karaf was actually getting the missing artifacts remotely and not really "offline"

      Delete
    3. I fixed all known offline Karaf issues last night. Please pull the latest from github and let me know if you experience any problems.

      One annoying aspect of Karaf is the sparse documentation. The behavior of karaf-maven-plugin ignoreDependencyFlag changed from Karaf 3 to Karaf 4. Now to download all the dependency libraries, the flag should be set to false in Karaf 4.

      Delete
    4. I was able to build offline karaf4 using the latest code in your github repo! Thanks again for your work! Especially your "prepare-package" in pom.xml file which supplemented any missing dependencies really did the tricks and helps on my previous issues

      It was really odd that the behavior of ignoreDependencyFlag was "flipped" without notice...
      But I'm more interested about how did you recognize this behavior change?

      Just wanted to learn from your experience on the problem investigation process so I could try to troubleshoot by myself next time :)

      Delete
    5. After building the offline Karaf, I turn off my internet connection and delete ~/.m2.

      Starting offline Karaf, I check data/log/karaf.log to see exceptions and what bundles are missing. I did a text search on the missing libraries in the file https://repo1.maven.org/maven2/org/apache/karaf/features/standard/4.0.1/standard-4.0.1-features.xml

      I noticed that all the missing bundles all had dependency="true" . So knowing that the ignoreDependencyFlag controlled how the karaf-maven-plugin reacts to this attribute, I tested my assumption by changing it to false. Luckily it worked.

      Unfortunately trial and error does not always work out so I usually just read the Karaf source code when I am fully stumped.

      Delete