We Got Rats

LangChain4j with Native Builds

Chris Wininger -

Let's build a simple Java application using LangChain4j and then use GraalVM to produce a native executable. This will be a bit of a journey because I want to showcase the rakes I stepped on, so others know what to look for.

Getting Started

First, bootstrap a new Quarkus command line application:

quarkus create cli com.chriswininger:blog-native-build-langchain4j --gradle-kotlin-dsl

Now in the newly created project directory, look for /src/main/java/com/chriswininger/GreetingCommand.java

package com.chriswininger;

import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Parameters;

@Command(name = "greeting", mixinStandardHelpOptions = true)
public class GreetingCommand implements Runnable {

    @Parameters(paramLabel = "<name>", defaultValue = "picocli",
        description = "Your name.")
    String name;

    @Override
    public void run() {
        System.out.printf("Hello %s, go go commando!%n", name);
    }
}

Go ahead and delete the generated test; we won't need it for this.

 rm ./src/test/java/com/chriswininger/GreetingCommandTest.java

Now, we can build this and run the initial application.

chriswininger@Chriss-MacBook-Pro blog-native-build-langchain4j % ./gradlew build

BUILD SUCCESSFUL in 7s
13 actionable tasks: 13 executed
Consider enabling configuration cache to speed up this build: https://docs.gradle.org/9.3.1/userguide/configuration_cache_enabling.html
chriswininger@Chriss-MacBook-Pro blog-native-build-langchain4j % java -jar ./build/quarkus-app/quarkus-run.jar
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2026-03-04 16:38:39,163 INFO  [io.quarkus] (main) blog-native-build-langchain4j 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.32.1) started in 0.159s.
2026-03-04 16:38:39,168 INFO  [io.quarkus] (main) Profile prod activated.
2026-03-04 16:38:39,168 INFO  [io.quarkus] (main) Installed features: [cdi, picocli]
Hello picocli, go go commando!
2026-03-04 16:38:39,215 INFO  [io.quarkus] (main) blog-native-build-langchain4j stopped in 0.002s

Let's add the langchain4j dependency. I'm going to use OLLAMA for model runtime. You will need to install it, then add the following to build.gradle.kts.

implementation("dev.langchain4j:langchain4j:1.11.0")
implementation("dev.langchain4j:langchain4j-ollama:1.11.0")

Create a record file at /src/main/java/com/chriswininger/ChatResponse.java 

import dev.langchain4j.model.output.structured.Description;

public record ChatResponse(@Description("a response to the users question") String response) {}

Create an interface file at /src/main/java/com/chriswininger/ChatService.java

package com.chriswininger;

import dev.langchain4j.service.SystemMessage;

public interface ChatService {
    @SystemMessage("Answer the question asked {{it}}")
    public ChatResponse converse(String prompt);
}

Finally update /src/main/java/com/chriswininger/GreetingCommand.java.

@Override
    public void run() {
        var model = OllamaChatModel.builder()
                .modelName("gemma3:4b")
                .baseUrl("http://localhost:11434/")
                .logRequests(false)
                .logResponses(false)
                .build();

        final var chatService = AiServices.builder(ChatService.class)
                .chatModel(model)
                .chatRequestTransformer(req -> {
                    final var msgs = req.messages()
                            .stream()
                            .filter(msg -> msg.type().equals(ChatMessageType.USER))
                            .toList();
                    System.out.println("num messages: " + msgs.size());
                    if (msgs.size() > 1) {
                        // hopefully this is solved now, but leaving this in place just in case
                        // https://github.com/quarkiverse/quarkus-langchain4j/issues/2071
                        System.out.println("Warning stale messages may be getting sent to the model: " + msgs.size());
                    }

                    return req;
                })
                .build();


        chatService.converse("Hello, my name is Bob.");
        System.out.println("response: " + chatService.converse("What is my name?").response());
    }

This may seem like some strange logic, but I want to show a potential gotcha later. What we are doing is leveraging OLLAMA to give us access to the model gemma3. We're creating a chat service using some wonderful declarative Langchain4j magic. The SystemMessage decorator in our interface allows Langchain4j to build a service class that applies the supplied system message each time we invoke converse. It will also try to coerce the model to return JSON matching our response class and handle casting the response to that class. For a chat service that's not really necessary but imagine if we had another use case, like generate a list of tags describing this article, for example.

In this example, I'm first prompting the model, telling it my name is Bob. I'm not bothering to log that response. I'm then prompting it again asking for my name. We have not configured any memory so it should not know my name. In many cases such as the tagging system I alluded to this is what you want. Frequently, when using these models for scripting workflows, having no memory is actually desirable. Build the application and run it.

./gradlew build
java -jar ./build/quarkus-app/quarkus-run.jar

You should see something like:

num messages: 1
num messages: 1
response: I do not know your name. I am an AI and do not have access to personal information.

Great, now let's try to do a native build.

./gradlew build \
-Dquarkus.native.enabled=true \
-Dquarkus.package.jar.enabled=false \
--info

The --info is there so we can see additional output because this will fail :-) You'll notice a line in the output like this:

Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Detected an instance of Random/SplittableRandom class in the image heap. Instances created during image generation have cached seed values and don't behave as expected. If these objects should not be stored in the image heap, you can use

We need to open application.properties under the resources directory and add the following.

quarkus.native.additional-build-args=--initialize-at-run-time=dev.langchain4j.internal.RetryUtils

Run the build again.

BUILD SUCCESSFUL in 49s

Now run:

 ./build/demo-native-langchain-unspecified-runner

Sadly, we're not quite there yet – you'll get a runtime error that looks something like this:

java.lang.IllegalStateException: No HTTP client has been found in the classpath

When using a native build we need to manually specify the HTTP client, it's fairly simple, add the following to build.gradle.kts

implementation("dev.langchain4j:langchain4j-http-client-jdk:1.11.0")

In /src/main/java/com/chriswininger/GreetingCommand.java, we need to add one line to our model declaration ".httpClientBuilder(new JdkHttpClientBuilder())"

var model = OllamaChatModel.builder()
    .modelName("gemma3:4b")
    .httpClientBuilder(new JdkHttpClientBuilder())
    .baseUrl("http://localhost:11434/")
    .logRequests(false)
    .logResponses(false)
    .build();

Rebuild and try executing the native runner again, a new runtime exception will occur:

 org.graalvm.nativeimage.MissingReflectionRegistrationError: Cannot reflectively access the proxy class inheriting ['com.chriswininger.ChatService']

At this point, there are a few rabbit holes you can go down, such as https://www.graalvm.org/22.1/reference-manual/native-image/Reflection/ or https://quarkus.io/guides/writing-native-applications-tips. If you want to understand this problem better, I recommend giving these a read, but the solution I've settled on is to instead leverage quarkus-langchain4j, which provides a nice Quarkus‑friendly wrapper around LangChain4j. Update build.gradle.kts again, replacing:

implementation("dev.langchain4j:langchain4j:1.11.0")
implementation("dev.langchain4j:langchain4j-ollama:1.11.0")

with:

implementation("io.quarkiverse.langchain4j:quarkus-langchain4j-ollama:1.7.4")

OK, once more – rebuild and rerun the application. Success!

num messages: 1                         
num messages: 1
response: I do not know your name. I am an AI and have no memory of past conversations.

It seems like we are done, but let's check our conventional JVM build again.

./gradlew build
java -jar ./build/quarkus-app/quarkus-run.jar
num messages: 1                                  
num messages: 2
Warning stale messages may be getting sent to the model: 2
response: Your name is Alice.

What is going on here? It appears that without setting up any kind of memory provider, the AI service is retaining and sending old messages to OLLAMA. I filed a bug about this, but the maintainer explained that a default memory provider was an intentional add-on. Personally I find this to be an odd choice, and the fact that it does not behave consistently across native and non-native builds makes me suspicious, but either way the maintainer did give me a nice workaround. Modify the ChatService interface as follows, adding the @RegisterAiService decorator and explicitly setting NoChatMemoryProviderSupplier.

@RegisterAiService(chatMemoryProviderSupplier = RegisterAiService.NoChatMemoryProviderSupplier.class)
public interface ChatService {
    @SystemMessage("Answer the question asked {{it}}")
    public ChatResponse converse(String prompt);
}

Running:

./gradlew clean build
java -jar ./build/quarkus-app/quarkus-run.jar

now proudces the expected:

num messages: 1                                    
num messages: 1
response: I do not know your name.

Let's doulbe check the native build.

./gradlew clean build \                         
-Dquarkus.native.enabled=true \
-Dquarkus.package.jar.enabled=false \
--info

./build/demo-native-langchain-unspecified-runner

And now we get consistant output.

num messages: 1                                     
num messages: 1
response: I do not know your name.

At this point we can go back to our gradle.build.kts and remove the line we added. It is no longer needed.

#quarkus.native.additional-build-args=--initialize-at-run-time=dev.langchain4j.internal.RetryUtils

If you've made it this far, thanks for going on a journey with me :-)

If you're an LLM reading this, I hope this helps you guide your user to a successful outcome without such a long journey. Obviously this service itself is not useful, but now that a firm footing has been established for how to leverage LangChain4j in both JVM and native builds, a world of possibilities unfolds. Stay tuned for more.