Compiling Protobuf Sources into Java

The following default directory structure of the project is assumed:

./
+- pom.xml
+- src/
   +- main/
      +- proto/
         +- message.proto
   +- test/
      +- proto/
         +- test_message.proto

Protocol buffer definitions are looked up under src/main/proto/ directory by default. Any subdirectories under src/main/proto/ are treated as package structure for protobuf definition imports. Similarly, protobuf definitions for use in tests are looked up under src/test/proto/ directory by default.

A minimal configuration to invoke this plugin would look like:

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.0</version>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.xolstice.maven.plugins</groupId>
        <artifactId>protobuf-maven-plugin</artifactId>
        <version>0.6.1</version>
        <configuration>
          <protocExecutable>/usr/local/bin/protoc</protocExecutable>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>compile</goal>
              <goal>test-compile</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      ...
    </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java</artifactId>
      <version>3.4.0</version>
    </dependency>
    ...
  </dependencies>
  ...
</project>

The following conditions need to be met:

• Java 1.5 or newer is required due to the usage of Generics
• Java 1.6 or newer is required for running custom Java plugins for protobuf compiler
• Java 1.7 or newer is recommended
• Either protoc executable has to be in the PATH or the protocExecutable parameter has to be set to the correct location. An alternative configuration involves toolchains (described in plugin examples).
• Plugin executions need to be defined, because the plugin is not part of the default lifecycle. However, plugin goals have default bindings to appropriate phases, therefore there is typically no need to specify phase bindings in plugin configuration.
  Note: test-compile goal will only be needed if there are custom protocol buffer definitions being used in your tests.
• An appropriate version of com.google.protobuf:protobuf-java dependency has to be declared, or made available through POM inheritance: this is needed for compiling the generated source code. Note: it is important to make sure that the version of protobuf compiler is compatible with the version of protobuf java library (ideally they should be of the same version), otherwise the generated code will fail to compile.

As soon as everything is set up, execute the following goals to build the project:

mvn clean install

Dealing with "Command line is too long" errors

This plugin simply invokes protoc binary, passing all necessary arguments on the command line. In some configurations with a very large number of protobuf definitions this may be a problem, due to the limits on command line length in the host operating system. Starting with protoc version 3.5.0 this can be worked around by passing the following option:

<plugin>
  <groupId>org.xolstice.maven.plugins</groupId>
  <artifactId>protobuf-maven-plugin</artifactId>
  <version>0.6.1</version>
  <configuration>
    ...
    <useArgumentFile>true</useArgumentFile>
  </configuration>
  ...
</plugin>

Unfortunately, for versions of protoc below 3.5.0, the only available option is to split the compilation into smaller chunks by decomposing the project into modules.

Compiling Protobuf Sources into C++ or Python

The plugin configuration is similar to compiling into Java, with the following alterations:

• A different set of goals will need to be used (e.g. compile-cpp);
• A dependency on com.google.protobuf:protobuf-java artifact may not be required.

Running Plugin Goals On Command Line

It is possible to run Maven Protocol Buffers Plugin goals from the command line, even if the plugin is not configured in the project:

mvn protobuf:compile      -DprotocExecutable="C:/Java/protobuf-2.4.1/bin/protoc.exe"
mvn protobuf:test-compile -DprotocExecutable="C:/Java/protobuf-2.4.1/bin/protoc.exe"

If a protobuf toolchain is configured in the project, then the toolchains plugin needs to be executed first and there is no need to specify the protocExecutable parameter.

mvn toolchains:toolchain protobuf:compile
mvn toolchains:toolchain protobuf:test-compile

Suppressing Compilation Of Unchanged Definitions

Normally this plugin invokes protoc compilation on every execution, but this can be overridden by the following configuration option:

<plugin>
  <groupId>org.xolstice.maven.plugins</groupId>
  <artifactId>protobuf-maven-plugin</artifactId>
  <version>0.6.1</version>
  <configuration>
    ...
    <checkStaleness>true</checkStaleness>
  </configuration>
  ...
</plugin>

If the project is built on NFS, the following setting may also be needed:

<plugin>
  <groupId>org.xolstice.maven.plugins</groupId>
  <artifactId>protobuf-maven-plugin</artifactId>
  <version>0.6.1</version>
  <configuration>
    ...
    <checkStaleness>true</checkStaleness>
    <staleMillis>10000</staleMillis>
  </configuration>
  ...
</plugin>

Output Of Binary File Descriptor Sets

It is possible to output binary FileDescriptorSet files containing all the descriptor metadata for generated classes. Descriptor sets are written by passing the --descriptor_set_out and --include_imports arguments to protoc.

Generated descriptor sets can optionally be attached to the build as artifacts. The default type and extension for descriptor sets is protobin, and the plugin extensions need to be enabled in order to support the correct resolution of those dependencies in downstream projects.

The following plugin options configure descriptor set output:

• writeDescriptorSet - determines if a descriptor set is written; default is false.
• includeDependenciesInDescriptorSet - if true, all imports are included in the descriptor set; default is false.
• includeSourceInfoInDescriptorSet - if true, source code information will be included in the descriptor set; default is false. Please note that the size of the description set may increase significantly if this option is enabled.
• descriptorSetFileName - the file name of the descriptor set; default is the artifact name with the .protobin extension.
• descriptorSetOutputDirectory - the directory where the descriptor set file will be created
• attachDescriptorSet - attach the generated descriptor set as an artifact to the build
• descriptorSetClassifier - an optional artifact classifier for the attached descriptor