4.0.0 Migration Guide

This migration guide provides examples in both Java and Scala. Use the buttons on the right to select your preferred language.

3.11.0 to 4.0.0

Overview of changes to be aware of in this release are:

  • Daffodil now requires Scala 3.3.6 and Java 17+
  • Daffodil has merged daffodil-udf, daffodil-runtime1, daffodil-runtime1-unparser, daffodil-lib, daffodil-sapi, daffodil-japi and daffodil-io into daffodil-core.
  • Daffodil has replaced the Validation Modes(withValidationMode)/withValidator with the withValidation(validatorName[, validationConfigurationURL]) method. Built-in validator names are xerces, daffodil, schematron and off.
  • com.typesafe.config class has been replaced with java.util.Properties class for Validators configuration
  • getDiagnostics returns a java.util.List instead of a Seq
  • Diagnostic removed getSomeMessage and getSomeCause
  • withExternalVariables expects a Java Map instead of Scala Map.
  • Diagnostic.getMessage() now returns just the message, and toString should be used if the diagnostic context is needed
  • getAnyRef in MetaData has been removed

Dependencies

The following dependencies have been merged into daffodil-core:

  • daffodil-udf
  • daffodil-sapi
  • daffodil-japi
  • daffodil-runtime1
  • daffodil-runtime1-unparser
  • daffodil-lib
  • daffodil-io

Dependencies to the listed jars should be changed to the following.

<dependency>
  <groupId>org.apache.daffodil</groupId>
  <artifactId>daffodil-core_3</artifactId>
  <version>4.0.0</version>
</dependency>

scalaVersion := "3.3.6"
libraryDependencies += "org.apache.daffodil" %% "daffodil-core" % "4.0.0"

Core Imports

The path to core api classes were updated from japi/sapi to api

// ----- BEFORE -----
import org.apache.daffodil.japi.ProcessorFactory;
import org.apache.daffodil.japi.Daffodil; // factory method for compiler
import org.apache.daffodil.japi.DataProcessor;
import org.apache.daffodil.japi.Diagnostic;
import org.apache.daffodil.japi.ParseResult;
import org.apache.daffodil.japi.UnparseResult;
import org.apache.daffodil.japi.DaffodilParseXMLReader;
import org.apache.daffodil.japi.DaffodilUnparseContentHandler;
import org.apache.daffodil.japi.io.InputSourceDataInputStream;
import org.apache.daffodil.japi.infoset.*; // all InfosetInputters and InfosetOutputters

// ----- AFTER -----
import org.apache.daffodil.api.ProcessorFactory;
import org.apache.daffodil.api.Daffodil; // factory method for compiler, InfosetInputter/Outputters and many others
import org.apache.daffodil.api.DataProcessor;
import org.apache.daffodil.api.Diagnostic;
import org.apache.daffodil.api.ParseResult;
import org.apache.daffodil.api.UnparseResult;
import org.apache.daffodil.api.DaffodilParseXMLReader;
import org.apache.daffodil.api.DaffodilUnparseContentHandler;
import org.apache.daffodil.api.InputSourceDataInputStream;
import org.apache.daffodil.api.infoset.*; // InfosetInputter, InfosetOutputter, JDOM/Scala/W3CDOM InfosetOutputters

// ----- BEFORE -----
import org.apache.daffodil.sapi.ProcessorFactory
import org.apache.daffodil.sapi.Daffodil // factory method for compiler
import org.apache.daffodil.sapi.DataProcessor
import org.apache.daffodil.sapi.Diagnostic
import org.apache.daffodil.sapi.ParseResult
import org.apache.daffodil.sapi.UnparseResult
import org.apache.daffodil.sapi.DaffodilParseXMLReader
import org.apache.daffodil.sapi.DaffodilUnparseContentHandler
import org.apache.daffodil.sapi.io.InputSourceDataInputStream
import org.apache.daffodil.sapi.infoset._ // all InfosetInputters and InfosetOutputters

// ----- AFTER -----
import org.apache.daffodil.api.ProcessorFactory
import org.apache.daffodil.api.Daffodil // factory method for compiler, InfosetInputter/Outputters and many others
import org.apache.daffodil.api.DataProcessor
import org.apache.daffodil.api.Diagnostic
import org.apache.daffodil.api.ParseResult
import org.apache.daffodil.api.UnparseResult
import org.apache.daffodil.api.DaffodilParseXMLReader
import org.apache.daffodil.api.DaffodilUnparseContentHandler
import org.apache.daffodil.api.InputSourceDataInputStream
import org.apache.daffodil.api.infoset.* // InfosetInputter, InfosetOutputter, JDOM/Scala/W3CDOM InfosetOutputters

CLI Validation

Values for the --validate option have been renamed: on has been replaced with xerces and limited has been replaced with daffodil. Both schematron and xerces accept option values that provide a file to the validator for validator configuration. If no value is provided, it will attempt to use the schema provided by --schema. The file can either be a .xsd, .sch, .conf or .properties file. Typesafe configs have been replaced with java.util.Properties. --validate {schematron|xerces}=value can be used with --parser, enabling Xerces/Schematron validation with a saved parser.

# ----- BEFORE -----
daffodil parse --validate on -s schema.xsd input.dat
daffodil parse --validate limited --parser schema.parser myData.dat
daffodil parse --validate on --parser schema.parser myData.dat # would lead to error
daffodil parse --validate schematron=schema.sch --parser schema.parser myData.dat # would lead to error

# ----- AFTER -----
daffodil parse --validate xerces -s schema.xsd input.dat
daffodil parse --validate daffodil --parser schema.parser myData.dat
daffiduk parse --validate xerces=schema.xsd --parser schema.parser myData.dat # no error and validates
daffiduk parse --validate schematron=schema.sch --parser schema.parser myData.dat # no error and validates

Compiling Schemas

This remains generally unchanged for Java users as passing in strings (or nulls) is still supported. For Scala users, Options are no longer supported in the api and strings (or nulls) must be passed in instead.

// unchanged save for import paths

// ----- BEFORE -----
val pf: ProcessorFactory = c.compileFile(schemaFile, None, None)

// ----- AFTER -----
val pf: ProcessorFactory = c.compileFile(schemaFile, null, null)
// or
val pf = c.compileFile(schemaFile)

DataProcessor Validation

Validation Modes (i.e withValidationMode(ValidationMode.*)) and the withValidator(validatorObj) were removed in place of withValidation(validator.name[, validationConfigurationURL]). Some validators have mandatory properties–if not provided an exception will be thrown. See the validator documentation for the list of those properties. Custom validators must be found via SPI and are not expected to be directly created.

// ----- BEFORE -----
DataProcessor dp = pf.onPath("/").withValidationMode(ValidationMode.Full);
// or
DataProcessor dp = pf.onPath("/").withValidator(new CustomValidator());

// ----- AFTER -----
DataProcessor dp = pf.onPath("/").withValidation("xerces", schemaUrl); 
//or
DataProcessor dp = pf.onPath("/").withValidation("CUSTOM-VALIDATOR-NAME", schemaUrl); 

// ----- BEFORE -----
val dp: DataProcessor = pf.onPath("/").withValidationMode(ValidationMode.Full)
// or
val dp: DataProcessor = pf.onPath("/").withValidator(new CustomValidator())

// ----- AFTER -----
val dp: DataProcessor = pf.onPath("/").withValidation("xerces", schemaUrl) 
//or
val dp: DataProcessor = pf.onPath("/").withValidation("CUSTOM-VALIDATOR-NAME", schemaUrl)

Parse

Factory methods to get InputSourceDataInputStream and InfosetOutputter objects have been added via Daffodil.newInputSourceDataInputStream and Infoset.new*InfosetOutputter

// ----- BEFORE -----
InputSourceDataInputStream dis = new InputSourceDataInputStream(fis);
JDOMInfosetOutputter outputter = new JDOMInfosetOutputter();
XMLTextInfosetOutputter outputter = new XMLTextInfosetOutputter();
JsonInfosetOutputter outputter = new JsonInfosetOutputter();

// ----- AFTER -----
InputSourceDataInputStream dis = Daffodil.newInputSourceDataInputStream(fis);
JDOMInfosetOutputter outputter = Daffodil.newJDOMInfosetOutputter();
InfosetOutputter outputter = Daffodil.newXMLTextInfosetOutputter();
InfosetOutputter outputter = Daffodil.newJsonInfosetOutputter();

// ----- BEFORE -----
val dis = new InputSourceDataInputStream(fis)
val outputter = new ScalaXMLInfosetOutputter()
val outputter = new XMLTextInfosetOutputter()
val outputter = new JsonInfosetOutputter()

// ----- AFTER -----
val dis = Daffodil.newInputSourceDataInputStream(fis)
val outputter = Daffodil.newScalaXMLInfosetOutputter()
val outputter = Daffodil.newXMLTextInfosetOutputter()
val outputter = Daffodil.newJsonInfosetOutputter()

Unparse

Factory methods to get an InfosetInputter object has been added via Daffodil.new*InfosetInputter

// ----- BEFORE -----
JDOMInfosetInputter inputter = new JDOMInfosetInputter(outputter.getResult());
  
// ----- AFTER -----
InfosetInputter inputter = Daffodil.newJDOMInfosetInputter(outputter.getResult());

// ----- BEFORE -----
val inputter = new ScalaXMLInfosetInputter(outputter.getResult())

// ----- AFTER -----
val inputter = Daffodil.newScalaXMLInfosetInputter(outputter.getResult())

Debuggers

The different ways debuggers are defined and enabled have changed. Factory methods to get a Debugger object have been added via Daffodil.newDaffodilDebugger.

Using a custom debugger:

// ----- BEFORE -----
import org.apache.daffodil.japi.debugger.Debugger;

final class CustomDebugger extends Debugger {
  // implementation details
}

// to use
dp = dp.withDebugger(new CustomDebugger());
dp = dp.withDebugging(true);

// ----- AFTER -----
import org.apache.daffodil.api.debugger.Debugger;

final class CustomDebugger extends Debugger {
  // implementation details
}
dp.withDebugger(new CustomDebugger())

// ----- BEFORE -----
import org.apache.daffodil.sapi.debugger.Debugger

class CustomDebugger extends Debugger { ... }
// to use
dp.withDebugger(new CustomDebugger())

// ----- AFTER -----
import org.apache.daffodil.api.debugger.Debugger

class CustomDebugger extends Debugger { ... }
// to use
dp.withDebugger(new CustomDebugger())

Using a custom debugger runner:

// ----- BEFORE -----
import org.apache.daffodil.japi.debugger.DebuggerRunner;

final class CustomDebuggerRunner extends DebuggerRunner {
  // implementation details
}

// to use
import org.apache.daffodil.japi.Debugger;

Debugger debugger = new CustomDebuggerRunner();
dp = dp.withDebuggerRunner(debugger);
dp = dp.withDebugging(true);

// ----- AFTER -----
import org.apache.daffodil.api.debugger.DaffodilDebuggerRunner;

final class CustomDebuggerRunner extends DaffodilDebuggerRunner {
  // implementation details
}

// to use
import org.apache.daffodil.api.debugger.Debugger;

Debugger debugger = Daffodil.newDaffodilDebugger(new CustomDebuggerRunner());
dp.withDebugger(debugger);

// ----- BEFORE -----
import  org.apache.daffodil.sapi.debugger.DebuggerRunner

final class CustomDebuggerRunner extends DebuggerRunner {...}

// to use
import org.apache.daffodil.sapi.debugger.Debugger

dp.withDebuggerRunner(new CustomDebuggerRunner())
  .withDebugging(true)

// ----- AFTER -----
import org.apache.daffodil.api.debugger.DaffodilDebuggerRunner

final class CustomDebuggerRunner extends DaffodilDebuggerRunner {...}

// to use
import org.apache.daffodil.api.debugger.Debugger
import org.apache.daffodil.api.Daffodil

val debugger = Daffodil.newDaffodilDebugger(new CustomDebuggerRunner())
dp.withDebugger(debugger)

Using the built-in trace debugger:

// ----- BEFORE -----
import org.apache.daffodil.japi.debugger.TraceDebuggerRunner;

dp = dp.withDebuggerRunner(new TraceDebuggerRunner());
dp = dp.withDebugging(true);

// ----- AFTER -----
import org.apache.daffodil.api.debugger.Debugger;

Debugger traceDebugger = Daffodil.newTraceDebugger(System.out);
dp.withDebugger(traceDebugger);

// ----- BEFORE -----
import org.apache.daffodil.sapi.debugger.TraceDebuggerRunner

dp.withDebuggerRunner(new TraceDebuggerRunner())
  .withDebugging(true)

// ----- AFTER -----
val traceDebugger = Daffodil.newTraceDebugger(System.out)
dp.withDebugger(traceDebugger)

Layers

Custom Plug-in layers must extend the org.apache.daffodil.api.layers.Layer class, and be referenced in a META-INF/services file with the same class reference as the name. This path has changed in Daffodil 4.0.0.

// ----- BEFORE -----
// example layer class
package com.example.layers;

import org.apache.daffodil.runtime1.layers.api.Layer;

public final class CustomLayer extends Layer {
// implementation details
}

// in META-INF/services/org.apache.daffodil.runtime1.layers.api.Layer
com.example.layers.CustomLayer

// ----- AFTER -----
// example layer class
package com.example.layers;

import org.apache.daffodil.api.layers.Layer;

public final class CustomLayer extends Layer {
  // implementation details
}

// in META-INF/services/org.apache.daffodil.api.layers.Layer
com.example.layers.CustomLayer

// ----- BEFORE -----
// example layer class
package com.example.layers

import org.apache.daffodil.runtime1.layers.api.Layer

final class CustomLayer extends Layer("customLayer", "com.example.layers.customLayer") {...}

// in META-INF/services/org.apache.daffodil.runtime1.layers.api.Layer
com.example.layers.CustomLayer

// ----- AFTER -----
// example layer class
package com.example.layers

import org.apache.daffodil.api.layers.Layer

final class CustomLayer extends Layer("customLayer", "com.example.layers.customLayer") {...}

// in META-INF/services/org.apache.daffodil.api.layers.Layer
com.example.layers.CustomLayer

User Defined Functions

UDF Providers must extend the org.apache.daffodil.api.udf.UserDefinedFunctionProvider class, and be referenced in a META-INF/services file with the same class reference as the name. This path has changed in Daffodil 4.0.0.

// ----- BEFORE -----
// example UDF Provider class
package com.example.udf;

import org.apache.daffodil.udf.UserDefinedFunctionProvider;

public class CustomUDFProvider extends UserDefinedFunctionProvider {
  // implementation details
}

// in META-INF/services/org.apache.daffodil.udf.UserDefinedFunctionProvider
com.example.udf.CustomUDFProvider

// ----- AFTER -----
// example UDF Provider class
package com.example.udf;

import org.apache.daffodil.api.udf.UserDefinedFunctionProvider;

public class CustomUDFProvider extends UserDefinedFunctionProvider {
  // implementation details
}

// in META-INF/services/org.apache.daffodil.api.udf.UserDefinedFunctionProvider
com.example.udf.CustomUDFProvider

// ----- BEFORE -----
// example UDF Provider class
package com.example.udf

import org.apache.daffodil.udf.UserDefinedFunctionProvider

class CustomUDFProvider extends UserDefinedFunctionProvider {...}

// in META-INF/services/org.apache.daffodil.udf.UserDefinedFunctionProvider
com.example.udf.CustomUDFProvider

// ----- AFTER -----
// example UDF Provider class
package com.example.udf

import org.apache.daffodil.api.udf.UserDefinedFunctionProvider

class CustomUDFProvider extends UserDefinedFunctionProvider {...}

// in META-INF/services/org.apache.daffodil.api.udf.UserDefinedFunctionProvider
com.example.udf.CustomUDFProvider