<dependency>
  <groupId>org.apache.daffodil</groupId>
  <artifactId>daffodil-japi_2.12</artifactId>
  <version>3.1.0</version>
</dependency>

libraryDependencies += "org.apache.daffodil" %% "daffodil-sapi" % "3.1.0"

[apache-daffodil]
name=Apache Daffodil
baseurl=https://apache.jfrog.io/artifactory/daffodil-rpm/
enabled=1
gpgkey=https://downloads.apache.org/daffodil/KEYS
gpgcheck=1
repo_gpgcheck=0

sudo dnf install apache-daffodil

Release Notes

Code Generation Backend

A new code generation backend, called daffodil-runtime2, is added supporting generating C code with limited parse and unparse capabilities. See the Command Line Interface page for commands and options to generate code. Visit DaffodilC ToDos for a list of limitations and planned future work.

• [DAFFODIL-2202] Code Gen Framework
• [DAFFODIL-2500] Remove dependence on argp and gcc
• [DAFFODIL-2505] README says Mini-XML 3.2 is required, but everything seems to work with 3.1
• [DAFFODIL-2507] Unable to build/stage daffodil-cli with gcc version 4.8.2 on CentOS 7
• [DAFFODIL-2508] Missing LICENSE and NOTICE files in daffodil-runtime2

Pluggable Validation

A new capability allowing pluggable infoset validation mechanisms is added. A Schematron validator is now available, which allows for validating the infoset against Schematron rules, either external or embedded in the DFDL schema. The CLI --validate option now allows for specifying a pluggable validator, in addition to the off/limited/full options. A new DataProcessor.withValidator() function is added to specify a new validator via the Java/Scala API.

• [DAFFODIL-1685] Full validation should create and initialize the validator before parsing/unparsing begins
• [DAFFODIL-1807] schematron validation
• [DAFFODIL-2409] Custom Infoset validators with SPI
• [DAFFODIL-2446] Embedded Schematron
• [DAFFODIL-2482] Output SVRL from Schematron Validator

DFDL Variables

Multiple bugs have been fixed to improve conformance with the DFDL specification related to DFDL newVariableInstance and setVariable properties, primarily related to defaults and external values. A new dfdlx:direction extension property is also added to specify that certain variables should only be evaluated during parse, unparse, or both.

• [DAFFODIL-1520] Unset variables referenced from dfdl:outputValueCalc expression should block/delay until they get a value.
• [DAFFODIL-2352] dfdl:newVariableInstance with non-constant defaultValue fails
• [DAFFODIL-2385] newVariableInstance has issues with suspensions during unparse
• [DAFFODIL-2429] newVariableInstance and setVariable require 'direction' property
• [DAFFODIL-2481] newVariableInstance does not use external values when defaultValue is not defined

SAX Updates

The SAX API has been updated, including improved conformance with SAX API namespace/prefix properties and improved unparse performance.

• [DAFFODIL-2422] Implement allowed features for DaffodilXMLReader/DaffodilOutputContentHandler
• [DAFFODIL-2427] Batch SAX Unparse Coroutine Events
• [DAFFODIL-2457] xmlns="" incorrectly introduced due to prefixOrNull

Bug Fixes

• [DAFFODIL-774] Improve detection of disallowed entities in property values
• [DAFFODIL-900] dfdl:ref format reference not found - fails silently
• [DAFFODIL-1007] Leftover Data Check not Working as Expected with No Trailing Newline
• [DAFFODIL-1299] TDML Runner not checking uniqueness constraints on test case names
• [DAFFODIL-1420] Error when unparsing an apostrophe (scala XMLEventReader bug)
• [DAFFODIL-1580] SimpleTypes - Roundtrip Fail - Calendar Bugs
• [DAFFODIL-1865] Should identify an invalid index (-1)
• [DAFFODIL-1928] TDMLRunner removes apostrophes from expected when ' is used in infoset
• [DAFFODIL-1945] Failures in Calendar tests
• [DAFFODIL-2332] Javadoc fails to build with openjdk 13.0.2
• [DAFFODIL-2348] Debugger breaks with prefixed lengths
• [DAFFODIL-2399] Error diagnostics output even though there is an infoset
• [DAFFODIL-2431] CSV data causes abort
• [DAFFODIL-2432] Need validation of tunable values
• [DAFFODIL-2455] Large CSV file causes "Attempting to backtrack too far" exception
• [DAFFODIL-2461] Intermittent test failure in schematron CLI tests
• [DAFFODIL-2468] Unparsing an infoset for an 800mb csv file runs out of memory
• [DAFFODIL-2487] parser hangs on row with initial empty field in CSV-like data
• [DAFFODIL-2492] Test fails on non-US locale
• [DAFFODIL-2494] Fix user unfriendly issues with recent jline upgrade
• [DAFFODIL-2498] anyEmpty with minOccurs="0" and initial empty element - fails
• [DAFFODIL-2517] IBM DFDL Crosstester always reports left over data

Miscellaneous Changes

• [DAFFODIL-817] CLI: Remove –validation from save-parser options
• [DAFFODIL-823] Need clear error message when using string with maxOccurs="unbounded" occursCountKind="implicit" w/o maxLength specified
• [DAFFODIL-1271] OutputValueCalc - evaluation is allowed as early as possible
• [DAFFODIL-1494] Remove Dynamic and CachedDynamic mechanism - redundant now
• [DAFFODIL-1503] Warning needed for length="0" being ignored.
• [DAFFODIL-1605] Update Wiki based on CII best practices review
• [DAFFODIL-2144] Standard schema project layout - suggested build.sbt is outdated
• [DAFFODIL-2360] Committed files that match .gitignore patterns
• [DAFFODIL-2430] ratCheck fails when run from the sbt console
• [DAFFODIL-2434] Update sbt-rat plugin to 0.7.0
• [DAFFODIL-2435] Prepare for 3.1.0
• [DAFFODIL-2436] Ensure Daffodil works with newer versions of Java
• [DAFFODIL-2440] TDML Runner must provide root namespace to processor
• [DAFFODIL-2442] daffodil-site link to DFDL specification is to very obsolete old one
• [DAFFODIL-2453] Debugger: support for displaying variable information
• [DAFFODIL-2454] Eclipse - Not able to Compile or Daffodil/stage daffodil 3.0.0
• [DAFFODIL-2463] Update Daffodil Incubator Status file/page
• [DAFFODIL-2464] Document Apache Daffodil Maturity Model Assessment
• [DAFFODIL-2465] Remove requirement of DAFFODIL_HOME for eclipse setup
• [DAFFODIL-2467] Bump copyright years to 2021
• [DAFFODIL-2469] Need to update unsupported features page
• [DAFFODIL-2470] Remove instances of incubating
• [DAFFODIL-2472] Add .asf.yml file to configure daffodil and daffodil-site github
• [DAFFODIL-2474] how to deal with Control chars and newlines in pattern.
• [DAFFODIL-2477] –validate option requires parameter
• [DAFFODIL-2484] Upgrade CI to Java 16
• [DAFFODIL-2491] Allow custom Debuggers through API
• [DAFFODIL-2495] Need way to dry-run release candidates from repos/branches other than apache/daffodil@main
• [DAFFODIL-2503] Add back missing unit tests for Coroutines library
• [DAFFODIL-2506] Remove bintray.com
• [DAFFODIL-2514] Prepare for 3.1.0 Release

Known Issues

A number of issues remain open that have been marked as critical, and are expected to be fixed in the next release. These issues are:

• [DAFFODIL-1422] disallow doctype decls in all XML & XSD that we read in
• [DAFFODIL-2400] New SAX API causes performance degradations
• [DAFFODIL-2512] Unordered sequences with initiated content or discriminators does not parse correctly

Deprecation/Compatibility

The use of the ParseResult.isAtEnd() function has been deprecated, because it can cause unexpected blocking in some cases. Instead, the final bit position should be accessed via the ParseResult.bitPos1b() function, and the result compared with the expected data length. Alternatively, a new InputSourceDataInputStream.hasData() function is added, which blocks until at least 1 byte of data is available to parse, or end-of-file is reached.

• [DAFFODIL-2502] Parse must behave properly for reading data from TCP sockets

Dependency Changes

The following dependencies have been added or updated

Core

• FasterXML Jackson Core 2.12.3 (update)
• FasterXML Woodstox Core 6.2.6 (update)
• ICU4J 69.1 (update)
• Xerces 2.12.1 (update)

Command Line Interface

• JLine 3.20.0 (update)
• Jansi 2.3.2 (update)
• Scallop 4.0.2 (update)

Code Generator

• OS-Lib 0.7.6 (new)

Schematron Validator

• Lightbend Config 1.4.1 (new)
• Saxon-HE 10.5 (new)

Test

• JUnit 4.13.2 (update)
• ScalaCheck 1.15.4 (update)