<dependency>
  <groupId>org.apache.daffodil</groupId>
  <artifactId>daffodil-japi_2.11</artifactId>
  <version>2.1.0</version>
</dependency>

libraryDependencies += "org.apache.daffodil" %% "daffodil-sapi" % "2.1.0"

[apache-daffodil]
name=Apache Daffodil
baseurl=https://apache.jfrog.io/artifactory/daffodil-rpm/
enabled=1
gpgkey=https://downloads.apache.org/daffodil/KEYS
gpgcheck=1
repo_gpgcheck=0

sudo dnf install apache-daffodil

Release Notes

Daffodil 2.1.0 is the first release of Daffodil as an Apache incubator project. This release includes relicensing the source to Apache v2, changing the namespace to org.apache.daffodil, and numerous bug fixes and DFDL feature additions to support more DFDL schemas.

This release contains changes which break compatibility with past releases. Please see the deprecation / compatibility notes section for more information.

Packed Decimal

Added support for binary packed decimal data types. In addition to standard, the dfdl:binaryNumberRep property now also accepts values of bcd, packed, and ibm4690Packed to support Binary Coded Decimal, IBM 390 Packed Decimal, and IBM 4690 Packed Decimal, respectively. The dfdl:binaryDecimalVirtualPoint property applies for the appropriate types. Also implemented are dfdl:binaryPackedSignCodes and dfdl:binaryNumberCheckPolicy to configure the valid nibble values when dfdl:binaryNumberRep is packed.

• [DAFFODIL-1739] Implement 'packed' and 'bcd'

Apache Incubation Support

Multiple changes were made to support the move to the Apache incubator. This includes changes to the source directory structure, relicensing files to Apache v2, changing the project package to org.apache.daffodil, updating the build infrastructure to SBT 1.x, and adding TravisCI continuous integration support.

• [DAFFODIL-1612] sbt - branches without tags cause sbt compile to fail
• [DAFFODIL-1841] daffodil root jar is published but shouldn't be, and its contents are wrong
• [DAFFODIL-1856] Changes necessary for apache incubation
• [DAFFODIL-1857] Update sbt to 1.x, simplifing build config and replacing deprecated syntax
• [DAFFODIL-1858] Add support for TravisCI
• [DAFFODIL-1859] Switch to apache.org namespace and Apache v2 license
• [DAFFODIL-1870] invalid and nonfunctional URLs in source files
• [DAFFODIL-1873] sbt warnings/errors
• [DAFFODIL-1890] Update user related links from the wiki to daffodil.apache.org
• [DAFFODIL-1896] Modify artifact names to include "apache" and "incubating"
• [DAFFODIL-1897] Some Schemas fail to compile with latest updates
• [DAFFODIL-1899] Fix NOTICE and LICENSE files
• [DAFFODIL-1901] Apache Rat cannot detect licenses for Passera, Scala libs, and W3C files
• [DAFFODIL-1906] Updates to LICENSE/NOTICE files

Ambiguous Path Expressions

Previously, ambiguous path expressions (i.e. path expressions that could potentially reference multiple non-array elements of the same name) were forbidden and caused a schema definition error at schema compilation time. As of 2.1.0, such ambiguities are detected at compile time but are now only considered schema definition warnings. However, if a path expression ever evaluates to more than one non-array element at runtime, it will then be considered an error and cause a runtime schema definition error. This change allows for schemas that appear to have ambiguous path expressions, but in only ever evaluate to a single element in practice.

• [DAFFODIL-1773] Choice ambiguous element name results in failed expression
• [DAFFODIL-1854] Remove concept of slots from InfosetImpl.scala
• [DAFFODIL-1860] Minimize the number of HashMap allocations/insertions in the Infoset
• [DAFFODIL-1869] Nato link16 doesn't work on 2.0.0 Daffodil
• [DAFFODIL-1877] Slow schema compilation due to unambiguous checks

TDML Runner Enhancements

Changes were made to the TDML runner to support an embedded defaultConfig option and to verify expected schema definition warnings.

• [DAFFODIL-1583] TDML Runner - expected warnings feature
• [DAFFODIL-1846] TDML Runner - Infoset.contents is stripping off all attributes
• [DAFFODIL-1868] TDML Runner defaultConfig cannot be embedded. Not checked properly.

BitOrder/ByteOrder/Encoding Refactoring

The way bitOrder, byteOrder, and encoding consistency was checked required fairly complicated logic at schema compilation time. Not only was this logic difficult to validate, which led to bugs, it also contributed to slow schema compilation times. This complex logic was removed and replaced with an efficient mechanism for runtime validation. This reduced code complexity, fixerd some bugs, and will facilitate future performance optimizations during schema compilation.

• [DAFFODIL-1001] Invalid bitOrder and byteOrder combination should produce SDE
• [DAFFODIL-1597] Too many ways that encoding, byteOrder, etc. are being setup
• [DAFFODIL-1732] BitOrderChangeParser not inserted correctly for bitmap schema
• [DAFFODIL-1767] byteOrderChangeParser optimization not working
• [DAFFODIL-1843] Unparser bitOrder change and OVC (outputValueCalc) interaction
• [DAFFODIL-1876] Improve performance regression due to FormatInfo changes
• [DAFFODIL-1884] Regression in bitOrder changing

XML Output

The XML infoset output (via the XMLTextInfosetOutputter) now writes an XML declaration <?xml version="1.0" ... ?> at the beginning of output. It also ensures output is flushed at the end of writing, which should prevent issues users reported of receiving only partial data.

• [DAFFODIL-1844] XMLTextInfosetOutputter should output an XML declaration
• [DAFFODIL-1871] XMLTextInfosetOutputter endDocument should flush the writer

Miscellaneous Bug Fixes

• [DAFFODIL-1473] DFDL Schema Validation not happening properly
• [DAFFODIL-1838] Need more warning suppression options
• [DAFFODIL-1851] Unparsing delimited data with escape characters could lead to truncation
• [DAFFODIL-1855] Property Resolution - resolve onto ref objects
• [DAFFODIL-1862] Better logging/reset mechanism for MarkPool
• [DAFFODIL-1864] spurious SDE about all-text schema being non-scannable
• [DAFFODIL-1866] Eclipse classpaths need fixing
• [DAFFODIL-1867] Memory leak in Register
• [DAFFODIL-1872] Eclipse classpaths for daffodil-io module need fixing
• [DAFFODIL-1874] Incorrect warning message about default value and no empty representation
• [DAFFODIL-1885] Initiators are not being inherited from parent choices
• [DAFFODIL-1893] TDML Runner not caching schema compiles - only caches schemas if compileAllTopLevel is true
• [DAFFODIL-1895] Eclipse classpath for daffodil-core needs fix
• [DAFFODIL-1903] Eclipse classpath broken for daffodil-test and other tests - now must have daffodil-lib-unittest
• [DAFFODIL-1907] Daffodil disallows attribute declarations for non-DFDL annotation elements.
• [DAFFODIL-1910] Cannot unparse when array with minOccurs="0" is last in a sequence
• [DAFFODIL-1911] Move daffodil-cli tests to use sbt's "integration" test capabilities
• [DAFFODIL-1913] Strange jar ends up in lib_managed of daffodil - trips up Eclipse IDE users
• [DAFFODIL-1914] Eclipse classpath broken for daffodil-cli
• [DAFFODIL-1920] toString representation of grammar prims and parser/unparsers need improvement
• [DAFFODIL-1921] option to bypass Java 8 check

Deprecation/Compatability

As of 2.1.0, the following changes have been made which affect compatibility with past releases:

Package Organization Change

Now that Daffodil is an Apache incubator project, the package organization has been changed from edu.illinois.ncsa to org.apache.daffodil. Developers using the Daffodil Java and Scala API will need to update their source code and SBT/Maven configurations to use the new organization.

Daffodil Extension daf XML Namespace Change

The Daffodil extension daf XML namespace has been modified to represent the move to Apache, changing from urn:ogf:dfdl:2013:imp:opensource.ncsa.illinois.edu:2012:ext to urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:ext. Backwards compatibility is maintained for schemas using the old daf namespace, though it should be considered deprecated. All future Daffodil extensions will only be supported in the new namespace, so it is recommended that one update schemas to use the new namespace.

Deprecation of built-in-formats.xsd

The built-in-formats.xsd file is a schema that defines a set defaults for a wide array of DFDL properties. This file was intended for use only during testing, but was errantly made available for use in real schemas. This was an oversight, and so use of this schema will now result in a deprecation warning. Instead, one should import/include org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd and set the dfdl:format ref value to GeneralFormat to get a reasonable set of default property values or define all necessary properties within a schema.