The above release files should be verified using the PGP signatures and the
project release KEYS. See
verification instructions
for a description of using the PGP and KEYS files for
verification. A SHA512 checksum is also provided as
an alternative verification method.
Create the file /etc/yum.repos.d/apache-daffodil.repo with the following content:
A new extension is added to support custom DFDL expression functions written in
Java or Scala. To add new functions, the UserDefinedFunctionProvider and
UserDefinedFunction interfaces must be implemented, compiled into a jar, and
added to the classpath. Once defined, the function can then be called just like
a normal DFDL expression function, such as pre:myUserDefinedFunction(args).
For more information on usage, see the User Defined Function
page.
[DAFFODIL-2186] User defined functions for DPath expressions
A new extension is added to support Binary Large Objects. Setting an element
type to xs:anyURI and setting the DFDL property
dfdlx:objectKind="bytes" will cause Daffodil to write the bytes associated
with the element to a file rather than to the infoset. The URI to the file is
stored in the infoset. The function setBlobAttributes is added to the
InfosetOutputter to support changing the directory and name of these BLOB
files.
Add support for unordered sequences via the dfdl:sequenceKind="unordered"
property. See section 14.3 of the DFDL specification for more information on
the behavior of unordered sequences.
[DAFFODIL-918] Unordered sequence cannot have empty content.
[DAFFODIL-1010] Re-enable checks for unordered seq and choice branch violations
[DAFFODIL-1091] NumberFormatException when Unbounded element in unordered sequence
[DAFFODIL-1100] Unordered sequence with dfdl:format value of occursCountKind != "parsed" fails to compile
[DAFFODIL-2261] Optional first element in unordered sequence fails to unparse
Memory Limitations
During parsing, Daffodil stores the data stream in a cache to allow for
backtracking. However, when streaming very large files that might need to
backtrack long distances it is possible that the cache could outgrow the size
of the heap, resulting in out of memory errors. In the majority of these cases,
although possible, a parse would never actually need to backtrack that far, so
we hold on to memory unnecessarily. New parameters are added to the input
stream to support the ability to limit the maximum size of this cache. Daffodil
will throw away old data when this limit is reached, and will only error if
trying to backtrack to the discarded data.
During unparsing, in some circumstances it was possible for Daffodil to cache
unparsed data that could reach a 2GB+ limit and cause an out of memory error.
To prevent this error, each unparse cache will store a maximum amount in memory
(defined by the maxByteArrayOutputStreamBufferSizeInBytes tunable). When
this maximum value is reached, Daffodil will switch to writing to a temporary
file (defined by the tempFilePath tunable). Eventually, the contents of the
file will be written to the unparse data stream in chunks (defined by the
outputStreamChunkSizeInBytes tunable).
With these changes and BLOB support, handling large files, including those
greater than 2GB is possible.
[DAFFODIL-2194] buffered data output stream has a chunk limit of 2GB
TDML
Improvements were made to the TDML runner, including API updates to support use in Java,
and improved output on failures.
[DAFFODIL-2204] unparserTestCase should dump hex data not just iso-8859-1 when mixed text/binary TDML doesn't compare
[DAFFODIL-2209] TDML Improperly handles MAC Style (CR) line endings
[DAFFODIL-2229] TDML Runner Class doesn't have access to simple constructors
Multiple infrastructure changes were made, including support for GitHub Actions
continuous integration, Windows CI tests, a new container based system for
creating releases, and website updates.
[DAFFODIL-2264] Add more automation to the release candidate container
Code Refactoring
Many non-functional changes were made, including improved internal type-safety,
improved separation of runtime objects to support different runtimes in the
future, initial changes to improve schema compilation speed, and other
miscellaneous improvements.