PilotFish Software Release Notes –
New Features & Improvements

Release 18R2 – Aug 2018

• New Apache Kafka Listener and Transport
  • Apache Kafka is a distributed streaming system which is commonly used as a fault-tolerant, high-throughput/low-latency enterprise messaging system
  • We’ve added both a Kafka Listener and Transport that allow consuming and publishing messages on Kafka architectures

• New AS2 Listener and Transport
  • AS2 is a secure point-to-point transmission protocol often used for transporting EDI Messages

• New UDP Listener and Transport
  • UDP is a core Internet protocol that uses stateless datagrams to transmit data.
  • Given is minimalistic guarantees, UDP has a low amount of overhead and latency making it better suited for some use-cases

• New Splunk® Support
  • Splunk is an output aggregator for enterprises, it allows for analysis and insight from machine-generated data and logs
  • We’ve added several modules to make integrating with Splunk simple and straightforward:
    • Splunk Search Listener – Execute Splunk search queries and process the output
    • Splunk Transport – Post transaction data as event data to Splunk
    • Splunk Processor – Send events to Splunk at specified stages of interface progression
    • Splunk Log Appender – A Log4J appender configuration provided in the logging configuration file

• New Microsoft Azure Storage Processor
  • We’ve expanded our cloud provider support and now include a Microsoft Azure Storage Processor to work with files and blobs storage on Azure

• Added Regression System
  • Developed behind the scenes for months, we’re releasing our Regression System which allows for recording of data flows and replay on future releases to reduce regressions. The GUI has been integrated into our eiDashboard product to allow for convenient operation.

• Added Capability to set Metadata on Sources and Targets along with the ability to search for them
  • The ability to add relevant metadata to formats and routes has been in the product for some time, in 18R2 we are expanding this capability in two ways:
    • Adding the ability to enter metadata on Sources and Targets
    • Adding the ability to search/filter on this metadata

• Added support for additional Encryption Algorithms for Encryption Processors
  • AES-192, AES-256 and TwoFish as selectable algorithms (Note: Requires full-security installer, contact us if you need more information)

• Added OGNL Member Access Security Support
  • OGNL is a powerful expression language that can be used for specifying dynamic configuration values
  • We’ve added the ability to enable or restrict specific packages, classes and/or methods via an ognlSettings.xml file for tailored security settings

• Added XSLT 2.0/3.0/3.1 items to Data Mapper palette
  • As a follow up to the inclusion of Saxon PE in 18R1, we’ve gone further to add XSLT 2.0-3.1 functions and constructs to the Data Mapper tools palette

• Added SSL Configurations to better support self-signed certificates and hostname validation options
  • Often while building interfaces in a dev environment, HTTPS/SSL endpoints will use non-commercial, self-signed certificates. To remove the need to inject these temporary certificates into a custom trustStore we’ve added configuration options that can relax the SSL connection validations.

• Added the ability to run Transformation Modules as Processors
  • When designing an interface, occasionally there will be a need to run a native transformation module mid-stream, for example, JSON to XML or vice versa. The traditional approach to do this would be to add a Call Route Processor to a separate route with a dedicated format. However,  as the number of interfaces scale, these types of single-purpose routes can distract from the overall flow.
  • In order to simplify interface design and construction, we’ve added the ability to call transformation modules directly as processors.

• Added the ability to use Route Callouts within XSLT for dynamic value evaluation
  • Traditional code value lookups in XSLT are done with hard-coded data mapping templates, i.e., code ‘2’ signifies ‘Personal Email Address’
  • However, in the real world, keys and values are often dynamic or updated from time to time. This non-fixed nature requires additional effort to keep code value mappings up to date.
  • To address this, we’ve added the ability to call routes in order to be able to determine values dynamically.
  • By using routes in this way, we can take full advantage of the eiPlatform’s connectivity options and capabilities to lookup values from databases, web services or any other endpoint the eiPlatform supports.

• Improved EDI Support, Functionality and Performance
  • The EDI Transformer is now more capable of parsing EDI that contains invalid or out-of-place segments
  • We’ve improved version recognition for older 4010 EDI and X12 minor releases
  • Errors that occur while transforming EDI to XML now provide better segment and element context to help debugging
  • Numerous enhancements to parsing performance, especially when transforming EDI to XML and when X12 table data is not available

• Improved XML to JSON Conversion Performance
  • After carefully reviewing how PilotFish converts XML contents to JSON, we were able to reduce the number of computationally-expensive operations further to improve conversion performance

• Improved NCPDP Support
  • NCPDP schemas could be previously read into PilotFish’s Data Mapper using the generic XSD Reader, however, in practice there was some confusion over which file needed to be read in. To address this we’ve added a dedicated format reader for NCPDP that will check the file selected, and if not the correct one, the software will attempt to identify the right one to use.

• Improved MongoDB Support
  • We’ve updated our support for MongoDB with a couple of changes:
    • Updated the Mongo libraries to support the latest MongoDB release
    • Added the ability to connect to MongoDB instances that do not require authentication
    • Added support for empty filters

• Improved JavaScript Processor Functionality
  • Several improvements to the JavaScript Processor have been made in 18R2:
    • JavaScript files can now be stored in the route directory, allowing them to be more easily deployed with other configurations
    • More control over injecting the contents of Transaction Data into the script
    • Ability to read and write Transaction Attributes in the JavaScript file
    • A built-in code editor for on-the-fly editing of the JavaScript content

• Improved eiConsole Look and Feel
  • The eiConsole has been updated to a newer Look and Feel that is more responsive and memory-efficient
  • Visual differences are minimal as efforts were undertaken to give a consistent experience

• Improved eiConsole Menu Bar Layout
  • Due in part to its long history, the eiConsole’s menu bar had accumulated many sub-items under just a couple of top-level menus.
  • With 18R2 we’ve arranged menu items more consistent with common expectations by utilizing sub-menus and better grouping related items

• Improved Syntax Highlighting in Stage Output Viewer of Testing and Debug Modes
  • Drop-down to add the ability to toggle between no syntax, XML and JSON
  • XML and JSON now have collapsible code segments

• Improved support for Character Encodings in XPath Value Retrieval Processor
  • Functionality and configuration options have been added to the XPath Value Retrieval Processor that will allow for:
    • Detection and use of declared character encoding in XML contents
    • Override character encoding declared in XML

• Improved Logging Performance and Capabilities
  • One of the underlying changes to eiPlatform is the switch from Log4j to Log4j2
  • Log4j2 provides much better logging performance and enables more functionality

• Improved XPath Processors by switching underlying engine
  • The XPath processors now use Saxon which provides XPath 2.0 & 3.1 support and enhances performance

• Improved Registration Workflow for White-Labeled Installers
  • We’ve refined our registration process for white-labeled installers by reducing the steps necessary for users to go through manually

• Improved Directory Transport performance for High File Volumes with Dynamic Naming
  • When Directory Transport is used with the ‘If File Exists’ setting, it is set to ‘Create New’.  It attempts to find the first non-conflicting filename by incrementing a counter
  • On interfaces with a very large number of files being created and dynamically named, it was possible to incur delays due to the conflict checking
  • As of 18R2, a new ‘File Name Conflict Pattern’ option was added in the Advanced tab which uses SimpleDateFormat patterns and defaults to ‘yyyyMMddHHmmss’ which should minimize the amount of conflict resolution operations necessary

• Improved XSLT Transformation Performance
  • We’ve been able to improve XSLT transformation performance by utilizing more of the XSLT engine’s specific functionality

• Improved Basic Statistics Functionality
  • Basic statistics are persisted to respond to REST API calls, storage of these statistics has been changed to a lower-overhead format

• Updated Socrata Open Data Transport for API changes
  • The Socrata Open Data Transport has been updated to work with the current API

• Updated ACORD Life And Annuity to Version 2.40.00
  • The ACORD Life and Annuity artifacts have been updated to the latest version (2.40.00) for the eiConsole for ACORD (available from ACORD.org)

• Updated 3rd-Party Dependencies
  • Several supporting libraries have been updated to include upstream fixes