Chapter 6. Cobol2SummerBatch

6.1. BLU AGE II 5.2.1 for WCB Cobol2SummerBatch Release Note

Global Notes:

  • 5.X products are parts of the “Blu Age II – Legacy to cloud” product family, built on Kepler (Eclipse 4.3);

  • Product documentation is available at http://wikimod.bluage.com/mediawiki/index.php/Main_Page ;

    • Starting with version 5.0, a new license management has been introduced, requiring the use of a new License Server (shipped separately – see corresponding documentation for details);

    • 5.X versions rely on the launch configurations mechanism introduced with the 4.X product series in replacement of the previous workflow mechanism. Migrating from the 3.X workflow to the 5.X launch configuration is described in a dedicated guide.

6.1.1. SummerBatch

6.1.1.1. Improvements

[Note]Note

This version of Cobol2SummerBatch introduces new features that require the version 1.1.0 of SummerBatch (which is the default version for newly created BluAge projects with this product). This new release of SummerBatch is available on nuget (see https://www.nuget.org/packages/SummerBatch/); The corresponding source code can be browsed at https://github.com/SummerBatch/SummerBatch.

6.1.1.1.1. Support for file comparison in the FileUtilsTasklet

The FileUtilsTasklet tasklet in SummerBatch provides a new mode : "Compare", that can be used to perform either text based or binary file comparison;

To provide access to these new facility, some additional elements have been introduced in the BluAge batch UML profile :

FileOperationMode.COMPARE: this new litteral has been added to the FileOperationMode enumeration. This new value can be picked to populate the mode tagged value of the "FILE_OPERATION_STEP" stereotype (see the Usage paragraph below).

Figure 6.1. FileOperationMode.COMPARE

FileOperationMode.COMPARE


FileComparisonMode enumeration: this enumeration lists the supported comparison mode for the FileUtilsTasklet;

  • TEXT : for text-based comparison
  • BINARY : for binary comparison

Figure 6.2. FileComparisonMode enumeration

FileComparisonMode enumeration


EqualityComparerType enumeration : this enumeration lists the supported text-based comparison modes (only taken into account if TEXT has been selected as FileComparisonMode)

  • DEFAULT : standard text comparison (using SequenceEqual on lines of compared text files)
  • IEBCOMPR_LIKE : similar to z/OS IEBCOMPR comparison

Figure 6.3. EqualityComparerType enumeration

EqualityComparerType enumeration


New tagged values for the "FILE_OPERATION_STEP" stereotype :

  • fileComparisonMode : to be picked in the FileComparisonMode enumeration detailed above; Will only be taken into account if the mode is FileOperationMode#COMPARE;
  • equalityComparer : to be picked in the EqualityComparerType enumeration detailed above; Will only be taken into account if the mode is FileOperationMode#COMPARE and if the fileComparisonMode is FileComparisonMode#TEXT.

Usage

Using the compare mode is similar to other "FILE_OPERATION_STEP" usages : simply create the dedicated steps, using the "FILE_OPERATION_STEP" stereotype:

Figure 6.4. "FILE_OPERATION_STEP" Compare usage : creating the steps

"FILE_OPERATION_STEP" Compare usage : creating the steps


and set the tagged values to suit your needs:

Figure 6.5. "FILE_OPERATION_STEP" Compare usage : default text compare setup

"FILE_OPERATION_STEP" Compare usage : default text compare setup


Figure 6.6. "FILE_OPERATION_STEP" Compare usage : IebCompr like text compare setup

"FILE_OPERATION_STEP" Compare usage : IebCompr like text compare setup


Figure 6.7. "FILE_OPERATION_STEP" Compare usage : binary compare setup

"FILE_OPERATION_STEP" Compare usage : binary compare setup


then use them in the proper job diagram:

Figure 6.8. "FILE_OPERATION_STEP" Compare usage : using the compare steps in the job

"FILE_OPERATION_STEP" Compare usage : using the compare steps in the job


The result of the file comparison is stored in the ExitStatus (the FileUtilsTasklet implements the IStepExecutionListener interface and uses the AfterStep method to return the ExitStatus). If the compared files are equals, the ExitStatus will be ExitStatus.Completed, otherwise the ExitStatus will be custom with "FILESNOTEQUAL" as ExitCode.

Feeding the steps with the proper files to be compared is achieved through the standard Settings.config mechanism; The key to be used is having the form "(Job Name).(Step Name in Job).sources" and the value to be given is a list of two semi-colon separated file paths pointing at the files to be compared.

e.g. : for the sample shown above, here is the corresponding Settings.config file content :

<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
       <add key="BA_COMPARE_FILES.fileCompareTextDefault.sources" value="data/inputs/text1.txt;data/inputs/text2.txt" />
       <add key="BA_COMPARE_FILES.fileCompareTextIebCompr.sources" value="data/inputs/ieb1.txt;data/inputs/ieb2.txt" />
       <add key="BA_COMPARE_FILES.fileCompareBinary.sources" value="data/inputs/bin1.zip;data/inputs/bin2.zip" />
</appSettings>

6.1.1.1.2. Support for PowerShell scripts invocation

In version 1.1.0 of SummerBatch, support for powershell scripts invocation has been introduced, using the Summer.Batch.Core.Step.Tasklet.PowerShellTasklet (and some other related code artifacts). To be able to use that feature, some additions neeeded to be made to the Blu Age batch UML profile.

"SHELL_SCRIPT_INVOCATION_STEP" stereotype has been introduced to be used on steps dedicated to invoke PowerShell scripts:

Figure 6.9. "SHELL_SCRIPT_INVOCATION_STEP" stereotype

"SHELL_SCRIPT_INVOCATION_STEP" stereotype


The usable tagged values are :
  • timeOut: defines in milliseconds the timeout to be applied to the script invocation;
  • timeOutBehaviour: used to set how the step will behave when the timeout limit is reached; The value is to be picked in the new TimeOutBehaviour enumeration :

    Figure 6.10. TimeOutBehaviour Enumeration

    TimeOutBehaviour Enumeration

    • SET_EXIT_STATUS_TO_FAILED: the ExitStatus for the Step will be set ExitStatus.Failed which permits to add some decision handling on whether the running job should be stopped or not.
    • THROW_EXCEPTION: throwing an Exception that will force the running job to stop.

Usage

Define one or several steps holding the "SHELL_SCRIPT_INVOCATION_STEP" stereotype,

Figure 6.11. "SHELL_SCRIPT_INVOCATION_STEP" Steps

"SHELL_SCRIPT_INVOCATION_STEP" Steps


and use them in the proper job diagram:

Figure 6.12. PowerShell Steps usage in job diagram

PowerShell Steps usage in job diagram


[Note]Note

  • Parameters owned by the Step will be used as parameters to be passed to the invoked PowerShell Script; the parameters values can be set at usage time in the job diagram (see the job diagram above : the input pins correspond to the values to be given to the DateTimeNow and scriptFile step parameters). Provided default values -- if any -- will be used if required.

    Given parameters are usable within the invoked PowerShell script; see the sample snippet below :

    [CmdletBinding()]
    Param(
      [Parameter(Mandatory=$True)]
       [string]$DateTimeNow,
    	
       [Parameter(Mandatory=$True)]
       [string]$scriptFile
    )	
    ...

  • Properties owned by the Step will be used as variables to be passed to the invoked PowerShell Script; a default value should be provided.

Specifying the script to be invoked by the PowerShellTasklet and the timeout to be used is achieved by filling the Settings.config file; The keys have the form :

  • (Job Name).(Step Name in Job).SCRIPT: to point at the script file path;
  • (Job Name).(Step Name in Job).TIMEOUT: to set the timeout value (in milliseconds);

E.g. - Here is the sample Settings.config file content for the sample shown above :

<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
  <add key="BA_PS_INVOKE.PS_Invoke.SCRIPT" value="data/scripts/ScriptExitStatus.ps1" />
  <add key="BA_PS_INVOKE.PS_Invoke.TIMEOUT" value="100000" />
  <add key="BA_PS_INVOKE.TO_PS_Invoke.SCRIPT" value="data/scripts/ScriptTimeOut.ps1" />
  <add key="BA_PS_INVOKE.TO_PS_Invoke.TIMEOUT" value="12000" />
</appSettings>

[Warning]Warning

For technical constraints, the ambient transaction needs to be suppressed before invoking the PowerShell script. Therefore, any required transaction support should be explicitely handled directly by the PowerShell script.

6.1.1.1.3. Support for multiple (unlimited) outputs for the SortTasklet

The Summer.Batch.Extra.Sort.SortTasklet used by "SORT_STEP" sterotyped steps is now able to handle multiple output files (vs. only one output file for SummerBatch version 1.0.0);

A new tagged value, named 'outfil' has been introduced in the "SORT_STEP" stereotype. This tagged value has to be filled with relevant information to activate the multiple outputs facility. Its content has to be a semi-colons separated list of configurations.

Each configuration is a comma-separated list of Commands, specified by the following syntax :"<COMMAND>=(<PARAMETERS>)" where COMMAND is the command name and PARAMETERS, its parameters.

The outfil tagged value support three commands: INCLUDE and OMIT to filter records, and OUTREC to format them. Each command support the same syntax as described in the SummerBatch documentation related section.

[Note]Note

If both the 'outrec' and 'outfil' tagged values are set for a given "SORT_STEP", the 'outrec' value will be ignored when generating code artifacts.

Usage

Stereotype a step with the "SORT_STEP" stereotype, feeding the 'outfil' tagged value with the convenient configurations:

Figure 6.13. "SORT_STEP" Step specification sample

"SORT_STEP" Step specification sample

and use it in the proper job diagram :

Figure 6.14. "SORT_STEP" Step in job diagram

"SORT_STEP" Step in job diagram


To match the given 'outfil' value, the outputs specification has to be given as a semi-colon separated list of file paths, in the Settings.config file.

<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
...
  <add key="BA_SORT_FILE.sort18.inputFiles" value="data/inputs/sort18.txt" />
  <add key="BA_SORT_FILE.sort18.outputFile" value="data/outputs/sort18-1.txt;data/outputs/sort18-2.txt;data/outputs/sort18-3.txt" />
</appSettings>

[Warning]Warning

The number of given output files has to match the number of defined configurations in the 'outfil' value. Failing to do so will raise an early error at run time, preventing the proper job execution.

6.1.1.1.4. Support for constants modelization in service interfaces and business objects
  • When modeling a service interface modelized properties with a default value will be generated as constants in the corresponding code artifact.

  • When modeling a business object modelized properties with a default value and holding the "constant" stereotype will be generated as constants in the corresponding code artifact.

e.g. : modelized constants in a service interface

Figure 6.15. Modelize constants in Service interfaces

Modelize constants in Service interfaces


and here is the corresponding generated C# code snippet :

...
namespace Com.Netfective.Bluage.Services.CompositeEbcdicReader
{
    /// <summary>
    /// Service ServiceReadEbcdic
    /// </summary>
    public class ServiceReadEbcdic : AbstractService
    {
        private const string testConstant = "This is a test constant";
        private const int testConstantNum = 42;
        ...

e.g. : modelized constants in a business object

Figure 6.16. Modelize constants in a business object

Modelize constants in a business object


and here is the corresponding generated C# code snippet :

...
using Com.Netfective.Bluage.Common.Tags;
using System;

namespace Com.Netfective.Bluage.Business.Batch.Datasourcewriter.Bo
{
    /// <summary>
    /// Entity DatasourceWriterBO.
    /// </summary>
	[Tag("DataAccess")]	
    [Serializable]
    public class DatasourceWriterBO
    {
    ...
        /// <summary>
        /// Constant Property DefEmailAddress.
        /// </summary>
        public const string DefEmailAddress =  "user@mycompany.com";
        ...

6.1.1.1.5. Switching to public and protected virtual methods

Setting the useVirtualMethods property to true for the com.netfective.bluage.dotnet.batch BSP in the BluAge launch configuration will ensure that all public and protected methods will also be marked as virtual (whenever possible), permitting to use performance auditing processes relying on code instrumentation mechanisms.

[Note]Note

The useVirtualMethods property defaults to true.

6.1.1.1.6. Ability to tag generated code using the Custom Tag attribute

The Tag custom C# attribute will be applied to any generated C# artifact code provided the originating UML node has been properly tagged using the "UnityTag" stereotype (in the BluAge service UML profile).

The "UnityTag" holds a single tagged value, named "value" that contains a free-form String, used to identify the "domain of interest" for a given element. The content of the "value" tagged value will be used "as-is" in the generated code.

e.g. : tagging the "EmployeeService" with the "UnityTag" stereotype, valued with "BatchService" string

Figure 6.17. UnityTag stereotype usage sample

UnityTag stereotype usage sample


the corresponding C# code artifact :
...
namespace Com.Netfective.Bluage.Services.Employee
{
    /// <summary>
    /// Service EmployeeService
    /// </summary>
	[Tag("BatchService")]	
    public class EmployeeService : AbstractService
    {...
6.1.1.1.7. Logger delegation

Calls to logger are now delegated to a custom logger mechanism (see generated Log.cs and LogManager.cs classes); The loggerFactory, loggerImport and logger properties for the com.netfective.bluage.dotnet.batch BSP in the BluAge launch configuration can be used to customize the underlying used logger framework and the way the loggers are being named in the C# generated code.

Current default values are:

  • loggerFactory: NLog.LogManager
  • loggerImport: NLog.Logger
  • logger: Logger

6.1.1.2. Fixes

The following issues have been fixed in this version

  • PLTC-3264: Badly formed generated documentation for ebcdic write operation.

  • PLTC-3272: Duplicate declaration of a shared dependency amongst pre/post-processors in an ExecutionListener.

  • PLTC-3273: Unsupported scalar return type for "sql_operation".

6.2. BLU AGE II 5.2.2 for WCB Cobol2SummerBatch Release Note

Global Notes:

  • 5.X products are parts of the “Blu Age II – Legacy to cloud” product family, built on Kepler (Eclipse 4.3);

  • Product documentation is available at http://wikimod.bluage.com/mediawiki/index.php/Main_Page ;

    • Starting with version 5.0, a new license management has been introduced, requiring the use of a new License Server (shipped separately – see corresponding documentation for details);

    • 5.X versions rely on the launch configurations mechanism introduced with the 4.X product series in replacement of the previous workflow mechanism. Migrating from the 3.X workflow to the 5.X launch configuration is described in a dedicated guide.

6.2.1. SummerBatch

6.2.1.1. Framework changes

This version of Cobol2SummerBatch use the version 1.1.2 of SummerBatch (which is the default version for newly created BluAge projects with this product). This new release of SummerBatch is available on nuget (see https://www.nuget.org/packages/SummerBatch/); The corresponding source code can be browsed at https://github.com/SummerBatch/SummerBatch.

The following issues have been fixed by this new framework version:

  • PLTC-3569 : Some instances where different writers on the same file did not work are now fixed.

  • PLTC-3574 : Some instances where a template writer produced empty files are now fixed.

  • PLTC-3583 : Some instances where a template writer wrote on top of the file instead of the bottom are now fixed.

6.2.1.2. Fixes

The following issues have been fixed in this version:

  • PLTC-3544: An incorrect using clause was removed from the template writers.

  • PLTC-3559: The decimal type has been added.

  • PLTC-3563: Some missing writer operations were added in the configuration.

  • PLTC-3701: Lists of VOs are now correctly generated.

  • PLTC-3704: ".equals" has been replaced with ".Equals".

  • PLTC-3801: Duplicate entries in CSPROJ files have been removed.

6.2.1.3. Improvements

The following improvements have been added in this version:

  • PLTC-3582 : Services can now have attributes.

  • PLTC-3588 : Collections can now be used in Sql Query parameters.

  • PLTC-3593 : Operations with <<template_element>> stereotype can now be used to supply custom implementation.

6.3. BLU AGE II 5.2.3 for WCB Cobol2SummerBatch Release Note

Global Notes:

  • 5.X products are parts of the “Blu Age II – Legacy to cloud” product family, built on Kepler (Eclipse 4.3);

  • Product documentation is available at http://wikimod.bluage.com/mediawiki/index.php/Main_Page ;

    • Starting with version 5.0, a new license management has been introduced, requiring the use of a new License Server (shipped separately – see corresponding documentation for details);

    • 5.X versions rely on the launch configurations mechanism introduced with the 4.X product series in replacement of the previous workflow mechanism. Migrating from the 3.X workflow to the 5.X launch configuration is described in a dedicated guide.

6.3.1. SummerBatch

6.3.1.1. Framework changes

This version of Cobol2SummerBatch use the version 1.1.3.2 of SummerBatch (which is the default version for newly created BluAge projects with this product). This new release of SummerBatch is available on nuget (see https://www.nuget.org/packages/SummerBatch/); The corresponding source code can be browsed at https://github.com/SummerBatch/SummerBatch.

The following issues have been fixed by this new framework version:

  • PLTC-3820 : GDG versions in Summer Batch now start with version 1 instead of version 0.

  • PLTC-3830 : Some instances where holds on a created file were not correctly released are now fixed.

  • PLTC-4032 : Summer Batch now correctly write packed decimals with trailing zeroes.

The following issue is expected in an upcoming framework version which will be compatible with this product:

  • Stable sort : An implementation of a stable sort can be used (a flag is used to switch between this mode and the initial unstable one). In this stable sort, the order of equal rows will be retained. A tradeoff might be a loss of performance.

6.3.1.2. Fixes

The following issues have been fixed in this version:

  • PLTC-3845: The Settings.config file now contains the tags for SQLScriptWriters.

  • PLTC-3869: In business objects, the generic Lists are now correctly generated.

  • PLTC-3970: Some Java constructs generated in the .NET code were removed.

  • PLTC-4096: Some instances where the tags for the EBCDIC readers and writers missed in the Settings.config file are now fixed.

  • PLTC-4130: In FTP_GET_STEPs, autoCreateLocalDirectory and deleteLocalFiles options do not give errors anymore.

  • PLTC-4131: In FTP_GET_STEPs, downloadFileAttempts option is now taken into account.

6.3.1.3. Improvements

The following improvement has been added in this version:

  • PLTC-3983 : An <<import>> stereotype can now be put on interfaces to specifiy using clauses.

  • PLTC-4261 : A stableSort tag was added to the <<SORT_STEP>> stereotype, for use with the upcoming stable sort evolution in the Summer Batch framework.

6.3.2. COBOL reverse

6.3.2.1. Fixes

The following issue has been fixed in this version:

  • PLTC-3899: NullPointerExceptions while transmodeling some files were fixed.

6.4. BLU AGE II 5.2.4 for WCB Cobol2SummerBatch Release Note

Global Notes:

  • 5.X products are parts of the “Blu Age II – Legacy to cloud” product family, built on Kepler (Eclipse 4.3);

  • Product documentation is available at http://wikimod.bluage.com/mediawiki/index.php/Main_Page ;

    • Starting with version 5.0, a new license management has been introduced, requiring the use of a new License Server (shipped separately – see corresponding documentation for details);

    • 5.X versions rely on the launch configurations mechanism introduced with the 4.X product series in replacement of the previous workflow mechanism. Migrating from the 3.X workflow to the 5.X launch configuration is described in a dedicated guide.

6.4.1. SummerBatch

6.4.1.1. Improvements

The following improvement has been added in this version:

  • PLTC-4363 : A conditional job branch directly leading to a final node is now generated as a <fail> tag.

6.4.2. COBOL reverse

6.4.2.1. Fixes

The following issue has been fixed in this version:

  • PLTC-4300: A trim is now made when comparing for Level 88 values, so that the test is accurate even if the literal has a different length.