Reading and writing EBCDIC files using COBOL copybooks

COBOL batches frequently involve dealing with EBCDIC files using copybooks. Modernizing these batches requires the ability to read and write to these files. Two classes have been developed to support EBCDIC Read/Write:

Summer.Batch.Extra.Ebcdic.EbcdicFileReader: to read from an EBCDIC file;
Summer.Batch.Extra.Ebcdic.EbcdicFileWriter: to write to an EBCDIC file;

Both classes require an XML version of a COBOL copybook provided at run time.

Using the `EbcdicFileReader`¶

To read records from an EBCDIC file, using an EbcdicFileReader, the following elements must be provided :

an XML version of the needed COBOL copybook (Copybook property);
a class in charge of transforming EBCDIC record into a business object (EbcdicReaderMapper property): this class must implement Summer.Batch.Extra.Copybook.IEbcdicReaderMapper<T> interface ;
a path to EBCDIC file (Resource property).

Caution

The two elements (copybook XML file and EBCDIC reader mapper class) are mandatory.

The COBOL copybook is used by reader to extract records from the EBCDIC file.

Example 7.1. Sample XML copybook export¶

<?xml version="1.0" encoding="ASCII"?>
<FileFormat ConversionTable="IBM037" dataFileImplementation="IBM i or z System" 
    distinguishFieldSize="0" newLineSize="0" headerSize="0">
  <RecordFormat cobolRecordName="BA_EBCDIC_READER" distinguishFieldValue="0">
    <FieldFormat Decimal="0" DependingOn="" ImpliedDecimal="true" Name="CODE" Occurs="1" 
        Picture="S9(5)" Signed="true" Size="5" Type="3" Value=""/>
    <FieldFormat Decimal="0" DependingOn="" ImpliedDecimal="true" Name="NAME" Occurs="1" 
        Picture="X(30)" Signed="false" Size="30" Type="X" Value=""/>
    <FieldFormat Decimal="0" DependingOn="" ImpliedDecimal="true" Name="DESCRIPTION" Occurs="1" 
        Picture="X(40)" Signed="false" Size="40" Type="X" Value=""/>
    <FieldFormat Decimal="0" DependingOn="" ImpliedDecimal="true" Name="DATE" Occurs="1" 
        Picture="S9(8)" Signed="true" Size="4" Type="B" Value=""/>
  </RecordFormat>
</FileFormat>

Note

ConversionTable (=encoding) is specified in the copybook XML file: this attribute must have a property that can be understood by the C# API. This requires using the C# encoding names (which differ from java encoding names convention, for example, java uses "Cp037" where C# uses "IBM037"). System.Text.Encoding has GetEncodings method to list all available encodings.

The XML copybook file must be compliant with XSD (full XSD source is given in dedicated appendix section)

Figure 7.1. EBCDIC File Format XML schema¶

EbcdicFileFormat_xml_schema

The corresponding bound C# classes are located in Summer.Batch.Extra.Copybook namespace:

CopybookElement.cs
FieldFormat.cs
FieldsGroup.cs
FileFormat.cs
IFieldsList.cs
RecordFormat.cs

Afterward the EBCDIC reader mapper is used to transform the records into business objects. Below is a business object whose properties will be mapped to the EBCDIC record described by the XML copybook above.

Example 7.2. Sample business object to which EBCDIC records will be mapped¶

using System;

namespace Com.Netfective.Bluage.Business.Batch.Ebcdic.Bos
{
    /// <summary>
    /// Entity EbcdicFileBO.
    /// </summary>
    [Serializable]
    public class EbcdicFileBO
    {
        /// <summary>
        /// Property Code.
        /// </summary>
        public int? Code { get; set; }

        /// <summary>
        /// Property Name.
        /// </summary>
        public string Name { get; set; }

        /// <summary>
        /// Property Description.
        /// </summary>
        public string Description { get; set; }

        /// <summary>
        /// Property Date.
        /// </summary>
        public DateTime? Date { get; set; }

    }
}

Now, Define a mapper in charge of transforming records into business objects. This mapper must implement Summer.Batch.Extra.Ebcdic.IEbcdicReaderMapper<T> interface. Summer.Batch.Extra.Ebcdic.AbstractEbcdicReaderMapper<T> is super class to inherit from in order to craft EBCDIC reader mapper quickly (see example below).

Example 7.3. Sample EBCDIC reader mapper¶

using Com.Netfective.Bluage.Business.Batch.Ebcdic.Bos;
using Summer.Batch.Extra.Ebcdic;
using System.Collections.Generic;

namespace Com.Netfective.Bluage.Business.Batch.Ebcdic.Bos.Mappers
{
    ///<summary>
    /// Ebcdic mapper for the EbcdicFileBO class.
    ///</summary>
    public class EbcdicFileEbcdicMapper : AbstractEbcdicReaderMapper<EbcdicFileBO>
    {
        private const int Code = 0;
        private const int Name = 1;
        private const int Description = 2;
        private const int Date = 3;

        ///<summary>
        /// Map a collection of properties to a EbcdicFileBO record. 
        ///</summary>
        /// <param name="values"> list of values to be mapped</param>
        /// <param name="itemCount"> item count; will be used as identifier 
        /// if this makes sense for the target class.</param>
        /// <returns>the EbcdicFileBO record build upon the given list of values.</returns>
        public override EbcdicFileBO Map(IList<object> values, int itemCount)
        {
            var record = new EbcdicFileBO
            {
                Code = (int) ((decimal) values[Code]),
                Name = ((string) values[Name]),
                Description = ((string) values[Description]),
                Date = ParseDate(values[Date]),
            };
            return record;
        }
    }
}

The EbcdicFileReader is then declared as any other reader in the job XML file :

Example 7.4. `EbcdicFileReader` declaration in the job XML file¶

<step id="EbcdicReader">
    <chunk item-count="1000">
        <reader ref="EbcdicReader/EbcdicFileReader" />
        …
    </chunk>
</step>

And here is the corresponding Unity configuration part :

Example 7.5. `EbcdicFileReader` Unity configuration¶

…
/// <summary>
/// Registers the artifacts required for step EbcdicReader.
/// </summary>
/// <param name="container">the unity container to use for registrations</param>
private void RegisterEbcdicReader(IUnityContainer container)
{
    // Reader - EbcdicReader/EbcdicFileReader
    container.StepScopeRegistration<IItemReader<EbcdicFileBO>, EbcdicFileReader<EbcdicFileBO>>
        ("EbcdicReader/EbcdicFileReader")
        .Property("Resource").Resource("#{settings['BA_EBCDIC_READER.EbcdicReader.FILENAME_IN']}")
        .Property("Copybook").Resource("#{settings['BA_EBCDIC_READER.EbcdicReader.COPYBOOK_IN']}")
        .Property("EbcdicReaderMapper")
            .Reference<IEbcdicReaderMapper<EbcdicFileBO>>("EbcdicReader/EbcdicFileReader/Mapper")
        .Register();
…

Note

Please note:

EBCDIC file reader is registered within a step scope, using the StepScopeRegistration extension;
EBCDIC file reader registration is done using the same name as it was declared in the job XML file (that is "EbcdicReader/EbcdicFileReader");

The

Property("Resource")
    .Resource("#{settings['BA_EBCDIC_READER.EbcdicReader.FILENAME_IN']}")

call indicates that the property named "Resource" will be filled with value read in Settings.config XML file. Here is the corresponding Settings.config file content :

<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
  <add key="BA_EBCDIC_READER.EbcdicReader.FILENAME_IN"
    value="data\inputs\BA_EBCDIC_READER.data" />
  <add key="BA_EBCDIC_READER.EbcdicReader.COPYBOOK_IN"
    value="data\copybooks\BA_EBCDIC_READER.fileformat" />
</appSettings>

Using the `EbcdicFileWriter`¶

To write records to an EBCDIC file using an EbcdicFileWriter, the following elements should be provided:

list of COBOL copybooks XML versions (Copybooks property);
class in charge of transforming EBCDIC record into a business object (EbcdicWriterMapper property): one can use the Summer.Batch.Extra.Ebcdic.EbcdicWriterMapper class or a custom sub-class inherited from it;
path to the targeted EBCDIC file (Resource property).

Caution

Writer takes a list of XML copybooks but only uses one copybook at a time; writer has a method (ChangeCopyBook) to change the current copybook being used.

These three elements (copybook XML files list, EBCDIC writer mapper class and path to targeted resource) are mandatory.

Let's review the corresponding configurations. The writer must be declared in the job XML file (as any other writer):

Example 7.6. `EbcdicFileWriter` declaration in the job XML file¶

<step id="EBCDIC_WRITER">
    <chunk item-count="1000">
        …
        <writer ref="EBCDIC_WRITER/EbcdicFileWriter" />
    </chunk>
</step>

Unity configuration:

Example 7.7. `EbcdicFileWriter` Unity configuration¶

…
/// <summary>
/// Registers the artifacts required for step EBCDIC_WRITER.
/// </summary>
/// <param name="container">the unity container to use for registrations</param>
private void RegisterEBCDIC_WRITER(IUnityContainer container)
{
   …

    // Writer - EBCDIC_WRITER/EbcdicFileWriter
    container
       .StepScopeRegistration<IItemWriter<EbcdicFileBO>, EbcdicFileWriter<EbcdicFileBO>>("EBCDIC_WRITER/EbcdicFileWriter")
       .Property("Resource").Resource("#{settings['BA_EBCDIC_WRITER.EBCDIC_WRITER.EbcdicFileWriter.FILENAME_OUT']}")
       .Property("Copybooks").Resources("#{settings['BA_EBCDIC_WRITER.EBCDIC_WRITER.EbcdicFileWriter.COPYBOOK_OUT']}")
       .Property("EbcdicWriterMapper").Reference<EbcdicWriterMapper>("EBCDIC_WRITER/EbcdicFileWriter/Mapper")
       .Register();

    // EBCDIC writer mapper for EBCDIC_WRITER/EbcdicFileWriter
    container
       .StepScopeRegistration<EbcdicWriterMapper>("EBCDIC_WRITER/EbcdicFileWriter/Mapper")
       .Register();

…

Note

EBCDIC file writer is registered within a step scope, using the StepScopeRegistration extension;
EBCDIC file writer registration is done using the same name as it was declared in the job XML file (that is "EBCDIC_WRITER/EbcdicFileWriter");
Regarding the list of copybooks resources loading :
```
.Property("Copybooks").Resources("#{settings[…]}")
```
.Resources() call (note the extra "s") is able to load a list of resource paths into a semicolon-separated path string.
Summer.Batch.Extra.Ebcdic.EbcdicWriterMapper automatically converts a business object into a list of values that can be written in an EBCDIC record (Refer to Summer.Batch.Extra.Ebcdic.EbcdicWriterMapper#Map method). The automatic mapping between business object properties and copybook records is made on a name convention basis. The property name must be equal to FieldFormat Name attribute, converted to camel case.

e.g.
```
<FieldFormat Decimal="0" DependingOn="" ImpliedDecimal="true"
    Name="DESCRIPTION" Occurs="1" Picture="X(40)" Signed="false"
    Size="40" Type="X" Value=""/>
```
will automatically map to the property named "Description" (= camel case version of "DESCRIPTION")
```
/// <summary>
/// Property Description.
/// </summary>
public string Description { get; set; }
```
To make sure you are using correct names, in order to guarantee the automatic mapping between records and business object properties, you can use the ToCamelCase method from the Summer.Batch.Extra.Ebcdic.AbstractEbcdicMapper class;

Reading and writing EBCDIC files using COBOL copybooks

Using the EbcdicFileReader¶

Example 7.1. Sample XML copybook export¶

Figure 7.1. EBCDIC File Format XML schema¶

Example 7.2. Sample business object to which EBCDIC records will be mapped¶

Example 7.3. Sample EBCDIC reader mapper¶

Example 7.4. EbcdicFileReader declaration in the job XML file¶

Example 7.5. EbcdicFileReader Unity configuration¶

Using the EbcdicFileWriter¶

Example 7.6. EbcdicFileWriter declaration in the job XML file¶

Example 7.7. EbcdicFileWriter Unity configuration¶

Using the `EbcdicFileReader`¶

Example 7.4. `EbcdicFileReader` declaration in the job XML file¶

Example 7.5. `EbcdicFileReader` Unity configuration¶

Using the `EbcdicFileWriter`¶

Example 7.6. `EbcdicFileWriter` declaration in the job XML file¶

Example 7.7. `EbcdicFileWriter` Unity configuration¶