Programming guidelines

Key objectives of batch programming are: reliability, performance ,and maintainability.

Performance is dependent on a large set of factors, but some guidelines should be kept in mind :

• Shorten the path between data and data processor, Reducing network overhead can have a significant positive impact on global batch performance when dealing with databases.

• For critical performance needs, using parallel processing facility, which requires some extra work to be done (smart data partitioning).

• Carefully specify chunk size :

  • A small chunk size will lower global batch performance by adding a lot of checkpoints operations;

  • A very large chunk size will increase global batch performance but

    • could have a significant negative impact on memory consumption;

    • could involve some serious perturbations if data are being consumed by other appliances, in case of a batch failure (as rollback will deal with a large set of data).

• Hunt for unnecessary operations in code; In particular, be careful with logger calls that can degrade overall performance when used on a granularity basis.

• Follow general SQL statement performance guidelines. Some examples:

  • Make sure SQL queries are performing well, and that all database indexes have been created;

  • Pay attention to the order of filtering ( WHERE clause) by inspecting execution plan, to ensure the most discriminant filters are invoked first.

  • Hunt and remove unnecessary outer joins.

  • Only select fields that are really needed by processing.

• Keep I/O consumption profile low;

Some additional suggestions for maintainability and reliability:

Keep it simple. Have some intricate steps sequences within a given job can make code hard to maintain and harder to test.

Important

This reference guide makes the assumption that you know Visual Studio and creating projects and solutions. However, to help users set up their first Summer Batch project, we wrote Getting Started guide for SummerBatch and SummerBatchCore. Please read this tutorial prior to starting this reference documentation.