Multi-database Batch Job Servers

Topics:

Overview

A new batch job server design targets a multi-database architecture to improve performance. An independent common distributor server (based on the BIZ server) listens for job messages from each database, then distributes jobs in turn to batch job servers. Each batch job server then handles the job based on the status and keeps track of that job as necessary.

This feature is only available to hosted customers on Azure.

Environment Requirements

  • At least 2 databases/agencies with Azure MSSQL.
  • One set of BIZ Server/CFMX/Web/Indexer.
  • At least 1 batch job server (add more as necessary).
  • 1 batch job distributor (can be BIZ server or any batch job server in a low transaction volume environment).
  • Open port 3445 and 3447 for JBoss JMS messaging communication.



Configuration

  1. Make sure BIZ and batch job server have the same code with the same multi-database configuration.
  2. ServerConfig.properties for BIZ server.
    1. Enable multi-database: multidatasource.enabled=true
    2. Disable batch job distributor: av.batchdistributor.enabled=false
    3. Disable batch job server: av.batchengine.interval=0
  3. ServerConfig.properties for batch job distributor.
    1. Enable multi-database: multidatasource.enabled=true
    2. Enable batch job distributor:
      Note: Any BIZ and Batch Job server can be a batch job distributor. However, only one distributor can be in a deployment.

      av.batchdistributor.enabled=true

       

      av.batchdistributor.interval={number}

      Note: 0 means disable the batch job server, larger than 0 means batch job server enable and checking batch job interval is the number assigned (minimum is 6 seconds). Increase the interval if more batch job servers are in the same deployment to reduce database transaction frequency.

      This value must also be larger than the value for av.batchengine.interval to prevent an error for expired or abandoned jobs.

      batchjobqueue.java.naming.provider.url=http-remoting://BATCH_1_IP:3447,http-remoting://BATCH_2_IP:3447
      Note: In case of change in deployment environment, make sure this points to the correct batch job servers.

      remote.batchjobserver.connectionprovider.create.options.org.xnio.Options.

      SSL_ENABLED=false

       

      remote.batchjobserver.connections=batchjobserver

       

      remote.connection.batchjobserver.connect.options.org.xnio.Options.

      SASL_POLICY_NOANONYMOUS=false

       

      remote.connection.batchjobserver.connect.options.org.xnio.Options.

      SASL_POLICY_NOPLAINTEXT=false

       

      remote.connection.batchjobserver.username=ejbremote.connection.

      batchjobserver.password=test

       

      #JMS connection time out in milliseconds;jboss.naming.client.connect.timeout=20000
    3. Disable batch job server: av.batchengine.interval=0
      Note: Any BIZ and batch job distributor can be a batch job server if this interval is a number value above 0.
    4. Configure built-in job distribution with multiple servers in the URL. If all batch job servers are put into the URL property, the distributor sends batch jobs batch jobs to each server in turn.

      batchjobqueue.java.naming.provider.url=http-remoting://BATCH_1_IP:3447,http-remoting://BATCH_2_IP:3447

      OR

      Configure load balancer job distribution for a single server in the URL. With a single URL, the distributor sends all jobs to an end-point, either a load balancer or a single batch job server.

      batchjobqueue.java.naming.provider.url=http-remoting://BATCH_1_IP:3447

  4. ServerConfig.properties for the batch job server.
    1. Enable multi-database: multidatasource.enabled=true
    2. Disable batch job distributor: av.batchdistributor.enabled=false
    3. Enable batch job server: av.batchengine.interval=30
  5. To trigger a redistribution of jobs, update jobinstance_no to null from the database side.