Introduction

These issues were mainly caused by memory enhancements made in earlier releases of K2 blackpearl 4.5, race conditions due to flaws in the threading model behind IPCs, contention on connections established between Parent and Child IPC servers as well as problems with messages between servers in an NLB cluster.

The issues ranged from:

  • Intermittent IPC failures with an Error Description of ‘1’;
  • Parent IPC processes stuck in Running or Active state while the Synchronous Child IPC process has completed successfully;
  • Child IPC processes not starting at all;
  • Duplicate Child IPC processes starting; to
  • ‘Object Reference not set to an instance of an object’ errors when starting and completing Child IPC processes
Note: The following architectural changes to the manner in which the IPC Event is operated, may have an impact on existing K2 Installations with active process instances. See the section at the end of this aticle for further information.

Architectural Enhancements

IPC Event Designation Enhancement

IPC Events were previously handled similar to a synchronous Server Events. Even though the Child Process was separate, it was still handled as though it was contained within a synchronous Server Event.

This meant that for a Synchronous Child Process, the Parent Process would continue executing in a Running state while the child process executed until it was fully complete (or up to the first asynchronous point – like a Client Event). This methodology meant that the Parent Process would be retained in memory until the Child Process was complete.

As a means to reduce the memory usage, IPC Events are now treated similar to Client Events.  Once the Child process has started, the Parent process is placed in an Active state and the SourceCode.Workflow.Client connection is passed back to the calling application (in the case of a Parent Process being started synchronously).  This resulted in a behavioral change for true Synchronous processing across IPC boundaries.

IPC Object Layer Enhancement

Important: The changes that have been made to the IPC Feature are primarily a refinement of the existing methodology.

Previously, the relationship between a Parent and Child process was governed by an IPC Object layer internal to both the Parent and Child processes. The inefficiency with this model was that if an event took place resulting in an error within the IPCObject layer - not within the process itself, the Parent or Child process had to be put into an error state to resolve the error.

 Image

To resolve the error, the complete IPC Event (Parent process) or Succeeding Rule of the last Activity (Child process) had to be re-executed in order to re-instantiate the IPCObject layer.

 Image

The new enhancement separates the IPC Object layer from both the Parent and Child process and creates a new intermediary object layer whose specific role is to manage the start and outcome of the Child process and any errors, within this layer, should they occur. As a result, an IPC Process will go into error state depending on where the problem is, i.e. if the error occurs in either the Parent or Child process, it will go into an error state. If the problem is with the now intermediary IPCObject layer, the error will surface from there and not hinder the Parent nor Child process inaccurately. As a further enhancement, to ensure that the IPC Object error generation does not become useless or not surface, K2 Workspace and K2’s Error Repairing Tools have also been enhanced to ensure that the errors surface with the K2 Management Console.

K2Server.setup File Amendments

As a further enhancement, additional thread pools have been added to assist with the K2 server performance. The enhancement's role is primarily to control the number of threads that are allocated for IPC Objects and Asynchronous Objects. If the environment makes extensive use of IPC the thread pool count can be altered to gain performance enhancement for IPC operations specifically.

The region in the K2Server.setup file ([installation drive]:\Program Files (x86)\K2 blackpearl\Host Server\Bin\K2Server.setup) where the entries are located are shown below:

<K2IPCThreadPool>
      <Threads Count="10" />
      <Priority Value="Normal" />
    </K2IPCThreadPool>
    <AsyncThreadPool Name="K2AsyncThreadPool" Assembly="SourceCode.Workflow.Runtime" Type="SourceCode.Workflow.Runtime.K2ThreadPool" />
    <K2AsyncThreadPool>
      <Threads Count="10" />
      <Priority Value="Normal" />
    </K2AsyncThreadPool>
    <IPCSettings ExpireOnDelete="false" IPCThreadInterval="60" />

 ExpireOnDelete – This setting enables control over the behavior of IPC events when a child process instance is deleted.

  • True: If a synchronous IPC event is active and the child process instance is deleted, the parent process will be notified and expires the IPC event. The parent process continues as if the child process completed correctly. This setting might be moved into the IPC event and the option set per IPC event instead of being a global setting
  • False: The above behavior will not take place, but the setting can be enabled per IPC event.

 
IPCThreadInterval – This setting enables control over how often the IPC thread polls the IPC tables for new entries to process.

Note: In most cases new IPC entries will be processed immediately, but in the case that it does not get processed immediately, it will be picked up with the next polling interval. The Value is set in seconds and the suggested value is 60 seconds.
Important: Setting the IPCThreadInterval below 10 seconds is not recommended as it will cause the IPC thread to put too much strain on the server owing to the SQL queries it has to run on every polling interval.

Error Scenarios and Additional Considerations

With these additional enhancements, existing or new processes may still go into error. The details below will help the user identify the behavioral changes:

Note: The  behavioral changes listed below are ones known to occur and not limited to these.
  1. With the introduction of the new IPC Object model, if an error takes place within the IPC Object model, neither the Parent nor Child process will go into error. The errors related to the IPC Object model can be viewed in the error logs.
  2. During Run Time (this may have been detected during the process development phase), there can be a delay of up to 1 minute before the child instance will start or return to the parent process. This is by design and is normal behavior, but is the exception as the Child process should start shortly after the IPC call is made.
  3. If a parent process is started synchronously (no relation to the actual IPC call at that point in time, in code for example the parent process is set to start synchronously) as soon as the IPC event is "hit" the synchronous call will complete when the IPC Process i.e. the Child is set to start synchronously.

Upgrade Considerations

Important: As a best practice ensure that the processes are all complete, in an Active only state or that no bulk processing is taking place before the K2 Server is stopped for an upgrade. If there is bulk processing taking place the known issue discussed below may be encountered. In other words, the process that is in the Running state may encounter the following error if the upgrade took place while the process was in the Running state.

If or when an upgrade is being performed from previous versions to Update KB001290, which includes versions previous to Update KB001230 the upgrade may encounter the following known issue.

Known Issue: When an existing installation is upgraded Parent Processes with Asynchronous IPC Child processes stay in the Active state but do not continue running. This is owing to the outstanding logging of ProcessInstances from the K2Server to K2ServerLog databases which was caused because the K2 Server was stopped for the upgrade.

Current Workaround:  Re-run the script creating the missing _IPC and _IPCAsync entries (‘K2ServerAlter.sql‘ - no service restart required).

Error Resolution

This issue can be resolved by installing the latest K2 4.5 Update.