The Identity Sync Service, introduced with K2 Five (5.2), manages identity synchronization and caching in K2. Use the buttons below to download the installer for your version of K2.



Installing the sync service changes the way K2 syncs and caches identities (users, groups and group memberships) from your Identity Provider (IdP), such as Active Directory or Azure Active Directory.

Installing the sync service provides the following benefits:

  • Faster, differential syncing of identities after initial (full) sync for most IdPs
  • Removes the auto-expiration of cached identities
  • Avoids the direct resolution of identities with the provider
  • Addresses performance issues present in the previous way identities were cached
  • K2 uses the same identity cache as before when enabling the new sync service, but the way the cache is populated is based on the sync service. The new service provides better performance and predictability for populating the identity cache.
  • The sync service does not change the way system URM SmartObjects query and return data from IdPs, which means that enabling the new sync service will not affect your solutions that use these SmartObjects.

Who should choose this functionality?

This new identity synchronization is recommended for all K2 Five (5.3) customers. The main goals are:

  • Consistent and reliable identity data through frequent, proactive caching – populate the K2 identity cache for users, groups, and group membership, independent of someone logging or refreshing their worklist. Also, the identity information cached always represents the latest information from the IdP. For example, a cached identity is active in the K2 identity cache until it is disabled or deleted from the provider.
  • Improve runtime performance through local identity resolution – anytime the K2 platform needs to know attributes about a user (for example their email or manager) or group membership, K2 only queries the local K2 identity cache, avoiding 'expensive' direct queries of the IdP to retrieve user or group data.
  • Scalable with better performance by leveraging differential queries – when possible, differential queries get changes since the last sync with supported identity stores (AD, AAD, and SharePoint).
    Only AD, AAD and SharePoint support differential syncs. Other providers such as SQLUM, LDAP, and custom UMs, do a full sync every time.

The identity caching model manages the same identity attributes as in the previous model. These include user information (name, login name, email address, manager), group information, and group membership. It is important to remember that passwords are never synchronized or stored in the K2 identity cache. The identity cache is the single source for identities in K2 which the sync service updates on a regular basis to maintain consistency with the provider.

Terminology

Identities
  • Users, groups, and group memberships
Identity Provider (IdP)
  • Source of identity record
  • Examples include AD, AAD, and SharePoint
Sync providers
  • Execute queries against an IdP
  • Stores identity data in an intermediary table
Sync types
  • Full - gets all users, groups and group memberships from an IdP
  • Differential (diff) - gets only the changes since the last sync

Installing and using the sync service

K2 recommends performing a sync on the provider with the most users that actively use K2 before running the installer. Performing a sync before you use the installer allows you can gauge the amount of time it will take to sync.
See the Run the first sync below for instructions about to run your first sync.
If you have multiple provider instances, sync one at a time.

High-level steps

  1. Install or upgrade to K2 Five 5.3 and apply the latest fix pack, if applicable. The K2 installation creates identity sync providers for each security label in your environment. These sync providers are referenced when using SmartObjects to manage your syncs.
  2. Download and run the Identity Sync Service installer on your K2 server.
  3. Run the first sync for each provider instance you want to sync.
  4. Configure recurring scheduled differential syncs.
    Only AD, AAD and SharePoint support differential syncs. Other providers such as SQLUM, LDAP, and custom UMs, do a full sync every time.

Sync service installer

The sync service installer makes changes in the database to turn off the legacy identity resolution behavior and use the sync service instead. Use the Sync Service SmartObjects in the K2 Management site to manage identity syncs. Download and run the installer on your K2 server. If you have a distributed installation, you must restart the K2 service on each node, but you do not have to run the installer again on each physical K2 server in the farm. Use the Change button to select your K2 database if the installer does not automatically choose the correct database.
Image

Run the first sync

The time it takes to complete the first sync depends on the number of identities in your Identity Provider (IdP) and the complexity of the membership of the users & groups synced. The first sync can take a few hours to complete because all users, groups, and group memberships are synced. While the sync is running, the identities that are not yet cached are not available to K2, which means they will not have access to K2 sites and tools.

For AD, AAD, and SharePoint IdPs, all syncs started after the initial full sync run as differential syncs. This retrieves only changes in identity information since the last sync and, as a result, is significantly faster than the initial full sync. See the Sync Service topic in the User Guide for information on re-executing a full sync.

If you enable the sync service, you must run a sync for each of the providers in your environment. If you do not, you may see errors like the following:
Could not connect to server [k2.denallix.com]:[5555]
Inner Exception : An identity for the user K2:DENALLIX\Administrator could not be found.
Source : SourceCode.Categories.Runtime
Image

Follow these steps to run the first full sync of identities for each IdP:

  1. Open the K2 Management site and browse to the Sync Service object in the System category.
  2. Select the Operation SmartObject and execute the Start Sync method.

    • The Provider Name matches the security labels configured on your system. Active Directory has a default value of K2.
    • The Provider Instance Name is the name of the provider instance and is the domain section of a fully qualified domain name. Use your domain name if you have Active Directory. The AAD provider does not have a Provider Instance Name so leave it blank. SharePoint requires a Provider Instance Name which you can find by executing the List Provider Instances method of the Provider Instance SmartObject.
    • Enter a value of Yes or No in the Skip Deleted box to exclude (skip) or include deleted identities from your provider. On your first sync, K2 recommends skipping deleted identities because you most likely do not want deleted items in your K2 cache. The sync service does not retrieve skipped items in future syncs unless they change.
      After the first sync, set the Skip Deleted option to No (or leave it blank) so that identities you delete after the first sync are also marked as deleted in your K2 cache.
    If you are upgrading your K2 environment to K2 Five 5.3, set the Skip Deleted to No (or leave it blank) to sync items in the cache that you’ve recently deleted. Doing this also syncs all deleted items from in your IdP but ensures that your K2 cache is up to date.
    You must do this initial sync for each IdP (each label) in your environment.
  1. Open the K2 Management site and browse to the Sync Service object in the System category.
  2. Select the Operation SmartObject and execute the Start Sync method.
    Image
    • The Provider Name matches the security labels configured on the system. Azur Active Directory has a default value of AAD.
    • The Provider Instance Name is the name of the provider instance and is the domain section of a fully qualified domain name. The AAD provider does not have a Provider Instance Name so leave it blank.
    • Enter a value of Yes or No in the Skip Deleted box to exclude (skip) or include deleted identities from your provider. On your first sync, K2 recommends skipping deleted identities because you most likely do not want deleted items in your K2 cache. The sync service does not retrieve skipped items in future syncs unless they change.
      After the first sync, set the Skip Deleted option to No (or leave it blank) so that identities you delete after the first sync are also marked as deleted in your K2 cache
    You must do this first-time sync for each IdP (and hence, each label) in your environment.

When you install the sync service, it automatic syncs SharePoint groups in activated site collections. Activating a new site collection automatically configures SharePoint groups synchronization. Scheduled syncs happen every 15 minutes. Note that if you try to sync groups in SharePoint by clicking the Sync Groups link on the K2 for SharePoint settings page, you see the following message "Automatic Synchronization is turned on in this environment":
Image

If you add a site collection, the sync and sync schedule is set up automatically when you run the registration and activation wizards for that site collection.

  1. Open the K2 Management site and browse to the Sync Service object in the System category.
  2. Select the Operation SmartObject and execute the Start Sync method.
    Image
    • The Provider Name matches the security labels configured on the system. Use the name of the label you want to sync.
    • Leave the Provider Instance Name blank for legacy providers.
    • Enter a value of Yes or No in the Skip Deleted box to exclude (skip) or include deleted identities from your provider. On your first sync, K2 recommends skipping deleted identities because you most likely do not want deleted items in your K2 cache. The sync service does not retrieve skipped items in future syncs unless they change.
      After the first sync, set the Skip Deleted option to No (or leave it blank) so that identities you delete after the first sync are also marked as deleted in your K2 cache
    You must do this first-time sync for each IdP (and hence, each label) in your environment.
When syncing with AD FS, only the first 1000 records (identities or groups) are returned by default. To sync all AD FS identities, change your configuration as described in KB002721 - How To Configure ADFS To Return More Than The Default 1000 Records.

For every sync (full and differential), identities are cached to the intermediary cache and then transformed and placed in the main identity cache one minute after the sync. A maximum of two syncs can run at a time. Once they finish, additional scheduled syncs are run.

New identities added to your IdP after the initial sync, and changes to existing identities, are not synced to K2 unless you configure a sync schedule as detailed in the next section.

Configure sync schedules

Follow these steps to configure the identity sync schedule:

  1. Open the K2 Management site and browse to the Sync Service object in the System category.
  2. Select the Operation SmartObject and execute the Set Provider Schedule method.

    You can schedule an amount of time to sync between three minutes and seven days. You must enter it in the following format: DD:HH:MM:SS. For example, 00:07:00:00 is seven hours.
The AD, AAD, and SharePoint providers perform differential syncs on schedule but other IdPs (SQLUM, ADFS, LDAP, and custom) do a full sync at the scheduled time. Differential syncs are quick and efficient but if you have a large number of identities in SQLUM, for example, each scheduled sync takes about as long to run as the initial sync. Take that into consideration when configuring schedules for these IdPs. You may want to schedule syncs less frequently for IdPs that do not have volatile user data that changes often.

Sync run history

Use the Run History SmartObject method to see when your last sync ran and its status. Follow these steps to see the sync history.

  1. Open the K2 Management site and browse to the Sync Service object in the System category.
  2. Select the Run History SmartObject and execute one of the methods such as Get Run History Entries By Date Range as shown here.
See the Sync Service broker topic in the K2 Five User Guide for information on the Sync Service SmartObjects.

Considerations

  • If you add a new provider to your system, you must run an initial first sync on it, and then configure a sync schedule for it.
  • By default, K2 limits the active sync jobs to two. When a sync finishes, the next one starts. You may want to stagger scheduled syncs based on the average runtime of scheduled sync tasks.
  • After the first sync, there may be a discrepancy between the number of identities in the runtime cache compared to the intermediary cache. This is normal because there may be identities cached from other IdPs like SharePoint that you have not yet synced.
  • Data in the intermediary table is transformed and added to the identity cache one minute after a sync.
  • If you install the sync service but do not run an initial sync, you may get errors in K2 because identities from the IdP are not in the cache. A typical error is "An identity for the user {FQN} could not be found." where FQN is the fully-qualified name of the identity with which you're logging into K2 sites and tools.
  • If possible, you may want to run the initial sync at a time when the K2 enviornment is used less frequently by your users. This can help to mitigate users experiencing the "An identity for the user {FQN} could not be found" error while the initial sync runs.
  • If you decide to revert to the prior method of identity syncing, you must log a support ticket.

Known Issues

  • Managers remain after delete - When you delete a user who is also a manager of another user, the manager-direct report relationship still exists in the identity cache. A future release will address the removal of the manager-direct report relationship when the manager is deleted.
  • When syncing with AD FS, only the first 1000 records (identities or groups) are returned by default - To sync all the AD FS identities, change your configuration as described in KB002721 - How To Configure ADFS To Return More Than The Default 1000 Records
  • After installing the sync service and executing synchronizations, the K2 database file size may grow excessively. This is most likely due to the transaction log in SQL. In this case, consider performing a SQL database backup and shrink (or other SQL maintenance tasks) to reduce the size of the transaction log file.