Integration Throughput Explained for ServiceNow Users

When choosing an integration solution for ServiceNow, integration throughput is one of the most important considerations.

But what is integration throughput, and why is it particularly relevant to ServiceNow users?

Table of contents:

1. What is Integration Throughput?
2. What Affects Integration Throughput?
3. ServiceNow Use Cases that Require High Integration Throughput
4. DataSync: The High-throughput ServiceNow Integration Solution

What is Integration Throughput?

Integration throughput is a key performance indicator for integrations. It measures the amount of data that can be passed between systems in a specific period of time. 

Measuring integration performance allows organizations to pinpoint bottlenecks that might be disrupting data availability. 

Key terms relating to integration throughput include:

• Data volume denotes the quantity of data processed within a specific timeframe.
• Transaction rate denotes the number of transactions/requests processed per unit of time.
• Latency is the time delay from initiating a request to its processing and response generation.
• Scalability is the system’s ability to maintain or enhance throughput as the workload scales.
• System capacity is the peak throughput the system can sustain without performance issues.

What Affects Integration Throughput?

Both technological constraints and data quality issues can affect integration throughput. Sometimes these issues are one and the same. 

A common example of a technology constraining integration throughput for ServiceNow users, is API-related performance degradation. 

The process used by API to retrieve data often requires making repeat calls to – and queries on – the source database. 

When such calls are inefficiently configured, occur too often, or are attempting to retrieve too much data, the performance of the source system is negatively impacted. 

• Related post: Why You Should Avoid API ServiceNow Integrations 

Other examples of technological constraints include network bandwidth, system architecture, and hardware & software performance.

Data quality issues can also affect integration throughput by preventing records from being ingested by the target system. This includes poorly formatted data (bad email addresses, phone numbers, etc) and other data quality issues. 

Technological constraints can influence and even cause data quality issues, such as the inability to update schema and transform data pre- or post- send.

Why is Integration Throughput Important?

Modern organizations are reliant on timely access to data. 

Delays can negatively impact everything from the service levels delivered by employees and experienced by customers, to the decisions made by business leaders.

Why is Integration Throughput Important to ServiceNow Users?

ITSM solutions such as ServiceNow create a lot of data, making high-throughput essential for replicating and transferring the sheer volumes of data present. 

In cases where an integration is reliant on ServiceNow’s operational bandwidth to function, higher demands in integration throughput can cause severe performance degradation on the platform itself. 

Not only does this limit integration throughput, but it also disrupts tasks native to the ServiceNow platform. 

ServiceNow Use Cases that Require High Integration Throughput

Some ServiceNow integration use cases require more throughput than others.

This could be down to the volume of ServiceNow data connected solutions require, or the time-sensitive nature of the data and its insight among other reasons.

Artificial Intelligence (AI) and Machine Learning (ML)

The output of AI and ML solutions is improved by the volume of quality data that they ingest. As such, poor data quality and low data volumes can limit the effectiveness of AI and ML initiatives.  

While ServiceNow has its own AI and ML capabilities, there is much more potential available to organizations via third-party AI and ML solutions.

So to truly explore the potential of initiatives such as AI Service Management (AISM), ServiceNow users need a way to transfer data from their ServiceNow instance into third-party AI and ML solutions. 

A high-throughput data integration solution is vital in enabling the transfer of the large data volumes required to support the initiative. 

• Related post: How AI for ITSM Will Revolutionize the Workplace (and How It Won’t)

Analytics, Reporting and Business Intelligence (BI)

While ServiceNow’s default reporting and analytics capabilities can suffice for basic, ServiceNow-specific reporting needs, it has many limitations. 

For instance, users can only run reporting and analytics on data within the Platform and can only access up to 180 days of historical data. 

Furthermore, ServiceNow lacks real-time reporting capabilities and platform performance suffers when dealing with data-extensive reports.

With a high-throughput integration solution, organizations can replicate ServiceNow data within other systems, making it available to departments around the enterprise and purpose-built reporting, analytics and BI solutions, unbound by the limitations of ServiceNow.

Data Lake and Data Warehouse Building

With a ServiceNow-to-data lake/data warehouse integration, organization’s can free ServiceNow data from its silo within the platform.

This improves data availability within organizations and helps create a single source of truth that encompass ServiceNow data and data from disparate sources.

With improved data availability, teams across an organization can access and use ServiceNow data on a self-serve basis, storing and retrieving data as per their needs.

High integration throughput can help populate data lakes and warehouses more quickly and efficiently. 

High-throughput bulk data transfer supports their creation, and high-throughput dynamic data transfers can support their upkeep over time, ensuring the vast volumes of data generated by platforms like ServiceNow are transferred in, or near real-time.

When choosing a ServiceNow-to-data lake or data warehouse integration data quality is also a key consideration as teams need confidence the data they access is accurate and of use.

Data transformation is also particularly important where data warehouses are concerned as the technology stores data in line with a set schema.

DataSync: The High-throughput ServiceNow Integration Solution

Organizations using ServiceNow benefit greatly from high integration throughput solutions that replicate and transfer quality data, securely. 

One such solution used internally by ServiceNow themselves, is Perspectium DataSync. 

Designed by David Loo, ServiceNow’s founding developer, DataSync was purpose-built to extract large volumes of ServiceNow data and replicate it within third-party solutions and repositories.

ServiceNow uses Perspectium to replicate their own ServiceNow data within an external repository. This provides greater flexibility as to how ServiceNow data can be used, and who it can be used by. 

Upon implementing Perspectium, Venu Malyala, ServiceNow’s Senior Director of BI, MDM & Analytics said:

“When we started our journey at ServiceNow, our goal was to develop a real-time data warehouse and provide analytics at lightspeed.”

ServiceNow needed a scalable solution to keep up with their rapid growth and meet the demands of the organization where “every department needed data to be available for self-service and analytics.”

Malyala shared that the solution ServiceNow originally implemented was “not able to scale and provide real-time integration.” 

They also met with a number of vendors before they “finally met the Perspectium team and were able to quickly do a POC and enable real-time integration, a scalable one. Our biggest challenge was the volume of the data that we were passing, and we were quickly able to scale with Perspectium and integrate our ServiceNow instances.”

With Perspectium, ServiceNow is able to feed 200+ dashboards and five predictive solutions in their big data environment, enable self-service access to various departments for analytics and reporting and transfer over 20 million records per day, without performance issues. 

This is made possible by Perspectium’s API-free method of data replication. Instead of relying on performance-degrading, external API calls, Perspectium is installed natively within ServiceNow and uses efficient push technology to facilitate transfers. 

With Perspectium, data is pushed off platform and into a message bus without ServiceNow having to process external API calls.

Target systems then retrieve data from the message bus, allowing ServiceNow performance to remain optimal, and integration throughput to be as high as possible.

Want to learn more about how DataSync delivers high-throughput integrations for ServiceNow? Talk to us!