The Swelling Maintenance of Custom ServiceNow Integrations

After a company builds a ServiceNow integration, it may seem like the integration maintenance is manageable. But maintaining that integration often takes more effort than expected.

A ServiceNow integration requires about 350 hours of maintenance per year, according to Perspectium’s survey of ServiceNow customers.

As you might imagine, we have talked with several ServiceNow customers who built their own integrations – before switching to a packaged integration process. Here are some of the factors that cause the hours of maintenance to add up for those who build their own integrations.

1. Ongoing Efforts to Reduce Performance Impacts

Web-service calls compete with users’ browser sessions. So these calls impact the experience for ServiceNow users across the organization. Both a growing volume of data and a push for higher velocity can impact ServiceNow performance. ServiceNow engineers must then continually tweak the integration to mitigate the harshest impacts on ServiceNow performance.

• Transfer volume: As an organization grows, its data volumes grow. Over time, you will be transferring an increasing volume of data. With APIs and other web services, this transfer means a larger number of rows per call, increasing the impact on ServiceNow performance
• Transfer velocity: To approach real-time delivery and to capture as many updates as possible, a ServiceNow engineer may poll data frequently. But a higher frequency of polling via web services creates greater impact on ServiceNow performance.

ServiceNow engineers often find themselves making repeated modifications to a web-services integration in an attempt to reduce performance impacts on ServiceNow.

2. Enabling New Talent to Maintain the Integration

The tech industry has one of the highest turnover rates. When an employee or consultant who helped to build a homegrown integration leaves or is no longer available, the integration is vulnerable. The next person to fill the role has a significant learning curve to overcome, which can include reverse-engineering the integration to identify exactly how it works.

3. Re-building Integrations That Upgrades Break

When you build a ServiceNow integration with web services, you are building a customization that can break when ServiceNow upgrades. By going back to out-of-box, you will likely need to rebuild the integration. With two major ServiceNow releases each year, customers generally upgrade once or twice per year.

4. Modifying and Re-trying Transfers That Time Out

ServiceNow has several built-in timeouts for web services. If the request for data transfer surpasses that timeout period, the transfer will fail. When this error occurs, none of the data transfers. You are responsible for retrying with smaller and smaller queries until one of them works.

5. Managing Data Deletion

Web services calls do not detect deletions. For example, if you create a location, a configuration item, or a user, and replicate it to a data warehouse, deleting it from ServiceNow will not automatically signal to the other endpoint that the record should also be deleted there. Someone will need to initiate a separate request to delete the replicated data.

To manage deletes, you can mark a table to be re-replicated completely whenever data is deleted. This would require two steps: deleting or purging the table from the target database, and then re-replicating that entire table. Clearly, this process is quite time-consuming.

6. Cleansing Data

When data leaves ServiceNow and arrives at a target, it can look very different, depending on the quality of the integration. For example, you add a new user such as a new hire to ServiceNow. You then assign that user to an incident. If the incident data transfers but the user data does not, you have an inaccurate or incomplete record. The work to clean up data errors such as this one adds to the data maintenance.

7. Denormalizing Data

Designed for transactional performance, ServiceNow has a highly normalized data architecture. When you transfer this data to another system, such as a data warehouse, you will need to denormalize the data references so that you can properly report and analyze the data.

8. Managing Field-level Accuracy

Data in target systems often looks different from how it looks in ServiceNow. And one reason may be changes to the data or its appearance as a result of the fields themselves.You may need to manage the field length (is the target field length truncated?), field type and conversions (e.g., date/time format vs unformatted), internationalization (e.g., language conversion), and missing or dropped fields. 

9. Handling Errors and Recovering Data

When an error occurs, you need to decide whether to retrieve the data, alert somebody, or perform another action. In many cases, because the data did not transfer, you will need to retry the transfer to maintain data accuracy at the integration target. Before you can even do so, it is necessary to identify which data transferred and which did not.

10. Monitoring for Failures

To maximize availability, monitoring and alerts should be in place. For each possible cause of integration failure, set up a process for alerting and recovery so that you can bring the integration back online as soon as possible.

11. Adding and Modifying Schema at the Target

Whenever you add a new field to a ServiceNow table, that field must be present in the target system before any of its data transfers. You must also configure the integration to perform the transfer for the new field. When someone changes the field type in ServiceNow, you will also need to change the field type in the target system, perhaps a data warehouse, so that the delivery will be successful.

12. Managing Security

Managing security means maintaining both authentication and authorization.

Authentication: Those roles that have access to ServiceNow tables require login credentials. For any login credential, the user stores the password in a location – whether within human memory (which can fail), or written on paper or digitally, which require their own security measures. Because of security risks, password rotations go into effect. But when an integration encounters an expired password, that integration will stop working until somebody intervenes.

Authorization: It’s best to avoid creating a table that has no restrictions on which accounts can access the data – both for security and performance reasons. So ServiceNow teams maintain access control lists (ACL). Appropriate management of ACLs can consume a great deal of time.

Offloading the Maintenance Effort

As a packaged integration process (PIP), Perspectium is integration as a service. PIP customers shift the burden of security, performance, and other maintenance concerns to the vendor. Check out stories of ServiceNow customers who offload maintenance efforts by relying on integration as a service.