Managing ServiceNow Data: Top Three ServiceNow Growing Pains and How to Address Them

Effectively managing ServiceNow data and its growth is key to maintaining a healthy ServiceNow instance. If left unchecked, ServiceNow data growth can become unmanageable, causing significant problems for organizations using the platform. 

This blog post will highlight the top three growing pains facing ServiceNow users and best practices for managing ServiceNow data growth. 

Table of contents:

1. Top Three ServiceNow Growing Pains
2. Growing Pain #1: Growing data volumes
3. Growing Pain #2: Integration sprawl
4. Growing Pain #3: Data Loss
5. Solution Recommendation

Top Three ServiceNow Growing Pains

As an organization’s ServiceNow usage increases, three common growing pains tend to reveal themselves making managing ServiceNow data more difficult. Such difficulties often lead to issues with performance, poor data availability, data loss and more.

As ServiceNow plays such a critical role in many organizations, these issues create a ripple effect throughout the organization, affecting overall business operations.

These bottlenecks usually stem from the over-customization of ServiceNow, the accumulation of large data volumes, and poorly implemented integrations.

Growing Pain #1: Growing data volumes

Data-driven organizations generate massive volumes of data – a lot of which they wish to retain for use cases including auditing, analysis, reporting, and other purposes. 

Data that is in active use should be available to users of the ServiceNow platform, via the platform’s production tables. Such data can be retrieved by querying the platform.

However, growing data volumes consume increasing amounts of system memory, which slows down instances and queries. Slow queries inevitably lead to challenges in data retrieval and data availability.

User’s ability to find relevant data quickly is also impacted as the large volumes of data become “noise” that organizations must filter out and comb through to get to the intended results.

Additionally, ServiceNow’s data retention policy dictates that users must pay additional charges for data storage if they exceed contractual storage limits.

While archiving using ServiceNow’s in-built archiving function can ease the burden, it is not an absolute solution. The platform’s in-built archiving function simply moves data from production tables into archive tables within the same instance. 

This can improve query speeds but ultimately, system resources are required to retain the archived data, and the total volume of data on the platform is not reduced, exposing organizations to ServiceNow’s additional storage fees.

• Related post: Troubleshooting ServiceNow Performance Degradation

Solution: 

The most practical solution for managing ServiceNow data volume growth is off-platform data archiving. 

As mentioned above, ServiceNow’s in-built archiving function only moves data into archived tables within the same instance. 

As such, to avoid additional storage fees and experience the most significant improvements to performance and data availability, it’s best to archive data off-platform within a more cost-effective, third-party storage solution.

Archiving should be supported by a data retention policy, outlining requirements and responsibilities including:

• Which data should be archived and under what conditions?
• Which data should be deleted?
• The legal mandates for retaining particular data and the processes that ensure compliance
• The employees/departments that should have access to particular data
• Who is responsible for data/systems storing data and their maintenance

For organizations that would benefit from off-platform archiving, the ServiceNow-native application Data Archive is available.

Learn more about Data Archive for ServiceNow and how it enables off-platform archiving and more.

Organization’s may also consider replicating active ServiceNow data into external storage solutions such as data lakes and/or warehouses. 

Such repositories allow for the fast retrieval of data – particularly in warehouses where a schema-on-read, structured approach to storage benefits query performance – and allow ServiceNow data to be used within third-party solutions for analysis, business intelligence, reporting, AI, ML and more.

By replicating data into an external storage solution, organizations can create a single source of truth, democratize data for all stakeholders, enable controlled, self-service access to data, and collate data from multiple sources for more comprehensive analysis.

Growing Pain #2: Integration sprawl 

Integrations allow ServiceNow data to be fed into and used within third-party systems (e.g. analysis, business intelligence, reporting, AI and ML solutions). 

Earlier on in an organization’s ServiceNow journey, introducing a point-to-point integration to serve a particular use case makes sense, and can help organizations with managing ServiceNow data. 

At this point, the frequency of data transfers and the volumes of data involved do not cross the threshold at which the platform’s performance suffers.

However, such integrations are typically web services-based (API and/or ETL) and as usage and requirements scale up, they affect performance. At this point, organizations capacity for managing ServiceNow data is impacted. 

• Related post: Why You Should Avoid API-based ServiceNow Integrations

Issues compound when organizations add more point-to-point integrations that compete for ServiceNow’s resources to retrieve data and execute tasks. 

Each one of these integrations must be monitored and maintained, typically by employees within the organization that have other duties.

The negative effects of such a poorly optimized approach to integrating ServiceNow can accumulate a sizable technical debt.

Solution:

To combat integration sprawl, multiple point-to-point web services-based integrations can be replaced with a single integration following the publish-once, subscribe-many model. 

The publish/subscribe model allows for ServiceNow data to be replicated into a message broker system (MBS).

The MBS acts as a cloud-enabled bridge between the target system and the source system(s) that queues data replicated from the source, ready for retrieval by multiple targets. 

This benefits the performance of ServiceNow by offloading the bulk of the processing required to distribute data to the MBS, preserving ServiceNow’s resources for tasks carried out on the platform. 

It also benefits the workloads of teams managing ServiceNow data and integrations as there is less to monitor and maintain. In some cases, even the maintenance of the publish/subscribe model integration can be offloaded to the integration provider.

• Related post: Integration Outsourcing Explained

For ServiceNow users interested in the publish/subscribe integration model for the platform, DataSync is available for ServiceNow partner’s Perspectium.

The data replication and integration solution is installed natively within ServiceNow, and leverages the publish/subscribe model and its MBS to enable over 20 million records to be replicated and transferred externally every day, without performance issues.

Growing Pain #3: Data Loss

Data loss events are an inevitability of modern technologies and ServiceNow is no exception. 

As usage of ServiceNow increases, the threat of data loss becomes more apparent. 

Not only because there is more data to lose, but with an increasing amount of customizations and employees interacting with the platform, there is simply more opportunity for things to go wrong.

As such, preparing for and recovering from a data loss event is a significant requirement for actively managing ServiceNow data.

• Related post: What Causes Data Loss in ServiceNow?

Most technical and business users of ServiceNow will recognise that backup and restore capabilities are vital for effectively managing ServiceNow data and preventing data loss.

However, the backup and restore function ServiceNow provides has a number of limitations that can increase the chance of unrecoverable data loss. 

• Backup schedules are managed entirely by ServiceNow, leaving users without control over when backups occur or the ability to schedule backups on demand.
• ServiceNow retains backups for only 14 days, which can be problematic if data loss goes unnoticed beyond this window, making recovery impossible.
• Restoring data through ServiceNow requires submitting a support ticket, involving a time-consuming process that can take several days. ServiceNow themselves state this should be a last resort.
• ServiceNow’s restore processes do not allow for granular control, as they are designed to replace entire databases rather than individual data elements.
• During a ServiceNow-led restore, the production instance must go offline, limiting access until the restoration is complete.

Solution:

While data loss in ServiceNow can’t be completely prevented, organizations can take a number of measures to ensure they are better prepared to recover.

The most significant elements of preparation is to create a disaster recovery plan and ensure the available backup and restore capabilities for ServiceNow, meet the organization’s disaster recovery requirements.

For many organization’s this will mean implementing a purpose-built, more robustly featured solution.

Organizations should prioritize solutions that provide them with more control over the backup and restore process, including control over:

• When and how often backups occur (including on-demand backups)
• What data is backed up and restored
• How long backups are retained for
• Initiation of the recovery process and use of the platform during restoration

One such solution is Snapshot, the purpose-built backup and restore solution for ServiceNow.

Learn more about Snapshot and how it lets users take control of their ServiceNow backups.

How Perspectium Helps with Managing ServiceNow Data

With the right tools, the growing pains associated with managing ServiceNow data can be addressed and eliminated. And in Perspectium, ServiceNow users have a solution for just that.

Created by the founding developer of ServiceNow, David Loo, Perspectium is a solution and service purpose-built to overcome common difficulties facing ServiceNow users and the management of the data it generates.

Perspectium applications and their use cases include:

DataSync: Publish/subscribe model integration and data replication solution for ServiceNow. Replicate over 20 million records per day externally, fast, and without performance issues.

Data Archive: Archive ServiceNow data off-platform to improve performance, avoid additional storage fees, and benefit data retrieval and availability.

Snapshot: Take control of the ServiceNow backup and restore process, enabling unlimited backups and backup retention, user-controlled backup scheduling, on-demand backups and more.

Perspectium applications are implemented by Perspectium’s expert delivery team, and installed natively within ServiceNow benefitting the learning-curve, implementation time, and time-to-value.

The applications were specifically designed to avoid the pitfalls of web services/API-based integration and for the highest possible throughput when replicating data externally.

To learn more about Perspectium’s user-friendly ServiceNow applications, contact us today!