It is often wiser to adapt our systems to what people do in practice, than the other way ‘round.
A big challenge with implementing a centralized authoritative repository representing an organization’s System of Record is that this is often not practical. Many traditional Enterprise Content Management (ECM) implementations failed because of the enormity of the project and the complexity of the implementation, not unlike big ERP and Enterprise Management projects. They put an excessive burden on IT and have required IT intervention for minor changes. ECM systems were often intrusive and required users to change the way they work, causing widespread user dissatisfaction. ECM systems as a result are best suited for the most critical business documents, often in specific departments (Finance, HR, etc.) of the enterprise with well-structured processes or in highly regulated industries (e.g. pharmaceutical).
The need for flexibility and to tap widespread user support across the global organization has moved organizations away from centralized repositories and towards decentralized distributed repositories unified by a holistic policy management approach.
The growth of unstructured information, particularly email, has resulted in the emergence of online archiving solutions as a System of Record. Archives are particularly effective in handling billions of small unstructured items such as email. As archiving solutions do not include collaboration and document management capabilities and integrate with email and file systems at the back-end, they do not impact end-users. The main focus of archiving has been for compliance and eDiscovery. Archiving solutions typically provide a rich set of capabilities for retention management, monitoring and supervision, search, legal-hold and early case assessment. With the growth of unstructured information and the use of expensive storage for production email and file systems, a secondary focus for archiving is to offload production systems. This is done by moving data that is inactive to the online archive (and deleting the production copy) or by stubbing the data (leaving a pointer in the production system to the data in the archive).
Online archiving of email has become a widespread practice, but the use of archiving systems for files and documents is still limited. The pervasive use of email for communication makes it a frequent target of internal investigations and external eDiscovery requests. This often results in the implementation of email archiving for all users in the organization. In many organizations however, the cost of moving large numbers of files and documents into an archive and then storing another copy is considered prohibitive relative to the perceived gains. File systems also scale a lot better than email systems do, making file archiving less of an urgent problem.
This brings us to backup. Any data that is important for the business has to be and is backed up. Backup has been established as a standard practice for all applications and file systems. Why not leverage this existing and comprehensive repository of information, which is kept safely and securely, for information governance? Why not bring the information governance capability to where the data resides, rather than moving the data to this capability?
Guess what? We know from experience that organizations are already using backup as an archive. Despite the admonishments of industry experts, tape has always been and continues to be used to meet archiving needs. The configuration of long term retention periods for a fraction of data on disk and cloud backup is an indication that it’s being used for archival purposes. eDiscovery requests often result in many days or weeks of manual effort by IT searching backup for relevant data.
Data protection solutions can be enhanced to make a backup repository a System of Record and address information governance needs. Next generation data protection solutions can incorporate archiving by indexing and classifying data as it is being backed up. This will support search for litigation and compliance. Adding sophisticated policy management, specifically granular policies for retention and disposition of data and policies for deletion of inactive source data will help address archiving needs.
Backup is changing in very significant ways. Will backup be busy adapting to the new virtualization and cloud environments? Or will it have a chance to incorporate archiving features and become a System of Record? Is this an idea whose time has finally come? What do you think?