How can TransVault Help With my Commvault Migration?
Available from global network of TransVault Certified partners, TransVault™ quickly and securely migrates legacy archived email (including archived user mailboxes and journals) from Commvault V10 and V11 (please call for other versions) into a wide array of different email archives and platforms.
It also migrates from a wide range of archives into the Commvault Express journal service.
All data movement is fully audited to ensure chain-of-custody, and mailbox shortcuts are fully managed to ensure a seamless experience for end users as they move.
How does TransVault connect into Commvault?
Where possible, TransVault connects directly via the Commvault REST (Representative State Transfer) API. In some instances, for example, when retrieving messages, we work directly with the source data. This is because the CommVault API can be slow and therefore not ideal for mass extractions at high speed.
What happens to shortcuts (stubs) when we migrate to the new system?
TransVault provides a comprehensive shortcut management service that ensures users have a seamless experience when they migrate.
As your data is moved, the corresponding shortcut is converted to point to the new archive. This activity can take place whilst users are online – there’s no need for any downtime or for users to logout.
In the situation where shortcuts aren’t supported by the target archive (for example, if you’re migrating from Commvault to Exchange or Office 365), legacy shortcuts are replaced (rehydrated) with the original item.
Another key feature designed to ensure a great user experience is that TransVault synchronizes with the current status of users’ shortcuts as they exist in their mailbox at the time of the migration.
For example, where users have re-foldered their shortcuts ‘post-archiving’, TransVault ensures the corresponding items end up in the right folders post-migration.
How can I find out if TransVault has moved all of our Commvault data?
Transvault migrations from Commvault start with a deep dive into the Commvault archive data to establish its exact contents.
This initial analysis enables you to create a baseline from which you can check that everything – emails, metadata and attachments – has been successfully migrated. It also enables forensic 1:1 auditing that provides evidence your data has been successfully transferred to the new environment.
Pre-analysis of the contents of your archive also gives you visibility of how much data there is to be migrated, how many mailboxes exist and how long the process is expected to take. This enables you to streamline and prioritise your migration, for example, migrating the IT department first, and apply policies that determine what you move and where. For example, move items less than 2 years old to primary Office 365 mailboxes, and anything between 8 and 10 years old to In-Place Archives.
The ability to audit is further bolstered by that fact that migrations are made in one transaction, direct from source to destination and with no interim steps that could lead to data being tampered with or going missing ‘in transit’.
In the event of a future eDiscovery, the audit reports that are available enable you to prove that you have not lost items during the move and that chain-of-custody has been maintained.
If the migration of an item fails (e.g. owing to data corruption or intermittent network issues) tasks can be rerun to target just these items, adding them into the new archive alongside items that are already migrated.
See also Q12 for more information about data remediation in the event of a permanent failure.
Reporting is also available to highlight how much data has been moved for each ‘mailbox‘ in the archive.
How does using manual extraction techniques compare with TransVault?
Individual archive mailboxes can be restored from Commvault into PSTs or Exchange mailboxes using the CommCell Browser. However this approach is a long and painful process and not suitable for large scale migrations for the following reasons:
Inability to cope with high-volumes: When restoring from Commvault to PSTs you need to be aware of PST size limits, and ‘chop’ large mailboxes into several PSTs less than 45GB. Similarly, if you
restore back to Exchange (with the plan to migrate from Exchange to Office 365, for example) you need to ensure that your Exchange servers can cope with high volumes of data. By comparison, TransVault is proven in many migration projects handling tens of TBs of data. It offers a highly scalable architecture and extreme flexibility for coping with the largest, most complex environments.
Breaks chain-of-custody: Any solution that migrates using interim files or steps risk breaking chain-of- custody. TransVault-powered migrations occur directly between the source and target, as a single, synchronous step, and as a fully audited process.
Poor reporting: TransVault generates detailed status reports that show totals of items found and migrated for each archived mailbox, along with any errors encountered, giving you total peace of mind. With manual Commvault restores, there’s limited reporting.
No flexibility – TransVault offers extremely advanced capabilities including detailed archive analysis, fine-grain data selection, sophisticated shortcut handling, address re-writing to support inter-domain migrations, re-assigning multiple archives that may have been created for the same individual (e.g. due to re-locations, name-changes) etc. None of these capabiities are available with Commvault restores.
Poor end-user experience –Manual restores do not give you awareness of which items have
shortcuts. This means if you export data, it will be everything in the archive, which will mean users are presented with data they thought they deleted years ago. So not only do you have a storage space issue, you have user support issues as well. By comparison TransVault offers sophisticated shortcut handling capability which includes the ability to exclude migrating an item where the corresponding shortcut has been deleted.
Generally speaking, any approach that relies on migration using interim files will encounter challenges:
- Need for interim storage space. PST files in particular are space-inefficient so you’ll need lots of extra storage to pre-stage your data.
- Potential loss of integrity. When using multiple intermediary conversions there is always the potential to lose information along the way with each conversion.
- Loss of chain-of-custody. Multiple steps risk loss of control over your data.
Performance comparison: A project to extract 10.5 million emails from a leading on-premise solution using built-in PST extraction took 5.5 man months to run (and re-run) TransVault is
benchmarked to migrate the same amount of data in 24 hours.
Cost comparison: Manually exporting the contents of a single mailbox into a PST file can take
1.5-3 hours (depending on size), with additional time required for checking completion, updating manual logs, fixing corruptions etc. Hiring a temp (at $18/hour) to manually migrate PST files, and an average mailbox migration time of 3 hours would cost around $54 per mailbox. Bearing in mind this is just half the process (the PST files would then need to be ingested into the target archive) manual migration costs can reach > $100 per mailbox.
How quickly can TransVault migrate archived items?
TransVault offers the fastest and safest migration times in the industry. This is confirmed by the many
Partners we have that have direct, project-based experience of TransVault vs other migration solutions.
TransVault’s multi-threaded, multi-server capability means that multiple extraction and ingestion pipes can be set up between your source and target system, thus driving your environment to capacity. You can also process a single mailbox using multiple migration threads – ideal for processing large archives and journals in the fastest possible times.
It is important to note that achieving the best performance is always dependent on environmental factors such as:
- Available network bandwidth
- Speed of the storage subsystem on which the legacy archive sits as well as the destination storage
- The ingestion performance of the target archive system
- The scheduling of other project elements such the commissioning of the target environment.
TransVault’s multi-threaded, multi-server capability means that multiple extraction and ingestion pipes can be set up between your source and target system, thus driving your environment to capacity.
Your chosen archive migration partner can establish a proof of concept (POC) to establish likely throughput rates in your specific environment. They will also be able to give you guideline speeds seen at other customer sites for a similar migration path.
Can TransVault cope with encrypted Commvault content?
We are involved in a de-merger and need to split up our archive. How does this work?
TransVault can filter items by user, groups, folders and dates, thus enabling data to be selectively and incisively migrated to one or more different locations.
TransVault can also re-write internal email addresses in sender and recipient fields so that the email is usable with any new domain naming or recipient-addressing conventions.
How does TransVault aid compliance when migrating?
Migration methods that rely on interim PST or EML files are subject to human error and have no tracking or auditing mechanism to prove that a migration was 100% successful.
As described earlier in this document, TransVault eliminates opportunity for human error: your data is automatically moved in 1 step, direct from the source to destination using banking-style transactions.
Each item moved is checked for integrity to ensure your data will be viable in the new environment.
There’s also complete auditing of the migration process, with detailed reports that show 1:1 mappings of the ID of the item in the source archive and the ID of the new item as it is moved to the destination archive, enabling you to demonstrate of a complete ‘Chain of Custody’ for your data while undergoing migration.
See also next question relating to the migration of journal compliance records.
Why do Journal Archives require special care?
By their very nature, moving email Journals and Journals stored in email archives demands the utmost care. From a legal and compliance perspective, the movement of journals should be closely managed and audited, leaving no room for error or loss of data integrity whilst your data is being moved.
TransVault delivers full auditing and full chain-of-custody in order to be able to prove to the relevant bodies that all due diligence has been made in migrating journal content. If this cannot be done, the results of a future eDiscovery request made be subject to dispute.
Secondly, Journal archives tend to be extremely large, making a manual extraction approach slow and subject to size-related problems that can occur when using PST or NSF files as an interim store.
TransVault allows journal and large mailboxes to be split into a number of separately handled virtual mailboxes of a user-defined size. This allows multiple processing threads to be applied to the migration of a single journal mailbox, significantly speeding up the migration task.
Thirdly, most of the organizations that capture emails into Exchange Journal mailboxes for compliance reasons use the Envelope Journaling feature.
This feature was developed by Microsoft as a way to preserve vital header information including BCC’d recipients and the expanded members of any distribution lists.
Again, from a compliance perspective, this data must be preserved and available for access when performing eDiscovery. Losing this vital recipient data would jeopardize the validity of any future eDiscovery case, as not all the people that received any given email would be included in an investigation.
The important thing to note is that different archive platforms store envelope information in different ways. For example, some store this information separately in the archive index whereas others store it in the archive store itself. This creates a very special requirement at migration time that neither manual methodologies nor vendor extraction tools can support.
TransVault reconstructs this information into the right format so that it can be passed into the target system.
If you’re moving data into Office 365, TransVault is able to migrate your legacy journal format into the new compliance model that has been introduced with Office 365, ensuring all your data is intact, fully discoverable and stored in accordance with Microsoft’s licencing policies.
What do you mean by ‘Chain of Custody’ and why is it important?
Chain of custody refers to the reliable recording of processes and procedures that occur while evidence
(physical or electronic) is being captured, held, transferred or disposed of.
It is vital that the evidence in question has remained free from alteration and that secure handling has been provided at all stages. This is important since any later data investigation or eDiscovery against migrated emails will seek to validate that the data has been handled correctly and that nothing was missed out.
If at all possible, transferring your archives in 1 step, end-to-end, is the best way to preserve chain-of- custody. This approach avoids any chance of manual intervention, loss of tight control or data spoliation by way of conversion into interim file formats.
Other migration techniques that involve extraction to interim storage areas and file formats (such as PST or EML files) risk your data being inadvertently lost, maliciously tampered with, or indeed corrupted while they are waiting to be ingested into the target archive. It is therefore difficult to vouch for the security of your data ‘in transit’.
TransVault’s direct, in memory, end-to-end, transactional transfers (e.g. direct from EV to Office 365) that involve no ‘touch down’ to interim storage and no interim file formats ensure defensible eDiscovery at a future date.
Additionally TransVault’s forensic 1:1 auditing provides evidence that your data has been successfully transferred to a new environment, and provides both a source ID and target ID.
What happens if an item fails to migrate?
As your data is moved, TransVault carries out a series of integrity checks to ensure your email records will be viable post-migration. Any items that fail to migrate are automatically re-processed a specified number of times and/or at a different time of day.
Failures to migrate an item may be temporary, owing to environmental issues such as poor network bandwidth or high loading on the legacy archive.
‘Permanent’ failures tend to be low – typically .001% of the overall email quantity. They are usually attributable to pre-existing problems in the source archive (i.e. not caused by the migration process). As such, it is likely that these items would NOT have been picked up by any audit or eDiscovery exercise.
In the event of a permanent failure, a full log of the item(s) in question is produced to enable investigation. You can attempt a manual retrieval of any failed messages directly from the log, a feature which massively reduces troubleshooting overheads.
If your organization requires further investigative work to be carried out on failed items, data remediation
services are available to help and where possible, fix the problem to the satisfaction of your legal team.
For example, in one migration project it was discovered that some emails only had the email ‘shell’ intact – the actual body of the email and any attachments were missing. The customer’s legal team requested for all the retrievable header data (i.e. Subject, To, From, Date) to be preserved and migrated along with explanatory text into the email body to explain to a future eDiscovery operator that the original message was broken prior to the migration. TransVault’s remediation service was able to meet this request.
NB – If your organization requires further investigative work and data remediation to be carried out in the event of a corrupted item, this should be discussed with your chosen TransVault service provider in advance.
Will I need services?
Typically, yes. Commvault can tend to be a complex environment that typically demands a more services-led approach.
One of our many specialist partners across the globe will be able to advise you on and deliver TransVault alongside services that include implementation, migration hand-holding, expert advice on best practices, compliance know-how, project management, archive-specific expertise, expert trouble-shooting and comprehensive management reporting.
To contact a reseller or partner, please see http://www.transvault.com/ for more information.