Email Archive Migration Tag tag

Although an archive might be something the IT department would prefer to put on a tape and forget about, most email archives need to be ‘kept alive and kicking’ over period that could extend well beyond our retirement – or our next job move!

At the extreme end of the scale, Child Services related records – including those in email form – must be retained by UK Government bodies until the person’s 75th birthday. Imagine that!

Even if you don’t have a legislative reason to retain and discover emails, there’s usually a whole bunch of business and productivity reasons you need to ensure archives are reliably maintained and readily accessible for staff – and for a longer time than you bargained for.

So it’s no surprise that an enterprise will need to tackle at least one archive migration – possibly several – before the emails in question reach the end of the road (that’s if someone wants to take responsibility for pressing the delete button).

In fact Essential has already migrated more than a handful of customers twice – we even have several ‘three timers’ – within a span of 7 years. More recently this includes law firm Ashfords.

The drivers behind multiple jumps can be down to whole range of scenarios, including:

CHANGING CIRCUMSTANCES – For example, customers have found themselves needing to move on from technically sound solutions that have unfortunately ‘fallen by the wayside’ following vendor acquisitions or lack of vendor focus.
OVER-OPTIMISM – Some customers have been tempted to take advantage of increased storage capacity in newer versions of Exchange. Where this can be successful, the lack of single instancing and ‘re-hydration’ effect as emails get moved ‘back to where they came from’ can lead to a bout of archive indigestion (triggering a return to a dedicated archive).
STORAGE RE-FRESH – Extricating archives from high-end specialist storage devices and end-of-life storage devices (EMC Centera fits both categories) is a common request and very justifiable in the face of spiralling storage costs. Being able to physically retrieve from the storage you may have purchased a decade ago is also important – we even had one customer whose disks had started to rust.
FINANCIAL ATTRACTION – Even a recent re-vamp of an on-premises archive can be ousted in favour of a pay-as-you-go cloud model if that is what the FD desires.The good news is that from an accounting perspective, most assets – including software – are depreciated of period of 3 or 5 years, so relatively frequent switching to a new long term email storage platform is not the end of the world. Similarly, newer archive systems and storage platforms tend to have lower overheads.

What needs to be accounted for, however, is the cost and complexity of switching to the replacement (i.e. migration).

I recall the time we moved house just a few doors down the street. To save costs we decided to move ourselves – it was such a short distance for Pete’s sake.

The reality was that my partner lost 2 stones in the process of our DIY move. I guess you could say that, in that respect, our strategy to cut costs paid off. But never again.

When you’re planning a move you need to plan in removal costs from a reputable firm that will ensure everything makes it successfully to the new destination, quickly, intact and fully accounted for.

This is all part of good information governance that should be adhered to throughout the lifecycle of your corporate records.

PS – If you’ve moved your archives into Office 365, it’s highly likely that this won’t be the ‘final destination’ for your email records. Anything can (and often, will) happen that could mean a re-location of your data. The good news is that extracting your data out of Office 365 should be a lot easier….

This is a very frequently asked question for anyone wanting to switch to Microsoft’s cloud.

The short answer is: If you’ve chosen the right software for your migration project, your migration ‘engine’ will never be your bottleneck.

These are the top 3 things that will have an impact on the time it takes to move your legacy archives to Microsoft 365:

1 – Your data source

If your archive (e.g. Enterprise Vault) sits on sluggish hardware this might be your bottleneck as you’ll need to be careful not to compromise the performance of your archive service if it’s still in use. Some initial migration tests during the daytime will help you understand what can be tolerated. If your migration solution can access your archive stores directly rather than going via the archive database (API), you can usually get faster results, but this may not always be possible.
If you use compliant storage devices like EMC Centera you’ll be limited by the access nodes available for your migration activity. It’s worth checking whether your migration software is smart enough to migrate from both your primary storage and any replicated devices to double your throughput.
If you have a distributed archive where data needs to come over a slow WAN from multiple sites you could consider bringing a copy of the archive stores locally to your migration hub. This demands the ability to access the archive store directly (as you would typically not want to replicate your archive application servers).

2 – The target (& chosen migration approach)

Microsoft 365 has inbuilt throttling, so your archive migration solution should provide the optimal regimes for uploading.

The fastest migrations will use a combination of multiple processing tasks, threads, batches and other techniques to optimise migration into Office 365.

For example, our projects from EV to Microsoft 365 have seen and range of between 10 and 25 messages per second per migration task, and we can have any number of these. You should expect that the migration can automatically tick over for 16+ hours per day and that you can really ramp up activity out of hours.

In some instances, your migration will be limited by your physical network connection to Microsoft’s cloud. We have seen different results in different parts of the world.

To eliminate this as a bottleneck you may consider a ‘double hop migration’ whereby the raw archived data is shipped to the cloud first (e.g. Microsoft Azure) and the data can then be migrated to Office 365 from there.

You might also use the Microsoft Drive Shipping service, but that would require you exporting content from your archives into PST files and onto a hard drive and then shipping it to Microsoft (who will then upload the contents to interim cloud storage prior to upload into Microsoft 365.

Apart from the extraction process taking a long time and creating and tracking PSTs an administrative nightmare (for example, you will need to indicate which PSTs are destined for the primary mailbox vs the archive mailbox on a per user basis), you will have an undetermined ‘wait time’ or lag before the data is uploaded by Microsoft.

Either way, if you’re using a ‘staged’ rather than a direct ‘end-to-end’ migration you’ll need to ensure your archive is static before you start, so no more archiving.

Also, you will need to pay close attention to the process of handling shortcuts (making sure they are only removed once you know the corresponding item has been ‘rehydrated’ into Microsoft 365).

Finally, if you use a multiple hop approach to migrating your data, you need to make sure that your compliance team are happy with the handling of data because your chain of custody will be compromised.

3 – Your project plan

Having worked with over 250 clients to migrate their archives, the biggest risk to timescales slipping is your ‘preparation phase’, especially if you’ve got a lot of hoops to jump through internally before getting your project off the ground.

First, your overall Microsoft 365 migration strategy will come into play. For example, unless you are planning a big-bang or ‘cut-over’ migration, it is likely you will move users across in batches over a period of time.

Also, if you are not planning to migrate using a Hybrid approach, you will need to migrate the users’ primary Exchange mailboxes first and then schedule the migration of their archived data to run directly after this.

There will be many ‘political’ and legal issues to consider around how you will ensure any journal (compliance) archives are treated (a subject of a future post) and whether you will take the opportunity to limit the amount of user data you move (e.g. just the last 5 years’). This does not address what you plan to do with leaver’s archives (another subject we plan to cover).

Getting agreement around these matters with the various stakeholders in your organisation can take a long time.

The migration procedure itself tends to be the easy bit and the quickest to get ‘in-flight’!

Contact us today for our quick start guide to getting your migration project moving!

Get in touch

When going through an email archive migration, understanding the way your end-users work with their archives is worth getting to grips with, otherwise you could be heading for a bumpy ride.

A long while back we worked on a migration where we took our client’s request at face value:

“Just migrate the last 6 months’ worth of email archives into the cloud”. And so we did.

Well, let me tell you, the staff at the organisation in question kicked off big time when they realised what had happened.

In short, the decision to migrate this limited amount of data was made by the IT and compliance teams ‘in isolation’. By that, I mean the end users themselves were not consulted on what they needed and what was happening: they just arrived on Monday morning to discover anything older than 6 months was gone.

The upshot was we had to re-run the migration to move many years’ worth of users’ emails.

Always consult end users

The moral in the story is, when you migrate your archived data, making technical and/or compliance-only-led decisions about what gets moved can lead to huge productivity hit on the business.

So how do you make sure your migration is also user aware?

To ensure your migration is truly user – or dare we say – business-aware, a key thing to bear in mind when you make plans to move archived emails to a new platform is: “What types of email users you have in your organisation.”?

Over the years we’ve encountered 4 main types of users:

First up: ‘The Filer’

These super-organised users put everything into the right folder. This could mean moving things several times to ensure they can instantly locate what they need to do their job.

Something to check before you migrate: If your archive isn’t fully synchronised with the latest shortcut locations, emails could get migrated to the wrong folders.

Next, the Sharer

Sharers make a habit of forwarding useful emails to co-workers or putting them into public folders. They’re a vital component in building on your enterprise knowledge resource.

Beware: It’s highly likely folk that used to have access to ‘shared’ archived emails will lose out when you migrate, as many archives don’t store this type of information.

Deleters

They are the ‘dream’ end user in many ways, as they systematically get rid of what they don’t need or what they have already dealt with.

Watch out: Again, if your migration path does not synchronise up with what users have deleted from their view of the archive (and bear in mind that even the best archives can ‘miss’ deletion activity), you risk having stuff re-appear for end-users post-migration – which will freak them out.

Searchers

They tend to be less disciplined in their filing and rely heavily on their archive search service to find past emails. They’ll often delete emails safe in the knowledge that they’ll still be able to hunt them out later.

Caution: If you have a lot of this type of user, you won’t get away with using filters or doing a stub only migration.

Conclusion

Migrating to Office 365 or another new platform should be a stress free experience.

By paying attention to the types of end-users you have, and not under-estimating their reliance on access to archived email, you can ensure a seamless transition, where:

Emails end up in the right folders post-migration
Data that has been shared is still available post-migration
1000’s of previously deleted emails don’t suddenly reappear back in users’ Office 365 mailboxes
Years’ worth of past reports, proposals, papers, technical responses, orders, etc., are still accessible.

Migrating to Office 365 or another new platform should be a stress free experience

For more tips on best practices for email archive migrations, get in touch.

Get in touch

Warning:

.psts are like vermin.

Experts agree they are a pest and IT departments across the world are all too familiar with the support nightmares they create. They also carry the risk of data loss & exposure.

Apart from being massively out-of-control in most organisations (difficult to locate on users’ hard disks and containing unknown evils), they are inherently difficult to grapple with.

Having worked on PST elimination projects for over fifteen years now, we have encountered quite a few challenges and gotchas. Here’s just a few to bear in mind if you are planning to get rid of your PSTs.

1. Users Get Attached to their PSTs

Having been forced to use PSTs for so many years to stay within enforced quotas, end users have become used to them ‘being there’. We know of law firms whose partners have created PSTs for each major client, and store them on their laptops. Removing PST files without explaining why; and without creating a familiar and easy way to access the contents of PSTs post migration, will be very dimly received.

2. PSTs are only accessible by one user/application at a time

PSTs are only really designed to be accessed by one user or application at a time. This creates a challenge where PSTs are opened when a user logs into Outlook. If this happens as a matter of course at your organisation you will need some way to ensure users don’t access their PSTs during your migration project (see also next point). You will also have to work out how you are going to tackle users with laptops that only connect to the network intermittently, and users that always have their PSTs open.

3. PSTs are usually an ‘all or nothing’ thing

If you look at a PST in a directory listing – it’s just one file. It’s effectively a container file, and with most solutions that you can use to move or eliminate PST contents, you’re typically looking at taking the whole file with you. This can mean taking a whole bunch of stuff you just don’t need or that falls outside of your retention policy. Having said this, it’s possible to take a more granular approach and just select the items you want to move according to different criteria. We have found this saves a lot of time and network bandwidth (as well as archive storage).

4. PSTs might be changing all the time

If you plan to tackle PSTs we recommend you create an Outlook Group Policy that makes the existing PSTs read only and prevents new PSTs from being created. This will put your PSTs in a ‘stable’ situation while you are tackling them.

5. Be Careful Assigning Ownership

From a legal and audit point of view, who owns a PST file can be vital information. Where PST files are stored on network shares it might seem to be obvious who the owner is, but this depends on how structured the shares are and permissions on each folder within the share. Where “team” folders exist, each PST file could contain messages which are “owned” by several people. Where PST files are stored on desktops then you need to examine what happens when one user leaves and another joins. Are the desktops completely rebuilt or just new profiles configured? In the latter case PST files could be lurking that belong to users who have left the organisation many moons ago!

6. What happens post PST removal?

Users who are working offline will tend to have PST on laptops. Using Outlook in cache mode (which creates an OST file) is the logical alternative for enabling offline access, so following a PST migration exercise you need to make allowance for the extra network traffic that will be created during the first offline synch after migrating them.

There are many more PST tips we have

Get in touch and find out more about our PST migration services.

Get in touch

Even though Microsoft offers free tools to import your PST files into Microsoft 365, does anyone really relish the thought of migrating ALL of their PST files into Exchange?

Probably not, because:

You end up moving old rubbish
It will pound your network
Users might not like it

Having worked with hundreds of customers to roll-out email archives, PST migration is always the painful final phase that often gets brushed under the carpet for the aforementioned reasons.

The over-arching requirements for anywhere, any device data accessibility and centralised data governance, however, may be the final nudge to get your PST migration project management backing.

So is it acceptable to filter out the rubbish before you siphon it into your shiny new Exchange environment?

At the very least it’s surely prudent to get a policy agreed that will define your PST migration including whether to migrate:

Content that outdates your email retention policy
Folders marked ‘Personal’
Content that pre-dates your switch on of Journal capture
Duplicate PSTs

A PST migration project means bracing yourself for decisions about what to migrate and no-one likes decisions! But we like the advice from Craig Ball, a Texan trial attorney and certified computer forensic examiner:

‘To Preserve broadly is safe, but expensive. To Preserve carefully is safe and cost-effective”.

The truth is, any enterprise-level PST migration exercise – even if you gather up everything – is not easy and you may struggle with free Microsoft tools.

PST Migration Challenges

Check out the top 6 PST Migration Challenges

Learn more

We read with interest a recent Mimecast blog which refers to a TechCrunch article on the News Corporation phone hacking scandal. The archiving vendor alludes to the concept of an email archive making it possible to permanently delete an email from circulation.

Perhaps we misread the intention of the writer, but here’s a home-truth speaking from experience:

Even if you maintain a central archive of your emails, be it on-premise or the cloud, deleting any given email from said archive is highly unlikely to delete all copies of the email.

This is because any given email is likely to exist in many other places besides an archive (and print outs in a crate), including:

on past backups of your email and archive servers
in personal email archives (aka PST files) that can exist on the user’s own hard disk or even a memory stick, and
assuming it wasn’t just an internal email, in the archive of the organisation with whom an incriminating email was exchanged.

The first 2 examples are becoming easier to tackle. Earlier this month Microsoft released its own tool to find and ’round up’ the contents of PST files and for many years now our UK-based email management organization has been delivering services and software to aid the search and recovery of emails from backups, PSTs and other locations.

So yes – implementing a robust archive along with sound and defensible policies for deletion can significantly reduce your eDiscovery costs and limit your exposure, but if the crunch comes, the incriminating emails can and will come out of the woodwork more easily than you think.

We’re currently involved in a number of different archive migration projects, helping customers to move legacy email information from one system to another. These are often challenging projects to manage, with a lot of variables to consider.

One scenario which has come up a couple of times in recent months, is where a customer has made the decision to move an archive with several Terabytes of data from one third party solution to another.

Understandably at the point of deciding to do this, one cost they decided they could live without was the support contract for the solution they are leaving behind- why support a solution they are actively moving away from?

Well fast forward to 8 months later, and due to numerous environmental, technical and people changes, the migration is still ongoing. This in turn impacts their migration to Exchange 2010 which cannot be completed as the old source archive is not compatible.

To get compatible clients, and upgrade the old archive will incur cost and time penalties that the customer has not accounted for on top of an already delayed project. The customer is faced with the unpleasant prospect of some unexpected costs if they want to carry on with their Exchange migration before the archive element is completed.

Our advice? If your business is looking to make a change of archive platform, consider the implications should the migration take longer than expected. And perhaps stay supported for the existing solution- just in case..