Table of Contents
This guide is aimed at giving you a complete end to end understanding of the fundamentals of email archiving, the various architectures possible, the core reasons why you should archive email, what you should consider while choosing a platform and what are the best practices for using an email archiving platform.
Why is Email Archiving Necessary?
The Trend of email consumption today
60% of business critical information is stored exclusively in email - IDC
4GB is the average annual growth in a user's archive account - Mithi
126 messages are expected to be sent and received, on average, per business user per day, by the end of 2019 – Radicati
This means that a 200 user company will accumulate more than 1 TB of data in 1 year and this will keep growing year after year.
What is the need for Email Archiving
60% of business critical information is stored exclusively in email – IDC That makes it vital to have an email archiving strategy to ensure eDiscovery, regulatory compliance, litigation support, Storage Management, information availability, data backup, and business continuity. Discovering useful information, Recovering ‘lost’ information, Uncovering valuable data, Gaining insights An email archiving solution ensures that all emails are collected and stored centrally, before they are delivered to the users. A central access to all the email information, reduces internal inefficiencies of finding information across multiple locations and dealing with deleted or lost emails. Plus having a central access to all the email data helps you easily find all data for contractual or obligation purposes, mandatory compliance and legal matters during internal audits. In addition, a central easily searchable data set, means you can quickly recover lost or missing email information, or gain useful business insights by leveraging analytics on the data. Improving Mail Service & End point performance with a split mail box By keeping only recent mails on the live storage and moving older mails to a secondary storage, the mail server and the end point performance improves due to reduced sync times and quicker searches along with simplifying email data management. For example, let’s say you have your primary mailbox which you are accessing via MS Outlook and you have a secondary mailbox which is your archive account. The strategy is to keep your frequently accessed primary mailbox very slim, possibly with an automated retention policy with only the recent email data, while the rest of all your infrequently accessed email data is on your archive account. Managing Risks and Ensuring Compliance Enterprises need to backup email data, which is older and still necessary for future reference and regulatory compliance. Archival along with an easy method to search and export data, will help protect an organisation against all kinds of situations, which cause data loss such as accidental or malicious deletion of mail, data corruption on desktops, laptops, ransomware attacks, loss of laptops, mail deletion, etc. Simplifying Email Data Management Traditionally, ensuring safety of email data required a lot of heavy lifting in terms of having a backup strategy, backup tools, backup storage, processes, redundancy, planning in case the backup itself fails, or in case the tape drives fail, in order to prevent data loss. Typically all this email data is stored in a hierarchical form, in a chronological order. Searching and finding information in these devices, in good time, is next to impossible. A good archival platform will store all your information live, search ready and always online. This means that you can locate any email of any period in a matter of a few seconds.
An Ideal Vision for Data Management and Data Mining
In our conversations with hundreds of decision makers in enterprises across industries and sizes, three fundamental outcomes showed up as required by all organizations from an email archiving platform:The common thread across the outcomes is the need to make email archival a mainstream day to day application to boost productivity of users and make the accumulated data more useful to the organization.
- One, the need to be ensured that, a copy of all your data is automatically ingested into a safe, secure and reliable platform, without having to run a human driven process
- Second, the need to decentralize access to historical data so that end users are more productive with self service, when it comes to finding earlier data pieces and restoring them
- Third, having access to fast discovery tools, which can help leverage accumulated data
What are the bottlenecks and roadblocks in realizing your ideal state for data management?
Most businesses have been managing their email data store using storage devices in house so far. More recently, some have begun to use some sort of hybrid combination with the cloud to overcome the limitations of the inhouse storage systems. Considering the above trends, however, on premise systems fall woefully inadequate to scale up to handle the massive and continuous growth in email data. Some of the challenges faced by businesses in trying to manage the accumulation of email data are -
- Inconsistent and inadequate reliability of locally deployed infrastructure: It takes a good deal of hard work and deployment of substantial resources to create in-house infrastructure, which is dependable by design and delivers an uptime upward of 99.99%. In addition planning a backup of the archived data and a disaster recovery system for this infrastructure is needed.
- Lack of elasticity of on-premise infrastructure: You would typically need to design scale into your infrastructure from the start and also provision extra capacity in advance to be ready for the growth. Components typically used to configure local solutions have limited head room (limited elasticity) to accommodate the growth.
- Planning for the security of in premise infrastructure and data: Ensuring security means working from your physical infrastructure, right upto the app access layers. This is no trivial task and requires specialized teams to work on this. Setting up a multilayered security framework, will cover technology systems like firewall, access controls, attack and intrusion prevention systems and more, processes like accessing the infrastructure, change management, access key management, people roles, data access policies and more.
- Setting up Management and Monitoring systems for infrastructure and application: This whole stack would need regular upkeep, hygiene and 24/7 monitoring to ensure consistent performance and reliability. This would involve automated monitoring to ensure service and infrastructure uptime, a 24/7 NOC team to respond to these alerts, and a management team to run routine cleanup and other maintenance jobs.
- Dealing with higher upfront Capex: The entire project can be an expensive proposition since you have to typically budget and procure compute and storage for 1-2 years of projected use.
So instead of working out an infrastructure and keeping your data where the infrastructure itself might be the limitation, organisations are opting for cloud archiving platforms to be assured that they have a copy of their data in a separate operational infrastructure with guaranteed durability.
So What are the Do’s and Don’ts of Archival
Recognize the difference between Archiving and Backups
Often people use the words Backup and Archiving interchangeably as solutions to store and restore data. Traditionally, data management equaled Backups to another medium, like drives, tapes, and in some cases the cloud. Most organisations have been taking email backups by either storing their PST files, or copying the mail from their servers, or using some end point backup tool to secure their email. Believing that their data security requirements are met as long as they are maintaining backups. However, these backups are for all practical purposes notional when it comes to supporting the business requirement of data analysis, selective email retrieval, business intelligence, knowledge mining, compliance and more. Along the way, there have been several attempts to develop tools and systems, which attempt to use the backup as a live source of data, but fail at this miserably. Backups are typically snap shots and periodic by nature, dont guarantee on capturing changes between two backup runs. The original purpose of Backups was for disaster recovery in case of a system crash. You use it to restore the last good state of the system from the most recent backup. While backups, managed by the application administrators, will stay on to manage the disaster recovery part of the deployment (at least for some more time), businesses are becoming aware that backup is not equal to data management or archival. Backups are necessary as a basic IT requirement, but an email archiving solution is what you need to help your business leverage accumulated data.
What is Email Archiving?
Email archiving (Verb): Keeping a read-only copy of every mail sent and received by your users in a digital vault. The essence of email archiving is to ensure that a copy of every mail sent and received is stored in a separate archival store. Also, it is preferable to have the following attributes for the email archiving system:
- Ensuring that the captured email is in its original form, this copy should be created before the mail is delivered to the end user (also known as in-line archiving).
- Safe access to their respective archive accounts to access their own historical email.
- Immutable, highly reliable, durable, and physically separate infrastructure ensuring safety and security of your business critical information.
- Fast ediscovery console to enable easy search of email of all your users from a central point.
By 2019, 75% of organizations will treat archived data as an active and “nearline” data source, and not simply as a separate repository to be viewed or searched periodically, up from less than 10% today. - Gartner
3 Types of Email Archival Architectures
Organisations looking at deploying an email archiving solution have three basic types of solutions to select from. Here’s a brief description on you go about setting up and managing them. Shared SaaS Email Archiving On Cloud With SaaS Email Archiving, Simply Subscribe, Configure and Consume the services. No infrastructure or management required at your end except an Internet connection to allow transport of mail from the Primary mail servers to the Archival service on the cloud. Private Email Archive On Cloud For a Private Cloud Email Archive, setup a server and storage on the dedicated cloud, on your own cloud account, and install the chosen email archiving software on it. The instance type and storage capacity will depend on the number of users and the retention period. Manage the archival set-up on the cloud on an ongoing basis. Dedicated Email Archive On Premise For on premise email archiving, provision server and storage infrastructure in premise (bare metal or virtualized). Next, deploy the Archiving software on this infrastructure and finally configure for high availability and security. The server capacity and storage capacity will depend on number of users and the retention period. The users access this over LAN/WAN. Manage the storage infrastructure and archival set-up on an ongoing basis.
A strong case for Cloud archival over In premise archival setups
IT Admins will also vouch for how setting up and maintaining on-premise archiving solutions is a complicated and time-consuming task. Using a cloud archiving system provides your organisation several advantages, a few of which are listed below:
- Zero Capex for hardware and software infrastructure
- "Pay as you go" model lets you easily scale up your storage with a sudden surge in storage requirement.
- Auto upgrades for hardware and software. No manual intervention required.
- Elastic compute and storage means no upfront provisioning and automatic infinite scale
Key Considerations While Choosing an Email Archive Solution
Always Archive on a separate operational Infrastructure A separate operational infrastructure gives you an additional layer of safety since you now have a copy in another location. In addition, knowing that a copy of your data is available elsewhere, you can freely trim your data on the primary mail platform to gain efficiencies. Go cloud to Gain from Elasticity, Scale, Reliability, Security and Performance The cloud platform takes away all the heavy lifting of building a reliable and high performance infrastructure to support the archival application. The most important piece of technology here is the elastic storage, which gives immense flexibility to scale automatically on demand. There is a deliberate movement of IT workloads to the cloud and for good reason. Go for Zero infrastructure and Simplified Management The cloud storage infrastructure is the safest storage platform available today. With guaranteed uptime of the infrastructure and extremely high durability on the storage, you are assured that once you put data on the cloud, it is safe for life. No need to maintain backup etc. Go SaaS to Gain from Guarantees via SLA SLA’s provided by a cloud service provider gives organisations the guarantees on the performance of the service thereby creating a legal agreement. Ensure Tools for Easy Data Retrieval in Bulk for Any Size Another big advantage that cloud has to offer is the tremendous on-demand compute necessary to process large volumes of data and multiple methods to retrieve large volumes of data.
Best Practices with Cloud Email Archival
Ensure All The Emails Are Always Online and Search Ready E-documents are scattered across locations including email servers, PST files on local desktops, laptops, mobile devices, backup devices and other removable media. An email archiving solution, ensures that all these email are collected and stored centrally, much before they are delivered to the user. End User Access via a Self Service Portal to Improve Productivity On-premise email archiving and backup solutions are not attuned to the needs of the end users. In such cases, it can be a colossal task to access information on demand. You would have to rely on your IT helpdesk and follow up (at times for days) to gain access to old email data. An email archiving solution with a self-help portal increases end user productivity and reduces the IT tickets, thereby boosting IT productivity as well. Use Ediscovery to Leverage All The Accumulated Data Ediscovery, is a very important tool which can be used to access the data from a centrally located vault for analysis business intelligence etc. With some quick queries, SMBs can also use the eDiscovery feature to obtain key statistics such as new leads, new accounts closed, customer appreciations, customer feedback, etc. Automate All Integrations with Primary Mail Platform A good archival platform will expose API to allow you to integrate email archiving and ediscovery with other business systems. Recognize that Archive and Backup are different Backups are periodic snapshot of the current state of the mail store and are primarily used to quickly recover an overwritten file or corrupted database and more importantly for disaster recovery. Email Archiving, on the other hand, typically happen in-line or continuously, thereby preventing the chance of missing out data and storing the emails which are no longer in use, but may be required in the future.
How it Works
Email Archiving for Office 365, Google Apps, Exchange and other such solutions
As you may be already aware, Office 365, Google Apps, Microsoft Exchange Server, Exchange Online, Zimbra, Lotus Notes etc are some of the most widely used enterprise email services today. Used by organizations both large and small alike, they also tend to rely on their built-in email archiving capabilities for eDiscovery and compliance. If you are considering moving or have already migrated to them, you should know that their in-built features may not suffice your enterprise’s email archiving needs. With threats on a constant rise and stringent compliance regulations, it is no longer enough to rely on its native archiving feature and therefore, what you need is a third-party email archiving solution.
Vaultastic was built around your vision or ideal state to cater to what enterprises and businesses were looking for. Vaultastic assures that their data has been automatically ingested into alternate immutable cloud storage, with very high durability and deep tamper resistant controls, and assures them a near 100% guarantee that their data is now safe. So what next? Is only a safe backup of all the data, going to help the business improve its bottomline, topline or operations? Here is where the other part of the vision kicks in, where each and every data piece is always online and search ready to return results instantly, which in turn helps the business leverage all that accumulated wealth of information, carrying a large percentage of the organisation’s critical data. Also, traditionally, email archiving has been in the domain of compliance or IT services, which limits its use and puts pressure on the backend to respond to seek requests for historical data. By expanding the scope of usage, safely and securely to all the end users, the accumulated archived data suddenly unlocks trapped value for the business. Each user can leverage the knowledge discovery tools, to leverage trapped knowledge, intelligence and communication, to help deliver better outcomes for the business. Lastly, we’ve specifically focused on tools to enable easy data out for information of any size.