Email Archiving and Compliance
What is an email archive?
It is an automatic means of recording the details and content of every email sent or received by an organisation in a tamper proof and auditable manner.
An appropriate email archive should enable you to ensure compliance with government guidelines and internationally published codes of good practice, including:
- BSI 7799
- BSI PD00008, PD0010, PD5000
- Data Protection Act
- RIP (Regulation of Investigatory Powers Act 2000)
- Employment Law
- SEC 17a
- Encourage adherence to company email usage policy
- Ensure maximum legal admissibility and evidential weight
The email archiving mechanism should also help manage the problem of increasing email storage requirements and provide a means of searching and retrieving archived email.
Who needs email archiving?
Any organisation that
- Regularly exchanges contractual or sensitive documentation
- Could be the subject of Subject Access Requests
- Is required to maintain records and data over a number of years
- Needs to maintain a reliable record of communication in case of future litigation
- Has large message stores which are difficult and time consuming to backup
- Has users constantly exceeding email storage quotas
- Almost any organisation would benefit from the deployment of an email archive.
However it is particularly applicable to organisations in the following market sectors: Finance, Construction, Police, Health, Government, Local Government, Legal, Defence companies.
Email Archiving Emerges As Critical Function
June 3, 2003
By Sharon Gaudin
On any given day, a resident of the City of Orlando can call the municipal offices and ask to see all the emails that, let’s say, the chief of police sent out and received in January 2002. They also might receive a request the same week, or even the same day, from someone asking to see all the emails sent in March relating to the fire department.
For John Matelski, deputy chief information officer for the city, it’s an organizational and storage nightmare. But with the aid of email archiving, he tries to think of it as just a challenge.
“It’s a storage challenge. That’s how I like to look at it,” says Matelski, whose WAN supports more than 100 facilities and his department supports 2,500 email accounts. “Since we live in Florida, we’re subject to the Florida Sunshine Laws, which say that all government activities are open to the public and all government business transactions must be provided… How you do that is up to you. But I can’t print it all out and put it in a folder. Automating that process is critical.”
The City of Orlando is part of a new trend picking up on a relatively old idea — email archiving.
With a majority of business users relying on email more than the telephone when it comes to business communications, an ever-increasing amount of critical information is passing through a company’s email system. Sendmail, Inc., a company that sells email archiving technology, reports that 60% of business critical information is stored within corporate messaging systems. That’s up from 33% from just four years ago. Sendmail also states that an average IT administrator spends five to six hours a week recovering old messages since more than 80% of end users cannot recover them on their own.
Email, in today’s corporate world, holds a company’s critical information. In emails, financial expectations are discussed. Stocks might be suggested. Sales are made. Customers make complaints. Companies make promises.
And it’s all recorded in email.
Now a growing number of businesses are realizing that they need to save these emails. And they just can’t pack them away like receipts you toss in a box at home. They need to be organized. They need to be easily accessible. When an archived email is needed, you don’t want to have to call in half the IT team just to find it.
“There’s a lot of buzz in the industry about email archiving right now and it’s mainly being driven by regulations coming down saying you have to save and be able to retrieve certain kinds of data,” says Doug Chandler, program director for storage services at IDC, an industry analyst firm based in Framingham, Mass. “It’s difficult because if you haven’t been saving this stuff in an organized way, the ability to go in and pull up certain messages from certain days five years ago is not an easy thing to do. You have to have sophisticated software and trained people to manage this process.”
And there are a lot of new regulations — HIPAA, the FDA, SEC, Employee Privacy Regulations. They all have rules for various industries governing what information must be stored away and accessible. The health care industry and financial services are the two generally considered to be the hardest hit. But the idea behind the regulations — credibility and not a little CYA (cover your assets) — is spreading from industry to industry.
Those regulations, coupled with companies’ fears of law suits, is pushing the need to archive. David Ferris, president of San Francisco-based Ferris Research, says there’s a $200 million email archiving market this year. And he expects it to experience 50% to 100% growth over each of the next four years.
For now, Matelski at the City of Orlando is using the archiving feature in Lotus Notes 6.0 to make sure the city’s employees are meeting the requirements of the state’s Sunshine Laws. The emails are archived locally on the individual hard drives, and each employee is provided with a backup mechanism and the training to use it.
Matelski says the employees are able to access and successfully search through their own archives — without the help of the IT department, saving Matelski and his workers an untold amount of time and distraction.
“That would be a great burden,” he says. “If I have to assist any of those 2,500 people, there’s no way I could appropriately staff to accommodate that many requests.”
The deputy chief information officer also says they are looking into buying a stand-alone email archiving package to enable them to archive to a Storage Area Network.
IDC’s Chandler says until they do, Matelski and his team won’t officially be archiving.
“Technically speaking, archiving is treated as a permanent copy being saved in a secondary location,” says Chandler. “I wouldn’t call it archiving if you’re talking about putting it on someone’s PC hard disk. They’re using that storage space everyday. You need to create a permanent copy somewhere so you can save it for five or seven years or more.”
The issue is that it’s an expensive and complex endeavor.
For Tony Spruill, a senior program analyst at Kemet Electronics Corp., a 6,000-employee company based in South Carolina, it’s an expensive proposition that his company is simply going to have to take. If Kemet employees don’t start archiving, the company will have to keep upgrading its email storage capacity. And that process has gotten old fast.
“We’re losing drive space because of the rate that the mail files are growing,” says Spruill, who adds Kemet just upgraded its mail servers and installed a data storage network to deal with their increasingly high mail volume. “We’ll double our space and a year later we’re out of space again.”
Right now, Kemet is employing the archiving feature in Lotus Notes 6.0, just like the City of Orlando. Spruill says it’s helping but it’s only a ‘stop gap’ for the problem. They’ve looked at IBM’s Common Store and like what they’ve seen — all except the price tag. But he’s hopeful they can adopt it soon.
“There’s a lot of document transaction and emails that each department thinks they need to keep forever,” says Spruill. “Accounting, customer sales — they all have emails they need to store away. And for them, archiving is important.”
There seems to be quite a large number of diffrent solutions available for users fo Mircosoft Exchange, Lotus Notes/Domino and Novel Groupware. I do not cover these products since I have no interest in any of them.
I found it very hard to find a solution that will work for an ISP environment running either sendmail or qmail as MTA. Below are a number of possibilities to investigate:
- MHonArc and mharc – MHonArc is a Perl mail-to-HTML converter. MHonArc provides HTML mail archiving with index, mail thread linking, etc; plus other capabilities including support for MIME and powerful user customization features.
- MPP (Message Processing Platform) – PP is an email security application that provides a complete antispam, antivirus and archival solution with a unique middleware approach. MPP ends the rip-out cycle of ineffective antispam products with an upgradeable, modular plug-in architecture. With MPP you set your service and provisioning environment once and plug-in the optimal commercial or open-source scanning technology.
- Hexamail Vault – Hexamail Vault provides a centralized automatic email archiving solution. Highly efficient full-text indexing and compression algorithms ensure that the most scalable and high performance email archive is generated automatically.
- Athena Archiver – Athena Archiver is a comprehensive electronic communications archiving system which helps companies meet their Sarbanes Oxley and SEC electronic record compliance needs. It minimizes the potential for fines, critical downtime, and lost or damaged data.