Data Retention: Email, Email Monitoring and ISPs

Following the recent news articles covering the issues of the government monitoring personal emails, storing personal data, and data retention,  numerous questions have arisen. This article attempts to answer these questions:

What powers does the UK government have to monitor emails at the moment?

Currently most of the powers for monitoring of data come from the Regulation of Interception Powers Act 2000 (RIPA). Which amongst, other things, allows for the interception of communications data.

RIPA requires that ISPs maintain the ability to allow for interception

The Anti-Terrorism, Crime and Security Act provides guidelines for data retention, though it is currently voluntary. The powers under this act have been condemned for overuse, even by the current government.

Do ISPs currently store data?

Yes, they do. There are two reasons for this.

Commercial reasons, obviously the more data they have about individual’s habits the better they can hone their service, and marketing.

Anti-Terrorism, Crime and Security Act. Currently the government has a voluntary code of practice, whereby the ISPs voluntarily collect the data

Who can currently authorize the monitoring of emails?

The authority to monitor emails and intercept communications comes from different people, depending on where the request comes from. For example, if MI5 or MI6 want to intercept communications need the permission of the Secretary of State (Home Secretary). The police, however, only require the permission of survelliance commissioner, under Section 36 of RIPA.

How are the emails intercepted?

Emails are currently intercepted via the ISP (Internet Service Provider). Technical details about this are not released. In the press the method of interception are referred to as “black boxes” at the ISP. In all probability these black boxes are an advanced a network tap/packet sniffer, which pulls out all of the required information for a given protocol. This data i  then probably stored/cached with the ISP and then sent to the government or maintained at the ISP for searching at the location. The latter model would be the more secure, so the government has probably gone for the former. The data is almost certainly indexed, which means that searches would be realtivley quick, seconds rather than days or months.

The ISPs are required under RIPA to provide the ability to maintain interception capability. This means that the government, when required, can monitor any person’s internet activity.

The police also have the powers to access personal computers directly, and covertly. This type of access would allow the monitoring of emails, as well as internet access, screen shots; even key strokes can be recorded.

What new laws are being created to monitor emails?

The government is not actually creating new laws, but rather a statutory instrument. This means that an act of parliament is not required

The statuary instrument, Data Retention (EC Directive) Regulations SI 2007/2199, issued in the UK is based on the EU directive 2006/24/EC which states, under Article 5, what data must be retained.

 EU directive 2006/24/EC, is a European directive the UK are required to transpose it into UK law.

6) What information will the government be collecting from the emails?

a. Currently the plans are to only collect the header information from the emails. i.e. The “To”, “From”, “BCC”, “Subject”, as well as information in the email about IP address it was sent from, how it was sent (Thunderbird, Outlook). This information is known as “traffic” data.

b. Article 5 of the EU directive states that content of the email should not be retained.

7) What is the difference between “traffic” and “communciations” data

a. Traffic” data is information about data that is being transmitted, e.g. IP addresses, phone numbers, to, from etc. This defined by RIPA and more information is available here

b. Communications” data is the actual body of the data package being sent.

c. Example. If an email was sent from Person A to Person B, the information about Person A, IP address, email address, subject of the email, and the email of Person B would be the “traffic” data. But the content of the actual email, the message, would be the “content”.

8 ) Will the government be reading the content of the email or header?

a. Currently the UK Government is only planning to store the “traffic” data, i.e. the header information. It should be emphasized that while only traffic data is stored both content and traffic can be intercepted and can be monitored

9) How long will the email data be retained for?

a. This email header information is to be detained for 12 months (1 year), minimum. But no more than 24 months (2 years).

b. This figure comes from the Data Retention (EC Directive) Regulations SI 2007/2199, which states that: [Email Traffic] data must be retained for a period of 12 months, in accordance with regulation 4(2). The data must be stored in accordance with the requirements in regulation 7.

10) Why did the government change the laws?

a. The government changed the laws for several different reasons, depending on your political perspective. Some of the documented reasons are below:

b. The EU Directive, in March 2006,  required nation states to have greater monitoring of email and internet traffic

c. Based on the EU Directive, the UK transposed this into UK law, via the statutory instrument 2007/2199

d. In December 2007 the UK government published a document entitled the Next Generation Telecoms Networks. This pointed out the failings of RIPA, because as networks have become more and more capable, it has been harder to monitor the communications traffic. The document states: “Under the Regulation of Investigatory Powers Act 2000,communications providers must allow lawful interception by police and intelligence services where reasonably practicable. This may become more difficult with NGNs. A phone call over the PSTN can be intercepted with a tap anywhere along the line dedicated to the call, but in an NGN, packets may travel along many different paths. However, there are points where traffic can be intercepted, and 21CN will allow lawful interception. The Home Office’s Interception Modernisation Programme aims to ensure that NGNs and other developments in communications do not impede lawful interception”

e. In short, the government feels it is losing control of the communications and want to able to tap into communications anywhere at anytime.

11) How much will this cost?

a. The current estimates for the Interception Modernisation Programme are estimated at £12 billion. But, as with all government projects, particularly IT projects, these figure can expect to increase radically. It will no doubt be closer to £20 billion before its finished

12) Has the government ever misused data it has collected before?

a. Yes, lots and regularly. In fact most databases appear to have been misused at sometime or another. Examples of data misuse are here.

13) Could the government lose the email data, or will it be secure?

a. It’s been reported on numerous occasions that the government has lost data many many times. Examples of data loss are here.

14) How much information can the government obtained from just the email addresses?

a. A lot. From the email subject, IP addresses, and email addresses the government will be able to generate a lot useful information. They will be able to build up who is talking to who, frequency of communication and link those to IP addresses.

b. Cross referencing the email addresses with searches on forums, social networking sites, and other databases will bring together greater information for the government to data mine.

c. The IP addresses alone can be used to great effect, and combined with entries in the search engine databases, i..e who has been searching for what, they can tell a lot about the user.

d. Finally, and perhaps most importantly, the email addresses, will build up a network of contacts for each person and so could be used for a fishing expedition.

e. The commonly held belief of a maximum “Six Degrees of separation” between any two pepople, which has been shown to be true on several occasions, could be used against any person using email. Based on the “6 degrees theory” it stands to reason that any person in the UK is linked to a “terrorist” by, at most, 6 other people. With the onset of huge social networking sites, mass emails, and bookmarking sites, its likely that many people will receive an email or be connected to a terrorist within a couple of steps. I.e. a perfectly innocent person may be just 1 step away from somebody involved with an extremist group. This would give the police the power to intercept the innocent individuals email, both content and traffic data as they are “linked” to the terrorsist.

15) How can I avoid my emails being read?

a. The technology to be put in place (or already in place). Allows the government to retain data on email traffic, but monitor email content as and when required. This cannot be stopped, but security can be put in place.

b. You can’t hide your email address nor can this be encrypted, it has to be sent in plain text (it’s the nature of the internet). But you can try using multiple email accounts, one for work friends, one for network friends, one for purchases, etc. Doing this makes it harder to link your different groups together; but not impossible

c. Encrypt your email content. You cannot encrypt the email traffic, but you can encrypt the content.

d. Use none-decrypt subject titles: The subject title will be an important part of the traffic data, but if you are use none-descript ones e.g “Test1” “Test2”, then this will make it harder to understand what you are talking about. Remove the “Re” or “Fw” from the subject title, this again limits the information available from monitoring the subject title

e. Change your IP address: Currently all the tools available to the public, e.g. Tor, only hide your IP address for web browsing not for email. Therefore your true IP address will still be recorded when you use your email. But, by hiding your IP address in web browsing it is harder to link your web browsing to your emailing.




Legislation Relating to Data Retention in the UK

EU approves law for 2 year data retention (2005)

In December 2005 the European Parliament approved rules forcing telephone companies to retain call and internet records for use in anti-terror investigations. The law allows records to  be kept for up to two year.

Police will have access to information about calls, text messages and internet data, but not the call content.

The UK, which pressed European member states to back the rules, said that data was the “golden thread” in terrorist investigations.

The parliament voted by 378 to 197 to approve the bill, which had already been agreed by the assembly’s two largest groups, the European People’s Party and the Socialists.

BBC Article

ISP Data Retention (US)

A survey by Wired Magazine was conducted in 2007 to find out about the privacy policies of the top ISPs. Out of the 8 largest ISPs asked the 10 question survey in the US only 4 responded: AOL, AT&T, Cox and Qwest. Comcast, EarthLink, Verizon and Time Warner didn’t respond to the survey at all.

IP Retention

Cox’s IP Log retention times is: six months.

AOL IP log retention time is: “limited period of time,”

AT&T IP log retention time si  “within industry standards.”

URL Retention

The question of “how long are the URLs” retained was asked of the companies. The URLs contain a lot of detail about an individuals habits: What they read, buy, and like how often, and how much.  w

AOL, AT&T and Cox all statd that they do not the URLs at all. Qwest avoided the question.

ISPs Opinion on Data Retention

Qwest said that the market should decide how long data is kept

Cox stated it was “studying the issue” of data retention

AOL stated it isis working with the industry and Congress.

A&T stated it is  “ready to work with all parties.”

In the UK the data retention laws of ISPs are currently governed by the Retention of communications data under part 11: Anti-Terrorism, Crime & Security Act 2001

Data Retention and Interception

Data Retention

(More detailed information on these retention times is available here)

In December 2001, the Parliament approved the Anti-terrorism Crime and Security Act 200 (ATCS)

This law allows the Home Secretary to issue a code of practice for the voluntary retention of communications data by communications providers” for the purpose of protecting national security or preventing or detecting crime that relates to national security.

It only applies to data that is already being held by the Communication Service Providers (e.g ISP/telecomms) for business purposes. The Code of Practice was first approved in December 2003

The government has since proposed modifying the ATCS and RIPA to make data retention mandatory and expanding its use to include serious crimes, not just terrorism offenses.

A leaked submission by the police and intelligence services to the Home Office in 2000 proposed a seven-year data retention policy, however this has not been followed up and the current voluntary times remain.

Despite the government constantly pushing data retention times, and increasing surveillance and interception, to stop the ever present threat of terrorism, the reality is that the data will almost certainly be used for reasons other than prevention and detection of terrorism.

An opinion commissioned by the Information Commissioner’s Office (ICO) found that the access to information retained under the act for non-national security purposes would violate human rights and would be unlawful.

Who has access to this data?

Despite this the government had fully intended to allow a whole host of government agencies to access the data, from local police to the  local council. And in June 2002, the Home Office stated that the list of government agencies allowed under RIPA to access communications data was being extended to over 1,000 different government departments including local authorities, health, environmental, trade departments and many other public authorities.

The ICO stated that “I clearly cannot carry out meaningful oversight of so many bodies without assistance“, following this and the pubic outcry of so many people accessing so much information the then Home Secretary (David Blunkett) withdrew the order. Though the governments intentions and mindset were clear. Monitor everyone and give access to even the smallest council.

Data Retention Times

The code provides for the following retention time periods:

  • SMS, EMS and MMS: Data retention period 6 months.
  • Email: Data retention period 6 months
  • ISP Logs: Data retention period 6 months
  • Web Activity Logs: Data Retention period 4 day

More detailed information on these retention times is available here.

Interception of Communications

Before the ATCS 2001 Act was created the government created RIPA, Regulation of Investigatory Powers Act, which covers a variety of aspects including encryption and interception of communications. Section 12 of RIPA makes it an obligation of CSP (Communication Service Providers) to maintain an ability to intercept traffic” and “content” of communications, which then allows the govermenment to monitor communications as and when needed, or the in the case of Echelon, all of the time.

An explanation of the terms “traffic” and “content” in relation to RIPA are available on other posts on this site.

RIPA is often in the news for its repeated misuse by councils, from covertly following families, to ensure they go to the right school, to setting up cameras and covert surveillance to monitor dog fouling.

Data Retention: Anti-Terrorism, Crime and Security Act

Currently the home office has put in place a voluntary code of practice for ISP and telecommunication service providers relating to the retention of data this is comes under the “Retention of communications data under part 11: Anti-Terrorism, Crime & Security Act 2001

The code provides for the following retention time periods:

  • SMS, EMS and MMS: Data retention period 6 months.
  • Email: Data retention period 6 months
  • ISP: Data retention period 6 months
  • Web Activity Logs: Data Retention period 4 days

The following data is required to be stored for the retention times mentioned above:

SMS, EMS and MMS: Calling number, IMEI – Called number, IMEI – Date and time of sending – Delivery receipt – if available – Location data when messages sent and received, in form of lat/long reference.

Email: Log-on (authentication user name, date and time of log-in/log-off, IP address logged-in from) – sent email (authentication user name, from/to/cc email addresses, date and time sent) – received email (authentication user name, from/to email addresses, date and time received)

ISP: Log-on (authentication user name, date and time of log-in/log-off, IP address assigned, Dial-up: CLI and number dialed, Always-on: ADSL end point/MAC address (If available)

Web Activity Logs: Proxy server logs (date/time, IP address used, URL’s visited, services)

The code is quite clear that information stored should on be “Communications Data” only and exclude content of communication.

The Web browsing information to be retained should only be to the extent that only the host machine or domain name is disclosed.

The example the Home Office gives is that if the URL visited was

then only the domain “” is to be stored . The reason is that the:

within a communication, data identifying would be traffic data, whereas data identifying would be content and not subject to retention.

Communications Data – RIPA

Communications Data is defined by RIPA as any of the following:
(i) any traffic data comprised in or attached to a communication (whether by the sender or otherwise) for the purposes of any postal service or telecommunication system by means of which it is being or may be transmitted;
(ii) any information which includes none of the contents of a communication [apart
from any information falling within paragraph (i)] and is about the use made by any
(1) of any telecommunications service; or
(2) in connection with the provision to or use by any person of any
telecommunications service, of any part of a telecommunication system;

(iii) any information not falling within paragraph (i) or (ii) that is held or obtained, in
relation to persons to whom he provides the service, by a person providing a
telecommunications service.