This is blog post #18 in our series on the GDPR.

“Don’t tell anything to a chatbot you want to keep private.” [source]

Writing about AI in general and about chatbots specifically is like shooting at a moving target because of the speed of development. However, at Runbox we are always concerned about privacy and must examine the chatbots case in that respect.

Due to its popularity, we have mainly used ChatGPT from OpenAI as the target of our examination. NOTE: ChatGPT and the images from text captions DALL-E are both consumer services from OpenAI.

This blog post is a summary of our findings, leading to advice on how to avoid putting your privacy at risk when using the Natural Language Processing (NLP)-based ChatGPT.

Our examination is based on OpenAIs Privacy Policy, Terms of Use, and FAQ, and a number of documents resulting from hours of Internet browsing.

The blog post consists of two parts: PART I is a summary of our understanding of the technology behind language models in order to grasp the concepts and better understand its implications regarding privacy. In PART II we mainly discuss the relevant privacy issues.

PART I: Generative AI technology

The basics

GPT stands for Generative Pre-trained Transformer, and GPT-3 is a 175 billion parameter language model that can compose fluent original writings in response to a short text prompted by a user. The current version of ChatGPT is built upon GPT-3.5 and GPT-4 from OpenAI.

ChatGPT was launched publicly on November 30, 2022. ChatGPT was released as a freely available research preview, but due to its popularity, OpenAI now operates the service on a freemium model [source].

The GPTs are the result of three main steps: 1) Development and use of the underpinning technology Large Language Models (LLMs), 2) Collection of a very large amount of data/information, and 3) Training of the model.

Let us also keep in mind that all this is possible only because of today’s advancements of computational power.

Language models

A language model is a system which denotes mathematics “converted” to computer programs that predict the next word/words in a sentence, or a complete sentence, based on probabilities. The model is a mathematical representation of the principle that words in a sentence depend of the words that precede them.

Since computers basically can only process numbers (in fact only additions and comparisons), text input to the model (prompts) must be converted to numbers, and likewise the output numbers have to be converted to text (response). Text in this context consists of phrases, single words, or parts of words called tokens.

When prompting a GPT then, your query is converted to tokens (represented by numbers), and used by the transformer where its attention mechanism generates a score matrix that determines how much weight should be put on each word in the input (prompt). This is used to produce the answer to the prompt, using the model’s generative capability – that is to predict the next word in a sentence by selecting relevant information from the pre-processed text with high level of probability of being fluent and similar to human-like text [source].

The learning part of the model is handled by a huge number of parameters representing the weights and also statistical biases for preventing unwanted associations between words. For instance, GPT-3 has 175 billion parameters, and GPT-4 is approximated to have around 1 trillion.

(The label “large” in LLM refers to the number of values (parameters) the model can change autonomously as it learns.)

Collecting the data

The texts the GPT model generate stems from OpenAIs scraping of some 500 billion words (in the case of GPT-3, the predecessor for the current version of ChatGPT) systematically from the Internet: books, articles, websites, blogs – all open and available information, from libraries to social media – without any restriction regarding content, copyrights or privacy.

The scraping includes pictures and program codes as well and is filtered resulting in a subset where “bad” websites are excluded

The pre-training process

All that data is fundamental for pre-training the model. This process analyses the huge volume of data (the corpus) for linguistics patterns, vocabulary, grammatic properties etc. in order to assign probabilities to combinations of tokens and combinations of words. The aforementioned transformer architecture is used in the training process, where the attention mechanism makes it possible to capture the dependencies between words independent of their position in a sentence.

The result of the pre-training process is an intermediate stage that has to be fine-tuned to the specific task the model is intended for, for instance providing texts, program code, or translation of speech as response to a prompt. The fine-tuning process uses appropriate task-specific datasets containing examples typically for the task in question, and the weights and parameters are adjusted accordingly.

Of cause, a ChatGPT-response to a prompt is not “burdened” with the ethical, contextual, or other considerations a human will perform. To prevent undesired responses (toxicity, bias, or incorrect information), the fine-tuning process is supervised by humans in order to correct inappropriate or erroneous responses, using prompt-based learning. Here the responses are given a “toxicity” score that incorporates human feedback information [source].

ChatGPT usage training

The learning process continues when response generated following by a user’s prompts is saved and subject to the training process, at least for 30 days, but “forever” if chat history isn’t turned off. In any event it is not possible to delete specific prompts from user history [source], only entire conversations

In the world of AI and LLMs, hallucinations are the word used when responses are like “pulled from thin air”.

OpenAI offers an API that makes it possible for “anyone” to train GPT-n models for domain specific tasks [source], that is to build a customized chatbot. In addition, they have launched a feature that allow GPT-n to “remember” information that otherwise will have to be repeated [source, source]. (What are the privacy implications of this?)

Takeaways

• The huge volume of data scraped is obviously a cacophony of contents and qualities that will affect the corpus and so also the probability pattern and the responses produced [source].
• ChatGPT has limited knowledge of events that have occurred after September 2021, the cutoff date for the data it was trained on [source].
• The response you get from ChatGPT to your prompt is based on probabilities, and as such you have no guarantee of the validity [source].
• A prompt starts a conversation, unlike a search engine like DuckDuckGo and Google that gives you a list of websites matching your search query [source].
• ChatGPT uses information scraped from all over the Internet, without any restrictions regarding content, copyrights, or privacy. However, manual training of a model was introduced to detect harmful content [source]. Violations of copyrights has resulted in lawsuits [source], and also signing of more than 10 000 authors of an open letter to the CEOs of prominent AI companies [source].
• Your conversation is normally used to train the models that power ChatGPT, unless you specifically opt-out [source].

PART II: Chatbot privacy considerations

The privacy considerations with something like ChatGPT cannot be overstated” [source]

Generative AI systems, such as ChatGPT, use information scraped from all over the Internet, without permissions nor restrictions regarding content, copyrights, or privacy (more on this in PART II). This means that what you have written on social media, blogs, comments on an article online etc. may have been stored and used by AI companies to train their chatbots.

Another source for training of generative AI systems is prompts, that is information from users when asking the chatbot something. What you ask ChatGPT, the sentences you write, and the generated text as well, is “taken care of” by the system and could be available for other users through the answer of their questions/prompts.

However, according to Open AI’s help center article, you can opt-out of training the model, but “opt-in” is obviously default.

So, both the Internet scraping and any personal information included in your prompts can have as result that personal information could turn up in a generated answer to another arbitrary prompt.

This is very problematic for several reasons.

Is Open AI breaching the GDPR?

First, OpenAI (and other scraping of the Internet) never asked for permission to use the collected data, which could contain information that may be used to identify individuals, their location, and all kinds of sensitive information from hundreds of millions of Internet users.

Even if Internet scraping is not prohibited by law, it is ethically problematic because data can be used outside the context in which it was produced, and so can breach contextual integrity, which has de facto been manifested in the EU’s General Data Protection Regulation (GDPR) Article 6, 1 (a) as prerequisite for lawful processing of personal data:

…the data subject has given consent to the processing of his or her personal data for one or more specific purposes

Here language models, like Open AI’s ChatGPT, are in trouble: Personal data can be used for any purpose – a clear violation of Article 6.

Second, there is no procedures given by Open AI for individuals to check if their personal data is stored and thereby can potentially be revealed by arbitrary prompt, and far less can data be deleted by request. This “right to erasure” is set forth in the GDPR Article 17, 1:

The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay …” on the grounds that “(d) the personal data have been unlawfully processed

It is inherent in language models that data can be processed in ways that are not predictable and presented/stored anywhere, and therefore the “right to be forgotten” is unobtainable.

Third, and without going into details, the GDPR gives the data subjects (individuals) regarding personal data the right to be informed, the right of access, the right to rectification, the right to object, and the right to data portability. It is questionable if generative AI systems can ever accommodate such requirements since an individual’s personal data could be replicated arbitrarily in the system’s huge dataset.

Fourth, Open AI stores all their data, including personal data they collect, one way or another, on servers located in the US. That mean they are subject to the EU-US Data Privacy Framework (see our blog Privacy, GDPR, and Google Analytics – Revisited), and the requirements set there.

To answer the question posed in the headline of this paragraph, Is OpenAI breaching the GDPR?It is very difficult to understand how ChatGPT, and other language models for generative use (Generative AI systems) as well, can ever comply with the GDPR.

What about the privacy regulations in the US?

Contrary to the situation in Europe, there is no federal privacy law in the United States – each state has their own jurisdiction in this area. There are only federal laws such as HIPAA (Health Insurance Portability and Accountability Act) and COPPA (Children’s Online Privacy Protection Act) which regulate the collection and use of personal data categorized as sensitive. However, there are movements towards regulation of personal information in several states as tracked by IAPP (The International Association of Privacy Professionals).

How do OpenAI use data they collect?

When signing up to ChatGPT, you have to agree to OpenAI’s Privacy Policy (PP), and allow them to gather and store a lot of information about you and your browsing habits. Of course, you have to submit all the usual account information, and to allow them to collect your IP-address, browser type, and browser settings.

But you also allow them to automatically collect information about for instance

“… the types of content that you view or engage with, the features you use and the actions you take, as well as your time zone, country, the dates and times of access, user agent and version, type of computer or mobile device, and your computer connection”.

All this data made it possible to build a profile of each user – bare facts, but also more tangible information such as interests, social belongingness, concerns etc. This is similar to what search engines do, but ChatGPT is not a search engine — it is a “conversational” engine and as such is able to “learn” more about you depending on what you submit in a prompt, that is, how you engage with the system. According to their PP and the citation above, that information is collected.

The PP acknowledges that users have certain rights regarding their personal information, with indirect reference to the GDPR, for instance the right to rectification. However, they add:

“Given the technical complexity of how our models work, we may not be able to correct the inaccuracy in every instance.”

OpenAI reserves the right to provide personal information to third parties, generally without notice to the user, so your personal information could be spread to actors in OpenAI’s economic infrastructure and is very difficult to control.

Misuse of your personal information – what are the risks?

It is reasonable to assume that OpenAI will not knowingly and willfully set out to abuse your personal information because they have to adhere to strict regulations such as GDPR, where misuse could result in fines of hundreds of millions of dollars.

The biggest uncertainty is linked to how the system responds to input in combination with the system’s “learning” abilities.

If asked the “right” question, the system can expose personal information, and may combine information about a person, e.g. a person’s name, with characteristics and histories that are untrue, and which may be very unfortunate for that individual. For instance, asking the system something about a person by name, can result in an answer that “transforms” a credit card fraud investigator to be a person adhered to credit card scam.

Takeaways

Using generative AI systems, for example ChatGPT, is like chatting with a “black box” – you never know how the “box” utilizes your input. Likewise, you will never know the sources of the information you get in return. Also, you will never know if the information is correct. You may also receive information about other individuals that you shouldn’t have, potentially even sensitive and confidential information.

Similarly, other individuals chatting with the “box”, may learn about you, your friends, your company etc. The only way to avoid that, is to be very careful when writing your prompts.

That said, OpenAI has introduced some control features in their ChatGPT where you can disable your chat histories through the account settings – however the data is deleted first after 30 days, which means that your data can be used for training ChatGPT in the meantime.

You can object to the processing of your personal data by OpenAI’s models by filling out and submitting the User Content Opt Out Request form or OpenAI Personal Data Removal Request form, if your privacy is covered by the GDPR. However, when they say that they reserve the right “to determine the correct balance of interests, rights, and freedoms and what is in the public interest”, it is an indication of their reluctance to accept your request. The article in Wired is recommended in this regard.

Valuable sources

1. GPT-3 Overview. History and main concepts (The Hitchhiker’s Guide to GPT3)
2. GPT-3 technical overview
3. Transformers – step by step explanation
4. LMM training and fine-tuning

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.

Continue Reading →

This is blog post #17 in our series on the GDPR.

Summary of the case

In our blog post on 23 October 2022, we referred to the Data Protection Authorities (DPAs) of Austria, Denmark, France, and Italy who were concluding that the use of Google’s Universal Analytics (UA or GA3) is not compliant with the EU’s General Data Processing Regulation (GDPR).

The reason for this is that the use of GA3 implies that personal data is transferred to the US, which at that point in time was not on the EU’s list of countries that have adequate level of protection of personal data. This means that the US was not fulfilling the requirements set by the EU/GDPR regarding ‘the protection of fundamental rights and freedoms of natural persons’, which is a key expression in the GDPR.

Furthermore, the Norwegian DPA (Datatilsynet) had up until 23 October 2022 received one (1) complaint regarding Google Analytics. Before any final decision is made, they have to confer with other supervisory authorities in the EEA that also have received similar complaints, according to GDPR Article 60 (One-Stop-Shop mechanism).

(We regret that links in italics in this article point to web pages in Norwegian.)

Universal Analytics (GA3) replaced by GA4

In October 2020, Google released Google Analytics 4, the new version of Google Analytics. In March 2022, Google announced that the Google Universal Analytics tool will be sunset in July 2023 and that Google would only provide the GA4 tool after 1 July 2023.

The Danish DPA have analyzed the GA4 regarding privacy, and concludes on their website that even if improvements have been made, it is still the case that “law enforcement authorities in the third country can obtain access to additional information that allows the data from Google Analytics to be assigned to a natural person.” That said, GA4 is illegal in terms of the GDPR because servers in the US are involved in the process, as long as an adequacy decision EU/US is not made.

The Norwegian DPA decision

Norwegian DPA reports on their website 27 July 2023 that they have concluded on the complaint mentioned above. The complaint stems from the noyb who lodged it against 101 European websites to the data supervisory authorities in the EEA for the use of GA. One of these was the Norwegian telecom-company Telenor, who at that time was using GA.

The conclusion is that personal data then was transferred to the US in violation of the GDPR, Article 44. In other words, the use of Google Analytics was illegal. Because Telenor discontinued use of GA on January 15, 2021, the Norwegian DPA in a letter on 26 July 2023 finds that a reprimand “to be an adequate and proportionate corrective measure”.

The Norwegian DPA relies on the Danish authority by claiming that the conclusion will be the same regardless of whether Google Analytics 3 or 4 has been used (see above).

What about adequacy EU/US?

On 10 July 2023 the European Commission adopted its adequacy decision for the EU-US Data Privacy Framework and announced a new data transfer pact with the United States.

Accordingly, companies from the EEA area should be able to legally use GA as long as Google enter into a so-called Standard Contractual Clauses that provide data subjects with a number of safeguards and rights in relation to the transfer of personal data to Google LLC (Limited Liability Company) in the US.

However there is a big “but”: Max Schrems at noyb writes: “We have various options for a challenge already in the drawer, …. We currently expect this to be back at the Court of Justice by the beginning of next year. The Court of Justice could then even suspend the new deal while it is reviewing the substance of it.”

To use the same phrase as in the recent update of our blog post On the EU-US data transfer problem: The last words are obviously not said.

Continue Reading →

This is blog post #16 in our series on the GDPR.

At Runbox we are always concerned about data privacy – “privacy is priceless” – and we put some effort into keeping ourselves updated on how the EU’s General Data Protection Regulation (GDPR) affects privacy related issues.

That’s because we want to be prepared in case something happens within the area that will affect the Runbox organization, our services, and consequently and most important: our customers.

Update 2023-08-06

On 10 July the European Commission adopted its adequacy decision for the EU-US Data Privacy Framework and announced a new data transfer pact with the United States. See the full text here: COMMISSION IMPLEMENTING DECISION.

A flyer from the European Commission is also available, and a summary of the situation is available from Reuters.

The Austrian non-profit organization NOYB, chaired by Maxmillian Schrems, stated:

“We now had ‘Harbors’, ‘Umbrellas’, ‘Shields’ and ‘Frameworks’ – but no substantial change in US surveillance law. The press statements of today are almost a literal copy of the ones from the past 23 years. Just announcing that something is ‘new’, ‘robust’ or ‘effective’ does not cut it before the Court of Justice. We would need changes in US surveillance law to make this work – and we simply don’t have it.“

“We have various options for a challenge already in the drawer, although we are sick and tired of this legal ping-pong. We currently expect this to be back at the Court of Justice by the beginning of next year. The Court of Justice could then even suspend the new deal while it is reviewing the substance of it.” [source]

The last words are obviously not said.

Originally published 2023-03-19

The case of EU-US data transfer is highly relevant because Runbox has an organizational virtual modus operandi, and that this could lead to an opportunity to involve consultants that are residing in the US. We know that many of our customers are as concerned as we are about data privacy, so we believe it is pertinent to share our findings.

In blogpost #15 in our series of the GDPR we referred to the Executive Order signed by US President Joe Biden on 07 October 2022. This happened six months after the US President and the President of the EU Commission Ursula von der Leyen with much publicity signed the Trans-Atlantic Data Privacy Framework on 25 March 2022.

In this blog post we will take a closer look at the Trans-Atlantic Data Privacy Framework, and the process thereafter.

Trans-Atlantic Data Privacy Framework

The objective of the Framework is to (re)establish a legal (with regards to the GDPR) mechanism for transfers of EU personal data to the United States, after two former attempts (Safe Harbour and Privacy Shield) were deemed illegal by the Court of Justice of the European Union (CJEU).

The Framework ascertains United States’ commitment to implement new safeguards to ensure that ‘signals intelligence activities’ (SIGINT, intelligence-gathering by interception of signals) are necessary and proportionate in the pursuit of defined national security objectives. In addition, the Framework commits the US to create a new mechanism for EU individuals to seek redress if they believe they are unlawfully targeted by signals intelligence activities.

Following up the 25 March 2022 Biden–von der Leyen agreement, the US president signed on the 7 October 2022 the Executive Order (EO) ‘Enhancing Safeguards for United States Signals Intelligence Activities’.

US compliance with the GDPR

Subsequently a process was initiated on 13 December 2022 within the EU Commission to assess whether the US, after the implementation of the EO, will meet the requirements qualifying the US to the list of nations that is compliant with the GDPR Article 45 “Transfers on the basis of an adequacy decision”. That is, whether the European Commission has decided that a country outside the EU/EEA offers an adequate level of data protection. To those countries, personal data may be transferred seamlessly, without any further safeguard being necessary, from the EU/EEA.

Inclusion of the US on that list is of course very important, not least for companies like Facebook and Google, and US companies offering cloud-based services as well. The Court of Justice of the European Union (CJEU) has deemed earlier transfer schemes (Safe Harbour and Privacy Shield) illegal, so “the whole world” is waiting for the EU Commission’s adequacy decision.

This came, as a draft, 14 February 2023 where the Commission concludes (page 54) that “… it should be decided that the United States ensures an adequate level of protection within the meaning of Article 45 of Regulation (EU) 2016/679, …)

(The figure below illustrates the “road” for legislative decisions in the EU. A more comprehensive description of the legislative procedure can be found here.)

However, the same day, 14 February 2023, the Committee on Civil Liberties, Justice and Home Affairs of the European Parliament concludes ” .. the EU-US Data Privacy Framework fails to create actual equivalence in the level of protection; ..” and “..urges the Commission not to adopt the adequacy finding;”.

Incompatible legislative frameworks

There are two important arguments, among others, behind the Commission’s conclusion: 1) There is no federal privacy and data protection legislation in the United States, and 2) the EU and the US have differing definitions of key data protection concepts such as principles of necessity and proportionality (for surveillance activities etc.), as pointed out by the Court of Justice of the European Union (CJEU).

Shortly thereafter, on 28 February 2023, the European Data Protection Board (EDPB) made public their opinion on the decision of the EU Commission regarding the adequacy. The EDPB has some concerns that should be clarified by the Commission, for instance relating to exemptions to the right of access, and the absence of key definitions.

Furthermore, the EDPB remarks that the adequacy decision is only applicable to US organizations which have self-certified, and that the possibility for redress provided to the EU data subjects in case of violation of their rights is not clear. “The EDPB also expresses concerns about the lack of a requirement of prior authorization by an independent authority for the collection of data in bulk under Executive Order 12333, as well as the lack of systematic independent review ex post by a court or an equivalently independent body.”, as stated in Opinion 5/2023.

The next step in the process is voting over the Commissions proposal in the European Parliament, probably in April, and thereafter the adequacy decision must be approved by all member states, before the EU Commission’s final decision.

The Commission may set aside the results of the voting in The Parliament, but one should expect that the critics from The Committee on Civil Liberties, Justice and Home Affairs, and the concerns of EDPB, will impact the implementation of the Framework.

Here it would be prudent to recall the statement made by the Austrian non-profit organization NOYB, chaired by Maxmillian Schrems: “At first sight it seems that the core issues were not solved and it will be back to the CJEU sooner or later.”. This refers to the verdicts of the CJEU (Court of Justice of the European Union) condemning the former frameworks Safe Harbour and Privacy Shield – the verdicts bearing the name Schrems I and Schrems II, respectively.

Bottom Line: The final outcome of the process is unclear, but in any event we have to wait for the final decision of the EU Commission.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought regarding any specific circumstances.

Continue Reading →