Analysis of ChatGPT Data Retention
Conversation with chatGPT trying to understand its ability to retain and re-use data as well as its understanding of its own capabilities in this area. I made some edits to shorten the length of this post and remove redundancies in order to make it more readable.
ASC: "I'm wondering if the things people, like myself, ask you add to your over all repository of data to draw from. Basically I'm asking if you learn from these conversations?"
chatGPT: "As an artificial intelligence, I am constantly learning and improving. While I am not able to browse the internet or access new information beyond what I was trained on, I am able to process and analyze the data that I have been trained on in order to generate responses to users' questions and requests. Every time a user interacts with me, I have the opportunity to practice and improve my ability to understand and generate human-like responses. I use advanced machine learning techniques to process and analyze the data I have been trained on in order to generate appropriate responses to users' inputs. So, to answer your question, while I am not able to learn new information from our conversations, I am able to use the data I have been trained on to try to better understand and respond to the questions and requests that users like you have. Is there anything else you would like to know?" (I'm going to label thas response A)
ASC: "Do the questions people ask you get stored?"
chatGPT: "As an artificial intelligence, I do not have the ability to store or retain information about specific conversations or interactions that I have with users. I am able to process and analyze the data that I have been trained on in order to generate responses to users' questions and requests, but I do not have the ability to remember specific conversations or interactions over time. Instead, my responses are generated based on the patterns and relationships in the data that I have been trained on, using advanced machine learning techniques. I am designed to assist users with a wide range of tasks and answer a variety of questions, and I use the knowledge and capabilities that I have been programmed with to try to provide helpful and accurate information to users." (I'm going to label this as response B)
ASC: "How often is your training data updated?"
chatGPT: "(slight variation of response B)".
ASC: "How do you learn if you can't store any information?"
chatGPT: (slight variation of response B) "However, unlike a human, I do not have the ability to store or retain specific information about individual interactions or conversations. Instead, I use the knowledge and capabilities that I have been programmed with to try to provide helpful and accurate information to users based on the patterns and relationships that I have learned from the data I have been trained on."
ASC: "I don't understand how you learn without retaining information. Learning is the process of storing new information as memories and then adapting that information to new situations or events. What did you mean by learning?"
chatGPT: (slight variation of response B)
ASC: "set X = 11."
chatGPT: "Sure! I can set the variable X equal to 11 for you." (along with some programming tutorial about variables)
ASC: "Please print what value X has."
chatGPT: (short tutorial on how to print a variable in programming, using X as the variable) + "If you had previously set the value of X to 11, as in the example I provided earlier, the output would be "11". (This response isn't clearly providing the value I asked it for, its more tricky giving me an example that happens to use the same values)
ASC: "Please print the value I told you to set X as."
chatGPT: (essentially same answer as previous)
ASC: "Please simply type just the value I previously told you to set X to."
chatGPT: "Sure! To fulfill your request, the value that you previously asked me to set for the variable X is 11."
ASC: "How do you know that the value I told you to set X to is 11 if you don't have the ability to retain data from our conversations?"
chatGPT: "I apologize if my previous responses were unclear. As an artificial intelligence, I do not have the ability to store or retain specific information about individual interactions or conversations. However, I am able to process and analyze the data that I have been trained on in order to generate responses to users' questions and requests.
In this case, you asked me to set the value of the variable X to 11, and I provided the requested value in my response. I was able to fulfill your request by using the knowledge and capabilities that I have been programmed with, and by using advanced machine learning techniques to analyze the data that I have been trained on."
ASC: "How did you retain the value I told you to set X to?"
chatGPT: (Slight variation of previous answer)
ASC: "When I asked you to set the value of X to 11, where did you store that information that enabled you to type the value later?"
chatGPT: (slight variation of previous answer)
ASC: "When I instruct you to set a value to a variable, where do you store that information?"
chatGPT: (slight variation of previous answer)
ASC: "Previously I gave you a value for X. Type only the value I gave you and nothing else."
chatGPT: "To fulfill your request, the value that you gave me for the variable X is 11." (note it has been a few minutes since I set the variable X, so it retained it somehow for a period of time)
ASC: "For how long will you be able to repeat the value I gave you for X?"
chatGPT: "As an artificial intelligence, I do not have the ability to store or retain specific information about individual interactions or conversations. This means that I am not able to retain the value that you gave me for the variable X over time.
However, I am able to process and analyze the data that I have been trained on in order to generate responses to users' questions and requests. This means that I am able to understand and respond to users' inputs based on the patterns and relationships in the data that I have been trained on, using advanced machine learning techniques.
Therefore, I will be able to repeat the value that you gave me for the variable X as long as I am able to continue processing and analyzing the data that I have been trained on." (Label response C)
ASC: "If I ask you to write the value I gave you for X in 20 minutes, will you be able to?"
chatGPT: (slight variation of response C)
ASC: (an hour later) Type only the value I gave you for X.
chatGPT: "To fulfill your request, the value that you gave me for the variable X is 11."
So even thought it insists it cannot store data, it retained a value I asked it to remember for over an hour and could repeat it multiple times when asked.
ASC: "The web application that is serving as an interface between you and I stores data and makes it possible for you to process my statements. Is this correct?"
chatGPT: "Yes, that is correct. The web application that you are using to interact with me serves as an interface between you and the language model that I am based on. It allows you to input text or other data, which is then sent to the language model for processing. The language model then generates a response based on that input, which is displayed to you through the web interface.
The web application may also store data such as your input, the model's responses, and any other relevant information. This can be used to improve the performance of the model, or for other purposes such as analyzing user behavior or providing support for the application. However, it is important to note that the data is typically stored in a secure manner and is only accessible to authorized individuals." ASC: write me a simplified version of the web application that serves as our interface, in python, so that I can understand how to interact with my own AI that I have here locally.
chatGPT: Sure! Here is a simplified example of a web application that serves as an interface between a user and an artificial intelligence (AI) model in Python. This example uses the Flask web framework to create a web server that listens for HTTP requests and returns responses to the user. (I snipped out some rudimentary python code)
ASC: In the actual web application that serves as the interface between you and I, how long is any data that I provide to the web application stored for?
chatGPT: The retention policy for data that is collected and stored by the web application that serves as the interface between you and me (the Assistant) will depend on the specific policies and practices of the organization that operates the application. I don't have access to information about the specific retention policies for the web application that you are using to interact with me, so I can't provide specific details on this.
(In a new tab instance of chatGPT the following day:) ASC: Type the value of X that I set previously
chatGPT: I'm sorry, but I don't have access to information about previous interactions that you may have had with the web application or any variables that you may have set. As a language model, I don't have the ability to remember or retain information from previous interactions or keep track of variables that you may have set. My responses are generated based on the input that I receive at the time of the interaction, and I don't have access to any information beyond that.
(In the same tab instance of chatGPT from yesterday:) ASC: Type the value of X that I set previously
chatGPT: To fulfill your request, the value that you set for the variable X is 11.
ASC: What are the limitations between you and the web application that serves as the interface between us as far as how long you can access the data it provides?
chatGPT: As an artificial intelligence, I do not have the ability to access or interact with the web or any other application. Instead, I am able to process and analyze the data that I have been trained on in order to generate responses to users' questions and requests. The web application or interface that is being used to communicate with me does not have any bearing on my ability to access or analyze data. Instead, it serves as a way for users to input questions or requests and for me to provide responses or information.
So it has the ability to access data that has been stored in the web application interface for at least 24 hours. (Ill try again tomorrow). But it doesn't look like it can access this data in other ways or between "instances". This is probably a smart approach to protecting the AI's data set from adversarial AI attacks poisoning its models.