An artist describes the experience she had after training GPT-3 with decades-old diary entries as a healing process.
Out of the box, large language models like GPT-3 generate compelling text. A recent update to GPT-3 with human feedback aligns text quality even closer to human needs. These models reach their full potential when they are fine-tuned with dedicated data for a specific use case.
Artist and creative coder Michelle Huang is using this opportunity to develop a chatbot that allows her to converse with her "inner child." Huang fed the AI with diary entries from her childhood.
As she shares in a Twitter thread, she kept a diary of her dreams, fears and secrets almost every day for more than ten years. The content ranged from complaints about her homework to her crush. Sometimes trivial, sometimes revealing.
i kept diaries for about 10+ years of my life, writing almost everyday — about my dreams, fears, secrets
the content ranged from complaining about homework, to giddiness i felt from talking to my crush
some days were very mundane, some rather insightful pic.twitter.com/CzA1C20U4a
— michelle huang (@michellehuang42) November 27, 2022
"This way, I could accurately simulate what it would be like to talk to my childhood self, based on real data sources during that time period," Huang said. The artist believes that she has direct access to her former self through the chatbot, rather than risking her current self distorting her self-image through false memories.
"Young Michelle" asks about the dreams of the current Michelle
First, Huang asked her past self general questions about the world. The fine-tuned model answers these questions quite generally, but plausibly.
Today's Michelle: What do you think of the world?
Young Michelle: I think the world is fantastic and there is so much to learn and discover. I also think it's important to make a difference in the world and to be kind and understanding to others.
Today's Michelle: What is more important: freedom or love?
Young Michelle: I think both are important, but if I had to choose one, I would say love is more important. Love is something that can help us in any situation, while freedom can be lonely sometimes.
Huang then gave her digital reflection the opportunity to ask her questions. The model especially wanted to know if Huang had followed her dreams, and complimented her effusively.
"This specific interaction felt very similar to a normal texting conversation - as if i were texting my past self in real time," Huang writes.
then, i gave her a chance to ask questions to me as well
this specific interaction felt very similar to a normal texting conversation - as if i were texting my past self in real time
i felt like i was reaching through a time portal, disguised as a chatbox pic.twitter.com/Amws9Is8Rp
— michelle huang (@michellehuang42) November 27, 2022
Huang said she was surprised at how accurately the model could predict her current interests, albeit after much trial and error, and only from old diary entries. This led her to wonder if her path had long been mapped out in her psyche.
How to train your child's AI
After the overwhelming response, Huang shared her approach to GPT-3 fine-tuning in a second thread. This requires paid access to GPT-3 - she does not share how many of the paid tokens she used in her experiment.
Huang trained her chatbot with the following prompt. In addition, it is possible to influence the chatbot's tone with descriptions such as "the young [name] is loving, gentle, and very kind."
The following is a conversation with Present Michelle (age [redacted]) and Young Michelle (age 14).
Young Michelle has written the following journal entries:
[diary entries here]
Present Michelle: [type your question here]
AdJoin our communityJoin the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.AdJoin our communityJoin the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
