CS50 Tech Talk with OpenAI - ChatGPT for Writers
JAY DIXIT: OK. OK, everybody, we're going to get started. I am Jay Dixit.
I work on the community team at OpenAI. It's so nice to be here. It's so nice to see all of you.
So today we're going to talk about ChatGPT for writers. I think that when we think about ChatGPT for writing, sometimes we tend to think about the idea of using ChatGPT to generate writing, and sometimes we worry about that. But in my role at OpenAI on the community team, I have the great privilege of talking to professional writers every day, novelists and comedy writers, screenwriters, playwrights, poets, screenwriters about how they use ChatGPT in their writing process, not to do their writing for them, but as a practical tool to support their own creativity as writers. They use it as a sounding board to talk out ideas, to get unstuck, to break through writer's block. They use it as a research assistant, as a thesaurus to find the exact right word. They use it as an editor to get feedback on their writing.
So these are professional writers who are using ChatGPT not to replace the creative process or do their writing for them, but to support their own creativity as writers. And today, I am thrilled to say that we are joined by two of these writers. So they're going to talk about how they use ChatGPT as a practical tool to support their own creativity. So today, we are honored to be joined by a legend of speculative fiction, Ken Liu. Ken is known for his award-winning works of science fiction and fantasy. He's the author of the Dandelion Dynasty series.
His short story "The Paper Menagerie" was the first work to win both the Hugo Award and the Nebula Award, and also the World Fantasy Award. Ken also wrote the Singularity series-- stories, I should say-- Singularity stories, which are the basis for the AMC series Pantheon on Netflix. He's also worked as a futurist, advising the UN and the US government. And fun fact-- Ken graduated from Harvard and Harvard Law and is also an alum of CS50.
We are also so lucky and so honored to be joined by Sarah Rose Siskind, a comedy writers. And Sarah isn't just a comedy writer, she's a science comedy writer. She's the founder of Hello SciCom, which is a company that makes science and tech funny. She was writer for StarTalk with Neil Degrasse Tyson on Nat Geo. She's the executive producer of two science comedy TV shows currently in development. Fun fact-- Sarah has actually written papers on chatbots and robotics.
And Sarah also went to Harvard and is also an alum of CS50. So without any further ado, I give you Sarah Rose Siskind. SARAH ROSE SISKIND: Massive applause.
Thank you. Thank you, Jay. Thank you.
OK, by round of applause, how many people here are in CS50 and more coders? Round of applause. Oh, my gosh. So we're all-- oh, you're raising your hand because you're afraid to applaud. That's great. Great way to start a talk.
OK, fine. By raising your hands, how many people here are writers? Writers? OK. Good. We had a mix.
She understood the assignment. She applauded. So thank you.
OK, great. And how many people are just here for the pizza? OK. Got that guy. I love the honesty. I love the honesty. OK, great.
So thank you, Jay, for the intro. It is wonderful to be here, to be back. So yeah, that's my background. I wrote for StarTalk for a couple of years.
Pretty much my career in a nutshell is teaching machines how to be funny, teaching funny people how to use machines, trying to convince scientists to be entertaining, and trying to convince TV networks to fund this stuff. All right, so that's my career. And next stop is convincing my parents that this is a real job and I am employed. OK, so that's me. So I am a science comedy writer and a comedian, but most importantly, I'm a survivor of this class.
Yes, please. I'm a comedian. I'm used to applause. Thank you. So this is me in 2012, the stone age of coding.
This was me and my coding buddy, Chelsea, doing a parody of the This is CS50 ad campaign showing what coding is really like. I hope you guys can relate to this. So 10 years ago, I struggled in CS50. Now, I get to struggle in front of CS50.
It's a real full-circle moment for me. As proof of my struggle in CS50, I actually went through my old emails and found my email to my TF trying to get answers to a problem set. Hopefully, you guys can relate to this. Hi, Karen, sorry for the basic question. How do I print Hello, World in python? Yeah. Yeah.
This was her response. Hey, Sarah. No worries.
The answer is pretty straightforward. Smiley face. You just have to implement the print function and refactor the parentheses into the array until the stack is stacked up to the queue.
Or to put it simply, the Hello pointer has to recurse into the world cache or your Hello, World object will fork up your RAM. Facepalm emoji. Been there. Unless you want to hard patch your forced loops, then you'll have to logic bomb the null craft.
No sweat, girly, and don't forget to Pointer Joust your byte-hole. Everyone forgets that part. And if your Hello, World still isn't outputting, you'll want to segment dollar sign CX parentheses x equals 5011017BEEF.
Boom! Coding is so fun, right? Karen. P.S. We're serving Pinocchio's at the pizza social tonight. Tyga announced he'll be performing.
It's all over Facebook. Hashtag Gangnam Style. So this was a real email, obviously, from CS50 in 2012. No, obviously, this was a fake email that I wrote.
So how did I write this? Hopefully, it was relatable for any of you guys who are struggling in this class as I was. It was very cathartic for me to write it. So how did I write it? Well, of course with AI.
And so I'm going to break down how I use ChatGPT to write that thing. But first, a warning. So EB White once said that explaining a joke is like dissecting a frog. You learn a lot about the frog, but you end up killing it.
So this is going to be very unfunny. I am going to be explaining humor. So let's break it down.
Let's dissect that frog, with no further ado. How comedy writers use AI or as DALL-E put it, How Comey Wers Use AI. First up, you're going to put that pencil right down on that keyboard, as we all do. Then you're going to shout ideas into your laughing robot computer. That's how it works, people. OK, so before I get into the process, one thing I find great is that AI is basically already doing a parody of being a human.
So it's kind of-- it's a good place to start using AI to be funny. Kind of ironic. Stage zero-- the therapist. So writing comedy is hard. Being a writer in general is hard, usually accompanied by a lot of sadness and being broke. And so AI can help ease you into the writing process by being your therapist.
So I trained a custom GPT on a lot of improv manuals and comedy texts and therapy modalities to help me ease into things. And you can complain about your career choices or your nervousness about returning to your high-status alma mater, hypothetically speaking. And just a quick tip, you can turn on temporary chats to be incognito.
I'm really hoping, Jay, it's incognito and that OpenAI is not collecting this info, because that would be very bad for me. JAY DIXIT: You're on YouTube right now. SARAH ROSE SISKIND: OK, I'm on YouTube right now. Cool, great. Love it.
OK. So I think of writer's block as loneliness, fundamentally. It's extreme solipsism that's hard to break out of. Now, luckily, I have the incredible advantage of running a writer's room of comedy writers in New York City with a table and humans and everything, and a window, even. But that's a huge privilege, and it's my job. And most people don't have access to that, that is, before AI.
Now you all have your own personal robot writer's room in your pocket whenever you need it. The tools for high production comedy writing are democratized. OK, so that's the therapy part. You are not alone.
Next up, finding the premise. So I have a notepad in the shower. That's where I come up with thoughts about what might be funny. Why do we wave to people on boats but not in cars? Weird, weird stuff. Flow-state type stuff. We'll look into that one.
Now, I actually use advanced voice mode while I'm in the shower for a perfect transcription of my thoughts. Not all of them are winners, but I recommend it. It's great, and it'll actually talk back with you, so that's useful for identifying the premise. OK, next, some premises I observed for today's subject-- so I used ChatGPT to come up-- to pull a lot of stereotypes about CS50. And then I used Google image search to kind of spark ideas. And one of the things I observed after some image searching is that the This is CS50 ad campaign's pretty dramatic.
My favorite part is David Malan in front of a bus that says, "This is CS50." If ever there were a metaphor for what it feels like to take CS50, it's being in front of a bus that says, "This is CS50," about to get Regina Georged. OK. Oh, good.
I'm glad people got that reference. All right, so I really like this. I was like, OK, maybe I could do a parody about how dramatic that is and self-serious. But then I ruled it out because I realized I didn't have-- it's a pretty visual concept. And I would need a team of filmmakers and a high budget, and I didn't have the time.
So I ruled that idea out. And instead I pivoted to the idea that I nearly failed CS50. That might be a better premise. It's relatable. It's self-deprecating.
And much more importantly, it's a much more text-based idea that I could execute and show off how ChatGPT helped me write it. OK, so my premise is CS50 is hard. Next step-- finding the game. So I asked ChatGPT to reference the UCB Improv Manual. This is where expertise comes in, because it's useful to ask ChatGPT when prompting what to reference.
That has been one of my most important tips. And so I asked it for some games I could go with starting from that premise. And it said I could make fun of David Malan for being seen as a superhuman. I could ridicule the ridiculous study methods or talk about TA dependency. Oh, poor, poor ChatGPT doesn't know that we Harvard people call them TFs.
How very droll, ChatGPT. So I decided to combine all of these and come up with a game where I'm dependent on my superhuman TF, who's super cheerful and incomprehensible. And essentially, this is what coding sounds like to non-coders. So that was the premise and the game. But here's the problem.
How do I write a comedy bit about coding when I am, as I mentioned, really bad at coding? With AI, of course. AI is the ultimate writer's assistant assistant. So I asked ChatGPT just to help me off with some coding terms. And what was interesting is I asked it for what are CS-related nouns and verbs, hoping to compile them. And it actually caught wise that I was trying to come up with fake words. And it asked me if I wanted some fake coding terms.
And I was like, hell, yeah. And those actually produced even better results because it knew what I was trying to go for, which was the more ridiculous types of terms. So boom, I got all my terms. Fork up that RAM.
The word "fork" is just hilarious. It's got a hard K sound. It sounds a little risqué. Fork up your RAM. Where do we got it? Oh, yeah. Pointer Joust your byte-hole.
Come on. Byte-hole? Love it. Love it.
So now it's just a matter of heightening this stuff. This is the next stage. And actually, ChatGPT is really good at this.
If you ask it to put a lot of these terms in order. Now, but what is the order? So in comedy, the order is heightening two different things at the same time. And in this case, it's the complexity and the absurdity of the terms, and it's juxtaposed with Karen's friendliness and very straightforward-sounding tone.
And then this is a very scientific graph. This is where the comedy is. It's in between. OK. So we put them in order. Next up-- the ending.
So the button of a sketch is really just like the culmination of heightening. And I really just chose that the best way to do that is for Karen to become a computer. That's where she just kind of devolves into coding speak.
I used ChatGPT a lot for this, so I was asking it for-- because it couldn't just be-- a lot of comedy is not just about being random or nonsensical. It's the sweet spot. It's about upending expectations, but not upending them so much that it's completely nonsensical. It's got to be that sweet spot. And so I didn't want her to just kind of devolve into absolute nothingness. It had to be somewhat familiar coding nothingness.
And so I asked ChatGPT to help me with that. It was fantastic. And just like with a real writer, when I'm in the zone riffing with somebody and we sort of mutually inspire each other, I realized it was exporting all this stuff in Courier Sans.
And so I was like, oh, in the email, she should also devolve into Courier Sans and kind of do a meta joke where she's truly, fully become a computer. So that was the ending. Then there's a thing called a tag. This is more taken from the world of standup.
A tag can be right after the ending. And it's sort of like-- it can be a surprise callback. So I mentioned that I was taking this class in 2012. So I thought I'd add a very dated 2012 invitation at the end about serving Pinocchio's at the pizza social, Tyga performing, all over Facebook, hashtag Gangnam Style. These were all-- I asked for details from ChatGPT about what was trending in 2012.
Comedy lives in the details. AI is fantastic at the details, so it was really a match made in heaven. And then I spent way too much time-- I got sidetracked thinking of the days of 2012. Anyway, so that was the tag. Finally, last stage is wordsmithing and editing.
This is the sort of iteration phase for-- I recommend Canvas to edit stuff. It's really great. I asked ChatGPT to just throw in some emojis where Karen might put them. It did. And then in the history of ChatGPT, I have done something unprecedented, I think, Jay. I asked ChatGPT to make sure that my text did not make sense from a coding perspective.
And I really want OpenAI to look into whether I'm the first person to do that ever. And luckily, it assured me none of what I wrote made sense, which is great. Just how I planned it.
And after endless iterations, boom, we have our comedy bit. So this was really fun to write. It was honestly kind of cathartic because CS50 is very hard.
I don't know if I've mentioned. And there's one point I wanted to bring out, which was just something that ChatGPT did not help me on that I'm curious if ChatGPT 05 or whatever will help on, and that is understanding human misunderstandings. So my favorite part of the text is where I say-- where Karen says, the Hello pointer has to recurse into the world cache because it's built on this fundamental misunderstanding that Hello, World is two different operations. You have to divide the text that's being printed.
And that's the kind of thing that is a-- it flourishes when a comedy writer understands how a human might misunderstand something. And I found it paradoxically hard for ChatGPT to understand misunderstandings. So anyway, that was kind of an interesting frontier. OK, boom, that was a classic email of CS50 in 2012. Vicoory! We DIT! Thank you, DALL-E.
OK, I'm going to conclude with a couple of thoughts. So I'm optimistic. I might be the only comedy writer who is, in fact, optimistic about AI. And I'm optimistic because I don't focus so much on replacement. I focus on how AI is opening up fundamentally new possibilities in humor writing. So the first one I talked about is that AI expands the writer's expertise, so new voices can parody expert topics.
I really would not have been able to write this sketch without the use of an incredible research assistant, because it's been a really long time since I took CS50. So you get more niche comedy, more expert comedy, and more voices able to write that comedy. Next up, I think AI itself might have kind of an interesting comedic voice, just like outsiders tend to have a novel perspective, like babies and aliens might have an interesting perspective itself.
It can translate humor internationally. I've always wanted to hear a Lebanese comedian and understand what those people find funny in his routine. So translate humor internationally. Finally, and most importantly, AI is multi-modal so that writers can now be filmmakers.
The same mind that wrote something can now execute it, which is really cool. And so writers can now make high-production-value videos cheaply and quickly and on niche topics they thought they did not have time for because it was too visual an idea, and she thought she didn't have a team of filmmakers and not a high enough budget and not enough time. Oh, right. So remember how I wanted to make fun of CS50's very serious ad campaign because it looks like the beginning of Succession? And there's no reason for a coding class to be this dramatic. Remember how I said I didn't have time to make a parody of that? Well, I did actually.
So this is going to be the thing I end you on. I end today with-- I present to you guys, This is CS50. Eagle coders. Monk meditating. Robot.
Flowers. Baby born. OK. Thank you all. Knock 'em dead.
KEN LIU: So it will be kind of hard to follow that. So I'm going to not try to do what Sarah did. So a little bit about me. I worked as a technologist, a software engineer, basically, for a number of years before I became a lawyer and then went into writing full time.
So I've spent my entire life, basically, working with symbols and constructing structures out of symbols. And so I think a lot about what does that really mean? I'm going to talk to you by giving you a little bit of a theoretical framework that I use to think about AI and what AI is really doing to writers before giving you a bunch of practical tips on how I actually use it. So one of the things that Sarah said that really resonated with me is this idea that AI today, at least as architected in the form of these large language models, basically the transformer architecture, it's a parody of a human being.
And I think that's a really, really important point to keep in mind. What does it mean to be a parody of a human being? Well, what's really happening here is today's AI is fundamentally playing the exact same game that Turing originally proposed as a substitute for trying to define what intelligence is and what thinking is. We still don't know what that means, right? So if you go look at the papers being published in the field, Apple's researchers have published a paper that fundamentally argues that LLMs do not think. And the response to that paper is basically half the people saying that humans don't think either. And the other half says, well, that's so obvious. Why are you even pointing that out? I don't think we've ever been at a point in an industry where the top researchers and thinkers in the field fundamentally disagree about what the hell is even going on with the technology.
So we still don't know what thinking really means. We don't really know if LLMs are doing it. And we're still basically doing what Turing told us to do, which is to just imitate, play the imitation game. We don't know what thinking is. All we can do is just imitate the result of human thinking.
I think that's really important. That explains why there's this parrotic nature to everything LLMs generate. It is a reflection back to us of our intelligence encoded in the form of language, so whether that's thinking or not is not really possible to define right now and not important. But knowing that LLMs are just giving you reflections of human thinking allows you to think about what to do with it in creative, interesting ways.
So let me give you two things to think about. The first thing is this. If you go onto YouTube and search for "how to write a novel using AI," you will come out with hundreds, if not thousands of results. This is one of the most popular things that influencers do.
They write about how you can use AI to write novels and put them on Amazon and sell them for money, thousands of these videos. Now, I want you to do another search, which is, "here is a novel written by AI that I really enjoyed, which you should read." You will find zero such videos. I tried very hard. I looked very hard for people who wrote specific recommendation lists for AI-written novels, so I can go read them. I found a few of these lists, and I looked up these books.
And they have either zero Amazon reviews or three Amazon reviews, and the reviews are one star. It seems that people are really interested in working with AI, but they find the result incredibly boring. And I think all of you recognize that. It's fun to use DALL-E to generate a bunch of images and to make fun of them and to think about them and talk about them. None of you will go actually buy one of these DALL-E images and frame it and admire it and think about the deeper meaning.
AI art as being generated is boring. But the process is interesting. So keep that in mind.
That's one. The other thing I want you to think about is, a lot of the current talk about AI, to me, is deeply misguided. A lot of the current fear and concern about AI is this idea that AI will now create novels. AI will write movie scripts.
AI will just generate movies, and all the artists will just be out of jobs. I think that's just not a very good way to think about it, because thinking about AI as a cheaper way to replace humans is very capitalist. And if there's one thing about capitalism, it's not very creative. And it doesn't-- it just knows how to make something cheaper, faster. It doesn't really know how to create anything fundamentally interesting, because interesting is not necessarily something that can be monetized. So try to think of AI not as a tool for existing mediums, but as a new medium itself.
What do I mean by that? If you go back to the 19th century, the very end, and you go to these exhibition halls, and you will see a new machine being demonstrated called the cinematograph. What is a cinematograph? It is constructed from the Greek word for "motion," "kine," and "writing," "graph." So it's a motion writer.
What is a motion writer? It is a motion picture generator. It is a machine that captures movement and plays it back. The earliest films being produced by these cinematographs were known as actualities. What are actualities? They are sort of like proto documentaries, so if you go search them, they're about a minute long because that's how long the reels were, about 17 meters. And I think they played them at 16 frames a second. But anyway, you watch these videos.
They're about a minute long. And they show things like women leaving a factory, a person trying to get on a horse, a baby trying to plunge her hand into a bowl of goldfish, things like that. And you're like, OK, what is this? If you were told to pay a Franc each to just go watch actualities for a night, you would probably feel like you're being ripped off, because what am I watching? Just people going through the motions of daily life. Come to think of it, that's not very different from a lot of TikTok. But anyway, so you watch it. And you sort of go, OK, what is this? But there's something really compelling about this too, right? If you just watch the actualities, you cannot imagine that one day motion pictures will become the dominant way in which we tell stories.
There's no way you could. But just watching these one-minute-long actualities, there's something really compelling about them because seeing human movement being captured and reflected back, that's new, right? But you cannot-- this is not-- this is a new medium, right? People had to learn how to develop a language of cinema, to be able to tell stories using motion pictures. A true motion picture is not just a filmed stage play. It's not a novel coming to life. It's not any of those things. It's a new way of telling a story.
That's what a new medium is. My argument is the reason why AI today is so boring in its output is because we're still using it to tell existing stories to as a tool for existing mediums. Until we figure out how to use this medium to do its own thing, it will not succeed. And what does that new medium look like? Well, think about the ways in which we actually enjoy AI, right? The ways that we enjoy are these trends that show up, the make the bodybuilder more muscular, more muscular, more muscular, until you have Galaxy Brain version of bodybuilders or the AI Mona Lisa multiverse. I don't know how many of you are familiar with that. This was a trend from a year ago where people used ChatGPT and DALL-E to generate a series of image stories about how people from different countries would perform heists on each other of the Mona Lisa.
It's a typical day in the life of an American, right? And you see this guy getting up from a flat covered bed, getting his gun and drinking his coffee, driving his truck to the airport to fly to France, take the Mona Lisa, and fly back and hold a news press briefing. It's fun because it's a community participatory thing. It's not the result. It's the participation.
So taking those two things together, the essential boringness of AI output right now, but the fun of the process, and how you can't tell, from the earliest examples, how a medium is going to evolve. This is where I end up in terms of using AI as a writer today. Because AI is a parody of a human being, you have to lean into the idea of what can a parody of a human being do for me that's actually useful and interesting? So number one, if you're trying to write something, tell your own story, having the AI generate the result is not going to work. Why? Because if you're trying to tell a story, one of the things you have to do is to create your own language, not the language of cliches.
And AI, being a machine that captures cliches and distills them and reflects them back at you, it's particularly bad at that. So using it to generate your own story is almost always a very bad idea. So what can you do? Well, think about the whole idea of brainstorming. One of the most interesting things I do with AI is I brainstorm with it.
But here's the thing. I don't use it to generate ideas for me. I don't brainstorm with it quite the same way I do with a human being.
I do it by leaning into its machine-ness, right? What does that mean? As I mentioned earlier, a cinematograph captures motion. I have a term for AI, which is a "noemamatograph." Comes from the Greek word "noema," which is idea, concept, or subjectivity. It is a subjectivity-capturing machine.
Current AI is very good at capturing and excavating these subjectivities we embody into our linguistic output. And then it's able to reflect that back. So you have to craft a personality for your AI and then collaborate with it as a parody of that type of person. So what I do is I say, you are a very skilled interviewer of writers. You enjoy talking with writers about their project.
You like to ask provocative questions about their work. You are not here to give me ideas. You are here to ask me questions and push me to explain myself to you. This is what makes AI interesting. If you ask AI to give you ideas, it's very difficult to not let the machine cliche-ness get a hold of you.
But if you force the AI to ask you to answer its questions, you're likely to come up with really interesting ideas. The AI will ask questions. It's tireless. It will keep on pushing you and saying, why do you find this interesting? What is-- OK, well, I've seen that before. Can you make that more unique? It will push you into coming up with new ideas. So I really love it.
A lot of subjectivity, it turns out, is intersubjectivity. As writers, we're taught to work alone. And I think that's actually deeply unnatural. Those of you who are writers know that a lot of your joy in writing comes from working with other people.
Sarah and I have both done writers' rooms. And one of the most amazing things about being in a writers' room is the way in which the intersubjectivity of working with other people multiplies your own creativity tenfold. You feel so much more creative that way.
And having the AI be the person to push you to be more original, to push you to explore your idea and explain your idea and develop your idea better is really helpful. It allows you to get out of that "I'm locked in my head" space and practice your subjectivity. So that's one way. The other way I find deeply helpful with AI is to have AI give you feedback.
Again, what you want to do is lean into the machine-ness of it. You tell the AI you're an expert reader of techno thrillers. You're a huge fan of Tom Clancy. Here is a manuscript that I've written, and you're going to critique my book from the perspective of someone who loves Tom Clancy books.
What don't you like about this book? What do you like about it? That sort of thing is really interesting. Again, if you just ask ChatGPT to give you feedback, it can't do it because it's trying to give you the averaging of all the subjectivity that's captured. But if you shape it and tell it to parody or imitate a specific subjectivity to give you critique and feedback, it's great at it. It will detect patterns, clichés, and other general trends in your work and tell you what's not working.
And then you can think to yourself whether that's something you want or not. So other ways I found really interesting with AI is just all the same things that Sarah said. You ask it to be a research assistant. Again, it's good at parody. And a lot of writing is actually parody. When you're writing a techno thriller, a lot of times what you do is you're pretending to be an expert in a domain, and you're trying to give the illusion of technobabble in a particular field.
You go watch Star Trek or something. That's just one of the things writers have to do to create that illusion of a self-contained linguistic field of expertise. And AI is great for that. You can have AI create technobabble for you and to critique your technobabble and to say, this is not good, but this part is good.
Do something with it. So for those kind of things, I enjoy using AI to help me figure out what works. And then one last thing that I find AI to be really good at is to just send it out to do research tasks that I can't do myself. I send out ChatGPT to gather interesting incidents of social engineering, for example, and then it will go and collect a bunch of stories and come back and gave me. And then I can review them and figure out which ones are interesting as potential material that I can use. So in conclusion, I think any time you're using AI to generate the final output and to think that all you have to do is to be like those YouTubers and say, write me a best-selling novel, it's not going to work.
AI is incredibly bad at doing that sort of thing. But if you are a very purposeful writer who's trying to increase your own subjectivity and trying to stay in touch with your story and trying to leverage AI to basically up your own game and be a more interesting subjectivity, then it's great. And you want to find ways to force the AI to force you to be more human, be more creative, to write better material. Thank you.
JAY DIXIT: Amazing. Amazing. Thank you both so much. I love the breadth. I love that you each have such different perspectives.
But then it's amazing to hear how you actually use the tool in your process. And I love that each of your presentations was kind of characteristic of the type of writer you are. So Sarah, your presentation-- SARAH ROSE SISKIND: Desperate for validation.
JAY DIXIT: You were begging us for laughs, practically on your knees. No, it was a hilarious comedy routine. And Ken, characteristically, a brilliant philosophy essay. I love it. So we're going to take questions. I want to take questions from the audience.
We want to keep the questions focused on the process that we talked about-- the use of ChatGPT for writers, by writers, in support of a writer's creativity. So let's have questions from the audience. SARAH ROSE SISKIND: We wait for microphones? Or are we passing out microphones? JAY DIXIT: Out here? Yeah. Let's pass around the handheld mic.
Is that Julia? SARAH ROSE SISKIND: All right. AUDIENCE: [INAUDIBLE] SARAH ROSE SISKIND: OK, I got it. AUDIENCE: Thank you.
JAY DIXIT: Thank you. AUDIENCE: Hello. Thank you both for the talks. I am a PhD student working on human and AI interaction, specifically on designing AI that complements rather than replaces humans. So my question is maybe to both of you, but around expertise. So you mentioned subjectivity and how AI can amplify people's subjectivity.
So now I think that that is something that you know because you're an experienced writers. But new writers may not come up with this idea of what makes good writing to begin with. So if you use these tools from the beginning, I feel that there's the lack of opportunity to develop this expertise to begin with.
So maybe a little bit on expertise and writing, and how can you maybe teach or use these tools to teach younger writers for developing their subjective voice? SARAH ROSE SISKIND: One thing I'll say about that while Ken actually answers your question and modulates his response-- while he's loading. One of the things I find really interesting is there's this huge discussion about, oh, is 03 or 01 or the Perplexity a grad student or a PhD level or a Nobel prize winner? What I find really interesting is that, OK, you could compare it to humans. But you never meet a grad student who's a grad student in everything.
I know there's a bunch of smarty pants in the room, but nobody's a PhD level in all of the things. And so to me, as a science comedy writer who lives at the edge of two different disciplines, to me it's that inner relationship between two different expertise. I will write a script and be like, what's the funniest example of the Heisenberg uncertainty principle? And to me, that's a very hard cross-referencing for most writer's assistants. So I encourage more writers to think about using-- in terms of expertise-- using ChatGPT as a cross-referencing kind of expertise.
KEN LIU: Yeah. So I guess, for me, when I think about this question, I try to divide expertise into two types of expertise. One of them is subject matter or domain expertise, where you're a writer, you're trying to write about a subject, and you're trying to learn about it.
The other kind of expertise is more about craft. Are you actually a good writer? Whatever that means, right? The first kind of expertise AI is incredibly good at helping you. But you want to be very careful here. You want to avoid the Dunning-Kruger effect where you know just enough to be dangerous. And the trouble with that is AI, even today, has a huge propensity to hallucinate, especially in a field in which there's not a whole lot of training material, and where it's liable to make up things.
And you, as someone who's not an expert in the field, will not know. So what I tend to advise people to do is first assess your own level of expertise by asking the AI to quiz you and see how much you actually know about the field, so establish a baseline of how much you actually know yourself. And then you ask the AI to go out and teach you stuff.
You can generally trust the AI when it sticks to textbook material, meaning intro stuff. It's not going to lead you astray. But when you're getting into specific details, you're going to have to verify them yourself because the AI is liable to make up things at that level.
And it's very hard for you to tell that's happening unless you're an expert. I mean, I think all of you have this experience. I certainly did. Whenever I read a newspaper article about technology or law, I find them to be full of flaws and misunderstandings and mistakes, which makes me realize that when I'm reading about something that I know nothing about, such as finance or the Middle East, I need to take all of that with large grains of salt, because if I can detect the errors in fields of my own expertise, then I'm just not detecting the errors there. With AI, it's the same sort of thing. You just have to keep on remembering you got to be able to tell when the AI is bullshitting, right? The other part where developing your expertise as a subjectivity matter, that's much harder.
So my view on this is not necessarily correct or whatever, but I feel very deeply that as an artist, what you're really trying to do is to say with words what cannot be said in words. That's what art actually is. Art, at least as a novelist, is about writing something that is not purely communicated. You're trying to say with words something that actually cannot be said with words.
So in order to do that, you have to push yourself to invent your own language. All writers who are worth reading ultimately do that. So if you end up relying on ChatGPT to just give you the words or to tell you whether your words are good when you're just starting out your journey, I think that's a very bad way to do it.
When you start out as a writer, you can have AI to push you to develop your ideas, but you still have to actually develop your own evaluation of how good you are in your craft by interacting with other humans. I think Sarah would agree with this. AI is a good substitute for a writers' room when you have no humans around. And for certain things, it's better than humans, even, because it's tireless. And it will just keep on going. SARAH ROSE SISKIND: Comedy writers get tired very quickly.
KEN LIU: But for other things, you have to have humans. There are certain things that you just cannot rely on machines to do for you. I rely on my beta readers a lot for my books. And fundamentally, after I've done the AI critiquing, I have to go to the humans because there's nothing like an actual human being who will experience the art to tell me this is actually garbage, or this is actually interesting.
And writers who are starting out absolutely have to do that. I do believe that intersubjectivity is critical. JAY DIXIT: And I'll just jump in and say also, I don't think you have to be an expert like Ken or Sarah to find ChatGPT useful. I'm really excited about the potential of ChatGPT to help writers develop and grow and learn as writers.
You can literally just tell ChatGPT, hey, I'm a beginning writer. Help me understand the principles of heightening so that I can get better at writing comedy. Or walk me through three-act structure so I can write a screenplay.
And you can kind of just tell it where you are and also just ask for feedback. Hey, look at this joke. Is this really hitting the principle of heightening? Does this sound like a cliche? And then get the feedback that you might need. KEN LIU: It's good at generating writing exercises for you. I found that to be a fairly good use for students to have ChatGPT generate writing exercises for you to improve specific techniques. SARAH ROSE SISKIND: Yeah.
And also observing-- I think, yeah, Ken would probably agree that you refine your taste sometimes before you refine your ability to produce. And so I find 01, the slower models, are good when you feed it a sketch and you're like, why is this funny? And Ira Glass has this great quote about creativity, which is like, there's-- you know the one I'm talking about? KEN LIU: I know exactly-- SARAH ROSE SISKIND: Yeah, it's the one-- oh, it's so relatable. He talks about how you refine your taste usually much sooner than you refine your abilities. So there's this moment where you're really good at determining when something's good and you suck at making that thing. And it's a very uncomfortable time. And if you're good, you never close that gap, actually.
You're always-- your taste is always slightly ahead of your ability to produce. And so, yeah, if you sign up to be a writer, you sign up to be unhappy for the rest of your life. Sorry. The end. JAY DIXIT: Exactly. I'll say one more thing about that.
Yeah. You may never get there, right? The French poet Paul Valery has a quote where he says, a poem is never finished, merely abandoned. You're always going to be able to see a way to make it better. But I think for me as a writer, one of the hardest parts of making something good and getting it to that point where the quality of the work matches my taste of knowing whether it's good or not is just knowing what to do, knowing how to make it better. And that's where I feel like having ChatGPT to give me feedback like, hey, I wrote this.
I've been staring at it for hours, I don't know-- I can take out this comment or put it back in, but I don't know how to make it better. Give me ideas for how to make it better. And then ChatGPT can give me ideas. Then I can rewrite based on the feedback and then do it again. Give me more feedback.
Rewrite it again. My favorite writing teacher, William Zinsser, wrote a book called On Writing Well, my favorite writing book. Says writing is rewriting. What he means is that the thing that makes writing good isn't that great writers, like people we have here-- just comes out perfectly the first time in the first draft.
It means that what makes writing good is rewriting it and rewriting it and revising it and editing it until you get to the point where it's good and having more feedback to say, hey, maybe your intro is a little wordy. Maybe your second paragraph is boring. In the third paragraph, the joke doesn't land. The fourth one you overexplain it. And the fifth one, that's really interesting. You should expand on that part.
Having that before it even sees a human editor, I think, is so helpful. And then you still send it to your human editor and get even more feedback. And if it gets-- I don't know-- 5% better with each round of feedback than the final product, hopefully, ideally, it's better than it would have been. Let's take another question.
AUDIENCE: I just want to say thank you. This was a really enlightening talk, as someone who's concentrating in CS and English and enrolled in a creative writing workshop and algorithms class in the same semester. I just had a question for Ken, but also I would love to get your perspective on this, in kind of leaning into the machine-ness of GPT. What I understand-- you both brought up parody. I understand that as maybe a manifestation of LLM's abilities to kind of make this Word2vec association and take the average of two completely unrelated things or seemingly unrelated vectors.
How does that-- in your head, what are the implications of that machine-ness and skill for work like translation or multilingual work? Because I feel like I can see a huge potential there for maybe even a more generative use of GPT to give me different ways to translate from different languages that might be a little bit more than just using it as an agent or an assistant. And what are the implications of that for ownership of your writing? KEN LIU: [INAUDIBLE] SARAH ROSE SISKIND: Oh, my god. Now you defer to me? Oh, you got the mic. OK, great. God, there was a lot there. Leaning into the machine-ness of something is great.
I just wrote a paper recently. I'm trying to write a paper about where's the line, when to tell a inappropriate joke. And I wanted to come up with an algorithm, a funny algorithm of all these different variables and they're weighted different weights. And one is timing.
How long has it been since the offensive thing? And then how bad was the harm? And are you related to it? All this stuff. And I was like, I want to come up with a funny-sounding acronym for this algorithm. And this was a perfect application of ChatGPT. Its wordsmithing abilities are bar none.
It's really incredible. And so I asked it for one. And it came up with PERFECTION was the algorithm where it was like, P stands for punch, O stands for object of ridicule.
It was fantastic. And then I just changed it such that N actually stands for nothing. I just wanted the acronym to work.
And it was like a beautiful mutual co-working that would have taken a writer's assistant forever. I don't if they ever could have come up with an acronym that spelled something perfectly. I think if you ever added any AI to the Punderdome, it would just explode everybody there.
And so leaning into the machine-ness is a really great concept that I'm actually still looking into and wondering, how can this make us not just swim, but swim with fins, accelerate our progress and be a complement tool to us. But that's the evolving thing. And this is why I encourage writers to do exactly what you're talking about.
Think not so much, what can it replace, although it is useful to replace the drudgery of writing, but think what are the fundamentally new portals it's opening up? And that's where true creativity comes in, both in terms of what you're writing and how you're writing it. KEN LIU: Yeah, I would echo that. I want to push on the whole idea of new portals.
So my big thing has always been that I think using AI to just imitate humans and do what humans already do is a dead end. I don't think that's interesting. Fundamentally, you have to get AI to do things that humans can't do. And you brought up a bunch of really interesting points. And I don't have a finished answer.
But I have directions and things that I'm excited about. So a lot of you probably do know that the Transformer architecture, which is the foundation for the modern AI revolution, was originally designed as a translation technique, which is fascinating that you bring that up. And it's deep insight is that you-- attention is all you need-- that you pay attention to all parts and how they all connect to each thing, that it's not a sequential output, but rather an [INAUDIBLE]. The machine-ness of it, in that sense, is deeply interesting to me, because that's not how we're taught to think about language as humans, even though deep down, we probably do something very similar. But it opens up new ways of thinking about language.
So one thing that I found to be interesting already as reading the papers in the field, there's a DeepSeek paper that just recently came out, which I don't know how many of you have read. I do encourage you to read it. It reveals some interesting things. One of the things it revealed was that the team behind DeepSeek originally realized that the model performed better when it used a multilingual approach, meaning you give it a prompt and it does the thinking by switching between different languages constantly. And it actually performed perform better than if you forced it to be monolingual.
Now, you can take this in multiple ways. One is that there's something deeply fascinating about the way humans are multilingual, in that our collective intelligence as encoded in these linguistic corpora is actually different, meaning that there are different fields of expertise, that if we are all multilingual, in some sense, we would be better off than if we're monolingual. The other one, which is more mundane, is simply that the training corpus appears to cover certain subjects in certain languages better than other subjects in other languages.
So the AI has to do that. The team, by the way, ended up forcing the model to be monolingual, simply because users found it very disconcerting to have the model switch between different languages, which is, I think, extremely human. Very interesting that we force things to be worse in the name of comfort.
The other thing that I think is interesting about languages is to think about actual translation. So when you're talking about human translations, the Star Trek model of the universal translator really is doing us a huge disservice because it implies that translation is a matter of mapping. Whereas the Transformer architecture shows that it's not actually a matter of mapping.
It's deeply contextual, and there are subjectivities embedded within it. So let me give you an example. So Emily Wilson, who is one of the most interesting translators of classical lit, she and other translators like her often are criticized for injecting subjectivity into the translation, which reflects a deep-seated bias towards certain kinds of subjectivities versus others. So if you actually talk to classical scholars, they'll say that, OK, here's the thing, right? Emily Wilson herself said this. Any linguistic utterance in one language contains a multitude of truths, not a single one, but a multitude. And depending on the subjectivity of the person doing the interpreting, it is not possible to encode all of those truths exactly into another utterance in a different language.
You're going to have to choose which truth to tell. AUDIENCE: First of all, thank you. I think a lot of good practical advice and also the philosophy. I'm really loving conversation, I think. Can you talk about the interpretation in some multiple languages? Now, clearly the discussion is primarily English.
So I just want to know, in other languages, have people tried? What are your experiences? Anything that you see as comparison or contrast between using it for something in English versus something other than English, and how we should think about it? Where are we also in terms of-- from OpenAI, what are your views on non-English usage of this? JAY DIXIT: Yeah, that's a great question. We're really excited and interested in non-English usage. ChatGPT speaks at least 50 languages, or it might be a lot more now. And that includes the voice mode.
So my father speaks to ChatGPT advanced voice mode in Hindi. I use it to practice my French. I had a cleaning person who came to my home one time who was from Ukraine and didn't speak English, so I was talking in English. It was translating into Ukrainian.
She was speaking Ukrainian, translating back to English. It worked pretty well. And then yeah, I'm curious to hear-- well, the two of you probably know a lot about other languages. SARAH ROSE SISKIND: Ken should mostly answer this.
I could come in with the periphery thing because he's the translator. OK. Two thoughts. Firstly, off of what Ken was talking about, about how it works best, DeepSeek works best when referencing lots of languages, that's very interesting to me. One of the classes I took at Harvard was about the history of the English language. And one of the theories behind English's relative success as a language in terms of popularity has to do with the fact that it incorporates other languages inside of it.
It's actually a really-- it's a multiverse of a language. It firstly started with Anglo and Saxon. But then sometimes, we import words like "schadenfreude" or whatever that just come straight from other languages. It has, I think, 750,000 distinct words. And most other languages only have 250,000 to 500,000 distinct words. And so it's a very successful language because it operates like multiple languages.
And then the second thing is, yeah, the translation aspect of AI is mind-blowing. I've been practicing my Hebrew. And one of the things I learned recently was a lot of Hebrew-speakers, when they're speaking English, they'll talk in gerunds. They'll be like, I am knowing something. And I asked ChatGPT.
I was like, why do native Hebrew speakers over-conjugate gerunds? And it was talking about how there is no conditional-- that they don't have a distinction between present tenses in Hebrew. And they overcorrect. And it had all these theories as to why. And one was that they want to sound sophisticated.
I was like, oh, that's great. I know how to sound sophisticated in Hebrew now. So what's great about it is that it, in that sense, understood human misunderstandings, which was great because it's a new dimension of teaching, where you can actually understand a mistake.
Because my former boss, Neil Degrasse Tyson, always used to say this, which is like, if you have a multiple-choice test, for a language test that's, let's say, how do you spell the word "cat" for a child? And it's like, A, C-A-T, B, K-A-T, C, is like 5, 4, 3, 9, and you choose K-A-T, that's equally as wrong as the random string of numbers. But in reality, there are gradations of wrong. And so what's great about AI for understanding languages in general is it understands those gradations. It understands human mistakes and misdirections. So it's like a nuance explosion. So yeah, hashtag nuance explosion.
Ken? KEN LIU: What I was going to say was just that I think one of the consequences of modernity is this tendency to let network effects get away from us. And so English is one of those examples of a network effect. There are lots of places on the internet where one of the top lines is, please use English, because this is the language that will allow all of us to understand each other.
And if you work in the literary field, you know that's actually very much a thing. So a lot of writers from countries in northern Europe or eastern Europe will choose to write in English, simply because this gives them the ability to reach a much wider audience than they would otherwise. It's necessity. So that also means that English, by default, becomes the largest corpus of training data.
It simply encodes more of our human wisdom than any other linguistic corpus. And this has a tendency to make those who speak English to assume that if it can be said, it must have been said in English, and it can be said in English, which is not necessarily true. If you study any kind of other foreign language, you'll know that there are certain things that are interesting and deeply expressive in other languages that don't really-- cannot be translated per se. Translation is always sort of a halfway measure, if you will.
And I worry a little bit about that. And I do wonder to what extent AI can help us do that. I sometimes ask AI to give me a sense of how another language works and how, for example-- Latin, for example, has gone through this very interesting transition where classical Latin, as spoken by Cicero and others, relied much more on you. It uses parataxis, which is this idea of putting words next to each other.
And you have to construct the logical relationship between them based on the declensions and conjugations and so on. Whereas medieval Latin was very different. Medieval Latin had a huge reliance on word order, perhaps reflective of the way scribes over time interpreted the original parataxis and gave them specific interpretations, which are then encoded in word order.
And of course, as we all know, modern descendants of Latin have almost all abandoned the free form syntax and go with word order in some way. There's an interesting-- something interesting has been sort of captured or lost, depending on your perspective in that transition. And I think using ChatGPT as a way to give me some of these examples of old texts and to think about them, has been helpful for me to think about it.
I just think one of the great things about ChatGPT is that you now have an inexhaustible and tireless machine that can help you excavate the linguistic richness of all the other things. You are no longer relying on a single interpretation. You can go excavate and ask, what is the hidden agenda behind your translation, and what are the other agendas possible? So I can in some way recover the richness of the original.
I certainly gained a better understanding of "The Odyssey" as a result of reading multiple different translations. And I think now we have the potential to do that with any text, which I think is deeply rewarding. JAY DIXIT: OK. SARAH ROSE SISKIND: Damn, a mic drop. JAY DIXIT: Yeah, mic drop. Mic drop.
Perfect note. Perfect note to end it on. It's so inspiring hearing you both talk about it. I'm really excited about the potential of ChatGPT to support writers in the writing process and support their creativity.
If you guys want to learn more, you can find us-- Chatgptforwriters on substack. We're also releasing tips and tricks specifically for students on LinkedIn. LinkedIn, OpenAI for Education. So thank you so much, Sarah Rose Siskind. Brilliant. Thank you so much, Ken Liu, an honor.
And thank you all for being here. Thank you so much. Thank you.
SARAH ROSE SISKIND: Thanks, guys.
2025-02-22 18:14
