Can we distinguish ChatGPT from human speech?
Question
We're can't have a linguist in here without talking about ChatGPT. It's been all over the news for months now, hasn't it? This is the artificial intelligence chat bot set up by a subsidiary of Microsoft. You sort of alluded to AI and language and human ability earlier, are we in a position where we are going to need to come up pretty promptly with ways to spot these things because of the threat they might pose?
Answer
Stefanie - I don't really see any major threats right now. Chatbots, for example, have been able for a long time to create harmful language that is concerning, that is threatening. But right now I wouldn't say that ChatGPT is really a threat. There are a lot of of people already working on developing systems that can automatically detect whether a text was generated by an AI or human. These do not work perfectly well yet, but considering how fast research is moving forward in this field I would assume that this will be possible in the near future. I've tried out ChatGPT, just out of curiosity, and I would say it does produce some pretty good text in terms of structure. It's got good syntax, good grammar, good vocabulary, but in terms of what a human being can produce and the idiosyncrasies that just having human experiences adds to how we would write a text, I think there are still huge differences and it's pretty easy to spot.
Chris - It fell down in tests on facts though, didn't it? It talked a good game but yeah when you go below the surface, it's extremely iffy indeed, isn't it? You're nodding Will?
Will - Well, as a vanity project, the other day I was asking ChatGPT some questions which I knew the answers to from my own research, to see if it would get it. And what concerned me was it was, is it able to tell good research from bad research? Does it have that hurdle of peer review? Once something's published, does it think it's fact?
Stefanie - I wouldn't trust it. For example, it does make up its own references, so I certainly wouldn't trust it to write my essay for university, for example.
Chris - Although some people did do that and submitted them as a test to their tutors and they got quite good marks.
Stefanie - It might force us to maybe reconsider how we read and how we mark essays. I wouldn't necessarily say that's a terrible thing. And also we might just want to adapt and rethink and consider how we can use this tool to our advantage. It can give you a pretty good first draft maybe of something, but then you can use your own tools and skills as a student, as a scholar, to make it really good. And then get a top grade and not just an average.
Chris - Let's hope so.
Comments
Add a comment