ChatGPT is Bullshit

🎯 Resumo

Abstract Original

Recently, there has been considerable interest in large language models: machine learning systems which produce humanlike text and dialogue. Applications of these systems have been plagued by persistent inaccuracies in their output; these are often called “AI hallucinations”. We argue that these falsehoods, and the overall activity of large language models, is better understood as bullshit in the sense explored by Frankfurt (On Bullshit, Princeton, 2005): the models are in an important way indifferent to the truth of their outputs. We distinguish two ways in which the models can be said to be bullshitters, and argue that they clearly meet at least one of these definitions. We further argue that describing AI misrepresentations as bullshit is both a more useful and more accurate way of predicting and discussing the behaviour of these systems.

📚 Bibliografia

HICKS, Michael Townsen; HUMPHRIES, James; SLATER, Joe. ChatGPT is bullshit. Ethics and Information Technology, v. 26, n. 2, p. 38, jun. 2024.

🧠 Minhas Notas & Análise

🎯 Objetivo

🧬 Método

🏆 Resultados

📝 Comentários Extras

🔗 Conexões do Cofre

• Temas Relacionados:
• Trabalhos que citam este:

🔣 Bibliografia Latex

🖍️ Notas e Destaques

• is bullshitting, in the Frankfurtian sense (Frankfurt, 2002, 2005) (p. 1)

• Because these programs cannot themselves be concerned with truth, and because they are designed to produce text that looks truth-apt without any actual concern for truth, it seems appropriate to call their outputs bullshit. (p. 1) Comentário: Essa é uma boa ideia do por que programas de IA estão na verdade falando besteira mesmo

• Descriptions of new technology, including metaphorical ones, guide policymakers’ and the public’s understanding of new technology; they also inform applications of the new technology. They tell us what the technology is for and what it can be expected to do. (p. 1) Comentário: Comunicação é sempre um processo chave e de vital importância

• We argue that neither of these ways of thinking are accurate, insofar as both lying and hallucinating require some concern with the truth of their statements, whereas LLMs are simply not designed to accurately represent the way the world is, but rather to (p. 1) Comentário: LLM são feitos para dar a impressão de que eles são desenhados para representar o mundo como ele é.

• in particular, there is a question about the nature and meaning of the text produced, and of its connection to truth. (p. 1)

• In this paper, we argue against the view that when ChatGPT and the like produce false claims they are lying or even hallucinating, and in favour of the position that the activity they are engaged in (p. 1)

• give the impression that this is what they’re doing. (p. 2)

• We draw a distinction between two sorts of bullshit, which we call ‘hard’ and ‘soft’ bullshit, where the former requires an active attempt to deceive the reader or listener as to the nature of the enterprise, and the latter only requires a lack of concern for truth. (p. 2) Comentário: Soft bulshit -> falta com a verdade.
  Hard bulshit -> Tentativa ativa de enganar o leitor/ouvinte sobre a natureza da sua atividade.

• So, with the caveat that the particular kind of bullshit ChatGPT outputs is dependent on particular views of mind or meaning, we conclude that it is appropriate to talk about ChatGPT-generated text as bullshit, and flag up why it matters that – rather than thinking of its untrue claims as lies or hallucinations – we call bullshit on ChatGPT (p. 2)

• The most obvious difference between an LLM and a human mind involves the goals of the system. (p. 2) Comentário: Os objetivos por trás de um ser humano e o ChatGPT são fundamentalmente diferentes.

• Large language models simply aim to replicate human speech or writing. (p. 2) Comentário: Esse é fundamentalmente o objetivo de um LLM

• But because these spaces are constructed using machine learning by repeated statistical analysis of large amounts of text, we can’t know what sorts of similarity are represented by the dimensions of this high-dimensional vector space (p. 2) Comentário: Esse é um ponto que eu sempre critico. As relação nõa indicam necessariamente o que interpretamos.

• Allowing the model to choose randomly amongst the more likely words produces more creative and human-like text; the parameter which controls this is called the ‘temperature’ of the model and increasing the model’s temperature makes it both seem more creative and more likely to produce falsehoods. (p. 2) Comentário: Temperatura é uma indicação de aleatoriedade na palavra mais provávle a se seguir.

• Given this process, it’s not surprising that LLMs have a problem with the truth. Their goal is to provide a normalseeming response to a prompt, not to convey information that is helpful to their interlocutor. (p. 2) Comentário: Novamente o objetivo do ChatGPT é bem diferente do que esperamos.

• The accuracy problem for LLMs and other generative Ais is often referred to as the problem of “AI hallucination”: the chatbot seems to be hallucinating sources and facts that don’t exist. (p. 3)

• hese errors are pretty minor if the only point of a chatbot is to mimic human speech or communication. But the companies designing and using these bots have grander plans (p. 3) Comentário: Essa é uma das minhas principais preocupações. O objetivo de se usar isso não é o objetivo para qual o chatbot foi construido.

• Here’s one way this can work: when a human interlocutor asks the language model a question, it can then translate the question into a query for the database. Then, it takes the response of the database as an input and builds a text from it to provide back to the human questioner (p. 3) Comentário: Isso parece algo semelhante a tarefa de Document Retrival.

• This is not unrelated to the fact that when the language model generates a query for the database or computational module, it does so in the same way it generates text for humans: by estimating the likelihood that some output “looks like’’ the kind of thing the database will correspond with (p. 3) Comentário: Se o modelo é treinado com texto corrido e essa é a principal fonte, dificilmente ele vai indicar como resposta estruturas diferentes.

• The problem here isn’t that large language models hallucinate, lie, or misrepresent the world in some way. It’s that they are not designed to represent the world at all; instead, they are designed to convey convincing lines of text. (p. 3)

• These models aren’t designed to transmit information, so we shouldn’t be too surprised when their assertions turn out to be false. (p. 3)

• For an utterance to be classed as bullshit, it must not be accompanied by the explicit intentions that one has when lying, i.e., to cause a false belief in the hearer. Of course, it must also not be accompanied by the intentions characterised by an honest utterance. (p. 4)

• These suggest a negative picture; that for an output to be classed as bullshit, it only needs to lack a certain relationship to the truth. (p. 4)

• jokes containing false propositions (p. 4) Comentário: Piadas seriam mentiras se elas efetivamente não aconteceram com quem as está contando.

• The suggestion that the speaker must intend to deceive is a common stipulation in literature on lies. (p. 4)

• Frankfurt understands bullshit to be characterized not by an intent to deceive but instead by a reckless disregard for the truth (p. 4) Comentário: Esse é o conceito principal de Bulshit

• Bullshit (general) Any utterance produced where a speaker has indifference towards the truth of the utterance. (p. 5)

• Hard bullshit Bullshit produced with the intention to mis lead the audience about the utterer’s agenda. (p. 5)

• Soft bullshit Bullshit produced without the intention to mis lead the hearer regarding the utterer’s agenda. (p. 5)

• hard bullshitter, but it is important to note that it seems to us that hard bullshit, like the two accounts cited here, requires one to take a stance on whether or not LLMs can be agents, and so comes with additional argumentative burdens. (p. 5) Comentário: Tudo depende do entendimento dos motivos do ChatGPT.

• As we argue, ChatGPT is at minimum a soft bullshitter or a bullshit machine, because if it is not an agent then it can neither hold any attitudes towards truth nor towards deceiving hearers about its (or, perhaps more properly, its users’) agenda. (p. 5)

• They are expected to say misleading things. (p. 5) Comentário: As vezes é esperado que alguém falando besteira esteja manipulando as informações. Isso é o caso de propaganda por exemplo.

• And while there is considerable disagreement concerning whether ChatGPT has intentions, it’s widely agreed that the sentences it produces are (typically) meaningful (p. 6)

• if we take it not to have intentions, there isn’t any attempt to mislead about the attitude towards truth, but it is nonetheless engaged in the business of outputting utterances that look as if they’re truth-apt. We conclude that ChatGPT is a soft bullshitter. (p. 6)

• irst, whether or not ChatGPT has agency, its creators and users do. And what they produce with it, we will argue, is bullshit. Second, we will argue that, regardless of whether it has agency, it does have a function; this function gives it characteristic goals, and possibly even intentions, which align with our definition of hard bullshit (p. 6) Comentário: Este são bons argumentos para o uso de qualquer ferramenta. Apesar de ela em si não carregar a inteção, que a usa tem intenção por vezes clara.

• By treating ChatGPT and similar LLMs as being in any way concerned with truth, or by speaking metaphorically as if they make mistakes or suffer “hallucinations” in pursuit of true claims, we risk exactly this acceptance of bullshit, and this squandering of meaning – so, irrespective of whether or not ChatGPT is a hard or a soft bullshitter, it does produce bullshit, and it does matter. (p. 6) Comentário: A forma como tratamos estes algoritimos é importante. No meu ponto de vista é também uma forma comercial de indicar relevância.

• We can produce an easy argument by cases for this. Either ChatGPT has intentions or it doesn’t. If ChatGPT has no intentions at all, it trivially doesn’t intend to convey truths. So, it is indifferent to the truth value of its utterances and so is a soft bullshitter. (p. 6)

• It is aimed at being convincing rather than accurate. (p. 6) Comentário: Esse é o objetivo fundamental de um LLM.

• It’s reasonable to assume that one way of being a likely continuation of a text is by being true; if humans are roughly more accurate than chance, true sentences will be more likely than false ones. This might make the chatbot more accurate than chance, but it does not give the chatbot any intention to convey truths. (p. 6) Comentário: Outra importante distinção da funcionalidade de um ChatBot.

• say not that ChatGPT is bullshit but that it outputs bullshit in a way that goes beyond being simply a vector of bullshit: it does not and cannot care about the truth of its output, and the person using it does so not to convey truth or falsehood but rather to convince the hearer that the text was written by a interested and attentive agent (p. 7) Comentário: Novamente esse parece ser o principal uso de um LLM. Escrever textos de forma convincente mesmo que distante da verdade.

• its goal is not to convince us of the content of its utterances, but instead to portray itself as a ‘normal’ interlocutor like ourselves. By contrast, there is no similarly strong sense in which ChatGPT confabulates, lies, or hallucinates. (p. 7) Comentário: O Objetivo do ChatGPT seria então se fazer parecer como um ser homano e não necessiriamente entregar informações precisas.

• The bullshitter is the person using it, since they (i) don’t care about the truth of what it says, (ii) want the reader to believe what the application outputs. (p. 7)

• conversely, there does seem to be something particular to ChatGPT, to do with the way that it operates, which makes it more than a mere tool, and which suggests that it might appropriately be thought of as an originator of bullshit. (p. 7) Comentário: Um LLM é mais do que apenas uma ferramenta usada para produzir textos que o usuário quer como um lapis e papel.

• Minimally, it churns out soft bullshit, and, given certain controversial assumptions about the nature of intentional ascription, it produces hard bullshit; the specific texture of the bullshit is not, for our purposes, important: either way, ChatGPT is a bullshitter. (p. 8)

• We object to the term hallucination because it carries certain misleading implications. When someone hallucinates they have a non-standard perceptual experience, but do not actually perceive some feature of the world (Macpherson, 2013), where “perceive” is understood as a success term, such that they do not actually perceive the object or property. (p. 8)

• namely to deceive the reader about the nature of the enterprise – in this case, to deceive the reader into thinking that they’re reading something produced by a being with intentions and beliefs. (p. 8) Comentário: Isso, como eu vejo, é tanto a inteção do ChatGPT como de quem o criou.

• When we adopt the intentional stance, we will be making bad predictions if we attribute any desire to convey truth to ChatGPT. (p. 8) Comentário: Intencional stance -> dizer o que é que o sistema quer fazer sem saber exatamente como ele faz o que faz.

• The very same process occurs when its outputs happen to be true. (p. 9) Comentário: O processo de halucinação é o mesmo para uma resposta verdedaira ou falsa.

• This term also suggests that there is something exceptional occurring when the LLM makes a false utterance, i.e., that in these occasions - and only these occasions - it “fills in” a gap in memory with something false (p. 9)

• This is why we favour characterising ChatGPT as a bullshit machine. (p. 9)

• Like the human bullshitter, some of the outputs will likely be true, while others not (p. 9) Comentário: Quando a pessoa não está preocupada em gerar um conteúdo de qualidade mas sim apenas gerar lero-lero.

• Calling their mistakes ‘hallucinations’ isn’t harmless: it lends itself to the confusion that the machines are in some way misperceiving but are nonetheless trying to convey something that they believe or have perceived. (p. 9) Comentário: Fundamental entender este ponto!