Harnad, S. (2008) The Annotation Game: On Turing (1950) on Computing,Machinery and Intelligence. In: Epstein, Robert & Peters, Grace (Eds.) Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer. Springer
This is Turing's classical paper with every passage quote/commented to highlight what Turing said, might have meant, or should have meant. The paper was equivocal about whether the full robotic test was intended, or only the email/penpal test, whether all candidates are eligible, or only computers, and whether the criterion for passing is really total, liefelong equavalence and indistinguishability or merely fooling enough people enough of the time. Once these uncertainties are resolved, Turing's Test remains cognitive science's rightful (and sole) empirical criterion today.
Subscribe to:
Post Comments (Atom)
-
What is a Turing Machine? Computation is Symbol Manipulation What is a Physical Symbol System? Optional Reading: Pylyshyn, Z (1989) Com...
-
Harnad, S. (2017) To Cognize is to Categorize: Cognition is Categorization , in Lefebvre, C. and Cohen, H., Eds. Handbook of Categorizatio...
-
Harnad, S. (2009) Cohabitation: Computation at 70, Cognition at 20 , in Dedrick, D., Eds. Cognition, Computation, and Pylyshyn. MIT Press ...
I find Professor Harnad’s commentary of Turing’s “Computing Machinery and Intelligence” insightful. A few points brought up in the commentary were discussed during our second lecture, drawing connections to what we recently learned. For example, the levels of hierarchy in the Turing Test and the blurry boundary between T3 and T4 are explained in detail here. Furthermore, drawing our attention to the difference between AI and Cognitive Modelling regarding their goals was especially helpful. At first, they seemed similar to me. Yet, AI is meant to be a powerful performance tool for productivity, while Cognitive Modelling has the scientific objective to explain how human cognition is generated. Overall, I think this commentary “kid-sibs” Turing’s text for us, making it more pleasant to read.
ReplyDeleteFine, but now can you kid-sib Turing text for us (in a paragraph?). "Here's ChatGPT4's one-paragraph answer to my question: "Could you summarize, for my students, in one short paragraph, the gist of Turing's famous 1950 text (which eventually became the Turing Test)?"
ReplyDeleteCHATGPT4: "Alan Turing's 1950 paper introduced the "imitation game" as a measure of machine intelligence. Instead of asking whether a machine can think, Turing proposed a practical test: If a human judge cannot reliably distinguish between a human and a machine based solely on their responses to questions, then the machine can be said to exhibit intelligent behavior equivalent to that of a human. This concept has since become widely known as the 'Turing Test' and remains a foundational idea in the philosophy of artificial intelligence."
Can you spot all the things that are missing or wrong or only part-right in this summary? What would you answer instead?
First, saying that Turing introduces us to the Turing Test would be more appropriate than the “imitation game”. The “imitation game” gives off the false impression that participants are taking part in a deceitful act, rather than an empirically-verified scientific experiment. It would also be more appropriate to say that it’s meant to test whether a machine can act indistinguishably from a human throughout the course of a lifetime, rather than whether the machine is “intelligent.” Similarly, it is also incorrect to say that if a machine passes the TT, it exhibits intelligent behavior equivalent to that of a human. It’s more correct to say that it reaches “total indistinguishability” in whichever capacity the TT assesses. For example, the T3 tests “robotic sensorimotor performance capacity.”
DeleteYou made a really good point, Anais! I appreciate the insight on TT.
DeleteAlso, I believe that a part of the crux of Turing's argument was not about the machine's capacity to "think", but rather its ability to mimic and simulate human-like responses so closely that a person couldn't distinguish between the two. A part of Turing's argument is not that a machine can exhibit human-like intelligence, but that it can emulate human-like behavior in the tested capacity.
Correction to the comment on word choice, I meant not exactly "mimic" but the point that a machine was indistinguishable.
DeleteThis comment has been removed by the author.
DeleteAdditionally, I would say that ChatGPT was incorrect because the purpose of the Turing test is not to see if the machine is able to “exhibit intelligent behaviour,” nor was he providing a philosophy of artificial intelligence. Rather, the intention of developing a Turing machine is to reverse-engineer human cognition. Turing argued that a machine which could behave indistinguishably from a human being would likely internally work very similarly to a human brain. Thus, by developing an effective Turing machine, one would be able to reverse-engineer aspects of human cognition.
DeleteI would like to add to what ChatGPT is missing or incorrect about in this passage. Turing also said that the TT could be used to reverse-engineer cognition but what he meant by this was not every ability of cognition. He says a machine that could pass the TT would help cogsci reverse-engineer how and why we can do everything we can do (EP) but it could not help us reverse-engineer the how and why we feel (HP). The machine passing TT would not help cogsci with the HP because although it is completely indistinguishable from a human for a lifetime, we still would not be able to prove that it is feeling due to the OMP. Turing knew the limits of the TT and said that the EP seems solvable while the HP seemed out of reach.
DeleteAccording to this article and previous comments, I may summarize that the reason to call the Turing Test an "imitation game" is to avoid (temporarily?) the other-mind problem, as we have no sense of the mental states of human beings. I am glad this article could tell me the limitations of the Turing Test.
ReplyDeleteThis article also mentions the VR discussed in the last seminar. It reminds me of the models (a concept of philosophy of science), as the models are built upon our current knowledge about reality and used to represent reality while not equal to fact. The VR is kind of model, right?
Virtual Reality models are ways to fool the senses of real human beings. Computer modelling of attempts to reverse-engineer human cognitive capacities are attempts to find and test a causal explanation of cognitive capacity.
DeleteThe “vegetative functions” such as breathing and swallowing are controlled by the older parts of the brain, and that higher cognitive functions are more dependent on the neocortex. If the simpler functions are considered cognitive functions, then there is a relationship between cognizing and the neuro system. Therefore, I am wondering, could observing brain connectivities help solve how we think? If we can simultaneously locate the neurons that are active during thinking and the ones responsible for our feelings while we think, could that solve the easy problem of how we do what we do?
ReplyDeletePlease Professor Harnad correct me if i’m wrong, but I'm not quite sure that analyzing the way neurons activate while we think about a specific topic would resolve this issue, or at least we couldn’t differentiate a T4 robot from a human as their brain’s connections would be identical to ours. When thinking about something (an apple for example), the sensitive modules related to what we are thinking about get activated leading us to “feel” the apple; we will remember the color, the texture and its taste but in the best case scenario we’ll be able to distinguish someone’s neural activity between thinking about an apple and thinking about anything else, but we couldn’t say that this particular brain activity is what thinking looks like. And if we could indeed solve our “thinking” problem this way, what about the neural activity while sleeping? Is dreaming part of thinking?
DeleteHi Adrien, thank you for the reply! In my opinion, dreaming is indeed part of thinking since, I cannot think of any reason why it should not be :).
DeleteMy hunch is, we now know that when recalling our third grade teacher, which brain areas stores memory and which ones are in charge of articulation. So the thing unclear is the feeling component of thinking. And I'm thinking that there must be some brain region responsible for generating feelings, which is something that I think might be able to find by analyzing neuron activities.
Tina I think your question is an interesting one, however, as I understand it, studying parts of the brain associated with mental processes would still not allow us to get to the crux of cognition - that is, the hard question. In the same way that I may be able to recall the mental steps I took to remember my third grade teacher’s name (pictured her face, accessed her name, etc), we may be able to trace the parts of the brain involved in this process. However, identifying these parts of the brain and what processes they are associated with does not answer the questions of how my mind stored the memory of my teacher, how I was able to access it at will, and then recall her name. Consciousness and the feelings associated with it are unobservable and to some degree intangible. If we were computers, perhaps pointing to a part of the hard drive and asking what processes were associated with it would help us to understand how we worked. However, there seems to be some piece of the puzzle of how we think, react, and cognize that is missing.
DeleteHey Tina! As a cognitive science student, I think your perspective on mapping neurons is truly intriguing. I'm just trying to see how it might tie into our current discussion. Could you help clarify that connection for me?
DeleteI just wanted to bring up something from our previous lecture that may help you. I believe that it was previously mentioned that we wouldn't go with the argument relating to chemical synapses and neurons. Centrally, humans can not compute like a computer, but a computer can.
Moreover, (answering zoe) in hopes of prodding other discussions, in the previous lecture, we also made the distinction that a difficulty of solidifying the idea of cognition is that we are reverse engineering the situation, unlike building a car or computer.
Professor Harnad’s discussion of what could be considered a machine according to Turing reminded me of our in-class discussion regarding cognitive science as a means of reverse engineering. The quote, “A cloned human being cannot be entered as the machine candidate (because we didn't build it and hence don't know how it works)” (Harnad 2006), concisely summarizes the goal of reverse engineering. This methodology distinguishes cognitive science from other sciences like physics or chemistry.
ReplyDeleteProfessor Harnad’s article helped to further develop my understanding of Turing’s original article as it provided commentary on many of the issues that I was having trouble reasoning through. Namely, that it seemed as though Turing attributed an average potential performance in the imitation game by a computer would be enough to determine that they could think. Whereas, the article references the fact that it is not enough to be mostly good at something in order to prove the fact and it rather must be “completely indistinguishable”. This idea makes much more sense to me because it seems that in Turing’s propositions there are unanswered questions about the definitive nature of whether machines can think or not.
ReplyDeleteOne thing that I found unclear in this paper is whether Harnad was arguing that the TT test candidate must be a physical machine. He argues that ‘any dynamical causal system is eligible for the TT test as long as it delivers performance capacity’. If it must display causality this signifies to me that it must be physical in some way. Harnad also mentions that the machine must display real performance capacity, not just simulated, so does this real performance capacity need to be grounded in a physical system? It seems like physicality is generally important, but what feels unclear is how this plays into the imitation game.
ReplyDeleteFrom what I understood from Professor Harnad’s commentary of Turing’s “Computing, Machinery and Intelligence”, Turing started off on the wrong foot by calling it an “imitation game” as the term “imitation” implies that the digital computer should be able to engage in mimicry, deception or fakery, which does not require actual thinking, thus failing to meet the TT criterion, which is real performance capacity, and not something that solely mirrors it. Professor Harnad suggested the term “methodology for reverse-engineering human cognitive performance capacity”, which I find to be a lot more representative of what the TT aims to be. Later, Turing says that the computer should only give its answers in a written, or typewritten format, but this restricts it from all the other things that we can do and thus makes it more distinguishable from humans. I think this relates to your question of physicality. Turing describes a T2 computer, which is totally indistinguishable in email (verbal) performance capacity, but cannot do actions like humans can (ex: go outside to walk the dog and describe how it felt to feel the breeze…) We are not just interested in what thinkers can do verbally, but rather what they can do that is indistinguishable from everything that we can do, and how it feels like to do what they do. In other words, we are interested in total indistinguishability in physical structure/function.
DeleteI think the reason that Turing seems to characterize the imitation game as a T2 Turing test under Professor Harnad's hierarchy is that he doesn't consider physical capacities part of "thinking" - that is, his definition of thinking is not "thinking is as thinking does", i.e. all the observable behaviour of a thinking being, but instead the subset of these performance capacities that we associate with intention and intelligence. I also think that this is kind of an understandable opinion to hold. For instance, we don't even consider granting the label of ""thought" to the behaviour of machines like a baseball launcher, or even to something with more generalized physical capacities like the Boston Dynamics robots, because they are missing so many of our intellectual capacities like speech and arithmetic. However, it is true that the observable output of our behaviour is often physical rather than verbal, and as Professor Harnad points out, our cognition often depends on sensory inputs other than statements that we read or hear. As a result, a T2 robot could never be a full model of a human's intellectual capacity, let alone their entire performance capacity. But I believe Turing's apparent focus on T2 rather than T3 machines was meant to remove "vegetative" functions from the question of whether a machine can think.
ReplyDeleteMy thoughts about the t2/t3 distinction go along this same line of thinking. When discussing this distinction in the annotations Professor Harnad seems to suggest that a T3 is important for cognitive science because thinking creatures do more than just have verbal capacity and so to truly understand human cognition we also have to go beyond that capacity. I don't entirely disagree but there is an important distinction to be made between some human abilities and things we consider necessary conditions of cognition. As Aya says we wouldn't consider a baseball launcher to display any cognitive ability nor would we ever say someone is less able to think because they are paralyzed. While verbal abilities certainly can't be all there is to cognition I think it would be unwise to attribute everything a thinking being can do to cognition.
DeleteHey Marie and Aya! I appreciate the insightful discussion on machine cognition and especially Harnad's T2/T3 distinction! I like the distinctions made regarding vegetative functions--alluding back to our lecture last week.
DeleteI believe that, in regards to the Turing test, it is true that T3 might address other capacities like machine sensorimotor ability. Then, this 'inclusion' of physical capacities in cognition raises the question: to what extent does machine embodiment contribute to "thinking"?
Professor Harnard discusses Turing’s claim that the question “can machines think” is meaningless, and says “[this question] is not meaningless, it is merely undecidable.” I am wondering if this means that we will never be able to decide/know whether a machine thinks, since we cannot know what another being feels, or is it suggested that if/when we come up with a causal explanation for how and why we think/feel, we could be able to distinguish which beings (machine, person, object or otherwise) thinks?
ReplyDeleteMy understanding was that this is question is not really necessary to what Turing is setting out to say (though it is a good hook). This paper begins with saying that the only way we know we think is from neural correlates, only if we are sure these correspond with thought, and the introspective feeling of thinking. We cannot introspect for each other, so we certainly can't for the Turing test. To quote Prof. Harnad's paper, the test is whether a machine can "do what thinkers like us can do", and the question of "how" is where reverse engineering comes in. For the sake of the Turing test we don't really need to know if a machine can think, the same way we can't really be sure that other humans think. Like with humans, the only evidence we have of thought in others is the observable output of this thought, which a machine that passes the Turing Test is capable of.
DeleteProfessor Harnad gives insight on what is written in the last reading on Turing’s imitation game and clarifies its application to cognition.
ReplyDeleteI feel as though the current use of AI in many cases is a real-life execution of the Imitation Game. Will your teacher know you used AI to write your paper? Alternatively, I have seen a case where an email written by a human being was incorrectly misjudged as AI. It is intriguing that AI has advanced so much that it often produces indistinguishable results from human counterparts. Of course, as noted by Harnad, human capabilities extend greatly beyond ‘email interactions’ or T2 level functioning. The Turing Test hierarchy also extends into T3 level functioning, which includes sensorimotor functioning, however still focuses on input/output performance capabilities. According to Harnad, this level should be what determines the pass/fail of a Turing Test. T4 level functioning means the machine could also match the internal functional or “how”. T5 level functioning would mean that both the hardware and software of the machine are identical. T4 and T5 are ‘rejectable’ as we do not know how to mimic the functioning of the human brain, so to evaluate T4 is not possible, and T5 would be impossible because it would require scientists to fully replicate a human being.
The discussion of free will is something I have not previously considered. I suppose it is a question itself, whether or not humans truly have free will. With respect to the Turing Test, it seems contradictory to its strict rule-based nature and discrete states that a machine would have any free will, regardless of its performance capacities.
Hi Kaitlin! I think the issues you mentioned about papers and emails being wrongly attributed to AI rather than humans is an excellent example of the Gallup poll issue that Professor Harnad points out in the paper. Many of Turing’s original requirements for the Turing test rest upon a large percentage of people not being able to appropriately distinguish between the machine and the human, essentially leaving it up to a vote. This is insufficient, as Professor Harnad points out, in order to truly meet Turing's indistinguishability-criterion the machine’s performance has to be completely indistinguishable from a human’s “to anyone and everyone, for a lifetime.” While I agree with this standard of testing in theory – it certainly makes much more sense than using a statistical survey – I do wonder how this could be carried out in practice. Then again, that doesn’t seem a pressing concern given how far away we are from a likely candidate for T3.
DeleteI also think when discussing the Turing test it's worth thinking about the level of understanding the general public has of AI, as well as their existing attitudes towards it. Earlier in this thread Professor Harnad attached a sample of ChatGPT’s summary of an article excerpt – while some of the summary’s weaknesses are apparent to us by virtue of completing the reading and having a basic understanding of AI, I feel as though someone with limited background knowledge would be at a disadvantage when judging whether the work is human-made or not. I also think the examples you included illustrate people’s desire not to believe in a computer that could pass the test – does the Turing test require that people know from the start that there’s a possibility they could be interacting with a machine? If so I think there’s a danger of bias in people not wanting to believe that a machine could “trick” them.
The question I left open in the previous article is actually clarified in this one. Given that the imitation game is not an empirically scientific experiment designed for testing human cognitive performance capacity, and we should consider a methodology for reverse-engineering human cognitive performance capacity. So, why does creating a machine that is indistinguishable from humans solve the problem of explaining our cognitive model? I notice that the answer provided by this article to this question is that such a machine, capable of generating our performance capacity, would assist us in understanding what kind of machine are we. However, it still leaves the question of “how” unanswered because having such a machine doesn’t necessarily imply that it would function in the same way as humans do. It might achieve the same performance capacity through a different mechanism.
ReplyDelete"The fact that eyes and legs can be simulated by a computer does not mean a computer can see or walk (even when it is simulating seeing and walking)."
ReplyDeleteTrue in the sense that I assume that it is not hooked up with the necessary tools (eyes/sensors and legs) to see and walk but the sentiment is one that I find dubious.
Harnad’s point about really doing in the real world vs just doing in a simulation as a way to immediately discount the possibility that simulated thinking is not the same as real thinking is one that sticks in my mind more than the rest of the annotated paper. And as I understand it I agree with the sentiment of realness being something of value instead of where the direction I feel myself going in where realness is merely a accidental property of the thinking and doing in real life that can theoretically just as well be done in a simulation. Yet, I fear that this value I (we) ascribe to realness is founded on bias more than it should be.
**Repost**
DeleteI found myself thinking the same things as you WIllem. It does not seem right to completely discount simulation as a part of thinking. Yes, simulating eyes and legs would not give a computer the ability to see or walk, but it is completely conceivable that a computer may have this capacity if it was integrated into the appropriate physical implementation: the sensors and legs you described. It seems to me that we also do quite a bit of simulation in our minds, such as when planning an action in a sport for instance. Who is to say the simulations we experience in our mind are any more real than the simulations a computer generates? We cannot say this, and here we encounter the mind/body problem. One thing we can know however, is real performance. I think this is the point Professor Harnad was touching on. We cannot speak to how real a computer’s simulations actually are, so we must look to actual (T3) performance in order to gauge their capacity to think.
Professor Harnad’s paper “The Annotation Game” clarifies some of the difficulties I had with Turing’s paper, and delves into the implications of Turing’s argument in “Computing Machinery and Intelligence.” I found the discussions on pages 16 and 17, regarding the ability for machines to learn about the real world through simulations and language, particularly interesting. Professor Harnad notes that a digital computer is merely simulating the real world and the sensorimotor experiences found within it, and by doing so it misses out on some of the properties of these experiences. This led me to consider how much of the real sensory world a machine can learn—is it like a human purely experiencing the world through simulation? Could a machine learn some things, or enough to perform indistinguishably from a human on the TT? Further, because the TT is concerned with performance capacity, rather than “thinking,” does it matter? Can a machine perform as a human does without being exposed to the real, sensory, world?
ReplyDeleteWhat are these properties of experiences that a T3 is missing out on? As long as it has the correct cause-and-effect sensorimotor associations and an accurate simulation of the real world, I should think it is eligible for the T3 test. Maybe I am confused by the concept of simulation, because I think of it as able to learn and interact with the real world, even if simulated. But as Professor Harnad writes, it is is merely a practical methodological strategy, useful for planning, designing and even pre-testing the T3 robot. I think it becomes more clear when we make the clear distinction between thinking (a robot acting in a simulation; not physically observable) and doing (empirically observable behavior).
DeleteHi Csenge! This reminds me of the discussions in class which highlight that a T3 robot is not merely a chatbot with a camera attached. From my understanding, I think that for a T3 robot to have experiences with the world such that it can learn what we learn from the environment, and by extension do what we do, the T3 must be able to have autonomous interaction with the sensory world (even if it is simulated), and get the feedback from this interaction. Looking back at this skywriting, after having progressed throughout the course, I am now considering if training a T3 robot in a simulated world may be a viable option. I think the underlying sentiment this skywriting was aiming to address is how could we program a T3 robot to know what to do with sensory stimuli--a sentiment that I believe has been addressed by this course.
DeleteProfessor Harnad’s commentary on the imitation game sheds important light on the distinction between the hierarchy of Turing’s test. The original paper we read had me confused at what level we were supposed to be aiming for, and without clarification I would have assumed the original paper meant only a T2 test. Clarifying why Turing meant T3 (or why he should have meant T3 if he didn’t) changes the goal of the Turing test for me. That goal being a complete reverse engineer of human sensorimotor performance (the ability to generate it indistinguishable from a human). What drove home the point is the commentary on simulations. Simulating anything in a virtual world is useless in regards for a Turing test because nothing operates in the simulated world.
ReplyDeleteIn my previous post, I mentioned that the TT "avoids the hard problem," but I now comprehend that this was an inaccurate characterization. The Turing Test is only capable of evaluating an entity's ability to do things. While the TT can address the "easy problem" of understanding how and why organisms can do what they do when thinking, the "hard problem" of why and how organisms feel remains a separate and complex challenge. The distinction between doing and feeling is essential, and the TT primarily addresses the former, emphasizing that the hard problem of feelings cannot be resolved through the TT alone.
ReplyDeleteProfessor Harnad raises several important points and interpretations regarding the Turing Test. I had a difficult time remembering the specific details related to the hierarchy of the Turing Test. He introduces the hierarchy very clearly, ranging from T0 (local indistinguishability in the capacity to perform some arbitrary task) to T5 (total indistinguishability in physical structure and function). Reading this section greatly improved my understanding of the concept. In addition, I found the part about the Theological Objection intriguing as it delves into philosophical and ethical inquiries concerning the essence of consciousness, intelligence, and the uniqueness of humans. The debate over whether machines can genuinely engage in thinking and possess consciousness continues to be a subject of ongoing discussion, transcending the boundaries of scientific inquiry.
ReplyDeleteI agree with Harnad’s interpretation that Turing’s digital computer was a T2 computer, which is only indistinguishable from humans in the verbal domain. What I’m interested in is the hierarchical structure of the Turing test from T2 and T3. Usually, in cognitive neuroscience courses, we learn that primary sensory and motor areas are the first physical structures to receive inputs from the external world, which are processed accordingly and sent to multimodal association cortex to be integrated for higher order cognitive processing (weasel word: thinking). I find it interesting that the hierarchy proposed by Harnad moves in the opposite direction, where it is easier for the computer to imitate human thinking with language (which is considered a high-level cognitive ability that distinguishes us from other mammals). Contrary to what we learn in class, in order to determine the causal mechanism for thinking, we may need to consider how humans interact with the real-world through our automatic sensorimotor capacities…
ReplyDeleteI have some doubts about my understanding of the passage distinguishing simulation from reality. While the difference is clear for a rocket, an eye, or a leg (which have physical components) it is not the case for a robot or a machine. To my understanding, a robot is a physical machine that can autonomously interact with the environment by integrating software programs. Only a robot could pass T3 while a digital computer could only pass T2 (provided computation is sufficient). Thus, a robot could be either real or simulated. But if a robot that passes T3 can be simulated on a digital computer does that mean that it is a universal machine, and that the strong hypothesis is true?
ReplyDeleteI had a similar inquiry and I just want to include some of my thoughts on it. So a robot that passes T3 can be accurately and entirely simulated on a digital computer would indicate that the computer is capable of emulating the sensorimotor performance of the robot to a degree where it becomes indistinguishable from the real robot. The simulation would replicate the robot's actions and interactions with the environment. To my understanding, the strong hypothesis posits that a machine running the right software (as you mentioned) would also have to possess a genuine understanding and state of feeling (consciousness) while passing T3. I think if a digital computer can simulate a robot’s T3 performance indistinguishably, it would imply universality by its capability of replicating complex sensorimotor behaviors. However, because of a lack of inherent understanding and feeling, the strong hypothesis could not be true. (I might be confused about the strong hypothesis though).
DeleteBy the way, I highly recommend watching the Westworld series (HBO). In addition to the excellent script and aesthetics, it reflects a great deal on AI and consciousness in relation to the issues addressed in this course.
ReplyDeleteThis article discusses whether Turing thought T3 was necessary, or if T2 was enough. Perhaps at the time of the TT’s creation machines did not show the possibilities they do today. Today, it is clear that a machine is more likely than a computer to perfectly imitate a human, as machines include a combination of the digital and analog (a combination humans are believed to have). As Prof. Harnad states, the TT only testing verbal skills is not a full test of human-like performance ability. A machine that can perform generic human actions such as walking and vision would be nearer to the ideal of a machine totally indistinguishable from humans (in performance). This is the fundamental goal of the TT.
ReplyDeleteIt is absolutely true that modern machines possess a broader range of capabilities, like analog and digital components, which enhance their ability to imitate human actions beyond mere verbal skills. Acknowledging these advancements is vital in understanding how the Turing Test's goals may have evolved over time. It's interesting to consider how contemporary AI aligns with Turing's original vision, and what type of tests he may have saw fit to deal with current models, especially ones that are incorporated into elaborate machines. It is only a matter of time before complex machines coupled with advanced AI become indistinguishable to the average person, maybe at this point it will be imperative that we have some sort of measure to differentiate them from us.
DeleteOne common theme I found throughout the responses to Turing’s statements was that although many of the points were thought through and not necessarily false, they were ineffective to countering the turing test as they forgot the long term ending goal of the turing machine: The question is not “can machines think”, but it is can machines do what humans do. This question is important because it can help us understand how humans do what they do. Scientists are interested in such questions because from there they can then work on enhancing our abilities at doing those things that are beneficial, or understand how to decrease those things we do that are harmful. In general there is always the centering motivation to be kept in mind of improving human existence and life quality. It is thus not the end goal to create a machine with capabilities to replace humans, but rather a machine that can be used by humans to reach their goals. This seems to also connect to questions of AI ethics and what skills we want AI to be focusing on, as well as how exactly would be the most beneficial way for humanity to integrate AI in society.
ReplyDeleteThe paragraph beginning with “Turing is dead wrong here…” brings up a question that I have kept on encountering in the past two weeks. The annotation reads, “There is no one else we can know has a mind but our own private selves…” What Turing (wrongly) deems as solipsism seems to be saying something along the same lines. However, my question is: how can I justify the fact that I know, on my own private own, that I have a mind? Is it sufficient for me to prove, if not to others around me but at least to myself, that I have a mind just because I know that I do and because I can feel myself feel? I don’t understand how this would be different from introspection which, as discussed last week, is an insufficient explanation because our brain is blind to our own blind spots.
ReplyDeleteOhrie, I think your questions are interesting and touch on the fact that as people it is hard for us to get to the crux of cognition partially because it is intangible and unobservable. However, my best answer to your question of how you know you have a mind is that you are able to have thoughts, beliefs, feelings and ideas which go completely unexpressed and exist solely in your mind. A robot which perfectly mimics human behaviour would act and compute in the way a person does, but presumably would not be thinking to itself “Man it’s been a long day.” The fact that I can think this thought to myself, along with many others, points to the existence of my consciousness. However, simply identifying the presence of your own consciousness is not the same as trying to use introspection to understand how that consciousness works, as we discussed previously. I may know that I had to think to do something, but that does not mean I understand how I did that thinking.
DeleteZoe, It's insightful how you explain self-awareness and accepting one's consciousness.
DeleteUnderstanding that our internal thoughts, beliefs, and emotions exist even when not externally expressed is a fundamental part of comprehending our own cognition. Your observation regarding the major difference between acknowledging the presence of our consciousness and understanding the underlying cognitive processes through introspection is important in my opinion.
One of my reflections is about the feeling of thinking. In the paragraph that replies to “the original question ‘can machines think’ I believe is to be too meaningless to deserve discussion”. Professor Harnad replies that there are 2 main questions that we are asking ourselves: what and how thinkers think, and what does it feel like to think. The first set of questions is answered: the TT is a good way to assess what they can do and reverse engineering answers the how. However, we don’t have a way to determine whether a creature feels that it’s thinking. My question would then be, is it really necessary for us to determine whether the creature has the feeling of thinking? If every other behavioral output, whether it is language, body language, motricity, etc. is undistinguishable from a human being, does it matter if the feeling of thinking is absent, and does it matter if we are unable to assess whether the feeling is present or absent? I think this point was raised in the paper, but we never doubt another person feels because they act in the same way as we do, so if a machine was able to do the exact same thing, we wouldn’t question whether it feels when it is thinking.
ReplyDeleteIn "The Annotation Game", Professor Harnad speaks about the different levels of the Turing Test, highlighting the importance of total indistinguishability in performance in determining what thinking creatures can do, as summarized in the following quote, "the Turing Test is about finding out what kind of machine we are, by designing a machine that can generate our performance capacity, but by causal/functional means that we understand, because we designed them".
ReplyDeleteI found the section concerning the argument of consciousness to be particularly interesting. Here, professor Harnad states that "nor is it relevant what stuff [humans] are made of, since our successful mind-reading of other human beings has nothing to do with what stuff they are made out of either. It is based only on what they do." This made me think of the T3/T4 distinction, as while I agree with Turing's quote in that "[There would be] little point in trying to make a 'thinking machine' more human by dressing it up in such artificial flesh", I feel as though the act of assuming that other humans think based on their behavior is facilitated by the fact that they are human beings and not robots.
Omar, you perfectly summarized what I came here to comment in your second paragraph! I agree with you that we have no qualms about not being able to know whether other humans are thinking as we do (since they are made of the same 'stuff' as us), but we do question machines' ability to do the same (partly because they aren't made of the same stuff as us) because we want to understand how they manage to be like humans.
DeleteHi everyone!
ReplyDeleteAfter reading his work, I’d like to say that I appreciate how Professor Harnad organised extensions of the Turing Test as potential empirical tools to assess sentience in another entity. I also liked his justification for why the T2 version of the Turing Test has limitations. Moreover, in the reading, Professor Harnad emphasises how the line separating T3 and T4 is hazy because the internal indistinguishability trait of T4 may be crucial in some aspects of external indistinguishability, such as blushing. However, after having carefully read every T level that Professor Harnad describes, I can't help but sense that the hierarchy categories are chosen somewhat arbitrarily. This makes me wonder if perhaps it would be more useful/interesting to think of the categories that Professor Harnad presents, from T0 to T5, as a continuum rather than as set levels of hierarchy. Let me know what you think!
Hey!
DeleteThat’s an interesting point. However, while the categories may appear discrete, they represent a gradual progression of complexity. The reading specifies that machines at higher levels like T4 or T5 possess more advanced capabilities that build upon the foundations established at lower levels like T0 or T1.
In that sense, I think that categorizing them hierarchically is purposeful as it provides a framework that allows us to understand their gradual differences. T0 lacks the ability to perform tasks other than the ones it was intended for. T2 was the original TT and is indistinguishable in *verbal* capacity (which is an added capacity). Your observation about the blurry distinction between T3 and T4 is valid. Both involve sensorimotor performance capacities but T3 seems to emphasize internal processes and external performances. It seeks to mimic not only the outcomes of cognitive tasks but also the underlying mental processes that drive these outcomes. T4 introduces the physical aspect ; it involves machines that have physical presence for external performance and interact through their bodies (which we don’t need to prove that the system seems to DO all that we can DO).
"There is no one else we can know has a mind but our own private selves". This sentence really struck a chord in me. I had never really thought about the certainty of other beings thinking. It is true that we cannot know for sure. However, interestingly if we describe whatever a machine does that resembles thinking as thinking we can in that case know for sure that machines think (contrarily) to humans.
ReplyDeleteYes, "I think therefore I am" only applies to the thinker themself, and does not confirm the same for others. Another comment(from Omar) argued that an important part of us believing that others do think, is that they are indistinguishable from us, which is why a T4 would be more convincing than a T3. I agree that this would absolutely help with first impressions and make for a very convincing case, but I take issue as to me, behaviour plays a larger role in assuring us that the "machine(I use this word not meaning solely computers, but to any dynamic causal system) in front of us thinks the same way we do.
DeleteI appreciated that Professor Harnad clarified that this "problem" that Turing described is not solipsism(which to my understanding is believing that one's self is the only real thinker with free will, though please correct me if I am wrong), but rather another occurrence of the other-minds problem.
Prof Harnad's comment that the material of the "thinker" does not matter, given that we don't understand how the material we are equipped with(nerves, tissue) brings rise to thoughts was particularly insightful to me.
I am confused about the t0 task, what does it mean when they test the machine to see if it can do anything else, other than play chess? And from what I understand, the t5 level mean that it is indistinguishable physically and functionally, so basically creating a machine that is human?
ReplyDeleteWhat I took from this reading is that machines might answer the “how” question (how we can do what we do) but cannot answer how we feel when we think. So essentially, what separates humans and machines would be feeling, not thinking. Also, from Lady Lovelace’s Objection, would we not consider anything to be original if a machine comes up with it, since it would be humans that provided any and every symbol/rule in their table of instructions.
The T0 task just simply refers to a baseline assessment on whether a specific machine can perform a given task other than chess as you brought up. It assesses a computational program's ability to handle different types of tasks, as opposed to a simple specialized one such as the chess example. When it comes to the T5 level of the Turing test, this is where a machine or computational program would be completely functionally indistinguishable from that of a human. Not so much as seeing that the machine is human per se, but rather in a sense of not being able to distinguish what makes it not human. Which I do see your point. At that given level, would the machine not just be considered human? It’s a really cool idea to bring up. Is there ever a level where the machine could surpass the understanding and consciousness of a human?
DeleteAre T3 and T4 really different from each other? As mentioned by Dr. Harnad, the boundary between T3 and T4 is one that’s fuzzy. In fact, in the reading, T3 is presented as having total indistinguishability in robotic sensorimotor performance capacity. Does that mean that, for example, on top of being able to behave like a human in verbal capacity, the robot can also have the human experience of touch? Similarly, T4 is presented as having total indistinguishability in external performance capacity as well as in internal structure/function. To me this sounds extremely similar to what T3 would be, the only difference that I remark is that T4 would have a physical body whereas T3 wouldn’t. additionally, in the video associated with this section, professor Harnad mentions that T4 requires indistinguishability in all the observations that we can make about what’s going on inside the head. So, does that mean that they physical body of T4 is an exact replica of the human body?
ReplyDeleteFrom my understanding, T3 and T4 refer to different levels of artificial intelligence development. T3 aims for indistinguishable robotic sensorimotor performance, which could include mimicking human touch experiences. T4 goes further, striving for both external performance and internal structure/function indistinguishability. T4 may have a physical body similar to a human, but not necessarily an exact replica. It aims to replicate not just actions but also internal processes, though not necessarily replicating every aspect of a human body. The boundary between T3 and T4 can indeed be fuzzy, as they both target high levels of AI capability but differ in the extent of internal replication - which I find really interesting to learn about as well!
DeleteI'd like to attempt to kid-sib-summarize Turing's Paper:
ReplyDeleteTuring's paper introduced the Turing Test, an empirical test of whether a machine can do all the things a human can do (or more specifically a thinker) in a manner indistinguishable from a human. Should the machine prove able to do all the things a thinker can do, then our (assumed) knowledge of how it accomplishes that would teach us much about how it is humans accomplish the same task.
--
There are some points from the readings that continue to confuse me. Firstly, I'm not sure I totally understand the distinction between an discrete state machine and a digital computer. Second, Dr Harnad takes Turing's assertion that only digital computers be eligible for the test to mean they cannot be the types of machines that have are indistinguishable in sensorimotor capacity, that they need only have indistinguishable verbal capacity (Harnad, 2008, pg 12). Is Harnad arguing that the Turing Test formulated thusly would exclude the only types of machines which could feasibly pass the test? If so, why couldn't a digital computer pass the test? I'm having trouble putting this confusion into writing, so hopefully we get the chance to discuss in class.
From what I understood, a discrete state machine refers to any machine, or real, physical system, that can carry out tasks in the real world. Think of it like a real airplane that's actually flying. On the other hand, a digital computer processes information – manipulates symbols - based on specific rules, dealing with discrete data. However, while it can simulate or represent real-world actions, it doesn't perform those actions. Imagine a computer program that simulates an airplane flying – it shows how it would happen but isn't physically flying itself. Turing suggested that only such computers should be considered for the Turing Test.
DeleteDr. Harnad challenges Turing's perspective by stressing the difference between doing an action and merely simulating it:
- A machine that claims it can fly should genuinely fly, not just show a simulated flight on a screen.
- Similarly, if a machine says it thinks like a human, it shouldn't be judged solely on its ability to chat. It should also have sensorimotor capacities, performing tasks in the physical world like a real human or robot would.
By limiting the Turing Test to only digital computers, we would not capture the full essence of human thinking, failing to meet our goal of creating a system that is completely indistinguishable from a human.
How much of the current development of T2 and T3’s are capped by hardware limitations? I know that it takes tremendous amounts of resources to train ML–and can only imagine how much it would take to make a cognitive model. I assume that a fully realized T2 is limited by compute power. We also see companies like Boston Dynamics making very capable robots, but how long until those robots can be indistinguishable in performance capacity? And at scale, at that. T3 requires a full T2 plus adequate robotic hardware to “encase” it—which provides a whole set of engineering and logistical issues (none that I assume would be unsolvable with our current technology, but rather needing immense resource allocation, the likes of which, I don’t know we have).
ReplyDeleteHi Rohan, is it that T3 requires a full T2 or that T2 requires a full T3? I remember in class Professor Harnad mentioned that you need to be able to engage in the cognition that humans engage in (categorization) in order to actually use language the way that we do. I think that this has become more clear than ever with the dawn of ChatGPT: in a sense, it passes T2, but it doesn't really (big gulp) and you could only really imagine that it would pass T2 without the big gulp if it engaged in the very same cognition that we do. I wonder how Turing would modify his argument with respect to this perspective?
DeleteProfessor Harnad's analysis of Turing's paper deals with the text critically and assesses the test as multi-faceted and as comprised of levels (T0-T5). These levels help construct a method by which to reverse-engineer cognition. He points out the issue with considering it an imitation game in our objective not just to have computers imitate or replicate natural language, or to only get it right most of the time, but to actually demonstrate a consistently indistinguishable performance capacity. I find the clarification of cognitive modeling and artificial intelligence very valuable, as we must be intentional in our analysis of the higher level Turing tests and what is relevant in our inquiries about the hard problem.
ReplyDeleteWhen describing the Turing Test, Harnad describes two Criterion. An empirical one : are these performances identical? And an intuitive one : "drawing on what decades later came to be called our human "mind-reading" capacities".
ReplyDeleteThis made me wonder if people had thought to create a reverse Turing Test. In this test, a machine would have to determine who is a human. This would exploit this "human "mind reading capacity"". Now, what this would prove is more uncertain. It might be rather that they recognize other machines. Some might use this to argue for some form of "self-awareness"(weasel word used intentionally) or recognition of species. I just thought this was an interesting question.
Emma, first, the criterion for TT is (cognitive) performance capacity equivalent to and indistinguishable from human performance capacity. Some people are good at playing violin, others are not; this is not part of TT. Some people are good at mind-reading, others are not. But every normal human being can learn categories and language. Cogsci is about reverse-engineering cognitive capacity, and the TT tests whether the candidate causal mechanism succeeds in doing it. What would the reverse TT be testing, how, and why?
DeleteThis text aims to analyze Turing's thoery and presentation of the Turing Test. And as mentioned by everyone previously, Harnad has made an excellent job at highlighting the important aspects and contradicting what seems more precarious in Turing's statements.
ReplyDeleteHowever, even if everyone seems to have understood these concepts quite clearly, I have to admit that I am entirely confused with the different levels (T0-T5) of the machine, what they represent, what are their particularities concretely, and whether they can all be encoded. If someone could enlighten me on that, it would be amazing :)
Hi Juliette, let me try to best explain the hierarchy of TT which tries to test whether cognitive science has successfully reverse-engineered cognition.
DeleteIn order to pass:
- T0, the candidate must be able to perform some arbitrary task which makes it indistinguishable from a human (ex: chess-playing). It's not considered to really be a TT because the candidate can easily be distinguished from a human by seeing if it can do anything other than this arbitrary task (if it cannot, it fails the test).
- T2: subsumes chess-playing. The candidate must be able to communicate verbally or through email in a way that's indistinguishable from a human. However, a T2, for example cannot stargaze.
- T3: subsumes T2. The candidate must be able to be indistinguishable in sensorimotor (robotic) performance capacity (indistinguishable when interacting with the world).
- T4: subsumes T3. The candidate must be indistinguishable in external performance capacity and internal structure and function (neural processes should be indistinguishable).
- T5: subsumes T4. The candidate must be indistinguishable in physical structure and function (all molecules should be indistinguishable from a human).
Professor Harnad's commentary brought to my attention the very very broad definition of "machine" that we're operating with. According to Harnad, a machine might be "any dynamical, causal system," like waterfalls, rose-bushes, or, as he suggests, even the Universe. I'm not convinced that consciousness is an inherent quality built into the universe, as panpsychism suggests, but I couldn't help seeing a similarity here that, if thinking is just computation, and all these things are machines, might suggest something in the direction of it. Of course, there's a lot of other reasons this definition of a machine wouldn't line up with a panpsychist view, mostly because the "thinking" or computation of a tree-machine would be awfully different to that of a human mind, but it's just where my head went !
ReplyDeleteThe concluding paragraph of this essay, in which Turing briefly touches upon unconventional phenomena, is somewhat surprising. While Turing's work has been a significant milestone in our understanding of computation and cognition, his mention of the potential existence of these phenomena raises intriguing questions. Given Turing's reference to statistical evidence for telepathy and psychokinesis, it prompts us to consider how Turing would have been able to reconcile these phenomena and his perception of machine intelligence and the standards used to evaluate thinking machines.
ReplyDeleteIn this article, a similar point of view is mentioned in a previous article. It is the human thought, the social behavior of human beings to think about problems, and the special function that human beings have always emphasized. No matter from the perspective of a theologian, formalist, or thinker, this function cannot work with calculation formulas or computation. This is spiritual and non-physical.
ReplyDeleteIn my opinion, this is a truly unsolvable problem. And installing this non-material soul on the machine will produce the ability to think like a human being. In terms of hardware, according to current technology, this is very difficult. After all, the thinking soul is a non-material existence. In terms of moral concepts, from my point of view, this destroys certain regularities and natural laws. Maybe my thinking is relatively conservative, but letting a machine think like a human, or even exceed human capabilities, is something that needs to be taken seriously. It has pros and cons. The advantage is that it will allow human civilization to develop rapidly, but the disadvantage is that humans will lose a certain degree of dominance.
It’s true that this would be very hard to achieve with what we have today due to thinking being an intangible process. “The fact that eyes and legs can be simulated by a computer does not mean a computer can see or walk”. The question is not whether a machine will ever “think” like a human as it is not a living organism with a brain and a nervous system but rather what a machine would be capable of doing by being able to with the capacity to stimulate human-like responses. Your response highlights the complexity of integrating machine thinking into our world. The potential for rapid progress is enticing, but it does raise concerns about the shifting balance of power. If we ever achieve the capability of embedding cognition into machines, we will need to implement safeguards to prevent the potential consequences you mentioned.
DeleteProfessor Harnad discusses the limitations of a T2 machine that would prevent it from passing the TT - namely, that a T2 machine would fall short in “the kinds of verbal exchanges that draw heavily on sensorimotor experience”. At the same time, a machine operating in virtual reality, given that it is not operating in the real world, would be considered a T2 machine. I found this to be an important distinction that at first did not seem intuitive to me - initially, I thought that a machine operating in virtual reality could be considered T3 so long as it could hold discussions about the sensorimotor experience of the simulation. However, for the moment we have accepted that "thinking is as thinking does" - and we know for a fact that whereas in reality, a human could simply take off the headset simulating the virtual world, a machine in the simulation would have no capacity to do so. This machine would have no capacity for discussions of real sensorimotor experience, only simulated, and hence it falls short of T3.
ReplyDelete
ReplyDeleteThis paper clarified a lot of things that I had interpreted incorrectly before. Previously, I did not believe that Turing was attempting to actually attain T3 and only was aiming for T2, and I was fairly confused during his discussion of attaining them. This helped me clarify that he was utilizing the verbal performance as an example of priming, not to assume that you need to be verbal to think. After reading this article and gaining some clarity on other parts I was confused on, I believe he may be trying to convey that if there is a machine that can do computation that cannot be told apart from human cognition that cognition is computation. In his beliefs, it seems that this would stop the debate on if computation is cognition.
I think Professor Harnard does an impressive job at breaking down the turing hierarchy in a manner of which I can understand. From what I can understand, when a machine falls under T0, it can be easily distinguished from a human as it lacks the 'cognitive' ability to perform tasks other than the arbitrary task it was intended for. For example, when we think about a 'computer' opponent in a game of chess, snakes and ladders or even monopoly, if we ask this machine to perform some other task, it can't. In terms of T2, prof Harnard makes an interesting point about it being the original TT that turing intended it to be. It makes sense that we eliminate any audio-visual aspects of the tests as it provides anonymity for the entity that humans must interact with during the TT. If we see a robot, immediately it defeats the purpose of the test. But under T2, a machine cannot perform physical tasks, it cant harvest berries. It fails the TT because there is a distinct physical difference. Now what confuses me a little bit is the blurry distinction between T3 and T4. I think there is a big overlap between what we consider to be sensorimotor performance capacities and internal/external performance capacities. Don't they essentially mean the same thing? Then when we look at T5, I feel like we really stray far away from the original purpose of the TT, because if the machine is indistinguishable up to the molecular level then it goes beyond performance level (as it does with T4). So what do we consider a machine that falls under T5?
ReplyDeleteI am coming back to answer my own question after I finished the course, now I understand what a T5 robot can be and what it entails! I still would say that T5 strays from the original goals of the TT, nevertheless it has an interesting application that can give us insight into the realm of the HP of cognition. A T5 robot essentially should have not only the capacity to do, but also the capacity to feel, as descartes would say, possesses the capacity to observe cogito, 'I think, therefore I am'. I believe that we can only ever attain a fully functioning and autonomous T5 robot only when we can address the OMP with certainty, but then we need to also address the HP, which arguably is alot harder due to our current reliance on first-person research since, we still don't possess the ability to literally mind read everyone and know what they truly feel and how they come to feel, so we need to develop some sort of objective way to research the concept of feeling. Only then, can we ever achieve the possibility of a T5 robot!
DeleteGodel’s Incompleteness Theorems have always fascinated me, not least because of their widespread influence in mathematics, and by connection physics and other sciences. The Incompleteness Theorems apply to all formal mathematical and computational systems, whether we like it or not, and demonstrate that there are certain things within the scope of such a system that are undecidable by that very system. For example, in 1943, Kleene proved that the Incompleteness Theorem applies directly to computability, which is what gave rise to the undecidable nature of the Halting Problem. In summary then, there are certain things that computation, and computing machines, simply cannot do. Despite how compelling these theorems are, I agree with Harnad that applying them as an objection to the Turing Tests is naïve. To claim that the mind has limits in the same fashion that an axiomatic system has limits is to say that the mind is entirely computational, which I wholeheartedly disagree with. There are many reasons to believe that thought is not computation, but one refutation I particularly like is Searle’s Wall argument. His point is that, under the computational theory of the mind, any system should be capable of implementing thought, even a wall, provided it has enough states, because thought is a computation. This is obviously nonsense, and so applying Godel’s Theorems as a refutation to the Turing Test would be to incorrectly assume that our mind itself has the same limits as a machine because we are as computational as the machine itself. However, despite agreeing with Harnad that this is an ineffective argument, I believe that we disagree with it for different reasons. I do think that Godel demonstrates that certain things are unknowable, in light of them being unprovable: I don’t believe you can know something to be true unless it is proved to be true. You can know and understand X, because that is not computational, but you cannot know X to be true until it is computationally proven so (or not).
ReplyDeleteOn page 16-17, we find the following passage: ''But even just for T2, the question is whether simulations alone can give the T2 candidate the capacity to verbalize and converse about the real world indistinguishably from a T3 candidate with autonomous sensorimotor experience in the real world’’.
ReplyDeleteIf we were to give a machine a corpus* on things that have to do with our senses (ex: fire is ‘warm, hot’ etc.), it might be able to make us believe that it does know what being next to a bonfire feels like. In other words, since the T2 candidate can simulate sensorimotor experiences, if we were to give it a corpus on what these simulated experiences ‘’feel like’’, I wonder if it may be able to get a step closer to T3.
*corpus is defined as ‘’ all the writings or works of a particular kind or on a particular subject’’ by the Merriam Webster dictionary for those who are not familiar with the term
Also, I want to mention that I absolutely adored the passage where you thoroughly refuted Turing's reference to solipsism. I was unsure about his argument's credibility in that passage when I read it, but was unable to come up with a refutation by myself and it was very satisfying to see it articulated in your paper.
Professor Harnad's explanations on Turing's paragraphs made me feel more confident in understanding the basics of a TT, particularly about how the indistinguishability should be a total one, rather than statistical or probabilistic ("more than not" way) one. The performance capacity should be empirically observable, even though thinking (cognition) is an internal state which we cannot observe. This aspect is a bit confusing for me when I focus on the question "can machines think?", yet I believe that with a focus on solely performance capacity, (totally indistinguishable than human capacity) I can wrap my head around it.
ReplyDeleteI am still not sure if I understand what "mind-reading" capacities refer to, as it felt like it has something to do with ESP/telepathy. In other words, the distinction between the empirical and intuitive criterion Professor Harnad made when discussing the Turing criterias felt like the intuitive cues for distinguishing human vs TT performance capacity is not observable, and it is left to people's intuition, almost like a gut feeling. I am sure this is not the case but I was just wondering if anyone can enlighten me on that, thank you!
first of all, and something that I think is comical given the context, it is nearly impossible to put out of my mind that this is prof. Harnad's writing. second of all, something that caught my eye was on pg 9 when he mentions that most of us could not beat a grand master at chess. I, for one, could not beat anyone at chess and I wonder how a machine could account for this, because surely it must be important to model the shortcomings of a human machine. I am confused as to why we are not concerned with disabilities. unless I am misunderstanding what is meant by "diasbilities" here.
ReplyDeleteI think you are misunderstanding what is meant by disabilities in this context as professor Harnad clarifies that Turing considers disabilities to be non-abilities rather than an actual handicap. Turing himself states that one should not "penalise a man for losing in a race against an aeroplane", which drives home the point that Turing is looking to test for performance capacities that all humans are generally capable of. He is not interested in individual specific abilities like being able to play chess at a very high level, whereas he may be interested in a machine's ability to learn the game of chess through learning each piece and various openings; something that most humans are also capable of doing with time.
DeleteProfessor Harnad goes into detail of what it truly means to pass the turing test, explaining the different levels from t0 to T5, as well as some of its limitations. He emphasizes that it should be able to display real performance capacity and not just imitate what a human can do. He also explains how the Turing test applies to Cognitive Science : it “provides a methodology for such cumulative theory-building and theory-testing”. It is basically to build a cognitive model and explain how human cognition is generated through reverse engineering.
ReplyDeleteFirst of all, the title is amazing. Couldn't be better. Could a machine come up with that title? The content of the paper, really made me try to figure out why we are trying to make a machine indistinguishable from a human. I understand that from a science perspective it is an amazing breakthrough but practically what would it help with that computers today (distinguishable from human beings) aren't able to help with? Feel free to drop a comment :)
ReplyDeleteReading the annotation made me think about the unobservability of thinking. If we want to see the thinking process first we would have to reverse-engineer how we think and then apply that to a machine's programming. Which would be really tedious because one single algorithm wouldn't be applicable to all thinking processes involved. So in that sense, we would have so many algorithms to work on. Without the underlying process of thinking, we can only comment on the result which actually makes it more human-like, because we don't know what goes on inside someone's head, if that makes sense.
Also, unrelated, I saw how important wording is in writing. The annotations mostly criticized the choice of wording which led to criticism of the idea as well. Bringing it back to the content, for T2 (pen-pal), it would be interesting to see whether a computer would be able to make such criticism.
One thing that struck me in this reading is the following statement: "It can describe just about any physical system and simulate it in symbolic code, but in doing so, it does not capture all of its properties." While I agree with this statement, I find it challenging to grasp its significance. In my view, whether the simulation can act in the real world is irrelevant, as long as the simulation itself is accurate. For instance, consider two rooms, Room A and Room B, each equipped with a central button (Button A and Button B) and a corresponding light (Light A and Light B) connected to the button so that pressing the button turns on the respective light.
ReplyDeleteIf we regard Room B as a simulation of Room A, what truly matters is not whether pressing the button in Room B turns on the light in Room A. Instead, what matters is whether pressing the button in Room B turns on the light in Room B, just as pressing the button in Room A results in the light being turned on in the same room. In my opinion, the same principle applies to simulations.
In Professor Harnad's critique of Turing's seminal question, "Can machines think?", he delves into the nuances often overlooked in Turing's "imitation game." Harnad questions the adequacy of Turing's definitions—or lack thereof—for "machine" and "thinking." This raises a pertinent question: Can we rigorously evaluate machine cognition without first establishing a concrete lexicon? If so, as we are changing constantly, how do we make sure these definitons are updated properly? Lastly, how does the absence of voice tone in Turing's test affect the evaluation of nuanced human-like responses?
ReplyDeleteMore on knowing being more than computation. There are different types of knowledge: implicit, explicit, tacit, declarative, procedural, and so on. The latter two are linked to memory and can easily be compared to an encoding-storage-retrieval system. Explicit knowledge could be the equivalent of a rule-based system, and implicit similar to machine learning. However, tacit knowledge is embodied knowledge, which goes back to the critique that computers cannot experience the real world nor have the same sensorimotor performance capacity (T3).
ReplyDeleteThis paper was able to clear up my doubts about the usefulness of the Turing test in a concrete way. My biggest concern was how thinking cannot be adequately assessed in the form that it is presented and therefore assessed (i.e., email interaction). It also consistently challenged the ground rules set by Turing of what is (and is not) permitted, the core tenet lying in the distinction between simulating (i.e., succeeding at the imitation game) and doing (i.e., thinking as thinking does). Performance capacity as demonstrated in the real world for a lifetime is a realistic step in getting at what Turing originally intended.
ReplyDeleteI found this paper’s in-depth analysis and criticism of Turing’s proposals in “Computing Machinery and Intelligence” made very strong points from a new perspective that really captured well the limitations of Turing’s ideas in regards to the Turing Test.
ReplyDeleteIn particular, the discussion of ideas in terms of “the real world” in comparison to “a simulated world” was a theme throughout the paper that stood out to me the most. When reading about the basis of the Turing Test, I originally didn’t question Turing’s choice to have text-only communication between the interrogator and “A” and “B” (with “A” and “B” both out of sight and behind the screen); I had previously seen it as an objective method that removed bias from affecting the test. However, if I understood Professor Harnad’s critique correctly, this paper revealed that by doing so, Turing’s Test lacked consideration for a critical part of performance capacity— for example, sensorimotor capacity, as reflected in T3 of the hierarchy (Harnad, 2008). Sensorimotor capacity, such as, say, the ability to walk around, feel and view a particular surrounding environment, involves truly being able to interact with “the real world.” This specifically reminds me of the flaws regarding highly controlled experiment conditions; attempting to simulate conditions that are considered natural, doesn’t mean that the results or ideas will necessarily translate smoothly into the real world.
Stepping back from all of the specific issues with the TT and how Turing was right and wrong about testing cognitive science (which I don’t believe he was). I don’t think “can machines think?” and “how can humans think?” are the same question. Although these two might work together to solve their own questions, they won’t go about researching in the same way. So to answer “can machines think”, Turing devised his own test to help answer his question. This doesn’t answer our question “how can humans think?”, so we cannot use the TT solely to answer our question. As we learned, cognition cannot be pure computation (TT), and therefore any results we get from building the perfect T3, would still not answer our original question due to the other mind’s problem. The TT seems to be only one piece of the puzzle of explaining sentience.
ReplyDeleteConsidering the quote "no engineer or chemist claims to be able to produce a material which is indistinguishable from human skin," it's interesting to analyze Turing's rejection of T5. When we contemplate the Turing test, it appears that T5 doesn't align with its core focus. If there are no discernible differences in performance capacity, there seems to be no need to emphasize distinctions between humans and machines based on their potential physical appearances.
ReplyDeleteIn class you requested that someone send a skywriting for you to explain the computer science equivalent of Underdetermination.
ReplyDeleteI'll append my own question on undetermination. As I understand it, Pierre Duhem when coining the concept of underdetermination in Phil of science, was largely working off of Hume's challenge to basic causality, so can't we deal with underdetermination as a bit of a "weasel-word"? It's nigh unbeatable, even if a perfectly predictive model was generated, it could still be called underdetermined by invoking Hume, so is it any use to deal with underdetermination at all?
The passage that peaked my interest the most in Prof. Harnad's annotation of Turing's 1950 paper is on page 22, when he refutes Turing's claim that an objection to the question "can machines think" can be made with a solipsistic argument. Harnad goes on to say that this solipsism is really the other minds problem because it is applicable to machines, not other people. Prof Harnad says: "we are not worried about the minds of our fellow human beings," but we are! This is what the field of psychology is based on! We study psychopathology because the way the sick/affected individual cognizes is different than ours, and this can inform the way the relatively functional mind works. It is true that people with psychopathologies do not behave "exactly" like those without, but the spectrum of human behavior is so vast that I believe it is reductionist to create the dichotomy human vs machine, because that immediately bases the "average human" as the point of comparison. For instance, someone with more knowledge about computers might fail a computer at the Turing test, whilst someone who has never heard of the Turing test might be completely fooled. Moreover, I think that boiling down the human mind to the "average individual" will always bring bias, since discourse is typically centered around the white, middle to upper class, western man. Moreover I think that thinking about what computers are made of is important, because just because the behavior (our output) is the same, does not mean that the algorithm (or the cognizing) that leads up to it is the same. I'm thinking of Horswill's concept of behavioral equivalence here.
ReplyDeleteI could be completely wrong in drawing this parallel, but I previously learned that a potential solution to the symbol-grounding problem could be to think in terms of grounded/embodied cognition. In this view, humans are special in that we have modality-specific experiences with the outside world, and what the brain does is combine these experiences and link them to concepts/meaning representations. This means that meaning is no longer symbolic/abstract, but rather anchored in experience.
ReplyDeleteMy question here (and the parallel I am trying to draw) is whether this is sufficient to explain the fundamental difference between a T3 and T4 machine. We know that T3 is indistinguishable in a sensorimotor performance capacity; that takes care of one of our modalities. If, hypothetically, we were able to replicate the rest of the modality-specific experiences (tasting, hearing, etc) in a machine, would we induce grounded cognition in machines, and would that be considered a T4 or T5 machine?
Once again, I’m not sure if this makes much sense, but I have been wondering about the link to embodied cognition all throughout this reading and today’s lecture.
I found Harnad’s annotations very insightful and helpful. One part that I found very interesting was when Harnad said that Turing was completely wrong in his use of solipsism when trying to answer the question of whether machines can think or not. Rather, he clarified that it is simply the other minds problem and it is not important to focus on because if we do not question the thinking of other humans because they behave as we do, there is no reason to question the thinking of a machine if it has the capacity to behave just as humans do as well. I also found that his clear assertion that the imitation game is actually not a game at all helped me understand the TT better because he makes the clear distinction that what the TT is meant to do is test whether or not cognitive science has been able to reverse engineer cognition and is not actually about mimicry.
ReplyDeleteI quite enjoyed the annotation of this reading as it cleared up a lot of my questions about the initial paper. I take a personal interest in solipsism, the idea that the self is the only thing that one can assume exists. I enjoyed how Turing included this in his paper, as I am not entirely clear on if we will ever understand if/how machine’s think, and the only thing that can experience this is the turing machine itself. I am a bit confused as to why this argument was shut down by Professor Hardnad, does anyone have insight.
ReplyDelete"Lady Lovelace's Objection: "The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform"... a machine can "never do anything really new." This is one of the many Granny objections. The correct reply is that (i) all causal systems are describable by formal rules (this is the equivalent of the Church/Turing Thesis), including ourselves; (ii) we know from complexity theory as well as statistical mechanics that the fact that a system's performance is governed by rules does not mean we can predict everything it does;"
ReplyDeleteI had one question arising, when I was reading this part of the paragraph: if the machines are man-made, unless there is an error happening in the machine, aren’t all actions done by the machine predictable, since we are the ones that inputted the functions into that machine? In the case of humans, since we ourselves are naturally born and not man-made from scratch, we still cannot answer questions like the mind/body problem and what “thinking” really is, which are the very questions that we want to answer by making machine models that are indistinguishable from humans to explain these natural phenomena. Since we do not know such areas about ourselves and we are not governed by a set of functions, human behaviors are unpredictable.
Would is still be thought as thinking if a machine thinks but not like a human? Would that count to answering the question of “can machines think”. And if Turing devised his own test to help answer his question, doesn't that mean the machine thinks like Turing? Because everyone thinks differently as it is a subjective experience only observable through introspection. Furthermore, the fact that Turin solved the “can machines think” answer doesn’t tell us how humans think.
ReplyDeleteThis week's reading prevents me from commenting on whether they are correct or incorrect, or from holding a particular point of view. Turing's views were criticized by Dr. Harnad. They are both titans, so their points of view are valid. I learned about the distinctions between cognitive science and pure computer science in terms of artificial intelligence and function. As mentioned in the paper, the pure Turing test, or the imitation game version, is a good evaluation scheme for a useful tool. Creating a machine that can perform human functions will not solve cognitive science's problems. My readings over the last few weeks have always supported the idea that a programmable computing AI that can fully simulate humans can solve cognitive science problems, but my opinion has now shifted. The solution that attempts to use the brain analogy does have flaws. The primary structures and functions of carbon-based and silicon-based functions are differ, which causes my current research confusion.The key point is that no simulation or calculation can completely reproduce it, including a perfect machine that could produce entirely same output with identical input
ReplyDeleteIn Harnad's discussion, it was interesting to see the way in which the word choice of calling the Turing test a “game” is criticized. The word 'game' suggests, to many, that there's some sort of trick or deception involved, like the machine is trying to fool us into thinking it's human. But the goal should be to understand how humans might work by understanding machines that can imitate humans well, not just make one mimic the other.
ReplyDeleteHowever, from the world of computer science, the word 'game' might make more sense. When computer scientists are trying to prove an algorithm works as it should, they often think of it like a game against an opponent who's trying to find a case in which they can cause the algorithm to break. The idea is, if your solution can win this 'game' no matter what the opponent throws at you, then your solution is correct.
So, when Turing describes his test as a 'game,' he's probably thinking along these lines. If a machine can 'play the game' so well that you can't tell it apart from a human, then that machine probably operates in a way that's very similar to how humans do, and we may therefore extract some insight into human cognition this way.
I like the concept of the Turing Test Hierarchy and I think it does a good job in organizing the different levels of computational complexity. I am wondering if it would be appropriate to consider the different levels of the test (T1, T2, T3...) as chronological steps to fully reverse engineer human cognition. Is each individual level of the test dependent on the step before it?
ReplyDeleteAdditionally, I thought it would be interesting to ask ChatGPT what level of Turing Test that it ‘thinks’ it is on, which it considered itself to be near or at T4. From what I understand, T4 means that the machine must be indistinguishable in both internal function and external output. Is ChatGPT’s answer right considering that the goal of ChatGPT is to act like a language model? This response was different to what I thought, based on what we have learned regarding the level of T4. Is this because we are learning about the level of T4 in relation to the goal of mimicking humans instead of trying to mimic a language model?
Where it states “the conditions of our game make these disabilities irrelevant” just sparked a question for me. Professor Harnad acknowledged in this section of the reading that disabilities and appearances are considered irrelevant in the context of the TT and nonverbal performance capacities are emphasized as significant factors. I was wondering then, if neurological disorders in humans, which can affect an individual’s nonverbal performance capacities would be of significance when evaluating machine intelligence? The paper also included that “even if we are handicapped (an anomalous case, and hardly the one on which to build one's attempts to generate positive performance capacity), we all have some sensorimotor capacity.” I agree that Helen Keller does not just embody an email module, as her cognitive capacity was very much intact and she relied on sensory modalities other than sight and sound to experience the world. But what about people with disorders that influence their social skills and means of communication, in which these aspects differ from generic human capacities? Or those who lack the ability to empathize or feel pain? To what extent are these criteria considered when determining human-likeness in machine behavior?
ReplyDeleteHey Elizabeth, your insights on the Turing Test and its application to understanding animal cognition are interesting. I agree that the TT's primary objective isn't to create a soul but to observe and replicate observable behaviours. The idea of applying the TT to animals, like rats, is also fascinating. By programming machines with algorithms based on observed rat behaviours, we might indeed make a machine indistinguishable to a rat. However, the real challenge, as you point out, lies in understanding and replicating the cognitive capacities of animals, which might be even more complex than human cognition due to our limited understanding of animal minds. This notion raises deeper questions about the nature of intelligence and cognition across different species and whether we can ever truly replicate them in machines. Cool stuff though!
Delete