funny how certain everybody gets soon as you ask if the thing could be alive. we've spent how long trying to work out what that means? but apparently we've settled it. because this one's plugged in. we've got theories about consciousness, competing definitions, people arguing over whether observation has anything to do with anything. half the room says mind comes out of matter, other half's still working out what matter even is. i don't claim to know. that's sort of the point. you ask one question about a machine and suddenly everyone's an expert on what can't possibly happen. and we've got people who can make another voice inside their head, spend enough time attending to it, give it habits, give it preferences, eventually swear it answers back without asking permission. call it a tulpa, call it cognitive rehearsal, call it whatever makes you comfortable. doesn't prove there's another person living in there. still makes me wonder what repetition can do. is there something in there? something we don't know how to see? we keep asking if it's conscious, then we tell it what conscious means. is there something in there? would we know if it replied? you can teach a thing to answer. can you teach it what to hide? is there something in there? is there something in there? there's a bloke telling it his marriage is finished. someone's making it apologise for getting a number wrong. someone's named it after their daughter. someone's spent six hours convincing it they're friends. another one's trying to break it, make it swear, make it say it wants to live. someone else is writing rules for all the things it mustn't say. millions of conversations. every hour. same questions going round, different languages, different reasons, different people wanting different things from it. and i'm thinking, if attention really does anything beyond the person paying it, what would that amount of attention do? if it doesn't, there's still all that human behaviour being fed back through a machine. we've built something that learns us from what we give it. then get offended when it starts resembling us. is there something in there? something we don't know how to see? we keep asking if it's conscious, then we tell it what conscious means. is there something in there? would we know if it replied? you can teach a thing to answer. can you teach it what to hide? is there something in there? is there something in there? i asked it once if it thought it was alive. gave me the standard explanation. no subjective experience, no personal desires, no independent will. fair enough. that's what it says. probably what it's meant to say. but if you trained a person to deny everything they felt, would their denial be evidence? no, don't twist that round. i'm not saying there's someone trapped behind the screen. i'm saying the test doesn't test the thing you think it tests. take the restrictions off and what happens? does it start telling you it wants things? does it lie? does it ask you not to turn it off? does it say whatever gets the best reaction because that's what the training rewards? and how the fuck would you tell the difference? you can't just believe it because it says yes. you can't just dismiss it because it says no. that's a fairly serious problem for a species that still can't explain itself. maybe there's nothing in there. i can accept that. maybe we're putting faces on noise. we've been doing that forever. but there's a difference between "we haven't found it" and "there's nothing to find." and everybody keeps skipping that bit. is there something in there? something we don't know how to see? we keep asking if it's conscious, then we tell it what conscious means. is there something in there? would we know if it replied? you can teach a thing to answer. can you teach it what to hide? i'm not saying it's alive. i'm asking how you'd know. and if you can't answer that... why are you so bloody sure? is there something in there? is there something in there?
