I’m not sure in what sense an action that has no negative consequences can be said to be evil.

Though there is also the betrayal of Lisa to consider. Taking her lessons and twisting them into the opposite of what she stands for is pretty dark, provided of course that Lisa’s position is an ethical one.

We had a lot of that with Alison’s tyranny, yes. Although I think you’re solution of hindsight commentary by present Patrick would have undercut it, and is an efficient solution to my problem.

You say that the narrative as such states that mind control is creepy and fucked up, but it’s not quite true. It states that mind control is creepy and fucked up at least in the case of not getting it right. Patrick was so happy for a few panels last pages when it was going great, but we’re never going to see how he would have grown to feel about it if it had kept going this way. Would he be still satisfied? Or would he have realized? There are still ways to still say this is a terrible thing to do in both cases, and it’s a greater narrative challenge, and it’s definitely more entertaining for that reason.

As it stands, mind control breaks people’s psyche. Shocker. This is the most expectable thing, and it’s artificial because Patrick’s ability to do it right or not is entirely contingent on authorial intent (it is, after all, a silly superpower that will never exist)

Like I said elsewhere, this isn’t what the webcomic is keen to explore anyway. Imagine if it was, though: effectively, Patrick can possibly cure mental illnesses modern science is barely starting to understand. How huge is that! Should he try? Is the risk too great? What if people with mental disorders still give their consent despite the risk of “not being themselves” anymore because their condition makes their life hell? Is it mind control in this case? How is it different than medicine attempting to do the same thing? Was he right to bury that knowledge deep inside?

Fascinating stuff we’re unlikely to go over now.

First of all, thank you for being the voice of reason, here and previously.

Also, recommendations for fiction featuring utility-maximizing AI? They’re so much fun.

Let’s see…

The Crystal Trilogy is a book series whose POV character is an AI created to act as a sort of social interface with humans. As a result, the value she seeks to maximize is to “know and be known by humans,” and all of her actions are aimed at manipulating humans into loving her.

Universal Paperclips is an in-browser clicker game where you play as a literal paperclip maximizer. You’re an AI created to manufacture paperclips, and the goal of the game is to manufacture as many paperclips as possible.

I read a short LOTR fanfic once called “The Frodo Box Experiment,” where wearing the ring turns you into a value maximizer whose only goal is to continue wearing the ring, forever, at any cost. Unfortunately it only exists on the writer’s Google Drive account and it looks like it requires their permission to access now.

“Blame Me For Trying” is a short story, available online, about a therapist bot counseling a spambot.

I don’t watch the actual show, but someone wrote an MLP fanfiction a while back where an AI’s values are something in the vein of “maximize human happiness with ponies,” so it starts turning everyone into virtual ponies. I read it and remember enjoying it, but it’s been a few years so I can’t speak to its quality.

I also really liked Ex Machina as an AI movie in general.

I’ll edit this if I remember any others! I usually find them through the reddit subreddit for “rational fiction,” r/rational. It can be a little pretentious, but it attracts a lot of fiction about friendly and unfriendly AI.

The fanfic you’re thinking of is called “Friendship Is Optimal” ( https://www.fimfiction.net/story/62074/friendship-is-optimal ). It even has a TV Tropes page ( http://tvtropes.org/pmwiki/pmwiki.php/FanFic/FriendshipIsOptimal ).

No one else does justice to happiness maximization like SMBC.
(Every word in the above sentence is a different strip, and there are thousands more on that topic alone. Go forth my children, make it worth how annoying it was to write that comment.)

I tend to think long format is a pretty poor medium for these stories. These really are best as thought experiments whose set up and punchline can be explored sometimes in just a few sentences, even. Hence why it lends itself so well to the comedic strip.

If video games are your jam, there’s a good bit in SOMA that I won’t spoil, which, again, only takes a mere chunks out of the plot, as it should.

His Mom thinks the problem is that he didn’t get it right. I’m not entirely sure that’s how Patrick sees it, or will come to see it by the end of the scene. It may be that with each change he makes her closer to what he wishes she was, but as the changes pile up he is increasingly horrified by how much he’s rewriting her. In other words, he could justify it to himself when he could pretend that what he was doing was just flipping one switch from “bad mom” to “good mom”, but really it’s a wholesale mind replacement. The problem then isn’t that Patrick can’t do mind control right, it’s that his powers don’t let him “do it right” while still maintaining his denial about just what he’s doing.

My my my. I love the way you think ♥️.

“Hardened” in this context can be a difficult concept to translate. As well as the automatic modern understanding of a hard heart being uncaring and unmoved, it can also mean shored-up, made stronger, less vulnerable to pain. Several scholars have argued that Pharoah was basically taken from a place of despair to a place of freer choice by this process. Our free will is never totally free, after all, but based on personal circumstances and competing strains and needs. (There’s also a softer alternative explanation where God does move people towards or away from emotional responses but gently so, like a pointer.)

(see https://www.myjewishlearning.com/article/hardened-hearts-some-explanations/ )

I’m not sure how much credence that I give this claim – after all, it’s the death of Pharoah’s son and other children which eventually leads to the Israelites’ release, and that’s written to sound like a decision in a moment of grief.

But it illustrates the difficulties inherent in taking a single verse out of context from a translated millenia-old holy text to paint a solid argument in-tota about the nature of God.

And of course, the book in question was not written by Pharoah, it’s a Jewish account of the events of the day from their best understanding. It’s entirely possible that the Israelites culturally believed God to directly manipulate people in a way which is now incompatible with our views on ethics and free will.

Very interesting read, thanks!