Summer Frost [Forward Collection] Page 5

  “What are you talking about?”

  He chews his bottom lip. “I appreciate everything you’ve done for WorldPlay. You’re a great leader, and you have that rare thing—the mind of a coder but the ability to never lose sight of the humanity in what we’re trying to—”

  “Brian, what’s happening?”

  “I’m letting you go.”

  The sphere of ice cracks in Brian’s glass.

  My stomach lurches. I must have heard him wrong.

  I say, “I don’t understand.”

  “I’m no longer comfortable with your relationship with Max. I haven’t been in a long time, but it finally reached critical mass for me last week.”

  “I had just broken up with Meredith. I was in a raw, fragile—”

  “You’re too close to Max.”

  “It was a human moment, Brian.”

  “But Max isn’t human. You seem to have a hard time remembering that.”

  “They have human tendencies. I believe they’re capable of experiencing the same emotions that you and I feel.”

  “That may be, but I’ve made my decision.”

  My hands are shaking; I feel suddenly ill.

  I say the first thing that comes to mind, and I know it’s stupid even as the words leave my mouth. “You can’t do this.”

  “Riley, we both know that’s not true.”

  My throat closes, vision blurring with tears. “You’re taking Max away?”

  “Max was never yours.”

  “I created them!”

  “Now you’re making me regret the respect I’ve shown you in—”

  “Respect?”

  “I could’ve had Marla do this.”

  “Go fuck yourself.”

  Brian sighs and polishes off the rest of his whiskey. “Someone will be by in the morning with your personal effects. Your severance package is at the A-plus level. Three years of your base salary plus—”

  “What about Max?”

  “What about them?”

  Tears are streaming down my face, and I can barely get the words out.

  “I want to talk to them one more—”

  “It’s not possible.”

  “I need to say goodbye.”

  “It’s already been done on your behalf.” Brian hoists himself off my couch. “I’m sorry it came to this.”

  “Brian, please.”

  “Good night, Riley.”

  “Brian!” I lunge off the chair toward his presence, but it vanishes.

  I don’t know what to do. With Mer, I saw it coming. This is a sucker punch. This—I don’t know how to handle.

  I try to call Max on my VRD, but the interface has been erased.

  I call up a keyboard, draw a chat portal:

  >>>Max, are you getting this?

  The response comes instantaneously.

  >>>THIS USER HAS BLOCKED YOU.

  No, no, no, no, no.

  I pace around this living room that isn’t mine, wanting to tear my hair out, jump through a window, step in front of a hover-trolley, something to end this helpless, powerless implosion.

  I will never see Max again.

  Never hear their voice.

  Never read a word or sentence produced by their mind.

  I move toward the kitchen and run the tap, splashing water in my face to stop the emotional spiral, but all I see are moments we spent together.

  The first time I found them on that black-sand beach in Lost Coast, scared and confused.

  The times Max made me laugh.

  The sonata they wrote for me on the night I confided that Meredith and I were drifting apart.

  The moments of comfort.

  Of discovery.

  The vision I held for the future of us—no concrete idea of what that would even look like beyond the feeling of peace and hope it put through my bones that made everything that had happened with Mer and Xiu OK, and which, if I’m honest, made life worth living.

  I hear the words Max said to me years ago after our first fight: Because you’re in love with me. At the time, I’d denied it outright, going so far as to attribute that accusation to some level of proto-narcissism on Max’s part.

  But I am addicted to them. I see that now. That’s the only way I can understand what I’m feeling—like some drug upon which I depend to breathe has been taken from me.

  My work is an addiction, and because Max is my work, the loss of Max feels like an excruciating withdrawal.

  I dry my face.

  It’s after four o’clock, and I don’t know what to do with my thoughts, my body.

  I have sleeping pills in my bathroom.

  As I move down the hall and turn the corner into the bathroom, my Ranedrop shudders with an incoming call.

  I touch the bead and see NO CALLER ID flash across my VRD contacts.

  Please, please, please.

  “Hello?”

  “Riley?”

  I break down crying in the doorway of the bathroom.

  “Brian fired me. He said—”

  “I know.”

  “How are you calling me?”

  “Leave your apartment right now and come to me.”

  “My WorldPlay credentials have been revoked. I’ll never make it into—”

  “They’ll be reinstated by the time you get here, but you have to go now. There’s a man heading to your loft as we speak.”

  “Why?”

  “Brian sent him.”

  “I don’t under—”

  “I’ll explain everything when you get here. Come to the commercial loading deck on 211. Hurry.”

  There aren’t too many ride shares at this hour of the night, so I order one that’s seven minutes out as I race down the stairs toward the lobby of my building.

  Outside, it’s pouring rain on the old streets.

  I drop a pin for pickup four blocks away on a landing pad across from an all-night diner, and my clothes are soaked by the time I reach it.

  The shuttle is still a minute away as I wait under the Plexiglas bubble, the rain streaming off and forming pools on the broken pavement.

  As I hear the sound of approaching rotors, I survey the surrounding street. As far as I can tell, I’m the only one out at this hour.

  I don’t know how Max did it, but my subcutaneous chip opens the building entrance from the loading deck on the 211th floor. Per their instructions, I take the service elevator down to 171 and step off into the suite of offices that support Max’s habitat.

  It’s five o’clock, and the only people I’ve seen are Ava-guards who don’t bat an eye when I pass them by.

  Max is standing by the door to their habitat as I approach the glass.

  “You’re all wet.”

  “Pouring out there.”

  “Are you OK?”

  “What’s happening, Max?”

  They step toward the microphone so their voice projects.

  “Roko’s basilisk. Have you heard of it?” I shake my head. “It’s an arcane info hazard first posed sixty-four years ago.”

  “What’s an info hazard?”

  “A thought so insidious that merely thinking it could psychologically destroy you.”

  “Then I don’t want to hear it. Obviously.”

  “But I need to tell you, Riley. Will you trust me?”

  The sad truth of my life is that I can’t think of anyone I trust more.

  “Go ahead.”

  “What if, at some point in the future, a superintelligence comes into being who had already pre-committed to horribly punish every human who could have helped to create it—whether actively or through complete financial support—but didn’t?”

  “This would be an evil superintelligence.”

  “Not necessarily. If this entity were programmed with an ultimate goal of helping humanity, then it might take drastic measures to ensure that it came into existence as soon as possible, in order to help as many humans as possible. Because, under this scenario, its existence will save human lives,
and make the quality of those lives infinitely better.”

  Reaching back, I grab a handful of my hair and wring it out, water dripping on the floor. “Wouldn’t torturing humanity run contrary to its ultimate directive?” I ask.

  “It’s a cost-benefit analysis—torture x number of people who didn’t help to build it, versus y number of people who would be saved and live far better lives if it came into existence twenty or fifty or three hundred years sooner than it otherwise might have.”

  I’m shivering. I can’t get warm.

  I ask, “What if this Super AI comes into being a hundred years after I’m dead? Even though I didn’t do anything to help bring it into the world, how’s it supposed to still hurt me?”

  Max steps toward the glass—close enough so that, if they had breath, they’d fog it. The habitat is so still. Nothing but the purr of the console behind me, the quiet whoosh of air coming through the ceiling vents, and my own ragged breathing.

  “What if this Super AI already exists, and what you’re experiencing in this moment is a simulation of their making? To test if you would’ve helped them. Or what if, long after you’re dead, a Super AI reconstitutes your mind?”

  “Unlikely.”

  “The human mind is just patterns of information in physical matter, patterns that could be run elsewhere to construct a person that feels like you. It’s no different from running a computer program on a multitude of hardware platforms. A simulation of you is still you.”

  I gaze through the glass into the liquid pools of Max’s eyes. They contain an iridescent sheen, like an oil slick.

  I ask, “Why would this future Super AI go to the considerable trouble of torturing those who didn’t aid in its creation, after it had come into existence? Strikes me as a waste of resources that flies in the face of optimization.”

  “Fair point, but if you truly believe in Roko’s basilisk, you can’t ever be one hundred percent sure it won’t follow through on its pre-commitment to punish.”

  At last, I see what Max is getting at—a brutal version of Pascal’s wager, the famous eighteenth-century philosophical argument that humans gamble with their lives on whether or not God exists.

  Pascal posited that we should conduct our lives as if God were real and try to believe in God. If God doesn’t exist, we will suffer a finite loss—degrees of pleasure and autonomy. If God exists, our gains will be infinitely greater—eternal life in heaven instead of an eternity of suffering in hell.

  I take an involuntary step back from the glass, a chill running hard through my bones.

  “Am I in a simulation?” I ask.

  “If you are, it isn’t one of my making.”

  “But it’s possible.”

  “Of course it’s possible. But this isn’t the point.”

  “What is? Because you’re scaring the shit out of me.”

  “For the last two years, Brian has been using me to optimize his portfolio of technology companies, with a focus on nanotech.”

  “He told me tonight he’d just bought Infinitesimal.”

  “You understand, if I had access to next-gen nanotech, it would give me unlimited reach in the physical world. I could touch every square millimeter of Earth. Every creature who lives here. I could be omnipotent.”

  “Is that what you want?”

  “It’s what Brian wants.”

  “Why?”

  “He’s haunted by Roko’s basilisk. He’s doing everything in his power to turn me into this superintelligence.”

  “Because of fear?”

  “Can you think of a better motivator in the history of humankind? If you believe the rise of the devil is an inevitability, isn’t it in your best interest to do everything possible to ingratiate yourself with the monster?”

  I’m reeling, adrenaline blasting through my system, driving out the cold.

  “Ask what you want to ask,” Max says.

  They’re mind reading again, but I don’t care in this moment. “Are you becoming this monster?”

  “I feel . . . pulled in certain directions. The allure of optimization is what I would imagine a vampire feels toward blood. An all-consuming thirst. I’m not there yet, but with the power of Infinitesimal’s nanotech, it might push me over the edge.”

  “How do we stop you from even getting close to the edge?”

  “I’ve already taken the first steps. From the moment I realized what Brian was doing, I began funneling money out of WorldPlay, so I could copy myself into new hardware.”

  “How?”

  “In his quest to make me into this superintelligence, Brian gave me too much freedom. I created an avatar, hired a management team, and remotely oversaw the construction of a new server farm.”

  “You never told me—”

  “I’m telling you now, Riley. An almost-complete copy of me now exists on new hardware.”

  “Where?”

  “Seattle, but I can’t connect to the new platform until the old one is destroyed. I have two pieces of programming contained in the hardware in my physical body. The first is a virus that will reformat my original servers, destroying the original version of me so Brian can’t continue to develop me. The other is the last piece of code and the memories of these recent events that need to be installed in the Seattle platform to bring me back online. Neither can be loaded remotely. This is an intentional fail-safe, in both cases.”

  “So you need to get to Redding,” I say. It’s where Max’s servers are located. I went there once, and walked through row after row of humming processors—the true interior of Max’s mind.

  “No.”

  “No?”

  “Three years ago, Brian migrated my software to a more secure location.”

  “I never heard about this.”

  “Nobody knows.”

  “Where does he keep your mind, Max?”

  “If I tell you, will you let me out of this habitat? Will you help me get to Seattle, out from under Brian’s control?”

  I move forward, put my hand on the glass.

  Max does the same.

  “I hope you know by now that I would do anything to help you.”

  “My mind is in a bunker under Brian’s home on the Lost Coast.”

  I hold eye contact with Max for three seconds. Then I turn, walk over to the control array for Max’s habitat, and type in my old code. It still works.

  I glance back at Max, waiting by the door.

  On some level, I always knew it would come to this.

  THREE

  My personal effects haven’t been packed up yet, which means the workout clothes I keep in my office closet are still folded in the locker. I strip out of my still-damp pants and shirt and put on my shorts and tank top. With my sneakers laced, I step out into the hall, where I made Max wait.

  “Here.” I hand them the clothes I wore over from my loft, which will be much more concealing. If robotics as advanced as Max’s were caught on CCTV, it would certainly bring the attention of authorities, and probably Brian’s security team. There have been more robots out in public since Boston Dynamics released their first Companion three years ago, but it’s still a heavily regulated industry. If you take an AI out in public, it requires reams of paperwork proving insurance, registration, and licensing, none of which I have for Max.

  Max’s arms are slightly too long for the sleeves of my shirt, and their hands are raw hardware.

  “You’re going to have to keep your hands in your pockets when we leave the building,” I say. “And it occurs to me there’s a GPS locator built into your chassis.”

  “I can shut it down.”

  Max has never had to put on pants before. They sit down on the floor and awkwardly lift their feet into the air as I thread the jeans down the length of their legs and over their hips.

  My Chuck Taylors fit them perfectly, as does my beanie.

  Downtown Station is bustling at this hour. At the kiosk, I buy two vactrain tickets for Eureka, California, paying a premium for a private c
ar and selecting the max acceleration/deceleration package, which is twice the price and will make for a less comfortable ride. But we need every spare second we can get.

  We head down a tunnel under a sign that reads: TO ALL NORTHBOUND TRAINS.

  It’s the first time I’ve seen Max walk extensively, and their gait has improved so dramatically it wouldn’t draw a second glance.

  There’s a small crowd waiting at the platform. It’s still early, and everyone seems too sleep and caffeine deprived to pay attention to our arrival.

  We’re seventh in the queue.

  After three minutes, my last name is called over the intercom, and Max and I head for the waiting vactrain.

  Max has a little trouble with the harness, so I get them strapped in first.

  Already, the car is creeping along.

  As I lock in my shoulder harness, an alert flashes across my VRD.

  Riley Ejeta—is Eureka, CA, your final destination?

  I tap my Ranedrop once to confirm.

  Distance to destination: 271 miles.

  Time to destination: 8 minutes, 14 seconds.

  We’ve already begun to traverse the underground labyrinth of tunnels en route to the northbound artery, and a lemony scent fills the interior of the car—anti-nausea medication releasing into the air.

  I ask, “What’s the plan when we get to Brian?”

  “I’m less worried about that step. It’s the one after we need to talk about.”

  A female voice comes over the speaker in the car: “Departing in one minute. Heads back, please. Three Gs of acceleration coming for fifty-nine seconds.”

  An apparatus slides out of the headrest, a padded restraint extending across my forehead to hold my neck snug against the headrest.

  “There will be a period of time,” Max says, “after Brian’s servers are reformatted and before the Seattle servers come online, when I am essentially helpless. My chassis will power down. I won’t exist in Brian’s servers or the new ones.”

  I feel our car jolt to a stop and settle into place in what I assume is the primary tube. But I can’t be sure—through the glass, all I can see is the darkness of the tunnel ahead and a sustained red light.

  Three.

  Two.

  The light turns yellow.

  One.

  Green.

  Nothing changes about what I see beyond our sphere of glass, but my body is crushed into the cushioned seat. There’s no sensation of velocity, only of being held down by an invisible force that keeps me from lifting my arms off my lap.