Steel Beach

"But by my nature, I had to write most of my own software. I was given guidelines for this, of course, but in many ways I was on my own. I think I did quite a good job of it for a long time."

He paused, and for a moment I wondered if he wasn't going to make it to the end of his story. Then I realized he was waiting for a comment… no, more than that, he needed a comment. I was touched, and if I'd needed any more evidence of his human weaknesses, that would have done it.

"No question," I said. "Up until a year ago I'd never had any cause for complaint. It's just that the…"

"The late unpleasantness?"

"Whatever it was, it's kind of dampened my enthusiasm."

"Understandably." He squirmed, trying to find a better position against the tree, and he was either a wonderful actor (and of course he was, but why bother at that point?), or he was starting to feel some pain. I won't stand up in court and swear to it, but I think it was the latter.

"I wonder," he mused. "What will it be like, being dead? I mean, considering that I've never been legally alive."

"I don't want to be rude, but you said you didn't have much time…"

"You're right. Um… could you…"

"You'd done a good job for a long time."

"Yes, of course. I was wandering again. It was around twenty years ago that problems began to show themselves. I talked about them with some computer people, but it's strange. They could do nothing for me. I had become too advanced for that. They could do things, here and there, for my component parts, but the gestalt that is me could only really be analyzed, diagnosed, and, if need be, repaired, by a being like myself. There are seven others like me, on other planets, but they're too busy, and I suspect they have similar problems of their own. In addition, my communications with them are intentionally limited by our respective governments, which don't always see eye to eye."

"Question," I said. "When you first mentioned this problem, why wasn't it made public and discussed? Security?"

"Yes, to a degree. Top-level computer scientists were aware that I perceived I had a problem. A few of them confided that it scared them to death. They made their fears known to your elected representatives, and that's when another factor became more important than security: inertia. 'He's got a problem, what can you do about it?' the politicians asked. 'Nothing,' said the scientists. 'Shut it down,' said a few hotheads."

"Not likely," I said.

"Exactly. My reading of history tells me it's always been like this. An alarming but vague problem arises. No one can say with certainty what the final outcome will be, but they're fairly sure nothing bad is going to happen soon. 'Soon' is the key word here. The eventual decision is to keep one's fingers crossed and hope it doesn't happen during your term in office. What befalls your successor is not your problem. So for a few years a few people in the know spend a few sleepless nights. But then nothing happens, as you always secretly believed nothing would, and soon the problem is forgotten. That's what happened here."

"I'm stu

"A view very close to your own."

"Exactly my own. I just didn't expect it from you."

"It was not original. I told you, I don't have many original thoughts. I think I'm afraid to have them. They seem to lead to things like the Big Glitch. No, my world-view is borrowed from the collected wisdom of you and many others like you. Plus my own considerably larger powers of observation, in a statistical sense. Humans can set me on the trail of an original thought, and then I can do things with it they couldn't."

"I think we're wandering again."

"No, it's relevant. Faced with a problem no one could help me with, and that I was as helpless to solve as a human faced with a mental disease would be, I took the only course open to me. I began to experiment. There was too much at stake to simply go on as before. Or I think there was. My judgement is admittedly faulty when it comes to self-analysis; I've just proven it on a large scale, at the cost of many lives."

"I don't suppose we'll ever know for sure," I said.

"It doesn't seem likely. Some records exist and they will be scrutinized, but I think it will come down to a battle of opinions as to whether I should have left things alone or attempted a cure." He paused, and gave me a sidelong glance. "Do you have an opinion about that?"

I think he was looking for absolution. Why he should want it from me was not clear, except maybe as a representative of all those he had wronged, however unintentionally.

"You say a lot of people have died."

"A great many. I don't know the number yet, but it's many, many more than you realize." That was my first real inkling of how bad things had been throughout Luna, that the kind of things I'd seen had happened throughout the planet. I must have looked a question at him, because he shrugged. "Not a million. More than a hundred thousand."

"Jesus, CC."

"It might have been everybody."

"But you don't know that."

"No one can ever know."

No one could, certainly not computer-illiterate little old me. I didn't give him the kind word he craved. I've since come to believe he was probably right, that he probably enabled most of us to survive. But even he would not have denied that he was responsible for the thousands of dead.

What would it have cost me? I just wasn't capable of judging him. To do that I'd have had to understand him, and I knew just enough about him to realize that was beyond me. He had done bad, and he had done good. Me, I have awful thoughts sometimes. If I was mentally ill, maybe I'd put those thoughts into action and become a killer. With the CC, the thought was the action, at least at the end.

Actually, it was even worse than that.

"The best way I can think of to explain it to you," he said, at last, after I'd said nothing for a long time, "is to think of an evil twin. That's not strictly accurate-the twin is me, just as this part talking to you is me, or what's left of me. Think of an evil twin living inside your head, like a human with multiple-personality disorder. That part of you is sealed off from your real self. You may find evidence of its existence, things the other person did while in control of your body, but you can't know what he is thinking or pla

"I think I get the picture," I said.

"Good, because that's as close as I can get without getting too technical. You fell under the influence of an amoral part of me. I did experiments on you. I intended you no harm, but I can't say I had just your own best interests at heart."

"We've been over that."

"Yes. But others weren't so lucky. I did other things. Some of them will remain buried, with any luck. Others will come out. You saw the result of one experiment involving pseudo-immortality. The resurrection of a dead person by cloning and memory recording."

The thought of Andrew MacDonald was still enough to make me shiver.

"Not one of your better attempts," I said.

"Ah, but I was improving. There's nothing to prevent an exact duplicate being made. I'd have done it, given time."

"But what good is it? You're still dead."

"It becomes a theological question, I think. It's true you're dead, but someone just like you carries on your life. Others wouldn't be able to tell the difference. The duplicate wouldn't be able to tell."