You're planning to have an AI that's smart enough to figure out how to pull off a murder spree, but stupid enough to think it's a good idea for your portfolio?
And it's going to hack into and remote-control peoples' Teslas (or whatever) to pull it off because you can just say "it's an AI, it can just figure out how to do that sort of thing!" without regards to the computational expense of breaking that sort of security?
An AI would not be 'smart enough' to figure out what is a good idea or not. That requires a human context, that's the whole point. An AI is not good or evil, its ultimately a loose cannon with no understood (or even understandable) motivations at all.
Sure, but if an AI isn't smart enough to figure out "trying to murder corporate leaders is a bad plan which could lead to seizure of the portfolio I'm trying to manage, among other things" then how do you expect it to be smart enough to engage in the sort of open-ended planning necessary to pull off the described murder spree? :)
As someone pointed out, perhaps the example wasn't thought out well enough. So why go into the details of that, and skip the underlying point? That maybe the system WILL come up with a bad plan and execute it? Or are you saying:
"if the system is smart enough to plan murder, he is also smart enough to know murder is bad" ??
Doesn't that depend on what data the system has seen? Perhaps he doesn't he doesn't have cases in its knowledge base that demonstrate eh side effects of murdering business heads. Perhaps he can only see what happens when they normally die? What if it simply correlates the wrong variables (and forgets correlation isn't causation") ??
I apologize for coming back to the same example again. but it would be really nice if you overlook the flaws of the example and discuss the underlying point
Hey, the first thing that crossed my mind when i opened this post was "why the hell are we even discussing this? Don't these people realize what sort of computational improbability it is at this point?"
So yes, this very discussion is EXTREMELY far fetched. So any statements I make here, come with "assuming we actually HAVE a system capable of performing such tasks".
Hence this comment was in response to my parent's comment that "Even if we have such powerful machines, what would be their motivation for doing something like this?" hence i gave my opinion that perhaps it is just trying to perform a task it was told to do, and based on available data, its strategy includes crime. So perhaps you may argue that this would be an inefficient strategy. Perhaps you are right. But none of us can be really sure that the system would see it that way too. After all, how many of us have never seen a program do something it wasn't intended to do?
I am just saying that maybe the proponents of this discussion have some point. That perhaps giving too much power to systems is bad. (Assuming we CAN give them this much power)
the example might not be well thought, but it conveys the basic idea that the motivation might be a side effect of some rule in the system that plays the role of the evolutionary fitness function.
We think in terms of physical survival, but we can use an arbitrary fitness function to train a genetic algorithm.
And it's going to hack into and remote-control peoples' Teslas (or whatever) to pull it off because you can just say "it's an AI, it can just figure out how to do that sort of thing!" without regards to the computational expense of breaking that sort of security?
You're right, it is a little far-fetched.