Q1). Can we even define what intelligence means? Isn’t that necessary for working on AI?
A1). Even if we can’t define intelligence precisely yet, in practical terms, we know what it does: intelligence allows us to build models of the world, manipulate these models to see which actions are the best for achieving our goals, and then execute those actions. There are plenty of things that could allow one to model the world better, such as faster processing and more memory.
Q2). What if intelligence isn’t linear or one-dimensional? Then, talking about greater- or below-human intelligences doesn’t make sense. For instance, talking about human-equivalent AI is pointless. A computer mind would of necessity be much smarter than humans in some fields: for instance, in the field of doing multiplication or addition. In this scenario, creating a truly “human-equivalent” AI would require needless work, and involve essentially crippling the AI.
A2). It is true that intelligence is hard to measure with a single, linear variable. It is also true that it will probably take a long time before we can build an AI which is superior to humans in literally every respect, because there are so many different ways of comparing humans and AIs. Ultimately, there will be no such thing as exactly human-level AI, just as there is no such thing as exactly bird-level flight: humans will have their own comparative advantages, while AIs will have their own comparative advantages. A simple calculator is already superintelligent, if speed of multiplication is the only thing being measured.
However, there are such things as rough human-equivalence and rough below-human equivalence. No human adult has exactly the same capabilities, yet we can still speak of adult-level intelligence. A calculator might be superintelligent in a single field, but obviously no manager would hire a calculator to be trained as an accountant, nor would he hire a monkey. A “human-level intelligence” simply means a mind that is roughly capable of learning and carrying out the things that humans are capable of learning and doing- it does not mean that an AI would have to have exactly the same capabilities as a human mind. Likewise, a “superhuman intelligence” is a mind that can do all the things humans can do, at least at a roughly equivalent level, in addition to being considerably better at many of them.
Q3). Don’t we already have super-intelligent entities in the form of corporations, combining humans with Google, and so on?
A3). Human civilizations and organizations as a whole have not been shown to be smarter than the individual humans within them. For instance, in 1932, Germany voted the Nazi Party into power, even though many smart Germans realized the devastation that they would cause if they were running the government. In fact, there is substantial evidence that large groups of people are dumber than individual people. A large crowd of people escaping a burning building might well jam the door and prevent anyone from getting out, even if everyone could have easily escaped had everyone been rational and filed through one at a time.