The world is full of problems: Pain, poverty, illness, war, pollution, to pick a few among thousands. Many of us feel like we need to Do Something about these problems. There’s just one problem (sorry):
Agree on the main point: Collaboration is what made humans special, and is the ultimate meta problem of humanity. The difficulty is in finding a balance between the cohesion needed for problem solving, and the individual liberty needed for innovation. How do we balance that? Can we use AI to help? Who knows, I’m researching on this topic…
Minor point 1: The Victorians and the Qings excluded large parts of their population from effective collaboration, which meant that they are less effective at solving problems, and also curtailed individual freedom. Human history may be seen as finding ways to have large scale collaboration, and failing, in some cases.
Minor point 2: Effective altruism ranks problems. This is fundamentally ineffective. In whose framework does one problem rank higher? How is any ranking justified? EA ignores the fact that human progress happens in collectives, which benefits when each of us are exercising our most individual contributions, not when we are submitting ourselves to “a higher-ranking good”.
Great analysis and writing btw! Loved to read careful thoughts and similar insights!
"human progress happens in collectives, which benefits when each of us are exercising our most individual contributions, not when we are submitting ourselves to “a higher-ranking good”."
I made the same basic point, but less eloquently, in response to the overtly Orwellian 'Cooperation Machine' idea they floated:
"A great many things are needed, but a figurative machine for hacking human nature to manipulate it toward an end is precisely the sort of thinking that's gotten us to where we are right now."
(1) not sure what you mean by natural, if you mean "require no active effort to maintain" then no, certainly not for humans beyond extended family / Dunbar scales.
(2) for civilization-scale cooperation this feels like asking a Victorian what the main bottleneck to getting to the moon is; like, there's a whole tech tree and massive industrial base missing. can a bottleneck be "sustained effort over decades"?
Interesting question, but I would like to challenge your main example. You quote Ada Palmer who speaks of Bacon: "Let us found a new method — the Scientific Method — and with it dedicate ourselves to the advancement of knowledge of the secret causes of things, and the expansion of the bounds of human empire to the achievement of all things possible." In your opinion, this solution - the Scientific Method - lead to a successful meta-intervention. I'm not so sure. I would not dispute that the scientific method led to many discoveries that alleviated suffering in many ways and allowed people to dominate the often-brutal natural world in ways that they could not have previously. I would question, though, if this is truly a *successful* meta-intervention. What is "success"? Modern medicine saves lives, but the same processes that brought about vaccines and penicillin have also brought about a huge increase in diabetes, obesity, and other terrible diseases and problems. Technology gave us unheard-of comforts, and brought about the death of tens (hundreds?) of millions from mechanized warfare. We seem to be more comfortable and more miserable than ever. Is this "success"?
To go back to Ada Palmer's quote: "the expansion of the bounds of human empire to the achievement of all things possible" doesn't seem to me to be such an amazing idea, to be honest. I kind of don't like the whole "human empire" thing, I think it is actually a major contributor to the kind of misery we are now experiencing. Perhaps we could address (not solve) some of the problems we see in the world by adopting a humbler stance, recognizing that our Tower of Babel is doomed, climbing down and reaching out to those who are right there, waiting for us to notice them?
P.S. I am speaking as a total layperson, having read very little of the kind of books that others in the comments have read
Yes, intelligence can be used for both good and bad, so having more is a mixed blessing. But so is cooperation. The Nazis in WW2 cooperated among themselves. Every war requires a lot of cooperation on each side. Even small societies in which everyone knows everyone sometimes engage in nasty cooperation like bullying, discrimination or witch hunts. So I'm so not sure that more ability to cooperate at scale will be for the better. Only if we also gain more ability to choose good goals to cooperate on.
agreed cooperation is a dual use weapon! even on a less extreme case than the Nazis, a lot of our civilizational problems are downstream of the incredible cooperative power unlocked by joint-stock limited-liability corporations.
not sure how to think about this, and honestly it might doom the whole cooperation machine project. I'm hoping there's a way to boost non-coercive cooperation without dramatically enhancing the coercive capacity of human groups
I really like this solution. The greatest practical problem to this effort is that cooperation requires some degree of overlap in aims and at least a semblance of agreement over the identification of the problem. Science offers some promise towards the latter goal but I’m not sure what helps us with the former. With faltering trust in science for various reasons, some more nefarious than others, how can we even agree on what the problem is?Never mind how to solve it or what a solution looks like. Not trying to be a downer or anything but just saying the obvious—this is a difficult problem to solve. More difficult than intelligence hands down. At least we can have some semblance of agreement over what intelligence is. Glad someone’s working on it though.
(epistemic caveat: I take the bible as myth, not to dismiss it but to find the useful metaphors)
The biblical story of Lucifer, an angel cast from heaven for his hubris and desire for power, serves as a cautionary tale against prioritizing intelligence above all else. You mention the pursuit of "solving intelligence and using it to solve everything else." This mirrors Lucifer's ambition, as he sought to elevate himself above his designated role, believing his intellect could surpass even that of God (again: mythically "the good").
Many extremely powerful tools like intelligence and economic markets are only as good as how we apply them. In a coordinated manner. Which is, if I understand correctly, what metanarratives can do in large-scale societies. Somewhat poetically, I suspect a resurrection of an evolved ~christian (specifically, WEIRD) metanarrative might be something like the ~answer? To be clear, this is *not* to say we need to all be Christian.
but yeah I expect that at some point we'll have a clear model of how exactly Christian (and other religions') ideas and rituals interfaced with human psychology in order to generate civilization-scale cooperative behavior. right now, in terms of practical cooperation engineering there's probably more to learn from religion than from psychology
You’ll always fail if you’re looking for “the” root. There are many, and some of the biggest and worst always run through things few are willing to give up.
You illustrate this point wonderfully in the bit where you point out that merely knowing more about X, Y and Z doesn't magically make people put it to use, or wisely. Knowledge isn't the bottleneck. *PEOPLE*, tribes we make and belief systems we make up, are where most of the bottlenecks come from, and more often than not a problem that starts in the real world is magnified and/or multiplied when tribes and ideologies get their bloody mittens on them.
You, of course, then make the same mistake in other parts, with silly and gross oversimplifications and false common mischaracterizations, but the thrust is in the right direction I think. Just keep going, apply the critical lens to you and yours more than anyone else, peel another layer off the onion, and never stop.
It's not "inability to understand and cooperate with each other at scale". That's another externalization, trying to pretend our nature is what stops us, when in reality it's our choices. It's *unwillingness*.
Another example: "Our old social technologies for cooperation did not scale to the complexity and enormity of the modern world"... because they're tools in the real world, not magic wands. The people using them are what determines what impacts they are involved in bringing into being.
They weren't "replaced by global capital markets and massive state and corporate bureaucracies". Those are examples of the same thing, and aggregate choices of billions of people.
Choose to trap yourself in caves like: "We need to find the principles underlying these cozy words, and find a way to make them scale.", and you're doing your part to keep the problems where they are, in the exact same way as those you rightly criticized earlier who made the same mistake of thinking we just need to find a different silver bullet of knowledge.
This is even worse than a cave... this is a fatwa/crusade to keep the cancerous paradigm we need to break out from in place even longer: "we need to build a Cooperation Machine that takes in atomized people and raw intelligence and produces mutual understanding and harmonious collective action."
A great many things are needed, but a figurative machine for hacking human nature to manipulate it toward an end is precisely the sort of thinking that's gotten us to where we are right now.
"we need to build a Cooperation Machine that takes in atomized people and raw intelligence and produces mutual understanding and harmonious collective action."
I am sympathetic to this view. The scientific revolution created exponential growth in collective intelligence, while the growth in wisdom across civilization has certainly not kept up. The wisdom-intelligence gap is dangerous and growing.
Agree on the main point: Collaboration is what made humans special, and is the ultimate meta problem of humanity. The difficulty is in finding a balance between the cohesion needed for problem solving, and the individual liberty needed for innovation. How do we balance that? Can we use AI to help? Who knows, I’m researching on this topic…
Minor point 1: The Victorians and the Qings excluded large parts of their population from effective collaboration, which meant that they are less effective at solving problems, and also curtailed individual freedom. Human history may be seen as finding ways to have large scale collaboration, and failing, in some cases.
Minor point 2: Effective altruism ranks problems. This is fundamentally ineffective. In whose framework does one problem rank higher? How is any ranking justified? EA ignores the fact that human progress happens in collectives, which benefits when each of us are exercising our most individual contributions, not when we are submitting ourselves to “a higher-ranking good”.
Great analysis and writing btw! Loved to read careful thoughts and similar insights!
Very, very well put:
"human progress happens in collectives, which benefits when each of us are exercising our most individual contributions, not when we are submitting ourselves to “a higher-ranking good”."
I made the same basic point, but less eloquently, in response to the overtly Orwellian 'Cooperation Machine' idea they floated:
"A great many things are needed, but a figurative machine for hacking human nature to manipulate it toward an end is precisely the sort of thinking that's gotten us to where we are right now."
1) Do you think that cooperation and understanding are "natural"/"default" ?
2) If so, what do you think the main bottlenecks to these likely are?
(1) not sure what you mean by natural, if you mean "require no active effort to maintain" then no, certainly not for humans beyond extended family / Dunbar scales.
(2) for civilization-scale cooperation this feels like asking a Victorian what the main bottleneck to getting to the moon is; like, there's a whole tech tree and massive industrial base missing. can a bottleneck be "sustained effort over decades"?
No such thing as default.
Interesting question, but I would like to challenge your main example. You quote Ada Palmer who speaks of Bacon: "Let us found a new method — the Scientific Method — and with it dedicate ourselves to the advancement of knowledge of the secret causes of things, and the expansion of the bounds of human empire to the achievement of all things possible." In your opinion, this solution - the Scientific Method - lead to a successful meta-intervention. I'm not so sure. I would not dispute that the scientific method led to many discoveries that alleviated suffering in many ways and allowed people to dominate the often-brutal natural world in ways that they could not have previously. I would question, though, if this is truly a *successful* meta-intervention. What is "success"? Modern medicine saves lives, but the same processes that brought about vaccines and penicillin have also brought about a huge increase in diabetes, obesity, and other terrible diseases and problems. Technology gave us unheard-of comforts, and brought about the death of tens (hundreds?) of millions from mechanized warfare. We seem to be more comfortable and more miserable than ever. Is this "success"?
To go back to Ada Palmer's quote: "the expansion of the bounds of human empire to the achievement of all things possible" doesn't seem to me to be such an amazing idea, to be honest. I kind of don't like the whole "human empire" thing, I think it is actually a major contributor to the kind of misery we are now experiencing. Perhaps we could address (not solve) some of the problems we see in the world by adopting a humbler stance, recognizing that our Tower of Babel is doomed, climbing down and reaching out to those who are right there, waiting for us to notice them?
P.S. I am speaking as a total layperson, having read very little of the kind of books that others in the comments have read
P.P.S. Here's an author I find inspirational speaking about something related https://youtu.be/dfWNMktE3xc?si=2DLyNS4FwLRNgdkz&t=829 (listen from about min 13 to 18)
P.P.S. In my opinion, "love" is not any of these other things you mention.
Yes, intelligence can be used for both good and bad, so having more is a mixed blessing. But so is cooperation. The Nazis in WW2 cooperated among themselves. Every war requires a lot of cooperation on each side. Even small societies in which everyone knows everyone sometimes engage in nasty cooperation like bullying, discrimination or witch hunts. So I'm so not sure that more ability to cooperate at scale will be for the better. Only if we also gain more ability to choose good goals to cooperate on.
agreed cooperation is a dual use weapon! even on a less extreme case than the Nazis, a lot of our civilizational problems are downstream of the incredible cooperative power unlocked by joint-stock limited-liability corporations.
not sure how to think about this, and honestly it might doom the whole cooperation machine project. I'm hoping there's a way to boost non-coercive cooperation without dramatically enhancing the coercive capacity of human groups
I really like this solution. The greatest practical problem to this effort is that cooperation requires some degree of overlap in aims and at least a semblance of agreement over the identification of the problem. Science offers some promise towards the latter goal but I’m not sure what helps us with the former. With faltering trust in science for various reasons, some more nefarious than others, how can we even agree on what the problem is?Never mind how to solve it or what a solution looks like. Not trying to be a downer or anything but just saying the obvious—this is a difficult problem to solve. More difficult than intelligence hands down. At least we can have some semblance of agreement over what intelligence is. Glad someone’s working on it though.
(epistemic caveat: I take the bible as myth, not to dismiss it but to find the useful metaphors)
The biblical story of Lucifer, an angel cast from heaven for his hubris and desire for power, serves as a cautionary tale against prioritizing intelligence above all else. You mention the pursuit of "solving intelligence and using it to solve everything else." This mirrors Lucifer's ambition, as he sought to elevate himself above his designated role, believing his intellect could surpass even that of God (again: mythically "the good").
Many extremely powerful tools like intelligence and economic markets are only as good as how we apply them. In a coordinated manner. Which is, if I understand correctly, what metanarratives can do in large-scale societies. Somewhat poetically, I suspect a resurrection of an evolved ~christian (specifically, WEIRD) metanarrative might be something like the ~answer? To be clear, this is *not* to say we need to all be Christian.
your last phrase reminds me of xiq's ideas: https://x.com/exgenesis/status/1559501005229883397
but yeah I expect that at some point we'll have a clear model of how exactly Christian (and other religions') ideas and rituals interfaced with human psychology in order to generate civilization-scale cooperative behavior. right now, in terms of practical cooperation engineering there's probably more to learn from religion than from psychology
Would I be completely mistaken in reading "some new religion" in place of Cooperation Machine? Or does that reveal my lack of imagination here?
Certainly ideology, among which religion is but one option.
You’ll always fail if you’re looking for “the” root. There are many, and some of the biggest and worst always run through things few are willing to give up.
You illustrate this point wonderfully in the bit where you point out that merely knowing more about X, Y and Z doesn't magically make people put it to use, or wisely. Knowledge isn't the bottleneck. *PEOPLE*, tribes we make and belief systems we make up, are where most of the bottlenecks come from, and more often than not a problem that starts in the real world is magnified and/or multiplied when tribes and ideologies get their bloody mittens on them.
You, of course, then make the same mistake in other parts, with silly and gross oversimplifications and false common mischaracterizations, but the thrust is in the right direction I think. Just keep going, apply the critical lens to you and yours more than anyone else, peel another layer off the onion, and never stop.
It's not "inability to understand and cooperate with each other at scale". That's another externalization, trying to pretend our nature is what stops us, when in reality it's our choices. It's *unwillingness*.
Another example: "Our old social technologies for cooperation did not scale to the complexity and enormity of the modern world"... because they're tools in the real world, not magic wands. The people using them are what determines what impacts they are involved in bringing into being.
They weren't "replaced by global capital markets and massive state and corporate bureaucracies". Those are examples of the same thing, and aggregate choices of billions of people.
Choose to trap yourself in caves like: "We need to find the principles underlying these cozy words, and find a way to make them scale.", and you're doing your part to keep the problems where they are, in the exact same way as those you rightly criticized earlier who made the same mistake of thinking we just need to find a different silver bullet of knowledge.
This is even worse than a cave... this is a fatwa/crusade to keep the cancerous paradigm we need to break out from in place even longer: "we need to build a Cooperation Machine that takes in atomized people and raw intelligence and produces mutual understanding and harmonious collective action."
A great many things are needed, but a figurative machine for hacking human nature to manipulate it toward an end is precisely the sort of thinking that's gotten us to where we are right now.
"we need to build a Cooperation Machine that takes in atomized people and raw intelligence and produces mutual understanding and harmonious collective action."
But what does that mean wrt innovation?
Stifling via punishing individuation.
I am sympathetic to this view. The scientific revolution created exponential growth in collective intelligence, while the growth in wisdom across civilization has certainly not kept up. The wisdom-intelligence gap is dangerous and growing.