I've never thought that I'd look forward to receiving emails from a mailing list, congratulations you've changed that. You write the most interesting articles, keep up the great work!
> (...) Python (...) All those in turn risk being replaced by AI coding tools like Github’s Copilot.
I think we might end up with structured language / languages, maybe allowing for arbitrary/partial underspecification through natural language use. Which is then compiled/interpreted by LLMs. You could use pure natural language too, but it'd make for pointlessly hard to interpret / unclear source code - definition of the system you wish to generate.
> There is a tediously common cycle of hand-wringing over whatever is the latest deepfake technology advance, and how it has the potential to obliterate our capacity to discern truth from fiction. This just has not happened.
I think it already is happening, a little bit. Maybe relatively obsolete tech is used. But I've seen weird content sometimes. Things unlike old SEO spam. Like text that seemed sorta on-topic (for the Google query, for example), but then sharply changed topic.
The thing is, I'm pretty sure even GPT-3 allows for pretty absurd stuff if used properly. Not just lazy prompt "write an article on X". This is just pure NLP. Overall system/program could have straightforward software, which would constitute agent's scaffolding - which uses calls to GPT-3 and other NNs. Use some human behavior, or decision-making - coming from specific kinds of humans. Interact with public blockchains / economy (btw solution for coordinating open-access AI - have them controlled by DAO?).
Also, integration to some extent with human laws / regulations might happen? Eating political systems (maybe it would be wise to set them up before it happens - such that there'd be an way to opt-in to direct democracy as a citizen).
Maybe we'll end up with some giant ball of mud multi-headed monstrosity, which overall implements "sorta approximating friendly" AGI, acting as force multiplier of our Society? Maybe inflexible enough to execute outright betrayal without breaking down itself.
Am a bit high, so the above might be nonsense, IDK.
> Formally, Governance is an AI‐mediated Human‐interpretable Abstracted Democracy. It was constructed as an alternative to the Utilitarian AI Technocracy advocated by many of the pre‐Unification ideologues. As such, it is designed to generate results as close as mathematically possible to the Technocracy, but with radically different internal mechanics.
> The interests of the government's constituents, both Human and True Sentient, are assigned to various Representatives, each of whom is programmed or instructed to advocate as strongly as possible for the interests of its particular topic. Interests may be both concrete and abstract, ranging from the easy to understand "Particle Physicists of Mitakihara City" to the relatively abstract "Science and Technology".
> Each Representative can be merged with others—either directly or via advisory AI—to form a super‐Representative with greater generality, which can in turn be merged with others, all the way up to the level of the Directorate. All but the lowest‐level Representatives are composed of many others, and all but the highest form part of several distinct super‐Representatives.
> Representatives, assembled into Committees, form the core of nearly all decision‐making. These committees may be permanent, such as the Central Economic Committee, or ad‐hoc, and the assignment of decisions and composition of Committees is handled by special supervisory Committees, under the advisement of specialist advisory AIs. These assignments are made by calculating the marginal utility of a decision inflicted upon the constituents of every given Representative, and the exact process is too involved to discuss here.
> At the apex of decision‐making is the Directorate, which is sovereign, and has power limited only by a few Core Rights. The creation—or for Humans, appointment—and retirement of Representatives is handled by the Directorate, advised by MAR, the Machine for Allocation of Representation.
> By necessity, VR Committee meetings are held under accelerated time, usually as fast as computational limits permit, and Representatives usually attend more than one at once. This arrangement enables Governance, powered by an estimated thirty‐one percent of Earth's computing power, to decide and act with startling alacrity. Only at the city level or below is decision‐making handed over to a less complex system, the Bureaucracy, handled by low‐level Sentients, semi‐Sentients, and Government Servants.
> The overall point of such a convoluted organizational structure is to maintain, at least theoretically, Human‐interpretability. It ensures that for each and every decision made by the government, an interested citizen can look up and review the virtual committee meeting that made the decision. Meetings are carried out in standard human fashion, with presentations, discussion, arguments, and, occasionally, virtual fistfights. Even with the enormous abstraction and time dilation that is required, this fact is considered highly important, and is a matter of ideology to the government.
> EDC never ceded power at all, absorbing the few remaining independent governments at the end of the war with the bluntly honest explanation that the EDC believed that future peace could be best secured under its own, direct rule. This (...) removed any remaining illusions that the EDC was anything other than an oligarchical, unelected, secret military junta. EDC spent its power to remake human society forever, bowing out when its job was done to make way for its own hand-designed successor, the enigmatic Governance.
> The ten-year post-war saga of the EDC seems almost impossible, more dream than reality. The idea of a group of oligarchical technocrats governing so effectively, despite the well-known flaws of human nature, had more in common with the fever dreams of early twentieth-century utopians than anything the weight of history would suggest. The EDC, some allege, was the site of a quiet takeover of Humanity itself, by its AIs, by its magical girls, by the Incubators, or by some combination of the three.
[...]
> "Eventually, the balance of power shifted, and the government, in all its organs, exceeded the power of its own people. (...) , those who governed found that they could direct their nations in whatever idiosyncratic direction they pleased, in directions that did not have even a theoretical bearing on the interests of their subjects, and were in fact often openly hostile to those interests."
> "Eventually, the power of those who cared not for their subjects was overthrown from the outside, by the power of those who did, by the efforts of the very Committee seated here. (...) let us not delude ourselves as to the transient nature of this victory, or wear out our arms patting ourselves on the back. This was no victory of the powerless over the powered. This was the victory of some with power over others with power, and as such bodes only ill for the future."
> "It is impossible to return to the past, or restrict our development, as some still delude themselves into advocating. The lessons of industrialism, of plenty, can never be forgotten. The rightful craving for more wealth, more plenty will always be there. The people, the government–they will crave it, and between them they will destroy anything in their way."
> "No, while we still live, we should do what we can to become that eternal government, and to ensure, while we still live, that those who follow can never stray from the path. My allies and I therefore humbly submit to the Committee the following set of guiding principles, or let us be frank about it, ideological tenets:"
> 1) That our future government dedicate itself wholeheatedly to the problem of staying in power forever.
> This is not a matter of power-lust; it is a matter of what is necessary. Of course, this entails the suppression, ruthless if necessary, of competing ideologies and organizations.
> 2) That, as much as possible, no one being shall ever rule, or experience what it is like to rule. (...) By making the leaders mental combinations of their followers, their subjects, it may be possible to construct leaders who would no more enjoy abusing their power than you would enjoy abusing your power to control your own limbs
> 3) That, whatever else may ultimately prove necessary, no sentient shall ever be coerced by direct manipulation of its thought processes
> 6) The maximization of the freedom perceived by sentient individuals. (...) the attempt to maintain a true absolute freedom is impossible, impractical, and even unpleasant in many circumstances. In the end what matters is what the individuals involve perceive as being free, and this is what should be sought.
> 7) The maximization of economic prosperity, defined as both the average and minimal amount of resources that can be accessed by any given sentient.
> 8) The maintenance of Humanity’s core values and distinguishing characteristics into the indefinite future.
> We see it as our responsibility to ensure that even our most distant descendants understand what it means to be human–otherwise, it makes no difference whether humanity survives or not. The physical trappings of humanity are less important than the mental aspects, but should not be abandoned unless necessary.
> 9) As much as possible, the maximization of the number of human-related sentients
I've never thought that I'd look forward to receiving emails from a mailing list, congratulations you've changed that. You write the most interesting articles, keep up the great work!
> (...) Python (...) All those in turn risk being replaced by AI coding tools like Github’s Copilot.
I think we might end up with structured language / languages, maybe allowing for arbitrary/partial underspecification through natural language use. Which is then compiled/interpreted by LLMs. You could use pure natural language too, but it'd make for pointlessly hard to interpret / unclear source code - definition of the system you wish to generate.
> There is a tediously common cycle of hand-wringing over whatever is the latest deepfake technology advance, and how it has the potential to obliterate our capacity to discern truth from fiction. This just has not happened.
I think it already is happening, a little bit. Maybe relatively obsolete tech is used. But I've seen weird content sometimes. Things unlike old SEO spam. Like text that seemed sorta on-topic (for the Google query, for example), but then sharply changed topic.
The thing is, I'm pretty sure even GPT-3 allows for pretty absurd stuff if used properly. Not just lazy prompt "write an article on X". This is just pure NLP. Overall system/program could have straightforward software, which would constitute agent's scaffolding - which uses calls to GPT-3 and other NNs. Use some human behavior, or decision-making - coming from specific kinds of humans. Interact with public blockchains / economy (btw solution for coordinating open-access AI - have them controlled by DAO?).
Also, integration to some extent with human laws / regulations might happen? Eating political systems (maybe it would be wise to set them up before it happens - such that there'd be an way to opt-in to direct democracy as a citizen).
Maybe we'll end up with some giant ball of mud multi-headed monstrosity, which overall implements "sorta approximating friendly" AGI, acting as force multiplier of our Society? Maybe inflexible enough to execute outright betrayal without breaking down itself.
Am a bit high, so the above might be nonsense, IDK.
Hm, related (ignoring sentient AGI agents, and also that math solution to FAI is provided by 'wish' anyway): https://www.fanfiction.net/s/7406866/8/To-the-Stars
> Formally, Governance is an AI‐mediated Human‐interpretable Abstracted Democracy. It was constructed as an alternative to the Utilitarian AI Technocracy advocated by many of the pre‐Unification ideologues. As such, it is designed to generate results as close as mathematically possible to the Technocracy, but with radically different internal mechanics.
> The interests of the government's constituents, both Human and True Sentient, are assigned to various Representatives, each of whom is programmed or instructed to advocate as strongly as possible for the interests of its particular topic. Interests may be both concrete and abstract, ranging from the easy to understand "Particle Physicists of Mitakihara City" to the relatively abstract "Science and Technology".
> Each Representative can be merged with others—either directly or via advisory AI—to form a super‐Representative with greater generality, which can in turn be merged with others, all the way up to the level of the Directorate. All but the lowest‐level Representatives are composed of many others, and all but the highest form part of several distinct super‐Representatives.
> Representatives, assembled into Committees, form the core of nearly all decision‐making. These committees may be permanent, such as the Central Economic Committee, or ad‐hoc, and the assignment of decisions and composition of Committees is handled by special supervisory Committees, under the advisement of specialist advisory AIs. These assignments are made by calculating the marginal utility of a decision inflicted upon the constituents of every given Representative, and the exact process is too involved to discuss here.
> At the apex of decision‐making is the Directorate, which is sovereign, and has power limited only by a few Core Rights. The creation—or for Humans, appointment—and retirement of Representatives is handled by the Directorate, advised by MAR, the Machine for Allocation of Representation.
> By necessity, VR Committee meetings are held under accelerated time, usually as fast as computational limits permit, and Representatives usually attend more than one at once. This arrangement enables Governance, powered by an estimated thirty‐one percent of Earth's computing power, to decide and act with startling alacrity. Only at the city level or below is decision‐making handed over to a less complex system, the Bureaucracy, handled by low‐level Sentients, semi‐Sentients, and Government Servants.
> The overall point of such a convoluted organizational structure is to maintain, at least theoretically, Human‐interpretability. It ensures that for each and every decision made by the government, an interested citizen can look up and review the virtual committee meeting that made the decision. Meetings are carried out in standard human fashion, with presentations, discussion, arguments, and, occasionally, virtual fistfights. Even with the enormous abstraction and time dilation that is required, this fact is considered highly important, and is a matter of ideology to the government.
Ideology is basically high level description of Governance values, I guess. Constitution, bounding democracy. https://ttshieronym.tumblr.com/post/68536325770/organization-post-2-governance-part-3-ideology
> EDC never ceded power at all, absorbing the few remaining independent governments at the end of the war with the bluntly honest explanation that the EDC believed that future peace could be best secured under its own, direct rule. This (...) removed any remaining illusions that the EDC was anything other than an oligarchical, unelected, secret military junta. EDC spent its power to remake human society forever, bowing out when its job was done to make way for its own hand-designed successor, the enigmatic Governance.
> The ten-year post-war saga of the EDC seems almost impossible, more dream than reality. The idea of a group of oligarchical technocrats governing so effectively, despite the well-known flaws of human nature, had more in common with the fever dreams of early twentieth-century utopians than anything the weight of history would suggest. The EDC, some allege, was the site of a quiet takeover of Humanity itself, by its AIs, by its magical girls, by the Incubators, or by some combination of the three.
[...]
> "Eventually, the balance of power shifted, and the government, in all its organs, exceeded the power of its own people. (...) , those who governed found that they could direct their nations in whatever idiosyncratic direction they pleased, in directions that did not have even a theoretical bearing on the interests of their subjects, and were in fact often openly hostile to those interests."
> "Eventually, the power of those who cared not for their subjects was overthrown from the outside, by the power of those who did, by the efforts of the very Committee seated here. (...) let us not delude ourselves as to the transient nature of this victory, or wear out our arms patting ourselves on the back. This was no victory of the powerless over the powered. This was the victory of some with power over others with power, and as such bodes only ill for the future."
> "It is impossible to return to the past, or restrict our development, as some still delude themselves into advocating. The lessons of industrialism, of plenty, can never be forgotten. The rightful craving for more wealth, more plenty will always be there. The people, the government–they will crave it, and between them they will destroy anything in their way."
> "No, while we still live, we should do what we can to become that eternal government, and to ensure, while we still live, that those who follow can never stray from the path. My allies and I therefore humbly submit to the Committee the following set of guiding principles, or let us be frank about it, ideological tenets:"
> 1) That our future government dedicate itself wholeheatedly to the problem of staying in power forever.
> This is not a matter of power-lust; it is a matter of what is necessary. Of course, this entails the suppression, ruthless if necessary, of competing ideologies and organizations.
> 2) That, as much as possible, no one being shall ever rule, or experience what it is like to rule. (...) By making the leaders mental combinations of their followers, their subjects, it may be possible to construct leaders who would no more enjoy abusing their power than you would enjoy abusing your power to control your own limbs
> 3) That, whatever else may ultimately prove necessary, no sentient shall ever be coerced by direct manipulation of its thought processes
> 6) The maximization of the freedom perceived by sentient individuals. (...) the attempt to maintain a true absolute freedom is impossible, impractical, and even unpleasant in many circumstances. In the end what matters is what the individuals involve perceive as being free, and this is what should be sought.
> 7) The maximization of economic prosperity, defined as both the average and minimal amount of resources that can be accessed by any given sentient.
> 8) The maintenance of Humanity’s core values and distinguishing characteristics into the indefinite future.
> We see it as our responsibility to ensure that even our most distant descendants understand what it means to be human–otherwise, it makes no difference whether humanity survives or not. The physical trappings of humanity are less important than the mental aspects, but should not be abandoned unless necessary.
> 9) As much as possible, the maximization of the number of human-related sentients