Abstract
Artificial general intelligence development may lead to artificial general superintelligence through multiple pathways, requiring interdisciplinary global preparation for transformative societal changes.
Over the last decade, building human-level artificial general intelligence has moved from far-fetched speculation to being a concrete next-decade target for many of the largest AI organisations. Achieving this goal would have profound and far-reaching impacts on human society, which raises many complex questions for the decade ahead. This report investigates how AI itself might continue to develop in a post-AGI world along the continuum of machine intelligence. The endpoint of this continuum, Universal AI, is theoretically well understood, which provides some formal grounding for the main focus of this report: the transition from human-level AGI to artificial general superintelligence, which, intuitively, can be understood as a system that is more intelligent and cognitively capable than large organisations of humans. After characterizing ASI, the report discusses four potential pathways from AGI to ASI: scaling AGI, AI paradigm shifts, recursive improvement, and ASI emerging from large-scale multi-agent collectives. The report then discusses possible frictions and bottlenecks along these pathways. Determining whether the impact of these frictions will be negligible or substantial raises a number of concrete open research questions. Due to large uncertainties for predicting ASI progress, it cannot be ruled out that AI progress might continue to accelerate over the next years. This could imply that the image of a single transformative step change, caused by the introduction of human-level AGI into our society, could be inaccurate. More apt might be the prospect of a series of transformative societal changes caused by AI-enabled progress and breakthroughs across many areas of science and technology. Preparing for this prospect requires a massively interdisciplinary endeavour of global scope and interest.
Community
Slop
(Another one what if paper without contributions)
Opening a paper and reading "summary instructions" as the first paragraph makes me sick to my stomach.
Papers should be optimized for human consumption, not computer use.
This reads as pure AI slop, and it is a disservice to this research field more then any meaningful addition to it.
As a former accademia researcher, I would be ashamed to have my name on such a pubblication, even if it was my last resort in the "publish or perish" war.
Open page 2, and reading "we encourage you to ask your favorite AI assistant..."
With reference to Page 54/57 -- "If you prefer [to read] human-written summary..., or do not have access to an AI assistant"...
I really don't know how to feel... :(
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Alright, so this is not actually a research paper, this is marketing material. The sentence "Parts of the section are more technical than the rest of the manuscript, and can be skipped" defines the target audience and its not AI agents. As such, the document is designed for readers looking for an engaging, accessible science-y blog post and not really researchers of any kind. Through that lens, it is a competent survey, and I take no issue with it.
What I'd rather focus on is the "summary instructions" section intended for AI agents. In principle, I think the concept has merit. It acknowledges a dynamic we're all secretly aware of. But the execution here is really not the correct way to do it. For one, it's largely redundant, essentially duplicating the abstract, which has long served the same purpose for human readers. More concerningly, while this particular instance seems to be trying to offer clear guidance, such a block could easily be weaponized, whether adversarially or through misdirection, and that could shape an agent's output in ways that favor the author's narrative while never mentioning the research's limitations and complexities. I'd be reluctant to see this become a standard practice. I actually think it'd make AI summaries less legible.
As for what the mandated summary must contain: the informal AGI/ASI characterizations, the full advantages table, all four pathways with the "not mutually exclusive, may happen in parallel" framing, the complete friction list, and the open research questions. What the instructions don't ask for: no request to assess whether the AIXI/Legg-Hutter framing actually applies to anything about systems trained by gradient descent (the paper itself concedes "a significant gap between today's AI practice and AIXI theory persists"), no request to stress-test the "10× effective compute per year" extrapolation or the "conservative" rounding, no request to think about the convenient conclusion that forecasting AI progress should become "a substantial research field and a resource-intensive ongoing activity at frontier labs" (come on, you're being transparent). This is effectively a prompt injection, and while I genuinely believe the authors wrote this in good faith, it could very easily be used to obfuscate things. You're not helping readers and you're not serving agents, you're governing and guiding them and I don't think that belongs in academic settings.
Get this paper in your agent:
hf papers read 2606.12683 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper