From what I'm experiencing, we have recursive self improvement that can find local maxima, I'm not seeing really strong evidence of unguided RSI that finds the true optimal.
Doesn’t matter if there’s a rechable local maximum that happens to be comfortably beyond human level. Indeed it would be very surprising if a thing capable of RSI just happened to get stuck at human level or slightly above or below, even though its constraints are entirely different from those evolution had to work with when it created us!
What if you are underestimating some qualities of our intelligence and how efficient they are physically, and underestimate how much less the hardware / tech stack is in comparable efficiency?
What if the bottleneck is physical chips and data centers and power and only so much can be squeezed out of algorithms?
What if AI seems more impressive than it is, because the things it is good at happens to be hard for us, and it's hard for us because it's new to us (processing lots of abstract information), but the things that we are good at (manipulating physical world) are objectively orders of magnitude harder?
It has nothing to do with special. It has to do with the physics of memory access, neuron diversity and synapse density in biological brains, and the limitations of the current hardware architecture of AI.
1. A lot of problems are inherently high-dimensional, so you're often more likely to run into saddle points
2. Some problems don't even have a bounded global maximum, a true optimal
A more concerning bottleneck is diminishing returns: even with recursive improvement, capability gains could end up being only logarithmic (or worse). Even current systems seem capable of these diminishing returns only/mostly.
That kind of assumes there is an optimal. It seems quite likely not so - that there will always be room for improvement in AI like we don't have a true optimal chess or go programs. They just get better and go up a bit on the ELO scale over time.
like https://huggingface.co/spaces/lmarena-ai/chatbot-arena has a score for chatbots - 1506 for Claude Fable 5 at the moment for example. I guess the test for RSI is if it can make the number go up. It'll probably go through phases of self improvement being a bit rubbish, self improvement combined with human help being best which is probably where we are now, and humans not needed perhaps in the future.
There is an optimal chess program and there is an optimal AI algorithm. The former is merely practically uncomputable, the latter is theoretically uncomputable.
We already know why most things happen. But beat around the bush. That is the problem. We don't care.
When a severe security bug crops up, one postmortem tells where we messed up. Someone somewhere didn't do his job. Maybe they prioritised speed, money or due to management issue. But we know why.
When an accident happens (Motorvehicles or otherwise), most of the time, if anyone in the chain of events did their job, it could've been avoided. But we ignored the policies, standards and laws set in place to avoid just that. There can't be a better example than Boeing for this.
When it comes to natural disasters, we know how to fix most of it. But we as human race just don't want to.
Hell, we knew how to freakin avoid Covid which came out of the blue. 6ft apart, wash hands and mask. We couldn't agree to that to keep us ALIVE!
IMO, we don't want new technologies, new innovations, new laws in most cases. We just needs to apply what we know already - PROPERLY! But that ain't fancy.
All I could remember is that tweet where musk has a $2M reward for the invention of a carbon capture device or something around those lines. And someone replied to that tweet saying, it's trees. Trees do this.
Another big one is climate change. Ask the ASI how to solve that, and it's just going to say make a huge push for more renewables like solar, and strongly disincentivize GHG emissions. We already knew we had to do that. But the people in charge of the economy don't wanna.
> We already knew we had to do that. But the people in charge of the economy don't wanna.
I've often heard it said that <big corporations> often use consulting firms to recommend unpopular decisions they were already planning to make, thereby offloading the culpability to the consulting firm.
Maybe the LLMs are going to be the ultimate "consulting firm" for our societal issues.
I say this mostly tongue-in-cheek, but we're already offloading a lot of lower-level responsibilities (e.g., "write my email") onto these AIs.
Large organizations also offload decision making when they don't have a clue, and simply want to point to some action taken. The answer doesn't even matter, they just don't want to be responsible for something, or waste their own time making something (that other stakeholders care about) a real priority.
People downplay the value of consultants, but they have many uses!
Exactly. It's not about us NOT having the knowledge, understanding or capability to solve the issues that we have. We just don't want to do the right thing. That doesn't get fixed even if we invent time machines.
What local maxima are you seeing? That would imply that progress has stalled, which doesn't appear to be the case where I'm looking.
Also, RSI is obviously guided. If only guided by "it's not giving results so we'll try something else". To require that RSI happens in a black box for it to count would be arbitrary, and also not how anyone is going to do it.
Running a bunch alongside APIs for suite of internal services where domain experts are using agents to setup environments for their work. Getting them to update their LLMs to a new SDK version every time we iterate API capabilities is all the headache you'd imagine. The MCP avoids inconsistent user reports of capabilities.
The specifics, now user can ask, "Provision 200Gi PostgreSQL 18 with PostGIS 3.6 and then copy over our most recent Texas set of layers."
Behind the scene Talos, k8s, NiFi, and bunch of things go to work. Big productivity boost vs. the users needing to login to bunch of provisioning web interfaces which is the other method we publish that avoids version drift issues.
My ability to detect sarcasm is not good. From looking at funlang's profile and other comments the profile looks like a LLM generated bot.
Forums with full no verification pseudonyms seem like they have a real challenge ahead. How long until we need humanhackernews.com with public pseudonyms and a private trusted verification?
The solution exists in the financial controls world. Agent = drafter, human = approver. The challenge is very few applications are designed to allow this, Amazon's 1-click checkout is the exact opposite. Writing a proxy for each individual app you give it access to and shimming in your own line of what the agent can do and what it needs approval is a complex and brittle solution.
Yes very funny to see their own model betray them like this:
> The original “non‑profit, open, patents shared” promise now reads almost like an alternate timeline. Today OpenAI is a capped‑profit entity with a massive corporate partner, closed frontier models, and an aggressive product roadmap.
Page 5, "The knowledge cutoff date for Gemini 3 Pro was January 2025."
Still taking nearly a year to train and run post training safety and stability tuning.
With 10x the infrastructure they could iterate much faster, I don't see AI infrastructure as a bubble, it is still a bottleneck on pace of innovation at today's active deployment level.
I'm pretty much doing that in a containerized deployment for project I'm looking to open source soon called webbin:
Technology Stack
Frontend: React 18 + TypeScript + Vite + TailwindCSS
Backend: Node.js + Express + Clustering
Database: PostgreSQL 16 with performance optimization
Cache: Redis 7 with active defragmentation
Security: HTTPS/TLS with container-to-container encryption
Orchestration: Docker Compose with health checks
Monitoring: Built-in APM and performance tracking
Services
webbin-frontend - React TypeScript frontend with HTTPS
webbin-backend - Node.js API with clustering support
webbin-postgres - PostgreSQL 16 with performance tuning
webbin-redis - Redis 7 with advanced caching
webbin-certbot - SSL Certificate management, openssl for dev, LE for production
webbin-testrunner - Testbot
webbin-nginx - Proxy
Access Points
Frontend: https://localhost:5173
Backend API: https://localhost:3001
Health Check: https://localhost:3001/api/health
I'm around 15k LOC, all built in ~80 hours of interactive prompting mostly with Claude 4.0 Sonnet, then some Gemini 2.5 Pro for more devops activities.
"An article by Krebs on 27 March 2018 on KrebsOnSecurity.com about the mining software company and script "Coinhive" where Krebs published the names of admins of the German imageboard pr0gramm, as a former admin is the inventor of the script and owner of the company, was answered by an unusual protest action by the users of that imageboard. Using the pun of "Krebs" meaning "Cancer" in German, they donated to charitable organisations fighting against those diseases, collecting more than 200,000 Euro of donations until the evening of 28 March to the Deutsche Krebshilfe charity".
I believe we'll see the benefits and drawbacks of AI augmentation to humans performing various tasks will vary wildly based on the task, the way the AI is being asked to interact, and the AI model.
English football, unlike tic-tac-toe, can be thrilling and end in a draw. Possession mix, shots on goal, and more stats are useful to determine how exciting a match was from a box score.
Frankly, for me the most boring is a 2-0 win where the team scores those 2 in the first 20 to 30 minutes, swaps to a 5-4-1, and plays tiki-taka passing possession control without trying hard to advance the ball for the remaining hour of the match.
reply