Though no one can predict the future, and though abandoning one of the two paths feels politically incorrect, we cannot rule out the possibility of such unipolar dominance.
As is widely known, AI has always been marked by the competition between two schools: symbolic rationalism and data-driven empiricism. Their fortunes have waxed and waned throughout history, but over the past 30+ years, the pendulum has shown no sign of swinging back toward symbolism.
Why?
The ongoing contemporary history of large language models is fascinating. Each time challenges and obstacles arise, the mainstream paradigm overcomes them from within. Whether this will continue remains to be seen, but the trend seems likely to persist.
When large language models (LLM) first emerged, people marveled at their capabilities. But soon, critiques arose: their simple "next token prediction" (NTP) objective and the statistical nature of their probabilistic models led many to conclude they were merely advanced statistical tools, like large parrots—lacking true "understanding."
Ilya Sutskever and Geoffrey Hinton had to step in repeatedly to explain: "Do not underestimate next token prediction. This is no mere statistical n-gram model from the past. It abstracts a system of understanding that integrates human knowledge. When next-token prediction grows increasingly accurate, deep comprehension of context becomes indispensable." Such explanations struggled to convince skeptics. Later, Ilya invoked Kolmogorov complexity as a theoretical foundation, but this framework remains esoteric and inaccessible to most audiences—even many PhDs and professors view it with bemused skepticism. Yet, no better explanation exists.
What ultimately dissolved the "statistical parlor trick" critique was firsthand experience. Users interacting with LLMs realized: these systems seem to genuinely understand. No matter how you phrase your queries, in any language, with nuance or subtext, large models grasp meaning more reliably than most humans.
With the "understanding" debate fading, critics shifted focus: "LLMs cannot reason."
As recently as last year, Yann LeCun cited this as one of his core arguments against the mainstream GPT-style LLM paradigm (advocating instead for vision-based world models as the true path). Many relished pointing out flaws—like LLMs failing at elementary arithmetic such as multi-digit multiplication.
But this critique no longer holds. With the advent of reasoning models like OpenAI’s "o-series" and DeepSeek’s "r-series," accusations of "no reasoning ability" have collapsed. Hardliners may still dismiss probabilistic reasoning as unstable, lacking the rigor of symbolic logic. Yet users deploying these models for mathematics, coding, or project planning overwhelmingly report breakthroughs. Large-model reasoning now rivals or surpasses human experts, approaching master’s or doctoral proficiency. Coding capabilities already exceed those of average engineers. This is just the beginning. It is well plausible that within a year or two, reasoning models could dominate Olympiad-level math or competitive programming.
Once again, barriers were breached through internal innovation—this time after large-model pretraining neared its limits. The core framework remains unchanged, though: reinforcement learning still relies on NTP for chain-of-thought (CoT) generation; reasoning models remain probabilistic. Symbolic AI contributed nothing. Symbols remain confined to input/output interfaces—even the "inner monologue" of CoT manifests as output tokens.
The sheer creative potential within this paradigm is staggering. Those of us from symbolic AI backgrounds once naively imagined that when neural approaches hit walls, our logic-and-grammar toolkit would ride to the rescue. Hybrid neuro-symbolic fantasies danced in our minds.
Zooming out, modern large models evolved from earlier statistical frameworks, with neural networks as a tributary. When those statistical models hit ceilings, breakthroughs came from within—via deep learning. Symbolism played no role.
A profound question arises: Why has the theoretically appealing vision of hybrid neuro-symbolic synergy remained an impractical or unnecessary dream?
Two possibilities stand out.
First, the data-driven empiricist approach possesses far greater resilience and potential than we imagined.
This hints at deeper truths. Artificial neural networks, inspired by biological brains, had languished for decades until the deep learning revolution. Over the past decade, their human-like (or superhuman) performances have forced us to confront a possibility: perhaps this is indeep how intelligence works. If artificial systems achieve human-level cognition through mechanisms mirroring our own biology—despite neuroscientists’ caveats about our limited brain knowledge—this alignment would powerfully validate the neural paradigm. Symbolic logic and statistical feature engineering, by contrast, are alien to biological cognition. Their limitations may stem from this fundamental mismatch. One might even argue that high-dimensional vector spaces in LLMs—where multimodal signals are embedded within neural frameworks—encode a "language of God," or the essence of universal information. Symbols, then, are mere human-imposed constructs, sensory accommodations divorced from reality’s substrate.
Second, when a paradigm harbors untapped potential, progress demands sufficient talent density to exploit it.
AI uniquely concentrates genius. Countless brilliant minds flock to this field, creating an intellectual critical mass unmatched in most domains.
With these conditions in play, we must never underestimate the internal momentum to break through barriers. AGI (Artificial General Intelligence) believers, via their "insane" grind, keep delivering results. Could they indeed be AI’s ultimate Terminators?
Addendum: Symbolic might just be "reduced" to a symbolic tool that may retain its irreplaceable cognitive value
yanyongxin:
What distinguishes humans from other animals is our evolved reasoning capacity. Though rooted in neurons, this ability represents a qualitative leap beyond mere "instinctive reactions." It abstracts object relationships, enabling multi-step reasoning that can be transmitted and memorized through linguistic symbol chains. Reasoning is inherently discrete—thus symbolizable—as a simulated system built atop neural architecture. This simulation likely requires structural differences in human neural systems compared to other animals.
The most striking contrast between reasoning systems and primal neural cognition lies in sustained deliberation. Unlike "muscle memory" or intuition, human reasoning varies dramatically. During my university years, I observed students who excelled at quick problem-solving yet froze when faced with complexity. Today's LLMs approximate the reasoning level of humanities undergraduates, but still lag behind trained STEM specialists—particularly in mathematics and physics. The essence of STEM disciplines lies in rigorously symbolizing real-world problems. Simulating such precision within biological neural systems demands rare opportunities (elite education), prolonged training, and specific neurostructural advantages ("talent"), all channeled through disciplined formalization. Yet achieving this capability bridges biology with mechanical rigor—enabling interfaces with tools like Mathematica.
This charts AI's next frontier: building superior logical simulation systems atop neural frameworks until seamless integration with pure symbolic tools is achieved. The brain's logical simulation system remains energy-intensive, error-prone, and costly to develop. Its key advantage? Seamless integration with underlying neural processes.
Li Wei: Well said.
Interfacing with symbolic systems manifests as tool use. For instance, when confronting complex math problems, instead of forcing probabilistic reasoning through natural-language chain-of-thought (CoT), LLMs should just generate code properly to invoke Mathematica. This tool-use capability is now defined as a fundamental trait of LLM-native agents—yet another innovation emerging from within the paradigm.
Thus, we see a clear evolutionary trajectory:
1. Traditional Statistical Models ("Artificial Idiocy"):
Failure: Little natural language understanding
Solution: LLMs (e.g., ChatGPT)
2. Pretrained LLM:
Failure: Lacking reasoning ability
Solution: Reasoning-reinforced LLMs (e.g., OpenAI’s o1, DeepSeek’s r1)
3. Reasoning LLM:
Failure: Insufficient symbolic rigor
Solution: LLM Agents (symbolic tool integration)
yanyongxin:
Traditional statistical models earned their "artificial idiocy" label because their parameter spaces and data structures proved inadequate to host the world models required for true language understanding.
【相关】
