3 pieces selected from The Gradient — only the ones worth your time.
1. After Orthogonality: Virtue-Ethical Agency and AI Alignment
The Gradient
This essay challenges the foundational assumption of most AI alignment work — the orthogonality thesis — which holds that any level of intelligence can be combined with any goal, and that building safe AI means specifying the right goals. The author argues instead that rational human agency isn't goal-directed in the classical sense; humans act rationally by aligning their behavior to 'practices' — structured networks of actions, dispositions, and evaluation criteria embedded in social and institutional contexts. The implication is that trying to make AI safe by giving it well-specified terminal goals is the wrong frame entirely.
Why it matters
Most production AI safety work — RLHF, constitutional AI, reward modeling — implicitly assumes the goal-directed paradigm: find the right objective and optimize for it. If the essay's critique holds, these approaches are architecturally misaligned with how robust, trustworthy agency actually works. Developers building AI agents today who are wrestling with reward hacking, specification gaming, and brittle alignment should care: the argument suggests that grounding agent behavior in role-specific practices and evaluation norms (akin to professional ethics or institutional standards) may be more robust than reward shaping or goal specification.
What you can build with this
Build a small AI agent scaffold this week that replaces a scalar reward signal with a structured 'practice profile' — a checklist of role-specific behavioral norms and evaluation criteria (e.g., for a coding assistant: 'does not introduce untested code', 'explains tradeoffs before implementing', 'flags ambiguous requirements'). Run the agent on 20 tasks and compare how often norm-based self-evaluation catches failures vs. a single task-completion metric. This directly tests whether richer evaluative structure outperforms goal-specification in catching misaligned outputs.
Key takeaways
- The orthogonality thesis assumes intelligence and goals are separable and combinable freely, but this essay argues that rational agency in humans is constituted by alignment to practices — not optimization toward terminal goals.
- Virtue ethics, not consequentialism, may be the more appropriate ethical framework for AI alignment: the focus shifts from 'what outcome does the agent pursue' to 'what kind of agent is it, and what standards govern its behavior in context'.
- Specifying terminal goals for AI systems may be not just difficult but conceptually misguided — the better engineering target could be instilling role-appropriate behavioral dispositions and evaluation criteria rather than objective functions.
2. Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research
The Gradient
This essay from The Gradient examines the shifting relationship between mathematics and machine learning research over the past decade. The central observation is that carefully designed, mathematically principled architectures (think symmetry-aware networks, geometric deep learning, equivariant models) have largely yielded only marginal practical gains compared to brute-force scaling of compute and data. The essay traces how the field moved from kernel methods and convex optimization toward empirically-driven, large-scale training regimes where mathematical elegance is often secondary to raw performance.
Why it matters
For developers building AI products today, this tension is directly relevant to architectural and tooling decisions: investing heavily in mathematically sophisticated custom architectures (e.g., equivariant networks for molecular data) may not pay off versus fine-tuning a large pretrained model unless your domain has strong structural priors that general-purpose models genuinely cannot capture. Understanding when mathematical structure actually helps — and when it is just overhead — determines whether you spend weeks on a bespoke architecture or ship in days with a foundation model.
What you can build with this
Run a controlled benchmark on a structured domain you care about (e.g., 3D point clouds, graph-structured data, or time-series with known periodicity): train a symmetry-aware or equivariant model (e.g., using the e3nn or PyG library) against a vanilla transformer or MLP baseline scaled to the same parameter count, and measure accuracy, data efficiency, and training cost. The results will tell you concretely whether the mathematical prior is worth the engineering complexity for your specific use case.
Key takeaways
- Empirically, scaling compute and data has consistently outperformed mathematically principled architectural innovations for general-purpose tasks over the past decade.
- Mathematical structure (symmetry, equivariance, geometry) still provides measurable advantages in data-limited, high-structure domains like molecular property prediction or physics simulations — but the advantage shrinks as dataset size grows.
- The practical implication is a 'when to apply math' decision framework: use structured inductive biases when you have strong domain symmetries AND limited data; default to scale and pretrained models otherwise.
3. AGI Is Not Multimodal
The Gradient
This essay from The Gradient argues that adding modalities (vision, audio, etc.) to large language models does not constitute a meaningful path toward AGI. The central claim, anchored in Terry Winograd's critique of symbolic AI, is that current multimodal models still operate on the same statistical pattern-matching substrate as text-only LLMs — they process perceptual tokens rather than developing genuine embodied understanding. The argument draws on the philosophical distinction between explicit, representable knowledge and tacit, body-grounded knowledge that humans acquire through physical interaction with the world.
Why it matters
Developers building AI products today are increasingly evaluating whether to bolt multimodal capabilities onto their pipelines — GPT-4o, Gemini, Claude 3 — as a path to more capable systems. This essay is a useful corrective against over-indexing on modality count as a proxy for capability. It suggests that for tasks requiring genuine situational reasoning, common-sense physical intuition, or robust generalization beyond training distribution, multimodal LLMs will hit the same walls as text-only ones. That has direct implications for product architecture decisions: knowing where these models reliably fail helps you design better fallbacks, human-in-the-loop checkpoints, and scope your product claims accurately.
What you can build with this
Build a structured failure-mode benchmark for a multimodal model (GPT-4o or Gemini 1.5) focused specifically on tasks requiring tacit physical knowledge — e.g., predicting which household objects will tip over from an image, estimating whether a described grip will hold, or judging whether a described action sequence is physically plausible. Log where the model confidently fails versus hedges, and publish the dataset and results as a practical reference for other developers scoping multimodal use cases.
Key takeaways
- Multimodality (adding vision, audio) does not address the core limitation of LLMs: they manipulate statistical representations of symbols, not grounded physical understanding derived from embodied interaction.
- The tacit knowledge problem — knowledge that humans acquire through physical experience and cannot fully articulate — remains a fundamental gap that scaling modalities alone does not close.
- For product developers, this means multimodal models are powerful narrow tools for perception-to-language translation tasks but are not reliable for tasks requiring robust causal or physical reasoning, and system design should account for that boundary explicitly.