4 pieces selected from AI Alignment Forum, The Gradient — only the ones worth your time.
1. Data filtering works a lot worse than you would expect
AI Alignment Forum
This work was largely done during Neel Nanda's MATS 10.0 Exploration Phase.
J Rosser and Dohun Lee are co-first authors for this post with equal contribution. Josh Engels and Neel Nanda supervised the project, and provided guidance and feedback throughout.
Tweet Thread
TLDR
Models can acquire undesirable traits from during supervised fine-tuning (SFT). A natural thing to try is to identify the
Key takeaways
2. After Orthogonality: Virtue-Ethical Agency and AI Alignment
The Gradient
This essay challenges the conventional goal-oriented approach to AI alignment, arguing that human rationality is not driven by fixed goals but by alignment to practices—networks of actions, dispositions, and evaluation criteria. The author proposes that AI systems should similarly be designed to align with ethical practices rather than rigid goals, drawing on virtue ethics to frame AI behavior as contextually adaptive and morally grounded.
Why it matters
Developers building AI products often rely on goal-based frameworks, which can lead to misalignment when real-world contexts shift. This essay provides a philosophical foundation for designing AI systems that adapt to ethical practices, reducing the risk of unintended consequences in dynamic environments.
What you can build with this
Design an AI-driven moderation system for an online community that evaluates user behavior not against fixed rules but against evolving ethical practices, using reinforcement learning to adapt to community norms over time.
Key takeaways
- Human rationality is better understood as alignment to practices rather than pursuit of fixed goals.
- AI systems should be designed to align with ethical practices, not just predefined objectives.
- Virtue ethics offers a framework for building contextually adaptive and morally grounded AI.
3. Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research
The Gradient
This essay examines the evolving role of mathematics in machine learning research over the past decade. It highlights a shift from mathematically principled architectures, which now yield only marginal improvements, to compute-intensive, engineering-driven approaches that scale with larger training sets. The author argues that while mathematical foundations remain important, the focus has moved toward empirical results and scalability, driven by advances in hardware and data availability.
Why it matters
For developers building AI products, this shift underscores the importance of prioritizing scalable, engineering-first solutions over purely theoretical advancements. It suggests that leveraging large-scale compute and data can often yield more practical improvements than refining mathematical architectures, especially in real-world applications where performance and scalability are critical.
What you can build with this
This week, start a project that compares the performance of a mathematically optimized model (e.g., a carefully designed neural network architecture) against a scaled-up, compute-intensive version of a simpler model (e.g., a larger transformer trained on more data) on a specific task like image classification or language modeling. Measure and analyze the trade-offs in accuracy, training time, and resource usage.
Key takeaways
- Mathematically principled architectures now offer only marginal improvements in machine learning.
- Compute-intensive, engineering-driven approaches are driving progress in modern machine learning.
- Scalability and empirical results are often more impactful than theoretical refinements in real-world AI applications.
4. AGI Is Not Multimodal
The Gradient
The essay argues that current multimodal AI models, despite their impressive capabilities, do not represent a path to Artificial General Intelligence (AGI). The author contends that these models lack the embodied, tacit understanding that underpins human intelligence, and that projecting language as the sole model for thought is insufficient for achieving AGI. The piece highlights the limitations of current AI systems, which excel in pattern recognition and generation but fail to grasp the deeper, contextual understanding that humans possess.
Why it matters
Developers building AI products need to understand the fundamental limitations of current multimodal models to set realistic expectations and avoid overpromising on capabilities. Recognizing these constraints can guide more effective product design and development, focusing on augmenting human intelligence rather than attempting to replicate it fully.
What you can build with this
Develop a hybrid AI system that combines multimodal AI with human-in-the-loop processes to handle tasks requiring deeper contextual understanding. For example, build a customer support tool that uses AI for initial responses but seamlessly escalates complex queries to human agents, leveraging AI's strengths while acknowledging its limitations.
Key takeaways
- Current multimodal AI models lack the embodied, tacit understanding that characterizes human intelligence.
- Projecting language as the sole model for thought is insufficient for achieving AGI.
- AI systems excel in pattern recognition and generation but fail to grasp deeper, contextual understanding.