“Back in 2017, most people were betting that the path to a truly general-purpose system would come from training agents from scratch on a curriculum of increasingly hard tasks, and through this, create a generally capable system. This was present in the research projects from all the major labs, like DeepMind and OpenAI, trying to train superhuman players in games like Starcraft, Dota 2, and AlphaGo. I think of this as basically a “tabula rasa” bet—start with a blank agent and bake it in some environment(s) until it becomes smart. Of course, as we all know now, this didn’t actually lead to general intelligences. At this time, people had started experimenting with a different approach, doing large-scale training on datasets and trying to build models that could predict and generate from these distributions. This ended up working extremely well.” – Jack Clark, co-founder of Anthropic (20260110)
Likely to be revisited after reading Judea PearlTODO
Related:
- 2-1b2b1.1 If it’s not worth doing well, it’s not worth doing at all
- E.g., are LLMs worth accelerating?
- 3-1d5 You can question the adequacy of the tools at hand insofar as it relates to some specific problems of yours. You don’t evaluate them ‘comparatively’ based on its ‘utility’ without explanations.
- 5-1b2 Don’t invest in prediction, because knowledge is inherently unpredictable
- 8-4 LLMs
- SMLs and ASICs need LLMs (at least according to my understanding as of 20260114)
Contradictory?
- 4-1a5 Don’t get obsessed with the failure rate, because what matters is where it’s going and not where it came from
- 9-2a1a キャラ (物語思考) ≠ キャリア (「自己分析」) — Don’t obsessed with where it came from, think about where it’s going)
- 10-1b4e Developing a taste means transitioning from being obsessed with where it came from (analog) to focusing on what it is and what it can do (digital)