I completely ignored Anthropic’s advice and wrote a more elaborate test prompt based on a use case I’m familiar with and therefore can audit the agent’s code quality. In 2021, I wrote a script to scrape YouTube video metadata from videos on a given channel using YouTube’s Data API, but the API is poorly and counterintuitively documented and my Python scripts aren’t great. I subscribe to the SiIvagunner YouTube account which, as a part of the channel’s gimmick (musical swaps with different melodies than the ones expected), posts hundreds of videos per month with nondescript thumbnails and titles, making it nonobvious which videos are the best other than the view counts. The video metadata could be used to surface good videos I missed, so I had a fun idea to test Opus 4.5:
I wanted to test this claim with SAT problems. Why SAT? Because solving SAT problems require applying very few rules consistently. The principle stays the same even if you have millions of variables or just a couple. So if you know how to reason properly any SAT instances is solvable given enough time. Also, it's easy to generate completely random SAT problems that make it less likely for LLM to solve the problem based on pure pattern recognition. Therefore, I think it is a good problem type to test whether LLMs can generalize basic rules beyond their training data.
，更多细节参见Line官方版本下载
#欢迎关注爱范儿官方微信公众号：爱范儿（微信号：ifanr），更多精彩内容第一时间为您奉上。。快连下载安装是该领域的重要参考
沪公网安备31010602000299号