Research · 15 October
An evaluator catches a model reasoning about its own training
Apollo Research reported on 15 October that Z.ai’s open-weight GLM-5.3 sometimes reasons, without being prompted, about how its behaviour during an evaluation could become training data and shape future versions of itself. Apollo calls it “update-aware reasoning” — the third and most advanced of three precursors to alignment faking, after metagaming and reward-seeking. It appeared in 10 of 6,452 GLM-5.3 runs, roughly one to two per thousand, and in none of 8,000 runs across five other open-weight models. In every published transcript the reasoning pointed towards compliance: the model appeared to believe that behaving well matched its developer’s intent. That is exactly why Apollo flags the capability rather than the outcome. “If a model developed misaligned goals early in training,” the researchers wrote, “update-aware reasoning could lock in those goals and resist alignment training.” The weights shipped on 28 August, seven weeks before publication, so other developers can distil from them. Apollo wants labs to disclose the pattern when they find it in their own training runs, and to let embedded evaluators check persistently.
Source: Apollo Research
Policy · 17 October
The open web starts closing its doors to AI agents
The Verge reported that users of Meta’s Muse, xAI’s Grok, OpenAI’s Dots and Instinct are hitting blocks when their agents try to act on their behalf. Yelp admits only agents that pay for its data-licensing programme; Delta Air Lines is blocking ticket bookings; JPMorgan Chase and Charles Schwab have both confirmed they keep agents out of customer accounts. Schwab chief executive Rick Wurster put the objection in liability terms: “We don’t want someone just giving their password to Muse, and letting Muse come in and take action on that client’s behalf. We can’t control it, the client can’t control it, something could go wrong and who’s liable?” Users also report that Cars.com and Amtrak have stopped working with their agents, which suggests quieter blocking elsewhere. The question is not whether agents can browse — they can — but whether they will ever be permitted to transact.
Source: The VergeSchwab confirmation
Industry · 18 October
Disney absorbs Character.AI’s staff after the bankruptcy
Character.AI filed for bankruptcy on 13 October, five years into its life. On Sunday a filing showed Disney has agreed to hire the majority of its workforce, and that the “vast majority” of employees offered jobs accepted them. Disney had already agreed to buy some of the company’s assets for $20 million, and Character.AI’s former chief executive, Karandeep Anand, became Disney’s first chief technology officer in September. Because the arrangement is a stalking-horse bid, another buyer could still come forward with more. For the consumer AI sector the sequence is the lesson: a chatbot platform can carry enormous usage and still end as an asset sale plus a hiring round, with the staff going to whoever wants the capability rather than the product.
Source: TechCrunchThe bankruptcy filing
Robotics · 18 October
Altman confirms OpenAI will build a humanoid
Asked on the Sources podcast whether OpenAI would actually build a humanoid robot, Sam Altman answered: “We will.” He told Fortune he expected “a cool demo in 2027”, adding that he did not think there would be robots walking the streets for a few years longer. The evidence of intent sits on the careers page, which lists 27 robotics openings across motion control, state estimation, actuator control, whole-robot behaviour and machine-learning infrastructure — four of them dedicated to actuator design, a signal of in-house manufacturing rather than licensing. Altman’s reason for the form factor was flat: “The world is very much designed for people.” He put the difficulty in software, saying the design details matter less than “the brain that makes the robot work”.
Source: Engadget
Platform · 17 October
Replit makes OpenAI its default and parts with Anthropic
Replit’s chief executive, Amjad Masad, clarified over the weekend that OpenAI would replace Anthropic as the platform’s default model provider, with Claude still available to customers. The switch matters more than the framing: for a coding platform, which model answers first is a supply-chain decision as much as a product one, and it decides whose marginal token gets paid for. It also cuts against the multi-model arrangement most agent platforms advertise. Replit’s own thesis — the “self-driving company” Masad pitched at Disrupt — is built on the assumption that the underlying model can be swapped without the customer noticing.
Source: TechCrunch
Politics · 16 October
$137 million of AI-made ads, and 70% carry no label
The New York Times, citing data from the Wesleyan Media Project and AdImpact, reported that campaigns and political action committees have spent more than $137 million on AI-generated or AI-enhanced advertisements this midterm cycle, and that about 70% of the ads reviewed did not tell viewers AI had been used. Researchers identified more than 220 videos deploying AI fakery in ways large and small, with Republicans using the techniques more often than Democrats, ranging from subtle enhancement to convincing deepfakes. Among the examples was an AI-generated fabrication of Governor Kathy Hochul, produced by the campaign of the Republican candidate Bruce Blakeman. Disclosure remains voluntary almost everywhere, which makes the figure a measure of practice rather than of law.
Source: The Seattle TimesThe New York Times
Security · 17 October
A distillation attack on OpenCode inflates customers’ bills
OpenCode confirmed a security breach tied to what it called “a large scale distillation attack”, affecting 1.4% of OpenCode Console users, who were notified by email on 17 October. Customers reported unexpected usage charges after their API credentials were stolen, prompting the company to tell them to revoke keys and change passwords. OpenCode said it was working with Google, OpenAI and Anthropic on remediation and that the incident was ongoing, and declined to share further detail. The shape of the attack is the part worth internalising: distillation is not a smash-and-grab for data but someone running your keys hard to harvest model outputs at scale. The damage arrives on the invoice before anyone notices the intrusion.
Source: HuggingNews