This week OpenAI announced a 750-task test to to measure "whether AI systems can support realistic life science research tasks, not just answer biology questions." But while OpenAI's top-performing GPT-Rosalind model led the rankings, Slashdot reader BrianFagioli notes that "it achieved a pass rate of just 36.1 percent, failing nearly two-thirds of benchmark tasks." Nerds.xyz points out that means "the best-performing model failed nearly two-thirds of the benchmark's tasks." The benchmark also revealed a familiar weakness. AI systems generally perform better when everything is presented as text. Once they are forced to work with supporting documents, figures, or complex datasets, performance drops noticeably. GPT-Rosalind's pass rate fell from 45.1 percent on text-only tasks to 28.1 percent on tasks involving artifacts or URLs. To be fair, the benchmark is not intended to suggest AI is useless in research. Quite the opposite. OpenAI found that models are becoming increasingly capable of scientific communication, evidence synthesis, and translating research findings into practical explanations. Those are valuable skills, particularly for researchers drowning in information. But LifeSciBench serves as a useful reminder that today's AI systems are still far from autonomous scientists. They can help. They can assist. They can sometimes provide surprisingly useful insights. What they cannot reliably do, however, is replace the expertise, judgment, and skepticism that real scientific research requires.

Read more of this story at Slashdot.

Long-time tech pundit Robert Cringely started his career at the Stanford Artificial Intelligence Lab back in 1978. Last month 73-year-old Cringely explained why his site went on a two-year hiatus — and it's not just because of a heart attack and a stroke last July: Just like everyone else, I've been busy all this time on Artificial Intelligence, founding with two partners a company called 2Brains... The work we were doing together is unfinished, but it's not stopped. The patents are filed, the architecture is documented, and the small team continuing the work includes me. Cringely's first piece made the cast that "the trillion-dollar bet the AI industry is making right now may be wrong, and that there's an architectural alternative we've patented and built." In Machines of Loving Grace, Amodei made the case that scaling compute would eventually solve essentially every hard problem in artificial intelligence. Buried in that optimism — or maybe not buried, maybe right out in the open — was a quiet absolution. Hallucinations, the embarrassing tendency of these systems to state falsehoods with total confidence, would take care of themselves. Make the models big enough, train them long enough, and the problem dissolves. You don't have to solve it. You just have to wait, and spend. And so the entire AI industry breathed a sigh of relief. I have spent forty years watching this industry, and I know a permission slip when I see one. Because that is what the essay became, whatever Amodei intended. It gave every other person writing nine- and ten-figure checks a reason not to worry about the one thing that should worry them most. The hallucination problem is the difference between a clever toy and a system a hospital or a bank or a court can actually rely on. It is the whole ballgame for enterprise AI. And the prevailing wisdom, blessed from the top, is that you needn't address it directly. Scale will provide... A small company I helped start, 2Brains Inc., set out in 2022 to solve hallucinations — before ChatGPT, before the scaling consensus hardened into received truth, back when the polite assumption was that the problem was simply insurmountable. We did not solve it by waiting for bigger models. We solved it architecturally, by separating the part of the system that generates language from the part that retrieves and verifies facts, and reconciling the two before anything reaches the user. It runs on ordinary processors. It is cheap. And on the industry's own benchmark for this kind of faithfulness, it more than doubles the published baseline, with no fabricated facts in the verified case at all. The article asks whether scaling will, at tremendous cost, eventually reduce hallucinations — or even worse, if the largest companies in the world "are spending a fortune chasing a cure that is not coming." And last week Cringely pitched more advantages for their solution, noting that most prompts aren't even chatbot-level creative prompts — but just requests to retrieve simple data: The reason 2Brains doesn't lie and the reason it's cheap are the same reason. It looks the fact up instead of guessing it — so it cannot fabricate, and the lookup runs on a processor that sips power instead of a chip that gulps it. Trust and thrift are not a trade-off you balance against each other. They fall out of a single design decision. You do not pay extra for the honest version. The honest version is the cheap version. That sentence is the whole company.

Read more of this story at Slashdot.

CNBC reports: Waymo is recalling almost 3,900 robotaxis in the U.S. to fix software issues after some cars drove into freeway construction zones, according to notices filed with the National Highway Traffic Safety Administration. The voluntary recall, the Alphabet-owned company's second in just over a month, followed 13 known incidents where Waymo robotaxis drove into construction zones on freeways in Phoenix, or entered freeway lanes with active construction in the San Francisco area, the filings published Thursday said... A letter posted to the regulator's website... noted that, "Driving through a closed construction zone increases the risk of a crash..." [Waymo said in a statement emailed to CNBC] "We voluntarily restricted freeway operations last month while making improvements, proactively notified state and federal regulators, and decided to file a voluntary software recall with NHTSA. We continue to safely serve riders on surface streets in all the cities where we operate...." The company implemented another voluntary recall in May after some of its robotaxis had driven into flooded zones or standing water. The NHTSA Safety Board also initiated a probe of Waymo after a January incident in which a robotaxi illegally passed a stopped school bus.

Read more of this story at Slashdot.