Evidence, evaluation and experiments
How do we know an agent is any good?
Claims about agentic AI are easy to make and much harder to prove. This pillar focuses on the evidence behind those claims and ask a practical question: what can the agent actually do, under what conditions, and compared with what baseline?
We examine peer-reviewed research, industry experiments, and first-party results (including the research behind REASONTOCHAIN) to understand what has actually been measured. Evaluation goes beyond accuracy to consider cost, latency, data and tool grounding, constraint handling, reliability, hallucination, coherence, and human auditability. We also examine the limitations behind the numbers: sample size, evaluation design, baselines, model versions, and potential sources of bias.
The objective is simple: build the measurement literacy needed to distinguish meaningful evidence from an impressive demo and to assess whether an agent is ready for real supply chain work.
Behind the Vendor Claims
Coming soon!
Enterprise software vendors are rapidly introducing AI agents into core business processes, but detailed public evaluations remain surprisingly scarce.
This article investigates what vendors actually disclose about agent performance, which evidence is missing, how published claims should be interpreted, and what enterprise buyers should demand before trusting agents with consequential decisions.
The Ultimate Supply Chain Benchmark
Coming soon!
Today’s benchmarks test pieces of the problem, reasoning, tool use, ERP execution, autonomy and economic outcomes. But what would it take to evaluate an agent as a real supply chain decision-maker?
This article defines the requirements for a decision-grade benchmark that tests not just capability, but reliability, authority, governance and business value under realistic operating conditions.
Can AI Agents Run a Supply Chain? What the Benchmarks Tell Us and What They Don't
AI agents can already perform meaningful parts of supply chain work, but can they actually run the system?
This article reviews leading public real-world, enterprise and supply chain benchmarks to separate demonstrated capability from hype, using the REASONTOCHAIN model of Agency, Autonomy, Capability, Authority, Governance and Value.
The evidence shows a clear frontier: agents are becoming capable operators in bounded environments, but reliable end-to-end supply chain ownership remains unproven.