❌

Normal view

AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents

arXiv:2510.21031v1 Announce Type: cross Abstract: The emergence of foundation models (FMs) has enabled the development of highly capable and autonomous agents, unlocking new application opportunities across a wide range of domains. Evaluating the architecture of agents is particularly important as the architectural decisions significantly impact the quality attributes of agents given their unique characteristics, including compound architecture, autonomous and non-deterministic behaviour, and continuous evolution. However, these traditional methods fall short in addressing the evaluation needs of agent architecture due to the unique characteristics of these agents. Therefore, in this paper, we present AgentArcEval, a novel agent architecture evaluation method designed specially to address the complexities of FM-based agent architecture and its evaluation. Moreover, we present a catalogue of agent-specific general scenarios, which serves as a guide for generating concrete scenarios to design and evaluate the agent architecture. We demonstrate the usefulness of AgentArcEval and the catalogue through a case study on the architecture evaluation of a real-world tax copilot, named Luna.

Early Screening and Subtype Identification of High-Risk Lung Nodules via Breathprint by Graphene eNose Platform: A Large Cohort Study

ACS Sens. 2025 Apr 25;10(4):3101-3111. doi: 10.1021/acssensors.5c00314. Epub 2025 Apr 7.

ABSTRACT

Early screening of individuals with high-risk lung nodules can significantly improve the prognosis of lung cancer patients, and accurate identification of lung nodule subtypes can provide guidance for medical treatment. Exhaled breath (EB) analysis via eNoses offers a quick and noninvasive approach, but current eNose technology lacks quality control and solid validation in large population studies. Herein, an eNose platform integrated with a metal ion-decorated graphene sensor array and a breath sampling accessory was established. EB samples from 427 healthy subjects and 2586 subjects with lung nodules, including various benign and malignant subtypes, were collected through the breath sampling accessory for quality control. The large-cohort clinical EB samples were analyzed by the eNose platform to acquire the cross-reactive resistance response. Breathprint analysis for high-risk lung nodules using SVM and age-matched training sets yielded strong and robust performance. Combined with baseline data, the model achieved an AUC of 0.93 (95% CI, 0.89-0.96) on the external test set, with 97% sensitivity and 73% specificity. Moreover, dimensionality reduction analysis of breathprints demonstrated separability across different lung nodule subtypes. This study demonstrates the reliability of the graphene eNose platform to identify high-risk lung nodules and classify lung nodule subtypes in a noninvasive and rapid method.

PMID:40193324 | DOI:10.1021/acssensors.5c00314

❌