how to eval performance?

[linxid]

This is a amazing work. What evaluation tool do you use, different benchmark use differ tool or use evalscope or ms-swift. We use evalscope can not reproduce performance. Can you share eval script.

[1GaryLi]

Thank you for your interest in our work! We are basically using the corresponding official codebase to evaluate different benchmarks. We have already shared the evaluation code for Tau²-Bench, using the official implementation here:
https://github.com/sierra-research/tau2-bench

For OfficeBench, our setup differs slightly from the official codebase: we add a “think” step before each tool call and run the environment locally instead of via Docker. We will share the evaluation scripts for OfficeBench and AgentBench in the next few days.

Please also let me know if you have any other questions in the meantime.