What is your feature suggestion?
Please add a supported way to initialize ADK session state per eval case when running agents-cli eval generate.
In ADK's native eval schema, EvalCase.session_input.state can be used to initialize session state before inference. This is useful for agents whose behavior depends on session state, including agents that expose different tools from a BaseToolset.get_tools(readonly_context) implementation depending on the current state.
ref: https://adk.dev/tools-custom/#toolsets-grouping-and-dynamically-providing-tools
In agents-cli v1.0.0, the eval surface uses the Agent Platform EvaluationDataset / EvalCase schema. The migration docs explicitly drop session_input and say state initialization should move into agent code. That makes it difficult to evaluate state-dependent behavior case by case.
ref: https://google.github.io/agents-cli/reference/eval-dataset-migration/#single-turn-case
What will this enable you to do?
This would make agents-cli eval usable for ADK agents whose behavior depends on session state.
Concrete use cases:
- Evaluate agents that choose a different Toolset based on tenant, plan, locale, role, or feature flags stored in session state.
- Write multiple eval cases against the same prompt but with different initial state, e.g. free user vs paid user, read-only vs admin, region A vs region B.
- Avoid moving test-only state initialization into production agent code.
Additional context
It looks like the underlying Agent Platform SDK already has an experimental SessionInput type with state, and some inference paths pass that state through to ADK or Agent Engine sessions.
https://docs.cloud.google.com/gemini-enterprise-agent-platform/optimize/evaluation/agent-evaluation#process
What is your feature suggestion?
Please add a supported way to initialize ADK session state per eval case when running
agents-cli eval generate.In ADK's native eval schema,
EvalCase.session_input.statecan be used to initialize session state before inference. This is useful for agents whose behavior depends on session state, including agents that expose different tools from aBaseToolset.get_tools(readonly_context)implementation depending on the current state.ref: https://adk.dev/tools-custom/#toolsets-grouping-and-dynamically-providing-tools
In
agents-cliv1.0.0, the eval surface uses the Agent PlatformEvaluationDataset/EvalCaseschema. The migration docs explicitly dropsession_inputand say state initialization should move into agent code. That makes it difficult to evaluate state-dependent behavior case by case.ref: https://google.github.io/agents-cli/reference/eval-dataset-migration/#single-turn-case
What will this enable you to do?
This would make
agents-cli evalusable for ADK agents whose behavior depends on session state.Concrete use cases:
Additional context
It looks like the underlying Agent Platform SDK already has an experimental
SessionInputtype withstate, and some inference paths pass that state through to ADK or Agent Engine sessions.https://docs.cloud.google.com/gemini-enterprise-agent-platform/optimize/evaluation/agent-evaluation#process