feat(agent): add verification configuration for agents and update related components#3174
Open
Dallas98 wants to merge 1 commit into
Open
feat(agent): add verification configuration for agents and update related components#3174Dallas98 wants to merge 1 commit into
Dallas98 wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a layered “self-verification” capability for ReAct-style agents, making verification configurable per agent, persisting it in storage, emitting verification events through the observer stream, and rendering them in the frontend UI.
Changes:
- Add
AgentVerificationConfigand persist it via backend APIs + DB JSONB column. - Implement a
VerificationControllerand integrate step-level + final-answer verification intoCoreAgentexecution/streaming flow. - Update frontend types, stores, streaming handlers, and task window renderer to surface verification events and allow enabling/disabling verification.
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/sdk/core/agents/test_core_agent.py | Adds unit tests for final-answer verification policy behavior and LLM-verifier normalization. |
| test/sdk/core/agents/test_agent_model.py | Adds tests for default AgentVerificationConfig behavior and validation. |
| test/backend/agents/test_create_agent_info.py | Extends backend test stubs/expectations to include verification_config. |
| sdk/nexent/core/utils/observer.py | Adds new ProcessType.VERIFICATION and maps it to a transformer. |
| sdk/nexent/core/agents/verification.py | New verification controller implementation (deterministic checks + optional LLM verifier). |
| sdk/nexent/core/agents/nexent_agent.py | Passes verification_config into CoreAgent during agent creation. |
| sdk/nexent/core/agents/core_agent.py | Integrates step pre/post verification and final-answer verification/repair loop. |
| sdk/nexent/core/agents/agent_model.py | Introduces AgentVerificationConfig and wires it into AgentConfig. |
| k8s/helm/nexent/charts/nexent-common/files/init.sql | Adds verification_config column to Helm init SQL. |
| frontend/types/chat.ts | Adds VERIFICATION to step content message type union. |
| frontend/types/agentConfig.ts | Defines AgentVerificationConfig TS type + default config; adds to Agent + update payload shape. |
| frontend/stores/agentConfigStore.ts | Tracks and dirty-checks verification_config, initializes defaults. |
| frontend/services/agentVersionService.ts | Adds verification_config field to agent version service types. |
| frontend/services/agentConfigService.ts | Includes verification_config in update payload and search response mapping. |
| frontend/public/locales/zh/common.json | Adds i18n strings for verification phases + agent verification toggle label. |
| frontend/public/locales/en/common.json | Adds i18n strings for verification phases + agent verification toggle label. |
| frontend/lib/chatMessageExtractor.ts | Refactors step creation helpers; adds extraction support for VERIFICATION messages. |
| frontend/hooks/agent/useSaveGuard.ts | Includes verification_config in guarded save/update payload. |
| frontend/const/chatConfig.ts | Registers VERIFICATION message/content type constants. |
| frontend/app/[locale]/chat/streaming/taskWindow.tsx | Renders verification events (icon + label + score) in the task window. |
| frontend/app/[locale]/chat/streaming/chatStreamHandler.tsx | Streams VERIFICATION events into the current step contents. |
| frontend/app/[locale]/agents/components/agentInfo/AgentGenerateDetail.tsx | Adds UI control to enable/disable verification for an agent. |
| docker/sql/v2.2.1_0601_add_agent_verification_config.sql | Adds migration to add verification_config column. |
| docker/init.sql | Adds verification_config column to Docker init SQL. |
| backend/utils/context_utils.py | Updates context skeleton instructions to describe verification behavior. |
| backend/services/agent_service.py | Persists/exports/imports verification_config via agent service flows. |
| backend/prompts/manager_system_prompt_template_zh.yaml | Adds verification instructions + verifier prompt section (manager, zh). |
| backend/prompts/manager_system_prompt_template_en.yaml | Adds verification instructions + verifier prompt section (manager, en). |
| backend/prompts/managed_system_prompt_template_zh.yaml | Adds verification instructions + verifier prompt section (managed, zh). |
| backend/prompts/managed_system_prompt_template_en.yaml | Adds verification instructions + verifier prompt section (managed, en). |
| backend/database/db_models.py | Adds verification_config JSONB column to ORM model. |
| backend/database/agent_db.py | Sets/returns verification_config in agent create/read mapping. |
| backend/consts/model.py | Adds verification_config to request/export models and normalizes it via Pydantic validation. |
| backend/agents/create_agent_info.py | Constructs SDK AgentConfig with validated verification_config. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+285
to
+290
| VerificationCheck( | ||
| name="observation_present", | ||
| passed=not self._EMPTY_RE.match(observation_text), | ||
| reason="" if observation_text.strip() else "The action produced no visible observation.", | ||
| fix_hint="Retry with better parameters, inspect tool errors, or explain that evidence is unavailable.", | ||
| ), |
Comment on lines
+79
to
+86
| VerificationEvent = Literal[ | ||
| "tool_precheck", | ||
| "tool_result", | ||
| "retrieval", | ||
| "code_execution", | ||
| "handoff", | ||
| "final_answer", | ||
| ] |
Comment on lines
+1193
to
+1212
| const fallbackReason = (() => { | ||
| if (data.event === "tool_precheck") { | ||
| return "动作非空、参数和语法已检查"; | ||
| } | ||
| if (data.event === "retrieval") { | ||
| return "检索结果和错误信号已检查"; | ||
| } | ||
| if (data.event === "handoff") { | ||
| return "子任务返回内容已检查"; | ||
| } | ||
| if (data.event === "tool_result" || data.event === "code_execution") { | ||
| return "执行结果非空，未发现未处理错误"; | ||
| } | ||
| if (data.event === "final_answer") { | ||
| return phase === "final_pass" | ||
| ? "答案完整、格式正常，未发现未处理错误" | ||
| : "答案非空、无内部标记、无占位符"; | ||
| } | ||
| return "未发现阻断问题"; | ||
| })(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.