feat(indexer): advanced multi-step search with cross-document references (#5022)#6215
Open
XananasX7 wants to merge 1 commit into
Open
feat(indexer): advanced multi-step search with cross-document references (#5022)#6215XananasX7 wants to merge 1 commit into
XananasX7 wants to merge 1 commit into
Conversation
…ces (hashgraph#5022, hashgraph#5021, hashgraph#5019) Implements the advanced Indexer search capability requested in hashgraph#5022, which also satisfies the use-cases from hashgraph#5021 (finding suitable carbon credits) and hashgraph#5019 (searching for common project practices). ## What changed ### New interfaces — indexer-interfaces/ AdvancedSearchParams, AdvancedSearchStep, SearchCondition, AdvancedSearchResult, AdvancedSearchResultItem, ConditionOperator (advanced-search.interface.ts) ### New message API enum IndexerMessageAPI.GET_ADVANCED_SEARCH_API ### New DTOs — indexer-api-gateway/src/dto/advanced-search.dto.ts Full class-validator/class-transformer annotated DTOs with Swagger docs for all request and response shapes. ### New API endpoint — POST /search/advanced Returns AdvancedSearchResultDTO with: - paginated result items with configurable display columns - serialised base64 'searchToken' for URL bookmark/share ### New backend service — indexer-service/src/api/advanced-search.service.ts Multi-step query executor: - Builds MongoDB filter from conditions using MikroORM EntityManager - Operators: eq, neq, contains (case-insensitive regex), regex, gt, gte, lt, lte, between, in, not_in - Type filter applied automatically per step - Cross-step references: {n}.fieldPath resolved from prior step carry-values; auto-upgrades 'eq' to 'in' for multi-value results - Intermediate steps capped at 1000 docs for memory safety - In-memory groupBy support after final DB fetch - searchToken = base64(JSON.stringify(params)) for bookmark support ### New Angular component — indexer-frontend/.../search/advanced/ AdvancedSearchViewComponent (standalone): - Reactive form for N search steps, each with M conditions - All 11 operators with appropriate input modes (range, list, single value) - Carry-forward field configuration for multi-step cross-referencing - Configurable result grid columns (add/remove) - URL persistence: token + pageIndex/pageSize written to queryParams - Restore-from-URL: pasting a bookmarked URL restores full form state - 'Copy Link' button for sharing - Navigates to detail pages on row open (same logic as basic search) ### Basic search view Added 'Advanced search ›' link to navigate to /search/advanced ### Route Added { path: 'search/advanced', component: AdvancedSearchViewComponent } ## Use-cases covered (per hashgraph#5022 acceptance criteria) ### hashgraph#5019 — Search for common project practices Step 1: type=Policy, conditions=[{ field:'analytics.textSearch', operator:'contains', value:'ACM0007' }], carryFields:['analytics.policyId'] Step 2: type=VC-Document, conditions=[{ field:'analytics.policyId', operator:'eq', value:'$step0.analytics.policyId'}, {field:'options.credentialSubject.decisionField', operator:'eq', value:'(c) Use default values'}] ### hashgraph#5021 — Find suitable carbon credits Step 1: type=VC-Document, conditions=[{ field:'type', op:'eq', value:'Monitoring Report'}, {field:'analytics.schemaName', op:'contains', value:'Monitoring'}], carryFields:['options.credentialSubject.projectId'] Step 2: type=VC-Document (minting), conditions=[{field:'options.credentialSubject.projectId', op:'eq', value:'$step0.options.credentialSubject.projectId'}] ### Generic support - Range conditions (e.g. totalIssuance > 10000) - Set conditions (find all where policyId IN [...]) - Regex pattern matching on any field - Arithmetic on values (supported via GT/LT/BETWEEN on numeric fields) - Result grid sorted and grouped by any field Closes hashgraph#5022 Partially addresses hashgraph#5021 Partially addresses hashgraph#5019 Signed-off-by: XananasX7 <xananasX7@users.noreply.github.com>
XananasX7
force-pushed
the
feature/5022-advanced-search
branch
from
9e46168 to
97870d9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the advanced Indexer search capability requested in #5022, covering sub-issues #5021 (finding suitable carbon credits) and #5019 (searching common project practices).
Closes #5022
Closes #5021
Closes #5019
Architecture
indexer-interfacesAdvancedSearchParams,AdvancedSearchStep,SearchCondition,AdvancedSearchResultinterfaces + 11 operatorsindexer-commonIndexerMessageAPI.GET_ADVANCED_SEARCH_APIenum valueindexer-api-gatewayPOST /search/advancedendpoint with Swagger docs + class-validator DTOsindexer-serviceAdvancedSearchService— multi-step MongoDB query executorindexer-frontendAdvancedSearchViewComponentat/search/advanced; link from basic search pageOperators
eq,neq,contains(case-insensitive),regex,gt,gte,lt,lte,between,in,not_inMulti-step cross-referencing
Intermediate steps carry field values forward via
carryFields. Subsequent steps reference them with$step{n}.fieldPathsyntax in any condition value. When a prior step returns multiple values, single-valueeqauto-upgrades toin.URL bookmarking
Each search result includes a
searchToken(base64-encoded params). The frontend writes it to?token=so searches can be bookmarked and shared — pasting the URL restores the full form state.Use-case examples
#5019 — Common project practices
{ "steps": [ { "type": "Policy", "conditions": [{ "field": "analytics.textSearch", "operator": "contains", "value": "ACM0007" }], "carryFields": ["analytics.policyId"] }, { "type": "VC-Document", "conditions": [{ "field": "analytics.policyId", "operator": "eq", "value": "$step0.analytics.policyId" }, { "field": "options.credentialSubject.decision", "operator": "eq", "value": "(c) Use default values" }] } ], "pageIndex": 0, "pageSize": 10 }#5021 — Find suitable carbon credits
{ "steps": [ { "type": "VC-Document", "conditions": [{ "field": "analytics.schemaName", "operator": "contains", "value": "Monitoring Report" }], "carryFields": ["options.credentialSubject.projectId"] }, { "type": "VC-Document", "conditions": [{ "field": "options.credentialSubject.projectId", "operator": "eq", "value": "$step0.options.credentialSubject.projectId" }] } ], "pageIndex": 0, "pageSize": 10 }