Feat/meta/sn by carpawell · Pull Request #3991 · nspcc-dev/neofs-node

carpawell · 2026-05-14T16:33:54Z

No description provided.

codecov · 2026-05-26T19:36:31Z

Codecov Report

❌ Patch coverage is 17.58794% with 492 lines in your changes missing coverage. Please review.
✅ Project coverage is 27.09%. Comparing base (6241c4c) to head (19291f2).

Files with missing lines	Patch %	Lines
cmd/neofs-node/meta.go	5.63%	134 Missing ⚠️
pkg/services/meta/meta.go	25.43%	81 Missing and 4 partials ⚠️
pkg/services/sidechain/sidechain.go	0.00%	80 Missing ⚠️
pkg/services/object/put/distributed.go	14.03%	48 Missing and 1 partial ⚠️
pkg/services/meta/blocks.go	0.00%	36 Missing ⚠️
pkg/core/object/replicate.go	13.15%	33 Missing ⚠️
pkg/services/meta/storage.go	7.69%	24 Missing ⚠️
pkg/services/object/put/streamer.go	31.81%	14 Missing and 1 partial ⚠️
pkg/services/meta/notifications.go	0.00%	14 Missing ⚠️
pkg/network/address.go	0.00%	8 Missing ⚠️
... and 6 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3991      +/-   ##
==========================================
- Coverage   27.79%   27.09%   -0.71%     
==========================================
  Files         681      683       +2     
  Lines       46797    46280     -517     
==========================================
- Hits        13006    12538     -468     
- Misses      32553    32564      +11     
+ Partials     1238     1178      -60

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

cthulhu-rider

not finished

cthulhu-rider · 2026-05-27T14:54:22Z

+
+func (m *Meta) blockHandler(ctx context.Context, buff <-chan *block.Header) {
+	for {
+		if len(buff) >= blockBuffSize-1 {


doubt it can be >

btw is it possible to do by channel writer(s). Would be more natural to me

doubt it can be >

why? we read an item, do some magic that may take time, and then check again what size we have now, a new item may have come

btw is it possible to do by channel writer(s). Would be more natural to me

writer does not belong to this repo, it is sent to neo-go code

and then check again

buff can initially have blockBuffSize-1 which is a capacity. This does not mean queue is full. Wouldnt be a big problem, but still

but there is len, not cap. not sure i understand you

I don't think this check is useful, you can't separate overflow from normally filled queue on the reader side, only writer can do this on attempt to write.

cthulhu-rider

finished. Lets fix everything and merge

@AnnaShaleva may u pls check chain configuration?

AnnaShaleva · 2026-06-02T15:30:16Z

 	// notification work; it is required to always read notifications without
 	// any blocking or making additional RPC.
-	notificationBuffSize = 100
+	notificationBuffSize = 10000


Is it approved by @roman-khimov?

this is an address + pointer buff channel, what cons may we have? ~703KB of memory

i can change it to any value. do you have a suggestion? i just had an experience with deadlock with neo-go subscription, so this is why this big value was chosen

That's why I'm asking. Increasing buffer size is not a solution to the deadlock problem. It will make the problem more hidden, that's it.

If there's a deadlock, it should be fixed in some other (explicit) way.

Increasing buffer size is not a solution to the deadlock problem

we are balancing b/w notification handling speed and storage speed. i guess it is another thing for exploring in load testing (yet not answered)

AnnaShaleva · 2026-06-02T15:41:28Z

+
+// Height returns current side chaih block height.
+func (s *SideChain) Height() uint32 {
+	return s.core.HeaderHeight()


Header height differs from block height. What exactly do you need here?

Header height differs from block height

in which way? you mean a 1-block difference in values? i need it for async, but the same VUB calculation

you mean a 1-block difference in values?

Not only. It can be large difference if, e.g. the node is in the process of syncing or stale. Headers synchronisation is a separate process, different from blocks synchronisation.

the only thing i can suggest is to block any operation before the node is in sync

The decision is up to you, I'm just warning. Either way, you should either adjust the documentation and method name or adjust the return value.

From what I see it's used in meta signing only where header is more appropriate as "the last known current height network it operating with", suggest renaming the method or fixing the comment.

AnnaShaleva

Tests are failing. Other than that looks legit as a prototype.

AnnaShaleva · 2026-06-04T15:51:48Z

+				config.HFCockatrice.String():    0,
+				config.HFDomovoi.String():       0,
+				config.HFEchidna.String():       0,
+				config.HFFaun.String():          0,


Add a compatibility unit-test: check that hardforks starting from Aspid ending with config.HFLatestStable are enabled in the resulting chainCfg. On subsequent NeoGo update if latest stable hardfork is changed, you'll know about it.

AnnaShaleva · 2026-06-04T15:53:32Z

+					configneogo.HFCockatrice.String():    0,
+					configneogo.HFDomovoi.String():       0,
+					configneogo.HFEchidna.String():       0,
+					configneogo.HFFaun.String():          0,


Ditto for this config.

AnnaShaleva · 2026-06-04T15:56:58Z

@@ -0,0 +1,21 @@
+package util


util is always a bad name, think about a better one. CC @roman-khimov

this is not the package i created, i just extended it. but agree, moved to another network package

carpawell · 2026-06-04T16:29:44Z

Tests are failing. Other than that looks legit as a prototype.

They should be adopted, yes. And there are other problems. Config check and changelog are not accepted to the experimental things.

cthulhu-rider

seems good to me overall

10_000_000 meant nothing, GAS factor is 1_0000_0000. Also, new value means something at least. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

It is available starting from v0.119.0. This should speed broadcast up, which is critical for metadata chain. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

Just drop the old one, use new instead, no functional changed in the new one. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

roman-khimov · 2026-06-10T13:51:48Z

+				TimePerBlock: 50 * time.Millisecond,
+			},
+			Magic:                           fsChainProtocol.Network + 1,
+			MemPoolSize:                     0,


roman-khimov · 2026-06-11T14:43:02Z

+
+func (m *Meta) blockHandler(ctx context.Context, buff <-chan *block.Header) {
+	for {
+		if len(buff) >= blockBuffSize-1 {


I don't think this check is useful, you can't separate overflow from normally filled queue on the reader side, only writer can do this on attempt to write.

roman-khimov · 2026-06-11T14:43:41Z

+
+func (m *Meta) notificationHandler(ctx context.Context, buff <-chan *state.ContainedNotificationEvent) {
+	for {
+		if len(buff) >= notificationBuffSize-1 {


roman-khimov · 2026-06-11T15:11:52Z

+	on.m.Unlock()
+
+	if ok {
+		on.metaSvc.taskQueue <- storageTask{addr: addr, o: &sub.objHeader}


Does it make any difference? Local HEAD should be pretty fast.

roman-khimov · 2026-06-11T15:14:12Z

+	wg.Go(func() { m.storager(ctx, m.taskQueue) })
+
+	m.chain.SubscribeForBlocks(m.bCh)
+	m.chain.SubscribeForNotifications(m.evsCh)


Seems like this can be done via JSON-RPC as well. Can we have this mode (non-local chain) for nodes with IR nearby?

roman-khimov · 2026-06-11T15:22:07Z

+
+// PutObject forces [Meta] to index provided object.
+func (m *Meta) PutObject(o *object.Object) error {
+	return m.metabase.Put(o)


Per-object transaction?

roman-khimov · 2026-06-11T15:25:05Z

 	if len(pubKeyBytes) != compressedECDSAPublicKeyLen {
 		panic(fmt.Sprintf("wrong public key len: expected %d, got %d", compressedECDSAPublicKeyLen, len(pubKeyBytes)))
 	}
-


roman-khimov · 2026-06-11T15:27:38Z

+
+// Height returns current side chaih block height.
+func (s *SideChain) Height() uint32 {
+	return s.core.HeaderHeight()


From what I see it's used in meta signing only where header is more appropriate as "the last known current height network it operating with", suggest renaming the method or fixing the comment.

roman-khimov · 2026-06-11T15:31:50Z

 	}))
 }

+func metadataChainHardforks() map[string]uint32 {


Should follow FS chain HF config (heights can be different, but just enabling it like this can be problematic as well). At least for 0-height ones.

carpawell mentioned this pull request May 15, 2026

ir: deploy IR nodes with new metadata chain nspcc-dev/neofs-testcases#1367

Merged

carpawell force-pushed the feat/meta/sn branch from 10a909b to 12a8fe6 Compare May 21, 2026 21:37

carpawell marked this pull request as ready for review May 21, 2026 21:37

carpawell requested review from End-rey, cthulhu-rider and roman-khimov as code owners May 21, 2026 21:37

carpawell force-pushed the feat/meta/sn branch 2 times, most recently from d26ab18 to f9b5259 Compare May 26, 2026 19:32

cthulhu-rider reviewed May 27, 2026

View reviewed changes

Comment thread pkg/services/meta_new/meta.go Outdated

Comment thread pkg/services/meta_new/meta.go Outdated

Comment thread pkg/services/meta_new/blocks.go Outdated

Comment thread pkg/services/meta_new/blocks.go Outdated

cthulhu-rider reviewed Jun 2, 2026

View reviewed changes

AnnaShaleva reviewed Jun 2, 2026

View reviewed changes

carpawell mentioned this pull request Jun 3, 2026

Find optimal metadata chain parameters #4009

Open

carpawell force-pushed the feat/meta/sn branch from f9b5259 to bf49760 Compare June 3, 2026 19:49

carpawell requested review from AnnaShaleva and cthulhu-rider June 3, 2026 19:50

carpawell force-pushed the feat/meta/sn branch from bf49760 to d0d5071 Compare June 3, 2026 19:51

AnnaShaleva reviewed Jun 4, 2026

View reviewed changes

carpawell force-pushed the feat/meta/sn branch from d0d5071 to f84dcfa Compare June 4, 2026 16:28

carpawell force-pushed the feat/meta/sn branch from f84dcfa to 18d4a4f Compare June 4, 2026 16:30

carpawell requested a review from AnnaShaleva June 4, 2026 16:30

carpawell mentioned this pull request Jun 4, 2026

Adapt tests to changed metaconfig nspcc-dev/neofs-testcases#1373

Open

carpawell force-pushed the feat/meta/sn branch from 18d4a4f to c4414b6 Compare June 8, 2026 07:35

cthulhu-rider approved these changes Jun 8, 2026

View reviewed changes

carpawell added 4 commits June 11, 2026 13:13

meta_test: change test network fee

0936c45

10_000_000 meant nothing, GAS factor is 1_0000_0000. Also, new value means something at least. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

sn: support new meta flow

df3624f

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

meta: add service metrics

fd62813

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

meta: optimize signature verifying

500e003

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

carpawell added 3 commits June 11, 2026 13:31

[ir,sn]/meta: enable neo-go's DirectRelay application configuration

658e41d

It is available starting from v0.119.0. This should speed broadcast up, which is critical for metadata chain. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

[ir,sn]/meta: enable every Neo hardfork in config explicitly

25dd3f1

Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

sn: eliminate old meta service

19291f2

Just drop the old one, use new instead, no functional changed in the new one. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>

carpawell force-pushed the feat/meta/sn branch from c4414b6 to 19291f2 Compare June 11, 2026 10:39

roman-khimov reviewed Jun 11, 2026

View reviewed changes

Conversation

carpawell commented May 14, 2026

Uh oh!

codecov Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cthulhu-rider left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cthulhu-rider left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnnaShaleva Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AnnaShaleva left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

codecov Bot commented May 26, 2026 •

edited

Loading

AnnaShaleva Jun 4, 2026 •

edited

Loading