explore more efficient approaches to inscription

[conduition]

Hey ordinal devs. There's a lot of debate going on in bitcoin core about whether to label inscription transactions as non-standard, thus preventing their propagation through the mempool.

My personal perspective is summed up in this article where I demonstrate that preventing or even identifying arbitrary data upload (i.e. inscribing) is actually impossible.

I'm hoping that this issue can serve as a concrete discussion of how to lessen the footprint of inscriptions on the bitcoin mempool, ideally in a way that meaningfully improves fee market conditions on the Bitcoin mainnet.

Some possible paths to explore:

• Analyze which types of inscriptions are taking up the most blockchain/mempool space today. This will tell us where effort is best spent.
• Dedicate specific magic byte markers for common content types. E.g. substitute \x12\x34 (2 bytes) in place of the content type header image/jpeg (10 bytes).
• (BRC-20) Use binary encoding. An inscription like this one [in text/utf8: {"p":"brc-20","op":"mint","tick":"iost","amt":"1000"}] could easily be compressed into just 5 or 6 bytes, instead of taking up 53 bytes of witness space. This would need to be done upstream in https://github.com/unisat-wallet/libbrc20-indexer and the spec
• (NFTs) Instead of uploading large blobs of data to the blockchain, store the blob on some other decentralized storage system, such as a torrent, IPFS hash, or Nostr note. Then commit only a reference to the blockchain.
• If the user insists on pushing data to the chain, use a compression scheme by default. This NFT inscription, for example, could have saved 4 kilobytes of witness space, 1000 vbytes, or enough for 5+ normal bitcoin transactions, just by wrapping its inner webp image in XZIP/GZIP.

[conduition]

Related: https://l1f.discourse.group/t/brc-20-optimization/33

[conduition]

The savings from content-type aliases would be nice, but they're nothing compared to what could be saved by optimizing the BRC20 inscription body format from JSON into a concise binary format. @casey if you know anyone who could help me action this, I'd be very grateful 🙏

[conduition]

• Analyze which types of inscriptions are taking up the most blockchain/mempool space today. This will tell us where effort is best spent.

I'm currently waiting on ord to finish indexing so i can run some analysis and get some hard numbers on which parts and kinds of inscriptions are impacting the blockchain most heavily.

[conduition]

Is it normal for this process to take 30 to 60 seconds to parse a single block? (built with --release)

[conduition]

Some results posted here: #2879 (reply in thread)

[conduition]

@casey @raphjaph I think this one is a no-brainer.

• Dedicate specific magic byte markers for common content types. E.g. substitute \x12\x34 (2 bytes) in place of the content type header image/jpeg (10 bytes).

Thoughts?

[conduition]

With pleasure 😄

One downside is that it would be an ecosystem wide change, if ord started doing it, other indexers would need to upgrade too.

Yes, updated indexers would be backwards-compatible, but old indexers would not recognize the new content type 'aliases'.

To avoid causing UTF8 decoding errors in old clients, you could use valid ASCII characters as the aliases, since you don't need many of them.

e.g:

Byte	ASCII Character	Content-Type Meaning
0x74	t	text/plain;charset=utf-8
0x6A	j	application/json
0x68	h	text/html

This way, older clients would still recognize inscriptions which use the new aliases, although they would appear to have weird one-character content-types.

[casey]

I'm not sure overloading the same field is a good idea, it might be better to go with a new field.

[conduition]

Sure, then older indexers would just see newer inscriptions as having no content type. That's still a valid inscription though IIRC.

I could prepare a PR for this into ord, the basics seem pretty straightforward. Do i need to update any docs first so we can agree on a format? Or should the code come first?

[casey]

I'm still not 100% sure that the juice is worth the squeeze here. Maybe open an issue, and we can let it marinate for a little bit?

[conduition]

#2944