indexer模块

blockchain/indexers 模块

该模块分为两个部分,一个是根据TxHash找Tx,另一个是根据地址找Tx. 主要是针对常用操作建立索引,加快信息查找.

索引管理

索引管理主要是为了提供统一的接口,让外部可以方便的管理地址索引和Tx的索引.
对外暴露接口

// NewManager returns a new index manager with the provided indexes enabled.
//
// The manager returned satisfies the blockchain.IndexManager interface and thus
// cleanly plugs into the normal blockchain processing path.
func NewManager(db database.DB, enabledIndexes []Indexer) *Manager
// Init initializes the enabled indexes.  This is called during chain
// initialization and primarily consists of catching up all indexes to the
// current best chain tip.  This is necessary since each index can be disabled
// and re-enabled at any time and attempting to catch-up indexes at the same
// time new blocks are being downloaded would lead to an overall longer time to
// catch up due to the I/O contention.
//
// This is part of the blockchain.IndexManager interface.
func (m *Manager) Init(chain *blockchain.BlockChain) error
// ConnectBlock must be invoked when a block is extending the main chain.  It
// keeps track of the state of each index it is managing, performs some sanity
// checks, and invokes each indexer.
//
// This is part of the blockchain.IndexManager interface.
func (m *Manager) ConnectBlock(dbTx database.Tx, block *btcutil.Block, view *blockchain.UtxoViewpoint) error
// DisconnectBlock must be invoked when a block is being disconnected from the
// end of the main chain.  It keeps track of the state of each index it is
// managing, performs some sanity checks, and invokes each indexer to remove
// the index entries associated with the block.
//
// This is part of the blockchain.IndexManager interface.
func (m *Manager) DisconnectBlock(dbTx database.Tx, block *btcutil.Block, view *blockchain.UtxoViewpoint) error

索引管理数据库

对应唯一的bucket idxtips
里面存着索引处理到哪一块了,有哪些索引,每个索引处理到第几块了等信息.

idxtips数据库中的信息

索引名 内容
txHash索引( txbyhashidx) 最新 blockHash+blockHeight
通过地址查找Tx( txbyaddridx) 最新 blockHash+blockHeight
// Indexer provides a generic interface for an indexer that is managed by an
// index manager such as the Manager type provided by this package.
type Indexer interface {
	// Key returns the key of the index as a byte slice.
	Key() []byte //这个就是给manager管理用的.

	// Name returns the human-readable name of the index.
	Name() string //给人看的

	// Create is invoked when the indexer manager determines the index needs
	// to be created for the first time.
	Create(dbTx database.Tx) error //初始化数据库

	// Init is invoked when the index manager is first initializing the
	// index.  This differs from the Create method in that it is called on
	// every load, including the case the index was just created. //初始化Indexer
	Init() error 

	// ConnectBlock is invoked when the index manager is notified that a new
	// block has been connected to the main chain. 来了一个新块
	ConnectBlock(dbTx database.Tx, block *btcutil.Block, view *blockchain.UtxoViewpoint) error

	// DisconnectBlock is invoked when the index manager is notified that a
	// block has been disconnected from the main chain. 发生了分叉, 移除一个块
	DisconnectBlock(dbTx database.Tx, block *btcutil.Block, view *blockchain.UtxoViewpoint) error
}

TxHash 找Tx

主要是处理ConnectBlock,每一个新块来以后,为相关的Tx建立索引.
DisconnectBlock,发生分叉需要移除一个block的时候,这个block中的索引也都删掉.

数据库

  1. bucket hashbyididx
    blockNumber->blockHash
    这个Bucket中存的是块号到块hash

  2. bucket idbyhashidx
    blockHash->blockNumber
    从blockhash到块号的映射

  3. bucket txbyhashidx
    TxHash->Tx索引 映射
    Tx索引结构如下:

| blockNumber | Tx在该块内字节偏移 | Tx长度 |
| ----------- | ------------------ | ------ |
| 4 | 4 | 4 |
这里说的偏移,指的是block序列化以后的偏移,Tx长度指的也是Tx序列化以后的字节长度

addr找Tx

主要是处理地址到交易的映射,这里说的Tx实际上是Tx索引,也就是上一节中说的blockNumber+TxStart+TxLength
因为addr可能会对应成千上万的交易,所以这部分索引的管理与维护是比较考验技巧的.
他用的是LSM Tree,需要以后研究一下.

数据库

bucket txbyaddridx

可以认为这个数据库中存的是一个地址关联的所有交易的集合.实际上就是把Tx按照地址进行分组整理.
[addr]-->[tx,tx,...] 并且tx是有序的,按照发生时间(也就是block number)进行了排序.

这里的存储思路需要好好研究一下

cfIndex

// Committed filters come in one flavor currently: basic. They are generated
// and dropped in pairs, and both are indexed by a block's hash. Besides
// holding different content, they also live in different buckets.
这里存储的是针对某个block建立的类似bloom过滤器一样的索引. 只不过gcs索引数据更小.
这里的索引针对的是block中消费的pkscript以及生成的pkscript

1. bucket

总bucket是cfIndexParentBucketKey
下面还有三个子bucket,分别是

cf0byhashidx cfIndexKeys

// cfIndexKeys is an array of db bucket names used to house indexes of
// block hashes to cfilters.

blockhash-->所有相关pkscript的索引

cf0headerbyhashidx cfHeaderKeys

// cfHeaderKeys is an array of db bucket names used to house indexes of
// block hashes to cf headers.

blockhash-->hash(cf0hashbyhashidx中的hash,上一块header的hash)

cf0hashbyhashidx cfHashKeys

// cfHashKeys is an array of db bucket names used to house indexes of
// block hashes to cf hashes.

blockhash-->所有相关pkscript的索引的hash值