Public commit record
45/100 · Thin
What the developer wrote
Authored by Kim
Dockerfile: update go base image
✓ Descriptive subject✓ Names a concrete action or component! No meaningful explanatory body
What changed, and why it matters
This commit is a large repository import or rebase that adds the entire btcd codebase plus new GitHub templates, CI workflows, a Makefile, and Dockerfiles. The stated purpose is 'Dockerfile: update go base image'. The actual Dockerfile change moves the build base image from a pinned SHA256 digest of golang:1.23.12-alpine3.21 to a tag-based golang:1.22.11-alpine3.21 in the GitHub Actions Dockerfile. There is no direct code-level security fix visible in the diff; the security relevance is limited to supply-chain/dependency hygiene of the Go base image used in Docker builds.
Recommended action
Verify whether the Go 1.22.11 base image is intentional and whether it addresses a specific known vulnerability. Prefer pinning the base image by SHA256 digest in .github/workflows/Dockerfile to reduce supply-chain risk. Review the full repository state for any additional security-relevant changes not captured in the supplied diff.
Security signals we found
01
Docker base image changed from SHA256-pinned golang:1.23.12-alpine3.21 to tag-based golang:1.22.11-alpine3.21 in .github/workflows/Dockerfile
02
No application code or consensus-critical changes visible in the diff
03
No CVE, advisory, researcher attribution, or vendor security disclosure present in commit or supplied references
04
Large repository import masks the actual Dockerfile change
Evidence from the diff
The commit imports the full btcd tree (3405 files, +200174 lines) and adds several repository-level files. The only Dockerfile change shown is in .github/workflows/Dockerfile, which switches the build stage from a SHA256-pinned golang:1.23.12-alpine3.21 image to a tag-based golang:1.22.11-alpine3.21 image. The top-level Dockerfile remains pinned by SHA256. No application code changes are present in the supplied diff. The commit does not describe a CVE, does not credit a researcher, and does not acknowledge a specific vulnerability. The base image downgrade from Go 1.23.12 to Go 1.22.11 is unusual for a ‘security update’ and could introduce rather than remove known Go runtime issues, but absent a disclosed advisory this is speculative.
Changed components
.github/workflows/DockerfileDocker build pipeline for btcdInspect captured patch +200174 / −0
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
new file mode 100644
index 0000000..709f0bf
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -0,0 +1,29 @@
+---
+name: Bug report
+about: Create a bug report. Please use the discussions section for general or troubleshooting questions.
+title: '[bug]: '
+labels: ["bug", "needs triage"]
+assignees: ''
+---
+
+### Background
+
+Describe your issue here.
+
+### Your environment
+
+* version of `btcd`
+* which operating system (`uname -a` on *Nix)
+* any other relevant environment details
+
+### Steps to reproduce
+
+Tell us how to reproduce this issue. Please provide stacktraces and links to code in question.
+
+### Expected behaviour
+
+Tell us what should happen
+
+### Actual behaviour
+
+Tell us what happens instead
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
new file mode 100644
index 0000000..8d32770
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -0,0 +1,11 @@
+blank_issues_enabled: false
+contact_links:
+ - name: Discussions
+ url: https://github.com/btcsuite/btcd/discussions
+ about: For general or troubleshooting questions or if you're not sure what issue type to pick.
+ - name: Community Slack
+ url: https://lightning.engineering/slack.html
+ about: Please ask and answer questions here.
+ - name: Security issue disclosure policy
+ url: https://github.com/lightningnetwork/lnd#security
+ about: Please refer to this document when reporting security related issues.
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
new file mode 100644
index 0000000..c96ee0a
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,19 @@
+---
+name: Feature request
+about: Suggest a new feature for `btcd`.
+title: '[feature]: '
+labels: enhancement
+assignees: ''
+---
+
+**Is your feature request related to a problem? Please describe.**
+<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
+
+**Describe the solution you'd like**
+<!-- A clear and concise description of what you want to happen. -->
+
+**Describe alternatives you've considered**
+<!-- A clear and concise description of any alternative solutions or features you've considered. -->
+
+**Additional context**
+<!-- Add any other context or screenshots about the feature request here. -->
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
new file mode 100644
index 0000000..dc1639a
--- /dev/null
+++ b/.github/pull_request_template.md
@@ -0,0 +1,19 @@
+## Change Description
+Description of change / link to associated issue.
+
+## Steps to Test
+Steps for reviewers to follow to test the change.
+
+## Pull Request Checklist
+### Testing
+- [ ] Your PR passes all CI checks.
+- [ ] Tests covering the positive and negative (error paths) are included.
+- [ ] Bug fixes contain tests triggering the bug to prevent regressions.
+
+### Code Style and Documentation
+- [ ] The change is not [insubstantial](https://github.com/btcsuite/btcd/blob/master/docs/code_contribution_guidelines.md#substantial-contributions-only). Typo fixes are not accepted to fight bot spam.
+- [ ] The change obeys the [Code Documentation and Commenting](https://github.com/btcsuite/btcd/blob/master/docs/code_contribution_guidelines.md#code-documentation-and-commenting) guidelines, and lines wrap at 80.
+- [ ] Commits follow the [Ideal Git Commit Structure](https://github.com/btcsuite/btcd/blob/master/docs/code_contribution_guidelines.md#model-git-commit-messages).
+- [ ] Any new logging statements use an appropriate subsystem and logging level.
+
+📝 Please see our [Contribution Guidelines](https://github.com/btcsuite/btcd/blob/master/docs/code_contribution_guidelines.md) for further guidance.
diff --git a/.github/workflows/Dockerfile b/.github/workflows/Dockerfile
new file mode 100644
index 0000000..a23b462
--- /dev/null
+++ b/.github/workflows/Dockerfile
@@ -0,0 +1,24 @@
+# GitHub action dockerfile
+# Requires docker experimental features as buildx and BuildKit so not suitable for developers regular use.
+# https://docs.docker.com/develop/develop-images/build_enhancements/#to-enable-buildkit-builds
+
+###########################
+# Build binaries stage
+###########################
+FROM --platform=$BUILDPLATFORM golang:1.22.11-alpine3.21 AS build
+ADD . /app
+WORKDIR /app
+# Arguments required to build binaries targetting the correct OS and CPU architectures
+ARG TARGETOS TARGETARCH
+# Actually building the binaries
+RUN GOOS=$TARGETOS GOARCH=$TARGETARCH go install -v . ./cmd/...
+
+###########################
+# Build docker image stage
+###########################
+FROM alpine:3.15
+COPY --from=build /go/bin /bin
+# 8333 Mainnet Bitcoin peer-to-peer port
+# 8334 Mainet RPC port
+EXPOSE 8333 8334
+ENTRYPOINT ["btcd"]
diff --git a/.github/workflows/dimagespub.yml b/.github/workflows/dimagespub.yml
new file mode 100644
index 0000000..d1e6355
--- /dev/null
+++ b/.github/workflows/dimagespub.yml
@@ -0,0 +1,52 @@
+name: Docker images build and publish
+
+on:
+ push:
+ tags:
+ - v*
+ # Allows you to run this workflow manually from the Actions tab
+ workflow_dispatch:
+
+env:
+ REGISTRY: ghcr.io
+ IMAGE_NAME: ${{ github.repository }}
+ # Build for default OS, linux, and common CPU architectures
+ # Reference https://github.com/docker/setup-buildx-action#quick-start
+ TPLATFORMS: linux/amd64,linux/arm64,linux/arm,linux/386
+
+jobs:
+ build-push:
+ runs-on: ubuntu-latest
+ permissions:
+ contents: read
+ packages: write
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+
+ - name: Docker Setup Buildx
+ id: buildx
+ uses: docker/setup-buildx-action@94ab11c41e45d028884a99163086648e898eed25
+
+ - name: Log in to the Container registry
+ uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+ with:
+ registry: ${{ env.REGISTRY }}
+ username: ${{ github.actor }}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Extract metadata (tags, labels) for Docker
+ id: meta
+ uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
+ with:
+ images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+
+ - name: Build and push Docker images
+ uses: docker/build-push-action@ac9327eae2b366085ac7f6a2d02df8aa8ead720a
+ with:
+ file: .github/workflows/Dockerfile
+ labels: ${{ steps.meta.outputs.labels }}
+ platforms: ${{ env.TPLATFORMS }}
+ push: true
+ tags: ${{ steps.meta.outputs.tags }}
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
new file mode 100644
index 0000000..50b4185
--- /dev/null
+++ b/.github/workflows/main.yml
@@ -0,0 +1,96 @@
+name: Build and Test
+on: [push, pull_request]
+
+env:
+ # go needs absolute directories, using the $HOME variable doesn't work here.
+ GOCACHE: /home/runner/work/go/pkg/build
+ GOPATH: /home/runner/work/go
+ GO_VERSION: 1.22.11
+
+jobs:
+ build:
+ name: Build
+ runs-on: ubuntu-latest
+ steps:
+ - name: Set up Go
+ uses: actions/setup-go@v5
+ with:
+ go-version: ${{ env.GO_VERSION }}
+
+ - name: Check out source
+ uses: actions/checkout@v4
+
+ - name: Build
+ run: make build
+
+ test-cover:
+ name: Unit coverage
+ runs-on: ubuntu-latest
+ steps:
+ - name: Set up Go
+ uses: actions/setup-go@v5
+ with:
+ go-version: ${{ env.GO_VERSION }}
+
+ - name: Check out source
+ uses: actions/checkout@v4
+
+ - name: Test
+ run: make unit-cover
+
+ - name: Send top-level coverage
+ uses: coverallsapp/github-action@v2
+ continue-on-error: true
+ with:
+ file: coverage.txt
+ flag-name: btcd
+ format: 'golang'
+ parallel: true
+
+ - name: Send btcec
+ uses: coverallsapp/github-action@v2
+ continue-on-error: true
+ with:
+ file: btcec/coverage.txt
+ flag-name: btcec
+ format: 'golang'
+ parallel: true
+
+ - name: Send btcutil coverage
+ uses: coverallsapp/github-action@v2
+ continue-on-error: true
+ with:
+ file: btcutil/coverage.txt
+ flag-name: btcutil
+ format: 'golang'
+ parallel: true
+
+ - name: Send btcutil coverage for psbt package
+ uses: coverallsapp/github-action@v2
+ continue-on-error: true
+ with:
+ file: btcutil/psbt/coverage.txt
+ flag-name: btcutilpsbt
+ format: 'golang'
+ parallel: true
+
+ - name: Notify coveralls all reports sent
+ uses: coverallsapp/github-action@v2
+ continue-on-error: true
+ with:
+ parallel-finished: true
+
+ test-race:
+ name: Unit race
+ runs-on: ubuntu-latest
+ steps:
+ - name: Set up Go
+ uses: actions/setup-go@v5
+ with:
+ go-version: ${{ env.GO_VERSION }}
+
+ - name: Check out source
+ uses: actions/checkout@v4
+
+ - name: Test
+ run: make unit-race
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..acfb8c4
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,60 @@
+# Temp files
+*~
+
+# Databases
+btcd.db
+*-shm
+*-wal
+
+# Log files
+*.log
+
+# Compiled Object files, Static and Dynamic libs (Shared Objects)
+*.o
+*.a
+*.so
+
+# Folders
+_obj
+_test
+vendor
+
+# Architecture specific extensions/prefixes
+*.[568vq]
+[568vq].out
+
+*.cgo1.go
+*.cgo2.c
+_cgo_defun.c
+_cgo_gotypes.go
+_cgo_export.*
+
+_testmain.go
+
+*.exe
+
+# Code coverage files
+profile.tmp
+profile.cov
+coverage.txt
+btcec/coverage.txt
+btcutil/coverage.txt
+btcutil/psbt/coverage.txt
+
+# vim
+*.swp
+*.swo
+/.vim
+
+#IDE
+.idea
+
+# Binaries produced by "make build"
+/addblock
+/btcctl
+/btcd
+/findcheckpoint
+/gencerts
+
+.DS_Store
+.aider*
diff --git a/CHANGES b/CHANGES
new file mode 100644
index 0000000..4e35922
--- /dev/null
+++ b/CHANGES
@@ -0,0 +1,1230 @@
+============================================================================
+User visible changes for btcd
+ A full-node bitcoin implementation written in Go
+============================================================================
+
+Changes in 0.22.0 (Tue Jun 01 2021)
+ - Protocol and network-related changes:
+ - Add support for witness tx and block in notfound msg (#1625)
+ - Add support for receiving sendaddrv2 messages from a peer (#1670)
+ - Fix bug in peer package causing last block height to go backwards
+ (#1606)
+ - Add chain parameters for connecting to the public Signet network
+ (#1692, #1718)
+ - Crypto changes:
+ - Fix bug causing panic due to bad R and S signature components in
+ btcec.RecoverCompact (#1691)
+ - Set the name (secp256k1) in the CurveParams of the S256 curve
+ (#1565)
+ - Notable developer-related package changes:
+ - Remove unknown block version warning in the blockchain package,
+ due to false positives triggered by AsicBoost (#1463)
+ - Add chaincfg.RegisterHDKeyID function to populate HD key ID pairs
+ (#1617)
+ - Add new method mining.AddWitnessCommitment to add the witness
+ commitment as an OP_RETURN output within the coinbase transaction.
+ (#1716)
+ - RPC changes:
+ - Support Batch JSON-RPC in rpcclient and server (#1583)
+ - Add rpcclient method to invoke getdescriptorinfo JSON-RPC command
+ (#1578)
+ - Update the rpcserver handler for validateaddress JSON-RPC command to
+ have parity with the bitcoind 0.20.0 interface (#1613)
+ - Add rpcclient method to invoke getblockfilter JSON-RPC command
+ (#1579)
+ - Add signmessagewithprivkey JSON-RPC command in rpcserver (#1585)
+ - Add rpcclient method to invoke importmulti JSON-RPC command (#1579)
+ - Add watchOnly argument in rpcclient method to invoke
+ listtransactions JSON-RPC command (#1628)
+ - Update btcjson.ListTransactionsResult for compatibility with Bitcoin
+ Core 0.20.0 (#1626)
+ - Support nullable optional JSON-RPC parameters (#1594)
+ - Add rpcclient and server method to invoke getnodeaddresses JSON-RPC
+ command (#1590)
+ - Add rpcclient methods to invoke PSBT JSON-RPC commands (#1596)
+ - Add rpcclient method to invoke listsinceblock with the
+ include_watchonly parameter enabled (#1451)
+ - Add rpcclient method to invoke deriveaddresses JSON-RPC command
+ (#1631)
+ - Add rpcclient method to invoke getblocktemplate JSON-RPC command
+ (#1629)
+ - Add rpcclient method to invoke getaddressinfo JSON-RPC command
+ (#1633)
+ - Add rpcclient method to invoke getwalletinfo JSON-RPC command
+ (#1638)
+ - Fix error message in rpcserver when an unknown RPC command is
+ encountered (#1695)
+ - Fix error message returned by estimatefee when the number of blocks
+ exceeds the max depth (#1678)
+ - Update btcjson.GetBlockChainInfoResult to include new fields in
+ Bitcoin Core (#1676)
+ - Add ExtraHeaders in rpcclient.ConnConfig struct (#1669)
+ - Fix bitcoind compatibility issue with the sendrawtransaction
+ JSON-RPC command (#1659)
+ - Add new JSON-RPC errors to btcjson package, and documented them
+ (#1648)
+ - Add rpcclient method to invoke createwallet JSON-RPC command
+ (#1650)
+ - Add rpcclient methods to invoke backupwallet, dumpwallet, loadwallet
+ and unloadwallet JSON-RPC commands (#1645)
+ - Fix unmarshalling error in getmininginfo JSON-RPC command, for valid
+ integers in scientific notation (#1644)
+ - Add rpcclient method to invoke gettxoutsetinfo JSON-RPC command
+ (#1641)
+ - Add rpcclient method to invoke signrawtransactionwithwallet JSON-RPC
+ command (#1642)
+ - Add txid to getblocktemplate response of rpcserver (#1639)
+ - Fix monetary unit used in createrawtransaction JSON-RPC command in
+ rpcserver (#1614)
+ - Add rawtx field to btcjson.GetBlockVerboseTxResult to provide
+ backwards compatibility with older versions of Bitcoin Core (#1677)
+ - Misc changes:
+ - Update btcutil dependency (#1704)
+ - Add Dockerfile to build and run btcd on Docker (#1465)
+ - Rework documentation and publish on https://btcd.readthedocs.io (#1468)
+ - Add support for Go 1.15 (#1619)
+ - Add Go 1.14 as the minimum supported version of Golang (#1621)
+ - Contributors (alphabetical order):
+ - 10gic
+ - Andrew Tugarinov
+ - Anirudha Bose
+ - Appelberg-s
+ - Armando Ochoa
+ - Aurèle Oulès
+ - Calvin Kim
+ - Christian Lehmann
+ - Conner Fromknecht
+ - Dan Cline
+ - David Mazary
+ - Elliott Minns
+ - Federico Bond
+ - Friedger Müffke
+ - Gustavo Chain
+ - Hanjun Kim
+ - Henry Fisher
+ - Iskander Sharipov
+ - Jake Sylvestre
+ - Johan T. Halseth
+ - John C. Vernaleo
+ - Liran Sharir
+ - Mikael Lindlof
+ - Olaoluwa Osuntokun
+ - Oliver Gugger
+ - Rjected
+ - Steven Kreuzer
+ - Torkel Rogstad
+ - Tristyn
+ - Victor Lavaud
+ - Vinayak Borkar
+ - Wilmer Paulino
+ - Yaacov Akiba Slama
+ - ebiiim
+ - ipriver
+ - wakiyamap
+ - yyforyongyu
+
+Changes in 0.21.0 (Thu Aug 27 2020)
+ - Network-related changes:
+ - Handle notfound messages from peers in netsync package (#1603)
+ - RPC changes:
+ - Add compatibility for getblock RPC changes in bitcoind 0.15.0 (#1529)
+ - Add new optional Params field to rpcclient.ConnConfig (#1467)
+ - Add new error code ErrRPCInWarmup in btcjson (#1541)
+ - Add compatibility for changes to getmempoolentry response in bitcoind
+ 0.19.0 (#1524)
+ - Add rpcclient methods for estimatesmartfee and generatetoaddress
+ commands (#1500)
+ - Add rpcclient method for getblockstats command (#1500)
+ - Parse serialized transaction from createrawtransaction command using
+ both segwit, and legacy format (#1502)
+ - Support cookie-based authentication in rpcclient (#1460)
+ - Add rpcclient method for getchaintxstats command (#1571)
+ - Add rpcclient method for fundrawtransaction command (#1553)
+ - Add rpcclient method for getbalances command (#1595)
+ - Add new method rpcclient.GetTransactionWatchOnly (#1592)
+ - Crypto changes:
+ - Fix panic in fieldVal.SetByteSlice when called with large values, and
+ improve the method to be 35% faster (#1602)
+ - btcctl changes:
+ - Add -regtest mode to btcctl (#1556)
+ - Misc changes:
+ - Fix a bug due to a deadlock in connmgr's dynamic ban scoring (#1509)
+ - Add blockchain.NewUtxoEntry() to directly create entries for
+ UtxoViewpoint (#1588)
+ - Replace LRU cache implementation in peer package with a generic one
+ from decred/dcrd (#1599)
+ - Contributors (alphabetical order):
+ - Anirudha Bose
+ - Antonin Hildebrand
+ - Dan Cline
+ - Daniel McNally
+ - David Hill
+ - Federico Bond
+ - George Tankersley
+ - Henry
+ - Henry Harder
+ - Iskander Sharipov
+ - Ivan Kuznetsov
+ - Jake Sylvestre
+ - Javed Khan
+ - JeremyRand
+ - Jin
+ - John C. Vernaleo
+ - Kulpreet Singh
+ - Mikael Lindlof
+ - Murray Nesbitt
+ - Nisen
+ - Olaoluwa Osuntokun
+ - Oliver Gugger
+ - Steven Roose
+ - Torkel Rogstad
+ - Tyler Chambers
+ - Wilmer Paulino
+ - Yash Bhutwala
+ - adiabat
+ - jalavosus
+ - mohanson
+ - qqjettkgjzhxmwj
+ - qshuai
+ - shuai.qi
+ - tpkeeper
+
+Changes in v0.20.1 (Wed Nov 13 2019)
+ - RPC changes:
+ - Add compatibility for bitcoind v0.19.0 in rpcclient and btcjson
+ packages (#1484)
+ - Contributors (alphabetical order):
+ - Eugene Zeigel
+ - Olaoluwa Osuntokun
+ - Wilmer Paulino
+
+Changes in v0.20.0 (Tue Oct 15 2019)
+ - Significant changes made since 0.12.0. See git log or refer to release
+ notes on GitHub for full details.
+ - Contributors (alphabetical order):
+ - Albert Puigsech Galicia
+ - Alex Akselrod
+ - Alex Bosworth
+ - Alex Manuskin
+ - Alok Menghrajani
+ - Anatoli Babenia
+ - Andy Weidenbaum
+ - Calvin McAnarney
+ - Chris Martin
+ - Chris Pacia
+ - Chris Shepherd
+ - Conner Fromknecht
+ - Craig Sturdy
+ - Cédric Félizard
+ - Daniel Krawisz
+ - Daniel Martí
+ - Daniel McNally
+ - Dario Nieuwenhuis
+ - Dave Collins
+ - David Hill
+ - David de Kloet
+ - GeertJohan
+ - Grace Noah
+ - Gregory Trubetskoy
+ - Hector Jusforgues
+ - Iskander (Alex) Sharipov
+ - Janus Troelsen
+ - Jasper
+ - Javed Khan
+ - Jeremiah Goyette
+ - Jim Posen
+ - Jimmy Song
+ - Johan T. Halseth
+ - John C. Vernaleo
+ - Jonathan Gillham
+ - Josh Rickmar
+ - Jon Underwood
+ - Jonathan Zeppettini
+ - Jouke Hofman
+ - Julian Meyer
+ - Kai
+ - Kamil Slowikowski
+ - Kefkius
+ - Leonardo Lazzaro
+ - Marco Peereboom
+ - Marko Bencun
+ - Mawueli Kofi Adzoe
+ - Michail Kargakis
+ - Mitchell Paull
+ - Nathan Bass
+ - Nicola 'tekNico' Larosa
+ - Olaoluwa Osuntokun
+ - Pedro Martelletto
+ - Ricardo Velhote
+ - Roei Erez
+ - Ruben de Vries
+ - Rune T. Aune
+ - Sad Pencil
+ - Shuai Qi
+ - Steven Roose
+ - Tadge Dryja
+ - Tibor Bősze
+ - Tomás Senart
+ - Tzu-Jung Lee
+ - Vadym Popov
+ - Waldir Pimenta
+ - Wilmer Paulino
+ - benma
+ - danda
+ - dskloet
+ - esemplastic
+ - jadeblaquiere
+ - nakagawa
+ - preminem
+ - qshuai
+
+Changes in 0.12.0 (Fri Nov 20 2015)
+ - Protocol and network related changes:
+ - Add a new checkpoint at block height 382320 (#555)
+ - Implement BIP0065 which includes support for version 4 blocks, a new
+ consensus opcode (OP_CHECKLOCKTIMEVERIFY) that enforces transaction
+ lock times, and a double-threshold switchover mechanism (#535, #459,
+ #455)
+ - Implement BIP0111 which provides a new bloom filter service flag and
+ hence provides support for protocol version 70011 (#499)
+ - Add a new parameter --nopeerbloomfilters to allow disabling bloom
+ filter support (#499)
+ - Reject non-canonically encoded variable length integers (#507)
+ - Add mainnet peer discovery DNS seed (seed.bitcoin.jonasschnelli.ch)
+ (#496)
+ - Correct reconnect handling for persistent peers (#463, #464)
+ - Ignore requests for block headers if not fully synced (#444)
+ - Add CLI support for specifying the zone id on IPv6 addresses (#538)
+ - Fix a couple of issues where the initial block sync could stall (#518,
+ #229, #486)
+ - Fix an issue which prevented the --onion option from working as
+ intended (#446)
+ - Transaction relay (memory pool) changes:
+ - Require transactions to only include signatures encoded with the
+ canonical 'low-s' encoding (#512)
+ - Add a new parameter --minrelaytxfee to allow the minimum transaction
+ fee in BTC/kB to be overridden (#520)
+ - Retain memory pool transactions when they redeem another one that is
+ removed when a block is accepted (#539)
+ - Do not send reject messages for a transaction if it is valid but
+ causes an orphan transaction which depends on it to be determined
+ as invalid (#546)
+ - Refrain from attempting to add orphans to the memory pool multiple
+ times when the transaction they redeem is added (#551)
+ - Modify minimum transaction fee calculations to scale based on bytes
+ instead of full kilobyte boundaries (#521, #537)
+ - Implement signature cache:
+ - Provides a limited memory cache of validated signatures which is a
+ huge optimization when verifying blocks for transactions that are
+ already in the memory pool (#506)
+ - Add a new parameter '--sigcachemaxsize' which allows the size of the
+ new cache to be manually changed if desired (#506)
+ - Mining support changes:
+ - Notify getblocktemplate long polling clients when a block is pushed
+ via submitblock (#488)
+ - Speed up getblocktemplate by making use of the new signature cache
+ (#506)
+ - RPC changes:
+ - Implement getmempoolinfo command (#453)
+ - Implement getblockheader command (#461)
+ - Modify createrawtransaction command to accept a new optional parameter
+ 'locktime' (#529)
+ - Modify listunspent result to include the 'spendable' field (#440)
+ - Modify getinfo command to include 'errors' field (#511)
+ - Add timestamps to blockconnected and blockdisconnected notifications
+ (#450)
+ - Several modifications to searchrawtranscations command:
+ - Accept a new optional parameter 'vinextra' which causes the results
+ to include information about the outputs referenced by a transaction's
+ inputs (#485, #487)
+ - Skip entries in the mempool too (#495)
+ - Accept a new optional parameter 'reverse' to return the results in
+ reverse order (most recent to oldest) (#497)
+ - Accept a new optional parameter 'filteraddrs' which causes the
+ results to only include inputs and outputs which involve the
+ provided addresses (#516)
+ - Change the notification order to notify clients about mined
+ transactions (recvtx, redeemingtx) before the blockconnected
+ notification (#449)
+ - Update verifymessage RPC to use the standard algorithm so it is
+ compatible with other implementations (#515)
+ - Improve ping statistics by pinging on an interval (#517)
+ - Websocket changes:
+ - Implement session command which returns a per-session unique id (#500,
+ #503)
+ - btcctl utility changes:
+ - Add getmempoolinfo command (#453)
+ - Add getblockheader command (#461)
+ - Add getwalletinfo command (#471)
+ - Notable developer-related package changes:
+ - Introduce a new peer package which acts a common base for creating and
+ concurrently managing bitcoin network peers (#445)
+ - Various cleanup of the new peer package (#528, #531, #524, #534,
+ #549)
+ - Blocks heights now consistently use int32 everywhere (#481)
+ - The BlockHeader type in the wire package now provides the BtcDecode
+ and BtcEncode methods (#467)
+ - Update wire package to recognize BIP0064 (getutxo) service bit (#489)
+ - Export LockTimeThreshold constant from txscript package (#454)
+ - Export MaxDataCarrierSize constant from txscript package (#466)
+ - Provide new IsUnspendable function from the txscript package (#478)
+ - Export variable length string functions from the wire package (#514)
+ - Export DNS Seeds for each network from the chaincfg package (#544)
+ - Preliminary work towards separating the memory pool into a separate
+ package (#525, #548)
+ - Misc changes:
+ - Various documentation updates (#442, #462, #465, #460, #470, #473,
+ #505, #530, #545)
+ - Add installation instructions for gentoo (#542)
+ - Ensure an error is shown if OS limits can't be set at startup (#498)
+ - Tighten the standardness checks for multisig scripts (#526)
+ - Test coverage improvement (#468, #494, #527, #543, #550)
+ - Several optimizations (#457, #474, #475, #476, #508, #509)
+ - Minor code cleanup and refactoring (#472, #479, #482, #519, #540)
+ - Contributors (alphabetical order):
+ - Ben Echols
+ - Bruno Clermont
+ - danda
+ - Daniel Krawisz
+ - Dario Nieuwenhuis
+ - Dave Collins
+ - David Hill
+ - Javed Khan
+ - Jonathan Gillham
+ - Joseph Becher
+ - Josh Rickmar
+ - Justus Ranvier
+ - Mawuli Adzoe
+ - Olaoluwa Osuntokun
+ - Rune T. Aune
+
+Changes in 0.11.1 (Wed May 27 2015)
+ - Protocol and network related changes:
+ - Use correct sub-command in reject message for rejected transactions
+ (#436, #437)
+ - Add a new parameter --torisolation which forces new circuits for each
+ connection when using tor (#430)
+ - Transaction relay (memory pool) changes:
+ - Reduce the default number max number of allowed orphan transactions
+ to 1000 (#419)
+ - Add a new parameter --maxorphantx which allows the maximum number of
+ orphan transactions stored in the mempool to be specified (#419)
+ - RPC changes:
+ - Modify listtransactions result to include the 'involveswatchonly' and
+ 'vout' fields (#427)
+ - Update getrawtransaction result to omit the 'confirmations' field
+ when it is 0 (#420, #422)
+ - Update signrawtransaction result to include errors (#423)
+ - btcctl utility changes:
+ - Add gettxoutproof command (#428)
+ - Add verifytxoutproof command (#428)
+ - Notable developer-related package changes:
+ - The btcec package now provides the ability to perform ECDH
+ encryption and decryption (#375)
+ - The block and header validation in the blockchain package has been
+ split to help pave the way toward concurrent downloads (#386)
+ - Misc changes:
+ - Minor peer optimization (#433)
+ - Contributors (alphabetical order):
+ - Dave Collins
+ - David Hill
+ - Federico Bond
+ - Ishbir Singh
+ - Josh Rickmar
+
+Changes in 0.11.0 (Wed May 06 2015)
+ - Protocol and network related changes:
+ - **IMPORTANT: Update is required due to the following point**
+ - Correct a few corner cases in script handling which could result in
+ forking from the network on non-standard transactions (#425)
+ - Add a new checkpoint at block height 352940 (#418)
+ - Optimized script execution (#395, #400, #404, #409)
+ - Fix a case that could lead stalled syncs (#138, #296)
+ - Network address manager changes:
+ - Implement eclipse attack countermeasures as proposed in
+ http://cs-people.bu.edu/heilman/eclipse (#370, #373)
+ - Optional address indexing changes:
+ - Fix an issue where a reorg could cause an orderly shutdown when the
+ address index is active (#340, #357)
+ - Transaction relay (memory pool) changes:
+ - Increase maximum allowed space for nulldata transactions to 80 bytes
+ (#331)
+ - Implement support for the following rules specified by BIP0062:
+ - The S value in ECDSA signature must be at most half the curve order
+ (rule 5) (#349)
+ - Script execution must result in a single non-zero value on the stack
+ (rule 6) (#347)
+ - NOTE: All 7 rules of BIP0062 are now implemented
+ - Use network adjusted time in finalized transaction checks to improve
+ consistency across nodes (#332)
+ - Process orphan transactions on acceptance of new transactions (#345)
+ - RPC changes:
+ - Add support for a limited RPC user which is not allowed admin level
+ operations on the server (#363)
+ - Implement node command for more unified control over connected peers
+ (#79, #341)
+ - Implement generate command for regtest/simnet to support
+ deterministically mining a specified number of blocks (#362, #407)
+ - Update searchrawtransactions to return the matching transactions in
+ order (#354)
+ - Correct an issue with searchrawtransactions where it could return
+ duplicates (#346, #354)
+ - Increase precision of 'difficulty' field in getblock result to 8
+ (#414, #415)
+ - Omit 'nextblockhash' field from getblock result when it is empty
+ (#416, #417)
+ - Add 'id' and 'timeoffset' fields to getpeerinfo result (#335)
+ - Websocket changes:
+ - Implement new commands stopnotifyspent, stopnotifyreceived,
+ stopnotifyblocks, and stopnotifynewtransactions to allow clients to
+ cancel notification registrations (#122, #342)
+ - btcctl utility changes:
+ - A single dash can now be used as an argument to cause that argument to
+ be read from stdin (#348)
+ - Add generate command
+ - Notable developer-related package changes:
+ - The new version 2 btcjson package has now replaced the deprecated
+ version 1 package (#368)
+ - The btcec package now performs all signing using RFC6979 deterministic
+ signatures (#358, #360)
+ - The txscript package has been significantly cleaned up and had a few
+ API changes (#387, #388, #389, #390, #391, #392, #393, #395, #396,
+ #400, #403, #404, #405, #406, #408, #409, #410, #412)
+ - A new PkScriptLocs function has been added to the wire package MsgTx
+ type which provides callers that deal with scripts optimization
+ opportunities (#343)
+ - Misc changes:
+ - Minor wire hashing optimizations (#366, #367)
+ - Other minor internal optimizations
+ - Contributors (alphabetical order):
+ - Alex Akselrod
+ - Arne Brutschy
+ - Chris Jepson
+ - Daniel Krawisz
+ - Dave Collins
+ - David Hill
+ - Jimmy Song
+ - Jonas Nick
+ - Josh Rickmar
+ - Olaoluwa Osuntokun
+ - Oleg Andreev
+
+Changes in 0.10.0 (Sun Mar 01 2015)
+ - Protocol and network related changes:
+ - Add a new checkpoint at block height 343185
+ - Implement BIP066 which includes support for version 3 blocks, a new
+ consensus rule which prevents non-DER encoded signatures, and a
+ double-threshold switchover mechanism
+ - Rather than announcing all known addresses on getaddr requests which
+ can possibly result in multiple messages, randomize the results and
+ limit them to the max allowed by a single message (1000 addresses)
+ - Add more reserved IP spaces to the address manager
+ - Transaction relay (memory pool) changes:
+ - Make transactions which contain reserved opcodes nonstandard
+ - No longer accept or relay free and low-fee transactions that have
+ insufficient priority to be mined in the next block
+ - Implement support for the following rules specified by BIP0062:
+ - ECDSA signature must use strict DER encoding (rule 1)
+ - The signature script must only contain push operations (rule 2)
+ - All push operations must use the smallest possible encoding (rule 3)
+ - All stack values interpreted as a number must be encoding using the
+ shortest possible form (rule 4)
+ - NOTE: Rule 1 was already enforced, however the entire script now
+ evaluates to false rather than only the signature verification as
+ required by BIP0062
+ - Allow transactions with nulldata transaction outputs to be treated as
+ standard
+ - Mining support changes:
+ - Modify the getblocktemplate RPC to generate and return block templates
+ for version 3 blocks which are compatible with BIP0066
+ - Allow getblocktemplate to serve blocks when the current time is
+ less than the minimum allowed time for a generated block template
+ (https://github.com/btcsuite/btcd/issues/209)
+ - Crypto changes:
+ - Optimize scalar multiplication by the base point by using a
+ pre-computed table which results in approximately a 35% speedup
+ (https://github.com/btcsuite/btcec/issues/2)
+ - Optimize general scalar multiplication by using the secp256k1
+ endomorphism which results in approximately a 17-20% speedup
+ (https://github.com/btcsuite/btcec/issues/1)
+ - Optimize general scalar multiplication by using non-adjacent form
+ which results in approximately an additional 8% speedup
+ (https://github.com/btcsuite/btcec/issues/3)
+ - Implement optional address indexing:
+ - Add a new parameter --addrindex which will enable the creation of an
+ address index which can be queried to determine all transactions which
+ involve a given address
+ (https://github.com/btcsuite/btcd/issues/190)
+ - Add a new logging subsystem for address index related operations
+ - Support new searchrawtransactions RPC
+ (https://github.com/btcsuite/btcd/issues/185)
+ - RPC changes:
+ - Require TLS version 1.2 as the minimum version for all TLS connections
+ - Provide support for disabling TLS when only listening on localhost
+ (https://github.com/btcsuite/btcd/pull/192)
+ - Modify help output for all commands to provide much more consistent
+ and detailed information
+ - Correct case in getrawtransaction which would refuse to serve certain
+ transactions with invalid scripts
+ (https://github.com/btcsuite/btcd/issues/210)
+ - Correct error handling in the getrawtransaction RPC which could lead
+ to a crash in rare cases
+ (https://github.com/btcsuite/btcd/issues/196)
+ - Update getinfo RPC to include the appropriate 'timeoffset' calculated
+ from the median network time
+ - Modify listreceivedbyaddress result type to include txids field so it
+ is compatible
+ - Add 'iswatchonly' field to validateaddress result
+ - Add 'startingpriority' and 'currentpriority' fields to getrawmempool
+ (https://github.com/btcsuite/btcd/issues/178)
+ - Don't omit the 'confirmations' field from getrawtransaction when it is
+ zero
+ - Websocket changes:
+ - Modify the behavior of the rescan command to automatically register
+ for notifications about transactions paying to rescanned addresses
+ or spending outputs from the final rescan utxo set when the rescan
+ is through the best block in the chain
+ - btcctl utility changes:
+ - Make the list of commands available via the -l option rather than
+ dumping the entire list on usage errors
+ - Alphabetize and categorize the list of commands by chain and wallet
+ - Make the help option only show the help options instead of also
+ dumping all of the commands
+ - Make the usage syntax much more consistent and correct a few cases of
+ misnamed fields
+ (https://github.com/btcsuite/btcd/issues/305)
+ - Improve usage errors to show the specific parameter number, reason,
+ and error code
+ - Only show the usage for specific command is shown when a valid command
+ is provided with invalid parameters
+ - Add support for a SOCK5 proxy
+ - Modify output for integer fields (such as timestamps) to display
+ normally instead in scientific notation
+ - Add invalidateblock command
+ - Add reconsiderblock command
+ - Add createnewaccount command
+ - Add renameaccount command
+ - Add searchrawtransactions command
+ - Add importaddress command
+ - Add importpubkey command
+ - showblock utility changes:
+ - Remove utility in favor of the RPC getblock method
+ - Notable developer-related package changes:
+ - Many of the core packages have been relocated into the btcd repository
+ (https://github.com/btcsuite/btcd/issues/214)
+ - A new version of the btcjson package that has been completely
+ redesigned from the ground up based based upon how the project has
+ evolved and lessons learned while using it since it was first written
+ is now available in the btcjson/v2/btcjson directory
+ - This will ultimately replace the current version so anyone making
+ use of this package will need to update their code accordingly
+ - The btcec package now provides better facilities for working directly
+ with its public and private keys without having to mix elements from
+ the ecdsa package
+ - Update the script builder to ensure all rules specified by BIP0062 are
+ adhered to when creating scripts
+ - The blockchain package now provides a MedianTimeSource interface and
+ concrete implementation for providing time samples from remote peers
+ and using that data to calculate an offset against the local time
+ - Misc changes:
+ - Fix a slow memory leak due to tickers not being stopped
+ (https://github.com/btcsuite/btcd/issues/189)
+ - Fix an issue where a mix of orphans and SPV clients could trigger a
+ condition where peers would no longer be served
+ (https://github.com/btcsuite/btcd/issues/231)
+ - The RPC username and password can now contain symbols which previously
+ conflicted with special symbols used in URLs
+ - Improve handling of obtaining random nonces to prevent cases where it
+ could error when not enough entropy was available
+ - Improve handling of home directory creation errors such as in the case
+ of unmounted symlinks (https://github.com/btcsuite/btcd/issues/193)
+ - Improve the error reporting for rejected transactions to include the
+ inputs which are missing and/or being double spent
+ - Update sample config file with new options and correct a comment
+ regarding the fact the RPC server only listens on localhost by default
+ (https://github.com/btcsuite/btcd/issues/218)
+ - Update the continuous integration builds to run several tools which
+ help keep code quality high
+ - Significant amount of internal code cleanup and improvements
+ - Other minor internal optimizations
+ - Code Contributors (alphabetical order):
+ - Beldur
+ - Ben Holden-Crowther
+ - Dave Collins
+ - David Evans
+ - David Hill
+ - Guilherme Salgado
+ - Javed Khan
+ - Jimmy Song
+ - John C. Vernaleo
+ - Jonathan Gillham
+ - Josh Rickmar
+ - Michael Ford
+ - Michail Kargakis
+ - kac
+ - Olaoluwa Osuntokun
+
+Changes in 0.9.0 (Sat Sep 20 2014)
+ - Protocol and network related changes:
+ - Add a new checkpoint at block height 319400
+ - Add support for BIP0037 bloom filters
+ (https://github.com/conformal/btcd/issues/132)
+ - Implement BIP0061 reject handling and hence support for protocol
+ version 70002 (https://github.com/conformal/btcd/issues/133)
+ - Add testnet DNS seeds for peer discovery (testnet-seed.alexykot.me
+ and testnet-seed.bitcoin.schildbach.de)
+ - Add mainnet DNS seed for peer discovery (seeds.bitcoin.open-nodes.org)
+ - Make multisig transactions with non-null dummy data nonstandard
+ (https://github.com/conformal/btcd/issues/131)
+ - Make transactions with an excessive number of signature operations
+ nonstandard
+ - Perform initial DNS lookups concurrently which allows connections
+ more quickly
+ - Improve the address manager to significantly reduce memory usage and
+ add tests
+ - Remove orphan transactions when they appear in a mined block
+ (https://github.com/conformal/btcd/issues/166)
+ - Apply incremental back off on connection retries for persistent peers
+ that give invalid replies to mirror the logic used for failed
+ connections (https://github.com/conformal/btcd/issues/103)
+ - Correct rate-limiting of free and low-fee transactions
+ - Mining support changes:
+ - Implement getblocktemplate RPC with the following support:
+ (https://github.com/conformal/btcd/issues/124)
+ - BIP0022 Non-Optional Sections
+ - BIP0022 Long Polling
+ - BIP0023 Basic Pool Extensions
+ - BIP0023 Mutation coinbase/append
+ - BIP0023 Mutations time, time/increment, and time/decrement
+ - BIP0023 Mutation transactions/add
+ - BIP0023 Mutations prevblock, coinbase, and generation
+ - BIP0023 Block Proposals
+ - Implement built-in concurrent CPU miner
+ (https://github.com/conformal/btcd/issues/137)
+ NOTE: CPU mining on mainnet is pointless. This has been provided
+ for testing purposes such as for the new simulation test network
+ - Add --generate flag to enable CPU mining
+ - Deprecate the --getworkkey flag in favor of --miningaddr which
+ specifies which addresses generated blocks will choose from to pay
+ the subsidy to
+ - RPC changes:
+ - Implement gettxout command
+ (https://github.com/conformal/btcd/issues/141)
+ - Implement validateaddress command
+ - Implement verifymessage command
+ - Mark getunconfirmedbalance RPC as wallet-only
+ - Mark getwalletinfo RPC as wallet-only
+ - Update getgenerate, setgenerate, gethashespersec, and getmininginfo
+ to return the appropriate information about new CPU mining status
+ - Modify getpeerinfo pingtime and pingwait field types to float64 so
+ they are compatible
+ - Improve disconnect handling for normal HTTP clients
+ - Make error code returns for invalid hex more consistent
+ - Websocket changes:
+ - Switch to a new more efficient websocket package
+ (https://github.com/conformal/btcd/issues/134)
+ - Add rescanfinished notification
+ - Modify the rescanprogress notification to include block hash as well
+ as height (https://github.com/conformal/btcd/issues/151)
+ - btcctl utility changes:
+ - Accept --simnet flag which automatically selects the appropriate port
+ and TLS certificates needed to communicate with btcd and btcwallet on
+ the simulation test network
+ - Fix createrawtransaction command to send amounts denominated in BTC
+ - Add estimatefee command
+ - Add estimatepriority command
+ - Add getmininginfo command
+ - Add getnetworkinfo command
+ - Add gettxout command
+ - Add lockunspent command
+ - Add signrawtransaction command
+ - addblock utility changes:
+ - Accept --simnet flag which automatically selects the appropriate port
+ and TLS certificates needed to communicate with btcd and btcwallet on
+ the simulation test network
+ - Notable developer-related package changes:
+ - Provide a new bloom package in btcutil which allows creating and
+ working with BIP0037 bloom filters
+ - Provide a new hdkeychain package in btcutil which allows working with
+ BIP0032 hierarchical deterministic key chains
+ - Introduce a new btcnet package which houses network parameters
+ - Provide new simnet network (--simnet) which is useful for private
+ simulation testing
+ - Enforce low S values in serialized signatures as detailed in BIP0062
+ - Return errors from all methods on the btcdb.Db interface
+ (https://github.com/conformal/btcdb/issues/5)
+ - Allow behavior flags to alter btcchain.ProcessBlock
+ (https://github.com/conformal/btcchain/issues/5)
+ - Provide a new SerializeSize API for blocks
+ (https://github.com/conformal/btcwire/issues/19)
+ - Several of the core packages now work with Google App Engine
+ - Misc changes:
+ - Correct an issue where the database could corrupt under certain
+ circumstances which would require a new chain download
+ - Slightly optimize deserialization
+ - Use the correct IP block for he.net
+ - Fix an issue where it was possible the block manager could hang on
+ shutdown
+ - Update sample config file so the comments are on a separate line
+ rather than the end of a line so they are not interpreted as settings
+ (https://github.com/conformal/btcd/issues/135)
+ - Correct an issue where getdata requests were not being properly
+ throttled which could lead to larger than necessary memory usage
+ - Always show help when given the help flag even when the config file
+ contains invalid entries
+ - General code cleanup and minor optimizations
+
+Changes in 0.8.0-beta (Sun May 25 2014)
+ - Btcd is now Beta (https://github.com/conformal/btcd/issues/130)
+ - Add a new checkpoint at block height 300255
+ - Protocol and network related changes:
+ - Lower the minimum transaction relay fee to 1000 satoshi to match
+ recent reference client changes
+ (https://github.com/conformal/btcd/issues/100)
+ - Raise the maximum signature script size to support standard 15-of-15
+ multi-signature pay-to-script-hash transactions with compressed pubkeys
+ to remain compatible with the reference client
+ (https://github.com/conformal/btcd/issues/128)
+ - Reduce max bytes allowed for a standard nulldata transaction to 40 for
+ compatibility with the reference client
+ - Introduce a new btcnet package which houses all of the network params
+ for each network (mainnet, testnet3, regtest) to ultimately enable
+ easier addition and tweaking of networks without needing to change
+ several packages
+ - Fix several script discrepancies found by reference client test data
+ - Add new DNS seed for peer discovery (seed.bitnodes.io)
+ - Reduce the max known inventory cache from 20000 items to 1000 items
+ - Fix an issue where unknown inventory types could lead to a hung peer
+ - Implement inventory rebroadcast handler for sendrawtransaction
+ (https://github.com/conformal/btcd/issues/99)
+ - Update user agent to fully support BIP0014
+ (https://github.com/conformal/btcwire/issues/10)
+ - Implement initial mining support:
+ - Add a new logging subsystem for mining related operations
+ - Implement infrastructure for creating block templates
+ - Provide options to control block template creation settings
+ - Support the getwork RPC
+ - Allow address identifiers to apply to more than one network since both
+ testnet3 and the regression test network unfortunately use the same
+ identifier
+ - RPC changes:
+ - Set the content type for HTTP POST RPC connections to application/json
+ (https://github.com/conformal/btcd/issues/121)
+ - Modified the RPC server startup so it only requires at least one valid
+ listen interface
+ - Correct an error path where it was possible certain errors would not
+ be returned
+ - Implement getwork command
+ (https://github.com/conformal/btcd/issues/125)
+ - Update sendrawtransaction command to reject orphans
+ - Update sendrawtransaction command to include the reason a transaction
+ was rejected
+ - Update getinfo command to populate connection count field
+ - Update getinfo command to include relay fee field
+ (https://github.com/conformal/btcd/issues/107)
+ - Allow transactions submitted with sendrawtransaction to bypass the
+ rate limiter
+ - Allow the getcurrentnet and getbestblock extensions to be accessed via
+ HTTP POST in addition to Websockets
+ (https://github.com/conformal/btcd/issues/127)
+ - Websocket changes:
+ - Rework notifications to ensure they are delivered in the order they
+ occur
+ - Rename notifynewtxs command to notifyreceived (funds received)
+ - Rename notifyallnewtxs command to notifynewtransactions
+ - Rename alltx notification to txaccepted
+ - Rename allverbosetx notification to txacceptedverbose
+ (https://github.com/conformal/btcd/issues/98)
+ - Add rescan progress notification
+ - Add recvtx notification
+ - Add redeemingtx notification
+ - Modify notifyspent command to accept an array of outpoints
+ (https://github.com/conformal/btcd/issues/123)
+ - Significantly optimize the rescan command to yield up to a 60x speed
+ increase
+ - btcctl utility changes:
+ - Add createencryptedwallet command
+ - Add getblockchaininfo command
+ - Add importwallet command
+ - Add addmultisigaddress command
+ - Add setgenerate command
+ - Accept --testnet and --wallet flags which automatically select
+ the appropriate port and TLS certificates needed to communicate
+ with btcd and btcwallet (https://github.com/conformal/btcd/issues/112)
+ - Allow path expansion from config file entries
+ (https://github.com/conformal/btcd/issues/113)
+ - Minor refactor simplify handling of options
+ - addblock utility changes:
+ - Improve logging by making it consistent with the logging provided by
+ btcd (https://github.com/conformal/btcd/issues/90)
+ - Improve several package APIs for developers:
+ - Add new amount type for consistently handling monetary values
+ - Add new coin selector API
+ - Add new WIF (Wallet Import Format) API
+ - Add new crypto types for private keys and signatures
+ - Add new API to sign transactions including script merging and hash
+ types
+ - Expose function to extract all pushed data from a script
+ (https://github.com/conformal/btcscript/issues/8)
+ - Misc changes:
+ - Optimize address manager shuffling to do 67% less work on average
+ - Resolve a couple of benign data races found by the race detector
+ (https://github.com/conformal/btcd/issues/101)
+ - Add IP address to all peer related errors to clarify which peer is the
+ cause (https://github.com/conformal/btcd/issues/102)
+ - Fix a UPNP case issue that prevented the --upnp option from working
+ with some UPNP servers
+ - Update documentation in the sample config file regarding debug levels
+ - Adjust some logging levels to improve debug messages
+ - Improve the throughput of query messages to the block manager
+ - Several minor optimizations to reduce GC churn and enhance speed
+ - Other minor refactoring
+ - General code cleanup
+
+Changes in 0.7.0 (Thu Feb 20 2014)
+ - Fix an issue when parsing scripts which contain a multi-signature script
+ which require zero signatures such as testnet block
+ 000000001881dccfeda317393c261f76d09e399e15e27d280e5368420f442632
+ (https://github.com/conformal/btcscript/issues/7)
+ - Add check to ensure all transactions accepted to mempool only contain
+ canonical data pushes (https://github.com/conformal/btcscript/issues/6)
+ - Fix an issue causing excessive memory consumption
+ - Significantly rework and improve the websocket notification system:
+ - Each client is now independent so slow clients no longer limit the
+ speed of other connected clients
+ - Potentially long-running operations such as rescans are now run in
+ their own handler and rate-limited to one operation at a time without
+ preventing simultaneous requests from the same client for the faster
+ requests or notifications
+ - A couple of scenarios which could cause shutdown to hang have been
+ resolved
+ - Update notifynewtx notifications to support all address types instead
+ of only pay-to-pubkey-hash
+ - Provide a --rpcmaxwebsockets option to allow limiting the number of
+ concurrent websocket clients
+ - Add a new websocket command notifyallnewtxs to request notifications
+ (https://github.com/conformal/btcd/issues/86) (thanks @flammit)
+ - Improve btcctl utility in the following ways:
+ - Add getnetworkhashps command
+ - Add gettransaction command (wallet-specific)
+ - Add signmessage command (wallet-specific)
+ - Update getwork command to accept
+ - Continue cleanup and work on implementing the RPC API:
+ - Implement getnettotals command
+ (https://github.com/conformal/btcd/issues/84)
+ - Implement networkhashps command
+ (https://github.com/conformal/btcd/issues/87)
+ - Update getpeerinfo to always include syncnode field even when false
+ - Remove help addenda for getpeerinfo now that it supports all fields
+ - Close standard RPC connections on auth failure
+ - Provide a --rpcmaxclients option to allow limiting the number of
+ concurrent RPC clients (https://github.com/conformal/btcd/issues/68)
+ - Include IP address in RPC auth failure log messages
+ - Resolve a rather harmless data races found by the race detector
+ (https://github.com/conformal/btcd/issues/94)
+ - Increase block priority size and max standard transaction size to 50k
+ and 100k, respectively (https://github.com/conformal/btcd/issues/71)
+ - Add rate limiting of free transactions to the memory pool to prevent
+ penny flooding (https://github.com/conformal/btcd/issues/40)
+ - Provide a --logdir option (https://github.com/conformal/btcd/issues/95)
+ - Change the default log file path to include the network
+ - Add a new ScriptBuilder interface to btcscript to support creation of
+ custom scripts (https://github.com/conformal/btcscript/issues/5)
+ - General code cleanup
+
+Changes in 0.6.0 (Tue Feb 04 2014)
+ - Fix an issue when parsing scripts which contain invalid signatures that
+ caused a chain fork on block
+ 0000000000000001e4241fd0b3469a713f41c5682605451c05d3033288fb2244
+ - Correct an issue which could lead to an error in removeBlockNode
+ (https://github.com/conformal/btcchain/issues/4)
+ - Improve addblock utility as follows:
+ - Check imported blocks against all chain rules and checkpoints
+ - Skip blocks which are already known so you can stop and restart the
+ import or start the import after you have already downloaded a portion
+ of the chain
+ - Correct an issue where the utility did not shutdown cleanly after
+ processing all blocks
+ - Add error on attempt to import orphan blocks
+ - Improve error handling and reporting
+ - Display statistics after input file has been fully processed
+ - Rework, optimize, and improve headers-first mode:
+ - Resuming the chain sync from any point before the final checkpoint
+ will now use headers-first mode
+ (https://github.com/conformal/btcd/issues/69)
+ - Verify all checkpoints as opposed to only the final one
+ - Reduce and bound memory usage
+ - Rollback to the last known good point when a header does not match a
+ checkpoint
+ - Log information about what is happening with headers
+ - Improve btcctl utility in the following ways:
+ - Add getaddednodeinfo command
+ - Add getnettotals command
+ - Add getblocktemplate command (wallet-specific)
+ - Add getwork command (wallet-specific)
+ - Add getnewaddress command (wallet-specific)
+ - Add walletpassphrasechange command (wallet-specific)
+ - Add walletlock command (wallet-specific)
+ - Add sendfrom command (wallet-specific)
+ - Add sendmany command (wallet-specific)
+ - Add settxfee command (wallet-specific)
+ - Add listsinceblock command (wallet-specific)
+ - Add listaccounts command (wallet-specific)
+ - Add keypoolrefill command (wallet-specific)
+ - Add getreceivedbyaccount command (wallet-specific)
+ - Add getrawchangeaddress command (wallet-specific)
+ - Add gettxoutsetinfo command (wallet-specific)
+ - Add listaddressgroupings command (wallet-specific)
+ - Add listlockunspent command (wallet-specific)
+ - Add listlock command (wallet-specific)
+ - Add listreceivedbyaccount command (wallet-specific)
+ - Add validateaddress command (wallet-specific)
+ - Add verifymessage command (wallet-specific)
+ - Add sendtoaddress command (wallet-specific)
+ - Continue cleanup and work on implementing the RPC API:
+ - Implement submitblock command
+ (https://github.com/conformal/btcd/issues/61)
+ - Implement help command
+ - Implement ping command
+ - Implement getaddednodeinfo command
+ (https://github.com/conformal/btcd/issues/78)
+ - Implement getinfo command
+ - Update getpeerinfo to support bytesrecv and bytessent
+ (https://github.com/conformal/btcd/issues/83)
+ - Improve and correct several RPC server and websocket areas:
+ - Change the connection endpoint for websockets from /wallet to /ws
+ (https://github.com/conformal/btcd/issues/80)
+ - Implement an alternative authentication for websockets so clients
+ such as javascript from browsers that don't support setting HTTP
+ headers can authenticate (https://github.com/conformal/btcd/issues/77)
+ - Add an authentication deadline for RPC connections
+ (https://github.com/conformal/btcd/issues/68)
+ - Use standard authentication failure responses for RPC connections
+ - Make automatically generated certificate more standard so it works
+ from client such as node.js and Firefox
+ - Correct some minor issues which could prevent the RPC server from
+ shutting down in an orderly fashion
+ - Make all websocket notifications require registration
+ - Change the data sent over websockets to text since it is JSON-RPC
+ - Allow connections that do not have an Origin header set
+ - Expose and track the number of bytes read and written per peer
+ (https://github.com/conformal/btcwire/issues/6)
+ - Correct an issue with sendrawtransaction when invoked via websockets
+ which prevented a minedtx notification from being added
+ - Rescan operations issued from remote wallets are no stopped when
+ the wallet disconnects mid-operation
+ (https://github.com/conformal/btcd/issues/66)
+ - Several optimizations related to fetching block information from the
+ database
+ - General code cleanup
+
+Changes in 0.5.0 (Mon Jan 13 2014)
+ - Optimize initial block download by introducing a new mode which
+ downloads the block headers first (up to the final checkpoint)
+ - Improve peer handling to remove the potential for slow peers to cause
+ sluggishness amongst all peers
+ (https://github.com/conformal/btcd/issues/63)
+ - Fix an issue where the initial block sync could stall when the sync peer
+ disconnects (https://github.com/conformal/btcd/issues/62)
+ - Correct an issue where --externalip was doing a DNS lookup on the full
+ host:port instead of just the host portion
+ (https://github.com/conformal/btcd/issues/38)
+ - Fix an issue which could lead to a panic on chain switches
+ (https://github.com/conformal/btcd/issues/70)
+ - Improve btcctl utility in the following ways:
+ - Show getdifficulty output as floating point to 6 digits of precision
+ - Show all JSON object replies formatted as standard JSON
+ - Allow btcctl getblock to accept optional params
+ - Add getaccount command (wallet-specific)
+ - Add getaccountaddress command (wallet-specific)
+ - Add sendrawtransaction command
+ - Continue cleanup and work on implementing RPC API calls
+ - Update getrawmempool to support new optional verbose flag
+ - Update getrawtransaction to match the reference client
+ - Update getblock to support new optional verbose flag
+ - Update raw transactions to fully match the reference client including
+ support for all transaction types and address types
+ - Correct getrawmempool fee field to return BTC instead of Satoshi
+ - Correct getpeerinfo service flag to return 8 digit string so it
+ matches the reference client
+ - Correct verifychain to return a boolean
+ - Implement decoderawtransaction command
+ - Implement createrawtransaction command
+ - Implement decodescript command
+ - Implement gethashespersec command
+ - Allow RPC handler overrides when invoked via a websocket versus
+ legacy connection
+ - Add new DNS seed for peer discovery
+ - Display user agent on new valid peer log message
+ (https://github.com/conformal/btcd/issues/64)
+ - Notify wallet when new transactions that pay to registered addresses
+ show up in the mempool before being mined into a block
+ - Support a tor-specific proxy in addition to a normal proxy
+ (https://github.com/conformal/btcd/issues/47)
+ - Remove deprecated sqlite3 imports from utilities
+ - Remove leftover profile write from addblock utility
+ - Quite a bit of code cleanup and refactoring to improve maintainability
+
+Changes in 0.4.0 (Thu Dec 12 2013)
+ - Allow listen interfaces to be specified via --listen instead of only the
+ port (https://github.com/conformal/btcd/issues/33)
+ - Allow listen interfaces for the RPC server to be specified via
+ --rpclisten instead of only the port
+ (https://github.com/conformal/btcd/issues/34)
+ - Only disable listening when --connect or --proxy are used when no
+ --listen interface are specified
+ (https://github.com/conformal/btcd/issues/10)
+ - Add several new standard transaction checks to transaction memory pool:
+ - Support nulldata scripts as standard
+ - Only allow a max of one nulldata output per transaction
+ - Enforce a maximum of 3 public keys in multi-signature transactions
+ - The number of signatures in multi-signature transactions must not
+ exceed the number of public keys
+ - The number of inputs to a signature script must match the expected
+ number of inputs for the script type
+ - The number of inputs pushed onto the stack by a redeeming signature
+ script must match the number of inputs consumed by the referenced
+ public key script
+ - When a block is connected, remove any transactions from the memory pool
+ which are now double spends as a result of the newly connected
+ transactions
+ - Don't relay transactions resurrected during a chain switch since
+ other peers will also be switching chains and therefore already know
+ about them
+ - Cleanup a few cases where rejected transactions showed as an error
+ rather than as a rejected transaction
+ - Ignore the default configuration file when --regtest (regression test
+ mode) is specified
+ - Implement TLS support for RPC including automatic certificate generation
+ - Support HTTP authentication headers for web sockets
+ - Update address manager to recognize and properly work with Tor
+ addresses (https://github.com/conformal/btcd/issues/36) and
+ (https://github.com/conformal/btcd/issues/37)
+ - Improve btcctl utility in the following ways:
+ - Add the ability to specify a configuration file
+ - Add a default entry for the RPC cert to point to the location
+ it will likely be in the btcd home directory
+ - Implement --version flag
+ - Provide a --notls option to support non-TLS configurations
+ - Fix a couple of minor races found by the Go race detector
+ - Improve logging
+ - Allow logging level to be specified on a per subsystem basis
+ (https://github.com/conformal/btcd/issues/48)
+ - Allow logging levels to be dynamically changed via RPC
+ (https://github.com/conformal/btcd/issues/15)
+ - Implement a rolling log file with a max of 10MB per file and a
+ rotation size of 3 which results in a max logging size of 30 MB
+ - Correct a minor issue with the rescanning websocket call
+ (https://github.com/conformal/btcd/issues/54)
+ - Fix a race with pushing address messages that could lead to a panic
+ (https://github.com/conformal/btcd/issues/58)
+ - Improve which external IP address is reported to peers based on which
+ interface they are connected through
+ (https://github.com/conformal/btcd/issues/35)
+ - Add --externalip option to allow an external IP address to be specified
+ for cases such as tor hidden services or advanced network configurations
+ (https://github.com/conformal/btcd/issues/38)
+ - Add --upnp option to support automatic port mapping via UPnP
+ (https://github.com/conformal/btcd/issues/51)
+ - Update Ctrl+C interrupt handler to properly sync address manager and
+ remove the UPnP port mapping (if needed)
+ - Continue cleanup and work on implementing RPC API calls
+ - Add importprivkey (import private key) command to btcctl
+ - Update getrawtransaction to provide addresses properly, support
+ new verbose param, and match the reference implementation with the
+ exception of MULTISIG (thanks @flammit)
+ - Update getblock with new verbose flag (thanks @flammit)
+ - Add listtransactions command to btcctl
+ - Add getbalance command to btcctl
+ - Add basic support for btcd to run as a native Windows service
+ (https://github.com/conformal/btcd/issues/42)
+ - Package addblock utility with Windows MSIs
+ - Add support for TravisCI (continuous build integration)
+ - Cleanup some documentation and usage
+ - Several other minor bug fixes and general code cleanup
+
+Changes in 0.3.3 (Wed Nov 13 2013)
+ - Significantly improve initial block chain download speed
+ (https://github.com/conformal/btcd/issues/20)
+ - Add a new checkpoint at block height 267300
+ - Optimize most recently used inventory handling
+ (https://github.com/conformal/btcd/issues/21)
+ - Optimize duplicate transaction input check
+ (https://github.com/conformal/btcchain/issues/2)
+ - Optimize transaction hashing
+ (https://github.com/conformal/btcd/issues/25)
+ - Rework and optimize wallet listener notifications
+ (https://github.com/conformal/btcd/issues/22)
+ - Optimize serialization and deserialization
+ (https://github.com/conformal/btcd/issues/27)
+ - Add support for minimum transaction fee to memory pool acceptance
+ (https://github.com/conformal/btcd/issues/29)
+ - Improve leveldb database performance by removing explicit GC call
+ - Fix an issue where Ctrl+C was not always finishing orderly database
+ shutdown
+ - Fix an issue in the script handling for OP_CHECKSIG
+ - Impose max limits on all variable length protocol entries to prevent
+ abuse from malicious peers
+ - Enforce DER signatures for transactions allowed into the memory pool
+ - Separate the debug profile http server from the RPC server
+ - Rework of the RPC code to improve performance and make the code cleaner
+ - The getrawtransaction RPC call now properly checks the memory pool
+ before consulting the db (https://github.com/conformal/btcd/issues/26)
+ - Add support for the following RPC calls: getpeerinfo, getconnectedcount,
+ addnode, verifychain
+ (https://github.com/conformal/btcd/issues/13)
+ (https://github.com/conformal/btcd/issues/17)
+ - Implement rescan websocket extension to allow wallet rescans
+ - Use correct paths for application data storage for all supported
+ operating systems (https://github.com/conformal/btcd/issues/30)
+ - Add a default redirect to the http profiling page when accessing the
+ http profile server
+ - Add a new --cpuprofile option which can be used to generate CPU
+ profiling data on platforms that support it
+ - Several other minor performance optimizations
+ - Other minor bug fixes and general code cleanup
+
+Changes in 0.3.2 (Tue Oct 22 2013)
+ - Fix an issue that could cause the download of the block chain to stall
+ (https://github.com/conformal/btcd/issues/12)
+ - Remove deprecated sqlite as an available database backend
+ - Close sqlite compile issue as sqlite has now been removed
+ (https://github.com/conformal/btcd/issues/11)
+ - Change default RPC ports to 8334 (mainnet) and 18334 (testnet)
+ - Continue cleanup and work on implementing RPC API calls
+ - Add support for the following RPC calls: getrawmempool,
+ getbestblockhash, decoderawtransaction, getdifficulty,
+ getconnectioncount, getpeerinfo, and addnode
+ - Improve the btcctl utility that is used to issue JSON-RPC commands
+ - Fix an issue preventing btcd from cleanly shutting down with the RPC
+ stop command
+ - Add a number of database interface tests to ensure backends implement
+ the expected interface
+ - Expose some additional information from btcscript to be used for
+ identifying "standard"" transactions
+ - Add support for plan9 - thanks @mischief
+ (https://github.com/conformal/btcd/pull/19)
+ - Other minor bug fixes and general code cleanup
+
+Changes in 0.3.1-alpha (Tue Oct 15 2013)
+ - Change default database to leveldb
+ NOTE: This does mean you will have to redownload the block chain. Since we
+ are still in alpha, we didn't feel writing a converter was worth the time as
+ it would take away from more important issues at this stage
+ - Add a warning if there are multiple block chain databases of different types
+ - Fix issue with unexpected EOF in leveldb -- https://github.com/conformal/btcd/issues/18
+ - Fix issue preventing block 21066 on testnet -- https://github.com/conformal/btcchain/issues/1
+ - Fix issue preventing block 96464 on testnet -- https://github.com/conformal/btcscript/issues/1
+ - Optimize transaction lookups
+ - Correct a few cases of list removal that could result in improper cleanup
+ of no longer needed orphans
+ - Add functionality to increase ulimits on non-Windows platforms
+ - Add support for mempool command which allows remote peers to query the
+ transaction memory pool via the bitcoin protocol
+ - Clean up logging a bit
+ - Add a flag to disable checkpoints for developers
+ - Add a lot of useful debug logging such as message summaries
+ - Other minor bug fixes and general code cleanup
+
+Initial Release 0.3.0-alpha (Sat Oct 05 2013):
+ - Initial release
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000..3ee61ef
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,45 @@
+# This Dockerfile builds btcd from source and creates a small (55 MB) docker container based on alpine linux.
+#
+# Clone this repository and run the following command to build and tag a fresh btcd amd64 container:
+#
+# docker build . -t yourregistry/btcd
+#
+# You can use the following command to build an arm64v8 container:
+#
+# docker build . -t yourregistry/btcd --build-arg ARCH=arm64v8
+#
+# For more information how to use this docker image visit:
+# https://github.com/btcsuite/btcd/tree/master/docs
+#
+# 8333 Mainnet Bitcoin peer-to-peer port
+# 8334 Mainet RPC port
+
+ARG ARCH=amd64
+# using the SHA256 instead of tags
+# https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests
+# https://cloud.google.com/architecture/using-container-images
+# https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md
+# ➜ ~ crane digest golang:1.23.12-alpine3.21
+# sha256:4bb4be21ac98da06bc26437ee870c4973f8039f13e9a1a36971b4517632b0fc6
+FROM golang@sha256:4bb4be21ac98da06bc26437ee870c4973f8039f13e9a1a36971b4517632b0fc6 AS build-container
+
+ARG ARCH
+
+ADD . /app
+WORKDIR /app
+RUN set -ex \
+ && if [ "${ARCH}" = "amd64" ]; then export GOARCH=amd64; fi \
+ && if [ "${ARCH}" = "arm32v7" ]; then export GOARCH=arm; fi \
+ && if [ "${ARCH}" = "arm64v8" ]; then export GOARCH=arm64; fi \
+ && echo "Compiling for $GOARCH" \
+ && go install -v . ./cmd/...
+
+FROM $ARCH/alpine:3.21
+
+COPY --from=build-container /go/bin /bin
+
+VOLUME ["/root/.btcd"]
+
+EXPOSE 8333 8334
+
+ENTRYPOINT ["btcd"]
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..5eed085
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,16 @@
+ISC License
+
+Copyright (c) 2013-2025 The btcsuite developers
+Copyright (c) 2015-2016 The Decred developers
+
+Permission to use, copy, modify, and distribute this software for any
+purpose with or without fee is hereby granted, provided that the above
+copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..3825005
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,165 @@
+PKG := github.com/btcsuite/btcd
+
+LINT_PKG := github.com/golangci/golangci-lint/cmd/golangci-lint
+GOIMPORTS_PKG := golang.org/x/tools/cmd/goimports
+
+GO_BIN := ${GOPATH}/bin
+LINT_BIN := $(GO_BIN)/golangci-lint
+
+LINT_COMMIT := v1.18.0
+
+DEPGET := cd /tmp && go install -v
+GOBUILD := go build -v
+GOINSTALL := go install -v
+DEV_TAGS := rpctest
+GOTEST_DEV = go test -v -tags=$(DEV_TAGS)
+GOTEST := go test -v
+COVER_FLAGS = -coverprofile=coverage.txt -covermode=atomic -coverpkg=$(PKG)/...
+
+GOFILES_NOVENDOR = $(shell find . -type f -name '*.go' -not -path "./vendor/*")
+
+RM := rm -f
+CP := cp
+MAKE := make
+XARGS := xargs -L 1
+
+# Linting uses a lot of memory, so keep it under control by limiting the number
+# of workers if requested.
+ifneq ($(workers),)
+LINT_WORKERS = --concurrency=$(workers)
+endif
+
+LINT = $(LINT_BIN) run -v $(LINT_WORKERS)
+
+GREEN := "\\033[0;32m"
+NC := "\\033[0m"
+define print
+ echo $(GREEN)$1$(NC)
+endef
+
+#? default: Run `make build`
+default: build
+
+#? all: Run `make build` and `make check`
+all: build check
+
+# ============
+# DEPENDENCIES
+# ============
+
+$(LINT_BIN):
+ @$(call print, "Fetching linter")
+ $(DEPGET) $(LINT_PKG)@$(LINT_COMMIT)
+
+#? goimports: Install goimports
+goimports:
+ @$(call print, "Installing goimports.")
+ $(DEPGET) $(GOIMPORTS_PKG)
+
+# ============
+# INSTALLATION
+# ============
+
+#? build: Build all binaries, place them in project directory
+build:
+ @$(call print, "Building all binaries")
+ $(GOBUILD) $(PKG)
+ $(GOBUILD) $(PKG)/cmd/btcctl
+ $(GOBUILD) $(PKG)/cmd/gencerts
+ $(GOBUILD) $(PKG)/cmd/findcheckpoint
+ $(GOBUILD) $(PKG)/cmd/addblock
+
+#? install: Install all binaries, place them in $GOPATH/bin
+install:
+ @$(call print, "Installing all binaries")
+ $(GOINSTALL) $(PKG)
+ $(GOINSTALL) $(PKG)/cmd/btcctl
+ $(GOINSTALL) $(PKG)/cmd/gencerts
+ $(GOINSTALL) $(PKG)/cmd/findcheckpoint
+ $(GOINSTALL) $(PKG)/cmd/addblock
+
+#? release-install: Install btcd and btcctl release binaries, place them in $GOPATH/bin
+release-install:
+ @$(call print, "Installing btcd and btcctl release binaries")
+ env CGO_ENABLED=0 $(GOINSTALL) -trimpath -ldflags="-s -w -buildid=" $(PKG)
+ env CGO_ENABLED=0 $(GOINSTALL) -trimpath -ldflags="-s -w -buildid=" $(PKG)/cmd/btcctl
+
+# =======
+# TESTING
+# =======
+
+#? check: Run `make unit`
+check: unit
+
+#? unit: Run unit tests
+unit:
+ @$(call print, "Running unit tests.")
+ $(GOTEST_DEV) ./... -test.timeout=20m
+ cd btcec; $(GOTEST_DEV) ./... -test.timeout=20m
+ cd btcutil; $(GOTEST_DEV) ./... -test.timeout=20m
+ cd btcutil/psbt; $(GOTEST_DEV) ./... -test.timeout=20m
+
+#? unit-cover: Run unit coverage tests
+unit-cover:
+ @$(call print, "Running unit coverage tests.")
+ $(GOTEST) $(COVER_FLAGS) ./...
+ # We need to remove the /v2 pathing from the module to have it work
+ # nicely with the CI tool we use to render live code coverage.
+ cd btcec; $(GOTEST) $(COVER_FLAGS) ./...; \
+ sed -i.bak 's/v2\///g' coverage.txt
+
+ cd btcutil; $(GOTEST) $(COVER_FLAGS) ./...
+
+ cd btcutil/psbt; $(GOTEST) $(COVER_FLAGS) ./...
+
+#? unit-race: Run unit race tests
+unit-race:
+ @$(call print, "Running unit race tests.")
+ env CGO_ENABLED=1 GORACE="history_size=7 halt_on_errors=1" $(GOTEST) -race -test.timeout=20m ./...
+ cd btcec; env CGO_ENABLED=1 GORACE="history_size=7 halt_on_errors=1" $(GOTEST) -race -test.timeout=20m ./...
+ cd btcutil; env CGO_ENABLED=1 GORACE="history_size=7 halt_on_errors=1" $(GOTEST) -race -test.timeout=20m ./...
+ cd btcutil/psbt; env CGO_ENABLED=1 GORACE="history_size=7 halt_on_errors=1" $(GOTEST) -race -test.timeout=20m ./...
+
+# =========
+# UTILITIES
+# =========
+
+#? fmt: Fix imports and formatting source
+fmt: goimports
+ @$(call print, "Fixing imports.")
+ goimports -w $(GOFILES_NOVENDOR)
+ @$(call print, "Formatting source.")
+ gofmt -l -w -s $(GOFILES_NOVENDOR)
+
+#? lint: Lint source
+lint: $(LINT_BIN)
+ @$(call print, "Linting source.")
+ $(LINT)
+
+#? clean: Clean source
+clean:
+ @$(call print, "Cleaning source.$(NC)")
+ $(RM) coverage.txt btcec/coverage.txt btcutil/coverage.txt btcutil/psbt/coverage.txt
+
+#? tidy-module: Run 'go mod tidy' for all modules
+tidy-module:
+ echo "Running 'go mod tidy' for all modules"
+ scripts/tidy_modules.sh
+
+.PHONY: all \
+ default \
+ build \
+ check \
+ unit \
+ unit-cover \
+ unit-race \
+ fmt \
+ lint \
+ clean
+
+#? help: Get more info on make commands
+help: Makefile
+ @echo " Choose a command run in btcd:"
+ @sed -n 's/^#?//p' $< | column -t -s ':' | sort | sed -e 's/^/ /'
+
+.PHONY: help
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..b5d9146
--- /dev/null
+++ b/README.md
@@ -0,0 +1,121 @@
+btcd
+====
+
+[](https://github.com/btcsuite/btcd/actions)
+[](https://coveralls.io/github/btcsuite/btcd?branch=master)
+[](http://copyfree.org)
+[](https://pkg.go.dev/github.com/btcsuite/btcd)
+
+btcd is an alternative full node bitcoin implementation written in Go (golang).
+
+This project is currently under active development and is in a Beta state. It
+is extremely stable and has been in production use since October 2013.
+
+It properly downloads, validates, and serves the block chain using the exact
+rules (including consensus bugs) for block acceptance as Bitcoin Core. We have
+taken great care to avoid btcd causing a fork to the block chain. It includes a
+full block validation testing framework which contains all of the 'official'
+block acceptance tests (and some additional ones) that is run on every pull
+request to help ensure it properly follows consensus. Also, it passes all of
+the JSON test data in the Bitcoin Core code.
+
+It also properly relays newly mined blocks, maintains a transaction pool, and
+relays individual transactions that have not yet made it into a block. It
+ensures all individual transactions admitted to the pool follow the rules
+required by the block chain and also includes more strict checks which filter
+transactions based on miner requirements ("standard" transactions).
+
+One key difference between btcd and Bitcoin Core is that btcd does *NOT* include
+wallet functionality and this was a very intentional design decision. See the
+blog entry [here](https://web.archive.org/web/20171125143919/https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon)
+for more details. This means you can't actually make or receive payments
+directly with btcd. That functionality is provided by the
+[btcwallet](https://github.com/btcsuite/btcwallet) and
+[Paymetheus](https://github.com/btcsuite/Paymetheus) (Windows-only) projects
+which are both under active development.
+
+## Requirements
+
+[Go](http://golang.org) 1.22 or newer.
+
+## Installation
+
+https://github.com/btcsuite/btcd/releases
+
+#### Linux/BSD/MacOSX/POSIX - Build from Source
+
+- Install Go according to the installation instructions here:
+ http://golang.org/doc/install
+
+- Ensure Go was installed properly and is a supported version:
+
+```bash
+$ go version
+$ go env GOROOT GOPATH
+```
+
+NOTE: The `GOROOT` and `GOPATH` above must not be the same path. It is
+recommended that `GOPATH` is set to a directory in your home directory such as
+`~/goprojects` to avoid write permission issues. It is also recommended to add
+`$GOPATH/bin` to your `PATH` at this point.
+
+- Run the following commands to obtain btcd, all dependencies, and install it:
+
+```bash
+$ cd $GOPATH/src/github.com/btcsuite/btcd
+$ go install -v . ./cmd/...
+```
+
+- btcd (and utilities) will now be installed in ```$GOPATH/bin```. If you did
+ not already add the bin directory to your system path during Go installation,
+ we recommend you do so now.
+
+## Updating
+
+#### Linux/BSD/MacOSX/POSIX - Build from Source
+
+- Run the following commands to update btcd, all dependencies, and install it:
+
+```bash
+$ cd $GOPATH/src/github.com/btcsuite/btcd
+$ git pull
+$ go install -v . ./cmd/...
+```
+
+## Getting Started
+
+btcd has several configuration options available to tweak how it runs, but all
+of the basic operations described in the intro section work with zero
+configuration.
+
+#### Linux/BSD/POSIX/Source
+
+```bash
+$ ./btcd
+```
+
+## IRC
+
+- irc.libera.chat
+- channel #btcd
+- [webchat](https://web.libera.chat/gamja/?channels=btcd)
+
+## Issue Tracker
+
+The [integrated github issue tracker](https://github.com/btcsuite/btcd/issues)
+is used for this project.
+
+## Documentation
+
+The documentation is a work-in-progress. It is located in the [docs](https://github.com/btcsuite/btcd/tree/master/docs) folder.
+
+## Release Verification
+
+Please see our [documentation on the current build/verification
+process](https://github.com/btcsuite/btcd/tree/master/release) for all our
+releases for information on how to verify the integrity of published releases
+using our reproducible build system.
+
+## License
+
+btcd is licensed under the [copyfree](http://copyfree.org) ISC License.
diff --git a/SECURITY.md b/SECURITY.md
new file mode 100644
index 0000000..e06625c
--- /dev/null
+++ b/SECURITY.md
@@ -0,0 +1,15 @@
+# Security Policy
+
+## Supported Versions
+
+The last major `btcd` release is to be considered the current support version.
+Given an issue severe enough, a backport will be issued either to the prior
+major release or the set of releases considered utilized enough.
+
+## Reporting a Vulnerability
+
+To report security issues, send an email to security@lightning.engineering
+(this list isn't to be used for support).
+
+The following key can be used to communicate sensitive information: `91FE 464C
+D751 01DA 6B6B AB60 555C 6465 E5BC B3AF`.
diff --git a/addrmgr/addrmanager.go b/addrmgr/addrmanager.go
new file mode 100644
index 0000000..bdfe909
--- /dev/null
+++ b/addrmgr/addrmanager.go
@@ -0,0 +1,1218 @@
+// Copyright (c) 2013-2016 The btcsuite developers
+// Copyright (c) 2015-2018 The Decred developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr
+
+import (
+ "container/list"
+ crand "crypto/rand" // for seeding
+ "encoding/base32"
+ "encoding/binary"
+ "encoding/json"
+ "fmt"
+ "io"
+ "math/rand"
+ "net"
+ "os"
+ "path/filepath"
+ "strconv"
+ "strings"
+ "sync"
+ "sync/atomic"
+ "time"
+
+ "github.com/btcsuite/btcd/chaincfg/chainhash"
+ "github.com/btcsuite/btcd/wire"
+)
+
+// AddrManager provides a concurrency safe address manager for caching potential
+// peers on the bitcoin network.
+type AddrManager struct {
+ mtx sync.RWMutex
+ peersFile string
+ lookupFunc func(string) ([]net.IP, error)
+ rand *rand.Rand
+ key [32]byte
+ addrIndex map[string]*KnownAddress // address key to ka for all addrs.
+ addrNew [newBucketCount]map[string]*KnownAddress
+ addrTried [triedBucketCount]*list.List
+ started int32
+ shutdown int32
+ wg sync.WaitGroup
+ quit chan struct{}
+ nTried int
+ nNew int
+ lamtx sync.Mutex
+ localAddresses map[string]*localAddress
+ version int
+}
+
+type serializedKnownAddress struct {
+ Addr string
+ Src string
+ Attempts int
+ TimeStamp int64
+ LastAttempt int64
+ LastSuccess int64
+ Services wire.ServiceFlag
+ SrcServices wire.ServiceFlag
+ // no refcount or tried, that is available from context.
+}
+
+type serializedAddrManager struct {
+ Version int
+ Key [32]byte
+ Addresses []*serializedKnownAddress
+ NewBuckets [newBucketCount][]string // string is NetAddressKey
+ TriedBuckets [triedBucketCount][]string
+}
+
+type localAddress struct {
+ na *wire.NetAddressV2
+ score AddressPriority
+}
+
+// AddressPriority type is used to describe the hierarchy of local address
+// discovery methods.
+type AddressPriority int
+
+const (
+ // InterfacePrio signifies the address is on a local interface
+ InterfacePrio AddressPriority = iota
+
+ // BoundPrio signifies the address has been explicitly bounded to.
+ BoundPrio
+
+ // UpnpPrio signifies the address was obtained from UPnP.
+ UpnpPrio
+
+ // HTTPPrio signifies the address was obtained from an external HTTP service.
+ HTTPPrio
+
+ // ManualPrio signifies the address was provided by --externalip.
+ ManualPrio
+)
+
+const (
+ // needAddressThreshold is the number of addresses under which the
+ // address manager will claim to need more addresses.
+ needAddressThreshold = 1000
+
+ // dumpAddressInterval is the interval used to dump the address
+ // cache to disk for future use.
+ dumpAddressInterval = time.Minute * 10
+
+ // triedBucketSize is the maximum number of addresses in each
+ // tried address bucket.
+ triedBucketSize = 256
+
+ // triedBucketCount is the number of buckets we split tried
+ // addresses over.
+ triedBucketCount = 64
+
+ // newBucketSize is the maximum number of addresses in each new address
+ // bucket.
+ newBucketSize = 64
+
+ // newBucketCount is the number of buckets that we spread new addresses
+ // over.
+ newBucketCount = 1024
+
+ // triedBucketsPerGroup is the number of tried buckets over which an
+ // address group will be spread.
+ triedBucketsPerGroup = 8
+
+ // newBucketsPerGroup is the number of new buckets over which an
+ // source address group will be spread.
+ newBucketsPerGroup = 64
+
+ // newBucketsPerAddress is the number of buckets a frequently seen new
+ // address may end up in.
+ newBucketsPerAddress = 8
+
+ // numMissingDays is the number of days before which we assume an
+ // address has vanished if we have not seen it announced in that long.
+ numMissingDays = 30
+
+ // numRetries is the number of tried without a single success before
+ // we assume an address is bad.
+ numRetries = 3
+
+ // maxFailures is the maximum number of failures we will accept without
+ // a success before considering an address bad.
+ maxFailures = 10
+
+ // minBadDays is the number of days since the last success before we
+ // will consider evicting an address.
+ minBadDays = 7
+
+ // getAddrMax is the most addresses that we will send in response
+ // to a getAddr (in practise the most addresses we will return from a
+ // call to AddressCache()).
+ getAddrMax = 2500
+
+ // getAddrPercent is the percentage of total addresses known that we
+ // will share with a call to AddressCache.
+ getAddrPercent = 23
+
+ // serialisationVersion is the current version of the on-disk format.
+ serialisationVersion = 2
+)
+
+// updateAddress is a helper function to either update an address already known
+// to the address manager, or to add the address if not already known.
+func (a *AddrManager) updateAddress(netAddr, srcAddr *wire.NetAddressV2) {
+ // Filter out non-routable addresses. Note that non-routable
+ // also includes invalid and local addresses.
+ if !IsRoutable(netAddr) {
+ return
+ }
+
+ addr := NetAddressKey(netAddr)
+ ka := a.find(netAddr)
+ if ka != nil {
+ // TODO: only update addresses periodically.
+ // Update the last seen time and services.
+ // note that to prevent causing excess garbage on getaddr
+ // messages the netaddresses in addrmanager are *immutable*,
+ // if we need to change them then we replace the pointer with a
+ // new copy so that we don't have to copy every na for getaddr.
+ if netAddr.Timestamp.After(ka.na.Timestamp) ||
+ (ka.na.Services&netAddr.Services) !=
+ netAddr.Services {
+
+ naCopy := *ka.na
+ naCopy.Timestamp = netAddr.Timestamp
+ naCopy.AddService(netAddr.Services)
+ ka.mtx.Lock()
+ ka.na = &naCopy
+ ka.mtx.Unlock()
+ }
+
+ // If already in tried, we have nothing to do here.
+ if ka.tried {
+ return
+ }
+
+ // Already at our max?
+ if ka.refs == newBucketsPerAddress {
+ return
+ }
+
+ // The more entries we have, the less likely we are to add more.
+ // likelihood is 2N.
+ factor := int32(2 * ka.refs)
+ if a.rand.Int31n(factor) != 0 {
+ return
+ }
+ } else {
+ // Make a copy of the net address to avoid races since it is
+ // updated elsewhere in the addrmanager code and would otherwise
+ // change the actual netaddress on the peer.
+ netAddrCopy := *netAddr
+ ka = &KnownAddress{na: &netAddrCopy, srcAddr: srcAddr}
+ a.addrIndex[addr] = ka
+ a.nNew++
+ // XXX time penalty?
+ }
+
+ bucket := a.getNewBucket(netAddr, srcAddr)
+
+ // Already exists?
+ if _, ok := a.addrNew[bucket][addr]; ok {
+ return
+ }
+
+ // Enforce max addresses.
+ if len(a.addrNew[bucket]) > newBucketSize {
+ log.Tracef("new bucket is full, expiring old")
+ a.expireNew(bucket)
+ }
+
+ // Add to new bucket.
+ ka.refs++
+ a.addrNew[bucket][addr] = ka
+
+ log.Tracef("Added new address %s for a total of %d addresses", addr,
+ a.nTried+a.nNew)
+}
+
+// expireNew makes space in the new buckets by expiring the really bad entries.
+// If no bad entries are available we look at a few and remove the oldest.
+func (a *AddrManager) expireNew(bucket int) {
+ // First see if there are any entries that are so bad we can just throw
+ // them away. otherwise we throw away the oldest entry in the cache.
+ // Bitcoind here chooses four random and just throws the oldest of
+ // those away, but we keep track of oldest in the initial traversal and
+ // use that information instead.
+ var oldest *KnownAddress
+ for k, v := range a.addrNew[bucket] {
+ if v.isBad() {
+ log.Tracef("expiring bad address %v", k)
+ delete(a.addrNew[bucket], k)
+ v.refs--
+ if v.refs == 0 {
+ a.nNew--
+ delete(a.addrIndex, k)
+ }
+ continue
+ }
+ if oldest == nil {
+ oldest = v
+ } else if !v.na.Timestamp.After(oldest.na.Timestamp) {
+ oldest = v
+ }
+ }
+
+ if oldest != nil {
+ key := NetAddressKey(oldest.na)
+ log.Tracef("expiring oldest address %v", key)
+
+ delete(a.addrNew[bucket], key)
+ oldest.refs--
+ if oldest.refs == 0 {
+ a.nNew--
+ delete(a.addrIndex, key)
+ }
+ }
+}
+
+// pickTried selects an address from the tried bucket to be evicted.
+// We just choose the eldest. Bitcoind selects 4 random entries and throws away
+// the older of them.
+func (a *AddrManager) pickTried(bucket int) *list.Element {
+ var oldest *KnownAddress
+ var oldestElem *list.Element
+ for e := a.addrTried[bucket].Front(); e != nil; e = e.Next() {
+ ka := e.Value.(*KnownAddress)
+ if oldest == nil || oldest.na.Timestamp.After(ka.na.Timestamp) {
+ oldestElem = e
+ oldest = ka
+ }
+
+ }
+ return oldestElem
+}
+
+func (a *AddrManager) getNewBucket(netAddr, srcAddr *wire.NetAddressV2) int {
+ // bitcoind:
+ // doublesha256(key + sourcegroup + int64(doublesha256(key + group + sourcegroup))%bucket_per_source_group) % num_new_buckets
+
+ data1 := []byte{}
+ data1 = append(data1, a.key[:]...)
+ data1 = append(data1, []byte(GroupKey(netAddr))...)
+ data1 = append(data1, []byte(GroupKey(srcAddr))...)
+ hash1 := chainhash.DoubleHashB(data1)
+ hash64 := binary.LittleEndian.Uint64(hash1)
+ hash64 %= newBucketsPerGroup
+ var hashbuf [8]byte
+ binary.LittleEndian.PutUint64(hashbuf[:], hash64)
+ data2 := []byte{}
+ data2 = append(data2, a.key[:]...)
+ data2 = append(data2, GroupKey(srcAddr)...)
+ data2 = append(data2, hashbuf[:]...)
+
+ hash2 := chainhash.DoubleHashB(data2)
+ return int(binary.LittleEndian.Uint64(hash2) % newBucketCount)
+}
+
+func (a *AddrManager) getTriedBucket(netAddr *wire.NetAddressV2) int {
+ // bitcoind hashes this as:
+ // doublesha256(key + group + truncate_to_64bits(doublesha256(key)) % buckets_per_group) % num_buckets
+ data1 := []byte{}
+ data1 = append(data1, a.key[:]...)
+ data1 = append(data1, []byte(NetAddressKey(netAddr))...)
+ hash1 := chainhash.DoubleHashB(data1)
+ hash64 := binary.LittleEndian.Uint64(hash1)
+ hash64 %= triedBucketsPerGroup
+ var hashbuf [8]byte
+ binary.LittleEndian.PutUint64(hashbuf[:], hash64)
+ data2 := []byte{}
+ data2 = append(data2, a.key[:]...)
+ data2 = append(data2, GroupKey(netAddr)...)
+ data2 = append(data2, hashbuf[:]...)
+
+ hash2 := chainhash.DoubleHashB(data2)
+ return int(binary.LittleEndian.Uint64(hash2) % triedBucketCount)
+}
+
+// addressHandler is the main handler for the address manager. It must be run
+// as a goroutine.
+func (a *AddrManager) addressHandler() {
+ dumpAddressTicker := time.NewTicker(dumpAddressInterval)
+ defer dumpAddressTicker.Stop()
+out:
+ for {
+ select {
+ case <-dumpAddressTicker.C:
+ a.savePeers()
+
+ case <-a.quit:
+ break out
+ }
+ }
+ a.savePeers()
+ a.wg.Done()
+ log.Trace("Address handler done")
+}
+
+// savePeers saves all the known addresses to a file so they can be read back
+// in at next run.
+func (a *AddrManager) savePeers() {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ // First we make a serialisable datastructure so we can encode it to
+ // json.
+ sam := new(serializedAddrManager)
+ sam.Version = a.version
+ copy(sam.Key[:], a.key[:])
+
+ sam.Addresses = make([]*serializedKnownAddress, len(a.addrIndex))
+ i := 0
+ for k, v := range a.addrIndex {
+ ska := new(serializedKnownAddress)
+ ska.Addr = k
+ ska.TimeStamp = v.na.Timestamp.Unix()
+ ska.Src = NetAddressKey(v.srcAddr)
+ ska.Attempts = v.attempts
+ ska.LastAttempt = v.lastattempt.Unix()
+ ska.LastSuccess = v.lastsuccess.Unix()
+ if a.version > 1 {
+ ska.Services = v.na.Services
+ ska.SrcServices = v.srcAddr.Services
+ }
+ // Tried and refs are implicit in the rest of the structure
+ // and will be worked out from context on unserialisation.
+ sam.Addresses[i] = ska
+ i++
+ }
+ for i := range a.addrNew {
+ sam.NewBuckets[i] = make([]string, len(a.addrNew[i]))
+ j := 0
+ for k := range a.addrNew[i] {
+ sam.NewBuckets[i][j] = k
+ j++
+ }
+ }
+ for i := range a.addrTried {
+ sam.TriedBuckets[i] = make([]string, a.addrTried[i].Len())
+ j := 0
+ for e := a.addrTried[i].Front(); e != nil; e = e.Next() {
+ ka := e.Value.(*KnownAddress)
+ sam.TriedBuckets[i][j] = NetAddressKey(ka.na)
+ j++
+ }
+ }
+
+ w, err := os.Create(a.peersFile)
+ if err != nil {
+ log.Errorf("Error opening file %s: %v", a.peersFile, err)
+ return
+ }
+ enc := json.NewEncoder(w)
+ defer w.Close()
+ if err := enc.Encode(&sam); err != nil {
+ log.Errorf("Failed to encode file %s: %v", a.peersFile, err)
+ return
+ }
+}
+
+// loadPeers loads the known address from the saved file. If empty, missing, or
+// malformed file, just don't load anything and start fresh
+func (a *AddrManager) loadPeers() {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ err := a.deserializePeers(a.peersFile)
+ if err != nil {
+ log.Errorf("Failed to parse file %s: %v", a.peersFile, err)
+ // if it is invalid we nuke the old one unconditionally.
+ err = os.Remove(a.peersFile)
+ if err != nil {
+ log.Warnf("Failed to remove corrupt peers file %s: %v",
+ a.peersFile, err)
+ }
+ a.reset()
+ return
+ }
+ log.Infof("Loaded %d addresses from file '%s'", a.numAddresses(), a.peersFile)
+}
+
+func (a *AddrManager) deserializePeers(filePath string) error {
+
+ _, err := os.Stat(filePath)
+ if os.IsNotExist(err) {
+ return nil
+ }
+ r, err := os.Open(filePath)
+ if err != nil {
+ return fmt.Errorf("%s error opening file: %v", filePath, err)
+ }
+ defer r.Close()
+
+ var sam serializedAddrManager
+ dec := json.NewDecoder(r)
+ err = dec.Decode(&sam)
+ if err != nil {
+ return fmt.Errorf("error reading %s: %v", filePath, err)
+ }
+
+ // Since decoding JSON is backwards compatible (i.e., only decodes
+ // fields it understands), we'll only return an error upon seeing a
+ // version past our latest supported version.
+ if sam.Version > serialisationVersion {
+ return fmt.Errorf("unknown version %v in serialized "+
+ "addrmanager", sam.Version)
+ }
+
+ copy(a.key[:], sam.Key[:])
+
+ for _, v := range sam.Addresses {
+ ka := new(KnownAddress)
+
+ // The first version of the serialized address manager was not
+ // aware of the service bits associated with this address, so
+ // we'll assign a default of SFNodeNetwork to it.
+ if sam.Version == 1 {
+ v.Services = wire.SFNodeNetwork
+ }
+ ka.na, err = a.DeserializeNetAddress(v.Addr, v.Services)
+ if err != nil {
+ return fmt.Errorf("failed to deserialize netaddress "+
+ "%s: %v", v.Addr, err)
+ }
+
+ // The first version of the serialized address manager was not
+ // aware of the service bits associated with the source address,
+ // so we'll assign a default of SFNodeNetwork to it.
+ if sam.Version == 1 {
+ v.SrcServices = wire.SFNodeNetwork
+ }
+ ka.srcAddr, err = a.DeserializeNetAddress(v.Src, v.SrcServices)
+ if err != nil {
+ return fmt.Errorf("failed to deserialize netaddress "+
+ "%s: %v", v.Src, err)
+ }
+
+ ka.attempts = v.Attempts
+ ka.lastattempt = time.Unix(v.LastAttempt, 0)
+ ka.lastsuccess = time.Unix(v.LastSuccess, 0)
+ a.addrIndex[NetAddressKey(ka.na)] = ka
+ }
+
+ for i := range sam.NewBuckets {
+ for _, val := range sam.NewBuckets[i] {
+ ka, ok := a.addrIndex[val]
+ if !ok {
+ return fmt.Errorf("newbucket contains %s but "+
+ "none in address list", val)
+ }
+
+ if ka.refs == 0 {
+ a.nNew++
+ }
+ ka.refs++
+ a.addrNew[i][val] = ka
+ }
+ }
+ for i := range sam.TriedBuckets {
+ for _, val := range sam.TriedBuckets[i] {
+ ka, ok := a.addrIndex[val]
+ if !ok {
+ return fmt.Errorf("Newbucket contains %s but "+
+ "none in address list", val)
+ }
+
+ ka.tried = true
+ a.nTried++
+ a.addrTried[i].PushBack(ka)
+ }
+ }
+
+ // Sanity checking.
+ for k, v := range a.addrIndex {
+ if v.refs == 0 && !v.tried {
+ return fmt.Errorf("address %s after serialisation "+
+ "with no references", k)
+ }
+
+ if v.refs > 0 && v.tried {
+ return fmt.Errorf("address %s after serialisation "+
+ "which is both new and tried!", k)
+ }
+ }
+
+ return nil
+}
+
+// DeserializeNetAddress converts a given address string to a *wire.NetAddress.
+func (a *AddrManager) DeserializeNetAddress(addr string,
+ services wire.ServiceFlag) (*wire.NetAddressV2, error) {
+
+ host, portStr, err := net.SplitHostPort(addr)
+ if err != nil {
+ return nil, err
+ }
+ port, err := strconv.ParseUint(portStr, 10, 16)
+ if err != nil {
+ return nil, err
+ }
+
+ return a.HostToNetAddress(host, uint16(port), services)
+}
+
+// Start begins the core address handler which manages a pool of known
+// addresses, timeouts, and interval based writes.
+func (a *AddrManager) Start() {
+ // Already started?
+ if atomic.AddInt32(&a.started, 1) != 1 {
+ return
+ }
+
+ log.Trace("Starting address manager")
+
+ // Load peers we already know about from file.
+ a.loadPeers()
+
+ // Start the address ticker to save addresses periodically.
+ a.wg.Add(1)
+ go a.addressHandler()
+}
+
+// Stop gracefully shuts down the address manager by stopping the main handler.
+func (a *AddrManager) Stop() error {
+ if atomic.AddInt32(&a.shutdown, 1) != 1 {
+ log.Warnf("Address manager is already in the process of " +
+ "shutting down")
+ return nil
+ }
+
+ log.Infof("Address manager shutting down")
+ close(a.quit)
+ a.wg.Wait()
+ return nil
+}
+
+// AddAddresses adds new addresses to the address manager. It enforces a max
+// number of addresses and silently ignores duplicate addresses. It is
+// safe for concurrent access.
+func (a *AddrManager) AddAddresses(addrs []*wire.NetAddressV2, srcAddr *wire.NetAddressV2) {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ for _, na := range addrs {
+ a.updateAddress(na, srcAddr)
+ }
+}
+
+// AddAddress adds a new address to the address manager. It enforces a max
+// number of addresses and silently ignores duplicate addresses. It is
+// safe for concurrent access.
+func (a *AddrManager) AddAddress(addr, srcAddr *wire.NetAddressV2) {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ a.updateAddress(addr, srcAddr)
+}
+
+// AddAddressByIP adds an address where we are given an ip:port and not a
+// wire.NetAddress.
+func (a *AddrManager) AddAddressByIP(addrIP string) error {
+ // Split IP and port
+ addr, portStr, err := net.SplitHostPort(addrIP)
+ if err != nil {
+ return err
+ }
+ // Put it in wire.Netaddress
+ ip := net.ParseIP(addr)
+ if ip == nil {
+ return fmt.Errorf("invalid ip address %s", addr)
+ }
+ port, err := strconv.ParseUint(portStr, 10, 0)
+ if err != nil {
+ return fmt.Errorf("invalid port %s: %v", portStr, err)
+ }
+ na := wire.NetAddressV2FromBytes(time.Now(), 0, ip, uint16(port))
+ a.AddAddress(na, na) // XXX use correct src address
+ return nil
+}
+
+// NumAddresses returns the number of addresses known to the address manager.
+func (a *AddrManager) numAddresses() int {
+ return a.nTried + a.nNew
+}
+
+// NumAddresses returns the number of addresses known to the address manager.
+func (a *AddrManager) NumAddresses() int {
+ a.mtx.RLock()
+ defer a.mtx.RUnlock()
+
+ return a.numAddresses()
+}
+
+// NeedMoreAddresses returns whether or not the address manager needs more
+// addresses.
+func (a *AddrManager) NeedMoreAddresses() bool {
+ a.mtx.RLock()
+ defer a.mtx.RUnlock()
+
+ return a.numAddresses() < needAddressThreshold
+}
+
+// AddressCache returns the current address cache. It must be treated as
+// read-only (but since it is a copy now, this is not as dangerous).
+func (a *AddrManager) AddressCache() []*wire.NetAddressV2 {
+ allAddr := a.getAddresses()
+
+ numAddresses := len(allAddr) * getAddrPercent / 100
+ if numAddresses > getAddrMax {
+ numAddresses = getAddrMax
+ }
+
+ // Fisher-Yates shuffle the array. We only need to do the first
+ // `numAddresses' since we are throwing the rest.
+ for i := 0; i < numAddresses; i++ {
+ // pick a number between current index and the end
+ j := rand.Intn(len(allAddr)-i) + i
+ allAddr[i], allAddr[j] = allAddr[j], allAddr[i]
+ }
+
+ // slice off the limit we are willing to share.
+ return allAddr[0:numAddresses]
+}
+
+// getAddresses returns all of the addresses currently found within the
+// manager's address cache.
+func (a *AddrManager) getAddresses() []*wire.NetAddressV2 {
+ a.mtx.RLock()
+ defer a.mtx.RUnlock()
+
+ addrIndexLen := len(a.addrIndex)
+ if addrIndexLen == 0 {
+ return nil
+ }
+
+ addrs := make([]*wire.NetAddressV2, 0, addrIndexLen)
+ for _, v := range a.addrIndex {
+ addrs = append(addrs, v.na)
+ }
+
+ return addrs
+}
+
+// reset resets the address manager by reinitialising the random source
+// and allocating fresh empty bucket storage.
+func (a *AddrManager) reset() {
+
+ a.addrIndex = make(map[string]*KnownAddress)
+
+ // fill key with bytes from a good random source.
+ io.ReadFull(crand.Reader, a.key[:])
+ for i := range a.addrNew {
+ a.addrNew[i] = make(map[string]*KnownAddress)
+ }
+ for i := range a.addrTried {
+ a.addrTried[i] = list.New()
+ }
+}
+
+// HostToNetAddress returns a netaddress given a host address. If the address
+// is a Tor .onion address this will be taken care of. Else if the host is
+// not an IP address it will be resolved (via Tor if required).
+func (a *AddrManager) HostToNetAddress(host string, port uint16,
+ services wire.ServiceFlag) (*wire.NetAddressV2, error) {
+
+ var (
+ na *wire.NetAddressV2
+ ip net.IP
+ )
+
+ // Tor v2 address is 16 char base32 + ".onion"
+ if len(host) == wire.TorV2EncodedSize && host[wire.TorV2EncodedSize-6:] == ".onion" {
+ // go base32 encoding uses capitals (as does the rfc
+ // but Tor and bitcoind tend to user lowercase, so we switch
+ // case here.
+ data, err := base32.StdEncoding.DecodeString(
+ strings.ToUpper(host[:wire.TorV2EncodedSize-6]))
+ if err != nil {
+ return nil, err
+ }
+
+ na = wire.NetAddressV2FromBytes(
+ time.Now(), services, data, port,
+ )
+ } else if len(host) == wire.TorV3EncodedSize && host[wire.TorV3EncodedSize-6:] == ".onion" {
+ // Tor v3 addresses are 56 base32 characters with the 6 byte
+ // onion suffix.
+ data, err := base32.StdEncoding.DecodeString(
+ strings.ToUpper(host[:wire.TorV3EncodedSize-6]),
+ )
+ if err != nil {
+ return nil, err
+ }
+
+ // The first 32 bytes is the ed25519 public key and is enough
+ // to reconstruct the .onion address.
+ na = wire.NetAddressV2FromBytes(
+ time.Now(), services, data[:wire.TorV3Size], port,
+ )
+ } else if ip = net.ParseIP(host); ip == nil {
+ ips, err := a.lookupFunc(host)
+ if err != nil {
+ return nil, err
+ }
+ if len(ips) == 0 {
+ return nil, fmt.Errorf("no addresses found for %s", host)
+ }
+ ip = ips[0]
+
+ na = wire.NetAddressV2FromBytes(time.Now(), services, ip, port)
+ } else {
+ // This is an non-nil IP address that was parsed in the else if
+ // above.
+ na = wire.NetAddressV2FromBytes(time.Now(), services, ip, port)
+ }
+
+ return na, nil
+}
+
+// NetAddressKey returns a string key in the form of ip:port for IPv4 addresses
+// or [ip]:port for IPv6 addresses. It also handles onion v2 and v3 addresses.
+func NetAddressKey(na *wire.NetAddressV2) string {
+ port := strconv.FormatUint(uint64(na.Port), 10)
+
+ return net.JoinHostPort(na.Addr.String(), port)
+}
+
+// GetAddress returns a single address that should be routable. It picks a
+// random one from the possible addresses with preference given to ones that
+// have not been used recently and should not pick 'close' addresses
+// consecutively.
+func (a *AddrManager) GetAddress() *KnownAddress {
+ // Protect concurrent access.
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ if a.numAddresses() == 0 {
+ return nil
+ }
+
+ // Use a 50% chance for choosing between tried and new table entries.
+ if a.nTried > 0 && (a.nNew == 0 || a.rand.Intn(2) == 0) {
+ // Tried entry.
+ large := 1 << 30
+ factor := 1.0
+ for {
+ // pick a random bucket.
+ bucket := a.rand.Intn(len(a.addrTried))
+ if a.addrTried[bucket].Len() == 0 {
+ continue
+ }
+
+ // Pick a random entry in the list
+ e := a.addrTried[bucket].Front()
+ for i :=
+ a.rand.Int63n(int64(a.addrTried[bucket].Len())); i > 0; i-- {
+ e = e.Next()
+ }
+ ka := e.Value.(*KnownAddress)
+ randval := a.rand.Intn(large)
+ if float64(randval) < (factor * ka.chance() * float64(large)) {
+ log.Tracef("Selected %v from tried bucket",
+ NetAddressKey(ka.na))
+ return ka
+ }
+ factor *= 1.2
+ }
+ } else {
+ // new node.
+ // XXX use a closure/function to avoid repeating this.
+ large := 1 << 30
+ factor := 1.0
+ for {
+ // Pick a random bucket.
+ bucket := a.rand.Intn(len(a.addrNew))
+ if len(a.addrNew[bucket]) == 0 {
+ continue
+ }
+ // Then, a random entry in it.
+ var ka *KnownAddress
+ nth := a.rand.Intn(len(a.addrNew[bucket]))
+ for _, value := range a.addrNew[bucket] {
+ if nth == 0 {
+ ka = value
+ }
+ nth--
+ }
+ randval := a.rand.Intn(large)
+ if float64(randval) < (factor * ka.chance() * float64(large)) {
+ log.Tracef("Selected %v from new bucket",
+ NetAddressKey(ka.na))
+ return ka
+ }
+ factor *= 1.2
+ }
+ }
+}
+
+func (a *AddrManager) find(addr *wire.NetAddressV2) *KnownAddress {
+ return a.addrIndex[NetAddressKey(addr)]
+}
+
+// Attempt increases the given address' attempt counter and updates
+// the last attempt time.
+func (a *AddrManager) Attempt(addr *wire.NetAddressV2) {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ // find address.
+ // Surely address will be in tried by now?
+ ka := a.find(addr)
+ if ka == nil {
+ return
+ }
+ // set last tried time to now
+ now := time.Now()
+ ka.mtx.Lock()
+ ka.attempts++
+ ka.lastattempt = now
+ ka.mtx.Unlock()
+}
+
+// Connected Marks the given address as currently connected and working at the
+// current time. The address must already be known to AddrManager else it will
+// be ignored.
+func (a *AddrManager) Connected(addr *wire.NetAddressV2) {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ ka := a.find(addr)
+ if ka == nil {
+ return
+ }
+
+ // Update the time as long as it has been 20 minutes since last we did
+ // so.
+ now := time.Now()
+ if now.After(ka.na.Timestamp.Add(time.Minute * 20)) {
+ // ka.na is immutable, so replace it.
+ naCopy := *ka.na
+ naCopy.Timestamp = time.Now()
+ ka.mtx.Lock()
+ ka.na = &naCopy
+ ka.mtx.Unlock()
+ }
+}
+
+// Good marks the given address as good. To be called after a successful
+// connection and version exchange. If the address is unknown to the address
+// manager it will be ignored.
+func (a *AddrManager) Good(addr *wire.NetAddressV2) {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ ka := a.find(addr)
+ if ka == nil {
+ return
+ }
+
+ // ka.Timestamp is not updated here to avoid leaking information
+ // about currently connected peers.
+ now := time.Now()
+ ka.mtx.Lock()
+ ka.lastsuccess = now
+ ka.lastattempt = now
+ ka.attempts = 0
+ ka.mtx.Unlock() // tried and refs synchronized via a.mtx
+
+ // move to tried set, optionally evicting other addresses if need.
+ if ka.tried {
+ return
+ }
+
+ // ok, need to move it to tried.
+
+ // remove from all new buckets.
+ // record one of the buckets in question and call it the `first'
+ addrKey := NetAddressKey(addr)
+ oldBucket := -1
+ for i := range a.addrNew {
+ // we check for existence so we can record the first one
+ if _, ok := a.addrNew[i][addrKey]; ok {
+ delete(a.addrNew[i], addrKey)
+ ka.refs--
+ if oldBucket == -1 {
+ oldBucket = i
+ }
+ }
+ }
+ a.nNew--
+
+ if oldBucket == -1 {
+ // What? wasn't in a bucket after all.... Panic?
+ return
+ }
+
+ bucket := a.getTriedBucket(ka.na)
+
+ // Room in this tried bucket?
+ if a.addrTried[bucket].Len() < triedBucketSize {
+ ka.tried = true
+ a.addrTried[bucket].PushBack(ka)
+ a.nTried++
+ return
+ }
+
+ // No room, we have to evict something else.
+ entry := a.pickTried(bucket)
+ rmka := entry.Value.(*KnownAddress)
+
+ // First bucket it would have been put in.
+ newBucket := a.getNewBucket(rmka.na, rmka.srcAddr)
+
+ // If no room in the original bucket, we put it in a bucket we just
+ // freed up a space in.
+ if len(a.addrNew[newBucket]) >= newBucketSize {
+ newBucket = oldBucket
+ }
+
+ // replace with ka in list.
+ ka.tried = true
+ entry.Value = ka
+
+ rmka.tried = false
+ rmka.refs++
+
+ // We don't touch a.nTried here since the number of tried stays the same
+ // but we decemented new above, raise it again since we're putting
+ // something back.
+ a.nNew++
+
+ rmkey := NetAddressKey(rmka.na)
+ log.Tracef("Replacing %s with %s in tried", rmkey, addrKey)
+
+ // We made sure there is space here just above.
+ a.addrNew[newBucket][rmkey] = rmka
+}
+
+// SetServices sets the services for the giiven address to the provided value.
+func (a *AddrManager) SetServices(addr *wire.NetAddressV2, services wire.ServiceFlag) {
+ a.mtx.Lock()
+ defer a.mtx.Unlock()
+
+ ka := a.find(addr)
+ if ka == nil {
+ return
+ }
+
+ // Update the services if needed.
+ if ka.na.Services != services {
+ // ka.na is immutable, so replace it.
+ naCopy := *ka.na
+ naCopy.Services = services
+ ka.mtx.Lock()
+ ka.na = &naCopy
+ ka.mtx.Unlock()
+ }
+}
+
+// AddLocalAddress adds na to the list of known local addresses to advertise
+// with the given priority.
+func (a *AddrManager) AddLocalAddress(na *wire.NetAddressV2, priority AddressPriority) error {
+ if !IsRoutable(na) {
+ return fmt.Errorf(
+ "address %s is not routable", na.Addr.String(),
+ )
+ }
+
+ a.lamtx.Lock()
+ defer a.lamtx.Unlock()
+
+ key := NetAddressKey(na)
+ la, ok := a.localAddresses[key]
+ if !ok || la.score < priority {
+ if ok {
+ la.score = priority + 1
+ } else {
+ a.localAddresses[key] = &localAddress{
+ na: na,
+ score: priority,
+ }
+ }
+ }
+ return nil
+}
+
+// getReachabilityFrom returns the relative reachability of the provided local
+// address to the provided remote address.
+func getReachabilityFrom(localAddr, remoteAddr *wire.NetAddressV2) int {
+ const (
+ Unreachable = 0
+ Default = iota
+ Teredo
+ Ipv6Weak
+ Ipv4
+ Ipv6Strong
+ Private
+ )
+
+ if !IsRoutable(remoteAddr) {
+ return Unreachable
+ }
+
+ if remoteAddr.IsTorV3() {
+ if localAddr.IsTorV3() {
+ return Private
+ }
+
+ lna := localAddr.ToLegacy()
+ if IsOnionCatTor(lna) {
+ // Modern v3 clients should not be able to connect to
+ // deprecated v2 hidden services.
+ return Unreachable
+ }
+
+ if IsRoutable(localAddr) && IsIPv4(lna) {
+ return Ipv4
+ }
+
+ return Default
+ }
+
+ // We can't be sure if the remote party can actually connect to this
+ // address or not.
+ if localAddr.IsTorV3() {
+ return Default
+ }
+
+ // Convert the V2 addresses into legacy to access the network
+ // functions.
+ remoteLna := remoteAddr.ToLegacy()
+ localLna := localAddr.ToLegacy()
+
+ if IsOnionCatTor(remoteLna) {
+ if IsOnionCatTor(localLna) {
+ return Private
+ }
+
+ if IsRoutable(localAddr) && IsIPv4(localLna) {
+ return Ipv4
+ }
+
+ return Default
+ }
+
+ if IsRFC4380(remoteLna) {
+ if !IsRoutable(localAddr) {
+ return Default
+ }
+
+ if IsRFC4380(localLna) {
+ return Teredo
+ }
+
+ if IsIPv4(localLna) {
+ return Ipv4
+ }
+
+ return Ipv6Weak
+ }
+
+ if IsIPv4(remoteLna) {
+ if IsRoutable(localAddr) && IsIPv4(localLna) {
+ return Ipv4
+ }
+ return Unreachable
+ }
+
+ /* ipv6 */
+ var tunnelled bool
+ // Is our v6 is tunnelled?
+ if IsRFC3964(localLna) || IsRFC6052(localLna) || IsRFC6145(localLna) {
+ tunnelled = true
+ }
+
+ if !IsRoutable(localAddr) {
+ return Default
+ }
+
+ if IsRFC4380(localLna) {
+ return Teredo
+ }
+
+ if IsIPv4(localLna) {
+ return Ipv4
+ }
+
+ if tunnelled {
+ // only prioritise ipv6 if we aren't tunnelling it.
+ return Ipv6Weak
+ }
+
+ return Ipv6Strong
+}
+
+// GetBestLocalAddress returns the most appropriate local address to use
+// for the given remote address.
+func (a *AddrManager) GetBestLocalAddress(remoteAddr *wire.NetAddressV2) *wire.NetAddressV2 {
+ a.lamtx.Lock()
+ defer a.lamtx.Unlock()
+
+ bestreach := 0
+ var bestscore AddressPriority
+ var bestAddress *wire.NetAddressV2
+ for _, la := range a.localAddresses {
+ reach := getReachabilityFrom(la.na, remoteAddr)
+ if reach > bestreach ||
+ (reach == bestreach && la.score > bestscore) {
+ bestreach = reach
+ bestscore = la.score
+ bestAddress = la.na
+ }
+ }
+ if bestAddress != nil {
+ log.Debugf("Suggesting address %s:%d for %s:%d",
+ bestAddress.Addr.String(), bestAddress.Port,
+ remoteAddr.Addr.String(), remoteAddr.Port)
+ } else {
+ log.Debugf("No worthy address for %s:%d",
+ remoteAddr.Addr.String(), remoteAddr.Port)
+
+ // Send something unroutable if nothing suitable.
+ var ip net.IP
+ if remoteAddr.IsTorV3() {
+ ip = net.IPv4zero
+ } else {
+ remoteLna := remoteAddr.ToLegacy()
+ if !IsIPv4(remoteLna) && !IsOnionCatTor(remoteLna) {
+ ip = net.IPv6zero
+ } else {
+ ip = net.IPv4zero
+ }
+ }
+ services := wire.SFNodeNetwork | wire.SFNodeWitness | wire.SFNodeBloom
+ bestAddress = wire.NetAddressV2FromBytes(
+ time.Now(), services, ip, 0,
+ )
+ }
+
+ return bestAddress
+}
+
+// New returns a new bitcoin address manager.
+// Use Start to begin processing asynchronous address updates.
+func New(dataDir string, lookupFunc func(string) ([]net.IP, error)) *AddrManager {
+ am := AddrManager{
+ peersFile: filepath.Join(dataDir, "peers.json"),
+ lookupFunc: lookupFunc,
+ rand: rand.New(rand.NewSource(time.Now().UnixNano())),
+ quit: make(chan struct{}),
+ localAddresses: make(map[string]*localAddress),
+ version: serialisationVersion,
+ }
+ am.reset()
+ return &am
+}
diff --git a/addrmgr/addrmanager_internal_test.go b/addrmgr/addrmanager_internal_test.go
new file mode 100644
index 0000000..a4ed50b
--- /dev/null
+++ b/addrmgr/addrmanager_internal_test.go
@@ -0,0 +1,207 @@
+package addrmgr
+
+import (
+ "math/rand"
+ "net"
+ "testing"
+ "time"
+
+ "github.com/btcsuite/btcd/wire"
+)
+
+// randAddr generates a *wire.NetAddressV2 backed by a random IPv4/IPv6
+// address. Some of the returned addresses may not be routable.
+func randAddr(t *testing.T) *wire.NetAddressV2 {
+ t.Helper()
+
+ ipv4 := rand.Intn(2) == 0
+ var ip net.IP
+ if ipv4 {
+ var b [4]byte
+ if _, err := rand.Read(b[:]); err != nil {
+ t.Fatal(err)
+ }
+ ip = b[:]
+ } else {
+ var b [16]byte
+ if _, err := rand.Read(b[:]); err != nil {
+ t.Fatal(err)
+ }
+ ip = b[:]
+ }
+
+ services := wire.ServiceFlag(rand.Uint64())
+ port := uint16(rand.Uint32())
+
+ return wire.NetAddressV2FromBytes(
+ time.Now(), services, ip, port,
+ )
+}
+
+// routableRandAddr generates a *wire.NetAddressV2 backed by a random IPv4/IPv6
+// address that is always routable.
+func routableRandAddr(t *testing.T) *wire.NetAddressV2 {
+ t.Helper()
+
+ var addr *wire.NetAddressV2
+
+ // If the address is not routable, try again.
+ routable := false
+ for !routable {
+ addr = randAddr(t)
+ routable = IsRoutable(addr)
+ }
+
+ return addr
+}
+
+// assertAddr ensures that the two addresses match. The timestamp is not
+// checked as it does not affect uniquely identifying a specific address.
+func assertAddr(t *testing.T, got, expected *wire.NetAddressV2) {
+ if got.Services != expected.Services {
+ t.Fatalf("expected address services %v, got %v",
+ expected.Services, got.Services)
+ }
+ gotAddr := got.Addr.String()
+ expectedAddr := expected.Addr.String()
+ if gotAddr != expectedAddr {
+ t.Fatalf("expected address IP %v, got %v", expectedAddr,
+ gotAddr)
+ }
+ if got.Port != expected.Port {
+ t.Fatalf("expected address port %d, got %d", expected.Port,
+ got.Port)
+ }
+}
+
+// assertAddrs ensures that the manager's address cache matches the given
+// expected addresses.
+func assertAddrs(t *testing.T, addrMgr *AddrManager,
+ expectedAddrs map[string]*wire.NetAddressV2) {
+
+ t.Helper()
+
+ addrs := addrMgr.getAddresses()
+
+ if len(addrs) != len(expectedAddrs) {
+ t.Fatalf("expected to find %d addresses, found %d",
+ len(expectedAddrs), len(addrs))
+ }
+
+ for _, addr := range addrs {
+ addrStr := NetAddressKey(addr)
+ expectedAddr, ok := expectedAddrs[addrStr]
+ if !ok {
+ t.Fatalf("expected to find address %v", addrStr)
+ }
+
+ assertAddr(t, addr, expectedAddr)
+ }
+}
+
+// TestAddrManagerSerialization ensures that we can properly serialize and
+// deserialize the manager's current address cache.
+func TestAddrManagerSerialization(t *testing.T) {
+ t.Parallel()
+
+ // We'll start by creating our address manager backed by a temporary
+ // directory.
+ tempDir := t.TempDir()
+
+ addrMgr := New(tempDir, nil)
+
+ // We'll be adding 5 random addresses to the manager.
+ const numAddrs = 5
+
+ expectedAddrs := make(map[string]*wire.NetAddressV2, numAddrs)
+ for i := 0; i < numAddrs; i++ {
+ addr := routableRandAddr(t)
+ expectedAddrs[NetAddressKey(addr)] = addr
+ addrMgr.AddAddress(addr, routableRandAddr(t))
+ }
+
+ // Now that the addresses have been added, we should be able to retrieve
+ // them.
+ assertAddrs(t, addrMgr, expectedAddrs)
+
+ // Then, we'll persist these addresses to disk and restart the address
+ // manager.
+ addrMgr.savePeers()
+ addrMgr = New(tempDir, nil)
+
+ // Finally, we'll read all of the addresses from disk and ensure they
+ // match as expected.
+ addrMgr.loadPeers()
+ assertAddrs(t, addrMgr, expectedAddrs)
+}
+
+// TestAddrManagerV1ToV2 ensures that we can properly upgrade the serialized
+// version of the address manager from v1 to v2.
+func TestAddrManagerV1ToV2(t *testing.T) {
+ t.Parallel()
+
+ // We'll start by creating our address manager backed by a temporary
+ // directory.
+ tempDir := t.TempDir()
+
+ addrMgr := New(tempDir, nil)
+
+ // As we're interested in testing the upgrade path from v1 to v2, we'll
+ // override the manager's current version.
+ addrMgr.version = 1
+
+ // We'll be adding 5 random addresses to the manager. Since this is v1,
+ // each addresses' services will not be stored.
+ const numAddrs = 5
+
+ expectedAddrs := make(map[string]*wire.NetAddressV2, numAddrs)
+ for i := 0; i < numAddrs; i++ {
+ addr := routableRandAddr(t)
+ expectedAddrs[NetAddressKey(addr)] = addr
+ addrMgr.AddAddress(addr, routableRandAddr(t))
+ }
+
+ // Then, we'll persist these addresses to disk and restart the address
+ // manager - overriding its version back to v1.
+ addrMgr.savePeers()
+ addrMgr = New(tempDir, nil)
+ addrMgr.version = 1
+
+ // When we read all of the addresses back from disk, we should expect to
+ // find all of them, but their services will be set to a default of
+ // SFNodeNetwork since they were not previously stored. After ensuring
+ // that this default is set, we'll override each addresses' services
+ // with the original value from when they were created.
+ addrMgr.loadPeers()
+ addrs := addrMgr.getAddresses()
+ if len(addrs) != len(expectedAddrs) {
+ t.Fatalf("expected to find %d addresses, found %d",
+ len(expectedAddrs), len(addrs))
+ }
+ for _, addr := range addrs {
+ addrStr := NetAddressKey(addr)
+ expectedAddr, ok := expectedAddrs[addrStr]
+ if !ok {
+ t.Fatalf("expected to find address %v", addrStr)
+ }
+
+ if addr.Services != wire.SFNodeNetwork {
+ t.Fatalf("expected address services to be %v, got %v",
+ wire.SFNodeNetwork, addr.Services)
+ }
+
+ addrMgr.SetServices(addr, expectedAddr.Services)
+ }
+
+ // We'll also bump up the manager's version to v2, which should signal
+ // that it should include the address services when persisting its
+ // state.
+ addrMgr.version = 2
+ addrMgr.savePeers()
+
+ // Finally, we'll recreate the manager and ensure that the services were
+ // persisted correctly.
+ addrMgr = New(tempDir, nil)
+ addrMgr.loadPeers()
+ assertAddrs(t, addrMgr, expectedAddrs)
+}
diff --git a/addrmgr/addrmanager_test.go b/addrmgr/addrmanager_test.go
new file mode 100644
index 0000000..4afe5fd
--- /dev/null
+++ b/addrmgr/addrmanager_test.go
@@ -0,0 +1,541 @@
+// Copyright (c) 2013-2014 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr_test
+
+import (
+ "errors"
+ "fmt"
+ "net"
+ "reflect"
+ "testing"
+ "time"
+
+ "github.com/btcsuite/btcd/addrmgr"
+ "github.com/btcsuite/btcd/wire"
+)
+
+// naTest is used to describe a test to be performed against the NetAddressKey
+// method.
+type naTest struct {
+ in wire.NetAddressV2
+ want string
+}
+
+// naTests houses all of the tests to be performed against the NetAddressKey
+// method.
+var naTests = make([]naTest, 0)
+
+// Put some IP in here for convenience. Points to google.
+var someIP = "173.194.115.66"
+
+// addNaTests
+func addNaTests() {
+ // IPv4
+ // Localhost
+ addNaTest("127.0.0.1", 8333, "127.0.0.1:8333")
+ addNaTest("127.0.0.1", 8334, "127.0.0.1:8334")
+
+ // Class A
+ addNaTest("1.0.0.1", 8333, "1.0.0.1:8333")
+ addNaTest("2.2.2.2", 8334, "2.2.2.2:8334")
+ addNaTest("27.253.252.251", 8335, "27.253.252.251:8335")
+ addNaTest("123.3.2.1", 8336, "123.3.2.1:8336")
+
+ // Private Class A
+ addNaTest("10.0.0.1", 8333, "10.0.0.1:8333")
+ addNaTest("10.1.1.1", 8334, "10.1.1.1:8334")
+ addNaTest("10.2.2.2", 8335, "10.2.2.2:8335")
+ addNaTest("10.10.10.10", 8336, "10.10.10.10:8336")
+
+ // Class B
+ addNaTest("128.0.0.1", 8333, "128.0.0.1:8333")
+ addNaTest("129.1.1.1", 8334, "129.1.1.1:8334")
+ addNaTest("180.2.2.2", 8335, "180.2.2.2:8335")
+ addNaTest("191.10.10.10", 8336, "191.10.10.10:8336")
+
+ // Private Class B
+ addNaTest("172.16.0.1", 8333, "172.16.0.1:8333")
+ addNaTest("172.16.1.1", 8334, "172.16.1.1:8334")
+ addNaTest("172.16.2.2", 8335, "172.16.2.2:8335")
+ addNaTest("172.16.172.172", 8336, "172.16.172.172:8336")
+
+ // Class C
+ addNaTest("193.0.0.1", 8333, "193.0.0.1:8333")
+ addNaTest("200.1.1.1", 8334, "200.1.1.1:8334")
+ addNaTest("205.2.2.2", 8335, "205.2.2.2:8335")
+ addNaTest("223.10.10.10", 8336, "223.10.10.10:8336")
+
+ // Private Class C
+ addNaTest("192.168.0.1", 8333, "192.168.0.1:8333")
+ addNaTest("192.168.1.1", 8334, "192.168.1.1:8334")
+ addNaTest("192.168.2.2", 8335, "192.168.2.2:8335")
+ addNaTest("192.168.192.192", 8336, "192.168.192.192:8336")
+
+ // IPv6
+ // Localhost
+ addNaTest("::1", 8333, "[::1]:8333")
+ addNaTest("fe80::1", 8334, "[fe80::1]:8334")
+
+ // Link-local
+ addNaTest("fe80::1:1", 8333, "[fe80::1:1]:8333")
+ addNaTest("fe91::2:2", 8334, "[fe91::2:2]:8334")
+ addNaTest("fea2::3:3", 8335, "[fea2::3:3]:8335")
+ addNaTest("feb3::4:4", 8336, "[feb3::4:4]:8336")
+
+ // Site-local
+ addNaTest("fec0::1:1", 8333, "[fec0::1:1]:8333")
+ addNaTest("fed1::2:2", 8334, "[fed1::2:2]:8334")
+ addNaTest("fee2::3:3", 8335, "[fee2::3:3]:8335")
+ addNaTest("fef3::4:4", 8336, "[fef3::4:4]:8336")
+}
+
+func addNaTest(ip string, port uint16, want string) {
+ nip := net.ParseIP(ip)
+ na := wire.NetAddressV2FromBytes(
+ time.Now(), wire.SFNodeNetwork, nip, port,
+ )
+ test := naTest{*na, want}
+ naTests = append(naTests, test)
+}
+
+func lookupFunc(host string) ([]net.IP, error) {
+ return nil, errors.New("not implemented")
+}
+
+func TestStartStop(t *testing.T) {
+ n := addrmgr.New("teststartstop", lookupFunc)
+ n.Start()
+ err := n.Stop()
+ if err != nil {
+ t.Fatalf("Address Manager failed to stop: %v", err)
+ }
+}
+
+func TestAddAddressByIP(t *testing.T) {
+ fmtErr := fmt.Errorf("")
+ addrErr := &net.AddrError{}
+ var tests = []struct {
+ addrIP string
+ err error
+ }{
+ {
+ someIP + ":8333",
+ nil,
+ },
+ {
+ someIP,
+ addrErr,
+ },
+ {
+ someIP[:12] + ":8333",
+ fmtErr,
+ },
+ {
+ someIP + ":abcd",
+ fmtErr,
+ },
+ }
+
+ amgr := addrmgr.New("testaddressbyip", nil)
+ for i, test := range tests {
+ err := amgr.AddAddressByIP(test.addrIP)
+ if test.err != nil && err == nil {
+ t.Errorf("TestGood test %d failed expected an error and got none", i)
+ continue
+ }
+ if test.err == nil && err != nil {
+ t.Errorf("TestGood test %d failed expected no error and got one", i)
+ continue
+ }
+ if reflect.TypeOf(err) != reflect.TypeOf(test.err) {
+ t.Errorf("TestGood test %d failed got %v, want %v", i,
+ reflect.TypeOf(err), reflect.TypeOf(test.err))
+ continue
+ }
+ }
+}
+
+func TestAddLocalAddress(t *testing.T) {
+ var tests = []struct {
+ address wire.NetAddressV2
+ priority addrmgr.AddressPriority
+ valid bool
+ }{
+ {
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("192.168.0.100"), 0,
+ ),
+ addrmgr.InterfacePrio,
+ false,
+ },
+ {
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("204.124.1.1"), 0,
+ ),
+ addrmgr.InterfacePrio,
+ true,
+ },
+ {
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("204.124.1.1"), 0,
+ ),
+ addrmgr.BoundPrio,
+ true,
+ },
+ {
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("::1"), 0,
+ ),
+ addrmgr.InterfacePrio,
+ false,
+ },
+ {
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("fe80::1"), 0,
+ ),
+ addrmgr.InterfacePrio,
+ false,
+ },
+ {
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("2620:100::1"), 0,
+ ),
+ addrmgr.InterfacePrio,
+ true,
+ },
+ }
+ amgr := addrmgr.New("testaddlocaladdress", nil)
+ for x, test := range tests {
+ result := amgr.AddLocalAddress(&test.address, test.priority)
+ if result == nil && !test.valid {
+ t.Errorf("TestAddLocalAddress test #%d failed: %s should have "+
+ "been accepted", x, test.address.Addr.String())
+ continue
+ }
+ if result != nil && test.valid {
+ t.Errorf("TestAddLocalAddress test #%d failed: %s should not have "+
+ "been accepted", x, test.address.Addr.String())
+ continue
+ }
+ }
+}
+
+func TestAttempt(t *testing.T) {
+ n := addrmgr.New("testattempt", lookupFunc)
+
+ // Add a new address and get it
+ err := n.AddAddressByIP(someIP + ":8333")
+ if err != nil {
+ t.Fatalf("Adding address failed: %v", err)
+ }
+ ka := n.GetAddress()
+
+ if !ka.LastAttempt().IsZero() {
+ t.Errorf("Address should not have attempts, but does")
+ }
+
+ na := ka.NetAddress()
+ n.Attempt(na)
+
+ if ka.LastAttempt().IsZero() {
+ t.Errorf("Address should have an attempt, but does not")
+ }
+}
+
+func TestConnected(t *testing.T) {
+ n := addrmgr.New("testconnected", lookupFunc)
+
+ // Add a new address and get it
+ err := n.AddAddressByIP(someIP + ":8333")
+ if err != nil {
+ t.Fatalf("Adding address failed: %v", err)
+ }
+ ka := n.GetAddress()
+ na := ka.NetAddress()
+ // make it an hour ago
+ na.Timestamp = time.Unix(time.Now().Add(time.Hour*-1).Unix(), 0)
+
+ n.Connected(na)
+
+ if !ka.NetAddress().Timestamp.After(na.Timestamp) {
+ t.Errorf("Address should have a new timestamp, but does not")
+ }
+}
+
+func TestNeedMoreAddresses(t *testing.T) {
+ n := addrmgr.New("testneedmoreaddresses", lookupFunc)
+ addrsToAdd := 1500
+ b := n.NeedMoreAddresses()
+ if !b {
+ t.Errorf("Expected that we need more addresses")
+ }
+ addrs := make([]*wire.NetAddressV2, addrsToAdd)
+
+ var err error
+ for i := 0; i < addrsToAdd; i++ {
+ s := fmt.Sprintf("%d.%d.173.147:8333", i/128+60, i%128+60)
+ addrs[i], err = n.DeserializeNetAddress(s, wire.SFNodeNetwork)
+ if err != nil {
+ t.Errorf("Failed to turn %s into an address: %v", s, err)
+ }
+ }
+
+ srcAddr := wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4(173, 144, 173, 111), 8333,
+ )
+
+ n.AddAddresses(addrs, srcAddr)
+ numAddrs := n.NumAddresses()
+ if numAddrs > addrsToAdd {
+ t.Errorf("Number of addresses is too many %d vs %d", numAddrs, addrsToAdd)
+ }
+
+ b = n.NeedMoreAddresses()
+ if b {
+ t.Errorf("Expected that we don't need more addresses")
+ }
+}
+
+func TestGood(t *testing.T) {
+ n := addrmgr.New("testgood", lookupFunc)
+ addrsToAdd := 64 * 64
+ addrs := make([]*wire.NetAddressV2, addrsToAdd)
+
+ var err error
+ for i := 0; i < addrsToAdd; i++ {
+ s := fmt.Sprintf("%d.173.147.%d:8333", i/64+60, i%64+60)
+ addrs[i], err = n.DeserializeNetAddress(s, wire.SFNodeNetwork)
+ if err != nil {
+ t.Errorf("Failed to turn %s into an address: %v", s, err)
+ }
+ }
+
+ srcAddr := wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4(173, 144, 173, 111), 8333,
+ )
+
+ n.AddAddresses(addrs, srcAddr)
+ for _, addr := range addrs {
+ n.Good(addr)
+ }
+
+ numAddrs := n.NumAddresses()
+ if numAddrs >= addrsToAdd {
+ t.Errorf("Number of addresses is too many: %d vs %d", numAddrs, addrsToAdd)
+ }
+
+ numCache := len(n.AddressCache())
+ if numCache >= numAddrs/4 {
+ t.Errorf("Number of addresses in cache: got %d, want %d", numCache, numAddrs/4)
+ }
+}
+
+func TestGetAddress(t *testing.T) {
+ n := addrmgr.New("testgetaddress", lookupFunc)
+
+ // Get an address from an empty set (should error)
+ if rv := n.GetAddress(); rv != nil {
+ t.Errorf("GetAddress failed: got: %v want: %v\n", rv, nil)
+ }
+
+ // Add a new address and get it
+ err := n.AddAddressByIP(someIP + ":8333")
+ if err != nil {
+ t.Fatalf("Adding address failed: %v", err)
+ }
+ ka := n.GetAddress()
+ if ka == nil {
+ t.Fatalf("Did not get an address where there is one in the pool")
+ }
+ if ka.NetAddress().Addr.String() != someIP {
+ t.Errorf("Wrong IP: got %v, want %v", ka.NetAddress().Addr.String(), someIP)
+ }
+
+ // Mark this as a good address and get it
+ n.Good(ka.NetAddress())
+ ka = n.GetAddress()
+ if ka == nil {
+ t.Fatalf("Did not get an address where there is one in the pool")
+ }
+ if ka.NetAddress().Addr.String() != someIP {
+ t.Errorf("Wrong IP: got %v, want %v", ka.NetAddress().Addr.String(), someIP)
+ }
+
+ numAddrs := n.NumAddresses()
+ if numAddrs != 1 {
+ t.Errorf("Wrong number of addresses: got %d, want %d", numAddrs, 1)
+ }
+}
+
+func TestGetBestLocalAddress(t *testing.T) {
+ localAddrs := []wire.NetAddressV2{
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("192.168.0.100"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("::1"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("fe80::1"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("2001:470::1"), 0,
+ ),
+ }
+
+ var tests = []struct {
+ remoteAddr wire.NetAddressV2
+ want0 wire.NetAddressV2
+ want1 wire.NetAddressV2
+ want2 wire.NetAddressV2
+ want3 wire.NetAddressV2
+ }{
+ {
+ // Remote connection from public IPv4
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("204.124.8.1"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4zero, 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4zero, 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("204.124.8.100"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0,
+ net.ParseIP("fd87:d87e:eb43:25::1"), 0,
+ ),
+ },
+ {
+ // Remote connection from private IPv4
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("172.16.0.254"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4zero, 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4zero, 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4zero, 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv4zero, 0,
+ ),
+ },
+ {
+ // Remote connection from public IPv6
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0,
+ net.ParseIP("2602:100:abcd::102"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.IPv6zero, 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("2001:470::1"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("2001:470::1"), 0,
+ ),
+ *wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("2001:470::1"), 0,
+ ),
+ },
+ /* XXX
+ {
+ // Remote connection from Tor
+ wire.NetAddress{IP: net.ParseIP("fd87:d87e:eb43::100")},
+ wire.NetAddress{IP: net.IPv4zero},
+ wire.NetAddress{IP: net.ParseIP("204.124.8.100")},
+ wire.NetAddress{IP: net.ParseIP("fd87:d87e:eb43:25::1")},
+ },
+ */
+ }
+
+ amgr := addrmgr.New("testgetbestlocaladdress", nil)
+
+ // Test against default when there's no address
+ for x, test := range tests {
+ got := amgr.GetBestLocalAddress(&test.remoteAddr)
+ wantAddr := test.want0.Addr.String()
+ gotAddr := got.Addr.String()
+ if wantAddr != gotAddr {
+ remoteAddr := test.remoteAddr.Addr.String()
+ t.Errorf("TestGetBestLocalAddress test1 #%d failed for remote address %s: want %s got %s",
+ x, remoteAddr, wantAddr, gotAddr)
+ continue
+ }
+ }
+
+ for _, localAddr := range localAddrs {
+ amgr.AddLocalAddress(&localAddr, addrmgr.InterfacePrio)
+ }
+
+ // Test against want1
+ for x, test := range tests {
+ got := amgr.GetBestLocalAddress(&test.remoteAddr)
+ wantAddr := test.want1.Addr.String()
+ gotAddr := got.Addr.String()
+ if wantAddr != gotAddr {
+ remoteAddr := test.remoteAddr.Addr.String()
+ t.Errorf("TestGetBestLocalAddress test1 #%d failed for remote address %s: want %s got %s",
+ x, remoteAddr, wantAddr, gotAddr)
+ continue
+ }
+ }
+
+ // Add a public IP to the list of local addresses.
+ localAddr := wire.NetAddressV2FromBytes(
+ time.Now(), 0, net.ParseIP("204.124.8.100"), 0,
+ )
+ amgr.AddLocalAddress(localAddr, addrmgr.InterfacePrio)
+
+ // Test against want2
+ for x, test := range tests {
+ got := amgr.GetBestLocalAddress(&test.remoteAddr)
+ wantAddr := test.want2.Addr.String()
+ gotAddr := got.Addr.String()
+ if wantAddr != gotAddr {
+ remoteAddr := test.remoteAddr.Addr.String()
+ t.Errorf("TestGetBestLocalAddress test2 #%d failed for remote address %s: want %s got %s",
+ x, remoteAddr, wantAddr, gotAddr)
+ continue
+ }
+ }
+ /*
+ // Add a Tor generated IP address
+ localAddr = wire.NetAddress{IP: net.ParseIP("fd87:d87e:eb43:25::1")}
+ amgr.AddLocalAddress(&localAddr, addrmgr.ManualPrio)
+
+ // Test against want3
+ for x, test := range tests {
+ got := amgr.GetBestLocalAddress(&test.remoteAddr)
+ if !test.want3.IP.Equal(got.IP) {
+ t.Errorf("TestGetBestLocalAddress test3 #%d failed for remote address %s: want %s got %s",
+ x, test.remoteAddr.IP, test.want3.IP, got.IP)
+ continue
+ }
+ }
+ */
+}
+
+func TestNetAddressKey(t *testing.T) {
+ addNaTests()
+
+ t.Logf("Running %d tests", len(naTests))
+ for i, test := range naTests {
+ key := addrmgr.NetAddressKey(&test.in)
+ if key != test.want {
+ t.Errorf("NetAddressKey #%d\n got: %s want: %s", i, key, test.want)
+ continue
+ }
+ }
+
+}
diff --git a/addrmgr/cov_report.sh b/addrmgr/cov_report.sh
new file mode 100755
index 0000000..e41c928
--- /dev/null
+++ b/addrmgr/cov_report.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+# This script uses the standard Go test coverage tools to generate a test coverage report.
+
+# Run tests with coverage enabled and generate coverage profile.
+go test -cover -coverprofile=coverage.txt ./...
+
+# Display function-level coverage statistics.
+go tool cover -func=coverage.txt
diff --git a/addrmgr/doc.go b/addrmgr/doc.go
new file mode 100644
index 0000000..c500fbb
--- /dev/null
+++ b/addrmgr/doc.go
@@ -0,0 +1,38 @@
+// Copyright (c) 2014 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+/*
+Package addrmgr implements concurrency safe Bitcoin address manager.
+
+# Address Manager Overview
+
+In order maintain the peer-to-peer Bitcoin network, there needs to be a source
+of addresses to connect to as nodes come and go. The Bitcoin protocol provides
+the getaddr and addr messages to allow peers to communicate known addresses with
+each other. However, there needs to a mechanism to store those results and
+select peers from them. It is also important to note that remote peers can't
+be trusted to send valid peers nor attempt to provide you with only peers they
+control with malicious intent.
+
+With that in mind, this package provides a concurrency safe address manager for
+caching and selecting peers in a non-deterministic manner. The general idea is
+the caller adds addresses to the address manager and notifies it when addresses
+are connected, known good, and attempted. The caller also requests addresses as
+it needs them.
+
+The address manager internally segregates the addresses into groups and
+non-deterministically selects groups in a cryptographically random manner. This
+reduce the chances multiple addresses from the same nets are selected which
+generally helps provide greater peer diversity, and perhaps more importantly,
+drastically reduces the chances an attacker is able to coerce your peer into
+only connecting to nodes they control.
+
+The address manager also understands routability and Tor addresses and tries
+hard to only return routable addresses. In addition, it uses the information
+provided by the caller about connected, known good, and attempted addresses to
+periodically purge peers which no longer appear to be good peers as well as
+bias the selection toward known good peers. The general idea is to make a best
+effort at only providing usable addresses.
+*/
+package addrmgr
diff --git a/addrmgr/internal_test.go b/addrmgr/internal_test.go
new file mode 100644
index 0000000..ab7644b
--- /dev/null
+++ b/addrmgr/internal_test.go
@@ -0,0 +1,25 @@
+// Copyright (c) 2013-2015 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr
+
+import (
+ "time"
+
+ "github.com/btcsuite/btcd/wire"
+)
+
+func TstKnownAddressIsBad(ka *KnownAddress) bool {
+ return ka.isBad()
+}
+
+func TstKnownAddressChance(ka *KnownAddress) float64 {
+ return ka.chance()
+}
+
+func TstNewKnownAddress(na *wire.NetAddressV2, attempts int,
+ lastattempt, lastsuccess time.Time, tried bool, refs int) *KnownAddress {
+ return &KnownAddress{na: na, attempts: attempts, lastattempt: lastattempt,
+ lastsuccess: lastsuccess, tried: tried, refs: refs}
+}
diff --git a/addrmgr/knownaddress.go b/addrmgr/knownaddress.go
new file mode 100644
index 0000000..b045365
--- /dev/null
+++ b/addrmgr/knownaddress.go
@@ -0,0 +1,113 @@
+// Copyright (c) 2013-2014 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr
+
+import (
+ "sync"
+ "time"
+
+ "github.com/btcsuite/btcd/wire"
+)
+
+// KnownAddress tracks information about a known network address that is used
+// to determine how viable an address is.
+type KnownAddress struct {
+ mtx sync.RWMutex // na and lastattempt
+ na *wire.NetAddressV2
+ srcAddr *wire.NetAddressV2
+ attempts int
+ lastattempt time.Time
+ lastsuccess time.Time
+ tried bool
+ refs int // reference count of new buckets
+}
+
+// NetAddress returns the underlying wire.NetAddressV2 associated with the
+// known address.
+func (ka *KnownAddress) NetAddress() *wire.NetAddressV2 {
+ ka.mtx.RLock()
+ defer ka.mtx.RUnlock()
+ return ka.na
+}
+
+// LastAttempt returns the last time the known address was attempted.
+func (ka *KnownAddress) LastAttempt() time.Time {
+ ka.mtx.RLock()
+ defer ka.mtx.RUnlock()
+ return ka.lastattempt
+}
+
+// Services returns the services supported by the peer with the known address.
+func (ka *KnownAddress) Services() wire.ServiceFlag {
+ ka.mtx.RLock()
+ defer ka.mtx.RUnlock()
+ return ka.na.Services
+}
+
+// The unexported methods, chance and isBad, are used from within AddrManager
+// where KnownAddress field access is synchronized via it's own Mutex.
+
+// chance returns the selection probability for a known address. The priority
+// depends upon how recently the address has been seen, how recently it was last
+// attempted and how often attempts to connect to it have failed.
+func (ka *KnownAddress) chance() float64 {
+ now := time.Now()
+ lastAttempt := now.Sub(ka.lastattempt)
+
+ if lastAttempt < 0 {
+ lastAttempt = 0
+ }
+
+ c := 1.0
+
+ // Very recent attempts are less likely to be retried.
+ if lastAttempt < 10*time.Minute {
+ c *= 0.01
+ }
+
+ // Failed attempts deprioritise.
+ for i := ka.attempts; i > 0; i-- {
+ c /= 1.5
+ }
+
+ return c
+}
+
+// isBad returns true if the address in question has not been tried in the last
+// minute and meets one of the following criteria:
+// 1) It claims to be from the future
+// 2) It hasn't been seen in over a month
+// 3) It has failed at least three times and never succeeded
+// 4) It has failed ten times in the last week
+// All addresses that meet these criteria are assumed to be worthless and not
+// worth keeping hold of.
+func (ka *KnownAddress) isBad() bool {
+ if ka.lastattempt.After(time.Now().Add(-1 * time.Minute)) {
+ return false
+ }
+
+ // From the future?
+ if ka.na.Timestamp.After(time.Now().Add(10 * time.Minute)) {
+ return true
+ }
+
+ // Over a month old?
+ if ka.na.Timestamp.Before(time.Now().Add(-1 * numMissingDays * time.Hour * 24)) {
+ return true
+ }
+
+ // Never succeeded?
+ if ka.lastsuccess.IsZero() && ka.attempts >= numRetries {
+ return true
+ }
+
+ // Hasn't succeeded in too long?
+ if !ka.lastsuccess.After(time.Now().Add(-1*minBadDays*time.Hour*24)) &&
+ ka.attempts >= maxFailures {
+ return true
+ }
+
+ return false
+}
diff --git a/addrmgr/knownaddress_test.go b/addrmgr/knownaddress_test.go
new file mode 100644
index 0000000..b4a2650
--- /dev/null
+++ b/addrmgr/knownaddress_test.go
@@ -0,0 +1,114 @@
+// Copyright (c) 2013-2015 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr_test
+
+import (
+ "math"
+ "testing"
+ "time"
+
+ "github.com/btcsuite/btcd/addrmgr"
+ "github.com/btcsuite/btcd/wire"
+)
+
+func TestChance(t *testing.T) {
+ now := time.Unix(time.Now().Unix(), 0)
+ var tests = []struct {
+ addr *addrmgr.KnownAddress
+ expected float64
+ }{
+ {
+ //Test normal case
+ addrmgr.TstNewKnownAddress(&wire.NetAddressV2{Timestamp: now.Add(-35 * time.Second)},
+ 0, time.Now().Add(-30*time.Minute), time.Now(), false, 0),
+ 1.0,
+ }, {
+ //Test case in which lastseen < 0
+ addrmgr.TstNewKnownAddress(&wire.NetAddressV2{Timestamp: now.Add(20 * time.Second)},
+ 0, time.Now().Add(-30*time.Minute), time.Now(), false, 0),
+ 1.0,
+ }, {
+ //Test case in which lastattempt < 0
+ addrmgr.TstNewKnownAddress(&wire.NetAddressV2{Timestamp: now.Add(-35 * time.Second)},
+ 0, time.Now().Add(30*time.Minute), time.Now(), false, 0),
+ 1.0 * .01,
+ }, {
+ //Test case in which lastattempt < ten minutes
+ addrmgr.TstNewKnownAddress(&wire.NetAddressV2{Timestamp: now.Add(-35 * time.Second)},
+ 0, time.Now().Add(-5*time.Minute), time.Now(), false, 0),
+ 1.0 * .01,
+ }, {
+ //Test case with several failed attempts.
+ addrmgr.TstNewKnownAddress(&wire.NetAddressV2{Timestamp: now.Add(-35 * time.Second)},
+ 2, time.Now().Add(-30*time.Minute), time.Now(), false, 0),
+ 1 / 1.5 / 1.5,
+ },
+ }
+
+ err := .0001
+ for i, test := range tests {
+ chance := addrmgr.TstKnownAddressChance(test.addr)
+ if math.Abs(test.expected-chance) >= err {
+ t.Errorf("case %d: got %f, expected %f", i, chance, test.expected)
+ }
+ }
+}
+
+func TestIsBad(t *testing.T) {
+ now := time.Unix(time.Now().Unix(), 0)
+ future := now.Add(35 * time.Minute)
+ monthOld := now.Add(-43 * time.Hour * 24)
+ secondsOld := now.Add(-2 * time.Second)
+ minutesOld := now.Add(-27 * time.Minute)
+ hoursOld := now.Add(-5 * time.Hour)
+ zeroTime := time.Time{}
+
+ futureNa := &wire.NetAddressV2{Timestamp: future}
+ minutesOldNa := &wire.NetAddressV2{Timestamp: minutesOld}
+ monthOldNa := &wire.NetAddressV2{Timestamp: monthOld}
+ currentNa := &wire.NetAddressV2{Timestamp: secondsOld}
+
+ //Test addresses that have been tried in the last minute.
+ if addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(futureNa, 3, secondsOld, zeroTime, false, 0)) {
+ t.Errorf("test case 1: addresses that have been tried in the last minute are not bad.")
+ }
+ if addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(monthOldNa, 3, secondsOld, zeroTime, false, 0)) {
+ t.Errorf("test case 2: addresses that have been tried in the last minute are not bad.")
+ }
+ if addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(currentNa, 3, secondsOld, zeroTime, false, 0)) {
+ t.Errorf("test case 3: addresses that have been tried in the last minute are not bad.")
+ }
+ if addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(currentNa, 3, secondsOld, monthOld, true, 0)) {
+ t.Errorf("test case 4: addresses that have been tried in the last minute are not bad.")
+ }
+ if addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(currentNa, 2, secondsOld, secondsOld, true, 0)) {
+ t.Errorf("test case 5: addresses that have been tried in the last minute are not bad.")
+ }
+
+ //Test address that claims to be from the future.
+ if !addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(futureNa, 0, minutesOld, hoursOld, true, 0)) {
+ t.Errorf("test case 6: addresses that claim to be from the future are bad.")
+ }
+
+ //Test address that has not been seen in over a month.
+ if !addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(monthOldNa, 0, minutesOld, hoursOld, true, 0)) {
+ t.Errorf("test case 7: addresses more than a month old are bad.")
+ }
+
+ //It has failed at least three times and never succeeded.
+ if !addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(minutesOldNa, 3, minutesOld, zeroTime, true, 0)) {
+ t.Errorf("test case 8: addresses that have never succeeded are bad.")
+ }
+
+ //It has failed ten times in the last week
+ if !addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(minutesOldNa, 10, minutesOld, monthOld, true, 0)) {
+ t.Errorf("test case 9: addresses that have not succeeded in too long are bad.")
+ }
+
+ //Test an address that should work.
+ if addrmgr.TstKnownAddressIsBad(addrmgr.TstNewKnownAddress(minutesOldNa, 2, minutesOld, hoursOld, true, 0)) {
+ t.Errorf("test case 10: This should be a valid address.")
+ }
+}
diff --git a/addrmgr/log.go b/addrmgr/log.go
new file mode 100644
index 0000000..b3ebbd1
--- /dev/null
+++ b/addrmgr/log.go
@@ -0,0 +1,32 @@
+// Copyright (c) 2013-2014 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr
+
+import (
+ "github.com/btcsuite/btclog"
+)
+
+// log is a logger that is initialized with no output filters. This
+// means the package will not perform any logging by default until the caller
+// requests it.
+var log btclog.Logger
+
+// The default amount of logging is none.
+func init() {
+ DisableLog()
+}
+
+// DisableLog disables all library log output. Logging output is disabled
+// by default until either UseLogger or SetLogWriter are called.
+func DisableLog() {
+ log = btclog.Disabled
+}
+
+// UseLogger uses a specified Logger to output package logging info.
+// This should be used in preference to SetLogWriter if the caller is also
+// using btclog.
+func UseLogger(logger btclog.Logger) {
+ log = logger
+}
diff --git a/addrmgr/network.go b/addrmgr/network.go
new file mode 100644
index 0000000..95555a6
--- /dev/null
+++ b/addrmgr/network.go
@@ -0,0 +1,298 @@
+// Copyright (c) 2013-2014 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr
+
+import (
+ "fmt"
+ "net"
+
+ "github.com/btcsuite/btcd/wire"
+)
+
+var (
+ // rfc1918Nets specifies the IPv4 private address blocks as defined by
+ // by RFC1918 (10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16).
+ rfc1918Nets = []net.IPNet{
+ ipNet("10.0.0.0", 8, 32),
+ ipNet("172.16.0.0", 12, 32),
+ ipNet("192.168.0.0", 16, 32),
+ }
+
+ // rfc2544Net specifies the IPv4 block as defined by RFC2544
+ // (198.18.0.0/15)
+ rfc2544Net = ipNet("198.18.0.0", 15, 32)
+
+ // rfc3849Net specifies the IPv6 documentation address block as defined
+ // by RFC3849 (2001:DB8::/32).
+ rfc3849Net = ipNet("2001:DB8::", 32, 128)
+
+ // rfc3927Net specifies the IPv4 auto configuration address block as
+ // defined by RFC3927 (169.254.0.0/16).
+ rfc3927Net = ipNet("169.254.0.0", 16, 32)
+
+ // rfc3964Net specifies the IPv6 to IPv4 encapsulation address block as
+ // defined by RFC3964 (2002::/16).
+ rfc3964Net = ipNet("2002::", 16, 128)
+
+ // rfc4193Net specifies the IPv6 unique local address block as defined
+ // by RFC4193 (FC00::/7).
+ rfc4193Net = ipNet("FC00::", 7, 128)
+
+ // rfc4380Net specifies the IPv6 teredo tunneling over UDP address block
+ // as defined by RFC4380 (2001::/32).
+ rfc4380Net = ipNet("2001::", 32, 128)
+
+ // rfc4843Net specifies the IPv6 ORCHID address block as defined by
+ // RFC4843 (2001:10::/28).
+ rfc4843Net = ipNet("2001:10::", 28, 128)
+
+ // rfc4862Net specifies the IPv6 stateless address autoconfiguration
+ // address block as defined by RFC4862 (FE80::/64).
+ rfc4862Net = ipNet("FE80::", 64, 128)
+
+ // rfc5737Net specifies the IPv4 documentation address blocks as defined
+ // by RFC5737 (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24)
+ rfc5737Net = []net.IPNet{
+ ipNet("192.0.2.0", 24, 32),
+ ipNet("198.51.100.0", 24, 32),
+ ipNet("203.0.113.0", 24, 32),
+ }
+
+ // rfc6052Net specifies the IPv6 well-known prefix address block as
+ // defined by RFC6052 (64:FF9B::/96).
+ rfc6052Net = ipNet("64:FF9B::", 96, 128)
+
+ // rfc6145Net specifies the IPv6 to IPv4 translated address range as
+ // defined by RFC6145 (::FFFF:0:0:0/96).
+ rfc6145Net = ipNet("::FFFF:0:0:0", 96, 128)
+
+ // rfc6598Net specifies the IPv4 block as defined by RFC6598 (100.64.0.0/10)
+ rfc6598Net = ipNet("100.64.0.0", 10, 32)
+
+ // onionCatNet defines the IPv6 address block used to support Tor.
+ // bitcoind encodes a .onion address as a 16 byte number by decoding the
+ // address prior to the .onion (i.e. the key hash) base32 into a ten
+ // byte number. It then stores the first 6 bytes of the address as
+ // 0xfd, 0x87, 0xd8, 0x7e, 0xeb, 0x43.
+ //
+ // This is the same range used by OnionCat, which is part part of the
+ // RFC4193 unique local IPv6 range.
+ //
+ // In summary the format is:
+ // { magic 6 bytes, 10 bytes base32 decode of key hash }
+ onionCatNet = ipNet("fd87:d87e:eb43::", 48, 128)
+
+ // zero4Net defines the IPv4 address block for address staring with 0
+ // (0.0.0.0/8).
+ zero4Net = ipNet("0.0.0.0", 8, 32)
+
+ // heNet defines the Hurricane Electric IPv6 address block.
+ heNet = ipNet("2001:470::", 32, 128)
+)
+
+// ipNet returns a net.IPNet struct given the passed IP address string, number
+// of one bits to include at the start of the mask, and the total number of bits
+// for the mask.
+func ipNet(ip string, ones, bits int) net.IPNet {
+ return net.IPNet{IP: net.ParseIP(ip), Mask: net.CIDRMask(ones, bits)}
+}
+
+// IsIPv4 returns whether or not the given address is an IPv4 address.
+func IsIPv4(na *wire.NetAddress) bool {
+ return na.IP.To4() != nil
+}
+
+// IsLocal returns whether or not the given address is a local address.
+func IsLocal(na *wire.NetAddress) bool {
+ return na.IP.IsLoopback() || zero4Net.Contains(na.IP)
+}
+
+// IsOnionCatTor returns whether or not the passed address is in the IPv6 range
+// used by bitcoin to support Tor (fd87:d87e:eb43::/48). Note that this range
+// is the same range used by OnionCat, which is part of the RFC4193 unique local
+// IPv6 range.
+func IsOnionCatTor(na *wire.NetAddress) bool {
+ return onionCatNet.Contains(na.IP)
+}
+
+// IsRFC1918 returns whether or not the passed address is part of the IPv4
+// private network address space as defined by RFC1918 (10.0.0.0/8,
+// 172.16.0.0/12, or 192.168.0.0/16).
+func IsRFC1918(na *wire.NetAddress) bool {
+ for _, rfc := range rfc1918Nets {
+ if rfc.Contains(na.IP) {
+ return true
+ }
+ }
+ return false
+}
+
+// IsRFC2544 returns whether or not the passed address is part of the IPv4
+// address space as defined by RFC2544 (198.18.0.0/15)
+func IsRFC2544(na *wire.NetAddress) bool {
+ return rfc2544Net.Contains(na.IP)
+}
+
+// IsRFC3849 returns whether or not the passed address is part of the IPv6
+// documentation range as defined by RFC3849 (2001:DB8::/32).
+func IsRFC3849(na *wire.NetAddress) bool {
+ return rfc3849Net.Contains(na.IP)
+}
+
+// IsRFC3927 returns whether or not the passed address is part of the IPv4
+// autoconfiguration range as defined by RFC3927 (169.254.0.0/16).
+func IsRFC3927(na *wire.NetAddress) bool {
+ return rfc3927Net.Contains(na.IP)
+}
+
+// IsRFC3964 returns whether or not the passed address is part of the IPv6 to
+// IPv4 encapsulation range as defined by RFC3964 (2002::/16).
+func IsRFC3964(na *wire.NetAddress) bool {
+ return rfc3964Net.Contains(na.IP)
+}
+
+// IsRFC4193 returns whether or not the passed address is part of the IPv6
+// unique local range as defined by RFC4193 (FC00::/7).
+func IsRFC4193(na *wire.NetAddress) bool {
+ return rfc4193Net.Contains(na.IP)
+}
+
+// IsRFC4380 returns whether or not the passed address is part of the IPv6
+// teredo tunneling over UDP range as defined by RFC4380 (2001::/32).
+func IsRFC4380(na *wire.NetAddress) bool {
+ return rfc4380Net.Contains(na.IP)
+}
+
+// IsRFC4843 returns whether or not the passed address is part of the IPv6
+// ORCHID range as defined by RFC4843 (2001:10::/28).
+func IsRFC4843(na *wire.NetAddress) bool {
+ return rfc4843Net.Contains(na.IP)
+}
+
+// IsRFC4862 returns whether or not the passed address is part of the IPv6
+// stateless address autoconfiguration range as defined by RFC4862 (FE80::/64).
+func IsRFC4862(na *wire.NetAddress) bool {
+ return rfc4862Net.Contains(na.IP)
+}
+
+// IsRFC5737 returns whether or not the passed address is part of the IPv4
+// documentation address space as defined by RFC5737 (192.0.2.0/24,
+// 198.51.100.0/24, 203.0.113.0/24)
+func IsRFC5737(na *wire.NetAddress) bool {
+ for _, rfc := range rfc5737Net {
+ if rfc.Contains(na.IP) {
+ return true
+ }
+ }
+
+ return false
+}
+
+// IsRFC6052 returns whether or not the passed address is part of the IPv6
+// well-known prefix range as defined by RFC6052 (64:FF9B::/96).
+func IsRFC6052(na *wire.NetAddress) bool {
+ return rfc6052Net.Contains(na.IP)
+}
+
+// IsRFC6145 returns whether or not the passed address is part of the IPv6 to
+// IPv4 translated address range as defined by RFC6145 (::FFFF:0:0:0/96).
+func IsRFC6145(na *wire.NetAddress) bool {
+ return rfc6145Net.Contains(na.IP)
+}
+
+// IsRFC6598 returns whether or not the passed address is part of the IPv4
+// shared address space specified by RFC6598 (100.64.0.0/10)
+func IsRFC6598(na *wire.NetAddress) bool {
+ return rfc6598Net.Contains(na.IP)
+}
+
+// IsValid returns whether or not the passed address is valid. The address is
+// considered invalid under the following circumstances:
+// IPv4: It is either a zero or all bits set address.
+// IPv6: It is either a zero or RFC3849 documentation address.
+func IsValid(na *wire.NetAddress) bool {
+ // IsUnspecified returns if address is 0, so only all bits set, and
+ // RFC3849 need to be explicitly checked.
+ return na.IP != nil && !(na.IP.IsUnspecified() ||
+ na.IP.Equal(net.IPv4bcast))
+}
+
+// IsRoutable returns whether or not the passed address is routable over
+// the public internet. This is true as long as the address is valid and is not
+// in any reserved ranges.
+func IsRoutable(na *wire.NetAddressV2) bool {
+ if na.IsTorV3() {
+ // na is a torv3 address, return true.
+ return true
+ }
+
+ // Else na can be represented as a legacy NetAddress since i2p and
+ // cjdns are unsupported.
+ lna := na.ToLegacy()
+ return IsValid(lna) && !(IsRFC1918(lna) || IsRFC2544(lna) ||
+ IsRFC3927(lna) || IsRFC4862(lna) || IsRFC3849(lna) ||
+ IsRFC4843(lna) || IsRFC5737(lna) || IsRFC6598(lna) ||
+ IsLocal(lna) || (IsRFC4193(lna) &&
+ !IsOnionCatTor(lna)))
+}
+
+// GroupKey returns a string representing the network group an address is part
+// of. This is the /16 for IPv4, the /32 (/36 for he.net) for IPv6, the string
+// "local" for a local address, the string "tor:key" where key is the /4 of the
+// onion address for Tor address, and the string "unroutable" for an unroutable
+// address.
+func GroupKey(na *wire.NetAddressV2) string {
+ if na.IsTorV3() {
+ // na is a torv3 address. Use the same network group keying as
+ // for torv2.
+ return fmt.Sprintf("tor:%d", na.TorV3Key()&((1<<4)-1))
+ }
+
+ lna := na.ToLegacy()
+
+ if IsLocal(lna) {
+ return "local"
+ }
+ if !IsRoutable(na) {
+ return "unroutable"
+ }
+ if IsIPv4(lna) {
+ return lna.IP.Mask(net.CIDRMask(16, 32)).String()
+ }
+ if IsRFC6145(lna) || IsRFC6052(lna) {
+ // last four bytes are the ip address
+ ip := lna.IP[12:16]
+ return ip.Mask(net.CIDRMask(16, 32)).String()
+ }
+
+ if IsRFC3964(lna) {
+ ip := lna.IP[2:6]
+ return ip.Mask(net.CIDRMask(16, 32)).String()
+
+ }
+ if IsRFC4380(lna) {
+ // teredo tunnels have the last 4 bytes as the v4 address XOR
+ // 0xff.
+ ip := net.IP(make([]byte, 4))
+ for i, byte := range lna.IP[12:16] {
+ ip[i] = byte ^ 0xff
+ }
+ return ip.Mask(net.CIDRMask(16, 32)).String()
+ }
+ if IsOnionCatTor(lna) {
+ // group is keyed off the first 4 bits of the actual onion key.
+ return fmt.Sprintf("tor:%d", lna.IP[6]&((1<<4)-1))
+ }
+
+ // OK, so now we know ourselves to be a IPv6 address.
+ // bitcoind uses /32 for everything, except for Hurricane Electric's
+ // (he.net) IP range, which it uses /36 for.
+ bits := 32
+ if heNet.Contains(lna.IP) {
+ bits = 36
+ }
+
+ return lna.IP.Mask(net.CIDRMask(bits, 128)).String()
+}
diff --git a/addrmgr/network_test.go b/addrmgr/network_test.go
new file mode 100644
index 0000000..f4bc5d8
--- /dev/null
+++ b/addrmgr/network_test.go
@@ -0,0 +1,208 @@
+// Copyright (c) 2013-2014 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package addrmgr_test
+
+import (
+ "net"
+ "testing"
+ "time"
+
+ "github.com/btcsuite/btcd/addrmgr"
+ "github.com/btcsuite/btcd/wire"
+)
+
+// TestIPTypes ensures the various functions which determine the type of an IP
+// address based on RFCs work as intended.
+func TestIPTypes(t *testing.T) {
+ type ipTest struct {
+ in wire.NetAddress
+ rfc1918 bool
+ rfc2544 bool
+ rfc3849 bool
+ rfc3927 bool
+ rfc3964 bool
+ rfc4193 bool
+ rfc4380 bool
+ rfc4843 bool
+ rfc4862 bool
+ rfc5737 bool
+ rfc6052 bool
+ rfc6145 bool
+ rfc6598 bool
+ local bool
+ valid bool
+ routable bool
+ }
+
+ newIPTest := func(ip string, rfc1918, rfc2544, rfc3849, rfc3927, rfc3964,
+ rfc4193, rfc4380, rfc4843, rfc4862, rfc5737, rfc6052, rfc6145, rfc6598,
+ local, valid, routable bool) ipTest {
+ nip := net.ParseIP(ip)
+ na := *wire.NewNetAddressIPPort(nip, 8333, wire.SFNodeNetwork)
+ test := ipTest{na, rfc1918, rfc2544, rfc3849, rfc3927, rfc3964, rfc4193, rfc4380,
+ rfc4843, rfc4862, rfc5737, rfc6052, rfc6145, rfc6598, local, valid, routable}
+ return test
+ }
+
+ tests := []ipTest{
+ newIPTest("10.255.255.255", true, false, false, false, false, false,
+ false, false, false, false, false, false, false, false, true, false),
+ newIPTest("192.168.0.1", true, false, false, false, false, false,
+ false, false, false, false, false, false, false, false, true, false),
+ newIPTest("172.31.255.1", true, false, false, false, false, false,
+ false, false, false, false, false, false, false, false, true, false),
+ newIPTest("172.32.1.1", false, false, false, false, false, false, false, false,
+ false, false, false, false, false, false, true, true),
+ newIPTest("169.254.250.120", false, false, false, true, false, false,
+ false, false, false, false, false, false, false, false, true, false),
+ newIPTest("0.0.0.0", false, false, false, false, false, false, false,
+ false, false, false, false, false, false, true, false, false),
+ newIPTest("255.255.255.255", false, false, false, false, false, false,
+ false, false, false, false, false, false, false, false, false, false),
+ newIPTest("127.0.0.1", false, false, false, false, false, false,
+ false, false, false, false, false, false, false, true, true, false),
+ newIPTest("fd00:dead::1", false, false, false, false, false, true,
+ false, false, false, false, false, false, false, false, true, false),
+ newIPTest("2001::1", false, false, false, false, false, false,
+ true, false, false, false, false, false, false, false, true, true),
+ newIPTest("2001:10:abcd::1:1", false, false, false, false, false, false,
+ false, true, false, false, false, false, false, false, true, false),
+ newIPTest("fe80::1", false, false, false, false, false, false,
+ false, false, true, false, false, false, false, false, true, false),
+ newIPTest("fe80:1::1", false, false, false, false, false, false,
+ false, false, false, false, false, false, false, false, true, true),
+ newIPTest("64:ff9b::1", false, false, false, false, false, false,
+ false, false, false, false, true, false, false, false, true, true),
+ newIPTest("::ffff:abcd:ef12:1", false, false, false, false, false, false,
+ false, false, false, false, false, false, false, false, true, true),
+ newIPTest("::1", false, false, false, false, false, false, false, false,
+ false, false, false, false, false, true, true, false),
+ newIPTest("198.18.0.1", false, true, false, false, false, false, false,
+ false, false, false, false, false, false, false, true, false),
+ newIPTest("100.127.255.1", false, false, false, false, false, false, false,
+ false, false, false, false, false, true, false, true, false),
+ newIPTest("203.0.113.1", false, false, false, false, false, false, false,
+ false, false, false, false, false, false, false, true, false),
+ }
+
+ t.Logf("Running %d tests", len(tests))
+ for _, test := range tests {
+ if rv := addrmgr.IsRFC1918(&test.in); rv != test.rfc1918 {
+ t.Errorf("IsRFC1918 %s\n got: %v want: %v", test.in.IP, rv, test.rfc1918)
+ }
+
+ if rv := addrmgr.IsRFC3849(&test.in); rv != test.rfc3849 {
+ t.Errorf("IsRFC3849 %s\n got: %v want: %v", test.in.IP, rv, test.rfc3849)
+ }
+
+ if rv := addrmgr.IsRFC3927(&test.in); rv != test.rfc3927 {
+ t.Errorf("IsRFC3927 %s\n got: %v want: %v", test.in.IP, rv, test.rfc3927)
+ }
+
+ if rv := addrmgr.IsRFC3964(&test.in); rv != test.rfc3964 {
+ t.Errorf("IsRFC3964 %s\n got: %v want: %v", test.in.IP, rv, test.rfc3964)
+ }
+
+ if rv := addrmgr.IsRFC4193(&test.in); rv != test.rfc4193 {
+ t.Errorf("IsRFC4193 %s\n got: %v want: %v", test.in.IP, rv, test.rfc4193)
+ }
+
+ if rv := addrmgr.IsRFC4380(&test.in); rv != test.rfc4380 {
+ t.Errorf("IsRFC4380 %s\n got: %v want: %v", test.in.IP, rv, test.rfc4380)
+ }
+
+ if rv := addrmgr.IsRFC4843(&test.in); rv != test.rfc4843 {
+ t.Errorf("IsRFC4843 %s\n got: %v want: %v", test.in.IP, rv, test.rfc4843)
+ }
+
+ if rv := addrmgr.IsRFC4862(&test.in); rv != test.rfc4862 {
+ t.Errorf("IsRFC4862 %s\n got: %v want: %v", test.in.IP, rv, test.rfc4862)
+ }
+
+ if rv := addrmgr.IsRFC6052(&test.in); rv != test.rfc6052 {
+ t.Errorf("isRFC6052 %s\n got: %v want: %v", test.in.IP, rv, test.rfc6052)
+ }
+
+ if rv := addrmgr.IsRFC6145(&test.in); rv != test.rfc6145 {
+ t.Errorf("IsRFC1918 %s\n got: %v want: %v", test.in.IP, rv, test.rfc6145)
+ }
+
+ if rv := addrmgr.IsLocal(&test.in); rv != test.local {
+ t.Errorf("IsLocal %s\n got: %v want: %v", test.in.IP, rv, test.local)
+ }
+
+ if rv := addrmgr.IsValid(&test.in); rv != test.valid {
+ t.Errorf("IsValid %s\n got: %v want: %v", test.in.IP, rv, test.valid)
+ }
+
+ currentNa := wire.NetAddressV2FromBytes(
+ time.Now(), test.in.Services, test.in.IP, test.in.Port,
+ )
+ if rv := addrmgr.IsRoutable(currentNa); rv != test.routable {
+ t.Errorf("IsRoutable %s\n got: %v want: %v", test.in.IP, rv, test.routable)
+ }
+ }
+}
+
+// TestGroupKey tests the GroupKey function to ensure it properly groups various
+// IP addresses.
+func TestGroupKey(t *testing.T) {
+ tests := []struct {
+ name string
+ ip string
+ expected string
+ }{
+ // Local addresses.
+ {name: "ipv4 localhost", ip: "127.0.0.1", expected: "local"},
+ {name: "ipv6 localhost", ip: "::1", expected: "local"},
+ {name: "ipv4 zero", ip: "0.0.0.0", expected: "local"},
+ {name: "ipv4 first octet zero", ip: "0.1.2.3", expected: "local"},
+
+ // Unroutable addresses.
+ {name: "ipv4 invalid bcast", ip: "255.255.255.255", expected: "unroutable"},
+ {name: "ipv4 rfc1918 10/8", ip: "10.1.2.3", expected: "unroutable"},
+ {name: "ipv4 rfc1918 172.16/12", ip: "172.16.1.2", expected: "unroutable"},
+ {name: "ipv4 rfc1918 192.168/16", ip: "192.168.1.2", expected: "unroutable"},
+ {name: "ipv6 rfc3849 2001:db8::/32", ip: "2001:db8::1234", expected: "unroutable"},
+ {name: "ipv4 rfc3927 169.254/16", ip: "169.254.1.2", expected: "unroutable"},
+ {name: "ipv6 rfc4193 fc00::/7", ip: "fc00::1234", expected: "unroutable"},
+ {name: "ipv6 rfc4843 2001:10::/28", ip: "2001:10::1234", expected: "unroutable"},
+ {name: "ipv6 rfc4862 fe80::/64", ip: "fe80::1234", expected: "unroutable"},
+
+ // IPv4 normal.
+ {name: "ipv4 normal class a", ip: "12.1.2.3", expected: "12.1.0.0"},
+ {name: "ipv4 normal class b", ip: "173.1.2.3", expected: "173.1.0.0"},
+ {name: "ipv4 normal class c", ip: "196.1.2.3", expected: "196.1.0.0"},
+
+ // IPv6/IPv4 translations.
+ {name: "ipv6 rfc3964 with ipv4 encap", ip: "2002:0c01:0203::", expected: "12.1.0.0"},
+ {name: "ipv6 rfc4380 toredo ipv4", ip: "2001:0:1234::f3fe:fdfc", expected: "12.1.0.0"},
+ {name: "ipv6 rfc6052 well-known prefix with ipv4", ip: "64:ff9b::0c01:0203", expected: "12.1.0.0"},
+ {name: "ipv6 rfc6145 translated ipv4", ip: "::ffff:0:0c01:0203", expected: "12.1.0.0"},
+
+ // Tor.
+ {name: "ipv6 tor onioncat", ip: "fd87:d87e:eb43:1234::5678", expected: "tor:2"},
+ {name: "ipv6 tor onioncat 2", ip: "fd87:d87e:eb43:1245::6789", expected: "tor:2"},
+ {name: "ipv6 tor onioncat 3", ip: "fd87:d87e:eb43:1345::6789", expected: "tor:3"},
+
+ // IPv6 normal.
+ {name: "ipv6 normal", ip: "2602:100::1", expected: "2602:100::"},
+ {name: "ipv6 normal 2", ip: "2602:0100::1234", expected: "2602:100::"},
+ {name: "ipv6 hurricane electric", ip: "2001:470:1f10:a1::2", expected: "2001:470:1000::"},
+ {name: "ipv6 hurricane electric 2", ip: "2001:0470:1f10:a1::2", expected: "2001:470:1000::"},
+ }
+
+ for i, test := range tests {
+ nip := net.ParseIP(test.ip)
+ na := wire.NetAddressV2FromBytes(
+ time.Now(), wire.SFNodeNetwork, nip, 8333,
+ )
+ if key := addrmgr.GroupKey(na); key != test.expected {
+ t.Errorf("TestGroupKey #%d (%s): unexpected group key "+
+ "- got '%s', want '%s'", i, test.name,
+ key, test.expected)
+ }
+ }
+}
diff --git a/addrmgr/test_coverage.txt b/addrmgr/test_coverage.txt
new file mode 100644
index 0000000..c67e0f6
--- /dev/null
+++ b/addrmgr/test_coverage.txt
@@ -0,0 +1,62 @@
+
+github.com/conformal/btcd/addrmgr/network.go GroupKey 100.00% (23/23)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.reset 100.00% (6/6)
+github.com/conformal/btcd/addrmgr/network.go IsRFC5737 100.00% (4/4)
+github.com/conformal/btcd/addrmgr/network.go IsRFC1918 100.00% (4/4)
+github.com/conformal/btcd/addrmgr/addrmanager.go New 100.00% (3/3)
+github.com/conformal/btcd/addrmgr/addrmanager.go NetAddressKey 100.00% (2/2)
+github.com/conformal/btcd/addrmgr/network.go IsRFC4862 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.numAddresses 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/log.go init 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/log.go DisableLog 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go ipNet 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsIPv4 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsLocal 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsOnionCatTor 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC2544 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC3849 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC3927 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC3964 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC4193 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC4380 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC4843 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC6052 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC6145 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRFC6598 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsValid 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/network.go IsRoutable 100.00% (1/1)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.GetBestLocalAddress 94.74% (18/19)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.AddLocalAddress 90.91% (10/11)
+github.com/conformal/btcd/addrmgr/addrmanager.go getReachabilityFrom 51.52% (17/33)
+github.com/conformal/btcd/addrmgr/addrmanager.go ipString 50.00% (2/4)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.GetAddress 9.30% (4/43)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.deserializePeers 0.00% (0/50)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.Good 0.00% (0/44)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.savePeers 0.00% (0/39)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.updateAddress 0.00% (0/30)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.expireNew 0.00% (0/22)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.AddressCache 0.00% (0/16)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.HostToNetAddress 0.00% (0/15)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.getNewBucket 0.00% (0/15)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.AddAddressByIP 0.00% (0/14)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.getTriedBucket 0.00% (0/14)
+github.com/conformal/btcd/addrmgr/knownaddress.go knownAddress.chance 0.00% (0/13)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.loadPeers 0.00% (0/11)
+github.com/conformal/btcd/addrmgr/knownaddress.go knownAddress.isBad 0.00% (0/11)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.Connected 0.00% (0/10)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.addressHandler 0.00% (0/9)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.pickTried 0.00% (0/8)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.DeserializeNetAddress 0.00% (0/7)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.Stop 0.00% (0/7)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.Attempt 0.00% (0/7)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.Start 0.00% (0/6)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.AddAddresses 0.00% (0/4)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.NeedMoreAddresses 0.00% (0/3)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.NumAddresses 0.00% (0/3)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.AddAddress 0.00% (0/3)
+github.com/conformal/btcd/addrmgr/knownaddress.go knownAddress.LastAttempt 0.00% (0/1)
+github.com/conformal/btcd/addrmgr/knownaddress.go knownAddress.NetAddress 0.00% (0/1)
+github.com/conformal/btcd/addrmgr/addrmanager.go AddrManager.find 0.00% (0/1)
+github.com/conformal/btcd/addrmgr/log.go UseLogger 0.00% (0/1)
+github.com/conformal/btcd/addrmgr --------------------------------- 21.04% (113/537)
+
diff --git a/blockchain/README.md b/blockchain/README.md
new file mode 100644
index 0000000..2237780
--- /dev/null
+++ b/blockchain/README.md
@@ -0,0 +1,103 @@
+blockchain
+==========
+
+[](https://github.com/btcsuite/btcd/actions)
+[](http://copyfree.org)
+[](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain)
+
+Package blockchain implements bitcoin block handling and chain selection rules.
+The test coverage is currently only around 60%, but will be increasing over
+time. See `test_coverage.txt` for the gocov coverage report. Alternatively, if
+you are running a POSIX OS, you can run the `cov_report.sh` script for a
+real-time report. Package blockchain is licensed under the liberal ISC license.
+
+There is an associated blog post about the release of this package
+[here](https://blog.conformal.com/btcchain-the-bitcoin-chain-package-from-bctd/).
+
+This package has intentionally been designed so it can be used as a standalone
+package for any projects needing to handle processing of blocks into the bitcoin
+block chain.
+
+## Installation and Updating
+
+```bash
+$ go get -u github.com/btcsuite/btcd/blockchain
+```
+
+## Bitcoin Chain Processing Overview
+
+Before a block is allowed into the block chain, it must go through an intensive
+series of validation rules. The following list serves as a general outline of
+those rules to provide some intuition into what is going on under the hood, but
+is by no means exhaustive:
+
+ - Reject duplicate blocks
+ - Perform a series of sanity checks on the block and its transactions such as
+ verifying proof of work, timestamps, number and character of transactions,
+ transaction amounts, script complexity, and merkle root calculations
+ - Compare the block against predetermined checkpoints for expected timestamps
+ and difficulty based on elapsed time since the checkpoint
+ - Save the most recent orphan blocks for a limited time in case their parent
+ blocks become available
+ - Stop processing if the block is an orphan as the rest of the processing
+ depends on the block's position within the block chain
+ - Perform a series of more thorough checks that depend on the block's position
+ within the block chain such as verifying block difficulties adhere to
+ difficulty retarget rules, timestamps are after the median of the last
+ several blocks, all transactions are finalized, checkpoint blocks match, and
+ block versions are in line with the previous blocks
+ - Determine how the block fits into the chain and perform different actions
+ accordingly in order to ensure any side chains which have higher difficulty
+ than the main chain become the new main chain
+ - When a block is being connected to the main chain (either through
+ reorganization of a side chain to the main chain or just extending the
+ main chain), perform further checks on the block's transactions such as
+ verifying transaction duplicates, script complexity for the combination of
+ connected scripts, coinbase maturity, double spends, and connected
+ transaction values
+ - Run the transaction scripts to verify the spender is allowed to spend the
+ coins
+ - Insert the block into the block database
+
+## Examples
+
+* [ProcessBlock Example](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain#example-BlockChain-ProcessBlock)
+ Demonstrates how to create a new chain instance and use ProcessBlock to
+ attempt to add a block to the chain. This example intentionally
+ attempts to insert a duplicate genesis block to illustrate how an invalid
+ block is handled.
+
+* [CompactToBig Example](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain#example-CompactToBig)
+ Demonstrates how to convert the compact "bits" in a block header which
+ represent the target difficulty to a big integer and display it using the
+ typical hex notation.
+
+* [BigToCompact Example](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain#example-BigToCompact)
+ Demonstrates how to convert a target difficulty into the
+ compact "bits" in a block header which represent that target difficulty.
+
+## GPG Verification Key
+
+All official release tags are signed by Conformal so users can ensure the code
+has not been tampered with and is coming from the btcsuite developers. To
+verify the signature perform the following:
+
+- Download the public key from the Conformal website at
+ https://opensource.conformal.com/GIT-GPG-KEY-conformal.txt
+
+- Import the public key into your GPG keyring:
+ ```bash
+ gpg --import GIT-GPG-KEY-conformal.txt
+ ```
+
+- Verify the release tag with the following command where `TAG_NAME` is a
+ placeholder for the specific tag:
+ ```bash
+ git tag -v TAG_NAME
+ ```
+
+## License
+
+
+Package blockchain is licensed under the [copyfree](http://copyfree.org) ISC
+License.
diff --git a/blockchain/accept.go b/blockchain/accept.go
new file mode 100644
index 0000000..4adc2f6
--- /dev/null
+++ b/blockchain/accept.go
@@ -0,0 +1,94 @@
+// Copyright (c) 2013-2017 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package blockchain
+
+import (
+ "fmt"
+
+ "github.com/btcsuite/btcd/btcutil"
+ "github.com/btcsuite/btcd/database"
+)
+
+// maybeAcceptBlock potentially accepts a block into the block chain and, if
+// accepted, returns whether or not it is on the main chain. It performs
+// several validation checks which depend on its position within the block chain
+// before adding it. The block is expected to have already gone through
+// ProcessBlock before calling this function with it.
+//
+// The flags are also passed to checkBlockContext and connectBestChain. See
+// their documentation for how the flags modify their behavior.
+//
+// This function MUST be called with the chain state lock held (for writes).
+func (b *BlockChain) maybeAcceptBlock(block *btcutil.Block, flags BehaviorFlags) (bool, error) {
+ // The height of this block is one more than the referenced previous
+ // block.
+ prevHash := &block.MsgBlock().Header.PrevBlock
+ prevNode := b.index.LookupNode(prevHash)
+ if prevNode == nil {
+ str := fmt.Sprintf("previous block %s is unknown", prevHash)
+ return false, ruleError(ErrPreviousBlockUnknown, str)
+ } else if b.index.NodeStatus(prevNode).KnownInvalid() {
+ str := fmt.Sprintf("previous block %s is known to be invalid", prevHash)
+ return false, ruleError(ErrInvalidAncestorBlock, str)
+ }
+
+ blockHeight := prevNode.height + 1
+ block.SetHeight(blockHeight)
+
+ // The block must pass all of the validation rules which depend on the
+ // position of the block within the block chain.
+ err := b.checkBlockContext(block, prevNode, flags)
+ if err != nil {
+ return false, err
+ }
+
+ // Insert the block into the database if it's not already there. Even
+ // though it is possible the block will ultimately fail to connect, it
+ // has already passed all proof-of-work and validity tests which means
+ // it would be prohibitively expensive for an attacker to fill up the
+ // disk with a bunch of blocks that fail to connect. This is necessary
+ // since it allows block download to be decoupled from the much more
+ // expensive connection logic. It also has some other nice properties
+ // such as making blocks that never become part of the main chain or
+ // blocks that fail to connect available for further analysis.
+ err = b.db.Update(func(dbTx database.Tx) error {
+ return dbStoreBlock(dbTx, block)
+ })
+ if err != nil {
+ return false, err
+ }
+
+ // Create a new block node for the block and add it to the node index. Even
+ // if the block ultimately gets connected to the main chain, it starts out
+ // on a side chain.
+ blockHeader := &block.MsgBlock().Header
+ newNode := newBlockNode(blockHeader, prevNode)
+ newNode.status = statusDataStored
+
+ b.index.AddNode(newNode)
+ err = b.index.flushToDB()
+ if err != nil {
+ return false, err
+ }
+
+ // Connect the passed block to the chain while respecting proper chain
+ // selection according to the chain with the most proof of work. This
+ // also handles validation of the transaction scripts.
+ isMainChain, err := b.connectBestChain(newNode, block, flags)
+ if err != nil {
+ return false, err
+ }
+
+ // Notify the caller that the new block was accepted into the block
+ // chain. The caller would typically want to react by relaying the
+ // inventory to other peers.
+ func() {
+ b.chainLock.Unlock()
+ defer b.chainLock.Lock()
+ b.sendNotification(NTBlockAccepted, block)
+ }()
+
+ return isMainChain, nil
+}
diff --git a/blockchain/bench_test.go b/blockchain/bench_test.go
new file mode 100644
index 0000000..db6f415
--- /dev/null
+++ b/blockchain/bench_test.go
@@ -0,0 +1,75 @@
+// Copyright (c) 2015 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package blockchain
+
+import (
+ "testing"
+
+ "github.com/btcsuite/btcd/btcutil"
+ "github.com/btcsuite/btcd/wire"
+)
+
+// BenchmarkIsCoinBase performs a simple benchmark against the IsCoinBase
+// function.
+func BenchmarkIsCoinBase(b *testing.B) {
+ tx, _ := btcutil.NewBlock(&Block100000).Tx(1)
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ IsCoinBase(tx)
+ }
+}
+
+// BenchmarkIsCoinBaseTx performs a simple benchmark against the IsCoinBaseTx
+// function.
+func BenchmarkIsCoinBaseTx(b *testing.B) {
+ tx := Block100000.Transactions[1]
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ IsCoinBaseTx(tx)
+ }
+}
+
+func BenchmarkUtxoFetchMap(b *testing.B) {
+ block := Block100000
+ transactions := block.Transactions
+ b.ResetTimer()
+
+ for i := 0; i < b.N; i++ {
+ needed := make(map[wire.OutPoint]struct{}, len(transactions))
+ for _, tx := range transactions[1:] {
+ for _, txIn := range tx.TxIn {
+ needed[txIn.PreviousOutPoint] = struct{}{}
+ }
+ }
+ }
+}
+
+func BenchmarkUtxoFetchSlices(b *testing.B) {
+ block := Block100000
+ transactions := block.Transactions
+ b.ResetTimer()
+
+ for i := 0; i < b.N; i++ {
+ needed := make([]wire.OutPoint, 0, len(transactions))
+ for _, tx := range transactions[1:] {
+ for _, txIn := range tx.TxIn {
+ needed = append(needed, txIn.PreviousOutPoint)
+ }
+ }
+ }
+}
+
+func BenchmarkAncestor(b *testing.B) {
+ height := 1 << 19
+ blockNodes := chainedNodes(nil, height)
+
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ blockNodes[len(blockNodes)-1].Ancestor(0)
+ for j := 0; j <= 19; j++ {
+ blockNodes[len(blockNodes)-1].Ancestor(1 << j)
+ }
+ }
+}
diff --git a/blockchain/blockindex.go b/blockchain/blockindex.go
new file mode 100644
index 0000000..5273cb4
--- /dev/null
+++ b/blockchain/blockindex.go
@@ -0,0 +1,517 @@
+// Copyright (c) 2015-2017 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package blockchain
+
+import (
+ "math/big"
+ "sort"
+ "sync"
+ "time"
+
+ "github.com/btcsuite/btcd/chaincfg"
+ "github.com/btcsuite/btcd/chaincfg/chainhash"
+ "github.com/btcsuite/btcd/database"
+ "github.com/btcsuite/btcd/wire"
+)
+
+// blockStatus is a bit field representing the validation state of the block.
+type blockStatus byte
+
+const (
+ // statusDataStored indicates that the block's payload is stored on disk.
+ statusDataStored blockStatus = 1 << iota
+
+ // statusValid indicates that the block has been fully validated.
+ statusValid
+
+ // statusValidateFailed indicates that the block has failed validation.
+ statusValidateFailed
+
+ // statusInvalidAncestor indicates that one of the block's ancestors has
+ // has failed validation, thus the block is also invalid.
+ statusInvalidAncestor
+
+ // statusNone indicates that the block has no validation state flags set.
+ //
+ // NOTE: This must be defined last in order to avoid influencing iota.
+ statusNone blockStatus = 0
+)
+
+// HaveData returns whether the full block data is stored in the database. This
+// will return false for a block node where only the header is downloaded or
+// kept.
+func (status blockStatus) HaveData() bool {
+ return status&statusDataStored != 0
+}
+
+// KnownValid returns whether the block is known to be valid. This will return
+// false for a valid block that has not been fully validated yet.
+func (status blockStatus) KnownValid() bool {
+ return status&statusValid != 0
+}
+
+// KnownInvalid returns whether the block is known to be invalid. This may be
+// because the block itself failed validation or any of its ancestors is
+// invalid. This will return false for invalid blocks that have not been proven
+// invalid yet.
+func (status blockStatus) KnownInvalid() bool {
+ return status&(statusValidateFailed|statusInvalidAncestor) != 0
+}
+
+// blockNode represents a block within the block chain and is primarily used to
+// aid in selecting the best chain to be the main chain. The main chain is
+// stored into the block database.
+type blockNode struct {
+ // NOTE: Additions, deletions, or modifications to the order of the
+ // definitions in this struct should not be changed without considering
+ // how it affects alignment on 64-bit platforms. The current order is
+ // specifically crafted to result in minimal padding. There will be
+ // hundreds of thousands of these in memory, so a few extra bytes of
+ // padding adds up.
+
+ // parent is the parent block for this node.
+ parent *blockNode
+
+ // ancestor is a block that is more than one block back from this node.
+ ancestor *blockNode
+
+ // hash is the double sha 256 of the block.
+ hash chainhash.Hash
+
+ // workSum is the total amount of work in the chain up to and including
+ // this node.
+ workSum *big.Int
+
+ // height is the position in the block chain.
+ height int32
+
+ // Some fields from block headers to aid in best chain selection and
+ // reconstructing headers from memory. These must be treated as
+ // immutable and are intentionally ordered to avoid padding on 64-bit
+ // platforms.
+ version int32
+ bits uint32
+ nonce uint32
+ timestamp int64
+ merkleRoot chainhash.Hash
+
+ // status is a bitfield representing the validation state of the block. The
+ // status field, unlike the other fields, may be written to and so should
+ // only be accessed using the concurrent-safe NodeStatus method on
+ // blockIndex once the node has been added to the global index.
+ status blockStatus
+}
+
+// initBlockNode initializes a block node from the given header and parent node,
+// calculating the height and workSum from the respective fields on the parent.
+// This function is NOT safe for concurrent access. It must only be called when
+// initially creating a node.
+func initBlockNode(node *blockNode, blockHeader *wire.BlockHeader, parent *blockNode) {
+ *node = blockNode{
+ hash: blockHeader.BlockHash(),
+ workSum: CalcWork(blockHeader.Bits),
+ version: blockHeader.Version,
+ bits: blockHeader.Bits,
+ nonce: blockHeader.Nonce,
+ timestamp: blockHeader.Timestamp.Unix(),
+ merkleRoot: blockHeader.MerkleRoot,
+ }
+ if parent != nil {
+ node.parent = parent
+ node.height = parent.height + 1
+ node.workSum = node.workSum.Add(parent.workSum, node.workSum)
+ node.buildAncestor()
+ }
+}
+
+// newBlockNode returns a new block node for the given block header and parent
+// node, calculating the height and workSum from the respective fields on the
+// parent. This function is NOT safe for concurrent access.
+func newBlockNode(blockHeader *wire.BlockHeader, parent *blockNode) *blockNode {
+ var node blockNode
+ initBlockNode(&node, blockHeader, parent)
+ return &node
+}
+
+// Equals compares all the fields of the block node except for the parent and
+// ancestor and returns true if they're equal.
+func (node *blockNode) Equals(other *blockNode) bool {
+ return node.hash == other.hash &&
+ node.workSum.Cmp(other.workSum) == 0 &&
+ node.height == other.height &&
+ node.version == other.version &&
+ node.bits == other.bits &&
+ node.nonce == other.nonce &&
+ node.timestamp == other.timestamp &&
+ node.merkleRoot == other.merkleRoot &&
+ node.status == other.status
+}
+
+// Header constructs a block header from the node and returns it.
+//
+// This function is safe for concurrent access.
+func (node *blockNode) Header() wire.BlockHeader {
+ // No lock is needed because all accessed fields are immutable.
+ prevHash := &zeroHash
+ if node.parent != nil {
+ prevHash = &node.parent.hash
+ }
+ return wire.BlockHeader{
+ Version: node.version,
+ PrevBlock: *prevHash,
+ MerkleRoot: node.merkleRoot,
+ Timestamp: time.Unix(node.timestamp, 0),
+ Bits: node.bits,
+ Nonce: node.nonce,
+ }
+}
+
+// invertLowestOne turns the lowest 1 bit in the binary representation of a number into a 0.
+func invertLowestOne(n int32) int32 {
+ return n & (n - 1)
+}
+
+// getAncestorHeight returns a suitable ancestor for the node at the given height.
+func getAncestorHeight(height int32) int32 {
+ // We pop off two 1 bits of the height.
+ // This results in a maximum of 330 steps to go back to an ancestor
+ // from height 1<<29.
+ return invertLowestOne(invertLowestOne(height))
+}
+
+// buildAncestor sets an ancestor for the given blocknode.
+func (node *blockNode) buildAncestor() {
+ if node.parent != nil {
+ node.ancestor = node.parent.Ancestor(getAncestorHeight(node.height))
+ }
+}
+
+// Ancestor returns the ancestor block node at the provided height by following
+// the chain backwards from this node. The returned block will be nil when a
+// height is requested that is after the height of the passed node or is less
+// than zero.
+//
+// This function is safe for concurrent access.
+func (node *blockNode) Ancestor(height int32) *blockNode {
+ if height < 0 || height > node.height {
+ return nil
+ }
+
+ // Traverse back until we find the desired node.
+ n := node
+ for n != nil && n.height != height {
+ // If there's an ancestor available, use it. Otherwise, just
+ // follow the parent.
+ if n.ancestor != nil {
+ // Calculate the height for this ancestor and
+ // check if we can take the ancestor skip.
+ if getAncestorHeight(n.height) >= height {
+ n = n.ancestor
+ continue
+ }
+ }
+
+ // We couldn't take the ancestor skip so traverse back to the parent.
+ n = n.parent
+ }
+
+ return n
+}
+
+// Height returns the blockNode's height in the chain.
+//
+// NOTE: Part of the HeaderCtx interface.
+func (node *blockNode) Height() int32 {
+ return node.height
+}
+
+// Bits returns the blockNode's nBits.
+//
+// NOTE: Part of the HeaderCtx interface.
+func (node *blockNode) Bits() uint32 {
+ return node.bits
+}
+
+// Timestamp returns the blockNode's timestamp.
+//
+// NOTE: Part of the HeaderCtx interface.
+func (node *blockNode) Timestamp() int64 {
+ return node.timestamp
+}
+
+// Parent returns the blockNode's parent.
+//
+// NOTE: Part of the HeaderCtx interface.
+func (node *blockNode) Parent() HeaderCtx {
+ if node.parent == nil {
+ // This is required since node.parent is a *blockNode and if we
+ // do not explicitly return nil here, the caller may fail when
+ // nil-checking this.
+ return nil
+ }
+
+ return node.parent
+}
+
+// RelativeAncestorCtx returns the blockNode's ancestor that is distance blocks
+// before it in the chain. This is equivalent to the RelativeAncestor function
+// below except that the return type is different.
+//
+// This function is safe for concurrent access.
+//
+// NOTE: Part of the HeaderCtx interface.
+func (node *blockNode) RelativeAncestorCtx(distance int32) HeaderCtx {
+ ancestor := node.RelativeAncestor(distance)
+ if ancestor == nil {
+ // This is required since RelativeAncestor returns a *blockNode
+ // and if we do not explicitly return nil here, the caller may
+ // fail when nil-checking this.
+ return nil
+ }
+
+ return ancestor
+}
+
+// IsAncestor returns if the other node is an ancestor of this block node.
+func (node *blockNode) IsAncestor(otherNode *blockNode) bool {
+ // Return early as false if the otherNode is nil.
+ if otherNode == nil {
+ return false
+ }
+
+ ancestor := node.Ancestor(otherNode.height)
+ if ancestor == nil {
+ return false
+ }
+
+ // If the otherNode has the same height as me, then the returned
+ // ancestor will be me. Return false since I'm not an ancestor of me.
+ if node.height == ancestor.height {
+ return false
+ }
+
+ // Return true if the fetched ancestor is other node.
+ return ancestor.Equals(otherNode)
+}
+
+// RelativeAncestor returns the ancestor block node a relative 'distance' blocks
+// before this node. This is equivalent to calling Ancestor with the node's
+// height minus provided distance.
+//
+// This function is safe for concurrent access.
+func (node *blockNode) RelativeAncestor(distance int32) *blockNode {
+ return node.Ancestor(node.height - distance)
+}
+
+// CalcPastMedianTime calculates the median time of the previous few blocks
+// prior to, and including, the block node.
+//
+// This function is safe for concurrent access.
+func CalcPastMedianTime(node HeaderCtx) time.Time {
+ // Create a slice of the previous few block timestamps used to calculate
+ // the median per the number defined by the constant medianTimeBlocks.
+ timestamps := make([]int64, medianTimeBlocks)
+ numNodes := 0
+ iterNode := node
+ for i := 0; i < medianTimeBlocks && iterNode != nil; i++ {
+ timestamps[i] = iterNode.Timestamp()
+ numNodes++
+
+ iterNode = iterNode.Parent()
+ }
+
+ // Prune the slice to the actual number of available timestamps which
+ // will be fewer than desired near the beginning of the block chain
+ // and sort them.
+ timestamps = timestamps[:numNodes]
+ sort.Sort(timeSorter(timestamps))
+
+ // NOTE: The consensus rules incorrectly calculate the median for even
+ // numbers of blocks. A true median averages the middle two elements
+ // for a set with an even number of elements in it. Since the constant
+ // for the previous number of blocks to be used is odd, this is only an
+ // issue for a few blocks near the beginning of the chain. I suspect
+ // this is an optimization even though the result is slightly wrong for
+ // a few of the first blocks since after the first few blocks, there
+ // will always be an odd number of blocks in the set per the constant.
+ //
+ // This code follows suit to ensure the same rules are used, however, be
+ // aware that should the medianTimeBlocks constant ever be changed to an
+ // even number, this code will be wrong.
+ medianTimestamp := timestamps[numNodes/2]
+ return time.Unix(medianTimestamp, 0)
+}
+
+// A compile-time assertion to ensure blockNode implements the HeaderCtx
+// interface.
+var _ HeaderCtx = (*blockNode)(nil)
+
+// blockIndex provides facilities for keeping track of an in-memory index of the
+// block chain. Although the name block chain suggests a single chain of
+// blocks, it is actually a tree-shaped structure where any node can have
+// multiple children. However, there can only be one active branch which does
+// indeed form a chain from the tip all the way back to the genesis block.
+type blockIndex struct {
+ // The following fields are set when the instance is created and can't
+ // be changed afterwards, so there is no need to protect them with a
+ // separate mutex.
+ db database.DB
+ chainParams *chaincfg.Params
+
+ sync.RWMutex
+ index map[chainhash.Hash]*blockNode
+ dirty map[*blockNode]struct{}
+}
+
+// newBlockIndex returns a new empty instance of a block index. The index will
+// be dynamically populated as block nodes are loaded from the database and
+// manually added.
+func newBlockIndex(db database.DB, chainParams *chaincfg.Params) *blockIndex {
+ return &blockIndex{
+ db: db,
+ chainParams: chainParams,
+ index: make(map[chainhash.Hash]*blockNode),
+ dirty: make(map[*blockNode]struct{}),
+ }
+}
+
+// HaveBlock returns whether or not the block index contains the provided hash.
+//
+// This function is safe for concurrent access.
+func (bi *blockIndex) HaveBlock(hash *chainhash.Hash) bool {
+ bi.RLock()
+ _, hasBlock := bi.index[*hash]
+ bi.RUnlock()
+ return hasBlock
+}
+
+// LookupNode returns the block node identified by the provided hash. It will
+// return nil if there is no entry for the hash.
+//
+// This function is safe for concurrent access.
+func (bi *blockIndex) LookupNode(hash *chainhash.Hash) *blockNode {
+ bi.RLock()
+ node := bi.index[*hash]
+ bi.RUnlock()
+ return node
+}
+
+// AddNode adds the provided node to the block index and marks it as dirty.
+// Duplicate entries are not checked so it is up to caller to avoid adding them.
+//
+// This function is safe for concurrent access.
+func (bi *blockIndex) AddNode(node *blockNode) {
+ bi.Lock()
+ bi.addNode(node)
+ bi.dirty[node] = struct{}{}
+ bi.Unlock()
+}
+
+// addNode adds the provided node to the block index, but does not mark it as
+// dirty. This can be used while initializing the block index.
+//
+// This function is NOT safe for concurrent access.
+func (bi *blockIndex) addNode(node *blockNode) {
+ bi.index[node.hash] = node
+}
+
+// NodeStatus provides concurrent-safe access to the status field of a node.
+//
+// This function is safe for concurrent access.
+func (bi *blockIndex) NodeStatus(node *blockNode) blockStatus {
+ bi.RLock()
+ status := node.status
+ bi.RUnlock()
+ return status
+}
+
+// SetStatusFlags flips the provided status flags on the block node to on,
+// regardless of whether they were on or off previously. This does not unset any
+// flags currently on.
+//
+// This function is safe for concurrent access.
+func (bi *blockIndex) SetStatusFlags(node *blockNode, flags blockStatus) {
+ bi.Lock()
+ node.status |= flags
+ bi.dirty[node] = struct{}{}
+ bi.Unlock()
+}
+
+// UnsetStatusFlags flips the provided status flags on the block node to off,
+// regardless of whether they were on or off previously.
+//
+// This function is safe for concurrent access.
+func (bi *blockIndex) UnsetStatusFlags(node *blockNode, flags blockStatus) {
+ bi.Lock()
+ node.status &^= flags
+ bi.dirty[node] = struct{}{}
+ bi.Unlock()
+}
+
+// InactiveTips returns all the block nodes that aren't in the best chain.
+//
+// This function is safe for concurrent access.
+func (bi *blockIndex) InactiveTips(bestChain *chainView) []*blockNode {
+ bi.RLock()
+ defer bi.RUnlock()
+
+ // Look through the entire blockindex and look for nodes that aren't in
+ // the best chain. We're gonna keep track of all the orphans and the parents
+ // of the orphans.
+ orphans := make(map[chainhash.Hash]*blockNode)
+ orphanParent := make(map[chainhash.Hash]*blockNode)
+ for hash, node := range bi.index {
+ found := bestChain.Contains(node)
+ if !found {
+ orphans[hash] = node
+ orphanParent[node.parent.hash] = node.parent
+ }
+ }
+
+ // If an orphan isn't pointed to by another orphan, it is a chain tip.
+ //
+ // We can check this by looking for the orphan in the orphan parent map.
+ // If the orphan exists in the orphan parent map, it means that another
+ // orphan is pointing to it.
+ tips := make([]*blockNode, 0, len(orphans))
+ for hash, orphan := range orphans {
+ _, found := orphanParent[hash]
+ if !found {
+ tips = append(tips, orphan)
+ }
+
+ delete(orphanParent, hash)
+ }
+
+ return tips
+}
+
+// flushToDB writes all dirty block nodes to the database. If all writes
+// succeed, this clears the dirty set.
+func (bi *blockIndex) flushToDB() error {
+ bi.Lock()
+ if len(bi.dirty) == 0 {
+ bi.Unlock()
+ return nil
+ }
+
+ err := bi.db.Update(func(dbTx database.Tx) error {
+ for node := range bi.dirty {
+ err := dbStoreBlockNode(dbTx, node)
+ if err != nil {
+ return err
+ }
+ }
+ return nil
+ })
+
+ // If write was successful, clear the dirty set.
+ if err == nil {
+ bi.dirty = make(map[*blockNode]struct{})
+ }
+
+ bi.Unlock()
+ return err
+}
diff --git a/blockchain/blockindex_test.go b/blockchain/blockindex_test.go
new file mode 100644
index 0000000..cd08969
--- /dev/null
+++ b/blockchain/blockindex_test.go
@@ -0,0 +1,42 @@
+// Copyright (c) 2023 The utreexo developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package blockchain
+
+import (
+ "math/rand"
+ "testing"
+)
+
+func TestAncestor(t *testing.T) {
+ height := 500_000
+ blockNodes := chainedNodes(nil, height)
+
+ for i, blockNode := range blockNodes {
+ // Grab a random node that's a child of this node
+ // and try to fetch the current blockNode with Ancestor.
+ randNode := blockNodes[rand.Intn(height-i)+i]
+ got := randNode.Ancestor(blockNode.height)
+
+ // See if we got the right one.
+ if got.hash != blockNode.hash {
+ t.Fatalf("expected ancestor at height %d "+
+ "but got a node at height %d",
+ blockNode.height, got.height)
+ }
+
+ // Gensis doesn't have ancestors so skip the check below.
+ if blockNode.height == 0 {
+ continue
+ }
+
+ // The ancestors are deterministic so check that this node's
+ // ancestor is the correct one.
+ if blockNode.ancestor.height != getAncestorHeight(blockNode.height) {
+ t.Fatalf("expected anestor at height %d, but it was at %d",
+ getAncestorHeight(blockNode.height),
+ blockNode.ancestor.height)
+ }
+ }
+}
diff --git a/blockchain/chain.go b/blockchain/chain.go
new file mode 100644
index 0000000..952d0bc
--- /dev/null
+++ b/blockchain/chain.go
@@ -0,0 +1,2257 @@
+// Copyright (c) 2013-2018 The btcsuite developers
+// Copyright (c) 2015-2018 The Decred developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package blockchain
+
+import (
+ "container/list"
+ "fmt"
+ "sync"
+ "time"
+
+ "github.com/btcsuite/btcd/btcutil"
+ "github.com/btcsuite/btcd/chaincfg"
+ "github.com/btcsuite/btcd/chaincfg/chainhash"
+ "github.com/btcsuite/btcd/database"
+ "github.com/btcsuite/btcd/txscript"
+ "github.com/btcsuite/btcd/wire"
+)
+
+const (
+ // maxOrphanBlocks is the maximum number of orphan blocks that can be
+ // queued.
+ maxOrphanBlocks = 100
+)
+
+// BlockLocator is used to help locate a specific block. The algorithm for
+// building the block locator is to add the hashes in reverse order until
+// the genesis block is reached. In order to keep the list of locator hashes
+// to a reasonable number of entries, first the most recent previous 12 block
+// hashes are added, then the step is doubled each loop iteration to
+// exponentially decrease the number of hashes as a function of the distance
+// from the block being located.
+//
+// For example, assume a block chain with a side chain as depicted below:
+//
+// genesis -> 1 -> 2 -> ... -> 15 -> 16 -> 17 -> 18
+// \-> 16a -> 17a
+//
+// The block locator for block 17a would be the hashes of blocks:
+// [17a 16a 15 14 13 12 11 10 9 8 7 6 4 genesis]
+type BlockLocator []*chainhash.Hash
+
+// orphanBlock represents a block that we don't yet have the parent for. It
+// is a normal block plus an expiration time to prevent caching the orphan
+// forever.
+type orphanBlock struct {
+ block *btcutil.Block
+ expiration time.Time
+}
+
+// BestState houses information about the current best block and other info
+// related to the state of the main chain as it exists from the point of view of
+// the current best block.
+//
+// The BestSnapshot method can be used to obtain access to this information
+// in a concurrent safe manner and the data will not be changed out from under
+// the caller when chain state changes occur as the function name implies.
+// However, the returned snapshot must be treated as immutable since it is
+// shared by all callers.
+type BestState struct {
+ Hash chainhash.Hash // The hash of the block.
+ Height int32 // The height of the block.
+ Bits uint32 // The difficulty bits of the block.
+ BlockSize uint64 // The size of the block.
+ BlockWeight uint64 // The weight of the block.
+ NumTxns uint64 // The number of txns in the block.
+ TotalTxns uint64 // The total number of txns in the chain.
+ MedianTime time.Time // Median time as per CalcPastMedianTime.
+}
+
+// newBestState returns a new best stats instance for the given parameters.
+func newBestState(node *blockNode, blockSize, blockWeight, numTxns,
+ totalTxns uint64, medianTime time.Time) *BestState {
+
+ return &BestState{
+ Hash: node.hash,
+ Height: node.height,
+ Bits: node.bits,
+ BlockSize: blockSize,
+ BlockWeight: blockWeight,
+ NumTxns: numTxns,
+ TotalTxns: totalTxns,
+ MedianTime: medianTime,
+ }
+}
+
+// BlockChain provides functions for working with the bitcoin block chain.
+// It includes functionality such as rejecting duplicate blocks, ensuring blocks
+// follow all rules, orphan handling, checkpoint handling, and best chain
+// selection with reorganization.
+type BlockChain struct {
+ // The following fields are set when the instance is created and can't
+ // be changed afterwards, so there is no need to protect them with a
+ // separate mutex.
+ checkpoints []chaincfg.Checkpoint
+ checkpointsByHeight map[int32]*chaincfg.Checkpoint
+ db database.DB
+ chainParams *chaincfg.Params
+ timeSource MedianTimeSource
+ sigCache *txscript.SigCache
+ indexManager IndexManager
+ hashCache *txscript.HashCache
+
+ // The following fields are calculated based upon the provided chain
+ // parameters. They are also set when the instance is created and
+ // can't be changed afterwards, so there is no need to protect them with
+ // a separate mutex.
+ minRetargetTimespan int64 // target timespan / adjustment factor
+ maxRetargetTimespan int64 // target timespan * adjustment factor
+ blocksPerRetarget int32 // target timespan / target time per block
+
+ // chainLock protects concurrent access to the vast majority of the
+ // fields in this struct below this point.
+ chainLock sync.RWMutex
+
+ // pruneTarget is the size in bytes the database targets for when the node
+ // is pruned.
+ pruneTarget uint64
+
+ // These fields are related to the memory block index. They both have
+ // their own locks, however they are often also protected by the chain
+ // lock to help prevent logic races when blocks are being processed.
+ //
+ // index houses the entire block index in memory. The block index is
+ // a tree-shaped structure.
+ //
+ // bestChain tracks the current active chain by making use of an
+ // efficient chain view into the block index.
+ index *blockIndex
+ bestChain *chainView
+
+ // The UTXO state holds a cached view of the UTXO state of the chain.
+ // It is protected by the chain lock.
+ utxoCache *utxoCache
+
+ // These fields are related to handling of orphan blocks. They are
+ // protected by a combination of the chain lock and the orphan lock.
+ orphanLock sync.RWMutex
+ orphans map[chainhash.Hash]*orphanBlock
+ prevOrphans map[chainhash.Hash][]*orphanBlock
+ oldestOrphan *orphanBlock
+
+ // These fields are related to checkpoint handling. They are protected
+ // by the chain lock.
+ nextCheckpoint *chaincfg.Checkpoint
+ checkpointNode *blockNode
+
+ // The state is used as a fairly efficient way to cache information
+ // about the current best chain state that is returned to callers when
+ // requested. It operates on the principle of MVCC such that any time a
+ // new block becomes the best block, the state pointer is replaced with
+ // a new struct and the old state is left untouched. In this way,
+ // multiple callers can be pointing to different best chain states.
+ // This is acceptable for most callers because the state is only being
+ // queried at a specific point in time.
+ //
+ // In addition, some of the fields are stored in the database so the
+ // chain state can be quickly reconstructed on load.
+ stateLock sync.RWMutex
+ stateSnapshot *BestState
+
+ // The following caches are used to efficiently keep track of the
+ // current deployment threshold state of each rule change deployment.
+ //
+ // This information is stored in the database so it can be quickly
+ // reconstructed on load.
+ //
+ // warningCaches caches the current deployment threshold state for blocks
+ // in each of the **possible** deployments. This is used in order to
+ // detect when new unrecognized rule changes are being voted on and/or
+ // have been activated such as will be the case when older versions of
+ // the software are being used
+ //
+ // deploymentCaches caches the current deployment threshold state for
+ // blocks in each of the actively defined deployments.
+ warningCaches []thresholdStateCache
+ deploymentCaches []thresholdStateCache
+
+ // The following fields are used to determine if certain warnings have
+ // already been shown.
+ //
+ // unknownRulesWarned refers to warnings due to unknown rules being
+ // activated.
+ unknownRulesWarned bool
+
+ // The notifications field stores a slice of callbacks to be executed on
+ // certain blockchain events.
+ notificationsLock sync.RWMutex
+ notifications []NotificationCallback
+}
+
+// HaveBlock returns whether or not the chain instance has the block represented
+// by the passed hash. This includes checking the various places a block can
+// be like part of the main chain, on a side chain, or in the orphan pool.
+//
+// This function is safe for concurrent access.
+func (b *BlockChain) HaveBlock(hash *chainhash.Hash) (bool, error) {
+ exists, err := b.blockExists(hash)
+ if err != nil {
+ return false, err
+ }
+ return exists || b.IsKnownOrphan(hash), nil
+}
+
+// IsKnownOrphan returns whether the passed hash is currently a known orphan.
+// Keep in mind that only a limited number of orphans are held onto for a
+// limited amount of time, so this function must not be used as an absolute
+// way to test if a block is an orphan block. A full block (as opposed to just
+// its hash) must be passed to ProcessBlock for that purpose. However, calling
+// ProcessBlock with an orphan that already exists results in an error, so this
+// function provides a mechanism for a caller to intelligently detect *recent*
+// duplicate orphans and react accordingly.
+//
+// This function is safe for concurrent access.
+func (b *BlockChain) IsKnownOrphan(hash *chainhash.Hash) bool {
+ // Protect concurrent access. Using a read lock only so multiple
+ // readers can query without blocking each other.
+ b.orphanLock.RLock()
+ _, exists := b.orphans[*hash]
+ b.orphanLock.RUnlock()
+
+ return exists
+}
+
+// GetOrphanRoot returns the head of the chain for the provided hash from the
+// map of orphan blocks.
+//
+// This function is safe for concurrent access.
+func (b *BlockChain) GetOrphanRoot(hash *chainhash.Hash) *chainhash.Hash {
+ // Protect concurrent access. Using a read lock only so multiple
+ // readers can query without blocking each other.
+ b.orphanLock.RLock()
+ defer b.orphanLock.RUnlock()
+
+ // Keep looping while the parent of each orphaned block is
+ // known and is an orphan itself.
+ orphanRoot := hash
+ prevHash := hash
+ for {
+ orphan, exists := b.orphans[*prevHash]
+ if !exists {
+ break
+ }
+ orphanRoot = prevHash
+ prevHash = &orphan.block.MsgBlock().Header.PrevBlock
+ }
+
+ return orphanRoot
+}
+
+// removeOrphanBlock removes the passed orphan block from the orphan pool and
+// previous orphan index.
+func (b *BlockChain) removeOrphanBlock(orphan *orphanBlock) {
+ // Protect concurrent access.
+ b.orphanLock.Lock()
+ defer b.orphanLock.Unlock()
+
+ // Remove the orphan block from the orphan pool.
+ orphanHash := orphan.block.Hash()
+ delete(b.orphans, *orphanHash)
+
+ // Remove the reference from the previous orphan index too. An indexing
+ // for loop is intentionally used over a range here as range does not
+ // reevaluate the slice on each iteration nor does it adjust the index
+ // for the modified slice.
+ prevHash := &orphan.block.MsgBlock().Header.PrevBlock
+ orphans := b.prevOrphans[*prevHash]
+ for i := 0; i < len(orphans); i++ {
+ hash := orphans[i].block.Hash()
+ if hash.IsEqual(orphanHash) {
+ copy(orphans[i:], orphans[i+1:])
+ orphans[len(orphans)-1] = nil
+ orphans = orphans[:len(orphans)-1]
+ i--
+ }
+ }
+ b.prevOrphans[*prevHash] = orphans
+
+ // Remove the map entry altogether if there are no longer any orphans
+ // which depend on the parent hash.
+ if len(b.prevOrphans[*prevHash]) == 0 {
+ delete(b.prevOrphans, *prevHash)
+ }
+}
+
+// addOrphanBlock adds the passed block (which is already determined to be
+// an orphan prior calling this function) to the orphan pool. It lazily cleans
+// up any expired blocks so a separate cleanup poller doesn't need to be run.
+// It also imposes a maximum limit on the number of outstanding orphan
+// blocks and will remove the oldest received orphan block if the limit is
+// exceeded.
+func (b *BlockChain) addOrphanBlock(block *btcutil.Block) {
+ // Remove expired orphan blocks.
+ for _, oBlock := range b.orphans {
+ if time.Now().After(oBlock.expiration) {
+ b.removeOrphanBlock(oBlock)
+ continue
+ }
+
+ // Update the oldest orphan block pointer so it can be discarded
+ // in case the orphan pool fills up.
+ if b.oldestOrphan == nil || oBlock.expiration.Before(b.oldestOrphan.expiration) {
+ b.oldestOrphan = oBlock
+ }
+ }
+
+ // Limit orphan blocks to prevent memory exhaustion.
+ if len(b.orphans)+1 > maxOrphanBlocks {
+ // Remove the oldest orphan to make room for the new one.
+ b.removeOrphanBlock(b.oldestOrphan)
+ b.oldestOrphan = nil
+ }
+
+ // Protect concurrent access. This is intentionally done here instead
+ // of near the top since removeOrphanBlock does its own locking and
+ // the range iterator is not invalidated by removing map entries.
+ b.orphanLock.Lock()
+ defer b.orphanLock.Unlock()
+
+ // Insert the block into the orphan map with an expiration time
+ // 1 hour from now.
+ expiration := time.Now().Add(time.Hour)
+ oBlock := &orphanBlock{
+ block: block,
+ expiration: expiration,
+ }
+ b.orphans[*block.Hash()] = oBlock
+
+ // Add to previous hash lookup index for faster dependency lookups.
+ prevHash := &block.MsgBlock().Header.PrevBlock
+ b.prevOrphans[*prevHash] = append(b.prevOrphans[*prevHash], oBlock)
+}
+
+// SequenceLock represents the converted relative lock-time in seconds, and
+// absolute block-height for a transaction input's relative lock-times.
+// According to SequenceLock, after the referenced input has been confirmed
+// within a block, a transaction spending that input can be included into a
+// block either after 'seconds' (according to past median time), or once the
+// 'BlockHeight' has been reached.
+type SequenceLock struct {
+ Seconds int64
+ BlockHeight int32
+}
+
+// CalcSequenceLock computes a relative lock-time SequenceLock for the passed
+// transaction using the passed UtxoViewpoint to obtain the past median time
+// for blocks in which the referenced inputs of the transactions were included
+// within. The generated SequenceLock lock can be used in conjunction with a
+// block height, and adjusted median block time to determine if all the inputs
+// referenced within a transaction have reached sufficient maturity allowing
+// the candidate transaction to be included in a block.
+//
+// This function is safe for concurrent access.
+func (b *BlockChain) CalcSequenceLock(tx *btcutil.Tx, utxoView *UtxoViewpoint, mempool bool) (*SequenceLock, error) {
+ b.chainLock.Lock()
+ defer b.chainLock.Unlock()
+
+ return b.calcSequenceLock(b.bestChain.Tip(), tx, utxoView, mempool)
+}
+
+// calcSequenceLock computes the relative lock-times for the passed
+// transaction. See the exported version, CalcSequenceLock for further details.
+//
+// This function MUST be called with the chain state lock held (for writes).
+func (b *BlockChain) calcSequenceLock(node *blockNode, tx *btcutil.Tx, utxoView *UtxoViewpoint, mempool bool) (*SequenceLock, error) {
+ // A value of -1 for each relative lock type represents a relative time
+ // lock value that will allow a transaction to be included in a block
+ // at any given height or time. This value is returned as the relative
+ // lock time in the case that BIP 68 is disabled, or has not yet been
+ // activated.
+ sequenceLock := &SequenceLock{Seconds: -1, BlockHeight: -1}
+
+ // The sequence locks semantics are always active for transactions
+ // within the mempool.
+ csvSoftforkActive := mempool
+
+ // If we're performing block validation, then we need to query the BIP9
+ // state.
+ if !csvSoftforkActive {
+ // Obtain the latest BIP9 version bits state for the
+ // CSV-package soft-fork deployment. The adherence of sequence
+ // locks depends on the current soft-fork state.
+ csvState, err := b.deploymentState(node.parent, chaincfg.DeploymentCSV)
+ if err != nil {
+ return nil, err
+ }
+ csvSoftforkActive = csvState == ThresholdActive
+ }
+
+ // If the transaction's version is less than 2, and BIP 68 has not yet
+ // been activated then sequence locks are disabled. Additionally,
+ // sequence locks don't apply to coinbase transactions Therefore, we
+ // return sequence lock values of -1 indicating that this transaction
+ // can be included within a block at any given height or time.
+ mTx := tx.MsgTx()
+ sequenceLockActive := uint32(mTx.Version) >= 2 && csvSoftforkActive
+ if !sequenceLockActive || IsCoinBase(tx) {
+ return sequenceLock, nil
+ }
+
+ // Grab the next height from the PoV of the passed blockNode to use for
+ // inputs present in the mempool.
+ nextHeight := node.height + 1
+
+ for txInIndex, txIn := range mTx.TxIn {
+ utxo := utxoView.LookupEntry(txIn.PreviousOutPoint)
+ if utxo == nil {
+ str := fmt.Sprintf("output %v referenced from "+
+ "transaction %s:%d either does not exist or "+
+ "has already been spent", txIn.PreviousOutPoint,
+ tx.Hash(), txInIndex)
+ return sequenceLock, ruleError(ErrMissingTxOut, str)
+ }
+
+ // If the input height is set to the mempool height, then we
+ // assume the transaction makes it into the next block when
+ // evaluating its sequence blocks.
+ inputHeight := utxo.BlockHeight()
+ if inputHeight == 0x7fffffff {
+ inputHeight = nextHeight
+ }
+
+ // Given a sequence number, we apply the relative time lock
+ // mask in order to obtain the time lock delta required before
+ // this input can be spent.
+ sequenceNum := txIn.Sequence
+ relativeLock := int64(sequenceNum & wire.SequenceLockTimeMask)
+
+ switch {
+ // Relative time locks are disabled for this input, so we can
+ // skip any further calculation.
+ case sequenceNum&wire.SequenceLockTimeDisabled == wire.SequenceLockTimeDisabled:
+ continue
+ case sequenceNum&wire.SequenceLockTimeIsSeconds == wire.SequenceLockTimeIsSeconds:
+ // This input requires a relative time lock expressed
+ // in seconds before it can be spent. Therefore, we
+ // need to query for the block prior to the one in
+ // which this input was included within so we can
+ // compute the past median time for the block prior to
+ // the one which included this referenced output.
+ prevInputHeight := inputHeight - 1
+ if prevInputHeight < 0 {
+ prevInputHeight = 0
+ }
+ blockNode := node.Ancestor(prevInputHeight)
+ medianTime := CalcPastMedianTime(blockNode)
+
+ // Time based relative time-locks as defined by BIP 68
+ // have a time granularity of RelativeLockSeconds, so
+ // we shift left by this amount to convert to the
+ // proper relative time-lock. We also subtract one from
+ // the relative lock to maintain the original lockTime
+ // semantics.
+ timeLockSeconds := (relativeLock << wire.SequenceLockTimeGranularity) - 1
+ timeLock := medianTime.Unix() + timeLockSeconds
+ if timeLock > sequenceLock.Seconds {
+ sequenceLock.Seconds = timeLock
+ }
+ default:
+ // The relative lock-time for this input is expressed
+ // in blocks so we calculate the relative offset from
+ // the input's height as its converted absolute
+ // lock-time. We subtract one from the relative lock in
+ // order to maintain the original lockTime semantics.
+ blockHeight := inputHeight + int32(relativeLock-1)
+ if blockHeight > sequenceLock.BlockHeight {
+ sequenceLock.BlockHeight = blockHeight
+ }
+ }
+ }
+
+ return sequenceLock, nil
+}
+
+// LockTimeToSequence converts the passed relative locktime to a sequence
+// number in accordance to BIP-68.
+// See: https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki
+// - (Compatibility)
+func LockTimeToSequence(isSeconds bool, locktime uint32) uint32 {
+ // If we're expressing the relative lock time in blocks, then the
+ // corresponding sequence number is simply the desired input age.
+ if !isSeconds {
+ return locktime
+ }
+
+ // Set the 22nd bit which indicates the lock time is in seconds, then
+ // shift the locktime over by 9 since the time granularity is in
+ // 512-second intervals (2^9). This results in a max lock-time of
+ // 33,553,920 seconds, or 1.1 years.
+ return wire.SequenceLockTimeIsSeconds |
+ locktime>>wire.SequenceLockTimeGranularity
+}
+
+// getReorganizeNodes finds the fork point between the main chain and the passed
+// node and returns a list of block nodes that would need to be detached from
+// the main chain and a list of block nodes that would need to be attached to
+// the fork point (which will be the end of the main chain after detaching the
+// returned list of block nodes) in order to reorganize the chain such that the
+// passed node is the new end of the main chain. The lists will be empty if the
+// passed node is not on a side chain.
+//
+// This function may modify node statuses in the block index without flushing.
+//
+// This function MUST be called with the chain state lock held (for reads).
+func (b *BlockChain) getReorganizeNodes(node *blockNode) (*list.List, *list.List) {
+ attachNodes := list.New()
+ detachNodes := list.New()
+
+ // Do not reorganize to a known invalid chain. Ancestors deeper than the
+ // direct parent are checked below but this is a quick check before doing
+ // more unnecessary work.
+ if b.index.NodeStatus(node.parent).KnownInvalid() {
+ b.index.SetStatusFlags(node, statusInvalidAncestor)
+ return detachNodes, attachNodes
+ }
+
+ // Find the fork point (if any) adding each block to the list of nodes
+ // to attach to the main tree. Push them onto the list in reverse order
+ // so they are attached in the appropriate order when iterating the list
+ // later.
+ forkNode := b.bestChain.FindFork(node)
+ invalidChain := false
+ for n := node; n != nil && n != forkNode; n = n.parent {
+ if b.index.NodeStatus(n).KnownInvalid() {
+ invalidChain = true
+ break
+ }
+ attachNodes.PushFront(n)
+ }
+
+ // If any of the node's ancestors are invalid, unwind attachNodes, marking
+ // each one as invalid for future reference.
+ if invalidChain {
+ var next *list.Element
+ for e := attachNodes.Front(); e != nil; e = next {
+ next = e.Next()
+ n := attachNodes.Remove(e).(*blockNode)
+ b.index.SetStatusFlags(n, statusInvalidAncestor)
+ }
+ return detachNodes, attachNodes
+ }
+
+ // Start from the end of the main chain and work backwards until the
+ // common ancestor adding each block to the list of nodes to detach from
+ // the main chain.
+ for n := b.bestChain.Tip(); n != nil && n != forkNode; n = n.parent {
+ detachNodes.PushBack(n)
+ }
+
+ return detachNodes, attachNodes
+}
+
+// connectBlock handles connecting the passed node/block to the end of the main
+// (best) chain.
+//
+// Passing in a utxo view is optional. If the passed in utxo view is nil,
+// connectBlock will assume that the utxo cache has already connected all the
+// txs in the block being connected.
+// If a utxo view is passed in, this passed utxo view must have all referenced
+// txos the block spends marked as spent and all of the new txos the block creates
+// added to it.
+//
+// The passed stxos slice must be populated with all of the information for the
+// spent txos. This approach is used because the connection validation that
+// must happen prior to calling this function requires the same details, so
+// it would be inefficient to repeat it.
+//
+// This function MUST be called with the chain state lock held (for writes).
+func (b *BlockChain) connectBlock(node *blockNode, block *btcutil.Block,
+ stxos []SpentTxOut) error {
+
+ // Make sure it's extending the end of the best chain.
+ prevHash := &block.MsgBlock().Header.PrevBlock
+ if !prevHash.IsEqual(&b.bestChain.Tip().hash) {
+ return AssertError("connectBlock must be called with a block " +
+ "that extends the main chain")
+ }
+
+ // Sanity check the correct number of stxos are provided.
+ if len(stxos) != countSpentOutputs(block) {
+ return AssertError("connectBlock called with inconsistent " +
+ "spent transaction out information")
+ }
+
+ // No warnings about unknown rules until the chain is current.
+ if b.isCurrent() {
+ // Warn if any unknown new rules are either about to activate or
+ // have already been activated.
+ if err := b.warnUnknownRuleActivations(node); err != nil {
+ return err
+ }
+ }
+
+ // Write any block status changes to DB before updating best state.
+ err := b.index.flushToDB()
+ if err != nil {
+ return err
+ }
+
+ // Generate a new best state snapshot that will be used to update the
+ // database and later memory if all database updates are successful.
+ b.stateLock.RLock()
+ curTotalTxns := b.stateSnapshot.TotalTxns
+ b.stateLock.RUnlock()
+ numTxns := uint64(len(block.MsgBlock().Transactions))
+ blockSize := uint64(block.MsgBlock().SerializeSize())
+ blockWeight := uint64(GetBlockWeight(block))
+ state := newBestState(node, blockSize, blockWeight, numTxns,
+ curTotalTxns+numTxns, CalcPastMedianTime(node),
+ )
+
+ // Atomically insert info into the database.
+ err = b.db.Update(func(dbTx database.Tx) error {
+ // If the pruneTarget isn't 0, we should attempt to delete older blocks
+ // from the database.
+ if b.pruneTarget != 0 {
+ // When the total block size is under the prune target, prune blocks is
+ // a no-op and the deleted hashes are nil.
+ deletedHashes, err := dbTx.PruneBlocks(b.pruneTarget)
+ if err != nil {
+ return err
+ }
+
+ // Only attempt to delete if we have any deleted blocks.
+ if len(deletedHashes) != 0 {
+ // Delete the spend journals of the pruned blocks.
+ err = dbPruneSpendJournalEntry(dbTx, deletedHashes)
+ if err != nil {
+ return err
+ }
+
+ // We may need to flush if the prune will delete blocks that
+ // are past our last flush block.
+ //
+ // NOTE: the database will never be inconsistent here as the
+ // actual blocks are not deleted until the db.Update returns.
+ needsFlush, err := b.flushNeededAfterPrune(deletedHashes)
+ if err != nil {
+ return err
+ }
+ if needsFlush {
+ // Since the deleted hashes are past our last
+ // flush block, flush the utxo cache now.
+ err = b.utxoCache.flush(dbTx, FlushRequired, state)
+ if err != nil {
+ return err
+ }
+ }
+ }
+ }
+
+ // Update best block state.
+ err := dbPutBestState(dbTx, state, node.workSum)
+ if err != nil {
+ return err
+ }
+
+ // Add the block hash and height to the block index which tracks
+ // the main chain.
+ err = dbPutBlockIndex(dbTx, block.Hash(), node.height)
+ if err != nil {
+ return err
+ }
+
+ // Update the transaction spend journal by adding a record for
+ // the block that contains all txos spent by it.
+ err = dbPutSpendJournalEntry(dbTx, block.Hash(), stxos)
+ if err != nil {
+ return err
+ }
+
+ // Allow the index manager to call each of the currently active
+ // optional indexes with the block being connected so they can
+ // update themselves accordingly.
+ if b.indexManager != nil {
+ err := b.indexManager.ConnectBlock(dbTx, block, stxos)
+ if err != nil {
+ return err
+ }
+ }
+
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+
+ // This node is now the end of the best chain.
+ b.bestChain.SetTip(node)
+
+ // Update the state for the best block. Notice how this replaces the
+ // entire struct instead of updating the existing one. This effectively
+ // allows the old version to act as a snapshot which callers can use
+ // freely without needing to hold a lock for the duration. See the
+ // comments on the state variable for more details.
+ b.stateLock.Lock()
+ b.stateSnapshot = state
+ b.stateLock.Unlock()
+
+ // Notify the caller that the block was connected to the main chain.
+ // The caller would typically want to react with actions such as
+ // updating wallets.
+ func() {
+ b.chainLock.Unlock()
+ defer b.chainLock.Lock()
+ b.sendNotification(NTBlockConnected, block)
+ }()
+
+ // Since we may have changed the UTXO cache, we make sure it didn't exceed its
+ // maximum size. If we're pruned and have flushed already, this will be a no-op.
+ return b.db.Update(func(dbTx database.Tx) error {
+ return b.utxoCache.flush(dbTx, FlushIfNeeded, state)
+ })
+}
+
+// disconnectBlock handles disconnecting the passed node/block from the end of
+// the main (best) chain.
+//
+// This function MUST be called with the chain state lock held (for writes).
+func (b *BlockChain) disconnectBlock(node *blockNode, block *btcutil.Block, view *UtxoViewpoint) error {
+ // Make sure the node being disconnected is the end of the best chain.
+ if !node.hash.IsEqual(&b.bestChain.Tip().hash) {
+ return AssertError("disconnectBlock must be called with the " +
+ "block at the end of the main chain")
+ }
+
+ // Load the previous block since some details for it are needed below.
+ prevNode := node.parent
+ var prevBlock *btcutil.Block
+ err := b.db.View(func(dbTx database.Tx) error {
+ var err error
+ prevBlock, err = dbFetchBlockByNode(dbTx, prevNode)
+ return err
+ })
+ if err != nil {
+ return err
+ }
+
+ // Write any block status changes to DB before updating best state.
+ err = b.index.flushToDB()
+ if err != nil {
+ return err
+ }
+
+ // Generate a new best state snapshot that will be used to update the
+ // database and later memory if all database updates are successful.
+ b.stateLock.RLock()
+ curTotalTxns := b.stateSnapshot.TotalTxns
+ b.stateLock.RUnlock()
+ numTxns := uint64(len(prevBlock.MsgBlock().Transactions))
+ blockSize := uint64(prevBlock.MsgBlock().SerializeSize())
+ blockWeight := uint64(GetBlockWeight(prevBlock))
+ newTotalTxns := curTotalTxns - uint64(len(block.MsgBlock().Transactions))
+ state := newBestState(prevNode, blockSize, blockWeight, numTxns,
+ newTotalTxns, CalcPastMedianTime(prevNode))
+
+ err = b.db.Update(func(dbTx database.Tx) error {
+ // Update best block state.
+ err := dbPutBestState(dbTx, state, node.workSum)
+ if err != nil {
+ return err
+ }
+
+ // Remove the block hash and height from the block index which
+ // tracks the main chain.
+ err = dbRemoveBlockIndex(dbTx, block.Hash(), node.height)
+ if err != nil {
+ return err
+ }
+
+ // Flush the cache on every disconnect. Since the code for
+ // reorganization modifies the database directly, the cache
+ // will be left in an inconsistent state if we don't flush it
+ // prior to the dbPutUtxoView that happens below.
+ err = b.utxoCache.flush(dbTx, FlushRequired, state)
+ if err != nil {
+ return err
+ }
+
+ // Update the utxo set using the state of the utxo view. This
+ // entails restoring all of the utxos spent and removing the new
+ // ones created by the block.
+ err = dbPutUtxoView(dbTx, view)
+ if err != nil {
+ return err
+ }
+
+ // Before we delete the spend journal entry for this back,
+ // we'll fetch it as is so the indexers can utilize if needed.
+ stxos, err := dbFetchSpendJournalEntry(dbTx, block)
+ if err != nil {
+ return err
+ }
+
+ // Update the transaction spend journal by removing the record
+ // that contains all txos spent by the block.
+ err = dbRemoveSpendJournalEntry(dbTx, block.Hash())
+ if err != nil {
+ return err
+ }
+
+ // Allow the index manager to call each of the currently active
+ // optional indexes with the block being disconnected so they
+ // can update themselves accordingly.
+ if b.indexManager != nil {
+ err := b.indexManager.DisconnectBlock(dbTx, block, stxos)
+ if err != nil {
+ return err
+ }
+ }
+
+ return nil
+ })
+ if err != nil {
+ return err
+ }
+
+ // Prune fully spent entries and mark all entries in the view unmodified
+ // now that the modifications have been committed to the database.
+ view.commit()
+
+ // This node's parent is now the end of the best chain.
+ b.bestChain.SetTip(node.parent)
+
+ // Update the state for the best block. Notice how this replaces the
+ // entire struct instead of updating the existing one. This effectively
+ // allows the old version to act as a snapshot which callers can use
+ // freely without needing to hold a lock for the duration. See the
+ // comments on the state variable for more details.
+ b.stateLock.Lock()
+ b.stateSnapshot = state
+ b.stateLock.Unlock()
+
+ // Notify the caller that the block was disconnected from the main
+ // chain. The caller would typically want to react with actions such as
+ // updating wallets.
+ func() {
+ b.chainLock.Unlock()
+ defer b.chainLock.Lock()
+ b.sendNotification(NTBlockDisconnected, block)
+ }()
+
+ return nil
+}
+
+// countSpentOutputs returns the number of utxos the passed block spends.
+func countSpentOutputs(block *btcutil.Block) int {
+ // Exclude the coinbase transaction since it can't spend anything.
+ var numSpent int
+ for _, tx := range block.Transactions()[1:] {
+ numSpent += len(tx.MsgTx().TxIn)
+ }
+ return numSpent
+}
+
+// reorganizeChain reorganizes the block chain by disconnecting the nodes in the
+// detachNodes list and connecting the nodes in the attach list. It expects
+// that the lists are already in the correct order and are in sync with the
+// end of the current best chain. Specifically, nodes that are being
+// disconnected must be in reverse order (think of popping them off the end of
+// the chain) and nodes the are being attached must be in forwards order
+// (think pushing them onto the end of the chain).
+//
+// This function may modify node statuses in the block index without flushing.
+//
+// This function never leaves the utxo set in an inconsistent state for block
+// disconnects.
+//
+// This function MUST be called with the chain state lock held (for writes).
+func (b *BlockChain) reorganizeChain(detachNodes, attachNodes *list.List) error {
+ // Check first that the detach and the attach nodes are valid and they
+ // pass verification.
+ detachBlocks, attachBlocks, detachSpentTxOuts,
+ err := b.verifyReorganizationValidity(detachNodes, attachNodes)
+ if err != nil {
+ return err
+ }
+
+ // Track the old and new best chains heads.
+ tip := b.bestChain.Tip()
+ oldBest := tip
+ newBest := tip
+
+ // Reset the view for the actual connection code below. This is
+ // required because the view was previously modified when checking if
+ // the reorg would be successful and the connection code requires the
+ // view to be valid from the viewpoint of each block being disconnected.
+ view := NewUtxoViewpoint()
+ view.SetBestHash(&b.bestChain.Tip().hash)
+
+ // Disconnect blocks from the main chain.
+ for i, e := 0, detachNodes.Front(); e != nil; i, e = i+1, e.Next() {
+ n := e.Value.(*blockNode)
+ block := detachBlocks[i]
+
+ // Load all of the utxos referenced by the block that aren't
+ // already in the view.
+ err := view.fetchInputUtxos(b.utxoCache, block)
+ if err != nil {
+ return err
+ }
+
+ // Update the view to unspend all of the spent txos and remove
+ // the utxos created by the block.
+ err = view.disconnectTransactions(
+ b.db, block, detachSpentTxOuts[i],
+ )
+ if err != nil {
+ return err
+ }
+
+ // Update the database and chain state. The cache will be flushed
+ // here before the utxoview modifications happen to the database.
+ err = b.disconnectBlock(n, block, view)
+ if err != nil {
+ return err
+ }
+
+ newBest = n.parent
+ }
+
+ // Set the fork point only if there are nodes to attach since otherwise
+ // blocks are only being disconnected and thus there is no fork point.
+ var forkNode *blockNode
+ if attachNodes.Len() > 0 {
+ forkNode = newBest
+ }
+
+ // Connect the new best chain blocks using the utxocache directly. It's more
+ // efficient and since we already checked that the blocks are correct and that
+ // the transactions connect properly, it's ok to access the cache. If we suddenly
+ // crash here, we are able to recover as well.
+ for i, e := 0, attachNodes.Front(); e != nil; i, e = i+1, e.Next() {
+ n := e.Value.(*blockNode)
+ block := attachBlocks[i]
+
+ // Update the cache to mark all utxos referenced by the block
+ // as spent and add all transactions being created by this block
+ // to it. Also, provide an stxo slice so the spent txout
+ // details are generated.
+ stxos := make([]SpentTxOut, 0, countSpentOutputs(block))
+ err = b.utxoCache.connectTransactions(block, &stxos)
+ if err != nil {
+ return err
+ }
+
+ // Update the database and chain state.
+ err = b.connectBlock(n, block, stxos)
+ if err != nil {
+ return err
+ }
+
+ newBest = n
+ }
+
+ // Log the point where the chain forked and old and new best chain
+ // heads.
+ if forkNode != nil {
+ log.Infof("REORGANIZE: Chain forks at %v (height %v)", forkNode.hash,
+ forkNode.height)
+ }
+ log.Infof("REORGANIZE: Old best chain head was %v (height %v)",
+ &oldBest.hash, oldBest.height)
+ log.Infof("REORGANIZE: New best chain head is %v (height %v)",
+ newBest.hash, newBest.height)
+
+ return nil
+}
+
+// verifyReorganizationValidity will verify that the disconnects and the connects
+// that are in the list are able to be processed without mutating the chain.
+//
+// For the attach nodes, it'll check that each of the blocks are valid and will
+// change the status of the block node in the list to invalid if the block fails
+// to pass verification. For the detach nodes, it'll check that the blocks being
+// detached and their spend journals are present on the database.
+func (b *BlockChain) verifyReorganizationValidity(detachNodes, attachNodes *list.List) (
+ []*btcutil.Block, []*btcutil.Block, [][]SpentTxOut, error) {
+
+ // Nothing to do if no reorganize nodes were provided.
+ if detachNodes.Len() == 0 && attachNodes.Len() == 0 {
+ return nil, nil, nil, nil
+ }
+
+ // Ensure the provided nodes match the current best chain.
+ tip := b.bestChain.Tip()
+ if detachNodes.Len() != 0 {
+ firstDetachNode := detachNodes.Front().Value.(*blockNode)
+ if firstDetachNode.hash != tip.hash {
+ return nil, nil, nil,
+ AssertError(fmt.Sprintf("reorganize nodes to detach are "+
+ "not for the current best chain -- first detach node %v, "+
+ "current chain %v", &firstDetachNode.hash, &tip.hash))
+ }
+ }
+
+ // Ensure the provided nodes are for the same fork point.
+ if attachNodes.Len() != 0 && detachNodes.Len() != 0 {
+ firstAttachNode := attachNodes.Front().Value.(*blockNode)
+ lastDetachNode := detachNodes.Back().Value.(*blockNode)
+ if firstAttachNode.parent.hash != lastDetachNode.parent.hash {
+ return nil, nil, nil,
+ AssertError(fmt.Sprintf("reorganize nodes do not have the "+
+ "same fork point -- first attach parent %v, last detach "+
+ "parent %v", &firstAttachNode.parent.hash,
+ &lastDetachNode.parent.hash))
+ }
+ }
+
+ // All of the blocks to detach and related spend journal entries needed
+ // to unspend transaction outputs in the blocks being disconnected must
+ // be loaded from the database during the reorg check phase below and
+ // then they are needed again when doing the actual database updates.
+ // Rather than doing two loads, cache the loaded data into these slices.
+ detachBlocks := make([]*btcutil.Block, 0, detachNodes.Len())
+ detachSpentTxOuts := make([][]SpentTxOut, 0, detachNodes.Len())
+ attachBlocks := make([]*btcutil.Block, 0, attachNodes.Len())
+
+ // Disconnect all of the blocks back to the point of the fork. This
+ // entails loading the blocks and their associated spent txos from the
+ // database and using that information to unspend all of the spent txos
+ // and remove the utxos created by the blocks.
+ view := NewUtxoViewpoint()
+ view.SetBestHash(&tip.hash)
+ for e := detachNodes.Front(); e != nil; e = e.Next() {
+ n := e.Value.(*blockNode)
+ var block *btcutil.Block
+ err := b.db.View(func(dbTx database.Tx) error {
+ var err error
+ block, err = dbFetchBlockByNode(dbTx, n)
+ return err
+ })
+ if err != nil {
+ return nil, nil, nil, err
+ }
+ if n.hash != *block.Hash() {
+ return nil, nil, nil, AssertError(
+ fmt.Sprintf("detach block node hash %v (height "+
+ "%v) does not match previous parent block hash %v",
+ &n.hash, n.height, block.Hash()))
+ }
+
+ // Load all of the utxos referenced by the block that aren't
+ // already in the view.
+ err = view.fetchInputUtxos(b.utxoCache, block)
+ if err != nil {
+ return nil, nil, nil, err
+ }
+
+ // Load all of the spent txos for the block from the spend
+ // journal.
+ var stxos []SpentTxOut
+ err = b.db.View(func(dbTx database.Tx) error {
+ stxos, err = dbFetchSpendJournalEntry(dbTx, block)
+ return err
+ })
+ if err != nil {
+ return nil, nil, nil, err
+ }
+
+ // Store the loaded block and spend journal entry for later.
+ detachBlocks = append(detachBlocks, block)
+ detachSpentTxOuts = append(detachSpentTxOuts, stxos)
+
+ err = view.disconnectTransactions(b.db, block, stxos)
+ if err != nil {
+ return nil, nil, nil, err
+ }
+ }
+
+ // Perform several checks to verify each block that needs to be attached
+ // to the main chain can be connected without violating any rules and
+ // without actually connecting the block.
+ //
+ // NOTE: These checks could be done directly when connecting a block,
+ // however the downside to that approach is that if any of these checks
+ // fail after disconnecting some blocks or attaching others, all of the
+ // operations have to be rolled back to get the chain back into the
+ // state it was before the rule violation (or other failure). There are
+ // at least a couple of ways accomplish that rollback, but both involve
+ // tweaking the chain and/or database. This approach catches these
+ // issues before ever modifying the chain.
+ for e := attachNodes.Front(); e != nil; e = e.Next() {
+ n := e.Value.(*blockNode)
+
+ var block *btcutil.Block
+ err := b.db.View(func(dbTx database.Tx) error {
+ var err error
+ block, err = dbFetchBlockByNode(dbTx, n)
+ return err
+ })
+ if err != nil {
+ return nil, nil, nil, err
+ }
+
+ // Store the loaded block for later.
+ attachBlocks = append(attachBlocks, block)
+
+ // Skip checks if node has already been fully validated. Although
+ // checkConnectBlock gets skipped, we still need to update the UTXO
+ // view.
+ if b.index.NodeStatus(n).KnownValid() {
+ err = view.fetchInputUtxos(b.utxoCache, block)
+ if err != nil {
+ return nil, nil, nil, err
+ }
+ err = view.connectTransactions(block, nil)
+ if err != nil {
+ return nil, nil, nil, err
+ }
+
+ continue
+ }
+
+ // Notice the spent txout details are not requested here and
+ // thus will not be generated. This is done because the state
+ // is not being immediately written to the database, so it is
+ // not needed.
+ //
+ // In the case the block is determined to be invalid due to a
+ // rule violation, mark it as invalid and mark all of its
+ // descendants as having an invalid ancestor.
+ err = b.checkConnectBlock(n, block, view, nil)
+ if err != nil {
+ if _, ok := err.(RuleError); ok {
+ b.index.SetStatusFlags(n, statusValidateFailed)
+ for de := e.Next(); de != nil; de = de.Next() {
+ dn := de.Value.(*blockNode)
+ b.index.SetStatusFlags(dn, statusInvalidAncestor)
+ }
+ }
+ return nil, nil, nil, err
+ }
+ b.index.SetStatusFlags(n, statusValid)
+ }
+
+ return detachBlocks, attachBlocks, detachSpentTxOuts, nil
+}
+
+// connectBestChain handles connecting the passed block to the chain while
+// respecting proper chain selection according to the chain with the most
+// proof of work. In the typical case, the new block simply extends the main
+// chain. However, it may also be extending (or creating) a side chain (fork)
+// which may or may not end up becoming the main chain depending on which fork
+// cumulatively has the most proof of work. It returns whether or not the block
+// ended up on the main chain (either due to extending the main chain or causing
+// a reorganization to become the main chain).
+//
+// The flags modify the behavior of this function as follows:
+// - BFFastAdd: Avoids several expensive transaction validation operations.
+// This is useful when using checkpoints.
+//
+// This function MUST be called with the chain state lock held (for writes).
+func (b *BlockChain) connectBestChain(node *blockNode, block *btcutil.Block, flags BehaviorFlags) (bool, error) {
+ fastAdd := flags&BFFastAdd == BFFastAdd
+
+ flushIndexState := func() {
+ // Intentionally ignore errors writing updated node status to DB. If
+ // it fails to write, it's not the end of the world. If the block is
+ // valid, we flush in connectBlock and if the block is invalid, the
+ // worst that can happen is we revalidate the block after a restart.
+ if writeErr := b.index.flushToDB(); writeErr != nil {
+ log.Warnf("Error flushing block index changes to disk: %v",
+ writeErr)
+ }
+ }
+
+ // We are extending the main (best) chain with a new block. This is the
+ // most common case.
+ parentHash := &block.MsgBlock().Header.PrevBlock
+ if parentHash.IsEqual(&b.bestChain.Tip().hash) {
+ // Skip checks if node has already been fully validated.
+ fastAdd = fastAdd || b.index.NodeStatus(node).KnownValid()
+
+ // Perform several checks to verify the block can be connected
+ // to the main chain without violating any rules and without
+ // actually connecting the block.
+ if !fastAdd {
+ // We create a viewpoint here to avoid spending or adding new
+ // coins to the utxo cache.
+ //
+ // checkConnectBlock spends and adds utxos before doing the
+ // signature validation and if the signature validation fails,
+ // we would be forced to undo the utxo cache.
+ //
+ // TODO (kcalvinalvin): Doing all of the validation before connecting
+ // the tx inside check connect block would allow us to pass the utxo
+ // cache directly to the check connect block. This would save on the
+ // expensive memory allocation done by fetch input utxos.
+ view := NewUtxoViewpoint()
+ view.SetBestHash(parentHash)
+ err := b.checkConnectBlock(node, block, view, nil)
+ if err == nil {
+ b.index.SetStatusFlags(node, statusValid)
+ } else if _, ok := err.(RuleError); ok {
+ b.index.SetStatusFlags(node, statusValidateFailed)
+ } else {
+ return false, err
+ }
+
+ flushIndexState()
+
+ if err != nil {
+ return false, err
+ }
+ }
+
+ // Connect the transactions to the cache. All the txs are considered valid
+ // at this point as they have passed validation or was considered valid already.
+ stxos := make([]SpentTxOut, 0, countSpentOutputs(block))
+ err := b.utxoCache.connectTransactions(block, &stxos)
+ if err != nil {
+ return false, erRisk score
Our methodology →Why this scored 17/100
Human-validated context
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
No validated notes yet.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.