…n be shared and reused

(cherry picked from commit 5e10922)

To make it clear where DNS resolves are happening

(cherry picked from commit e9fc71e)

Note: Some seeds aren't actually returning an IP for their name entries, so
they're being added to addrman with a source of [::].

This commit shouldn't change that behavior, for better or worse.

(cherry picked from commit a98cd1f)

Rather than allowing CNetAddr/CService/CSubNet to launch DNS queries, require
that addresses are already resolved.

This greatly simplifies async resolve logic, and makes it harder to
accidentally leak DNS queries.

(cherry picked from commit 3675699)

CNetAddr/CService/CSubnet can no longer resolve DNS.

(cherry picked from commit d39f5b4)

(cherry picked from commit ee06e04)

…round use of nServices

(cherry picked from commit 31d6b1d)

(cherry picked from commit f96c7c4)

(cherry picked from commit b6c3ff3)

(cherry picked from commit 1017b8a)

(cherry picked from commit 21e5b96)

Net functionality is no longer needed for CAddress/CAddrman/etc. now that
CNetAddr/CService/CSubNet are dumb storage classes.

(cherry picked from commit 21ba407)

Also fix up a few small issues:
- Lookup with "badip:port" now sets the port to 0
- Don't allow assert to have side-effects

(cherry picked from commit 8945384)

(cherry picked from commit 9e9d644)

(cherry picked from commit ccef5d7)

-BEGIN VERIFY SCRIPT-

sed --in-place'' --expression='s/NET_TOR/NET_ONION/g' $(git grep -I --files-with-matches 'NET_TOR')

-END VERIFY SCRIPT-

The --in-place'' hack is required for sed on macOS to edit files in-place without passing a backup extension.

(cherry picked from commit 07c493f)

( cherry-plucked from: b691f2d )

In github context:

bitcoin/bitcoin@b691f2d#diff-9281caf81406a5c4e11f572f613a4cc9769f39c66579ed75a4c957f30b39d836

Do not access `CNetAddr::ip` directly from `CService` methods.

This improvement will help later when we change the type of
`CNetAddr::ip` (in the BIP155 implementation).

Co-authored-by: Carl Dong <[email protected]>
(cherry picked from commit bc74a40)

(cherry picked from commit 100c64a)

We currently do two resolves for dns seeds: one for the results, and one to
serve in addrman as the source for those addresses.

There's no requirement that the source hostname resolves to the stored
identifier, only that the mapping is unique. So rather than incurring the
second lookup, combine a private subnet with a hash of the hostname.

The resulting v6 ip is guaranteed not to be publicy routable, and has only a
negligible chance of colliding with a user's internal network (which would be
of no consequence anyway).

(cherry picked from commit 7f31762)

(cherry picked from commit 100c64a)

(cherry picked from commit 833bc08)

The original ORCHID prefix was deprecated as of 2014-03, the new
ORCHIDv2 prefix was allocated by RFC7343 as of 2014-07. We did not
consider the original ORCHID prefix routable, and I don't see any reason
to consider the new one to be either.

(cherry picked from commit 8be3f30)

and remove redundant public declaration

(cherry picked from commit c90b9a2)

(cherry picked from commit d6a1287)

This new approach uses a static method which takes the object as
a argument. This has the advantage that its constness can be a
template parameter, allowing a single implementation that sees the
object as const for serialization and non-const for deserialization,
without casts.

More boilerplate is included in the new macro as well.

(cherry picked from commit ca33451)

(cherry picked from commit 0aadc11)

This adds a READWRITEAS(type, obj) macro which serializes obj as if it
were casted to (const type&) when const, and to (type&) when non-const.

This makes it usable in serialization code that uses a single
implementation for both serialization and deserializing, which doesn't
know the constness of the object involved.

(cherry picked from commit 818dc74)

(cherry picked from commit 9250a08)

This is necessary later, when we drop the nVersion field from the undo
data. At that point deserializing and reserializing the data won't
roundtrip anymore, and thus that approach can't be used to verify
checksums anymore.

With this CHashVerifier approach, we can deserialize while hashing the
exact serialized form that was used. This is both more efficient and
more correct in that case.

(cherry picked from commit e484652)

(cherry picked from commit cf68a48)

Currently, the READWRITE macro cannot be passed any non-const temporaries, as
the SerReadWrite function only accepts lvalue references.

Deserializing into a temporary is very common, however. See for example
things like 's >> VARINT(n)'. The VARINT macro produces a temporary wrapper
that holds a reference to n.

Fix this by accepting non-const rvalue references instead of lvalue references.
We don't propagate the rvalue-ness down, as there are no useful optimizations
that only apply to temporaries.

Then use this new functionality to get rid of many (but not all) uses of the
'REF' macro (which casts away constness).

(cherry picked from commit 172f5fa)

Using VARINT with signed types is dangerous because negative values will appear
to serialize correctly, but then deserialize as positive values mod 128.

This commit changes the VARINT macro to trigger an error by default if called
with an signed value, and updates broken uses of VARINT to pass a special flag
that lets them keep working with no change in behavior.

(cherry picked from commit 499d95e)

… other classes

This adds the (internal) Wrapper class, and the Using function that uses it. Given
a class F that implements Ser(stream, const object&) and Unser(stream, object&)
functions, this permits writing e.g. READWRITE(Using<F>(object)).

(cherry picked from commit ca62563)

( cherry-picked _MANUALLY_ from 2f1b2f4 )

This removes the need for the GNU C++ extension of variadic macros.

(cherry picked from commit 0e0fa27)

This returns a pointer to the beginning of the vector's data.

(cherry picked from commit 47314e6)

Implement `begin_ptr` and `end_ptr` in terms of C++11 code,
and add a comment that they are deprecated.

Follow-up to developer notes update in 654a211.

(cherry picked from commit f00705a)

(cherry picked from commit 8c1dbc5)

Support is added to serialize arrays of type char or unsigned char directly,
without any wrappers. All invocations of the FLATDATA wrappers that are
obsoleted by this are removed.

This includes a patch by Russell Yanofsky to make char casting type safe.

The serialization of CSubNet is changed to serialize a bool directly rather
than though FLATDATA. This makes the serialization independent of the size
of the bool type (and will use 1 byte everywhere).

(cherry picked from commit a7c45bc)

(cherry picked from commit 9272d70)

(cherry picked from commit ca34c5c)

(cherry picked from commit 37d800b)

This allows a very compact notation for serialization of vectors whose
elements are not serialized using their default encoding.

(cherry picked from commit abf8624)

(cherry picked from commit 3cd8ab9)

(cherry picked from commit 3ca574c)

(cherry picked from commit 56dd9f0)

(cherry picked from commit e574fff)

(cherry picked from commit 769ee5f)

Extracted by Pieter Wuille from a comment by Russ Yanofsky, see
bitcoin/bitcoin#18317 (comment).

(cherry picked from commit 6f9a1e5)

Extracted and extended by Pieter Wuille from a comment by Russ
Yanofsky (see
bitcoin/bitcoin#18317 (comment)).

(cherry picked from commit d06fedd)

(only picked changes to tests and netaddress.h, other changes are
outside the scope of this cherry-pick)

(cherry picked from commit 4eb5643)

(cherry picked from commit b7c349d)

Before this change, we would analyze the contents of `CNetAddr::ip[16]`
in order to tell which type is an address. Change this by introducing a
new member `CNetAddr::m_net` that explicitly tells the type of the
address.

This is necessary because in BIP155 we will not be able to tell the
address type by just looking at its raw representation (e.g. both TORv3
and I2P are "seemingly random" 32 bytes).

As a side effect of this change we no longer need to store IPv4
addresses encoded as IPv6 addresses - we can store them in proper 4
bytes (will be done in a separate commit). Also the code gets
somewhat simplified - instead of
`memcmp(ip, pchIPv4, sizeof(pchIPv4)) == 0` we can use
`m_net == NET_IPV4`.

Co-authored-by: Carl Dong <[email protected]>
(cherry picked from commit bcfebb6)

A netmask that contains 1-bits after 0-bits (the 1-bits are not
contiguous on the left side) is invalid [1] [2].

The code before this PR used to parse and accept such
non-left-contiguous netmasks. However, a coming change that will alter
`CNetAddr::ip` to have flexible size would make juggling with such
netmasks more difficult, thus drop support for those.

[1] https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#Subnet_masks
[2] https://tools.ietf.org/html/rfc4632#section-5.1

(cherry picked from commit 1ea57ad)

Do not access `CNetAddr::ip` directly from `CService` methods.

This improvement will help later when we change the type of
`CNetAddr::ip` (in the BIP155 implementation).

Co-authored-by: Carl Dong <[email protected]>
(cherry picked from commit bc74a40)

(cherry picked from commit 2c084a6)

Reachable from either place where SetIP is used when our best-guess
addrLocal for a peer is IPv4, but the peer tells us it's reaching us at
an IPv6 address.

In that case, SetIP turns an IPv4 address into an IPv6 address without
setting the scopeId, which is subsequently read in GetSockAddr during
CNetAddr::ToStringIP and passed to getnameinfo. Fix by ensuring every
constructor initializes the scopeId field with something.

(cherry picked from commit b7b36de)

This slows the increase of the nAttempts in addrman while partitioned,
 even if the node hasn't yet noticed the partitioning.

(cherry picked from commit 6182d10)

This introduces a rudimentary begin(), end(), operator[], and subspan to Span.

(cherry picked from commit 29943a9)

This allows for very cheap transformations on the range of elements that
are to be passed to ExecuteWitnessScript.

(cherry picked from commit 2b0fcff)

(cherry picked from commit 0fbde48)

This matches a change in the C++20 std::span proposal.

(cherry picked from commit 1f790a1)

This prevents constructing a Span<A> given two pointers into an array
of B (where B is a subclass of A), at least without explicit cast to
pointers to A.

(cherry picked from commit bb3d38f)

(cherry picked from commit ab303a1)

(cherry picked from commit 26acc8d)

Based on a suggestion by Russell Yanofsky.

(cherry picked from commit e63dcc3)

(cherry picked from commit 3502a60)

(cherry picked from commit fa2ae0a)

Tests if addresses are online or offline by briefly connecting to them. These short lived connections are referred to as feeler connections. Feeler connections are designed to increase the number of fresh online addresses in tried by selecting and connecting to addresses in new. One feeler connection is attempted on average once every two minutes.

This change was suggested as Countermeasure 4 in
Eclipse Attacks on Bitcoin’s Peer-to-Peer Network, Ethan Heilman,
Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive Report
2015/263. March 2015.

(cherry picked from commit dbb1f64)

…ttempts. THIS NEEDS REVIEW

Changes addrman to use the test-before-evict discipline in which an
address is to be evicted from the tried table is first tested and if
it is still online it is not evicted.

Adds tests to provide test coverage for this change.

This change was suggested as Countermeasure 3 in
Eclipse Attacks on Bitcoin’s Peer-to-Peer Network, Ethan Heilman,
Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive Report
2015/263. March 2015.

(cherry picked from commit e68172e)

The scripts for creating a compact IP->ASN mapping are here:
https://github.com/sipa/asmap

Co-authored-by: Pieter Wuille <[email protected]>
(cherry picked from commit 8feb4e4)

(cherry picked from commit cd16f48)

(cherry picked from commit 02137f1)

(cherry picked from commit d7349ca)

(cherry picked from commit b1a5f43)