Identifying and Resolving the Conflict Between 0-conf and Permissionless Innovation

(originally posted on yours.org in Oct 2018)

It may not be immediately apparent, but permissionless innovation of layer 2 blockchain features and reliable acceptance of payment before confirmation (0-conf) are in conflict. The conflict arises around the concept of “standard transactions”. Standard transactions are ones that are relayed between bitcoin nodes, whereas non-standard transactions are allowed in blocks but not relayed.

To defeat 0-conf, one strategy is to pay the merchant with a standard transaction but inject an unrelayed non-standard transaction directly to miners. The merchant only sees the standard transaction, but the miner mines the non-standard transaction. But this attack is difficult in practice because mining nodes are generally not directly exposed to the P2P layer, those that are are difficult to identify, and these nodes do not expose an interface that accepts nonstandard transactions.

However, careless innovation might make this attack easy. Let us imagine that a full node client A changes its concept of “standard transaction” to enable a new feature, but due to permissionless innovation, a competing full node client B does not. This means that these transactions will be relayed to the subset of the network reachable through nodes running A, but will not be relayed to B nodes (or more accurately, relayed to the first connection from A to B, but then dropped at B), or to “islands” of A nodes fully isolated by connections to B.

So to create a probabilistic double spend, an attacker need only spend using the new feature in the upgraded subset of the network, and then spend using a standard transaction in the non-upgraded portion. The likelihood of one transaction being mined verses another depends on the relative proportion of hash power contained in each section of the network.

There are unfortunately many types of changes that will break 0-conf:

1. Any change to the minimum fee, or even variations in the fee structure
2. Changes to OP_RETURN limits
3. Multiple OP_RETURN outputs
4. Any addition to the set of “standard” transaction templates
5. Any changes to the allowed number of unconfirmed child or ancestor transactions

One proposal I have heard is to simply remove the IsStandard() rules. As a reminder, these rules prevent end-users from creating certain types of transactions but do not constrain miners from doing so. They were likely added (before the BTC/BCH split) to prevent end-user access to exploits that are better solved (and many have been solved) via hard forks. Presumably miners are heavily invested in the success of the cryptocurrency so will not deliberately create blocks containing these exploits.

Unfortunately #1 is not fixed by removing IsStandard(). And the concept of IsStandard() remains a very powerful emergency remediation — it is a lot easier, safer, and therefore quicker for clients to deploy changes that simply prevent a transaction from entering their mempools than to organise an industry-wide hard or soft fork. So our removal of it might only last until the next emergency issue.

Solutions

Super-Standard Transactions

Tom Harding’s “super-standard” transaction proposal partly addresses this problem. With it, new features that create new types of transactions would not be considered 0-conf acceptable. But the subset of normal, everyday transactions would be 0-conf. The problem with this is that new features are never allowed to be 0-conf, at least not until every client agrees to implement them. And transitioning a new feature into the “super-standard” set (really, ANY changes to the definition of a super-standard transaction) needs to be coordinated to happen simultaneously across all clients, like a hard fork but with the added difficulty that old versions of client software do not fall away from the main chain.

Double-spend Proofs

What we really need is a way to flag a transaction to be relayed despite “is-standard” rules so that double-spends can be relayed to all nodes no matter their rules. However, a simple flag cannot work because anyone could set the flag, overriding a node’s relay policy and spamming the network. This is where the concept of a double spend proof finds real justification. It can be used to prove the existence of a double spend and be relayed network-wide relatively efficiently (as compared to the possible spam transaction it replaces).

In sum, to enable permissionless innovation, it seems we need double-spend proofs or an equivalent technology.