In a blockchain, users submit transactions which are validated and added to the blocks by the block proposers(aka miners) and all the miners come to a common decision about which block to add by using consensus algorithm.
This process involves different costs for different participants:
- A private benefit to its sender.
- A private cost to the proposer that includes the transaction in a block.
- A social cost to other nodes in the network, including a) All block proposers and other users and b) Present and future users.
Let’s explore more on these costs.
Costs on User (aka Transaction Sender)
Under the current fee paradigm, a user wishing to submit a transaction must figure out an appropriate fee. This turns out to be a very difficult task.
This seems straightforward, but there are problems.
Fees and Cognitive Load
The first problem is the cognitive load on the user: it is difficult to decide exactly how much to bid, whether to go over or under. Evaluating the right price to get in depends on a lot of other factors. Exactly how important is this transaction to me right now? How full is the mempool? What are the competing bids? Given that there will be, in expectation, another 10 minutes of transactions streaming into the miners before the next block is discovered, how low a fee can she get away with while still getting her transaction included in the blockchain? These are clearly difficult questions. Doing the right thing involves watching the chain closely and monitoring the transaction until it is included, an act that detracts from the hassle-free use of one’s money. One may be tempted to just go through a middleman, such as an exchange, to handle it all, which creates centralization and simply recreates the current banking system except with unregulated exchanges as centralized custodians. This has, historically, led to a stream of exit scams and other SFYL-events.
Broken Attempts to Solve this Issue
But the real problem is much more fundamental, and it stems from the fact that the fee mechanism in Bitcoin and other currencies is implemented as a pay-what-you-bid, or multi-unit first-price, auction. This fee behavior will lead to “sticky” and unnecessarily high fees, followed by sudden fee collapses, just as we have seen over the course of the last few years.
To see why, imagine a universe where everyone is using a simple historical fee estimator. Specifically, imagine a fee estimator that looks back on historical transactions and suggests a fee based on what happened on average in the past. If transactions were paying an average of 100 satoshis per byte in the recent past, then a user will simply attach a fee of 100 or more satoshis per byte.
This approach is completely broken. During times of congestion(as the red line goes up), the fees to get included will naturally go up. This is because all the users will try to pay the same price they were paying before the congestion, but some transactions will not get through as a lot of people are paying the same price(miners will have to drop some transactions, as there are a limited number of transactions that can go into a block). So the price starts rising up. But the price will remain high even after the congestion has ended(the blue line is still up even the red line drops). This is because the users will still pay the average price that the network was paying at the time of congestion. Even if some users try to underbid(in hope that maybe their transaction will get through), their transactions will not get through as miners are still getting transactions with high fees. This leads to users dropping their transactions and eventually when there are no more high fee transaction bids (which was at the time of congestion) the prices will drop drastically.
This approach will force the users to pay fees as if they were operating at the height of congestion even if the congestion has ended. The fee (the elevated fee) that arises during congestion remains the same in the system even though the conditions have changed, an artifact of bad mechanism design.
Blockchains are not the only place where similar auction mechanisms have led to poor user experiences. When ad placement in search engines was based on first price auctions, researchers observed similar patterns. Advertisers would compete with each other in order to get better placement, driving the fees sky high. This would be followed by many participants quitting the game, which would cause a precipitous drop, only to restart the unstable cycle all over again.
Solution: Multi-Unit Second-Price Auctions
The heading seems a bit complex, but the proposed mechanism is fairly simple: transactions specify a fee, just like before, and miners place transactions in a block, just like before. Except, instead of charging each transaction the fee it bid, we charge each transaction the lowest fee charged to any transaction in that block. Any surplus fees a transaction offered are returned to that user, to a designated address they specify. Hence, a transaction in effect says “I am willing to pay up to $30 for this transaction,” but is charged only $5 if the lowest fee transaction in that block paid $5. The remaining $25 are returned to the user.
This proposed mechanism brings insights from multi-unit second price auctions(used in Advertisement auctions) to the world of cryptocurrencies where currently multi-unit first price auctions are the norm. Whereas before, fee selection was a stressful and difficult task, with this mechanism, users can simply attach to their transaction the true maximum value they would be happy to pay. This is because they are not going to be charged that value: instead, they are charged whatever the minimum was to get into that block. In essence, the lowest paying transaction establishes just how little it took to get into that block, and everyone within that block is charged the same amount per byte. This is not only equitable and fair, but it takes away the pressure to play games with fee selection. As an additional bonus, the benefit from playing such games decreases as the blockchain gets more popular, further disincentivizing strategic fee selection. Note also that it picks the highest-paying transactions, just like first-price auctions, though it charges them strictly less than what they would be willing to bear…
Cost on Block Proposer/Validator(Miner)
Block proposers/validators also suffer from the current Multi-Unit First-Price Model. As we saw in Fig 1(above) the price suddenly drops. This kind of cycle leads to an unstable revenue, which makes transaction fees an unreliable source of income for proposers. This forces them to heavily rely on another source of revenue, the block reward. But as block reward will end up eventually, this(current) model is not sustainable for the long run.
Solution: Multi-Unit Second-Price Auctions
Using the Multi-Unit Second-Price Model results in better stability in revenue from the transaction fees. But there are some ways in which miners can try to manipulate the system to maximize their revenue. Different papers(discussed below) suggest different ways to mitigate these loopholes. Before discussing the mitigations these papers propose, let’s see the issues that may arise when we use this new model.
Malicious Miner Behaviour
The above graph shows the policy by which a miner fills transactions in a block. The grey portion in the Miner Revenue and the black region are the revenue that a miner pays to himself (which means the fees he paid for adding his own transactions in the block). The goal of the miner is to maximize the grey area.
The miner receives a number of transactions offering different transaction fees(represented by the Bids line). Now miner chooses the minimum Gas price for which he will add a transaction to the block. Higher the average gas price is the more the revenue will be.
In order to manipulate the market(increasing the average gas price to get included in the block), miners can jack up the minimum price to get in and add their self-generated transactions(with high gas prize) into the block to fill it. So only the transactions offering high gas price will get in. Eventually, in desperation, users will start offering a higher gas price for their transactions, thus increasing the average gas price to get in.
Also, miners can make private deals with particular users to send them transactions with a high gas price, and refund them under the table. This tactic will eventually increase the average gas price.
Mitigation of Malicious miner Problem
In my opinion, the best approach to mitigate these issues was provided by this paper.
This paper couples Multi-Unit Second-Price model idea with three other mechanisms to provide a comprehensive solution that prevents malicious behavior by miners.
- First, if a miner fails to fill a block, they cannot charge any fees. So a miner cannot take a high-paying transaction, ignore the rest of the mempool, and collect the entirety of the fee paid by that transaction while refusing to fill a block. They are, of course, free to fill the rest of the block with their own synthetic transactions, but they will have to pay out of their own pockets for those (and the next point addresses why the miner cannot just pay those additional fees directly to himself).
- Second, a miner is rewarded the average fee collected from the last B blocks, not just the fees from the block they themselves mined. This ensures that miners also have little incentive to act strategically. Instead, their best interests are aligned with maximizing the number of high-value transactions cleared per second. The only issue occurs when a majority of miners start manipulating the system(filling blocks with self-generated high fees transactions). Thus increasing the average fee paid. But this kind of attack can increase the average price only by a small factor as miners will benefit only if users are desperate enough to offer the price that the miners are asking for. So, increasing the gas price too much will not be a sustainable attack.
- Finally, the paper proposes that every block reserves some space, around 20%, which are exempt from this mechanism. This enables a miner to include transactions of high importance to themselves, such as those used for pool rewards, without affecting the fee mechanism and without being penalized. This addresses the miners’ needs and provides a simple migration path from the current state of affairs.
The above approach is also prone to economic attacks in which miners refund some users under the table to submit high gas price transactions.
Other papers such as this, by Vitalik Buterin, suggests a slightly different approach.
The idea in this paper is that you basically establish a minimum fee(to get in the block) that gets burned. The burned fee either gets deleted from the supply or it gets paid out as a block reward to the entire set of miners of validators over a very long period of time. The fee is adjusted within the protocol so that on an average the blocks are, say 50% full. This paper focuses more on the social cost which is incurred by the blockchain network(which we will discuss in the next section). This approach also successfully restricts the miners from manipulating the market to some extent.
Another paper, by Ron Lavi, Or Sattath, and Aviv Zohar gives some similar solutions(uses the Multi-Unit Second-Price model) but also differs in many ways. The major difference in this paper is that it considers the collective of miners as a single monopolist that can coherently charge non-competitive prices from the users. Here is the outline of their monopolistic model:
- Each transaction specifies a bid which is the maximal fee it is willing to pay.
- Miners can choose which subset of transactions they include in their block.
- All transactions in the block pay the exact same fee, which is given by the lowest bid contained in the block.
- Miners are expected to choose transactions in a way that will maximize the revenue obtained, i.e., maximize the number of included transactions times the minimum bid.
It also states that as the mechanism above provides miners the maximal revenue (by allowing them to select the set of included transactions and effectively set the monopolistic price), myopic miners gain nothing from a range of manipulations of the protocol, mainly, adding fake transactions, or excluding legitimate ones.
They also focus on a set of users, called “impatient users”, defined as those who desire that their transaction will be included in the next block, and have no utility from inclusion in later blocks. This effectively makes it a single shot game, rather than a repeated game which is often much harder to analyze.
This approach of mainly considering “impatient users”(and not all the users that also include users who are willing to wait and thus will not bid as high as the impatient users, hence decreasing miner revenue) seems to neglect some of the dynamics of the fee economy. But as stated by Aviv Zohar in a recent conversation, this critique applies to almost all the works in auction theory, including other fee market papers.
The social cost is an externality(external cost) that are paid by third parties that have no choice in the decision of whether or not this transaction gets included in the blockchain.
The social costs can be broken down in the following categories in terms of resource expenditure:
- Bandwidth cost: the cost of all nodes downloading each submitted transaction, bundling it into a block and then rebroadcasting the
transaction as part of some block.
- Computational cost: the cost of every node verifying each transaction.
- History storage cost: the cost of storing the transaction for all nodes
that store the blockchain’s history, for the time for which the history is
stored (possibly infinity).
- State storage cost: the marginal cost of the impact of the transaction
on the size of the state (eg. contract code, account balances) that every node must store to be able to process further transactions.
Note that the first two costs are costs that are paid by the nodes that are
online at the exact time when the transaction is included, the third cost is
paid by nodes that are online at that time or in the near future, but the fourth cost must be paid by all nodes forever unless a scheme is introduced
that makes all state entries temporary.
These external costs are unaccounted for if proposers(miners) have full free choice of what to include. So, if the proposers have full freedom to decide then proposers are going to accept all of the transactions that are worth at least their private cost. But the social cost is much much higher than the private cost. So if proposers accept all of the transactions that they want, the blockchain will just have way-way more transactions in it than is optimal, and eventually, it would bloat the blockchain. We will have nodes which take an extremely large amount of computational resources to run(which may also not even worth the revenue)…And we don’t want that.
Pricing Social Externalities
One question that we should ask ourselves is that “What is the social cost of including a transaction in the blockchain?” because the answer to this question is going to decide the answer to other questions like:
- How big should be the block size(so as to balance the private and social costs)?
- Should there be a limit or should we instead get rid of the limit and instead have the protocol charge a fixed fee?
In order to answer this kind of question we need to look at not just the social cost at any one point but the social cost across many different points.
The chart on the left is a study made by Cornell in 2014. They basically tried to estimate if the size of blocks in the Bitcoin blockchain were to increase how many nodes would continue to be able to process the blocks and stay synchronized to the network.
At block size:
- 4-megabytes, the number of nodes would decrease to 90% of the original value.
- 15 megabytes it would decrease to 75% of the original value.
- 38 megabytes it would decrease to 50% of the original value.
- 211 megabytes it would decrease to 10% of the original value.
So, this is one kind of social cost of shoving your transaction into a blockchain. The more transactions you shove the harder it will be for people to run a node and the fewer people you’ll have running a node.
This was one type of cost in which every computer has to do more work as the number of transactions increase.
The other cost is that as the number of nodes in the network decrease, the level of decentralization also decreases, thus becoming more centralized. This makes the network more vulnerable to political and economic risks.
The chart on the right depicts a crude measurement of the second type of cost. Assuming that the Decentralization utility(D) is a logarithmic function of the number of nodes, will mean that cutting down the number of nodes by a factor of two always has the same cost. So, it’s just harmful to go from 10,000 to 5000 as it is to go from 1000 to 500.
So, combining this with the first observation(the % drop in nodes as we increase the block size), the drop in utility as we increase the block limit starts off high(as a large number of nodes leave when we increase block limit from 4mb to 38mb) and it eventually becomes horizontal(as a small number of nodes leave as the block limit increases from say 150mb to 211mb) at large block limits.
The intuition for this curve comes from the fact that if you have a block limit of 100kb you can use every smartphone, IOT device as a node. If you go form 100kb to a block limit of 400kb, chances are a lot of devices will not be able to support as a node. Now, imagine going from 1.1 MB to 1.4 MB, sure you lose some devices but not as much as going from 100kb to 400kb.
This is one way to see the correlation between the block size and the social cost incurred by the network. Below is another way to see this.
The above graph shows the correlation between the gas limit and the uncle rates (uncle blocks are blocks which are valid blocks but are not a part of the main chain). We can see that the uncle rate increases as the block limit increases.
The uncle rate depicts the inefficiency of a network to process the blocks, making the network more vulnerable problems like to double-spending attacks, selfish mining. Also, as the block limit goes the nodes start dropping off(as they are not getting enough ROI), and as the nodes drop off the hardware of the nodes can be bought by the attackers which further worsens the security of the network.
So, combining the above 2 figures, we could argue that the social cost curves against the block size limit will look something like this:
So, in the left side, you have a high marginal cost initially, as we move from very small blocks(100kb) to small blocks(400kb) a large number of nodes drop off(No more full nodes on every smartphone). In the middle, it becomes a bit linear as NOT a lot of nodes are dropping and uncle rates are not that high. But on the right side, the uncle rate starts to rise, imposing several social threats to the network, thus increasing the marginal social cost.
This was one way of seeing a social cost. There is another social cost, which is storage fees. The thing that is different about the storage fees with respect to computational cost is that computational cost is incurred by the current users as only current users have to process the transactions, but storage is a thing that every node, that is present today, or will join in tomorrow, all will have to do.
This kind of cost is bigger for platforms which allow programmability such as Ethereum. Let’s see what are the specific problems does the storage part faces.
- First, there is a one-time storage cost norm. This is not a good policy as storage is a cost which is incurred not just by a present user but also by the future users.
- Second, storage is really underpriced today as compared to computational costs.
- There is not enough incentive to clear storage.
- There is not enough incentive to clear storage earlier rather than later.
- An incentive to create second-layer markets(like GasToken), which if implemented will create it’s own inefficiencies and issues, thus creating more problems.
Solution: On-Going Storage Fees
The idea here is that instead of making storage fees a one-time cost, it will be charged on an ongoing basis. So if you have to pay some cost per byte per block for every byte you are storing on the blockchain. The advantage of doing this is that it aligns the incentives in a better way, because if you are adding something which everybody has to store, not just today, but also for the future, then this type of fee model makes more sense. If you don’t pay for it then it will be eventually deleted.
Now, this model has been opposed a lot because of a few reasons:
- It affects the guarantee of the immutability of the code on the blockchain. (As the code can be deleted anytime if the fee has not been paid)
- Also, there are some cases like if you build a contract, which depends on another contract(not your contract), which goes bankrupt. A lot of contract like this exist today which allow other contracts to add storage to it, thus increasing its storage size. This will cause the rent of the shared contract going so high that the owner can’t manage to keep up with the payments.
- Also, if there a contract that is a public good(like ENS contract) which provides a value to a very large set of users, who will pay for it?
One interesting mitigation to these(problem 2,3) is Hibernation.
The idea behind hibernation is that if the contract fails to provide storage fees, it doesn’t get deleted, it gets removed from the active state and is put in the history. While it is in the “hibernated” state, no one can interact with it, but if someone whats to interact with it you can basically supply proofs(proofs about when the contract was hibernated) and you can revive the contract. You can provide a pointer which will tell where was the contract previously hibernated, and what was its state when it was hibernated and provide a proof that the contract was not woken before and now I am going to wake it up. This basically resets the contract to that particular state.
There is a vulnerability that is possible. This vulnerability allows a contract to be woken up twice. But there are ways to mitigate this kind of problem.