Possible Solutions for Unsustainable Storage Growth

Development
,
2

hey, Aaron, thank you very much for bringing it up!

Given the technical nature of the solution, it is really important for devs building on Moonbeam & Moonriver to actively participate in this topic and ask questions / provide input. I believe that a collaborative approach allows us to better understand the merits of the proposal and the potential consequences.

By involving the development teams, they can contribute their expertise and insights, ensuring that the proposed changes align with the needs and expectations of the projects being built on Moonbeam & Moonriver. involvement of devs can also help identify any potential challenges or considerations that may arise during the implementation. their input can ensure that the solution is well-rounded, addresses the technical aspects effectively, and takes into account the needs and concerns of the ecosystem’s stakeholders

3 Likes
3

Although I understand approaches 1-3, I’m worried how they deviate from the typical deployment structure and interaction when compared to other EVM implementations. This will just create very complicated dynamics that are not standard and have a big impact in both user experience and developer experience. This might drive devs away from Moonbeam.

On the other hand, proposal 4 & 5 will also create some differences with typical EVM implementations, but at least they’ll be sort of hidden behind the Ethereum JSON RPC. Therefore, in theory, while these implementations are not standard, they should have minimal impact on developer and user experience. Generally it will just be higher gas fees on some scenarios.

I personally like option 5. Specially if EVM gas estimations can take into account the change in state the transaction is doing and reflect that by increasing the minimum gas requirements of that tx. It is true that this should mean tx throughput might be affected, but:

  1. With Async Backing and further enhancements down the road, the maximum gas per block could be easily increase by 4x - 10x. Therefore (hopefully) throughput should not be impacted
  2. Devs should store only the bare minimum info on the Blockchain, and rely on events that can be retrieved by indexers to store and retrieve data in a cheaper and more efficient way

The main problem I see with 5 is what was mentioned of contracts that might have some gas limit hard coded for some calls. But this is not widely used IMO.

Anyways, my two cents, but would love to see discussions on the different ptoposals!

7 Likes
4

Overall, I am leaning more towards MBIP-5 as it seems to offer the most transparency for EVM users and retains a mechanism that feels more in line with Ethereum’s approach to managing smart contracts. In addition to this, I feel like this MBIP encourage developers to optimize and take care of their contracts as the cost for a user to use their protocol may become a key difference with protocols that offer similar functionalities.

Appart from this I want to drop one question regarding the deposits on smart contract creation,

From MBIP-1

When the Smart Contract gets destroyed, the deposit is restored to the depositor.

How would the contract destruction work? The selfdestruct opcode got deprecated and will be replaced by a new OPCODE with a different functionality to fit better on Ethereum’s roadmap. Smart Contracts using an older compiler version should still be able to be destroyed but not the ones using the newest compiler versions. And a following up question, would we consider that the deposit is lost/locked in the case Ethereum finally replaces selfdestruct and we follow their path due to compatibility ?

4 Likes
5

Hi Guys,

I’ve spoken to the contracts developers within the project and from a genuine on-chain usecase, our preferred solution would be 1 and 3 as 2 and 4 i think it will affect our base transaction cost which is something our users care a lot about as we do a lot of interactions onchain and increasing tx cost would affect our user-base.

6 Likes
6

Updated the account growth chart in the main post so that it’s as of May 25th, 2023.

3 Likes
7

Great post with a lot of juicy detail @aaron.mbf. Reducing complexity for those that are experienced in using and deploying to Ethereum compatible chains seems like it should have a higher weighting, and that lends itself to 4/5. But as I am not a developer my own view should be weighted less, so I would rather support the consensus of those actually building the dApps (assuming they have user experience in mind when considering the options).

With regard to address creation, is it not such of an issue that an existential deposit has not been implemented or considered for this?

2 Likes
8

Hello everyone,

I’m part of the developer team at StellaSwap. Evaluating the effects of MBIPs 1-3, I find that they complicate matters, negatively affecting both the user and developer experience.

However, MBIP-5 presents a promising solution, where it minimizes the negative impact on both user and developer experiences. From a computational under utilization standpoint, it could be okay if it means simplifying the user experience.

When it comes to employing {gas:} in our smart contracts, we always pass gasLimit as a parameter. I propose we look into the quantity of contracts on moonscan that use hardcoded values, which I suspect will be few.

Overall, this approach promotes optimal practices in smart contract development.

6 Likes
9

Noah from DAM here!
Not to parrot exactly what Alberto has written down, but I also agree that 1-3 are complicated from a UI pov as well as straying from the well-known baselines of EVM contract usage. Barriers to entry should always be kept extremely low in my opinion, and these proposals would directly contradict that philosophy.

4 and 5 make the most sense to me, with 5 probably edging out the win in my head, again because of low barrier to entry from a user POV. I would also agree that hardcoded values on gas usage are (or at least shouldn’t be) common anyway, so that drawback doesn’t really hold much weight imo. Underutilization from a compute perspective makes the alarm bells start ringing a little bit, but I would need to know more in order to weigh it properly. Is it the case where blocks are getting to 80% usage and storage says no more, or one where they’re getting to 20% usage and storage says no more? Do we have any benchmarks on that?

6 Likes
10

My main issue with 4 and 5 is that they seem to be an ingress-based model that does not account for the time dimension of storage. If we want to store some data indefinitely like Arweave then it could work, but if we want to incentivize state size hygiene, then it’s not the right economic model.

Would it be possible to ask users to pay an extra “storage subsidy” when they use a contract? Something like…

total cost = (units of gas used + subsidy gas surcharge) * gas base fee

The subsidy would be based on

  • current storage rent payable = storage * time - previous paid subsidies
  • contract tx rate to split the subsidy among multiple users

At the end of the day, the economic model must find a way to balance contract utility with storage costs.

  • A contract with large state should be OK if users are OK with paying for its high TX fees (the assumption is that are OK because of its high utility).
  • Lots of transactions indicate higher utility and decrease gas costs because the subsidy is split among more (active) users.
  • As the contract state grows txs will become more expensive, so developers are incentivized to design contracts that keep state size under control.
  • The contract storage funding scales up with active users/transactions

Potential issues

  • Not sure if wallets would play along the contract-dependant gas fee calculation
  • A contract with many dormant users and a lot of state would become expensive for new users
2 Likes
11

total cost = (units of gas used + subsidy gas surcharge) * gas base fee

I’m intrigued by this idea, but I see some problems with it. I’ll mention just a couple here:

  • There’s sort of a “Tragedy of the Commons” problem because if I want to use some contract that is expensive, what I really want is to wait for someone else to use it first and then use it immediately afterward. If this is everyone’s strategy, then no one will use a contract and the longer this “game” goes on, the worse it gets.

  • The model seems clear when a transaction directly calls a contract, but what about various forms of subcalls? Should I have to pay this rent on contract B when contract A calls contract B? For that matter, not all accesses to a contract are equal. The approach could be modified to take the size of an operation into account, but this gets complicated quickly.

12

Concerning the state size hygiene, I feel it is acceptable to consider a constant growth, as our hardware and technology is evolving over time, however we have to make sure it doesn’t grow faster

13

@aaron.mbf GM!

I feel that this is unneeded. Should we not be talking about storage itself and figuring out ways to better handle storage? I think that these MBIP’s target active users and the cost would drive users from Moonbeam. What drives people to moonbeam is the low transaction fees, that is what will drive both EVM users and XCM users to moonbeam. We want to incentivize people using our chain, right now that incentive is lower gas fees. By driving the fees up, people will go to other chains. If I have to pay a fee that I don’t even get to see or know, every time I create a new LP contract or a fee to provide LP or vault or other contract interaction why would I want to do that? Masking low transaction fees with a hidden deposit seems like stealing to me. Some of the MBIP’s remind me of a monolithic chain, like Solana, every time a new token is deposited onto my account I pay a small fee for the creation of that token address for the first time, if I close out that token account (sell all of those tokens) I get that fee back, or if I get an NFT that was dusted to me and I burn it I get a small fee, etc, etc…

I feel the real solution (rather than taxing XEN and XENft users) is to address the storage system and improving that, or get more storage space. Look at new compression models, or even forgetting merkle trees until needed. Thinking that the setup someone bought 17 months ago should still work means someone did not take scalability and use into consideration. Since I brought up Solana look at what metaplex is doing with their compressed NFT’s, 5 million NFT mints cost 5 SOL due to their compression system of storage.

If anything, taxing ‘use of chain’ can lead to serious problems, maybe you should tax people MORE the LESS they use the chain, but taxing is just stupid and against what blockchain/DeFi stands for. We want people to use the chain, and over 6000 individual people came to moonbeam and 10,000 addresses were created, I can show you the data from my last chain analysis in April chain data from april
where you can see the addresses of +6000 people using multiple chains (6 that I tracked) some people are so new to crypto from the XEN community they thought they needed a new wallet for each chain.

So instead of trying to push users off because storage was not considered when setups were made, figure out how to make better use of the storage at hand. User growth is wanted and should be sung uphigh, not finding a way to tax and pay for validators/nodes and collators that didn’t set up enough storage.

1 Like
14

I don’t have a lot of developer experience, but I will give my 2 cents:

Keeping transaction fees low, like they are now, is a crucial component in the decision-making progress. High transaction fees are one of the main reasons why non-crypto people don’t bother to explore blockchain applications. There are other technical implications to keep in mind, but keeping transaction fees low is, in my opinion, a priority.

1 Like
15

That is a valuable feedback, thank you.

I would like to address multiple points:

…like Solana, every time a new token is deposited onto my account I pay a small fee for the creation

This is how Polkadot works. This is call the existential deposit and is an effective mechanism to prevent uncontrolled growth of addresses like we have on Moonbeam. However this can only work with a better security mechanism than Ethereum (where the nonce is the only element preventing replay attack). That is why we are not proposing it (all other parachains do).

I feel the real solution (rather than taxing XEN and XENft users) is to address the storage system and improving that.

We don’t try to tax any specific project. The reality is that Moonbeam is a Smart contract execution blockchain, not a storage blockchain. The Moonbeam protocol offered storage at almost no cost and would have keep it so if the XEN project didn’t saturate it (using 99% of the storage by itself).

The actual storage issue we have today is not only about space (it requires 2TB as opposed to 100GB some month ago), but also about the execution. Some operations are now taking 5x more times to execute because of the size of the storage. This means that users will have to pay more for their contract because it will take more time.

Some of the solution you brough up like compression of storage would make this even more expensive to execute.

So, to me, penalizing users who are using correctly the chain (by doing code execution) because others are saturating the storage is unfair.

If anything, taxing ‘use of chain’ can lead to serious problems,…

Taxing is never something that people (even the one implementing it) want, but right now the people “using the chain” are already paying more because of the impact of the big storage.

Some of the MBIPs like MBIP-4 are actually proposing a solution that:

  1. Has very low fee if your contract is not abusing storage (which is the case for all but 1 project so far)
  2. Is visible to the user (the user transaction gasPrice, which is visible and decided by the user, will be slightly higher, but the user will get refunded the extra part that was reserved for the storage but not used)

So instead of trying to push users off because storage was not considered when setups were made, figure out how to make better use of the storage at hand.

We are not trying to push users off, we are trying to prevent the rise of the network cost because of the abuse of the storage resources.
Now, of course, if a project want to store 20M times the same bytes as some did, maybe an Ethereum compatible chain is not the right place :slight_smile:

4 Likes
16

Yes, correct. That is a very important point to keep in mind. Not everything should be stored on the Moonbeam blockchain. Preventing the rise of network costs is crucial.

1 Like
17

Actually solution 4 is not going to increase the fees if the storage usage is not big. What it would change is that the user will have to provide a higher fee, but on the other hand will get refunded more, allowing the transaction to be at the same cost at the end

18

Update: A snapshot vote is up now where GLMR token holders can vote on their preferred solution (or vote for “no change”). The voting will be open for roughly 1 week.

The results of this vote will have implications for the entire community so it’s very important that community members make their voice heard by voting.

Thank you to those that took the time to read through these proposals and especially those that took time to respond with their thoughts on the matter. Some excellent points were raised in the discussion and it’s wonderful to see community engagement on such an important matter.

4 Likes
19

Hi, while traveling here is my 2cents

It is clear that all solutions have certain advantages and disadvantages.
However in my opinion there are some fundamental principles

The chain should be used for calculations and only for the very necessary storage.

It is developers’ responsibility to take that into account when designing their smart contract both in case of user interaction and smart contract factories

Having said that with no IPFS native storage pallet and only other option of using XCM (and XCM to EVM) to access storage and smart front end design and as AlbertoV19 mentioned use of indexers, the developer can easily get carried away and switch to using the chain for storage.

Yet such storage usage puts strain on the chain for all of us. A common storage fee that increases from one block to another sounds like punishing the whole ecosystem for a developer’s project poor design and implementation.

Punishing the user by increasing the user’s fee across the chain is also hurting our ecosystem

In addition to the above with the chain been 2y old and having already 12Million smart contracts where does this lead to?

Is it unrealistic to say that in next 2years with the chain and Polkadot ecosystem much more mature, with halving around the corner, with new mature communication layers, a) horizontal (for XCM (and XCM to EVM)) and b) vertical (interconnected contracts) that the chain will hit 100mln or even 1billion smart contracts

Sure increasing blockspace 4x and asynchronous backing will facilitate all the above to a degree but I strongly believe that unused smart contracts that belong to a fictional decentralised desert should be buried to leave the vibrant ecosystem use its resources in the most efficient way

Based on the above and bearing in mind that all proposals have a high value I believe that the 2 proposals that describe most of my thesis above are

To enforce good and efficient usage of storage on the chain we can use the good actor / bad actor approach

The good actor

MBIP-1

An increased deposit when creating a smart contract

It is assumed that the developer has done tests ( off chain and on Moonbase Alpha ) and brings upgradable smart contracts on Moonbeam. He then deposits a very significant fee for the smart contracts deployed on Moonbeam.

The fee is partially refundable when he upgrades his smart contract and terminates older smart contracts

This mimics what we have in Supermarkets in Europe where you put a coin in the trolley to use it and you can get it back once you return the trolley (so a “good” player)

This means in our road to hitting 100million or 1 billion smart contacts, we as developers keep the healthy-modern ones

Note: It is important that we need a mechanism to terminate the smart contract (remove from chain) in modern solidity with funds being returned to smart contract deployer account

MBIP-2

The bad actor (not bad necessarily)

Sometimes people get lazy, fail to study and search for optimum solutions or are honestly unaware of better solutions but willing to learn. In all cases these developers have to be informed about better design alternatives and prompt them to study and look for better implementations

One way to enforce this is when a user of a specific project is forced to pay more fees because of the project inefficient implementation, without affecting the fees of users of other projects and the rest of the ecosystem.

It is my belief that if MBIP-1 was implemented it would contribute to the medium-long term ecosystem equilibrium. If MBIP-2 is also imposed then it responds to the short-medium term ecosystem equilibrium. Both are needed in my opinion

3 Likes
20

The snapshot vote has been closed with MBIP-4 garnering the most support and MBIP-5 second - both being the clear front runners based on this sampling of voters.

Given the magnitude of such a change to the core protocol and the limitations of the snapshot platform for participation, it has been suggested that any course of action should be ratified through on-chain governance.

Stay tuned…

3 Likes
21

Introduction

The Moonbeam and Moonriver community had previously discussed possible solutions for unsustainable storage growth in this forum post. This discussion then led to a Snapshot vote.

While the community appeared to give preference to MBIP-4: Storage Base Fee and MBIP-5: Gas-Based Storage Limit; overall turnout for the vote was rather low - in all 729 wallets voted, with a combined 1,616,014.49 GLMR. Furthermore, while MBIP-4 ended up winning the snapshot vote, this result was mostly determined by 1 single wallet voting with 1.1M GLMR. As a result, 180 wallets voted for MBIP-4, with a median vote of 2.48GLMR; whereas 348 wallets voted for MBIP-5, with a median vote of 5.6GLMR.

Because of the low turnout overall for the snapshot vote, the criticality of this decision and 1 single wallet potentially swaying the vote, an on-chain run-off vote will be conducted between MBIP-4 and MBIP-5. The general implementation details of these two solutions are explained in the original forum post, but this post will expand on some of the trade-offs between the two solutions; as well as provide a recommendation from the Moonbeam Foundation. However, the community can of course choose to ignore this recommendation.

MBIP-4: Add a StorageBaseFee - Pros and Cons

Pros:

  • Transaction fees stay the same for transactions with low storage increase (most of transactions). A user would see a higher up front transaction fee that would be refunded once the transaction is executed
  • The price for storage does not impact the price for CPU usage. This means that if your transaction has no impact on storage, you’ll have 100% of the StorageBaseFee refunded.
  • A block can be filled with both storage and CPU. In other words, transactions that only modify storage are not penalized via gas used, only through gas pricing. As a result, to fill the block with transactions that only modify storage will result in very high transaction fees
  • The maximum growth is roughly ~6.5GB/y. The exact number will vary based on the soft-limit per block (see below).

Cons:

  • This solution is complex to implement and also complex to explain to someone not intimately familiar with the code; because the actual fee would vary on a per transaction basis.
  • Querying the baseFee through an RPC call might show a higher value than what will actually be used. However, users that don’t use the maximum storage allowed will get a refund.
  • Fees paid don’t match the expected (baseFee + MaxPriorityFee) * gasUsed formula (or gasPrice * gasUsed for legacy transactions). Nevertheless, the effectiveGasPrice of the transaction receipt will return the resultant gasPrice used for a specific transaction. Consequently, fees paid can be calculated using effectiveGasPrice * gasUsed from the transaction receipt.
  • Storage heavy contracts will require manually providing higher fees than the baseFee. As a result, this might break a typical deployment script or scripts that build transactions that are very storage heavy
  • This has a higher impact on Ethereum RPC compatibility and scripts that work on Ethereum might not work out-of-the-box on Moonbeam, as some manual gas price adjustments might be needed
  • Each block would have a soft-limit of 40K bytes storage per block. If this limit is exceeded, it would require more fees to get the transaction to pass. However, this limit can be changed.

MBIP-5: Add a Storage limit per gas - Pros and Cons

Pros:

  • This solution is simpler to explain and to implement
  • There is no change to the base fee (when compared to MBIP-4). The transaction fee calculation follows the EIP-1559 standard, as this MBIP is focused on gas limit
  • There is a lower impact on Ethereum RPC compatibility. However, there is still a risk that deployments and transactions that might have worked in the past might not work after implementing this MBIP. For example, a transaction that uses a lot of gas already, without accounting for its impact on storage, might run over the transaction gas limit when this MBIP is implemented
  • The maximum growth is roughly ~6.5GB/y. The exact number will vary based on the hard-limit per block (see below).

Cons:

  • Each Block has a hard limit of 40k bytes storage, which cannot be exceeded.This limit is still open for discussion and could later be changed with a runtime upgrade.
  • Block storage and CPU are combined, 1 limiting the other. In other words, there can be blocks that are not doing any CPU operations (which is what typical smart contract execution is for) as their available gas is used with storage based operations
  • Gas price will increase with storage being used. This is a direct consequence of using gas to quantify storage usage. If storage usage is high, the gas used per block is high. Consequently, the dynamic gas price mechanism (similar to EIP-1559) will increase the baseFee of the block, increasing the transaction fees for the entire network

Foundation Recommendation

It is the Moonbeam Foundations recommendation that the community adopt MBIP-5. Not only is it simpler to explain to a non-technical user, it is generally more inline with Ethereum standards (both EIP-1559 and RPC compatibility) and so will lead to fewer integration problems for Solidity developers.

That being said, the foundation will of course abide by the community’s decision if it decides to go with MBIP-4 instead.

Voting

Given the importance of this decision and the relatively low turnout of the snapshot vote, an on-chain vote will be held. Two competing proposals have been submitted on-chain; these are:

Both proposals are system.remarks and are merely used to capture the will of the community. The vote for both has been fast-tracked to 7 days, and the foundation will implement whichever MBIP has more votes at the end of the voting period.

3 Likes