Understanding the exploit that made EOS.IO “unusable” for two hours
On September 13, an attacker flooded the EOSIO network to drain $110,000 in EOS from a gambling dApp.
On Sept. 13, an attacker flooded the EOSIO network to drain $110,000 in EOS from a gambling dApp. During the process, many user-facing applications were unusable due to congestion. Here’s how the hacker did it, in detail.
Basics of the network congestion exploit
Four days ago, an attacker pushed the EOS network into “high congestion mode” as part of a smart contract exploit. The maneuver temporarily made some free network resources unavailable, making many applications on the network “unusable” to smaller token holders for over two hours.
Although the network was still accessible (for example, a block explorer would still work), many were “prevented from publishing updates” or “doing anything actively on the chain” unless they paid for prohibitively costly network resources.
At the peak of network congestion, it required nearly 12 EOS to make a single feeless transaction on the network, said one community member. For context, Most blockchains attach a fee directly to transactions. EOSIO allows users to stake their tokens in exchange for network resources.
The attacker was able to rent a huge amount of network resources on a recently opened resource exchange. These resources were leveraged to select which valid transactions would get included on the blockchain to manipulate gambling dApp outcomes.
During this time, the maintainers of the gambling dApp did not have enough EOS on hand to take their contract offline (or take any preventative actions at all). This allowed the attacker to drain the smart contract for 30,000 EOS, at the cost of 300 EOS in rented network resources, at their leisure.
Identifying the attacker
Beginning Aug. 17, the user “mumachayinmm” started conducting tests against a variety of gambling dApps. After just under a month of testing, mumachayinmm rented the equivalent of 1.45 million EOS in network resources.
Previously, this would have required some $5.8 million in tokens. But REX, a new service launched in May, allows users to stake their EOS for security and voting purposes while selling the network resources their stake entitles them to. After REX, 1.45 million EOS in network resources cost just $1,200.
On Sept. 13, mumachayinmm started flooding EOSIO with hundreds of thousands of transactions.
Technical details behind the gambling dApp exploit
EOSPlay is a decentralized gambling dApp that offers games such as poker and dice. What made the service exploitable was how it generated random numbers for these games.
Instead of using a secure source of randomness, EOSPlay used the EOSIO blockchain as its source of entropy. Unfortunately, information on a blockchain can be manipulated.
As an example, on Bitcoin miners who find a block get to select which transactions are included at their discretion, so long as they’re legal transactions. Theoretically, if a dApp used transactions on Bitcoin to make calculations then large miners could game it.
On EOSIO, a similar way to manipulate the blockchain is to amass enough network resources to include whichever transactions are desired over all other users.
Specifically, what the attacker did was put deferred transactions into each block, said Dexaran, a respected smart contract developer. These blocks were the ones EOSPlay used to calculate random numbers.
By monopolizing network resources, the attacker could then calculate the random number before the contract could. If the number was a losing number, then the deferred transactions started an “infinite loop,” pushing random number generation to the next block, said Dexaran.
The maneuver allowed mumachayinmm to win on EOSPlay over and over again.
EOSPlay helpless during the attack
To make matters worse, the maintainers behind the gambling dApp did not stake enough EOS to cover their contract operation costs when EOSIO’s conservative mode was triggered. This was an oversight on the part of the maintainers.
With network resources monopolized the maintainers needed to have enough liquid EOS on hand to ensure a transaction to halt the contract would go through. It appears they didn’t have the tokens on hands, allowing the attacker to bide their time as the contract was drained.
These spam attacks aren’t unique to EOS. Networks such as Bitcoin and Ethereum are also vulnerable to spam attacks should a wealthy token holder wish to pay for them (though they are prohibitively expensive in most cases).
Block.one executives respond
#EOS is operating correctly. This is no different than when attackers flood eth or bitcoin with high fee transaction spam. The network didn’t freeze for token holders, there was just no extra bandwidth available for free usehttps://t.co/nZQmCTlXFa
— Daniel Larimer (@bytemaster7) September 14, 2019
Yet, these assertions are in conflict with Larimer’s May 2018 comments while he was touting the “feeless” design of EOSIO:
“On EOSIO, no single user has the ability to saturate the entire network no matter how much money they’re willing to spend.”
Yet, that is exactly what happened during this exploit. The attacker saturated the network by spending a paltry $1,200.
Block.one CEO Brendan Blumer also took to social media to defend EOSIO. Though, he was rather vague on specific actions until pressed by a community member.
We follow discussions on #EOS network evolution closely, and are aligned with our peers in maximising network security, performance, and capabilities. We are planning next steps carefully with a global set of sensitivities in mind, and a goal of healthy participation at heart
— Brendan Blumer (@BrendanBlumer) September 15, 2019
If a user stakes EOS they will always have access to network resources, he claims. But the amount will vary substantially, and when paying customers are using it all, it’ll be necessary to pay to maintain the same level of access, stated Blumer.
If you stake EOS then you won’t need to worry about losing access to bandwidth. If you expect an infinite amount of free bandwidth without ever paying for it, then you’ll need to find someone willing to subsidise your use to avoid disruption when paying customers are using it all
— Brendan Blumer (@BrendanBlumer) September 15, 2019
The recent exploit raises serious questions about the EOSIO blockchain. Jared Moore, an active community member asked: If the network is at risk of sudden spikes in resource cost, how much liquid EOS should developers have on hand to ensure they’re protected? Without guidance, dApp developers will continue to be vulnerable to these kinds of exploits, he argued.
Another issue is access. As EOS gains more usage it’s likely the network will eventually enter a state of constant “high congestion mode,” voiced another enthusiast.
This means developers and corporations, rather than small-time users, will dominate access to resources on the network—raising questions as to who the network is built for. These same corporations could also monopolize resources on the network, said Moore, in essence becoming gatekeepers.
On the bright side, such a scenario would make EOS like owning land, said another commentator, giving the token value through the network resources it entitles the owner to.
Dexaran, a security engineer and the creator of the ERC-223 token standard, made the following suggestion to mitigate future congestion attacks on dApps:
“It would be nice to calculate how much EOS you need to put into a ‘reserve’ account to make sure you have access to your contracts even during harsh congestion,” he commented.
Another community member voiced a need for better ways to calculate staked EOS needs under different network conditions:
“The key issue here is that the community has gotten used to the amount of free transactions they receive when the network is relatively unused. We need better estimates of how much EOS you need staked during different network conditions.”
He went on to describe problems with how staking is treated on the network.
“I also have a really big issue with the fact that EOSIO does not prioritize ‘staking’ transactions. When these conditions happen, folks attempting to stake more EOS should be allowed to (once per account) as a priority transaction. When I’ve paid for huge sums of EOS, it’s ridiculous when I get locked out and can’t allocate more to my account. I can’t ‘pay for more’ even if I wanted to.”
Designing a public blockchain is a complicated business. Things will go wrong. Right now, it’s very costly to build useful apps on any blockchain. Block.one executives should take the lead to make the development experience easier and less risky, paving the way for mass adoption, rather than maintaining hardliner positions that ‘nothing’s wrong.’