Sharding vs. Off-Chain: What’s Better For Blockchain Scalability?
Blockchain scalability is a vital issue that affects further adoption of this revolutionary technology. There are different solutions to this problem, and in this article, we will consider two of them: sharding and off-chain. We will do so by referencing examples of several scalability projects: FLETA, Elrond and GEO Protocol.
The problem
The problem of blockchain scalability can be divided into several parts: data size, transaction speed, and cost.
Data size
New transactions in every block (especially a popular one, like Bitcoin or Ethereum) lead to constant block additions to the ledger. This, in turn, leads to a continued increase of legacy data that should be stored by the blockchain’s full nodes (since that is the very essence of distributed ledger technology – DLT). When the number of users and transactions increases, this accumulating data swells enormously and there is no way to somehow reverse or slow the process.
Transaction speed
Each blockchain transaction requires peer-to-peer verification by other network members (consensus mechanism) that can become significantly slower when the number of users and transaction grows. Bitcoin can handle only 6 transactions per second. Compare it with Visa’s average of 8,000 or with its peak capacity of 43,000 transactions per second.
Transaction costs
The same goes for transaction costs. We can see the costs rise in times when there are more users and transactions on popular blockchain networks. That’s because more nodes (miners or validators) are needed with the network growth to process them (in the case of the most widely used proof-of-work – PoW – consensus mechanism) which involves more operational costs. Also, when the price of a particular coin grows, it makes the transactions that used to cost cents in fees cost dollars or even dozens of dollars.
Of course, some prominent blockchains like Ethereum are much faster and cheaper than Bitcoin – but this doesn’t solve the problem entirely. The thing is, in blockchain, you can get decentralization and censorship-resistance but always at a cost of scalability and speed. Attempts to make blockchain systems more scalable and fast by design lead to more centralization and/or less security, which contradicts the very idea of the technology.
Solutions
These issues are well known and addressed in a number of different ways, from improving blockchain’s codebase, soft forks and hard forks (which overall adds merely incremental improvements) to various third-party Layer 2 projects aimed at solving the scalability problem of particular blockchains.
Let us review in detail just two approaches to the blockchain scalability issue, namely sharding and off-chain.
Sharding solution
Sharding is one way to solve the scalability problem, and it has been used for a long time in database management due to its efficiency. The idea is to divide a database into shards – essentially horizontal segments – each of which is stored on a separate server which spreads the load and improves performance.
When applied to the blockchain realm (which is a relatively new approach), sharding means that each node contains only a certain part of the blockchain’s data and not the whole ledger copy (as is the case with a ‘normal’ blockchain system). In doing so, the nodes maintain those parts of the entire blockchain in a shared manner, preserving the decentralized principle that is native to the DLT. Thus, in sharding, a single node neither contains the whole ledger copy nor loads information of its shard onto all other nodes on the blockchain.
Sharding blockchains normally use proof-of-stake consensus algorithms (or a modification thereof), since the more common proof-of-work isn’t possible to implement along with sharding. PoS has a number of advantages over PoW, such as significantly lower energy consumption and no need for special hardware. Naturally, it has some flaws as well.
But let’s get more illustrative and consider a few projects currently using sharding.
Elrond
Built by a team of entrepreneurs, engineers and researchers with versatile experience at Intel, Microsoft, Google, NTT and NEM, Elrond is a public blockchain project focused on providing scalability – comparable in throughput to centralized systems while retaining a decentralized nature – minimal energy and computational requirements, and robust security. The project also claims to offer cross-chain interoperability that can be implemented by using an adapter mechanism at the Ethereum Virtual Machine level. (Despite it using Ethereum VM, Elrond isn’t positioned as a Layer 2 project for Ethereum – it just uses the existing Ethereum frameworks.)
All this is achieved by utilizing two key features: Adaptive State Sharding and a Secure Proof of Stake (SPoS) consensus mechanism.
Adaptive State Sharding is an approach to scalability that combines three sharding types: network/communication, transaction/processing, state/storage. According to the project team, it will improve communication between the shards, reduce storage and increase performance thanks to parallel processing. This approach allows the number of shards to change dynamically following available validation resources and network usage. Thus, the network can adapt to the number of users and demand changes without compromising security, availability and decentralization. This leads to linear scalability: scalability of the network is designed to grow with the number of nodes in the network. Also, Elrond’s network throughput is reported to exceed 10,000 TPS. During the Latest meetup, Elrond achieved 65k TPS with
The emergence of a new shard is possible only when the existing shard reaches 400 nodes. That is, if there are less than 800 nodes, there will be no two shards. And if there are more than 1,200 nodes, three shards will be created automatically.
As for Secure Proof of Stake, in essence, Elrond uses a mixture of PoS and pBFT consensus mechanisms. In general, it is a modified Byzantine Fault Tolerance mechanism, where consensus is achieved if 1/3 of the nodes, which have been randomly selected a two-level lottery, are honest, while the staked tokens still play a role in the voting and even in the probability of winning the lottery. In addition to penalties and the weight of the vote, a rating is applied to the staked deposit. If a node plays by the rules, its rating grows. But if it attempts to push something that opposes the general consensus, the rating drops.
Fig.1. Elrond’s Secure Proof of Stake consensus mechanism.
At this point, the sharding part comes into play. In other PoS blockchains it can take around 12 seconds to select a consensus group, since the entire heavy blockchain and numerous participants are involved, and information is distributed throughout the whole network. Elrond, in contrast, has a separate consensus group in each of its shards, and information exchange for reaching consensus occurs within each of the groups separately. As such, in general, it takes four seconds to set up a group and achieve consensus. Moreover, for better safety and fault tolerance, validators migrate from shard to shard by moving into a kind of buffer zone first – the validators pool – from which they are assigned to shards in random order. Plus the nodes themselves can also be shuffled within the shards. Due to these shuffles, the letter S has been added to the name of Elrond’s consensus mechanism: SPoS (Secure PoS).
After reaching consensus, when 2/3+1 signatures from members of the consensus group are collected, the block is approved and signed with the group’s aggregated signature. Then the block is distributed throughout the shard, and its header is sent to the metachain for commit. The rest of the shards receive information about each other’s state from the metachain.
Elrond is currently in the testnet phase (released in May 2019) and aims to become a platform for easy deployment of programmable money and dApps. Recently (in June 2019) the project went open source.
Token Sale: Elrond IEO has been launched at Binance launchpad and scheduled on 1st of July.
Image: Elrond achieved 36kTPS during a meetup
FLETA
FLETA aims to provide a sustainable, developer-friendly ecosystem for dApps. The project attempts to solve existing Layer 1 problems like scalability and transaction speed, as well as bring even more decentralization to blockchain. FLETA employs its own technologies for that: multi-chain structure, block redesign, parallel sharding, and a novel proof-of-formulation (PoF) consensus algorithm, in contrast to blockchain’s more common PoW and PoS.
Due to its parallel sharding technology with enhanced TPS of each individual shard, as well as overall TPS increase when the number of shards on the network rises, FLETA claims that it can handle up to 20,000 transactions per second. That is the highest TPS to date among existing blockchain projects, almost twice as much as the peak TPS of the Elrond testnet (referenced above), and can already be compared to centralized systems like the aforementioned Visa. With 500 shards, the FLETA network is capable of handling 10 million transactions per second!
Shards on the FLETA network can operate in parallel, without double spending, since each individual shard is separated and the data is not shared among them. So, apparently, it levels up decentralization compared to other blockchain projects, including Elrond, where shards are overly connected through the metachain (see above).
Also, FLETA uses an independent multi-chain structure, where sub chains are separated from the main chain. This drastically reduces the main chain overload along with excessive fees for dApp users, as well as excessive development costs. Apart from the main chain, FLETA’s multi-chain structure consists of a token chain and dApp chain.
The main chain is where the issue transaction happens. It creates the issue contract and receives tokens. The tokens, in turn, are traded on the token chain, which also lists the block number and hash value of the main chain to create a new block. Plus, the token chain can create a sub-chain to run dApps.
The most interesting point about FLETA is its novel consensus mechanism – the PoF, which is used to prevent unnecessary forks.
Fig. 2. FLETA’s Proof-of-Formulation consensus mechanism.
There are formulators who are directly responsible for block generation; observers who verify the blocks; and finally, consenters who agree on the block generation sequence.
The consensus is made up of a block mining procedure, where blocks are generated in a particular order (agreed by the consenters). There is a group of 20 formulators with ranks from 1 to 20 that is defined by their score, which determines formulator mining and reward sequence.
According to the developers, this procedure reduces the block’s dissemination range, which results in faster block generation. The observer node provides instant confirmations and prevents potential forks. Thus, the network is supposed to become stable and transaction speed is increased. However, only time and practice will tell whether this new consensus mechanism is more effective than more traditional ones.
Also, FLETA doesn’t address the interoperability issue as Elrond does.
Off-chain solution
While the sharding approach attempts to improve blockchain scalability from within, there are alternatives taking a more disruptive approach to the problem. Off-chain is one of the most ‘radical’ ones from a Layer 1 blockchain perspective. But it brings a number of major improvements and new possibilities that could drive adoption to new heights.
Sharding still uses a single distributed database, which is common for the whole network and imposes particular limitations by design. Also, the division of the database into separate shards brings up the new problem of effective and secure synchronization of its parts between each other, adding yet another point of potential failure to the whole system.
On the other hand, the off-chain approach solves both scalability and interoperability problems of blockchain (and non-blockchain) systems, improves speed and throughput limitations, increases privacy and makes the system even less centralized to extend when it becomes truly distributed.
So, let’s consider an example to gain a deeper insight into how all this could be implemented.
GEO Protocol
GEO Protocol is an open source infrastructure protocol that provides interoperability of value transfer networks in a lightweight, cost-efficient way, as well as enabling virtually unlimited scalability. The main goal of the project is to create a transport protocol for the future Internet of Value.
To achieve this, the basic problem of blockchain scalability should be ultimately solved first, and then the interoperability issue of the otherwise siloed blockchain ecosystems, as well as lack of connection of blockchain world to the world of traditional finance, should be dealt with. GEO Protocol addresses both problems, but here we will concentrate on the scalability part more.
The main bottleneck with blockchain systems in terms of scalability is the common ledger itself, as well as the need for the general consensus of this or that kind.
Being blockchain agnostic, GEO Protocol deprives itself of a common ledger altogether, relying on a fully distributed network of nodes and a disruptive local consensus approach, where only the nodes that directly participate in a transaction actually confirm that transaction. Special entities – observers – guarantee cross-chain atomicity of payments and are addressed only in a rare case of possible conflicts that could be the result of network connectivity issues for particular nodes etc.
Fig. 3. GEO Protocol network structure.
In addition, all the transaction-related information is stored locally by participating nodes only. Neither the network as a whole, nor observers, nor any other participants are aware of the content of a particular transaction or even know about its existence.
This enables ultimate privacy, as well as possibilities to scale the network almost endlessly since there is no central authority that controls the network, nor even a distributed one (like miners or validators in blockchain-based systems). Thus, the GEO network is completely decentralized, and yet it can provide full-scale functionality and operation as it’s supposed to.
In terms of throughput, since there is no need to unnecessarily share excessive information across the networks, every payment path will function separately and independently. Thus, the increase in the number of transaction doesn’t affect the overall TPS of the network.
An increase in the number of GEO network users and participants will only create more payment path possibilities and more channel liquidity, strengthening the network effect and the overall efficiency of the network.
Thus, any blockchain can use the GEO network for its own scalability as well as for interoperability with other blockchain and non-blockchain ecosystems. And from the point of view of the network, blockchain can serve as the asset-backing layer for GEO asset equivalents. This leads to a healthy symbiotic relationship between systems of different types on the global Internet of Value.
Conclusion
Scalability is one of the primary problems standing in the way of mass adoption of blockchain technology. This challenge could have various solutions, including sharding and off-chain. Each of these has its advantages and drawbacks.
Sharding, while greatly improving blockchain scalability, still greatly relies on the common ledger, an issue you can’t really overcome from within. Off-chain eliminates this entirely, becoming blockchain agnostic, lifting most of the throughput limitations and enabling interoperability.
On the other hand, off-chain solutions still require blockchain for their trustless nature and other perks. The good thing here is that we don’t have to choose between ways of solving the scalability issue, since they can efficiently work together, bringing new possibilities to all crypto networks.
