Teechain is a layer two payment network that offers high performance and strong security for blockchains and cryptocurrencies. In this blogpost we evaluate the performance of Teechain for the Bitcoin network: we look at the maximum throughput of several Teechain payment networks and measure payment latencies. For those who missed our first blogpost where we introduced Teechain and discussed how it works, you can catch up here. Otherwise, we provide a brief overview before diving into experiments and results.

How does Teechain work?

Teechain uses trusted execution environments (TEEs), such as Intel SGX, in recent CPUs to construct point-to-point payment channels, where two users can exchange funds directly without writing to the blockchain for each payment. Teechain also offers a protocol for secure multi-hop payments, allowing users to chain multiple payment channels together to execute payments between users who do not have a direct payment channel. To overcome any potential TEE failures and compromises, Teechain uses m-out-of-n secret sharing to construct committee chains, where funds in the Teechain network are protected by sets of TEEs that form committees and require a threshold number of signatures (e.g. 2-out-of-3 signatures) in order to spend. This allows Teechain to overcome TEE Byzantine failures, and achieve strong security guarantees. For more information about how Teechain works, see our first blogpost.

Measuring performance using Bitcoin transactions

To evaluate Teechain’s performance, we first require a dataset of payments to replay in order to measure throughputs and latencies in the network. As there exist no public micro-payment datasets at present, we use the transactions found in the Bitcoin blockchain. To adapt the Bitcoin transaction history for this purpose, we filter out transactions that are not appropriate for replaying, such as those that spend to/from multi-signature addresses, or payments of value over a certain threshold (i.e. $100). For transactions with multi-input and output addresses, we choose only one. This results in a dataset of over 100 million payments from a sender to a recipient and acts as a workload by which to evaluate performance.

Payment network setup

We deploy several Teechain payment networks using a cluster of 30 Intel Xeon E3-1280 SGX machines located in London. We set up payment channels between the machines, forming a complete payment graph; every machine has a direct payment channel to every other machine and we emulate network overheads by adding ~100ms latencies between each machine-- the worst case latencies measured for cross-Atlantic communication. We vary the number of machines in each deployment (N), constructing complete graphs for N=5 up to N=30 machines.

For fault tolerance and to protect from hardware attacks, we operate committee chains of length 3 at each node in the graph, requiring 2-out-of-3 committee TEE signatures in order to spend funds; TEE payment state is therefore replicated across 3 SGX machines at each node in the payment network. Figure 1 shows a complete graph construction for N = 5, where each node has a direct payment channel to every other node. Figure 1 also shows the 2-out-of-3 committee chain for node A, with all other nodes in the graph operating similarly. For more information about committee chains, see our first blogpost.

To execute payments in the Teechain deployments, we assign Bitcoin addresses to the machines. For all payments, the machine assigned ownership over the sending address is responsible for issuing payments to the recipient addresses and corresponding machines. We randomly and evenly distribute all Bitcoin addresses between the machines. To limit the number of in-flight and unconfirmed payments in the network, we implement a sliding window of payments at each machine. Each machine issues W=1000 payments across the network in parallel, and waits for acknowledgements before moving the sliding window. Due to the use of TCP to maintain connections between each node, all payments execute in-order according to their issuing node.

Achieving 1 million transactions per second (tx/s)

Figure 2 below shows the payment throughputs for each deployment when executing the Bitcoin workload. Teechain averages over 1 million tx/s with a complete graph for N=30 machines, where each node in the graph operates a 2-out-of-3 committee chain. The average payment latencies we measure during our experiments are around 1000 milliseconds, or a second, for a payment to complete. This is due to the fact that Teechain must send a single packet and receive an acknowledgement before a payment can complete, and each payment must update both the sender’s and recipient’s committee chains. Payment latencies are therefore bound by the network latencies between nodes in the graph.

Our results show that in a complete graph, Teechain scales linearly with the number of machines. Although these experiments allow us to replay the entire Bitcoin transaction history (8+ years) in just several minutes, there are several caveats:

• The Bitcoin transaction history is not necessarily a good workload for evaluating off-chain micropayment networks, as the transaction patterns found in Bitcoin may not reflect those of a micropayment network in practice. However, as there are very few datasets available, we deem this workload an interesting place to start.
• Although we only require 30 machines and 435 payment channels to achieve 1 million tx/s, assuming a complete graph where there are payment channels between each sender and recipient may not be the case in practice, and rather multi-hop payments may be required in a larger payment network. However, for deployments that wish to achieve high throughputs, we don’t deem complete graphs of N < 100 nodes unrealistic. We evaluate other payment network topologies, such as a hub-and-spoke topology, in a future blogpost.
• Assigning Bitcoin addresses randomly between the nodes in the deployment reduces one possible source of payment data skew. In practise it’s likely to have several big nodes (or hubs) responsible for a larger proportion of Bitcoin addresses than others. However, it is not unreasonable to assume that any big payment processing hub that offers this service to users may then just run several instances of Teechain on seperate machines to share load, rather than a single instance on one machine.

Why isn’t performance higher?

Those with a bit of background knowledge will know that Teechain payment channels can operate between 33,000 tx/s to 145,000 tx/s, per individual payment channel, depending on factors such as the size of the committee chains used at each endpoint. We present and discuss these numbers in more detail in a future blogpost. However, using these numbers alone, one would expect Teechain to scale past 1 million tx/s with only 10-20 payments channels (not 435 channels!). Well, there are several reasons for the performance results in our experiments:

• The Bitcoin transaction workload is skewed; some nodes issue more payments than others, meaning higher contention across individual payment channels in our deployments. We observe a skew of factor 5; some nodes process 5 times more payments than others.
• The sliding payment window limits the number of concurrent in-flight payment requests being made at any one time. It’s possible that payments for other payment channels can be issued concurrently, however, the sliding-window may prevent this while waiting for an acknowledgement to be received.
• The machines in our deployment only have 2 physical cores (4 with hyper-threading). Although there may be packets for different payment channels that can be processed in parallel, CPU contention prevents maximum parallelism across payment channels. Moreover, our experiments do not employ any optimizations such as transaction batching, where multiple transactions to the same sender are batched into a single transaction to increase throughput.

Unanswered questions?

Looking at the results above and thinking about payment network performance, there are several unanswered questions for Teechain that still remain:

• How does Teechain perform under other workloads and datasets other than Bitcoin, where payment patterns may be much more skewed?
• How does the payment network topology affect throughput? All of the experiments in this blogpost assumed a complete graph, where users have a direct payment channel to everyone they wish to pay. In other network topologies, such as hub-and-spoke topologies, this is not the case, and thus multi-hop payments are required.
• What is the performance of individual payment channels and multi-hop payments in Teechain? And how do these compare to other payment network solutions for Bitcoin, such as the Lightning Network?

We investigate these questions and provide answers to them in future blogposts -- so stay tuned! In the meantime, if you have any thoughts/questions and so on, feel free to find us at: teechain.network or email us at: teechan-team@systems.cs.cornell.edu

Written by Joshua Lind.