Decentralized (Peer-owned Orderer) Setup with No System Channel

8 min readApr 2, 2021

Introduction

In Hyperledger Fabric, ordering service plays an important role of keeping the consortium configuration and creating blocks. Over the past two years, the implementation of ordering service keeps evolving. Raft-based ordering service enables a type of decentralization, as participating organizations (peer organizations) can now have their own ordering service nodes. This means fabric can now support a setup with no orderer organization. Meanwhile, Fabric keeps using the system channel to manage the consortium, which creates an additional layer of administration. Fabric v2.3 supports a fabric network setup without a system channel. This provides flexibility in design and operation of a fabric deployment.

This article is showing a sample setup combining both. This setup does not have an orderer organization, and orderers are owned by participating organizations. No system channel is needed as we can bring up the application channel directly with channel genesis block.

Overview

Fabric documentation provides the best explanation on both raft-based implementation and setup of no system channel. Here is a quick overview.

Cluster implementation of ordering service (link) is desired in a production fabric network. Prior to raft, the Kafka-based cluster relies on a kafka cluster to support the orderer cluster. You can easily see how bulky the implementation is by comparing raft-based one, introduced in v1.4.3. Besides the complexity, the raft-based solution also supports “decentralization” of ordering service, which does not require a separate orderer organization. In this article we are showing how the orderer is owned by participating peer organizations. Note that a similar article was written more than a year ago in which v1.4 was used. This article can be seen as an updated version for v2.0+.

System channel (link) has been in Fabric since its launch. It is used by orderers to keep the consortium configuration. In a typical setup, we first bring up the system channel (with orderer genesis block file) in the orderers. After that, we can generate the channel block file through transaction updates per each application channel. Fabric v2.3 supports a setup with no system channel required. We can generate the genesis block file for the channel directly, and let both orderers and peers join with it. An article was written not long ago about this setup, but that article still used an orderer organization. In this article we further implement this no-system-channel setup on a decentralized orderer cluster.

Fabric Network Setup

I modified the Fabric CA tutorial setup (link), from which I also reworked this setup for Fabric v2.2 in this article. Here is our setup in this article.

In summary, this network is composed of two organizations, org1 and org2. In each organization, an orderer and a peer is provisioned. An administrator (admin) user is also created. An application channel mychannel is created and joined by orderers and peers of each organization. After mychannel is up and running, we will deploy the Simple Asset Chaincode (sacc) and see if everything works fine.

The identity crypto material for each organization is generated and issued by the root CA for that organization. Besides, TLS crypto material for both organizations is issued by CA TLS. More detail can be seen in step 2 of demonstration below.

Demonstration

Step 1: Prepare a Fabric host and obtain the setup

Prepare a single host with Fabric v2.3 properly installed. You can refer to the prerequisite and the installation guide for the detail.

Clone the repository inside fabric-samples/ directory. You can place it anywhere, but you may have to modify some path setting in some files.

cd fabric-samples/
git clone https://github.com/kctam/peerowned-nosyschannel.gitcd peerowned-nosyschannel

In my setup I am using virtual box and vagrant, running in my local host.

Step 2: Create crypto material for the whole network

As mentioned above, we are adopting a setup using Fabric-CA to create all the crypt material for both organizations. All materials generated are stored inside /tmp/hyperledger-fabric/, and this directory is mapped in the components later.

First, clean up everything and prepare the directory.

docker-compose downmkdir -p /tmp/hyperledger/
rm -rf /tmp/hyperledger/*

Bring up the three Fabric-CA.

docker-compose up -d rca-org1 rca-org2 ca-tls

And certain directories are now seen in /tmp/hyperledger/, which is mainly mapped from these Fabric CA containers.

You probably notice that these directories are created by root. To avoid permission problems (I saw this problem in this setup, while no problem in my Mac.) We now change it back to our user vagrant.

sudo chown -R vagrant:vagrant /tmp/hyperledger/*

Two scripts are prepared for both registration and enrollment of each component and user. You can modify them if you need additional orderers, peers or users. Make sure you have the fabric-samples/bin/ in your path.

./allCAnReg.sh./enrollAllOrgs.sh

After the scripts are completely executed, the results are kept in /tmp/hyperledger/. You can browse the directory to locate the crypto material for org1 and org2.

This is how the various CAs and the crypto material for each entity. Note a TLS client crypto material is generated for the administrator in each organization. In a no-system-channel setup, administrator accesses orderer to perform the configuration, which requires client authentication.

Step 3: Bring up all components

Now we are ready to bring up the network. All the peers and orderers are defined inside docker-compose.yaml. The three CAs in step 2 are not used from now on.

Take a look in the orderer1-org1 and orderer1-org2. They are properly configured such that no system channel is needed. As mentioned in the previous article, here is a quick summary

take out (comment) the use of genesis method and file
set bootstrap method to none
set listen address and enable TLS client authentication for administrator
set true to channel participation enabled
map the admin directory
map the port accessing the administrator port

To bring up all other components,

docker-compose up -d

To check if everything works according to our wish, first we see orderer and peer for each organization is up and running (note the four new instantiated containers).

Then we check the chain running in each orderer, and we see no chain is running at this moment. If we were using system channel, this should be shown in this directory.

no system channel is running in orderer in our setup

With this, all components are ready.

Step 4: Create application channel and join all components to the channel

To create application channel mychannel, we are now generate the genesis block for mychannel. Note that this is not the genesis block for the system channel as we are not using the system channel any more.

Now we need to make our terminal with proper environment variables. In the repository two files are prepared: term-org1 and term-org2. We will set the current terminal for org1, and open another terminal for term-org2.

For our current terminal (org1)

source term-org1

And in a new terminal (org2)

source term-org2

Note that each org points to its own orderer.

Terminal for org1 orderer1-org1 is used

Terminal for org2 orderer1-org2 is used

We will skip the details of configuration file configtx.yaml. You can refer to this article how to create decentralized peer-owned orderer and this article for the profile for no-system-channel.

On terminal for org1

configtxgen -profile SampleAppChannelEtcdRaft -configPath ${PWD} -outputBlock mychannel.block -channelID mychannel

Now we have the mychannel.block, the genesis block file for mychannel.

We first join orderer from both organizations with this block file, using osnadmin

On terminal for org1 (joining orderer1-org1)

# to show channel status
osnadmin channel list -o localhost:7080 --ca-file $ORDERER_CA --client-cert /tmp/hyperledger/org1/admin/tls-msp/signcerts/cert.pem --client-key /tmp/hyperledger/org1/admin/tls-msp/keystore/key.pem# join channel
osnadmin channel join --channelID mychannel --config-block mychannel.block -o localhost:7080 --ca-file $ORDERER_CA --client-cert /tmp/hyperledger/org1/admin/tls-msp/signcerts/cert.pem --client-key /tmp/hyperledger/org1/admin/tls-msp/keystore/key.pem

Check before joining

No system channel and application channels are in orderer1-org1

Join orderer1-org1 to application channel.

Now check again.

We repeat the same and join orderer1-org2 in terminal 2.

# to show channel status
osnadmin channel list -o localhost:8080 --ca-file $ORDERER_CA --client-cert /tmp/hyperledger/org2/admin/tls-msp/signcerts/cert.pem --client-key /tmp/hyperledger/org2/admin/tls-msp/keystore/key.pem# join channel
osnadmin channel join --channelID mychannel --config-block mychannel.block -o localhost:8080 --ca-file $ORDERER_CA --client-cert /tmp/hyperledger/org2/admin/tls-msp/signcerts/cert.pem --client-key /tmp/hyperledger/org2/admin/tls-msp/keystore/key.pem

And here is the result.

After joining the orderers, we are ready to join peers to mychannel. On both terminals, use command

peer channel join -b mychannel.block

And by listing the channel, we see peers of both organizations having joined mychannel.

Our network is ready. To test it is working we deploy sacc and make observations by invoking and querying functions.

Here is how various components are joined to mychannel.

Step 5: Test the network with chaincode sacc

This is the standard lifecycle chaincode process. I omit all the commands. You can make reference to this article to find out more.

We invoke the set function in the terminal for org1, and query the value with get function. This happens in peer1-org1 and using orderer1-org1.

And we can query the value in the terminal for org2, which is from peer1-org2.

And now we invoke the set function from org2, which is using orderer1-org2.

And now we query the updated value from peer1-org1.

It is complete. The orderer from both organizations can handle transactions from client applications, and the chaincode works properly on this decentralized setup with no system channel.

Summary

We have walked through a setup with decentralized ordering service (orderer owned by peer organization) and no system channel when bringing up an application channel. Hope this setup gives you more insight how to build things up with these new features.