From “First Network” to “Test Network”: A New Test Network introduced in Fabric v2.0
Overview
Another newly introduced item in Hyperledger Fabric release 2.0 is Test Network. If you have followed my articles on Hyperledger Fabric, the most common network for demonstration is always the First Network. An eye-catching description in the official document is that
(T)he test network is being introduced in Fabric v2.0 as the long term replacement for the first-network sample.
Therefore it is time to have a first understanding on the Test Network.
In this article I will describe the Test Network, mainly with some parallel comparison with First Network. Through this comparison we can understand some advantages of using Test Network, while knowing what is not available in this new network.
You can find the official information about Test Network here.
Some Key Findings
There are some quick findings when compared with First Network.
- Similar to First Network, Test Network comes with a well designed script called
network.sh. In fact it inherits a lot of script code from the scriptbyfn.shof First Network. - Ordering Service in Test Network is implemented as Solo, which means only one orderer is there providing the service. Bear this in mind though this does not have impact on chaincode development and testing. In contrast, five orderers are deployed as an orderer cluster using Raft in First Network.
- Each peer organization comes with one peer in Test Network, while First Network comes with two peers for each peer organization.
- In Test Network, crypto material can be generated either through
cryptogenor by organization certificate authority. In First Network, onlycryptogenis available. If organization CA is needed, it is created through material created incryptogen. - We can create more than one channels with the Test Network script
network.sh. In First Network we can create only one channel. - The sample chaincode in the Test Network script is fabcar, while that in the First Network script is abstore (was named chaincode_example02 in previous 1.x releases).
- Test Network does not bring up a CLI container. Peer commands are issued from local host. Proper environment variables are required before we can use the CLI command.
With this quick comparison, we can now describe Test Network.
Test Network Architecture
Components
Test Network contains three organization: one Orderer Organization and two Peer Organizations.
There is one ordering service node (orderer) in the Orderer Organization, which means the ordering service is implemented as Solo.
The two peer organizations are Org1 and Org2. Each peer organization is deployed with one peer (peer0).
Test Network allows two ways of crypto material generation. By crypto material, we are referring to the digital certificate and the signing key (secret key) for all the components and users. The first way is using the binary tools cryptogen, the one we have been using in First Network for long time. The other way is a complete build-up from a certificate authority (CA). In this case, one CA is deployed in each organization (orderer organization and peer organization) and crypto material is generated in the CA. We will cover the script about the CA later in the script session.
Channels
Test Network provides a flexible way to create channel, in a way that we can create more than one channels. It is done by restructuring the channel related activities into one single step and into one script. Once we specify the channel ID when executing the script, the configuration transactions, creation of channel genesis block and peers of member organizations joining channel will be done. As a comparison, First Network comes with one channel when the script executed. We will take a look on these channel related activities on the script session.
Sample Chaincode
While chaincode is not part of a fabric network (instead we say chaincode is deployed on a fabric network), the scripts provided by both Test Network and First Network come with sample chaincode deployment. It helps to verify that the fabric network is operating correctly with this sample chaincode.
Test Network comes with the fabcar example. Fabcar is a good sample chaincode created before Fabric v2.0. While the logic is not changed, the code in Golang coming with Fabric v2.0 is re-written with contract API. You can refer to my previous article about the overview of fabcar, and I am planning another article describing the difference when using contract API later.
As a comparison, First Network defaults the abstore, which was named as chaincode_example02 in previous releases. It only comes with some simple illustration of how chaincode is structured and implemented, despite no specific practical use. Nevertheless, it is the same code used in First Network before v2.0.
Here is a quick side-by-side illustration on both Test Network and First Network.
Review the First Network Script
As said, network.sh is a well designed script for Test Network. It comes with modes and flags to perform several tasks. Most of the scripts are inherited from the First Network, with some modification and restructuring to make the script more flexible. Therefore, it make sense to first review the flow of the script in First Network. This helps to understand the design of network.sh.
The script byfn.sh is the master script that bring up the network and deploy the sample chaincode. With no option when running the script, here is the overall flow, when you execute ./byfn.sh up.
Step 1: Crypto material generation. This involves using cryptogen on the configuration file crypto-config.yaml. The result is the crypto material stored inside crypto-config directory.
Step 2: Channel artifacts generation. This involves using configtxgen on the configuration file configtx.yaml. The channel artifacts contains
- a genesis block,
- a configuration transaction for the channel specified, and
- two anchor peer update transactions
The first item genesis block is a generic one, channel-agnostic. It is kept in the orderer service node(s), and will help generating the the channel genesis block when applying the configuration transaction.
Step 3: Bring up the components. All the components are described in the docker-compose files. It will bring up
- raft-based orderer cluster (five orderers)
- four peers (belonging to two peer organizations)
- a CLI container
The last item is not part of the fabric network. It only serves as a tool when dealing with chaincode operation and interaction.
Step 4: Create channel and join peers to the channel. Assuming we use the default mychannel. This step takes what we have generated in Step 2 to create a channel genesis block. The result is a block file used when all participating peers join the channel. Finally apply anchor peers for both peer organizations.
At this stage, the fabric network is up and running. The next step are relating to chaincode operation, that is, to make chaincode useable, and chaincode interaction, that is, to invoke and query chaincode functions.
Step 5: Install chaincode abstore to selected peers (peer0.org1 and peer0.org2). Instantiate chaincode abstore to mychannel. As abstore requires chaincode initialization (Init in the chaincode), argument is provided in this step.
After chaincode is useable, the script will invoke and query chaincode functions. Note in First Network, it also shows to install chaincode to another peer before chaincode function query can be done. We omit the detail here.
This ends the overall byfn.sh script.
Besides, there are various options provided in First Network. Here I just listed those of interests.
-s couchdb: bring up CouchDB containers for all the four peers
-n: bring up the First Network without running the default abstore chaincode
-a: bring up the Certificate Authority for the two peer organizations. This is needed when we need an organization CA when generating additional user (client) certificates
Test Network: network.sh
The master script for Test Network is network.sh. As mentioned before, network.sh inherits many script modules from byfn.sh. The most important part is that network.sh makes restructuring on the function such that it provides the following options when running this script.
Overall Flow
This is a parallel flow for network.sh as what we compare with First Network.
Step 1: Crypto material generation. Here Test Network provides two methods to generate the crypto material.
Using cryptogen: it is similar to how byfn.sh in First Network is doing. The crypto-config file is broken into three directory under organizations/cryptogen/. The result is stored inside organizations, with orderOrganizations and peerOrganizations, similar to the result previous.
Using Organization Certificate Authority (CA): this is newly added way without using the pre-built cryptogen. The idea is to first launch CA in each organization (orderer organization and the two peer organizations). Then use fabric-client to generate all the components (orderer, peer) and users (admin, user1) through registering and enrolling the Organization CA. The result is organized into the similar directory structure as using cryptogen. The option is turned on by putting -ca as option when bring up the Test Network. The script handling CA is in organizations/fabric-ca/registerEnroll.sh.
No matter which methods are used, the result is correctly mapped to the MSP of each components.
Step 2: Generate the genesis block. This is equivalent to the first item in Step 2 of First Network. Here the channel (e.g. mychannel) specific artifacts are not generated here. It is being moved to channel creation in a later step. The result is stored in system-genesis-block, which is correctly mapped to orderer’s directory.
Step 3: Bring up the components. All the components are described in the docker-compose files. These files are stored in docker directory. The one being used is docker-compose-test-net.yaml. It will bring up
- one orderer
- two peers (for two peer organizations)
Note that there is no CLI container in the docker-compose file.
Besides, if -ca is specified, it is the docker-compose-ca.yaml bringing up first in Step 1. This will bring up the three CAs, one for each organization.
Step 4: Create channel and join peers to the channel. This step involves creating and applying channel-specific material to Test Network, including the following steps
- generate configuration transaction for channel
- generate anchor peer update transactions for channel
- apply the configuration transaction to the network, resulting the channel genesis block file
- join the peers with the channel genesis block file
- apply the anchor peer update transactions to the network
As seen, the first two items are the missing part in Step 2 in First Network. It is moved to this Step 4. The remaining is the standard process to join peers to a channel. This restructuring makes sense as the channel related activities are now grouped together, and it can provide an option of creating multiple channels. We will demonstrate this later. The script performing this task is scripts/createChannel.sh.
At this stage, the Test Network is up and running. The next step are relating to chaincode operation, that is, to make chaincode useable, and chaincode interaction, that is, to invoke and query chaincode functions.
Step 5: Use lifecycle chaincode to deploy chaincode fabcar to the channel. In Hyperledger 2.0 we now use lifecycle chaincode, which is a four-step process. In a nutshell, chaincode is first packaged and installed in the select peers. With default setting, each peer organization needs to approve chaincode with the same set of parameters (chaincode definition). After approvals are done by all peer organizations, chaincode can be committed to channel and become useable. You can refer to whole process in the official document, and refer to my previous articles (this, this) for further elaboration.
After chaincode is useable, the script will invoke and query chaincode functions. This simply invoke a fabcar function and query the result.
The two steps above are performed in scripts/deployCC.sh.
Various Modes in network.sh
With this flow, network.sh comes with modes and options that provides flexibility when one brings up the Test Network.
network.sh up This brings up the components without configuring any channel. This includes one orderer and two peers. In case -ca option is given, there are also three CA containers. If -s option is given, it also brings up the two CouchDB containers, one for each peer.
network.sh up createChannel This brings up the components (orderer, peers and optionally CA and CouchDB). In additional it also creates a channel and the two peers join the channel. The default is mychannel, but one can specify -c with a new channel ID. The code is in scripts/createChannel.sh.
network.sh createChannel This is used to create a new channel after a network is up and running. We can use this after network.up to bring the first channel, or after network.sh up createChannel to create one more channel. Again channel ID is specified with -c option, and the two peers will join this new channel. The script does not allow customized channel member and therefore both organizations (both peers) will join. The code is in both network.sh and scripts/createChannel.sh.
network.sh deployCC This is used after a channel is deployed. The script will bring up the fabcar chaincode and invoke two functions (InitLedger and QueryAllCars) to see if chaincode is running. We can use -l option to specify what chaincode language is used. The default is Golang, while options are JavaScript and Java. The code is in scripts/deployCC.sh
network.sh down This can tear down everything, including components and any chaincode related containers and images. Use this whenever we need a clean environment.
This summarizes the various modes in network.sh script.
Demonstration on Test Network
Finally here is just a demonstration on what happens when we run network.sh. We will show several scenarios.
Directory Structure
Here is the directory structure of Test Network.
Those directories of our interest in this article are
configtx: where the configtx.yaml is keptdocker: where all the docker compose files are keptorganizations: where the crypto material configuration files and generated material are keptscripts: keeping the scripts for different modessystem-genesis-block: where the genesis block (system one, stored in orderers) is kept and mapped to the orderer
Bring Up Test Network Components
We begin with no options specified.
./network.sh updocker psdocker exec peer0.org1.example.com peer channel list
We will see three containers are running, one orderer and two peers for two organizations.
Meanwhile we see no channel is created with this command.
See if we bring up with -s couchdb option.
./network.sh down # tear down previous network
./network.sh up -s couchdbdocker ps
We see two CouchDB containers there, one for each peer.
If we bring up the network with -ca option, we will see the additional three containers on CAs as well, one for each organization.
Here are the containers running with -ca option.
./network.sh down # tear down previous network
./network.sh up -cadocker ps
Note: when using network.sh down, there are permission error messages found when deleting directories. It does not impact the upcoming demonstration as we are not using -ca any more. Will be updated once this issue is fixed.
Bring Up Test Network Components with Channel
Now we shift our focus on channel. We first bring up the network with default mychannel.
./network.sh down # tear down previous network
./network.sh up createChanneldocker psdocker exec peer0.org1.example.com peer channel list
On top of this, we can add one more channel, say newchannel.
./network.sh createChannel -c newchanneldocker exec peer0.org1.example.com peer channel list
docker exec peer0.org2.example.com peer channel list
Now we see one more channel created and the peers have joint this new channel.
Deploy Chaincode on a Channel
If we first tear down everything and directly deploy with this mode, it fails as nothing exist yet.
./network.sh down # tear down previous network
./network.sh deployCCNow we try to bring up the network without forming the channel, and try deploy chaincode again. It still fails as there is no channel.
./network.sh down # tear down previous network
./network.sh up
./network.sh deployCCFinally, we will bring up the Test Network with default channel, and then deploy the chaincode.
./network.sh down # tear down previous network
./network.sh up createChannel
./network.sh deployCCWe will get back the expected result, which shows the response when query the chaincode.
A note for Interacting with Chaincode
In the First Network we have a CLI container, and all peer commands can be issued from the container either using docker exec cli <command> or go into a shell docker exec -it cli bash. There is no CLI container defined in Test Network. Those peer command are instead executed from local host.
Here is where the peer executable is.
The network.sh is well scripted such that using the peer command with correct environment variables set. If we need to interact the Test Network similarly, we need to include the following setting.
cd fabric-samples/test-networkexport PATH=${PWD}/../bin:${PWD}:$PATH
export FABRIC_CFG_PATH=$PWD/../config/# for peer command issued to peer0.org1.example.com
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org1MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CORE_PEER_ADDRESS=localhost:7051# for peer command issued to peer0.org2.example.com
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org2MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp
export CORE_PEER_ADDRESS=localhost:9051
With this, we can use peer command upon peer0.org1 (or peer0.org2). For example, for peer0.org1.example.com,
With these variables, we can check the peer channel and peer chaincode upon peer0.org1.
peer channel listpeer chaincode query -C mychannel -n fabcar -c '{"Args":["queryAllCars"]}'
Summary
In this article we have gone through some detail about Test Network with a reference to and comparison with First Network. Certain enhancement and modification is made (like channel creation, CA generated crypto material), while some changes are not as handy as before (CLI container, and if you wish to test Raft-based setup). Nevertheless, it is good enough for a network when you development and test your own chaincode. If you have been working with First Network for a while, there should not be difficulty to get hold of this newly introduced Test Network.
