Introduction
In training classes, I always hear people say that Hyperledger Fabric is really complicated. And among everything, it is always the infrastructure part, that is, bringing up the fabric network with all the components, say the Peers, Orderers, Certificate Authority, with proper setup on the organizations and channels. After that, one can run chaincode on top of the network. That part is more “developer-friendly” compared to the underlying infrastructure.
If bringing up a fabric network is challenging, adding an organization in an existing network must be even more. There is a good tutorial in Hyperledger Fabric documentation (link). They provide very detail step-by-step instruction on building things up.
As usual I try to make it systematic such that learner can always have a bigger picture first, and detail step-by-step instruction with my own explanation. This article is serving the same purpose.
This article assumes one has already grasped some ideas about Hyperledger Fabric, both the infrastructure and the chaincode operation. Understanding the overall process of the Build Your First Network (BYFN) example is the foundation of learning this “Adding an Org to a Channel” Tutorial. Therefore make sure you have walked through the detail about BYFN. Of course, you can find good material in the documentation (link), or refer to my previous works on exploring the First Network.
We will first introduce the big picture of the whole process, understanding the overall flow before we go into each one by one.
Overall Process
Review: Build Your First Network
To make a good comparison, let’s review the Build Your First Network flow.
First Network contains two organizations (Org1 and Org2), each of which has two peers deployed (Peer0 and Peer1). One Orderer is setup for the network. A channel mychannel is established, and both organizations and peers join mychannel. After that a chaincode is installed in all peers and instantiated on mychannel. Finally we invoke and query on different nodes and observe the behaviour we define in the chaincode.
What we have done to bring up the First Network.
- Prepare the configuration files for crypto-config and configuration transactions.
- Use cryptogen and configtxgen tools to create the crypto material and the files of transactions, respectively.
- Bring up the nodes of First Network using docker-compose.
- Use command line interface (cli) to create the channel mychannel and all peers joining the channel.
With that the First Network is done. We then perform the chaincode operations.
- Install the chaincode to all peers.
- Instantiate the chaincode from a peer, with the initial arguments required per chaincode.
- Query the value stored in the ledger (in any peers).
- Invoke the chaincode and query the value again after invocation (in any peers).
Here is where we stop in the Build Your First Network process.
Process of Adding Org3 into First Network
We will walk through what is performed in the eyfn.sh script, and do it a step-by-step way using cli.
- Prepare Org3 artifacts
- Update the network to include Org3 configuration
- Bring up Peers of Org3 and join mychannel
- Chaincode operations after Org3 is part of the network
We will go into more detail on each step.
0. Bring Up First Network with byfn.sh
The overall process is built upon a complete First Network setup. So make sure you have complete the whole process.
$ cd fabric-samples/first-network
$ ./byfn.sh up(In case of any problems, try run ./byfn.sh down and then bring up again.)
After running the script you should see the END sign.
And there will be several points for further checking.
Check running containers: you should see NINE containers running. One Orderer, Four Peers, One CLI, and Three chaincode-related Containers.
Also access a Peer (say peer0.org1) and check the block height. It should be 5 after running the script.
We will explain what’s inside the block later, as it is relevant to the Org3 setup.
1. Prepare Org3 artifacts
The idea is close to what we have done on First Network setup. There are two files under first-network/org3-artifacts.
The configuration files to be used
- org3-crypto.yaml
- configtx.yaml
org3-crypto.yaml
The crypto part org3-crypto.yaml is similar to First Network script. Use cryptogen to bring up the directory structure for Org3.
According to the configuration, two peer nodes are created in Org3, with an Admin and an additional User. This setup is identical to First Network for Org1 and Org2.
To generate the crypto artifacts, use cryptogen,
// in first-network/org3-artifacts directory
$ ../../bin/cryptogen generate --config=./org3-crypto.yamlA new directory crypto-config is created, and the resulting directory structure looks like this.
We also need to copy the Orderer from First Network to crypto-config as well.
// in first-network/org3-artifacts directory
$ cp -r ../crypto-config/ordererOrganizations crypto-config/After that we will have the complete directory structure.
configtx.yaml
Unlike BYFN, this file only contains Org3 information. Obviously we are not following what we have done on BYFN script, using configtxgen to generate everything. Here we use option -printOrg to create a JSON file based on this configtx.yaml. The result is stored in org3.json, and store it into first-network/channel-artifacts directory.
// in first-network/org3-artifacts directory
$ export FABRIC_CFG_PATH=$PWD && ../../bin/configtxgen -printOrg Org3MSP > ../channel-artifacts/org3.jsonHere we can take a look on the structure of org3.json, an object that contains the required configuration of Org3. Later we will revisit this file when creating the update.
2. Update the network to include Org3 configuration
The Idea
As a permissioned blockchain, “who can do what” is of the utmost importance. In Hyperledger Fabric, the network setup is in fact written onto the ledger, inside the block. We can perform this easily when we build a brand new fabric network, but becomes very challenging when we need to modify the fabric network when it is already running.
The whole idea is to create a new block containing the new organization configuration. After that the peers of new organization can join the channel, and the whole network knows this organization is part of the network.
It takes several steps “to create a new block containing the new organization configuration”. In this session we are exploring this.
Tool: configtxlator
As you will see, there is a tool we will use quite often. It is configtxlator. This tool comes in Hyperledger Fabric, usually inside fabric-sample/bin file. The tool is accessible in CLI.
The tool deals with a format called protobuf. Without further going into detail, we just treat it a binary format. It is corresponding to JSON format, which we will deal with quite a lot in this session.
There are several options we will use.
- proto_encode: encodes JSON format to protobuf format
- proto_decode: decodes protobuf format to JSON format
- compute_update: computes the delta of two protobuf format in configuration and produces the configuration update from the first protobuf to the second
2.1 fetch the latest configuration block from the ledger
The first step is to fetch a configuration block from the ledger. And we will act upon this configuration block after.
Before everything, we first examine what are inside the blocks in the blockchain (ledger). As indicated in my previous work (link) after the byfn.sh, there are FIVE blocks on the ledger.
- block #0: Genesis block (mychannel.block)
- block #1: Anchor peer for Org1 update
- block #2: Anchor peer for Org2 update
- block #3: chaincode mycc instantiation (a, 100, b, 200)
- block #4: chaincode mycc invoke (a, b, 10)
The first three are the configuration, and the last two involved chaincode operations. We need to get the latest configuration block, that is, #2.
First shell to cli. Note that the environment variables by default is on peer0.org1.example.com.
$ docker exec -it cli bashThe command to get the configuration block is peer channel fetch config. From cli to get the latest configuration block on the specific channel.
# export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem && export CHANNEL_NAME=mychannel
# peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CAFrom the result we see block #2 is received, and it is stored in a file config_block.pb, which is a protobuf binary format.
2.2 prepare the configuration update for Org3
We begin with the config_block.pb file. This file has all the information inside the block. We do not need the whole block. Instead we extract the portion relevant to our construction of configuration update.
Use configtxlator proto_decode config_block.pb into JSON format, and use jq to retrieve the required portion. The result is kept in config.json.
# configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.jsonInside config.json, we clearly see configuration of Org1MSP and Org2MSP. We are now to include the Org3MSP into it, and the Org3MSP is from org3.json we generated in Step 1.
# jq -s '.[0] * {"channel_group":{"groups":{"Application":{"groups": {"Org3MSP":.[1]}}}}}' config.json ./channel-artifacts/org3.json > modified_config.jsonThe command is to use jq to insert org3.json into config.json, at the right location and the key name “Org3MSP”. The result is modified_config.json.
We now have the two JSON files,
- config.json (which contains org1 and org2)
- modified_config.json (in which org3 is included)
The next step is to find out the delta between the two files in protobuf binary format. We first encode it back to protobuf format, and then find out the difference, which is the configuration update needed.
It can be done in three instructions using configtxlator. The result is org3_update.pb.
# configtxlator proto_encode --input config.json --type common.Config --output config.pb# configtxlator proto_encode --input modified_config.json --type common.Config --output modified_config.pb# configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output org3_update.pb
This file cannot be used yet. We have to add an envelope on it. Again, the envelope is in JSON. As the last step, we decode the org3_update.pb again, add a wrapper, and encode it back to org3_update_in_envelope.pb.
Here are the instructions.
# configtxlator proto_decode --input org3_update.pb --type common.ConfigUpdate | jq . > org3_update.json# echo '{"payload":{"header":{"channel_header":{"channel_id":"mychannel", "type":2}},"data":{"config_update":'$(cat org3_update.json)'}}}' | jq . > org3_update_in_envelope.json# configtxlator proto_encode --input org3_update_in_envelope.json --type common.Envelope --output org3_update_in_envelope.pb
We are curious enough to explore what is inside org3_update.json and org3_update_in_envelope.json.
The result org3_update_in_envelope.pb is the file for the next step.
It is not an easy task, but it is really logical when creating this configuration update. Here is a simple summary on what we have done so far.
2.3 sends update to Orderer and a new block containing Org3 configuration is committed back to the ledger
The file org3_update_in_envelope.pb is the update transaction for the First Network.
The default policy to update the channel is by majority, which means a majority of administrator from existing organizations. In BYFN we only have Org1 and Org2, which means that we need both signatures.
By default, cli is acting on peer0.org1.example.com. We will open up another terminal for another cli acting on peer0.org2.example.com.
Open a new terminal with environment parameters.
// a new terminal for peer0.org2$ docker exec -e CORE_PEER_LOCALMSPID="Org2MSP" -e CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -e CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp -e CORE_PEER_ADDRESS=peer0.org2.example.com:7051 -it cli bash// perform export environment as well# export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem && export CHANNEL_NAME=mychannel
We will sign the update in peer0.org1.
// cli for peer0.org1.example.com# peer channel signconfigtx -f org3_update_in_envelope.pb
And then we send the update in peer0.org2, which will include signature from Org2 as well.
// cli for peer0.org2.example.com# peer channel update -f org3_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA
After the update is sent to Orderer, Orderer will process it and send a new block to the fabric network. Now the whole network should see the new block containing Org3 information.
Let’s check the block height now on both org1 and org2 cli.
// cli for both peer0.org1 and peer0.org2# peer channel getinfo -c mychannel
It should show “height” as 6. Compared to Step 2.1, the ledger has one more block, and the network is ready for Peers of Org3 joining.
3. Bring up Peers of Org3 and Join mychannel
We have gone through the most tedious steps in updating the whole First Network with Org3 configuration. What remains is just normal steps. In this session we bring up the peers of Org3 and a cli by default acting on peer0.org3.example.com.
The docker-compose file for org3 is docker-compose-org3.yaml. Inside it two peers and one cli (Org3cli) will be up and running.
From a new terminal,
// a new terminal for Org3cli$ docker-compose -f docker-compose-org3.yaml up -d
Shell to Org3cli and set the environment variables.
// Org3cli$ docker exec -it Org3cli bash# export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem && export CHANNEL_NAME=mychannel
To make peers of Org3 joining the mychannel, we need the genesis block (block #0).
Go to peer0.org1 cli.
// cli for peer0.org1.example.com# peer channel fetch 0 mychannel.block -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
The file mychannel.block is the genesis block. We copy mychannel.block to Org3cli such that we can use it in Org3cli.
// my localhost$ docker cp cli:/opt/gopath/src/github.com/hyperledger/fabric/peer/mychannel.block .$ docker cp mychannel.block Org3cli:/opt/gopath/src/github.com/hyperledger/fabric/peer/
Now we can join the peers of Org3 using this file.
// Org3cli# peer channel join -b mychannel.block
Optionally we can use the same Org3cli to join peer1.org3, by specifying the environment variables.
// Org3cli# CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org3.example.com/peers/peer1.org3.example.com/tls/ca.crt CORE_PEER_ADDRESS=peer1.org3.example.com:7051 peer channel join -b mychannel.block
How can we know the peers of Org3 already “in-sync”? We can check the blockchain height again.
// Org3cli# peer channel getinfo -c mychannel
We see “height” 6 again. That means peer0.org3 has already gets the ledger update.
4. Chaincode operations after Org3 is part of the network
The last step is about the chaincode. In BYFN script, a chaincode mycc is installed in the peers of Org1 and Org2, instantiated in mychannel. An invocation is done such that the state has been updated (for your information, the current state is the value of a is 90, of b is 210).
The interesting part is the the original chaincode mycc (version 1.0) is not yet installed in peers of Org3. Also, the endorsing policy should be updated such that Org3 is also a valid member to endorse invocation.
First we install a new version of chaincode (2.0) to all organizations. Now we just focus on peer0 of each organization.
// cli for peer0.org1.example.com# peer chaincode install -n mycc -v 2.0 -p github.com/chaincode/chaincode_example02/go/// cli for peer0.org2.example.com# peer chaincode install -n mycc -v 2.0 -p github.com/chaincode/chaincode_example02/go/// Org3cli# peer chaincode install -n mycc -v 2.0 -p github.com/chaincode/chaincode_example02/go/
Then we can upgrade the chaincode. Like instantiation, upgrade will execute Init() inside the chaincode. If any arguments are needed when we instantiate the chaincode, we also need to specify. Also, we specify the endorsement policy to include Org3.
// cli for peer0.org1.example.com# peer chaincode upgrade -o orderer.example.com:7050 —-tls $CORE_PEER_TLS_ENABLED —-cafile $ORDERER_CA -C $CHANNEL_NAME -n mycc -v 2.0 -c '{"Args":["init","a","90","b","210"]}' -P "OR ('Org1MSP.peer','Org2MSP.peer','Org3MSP.peer')"
As usual, it takes some time to build the container images and instantiate a running container. Once we see the instruction is done, we can issue query value from Org3cli.
// Org3cli# peer chaincode query -C $CHANNEL_NAME -n mycc -c '{"Args":["query","a"]}'
It takes the same a while, and the result 90 is shown.
Finally we will perform an invocation on Org3cli and query the value on Org2 for sake of demonstration.
// Org3cli# peer chaincode invoke -o orderer.example.com:7050 --tls $CORE_PEER_TLS_ENABLED --cafile $ORDERER_CA -C $CHANNEL_NAME -n mycc -c '{"Args":["invoke","a","b","10"]}'// cli for peer0.org2.example.com# peer chaincode query -C $CHANNEL_NAME -n mycc -c '{"Args":["query","a"]}'
Now we see the result on peer0.org2 cli it is 80, which means the invocation is completely processed and the ledger of the whole network is updated accordingly.
5. Clean Up
As good practice, we clean up everything after the exercise.
Exit all CLIs.
Use ./byfn.sh down to bring down the network. This script will tear down all configuration including that of Org3 as well.
// in first-network directory$ ./byfn.sh down
Closing
This ends our lengthy process. Again, adding one more organization is never an easy task due to the permissioning nature in permissioned blockchain like Hyperledger Fabric. Nevertheless, during this process we know more about some in-depth architecture of Hyperledger Fabric. I have to say the overall process is logical, though I am expecting some improvement later to make life easier.
Wish to remove an organization from a running fabric network? Here my latest article may help. See here!
