Observing Private Data in Ledger: Adding and Removing Organization in Distribution Policy
Overview
Private data in Hyperledger Fabric provides a way to keep privacy on certain data within a channel. While in a typical setup, all member organizations of a channel keep the same set of data, there are situations in which some data needs to be kept by a subset of organizations. It is largely due to business requirements. Other organizations in the same channel will not keep this data, but a hash of data as an existential evidence of such a piece of data. More detail about private data can be seen here.
In the architecture reference about private data,
“When updating a chaincode definition, you can … update existing private data collections, for example to add new members to an existing collection … .” (Updating a collection definition)
“Starting in v1.4, peers of organizations that are added to an existing collection will automatically fetch private data that was committed to the collection before they joined the collection.” (Private data reconciliation)
In this article we create a test to observe these points. The test environment is a three-organization setup. By tuning the collection definition, we will observe how private data is being handled when a new organization is added and then another is removed.
Test Setup
Our fabric network comes with three peer organizations, each of which a peer (peer0) is deployed. A channel mychannel is created and all the peers join mychannel. We only designated an anchor peer on peer0.org1.example.com.
The configuration is adopted from First Network, with proper modification on crypto-config.yaml and configtx.yaml to reflect our network setup.
Crypto material (inside crypto-config/) and channel artifacts (inside channel-artifacts/) are properly generated. For simplicity we just use this set of pre-generated material.
To demonstrate private data, a chaincode sacc_private3org is developed. It adopts from Simple Asset Chaincode (sacc), by adding two functions setPrivate() and getPrivate(). The data collection is called privateall. This chaincode is inside chaincode/sacc_private3org/, which is mapped to the CLI container.
Besides, we have created three collection definition files. They are,
org1–2.json(privateall for Org1 and Org2)org1–2–3.json(privateall for Org1, Org2 and Org3)org1–3.json(privateall for Org1 and Org3)
As an example, in org1–2.json, we specify a collection with name privateall, and peers of organization Org1 and Org2 keep this data.
These collection definition files are stored under collection/, which is also mapped to the CLI container. When we approve and commit chaincode definition, we will refer to one of them for our testing.
Finally, we include a CouchDB for each peer. The purpose is to observe the databases for private data in each peer.
The test is done following the sequence of collection files. We first set Org1 and Org2 keeping the private data (Scenario 1). After everything is working well, we will add Org3 into collection policy (Scenario 2). As far as we are keeping the chaincode name and tuning the sequence number, it is equivalent to an update and existing ledger will be maintained. Finally, we will remove Org2 from the policy, that is, Org1 and Org3 in collection policy (Scenario 3) and see what happens in the private data still in Org2.
Test Step
Preparation
Step 1: Deploy network
Clone the repository in the fabric-samples/. Note that the crypto material and channel artifacts are already generated. A script bringupnetwork.sh is created to deploy our network.
cd fabric-samples
git clone https://github.com/kctam/3org-privatedata-200
cd 3org-privatedata-200./bringupnetwork.sh
Upon completion we see three peers, one for each organization, and three couchdb containers, one for each peer.
And with anchor peers peer0.org1.example.com is configured, peers know the channel member of cross-organization. Check logs of all peers.
With this the network is ready for private data testing.
Step 2: Package and install chaincode on all the three peers
As we are not modifying the chaincode in these scenarios, this packaging and installation is done only once.
# done once
cd chaincode/sacc_private3org
GO111MODULE=on go mod vendor# packaging
docker exec cli peer lifecycle chaincode package sacc.tar.gz --path github.com/hyperledger/fabric-samples/chaincode/sacc_private3org/ --label sacc_1# peer0.org1
docker exec cli peer lifecycle chaincode install sacc.tar.gz# peer0.org2
docker exec -e CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp -e CORE_PEER_ADDRESS=peer0.org2.example.com:9051 -e CORE_PEER_LOCALMSPID="Org2MSP" -e CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt cli peer lifecycle chaincode install sacc.tar.gz# peer0.org3
docker exec -e CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org3.example.com/users/Admin@org3.example.com/msp -e CORE_PEER_ADDRESS=peer0.org3.example.com:11051 -e CORE_PEER_LOCALMSPID="Org3MSP" -e CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org3.example.com/peers/peer0.org3.example.com/tls/ca.crt cli peer lifecycle chaincode install sacc.tar.gz
Scenario 1: Private Data for Org1 and Org2
First we deploy chaincode with collection/org1–2.json, marked as sequence 1
Step 3: Approve and commit chaincode
Approve chaincode from three organizations
Commit chaincode
Step 4: Invoke setPrivate() to set a private data
docker exec cli peer chaincode invoke -o orderer.example.com:7050 --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n mycc --peerAddresses peer0.org1.example.com:7051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses peer0.org2.example.com:9051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"Args":["setPrivate","name","Alice"]}'Step 5: Query getPrivate() from all peers
We will see private data can be accessed in peers of Org1 and Org2, but not in Org3 as it is outside our collection policy.
Step 6: Observe data collection in ledger
We can see the private data is stored in Org1 and Org2, while only the hash is seen in Org3. This shows our collection policy works well
Scenario 2: Add Org3 into Policy
Then we deploy chaincode with collection/org1–2-3.json, marked as sequence 2
Step 7: Approve and commit chaincode
Approve chaincode from three organizations
Commit chaincode
Step 8: Query getPrivate() from all peers
We see private data is accessible from all organizations, per our updated collection policy. Obviously peer of Org3 obtains the private data as it is not visible previously (Step 5).
Step 9: Observe data collection in ledger
And from the log we see the reconciliation happens, and peer0.org3.example.com learns back the private data.
Scenario 3: Remove Org2 from Policy
Finally we deploy chaincode with collection/org1-3.json, marked as sequence 3
Step 10: Approve and commit chaincode
Approve chaincode from three organizations
Commit chaincode
Step 11: Query getPrivate() from all peers
We see private data is now not accessible in Org2, as in our new collection policy we include only Org1 and Org3.
Step 12: Observe data collection in ledger of peer0.org2.example.com
We still see the previous private data stored in peer0.org2.example.com, although it cannot be read per our new collection policy.
Step 13: Invoke a new data
Let’s invoke to set a new private data.
docker exec cli peer chaincode invoke -o orderer.example.com:7050 --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n mycc --peerAddresses peer0.org1.example.com:7051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses peer0.org3.example.com:11051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org3.example.com/peers/peer0.org3.example.com/tls/ca.crt -c '{"Args":["setPrivate","age","30"]}'From the ledger we know that only peers of Org1 and Org3 get the new private data, and peer of Org2 does not. Its hash is updated (as other peers), but not the private data.
Observation
When a new organization is added to policy, like Org3 in Scenario 2, as far as we are maintaining the same chaincode name and deploying with a new sequence number, Org3 will learn the private data and keep it in its ledger. When we make a query on peer of Org3, we get back the private data.
When we remove an organization from a policy, like Org2 in Scenario 3, the private data immediately cannot be read from the peer of Org2. This is our expected result. However the previous data stored in private data is still in the ledger. Therefore, removing an organization from collection policy does not remove the data which is already in the ledger. In fact I do not expect any possible way to remove them. Nevertheless, new private data is not written in Org2 as it is already out of the policy. To a certain extent it always meets our intention.
Summary
In this article we have created a three-organization network for private data testing. By adding and removing an organization from the collection policy, we can see how Hyperledger Fabric handles the private data according to the collection policy.
