Two Ways to Generate Crypto Materials in Hyperledger Fabric: Cryptogen and CA Server

Overview

One of the most asked questions on Hyperledger Fabric is about identity. A permissioned blockchain requires that an entity, be it a client, an administrator or a network component, must be first identified and permissioned before accessing a consortium network. Hyperledger Fabric uses several concepts like organization, membership service providers, digital certificate, which makes the understanding of the whole picture a little challenging for new learners. Here in this article again I try to elaborate these concepts through analyzing Test Network, a network example in fabric material, and the two different ways to generate the crypto material for these entities. I hope to give you another perspective on this topic.

Organization and Membership Service Provider

Hyperledger Fabric is always chosen to build a consortium network to address a business problem. The participants are usually business entities in this case. An example is a trade finance platform, where participants are mainly the banks, plus some governance or regulatory bodies. Hyperledger Fabric uses organizations to represent these participating entities. In the trade finance platform, each bank is an organization. A governance body is also another organization. This is how we understand organization from a business perspective.

In Hyperledger Fabric, each organization is characterized by its Membership Service Provider (MSP). On one side, MSP creates credential material to all entities within an organization. On the other side, MSP represents the organization to participate in a consortium network. Technically the consortium network is formed by the MSP of all organizations. They trust one another when the MSPs are functioning well.

MSP is implemented in Public Key Infrastructure (PKI). In a typical PKI environment each entity is holding a private key and a digital certificate (certificate) containing the public key and relevant information about this entity. This certificate is signed and issued by a Certificate Authority (CA) and becomes the identity of this entity.

When an entity signs a message with its private key, it produces a signature on that message. Everyone who obtain this message, signature and the signer’s certificate, can

  • tell that only the private key owner can create this signature (through public key from certificate)

This signing and verification process happens everywhere in a consortium network. Each organization knows the MSP of other organizations in a consortium network, and trust of this consortium network is built up from a “who’s who” perspective. As an example, here is a part of a transaction showing the endorsement, and we can see those information mentioned above.

Image for post
Image for post
A capture of a transaction showing identity of an entity (endorser) and the signature.

In this transaction two endorsements are included. Each endorsement contains

  • endorser: the certificate (identity) of endorser in base64

Everyone in the consortium can verify that it is the endorser in the certificate signing this proposal response, and therefore this endorsement is accepted.

Test Network

It is always too abstract if just reading the material without examples. Here we use Test Network from fabric samples as an example. Similar to First Network in previous releases, Test Network comes with a well-designed script in fabric v2.0. It is a three-organization design, with one orderer organization and two peer organizations.

Image for post
Image for post
Test Network: Organization and entities inside each organization

In general the structure of an organization includes

  • MSP, or we use the term Identity CA (or simply CA) representing it

We use network components and network users separately if the discussion is specific to each of them. If something applies to both network components and network users, we will use the term entities of an organization.

CA is responsible for handling the identity of all entities within an organization. Each entity obtains a credential (crypto material) from CA. The credential includes a private key and a digital certificate representing this entity. The security is maintained as long as the entity is keeping the private key safe.

While the main purpose of CA is on identity area (prove an entity is what it claims to be), we will encounter another set of crypto material for another purpose. In Hyperledger Fabric communication between components are also secured by TLS, for example, client application to peers, cluster communication between orderers, etc. Although this article is more focused on identity, we will also take a look on TLS, but you should always see them as two separate systems for different purposes.

A Note about Directory Structure of Crypto Material

For those who have gained some experience in First Network in the previous releases or Test Network in v2.0, you may have noticed that the crypto material is always stored in a fixed directory structure.

Image for post
Image for post

We will examine the directory in detail, but I wish to point out that there is not a must to follow such a data structure. If we trace back a bit history, at the beginning we are using cryptogen (more detail in next session) to create the crypto material, and docker-compose to bring up network components. Inside the docker-compose configuration files directories are mapped to each network component. For example, crypto material for peer0.org1.example.com is correctly installed in this peer through docker-compose files.

Image for post
Image for post
crypto material of peer0.org1 is correctly mapped to peer0.org1 containers in docker-compose files

It is a decent and systematic way as it can be coded into configuration. Nevertheless, you also can mount the crypto material one-by-one and manually, if it is desired.

As we still rely on docker-compose files for bringing up network components, we will follow this directory structure. It is automatically generated with cryptogen, but some more copy-n-paste is needed when we are using CA Server.

With these, we can start looking into the first way to generate crypto material: cryptogen.

Generating Crypto Material using Cryptogen

Hyperledger Fabric provides a tool that crypto material can be generated with minimum configuration. The tool is bin/cryptogen. Working with a configuration file, the crypto material of Test Network is generated and the result is kept as the directory structure shown above. With that, we can bring up the consortium network with docker compose files.

Directory Structure

We first use ./network.sh script to bring up Test Network and make some observations on the directory structure.

cd test-network
./network.sh up

Per script design the material is kept in organizations/. Here we take Org1 as an example for exploration.

Image for post
Image for post

Here are a quick summary for all these components

  • ca/: a directory holding crypto material for CA of Org1

We break it further down and observe some relevant items.

CA Certificate

We first take a look at the CA of Org1. It is this CA issuing identity (certificate) to all entities within Org1.

Image for post
Image for post

From the directory we see a private secret key and a certificate. We will inspect the subject and issuer of the certificate.

Image for post
Image for post

It is a self-signed certificate (subject == issuer). It is a typical design for non-production environments, as we allow a new self-signed certificate as the CA to issue identity for the whole organization. This is what cryptogen can provide, and this is also its limitation (see discussion later).

Network Components (peer)

As Org1 is a peer organization, the network component is “peer”. In the peers/ we see a list of peers defined per the crypto configuration file (will be examined later). In the Test Network we have only one peer.

Image for post
Image for post

Double click to this directory, there are two directories. msp/ is for identity, and tls/ is for TLS material. This directory structure will be seen in every entity.

Image for post
Image for post

We focus on the identity part. We will revisit the TLS material when we cover TLS later.

Inside msp/ the directory structure looks like this. We will be very familiarized with this directory structure later, as it is the structure for all entities, both network components and network users.

Image for post
Image for post

For identity, the private secret key of this peer is stored in msp/keystore/, while the certificate is kept in msp/signcerts/.

Image for post
Image for post

We take a look at the certificate of this peer. The issuer is the CA of Org1 mentioned above, and the subject is CN=peer0.org1.example.com and OU=peer.

Image for post
Image for post

Network Users

Network users are kept in users/. From the directory we see a list of users defined in the crypto configuration file (will be examined later). In Test Network we have one Admin, and one user named User1.

Image for post
Image for post

Let’s take Admin as an example. Again there are two directories. msp/ is for identity, and tls/ is for TLS material.

Image for post
Image for post
Image for post
Image for post

Here is the private key and certificate for Admin.

Image for post
Image for post

Finally we take a look at the certificate of Admin, both the subject and issuer. We see that the subject contains OU=admin, and the certificate is issued by the CA of Org1.

Image for post
Image for post

Same directory structure for User1. There is a difference in the certificate subject. We see OU=client for User1. Obviously it is the difference of roles in these two network users. Admin is an admin, while User1 is a client user. If some actions only allow admin role to perform, User1 is denied.

Image for post
Image for post

TLS

Though TLS is not relevant to the identity system, the crypto material for TLS is also generated during the cryptogen process. Here we just make a highlight and locate those materials inside the directory structure.

A TLS CA is also created for Org1. The TLS CA Certificate is stored in tlsca/. TLS crypto material for all entities in Org1 is generated from this TLS CA.

Image for post
Image for post

We see this is a self-signed certificate. Therefore in cryptogen the Identity CA and TLS CA are different systems, and do not have any relationship to each other.

Image for post
Image for post

In a TLS communication, network components play a TLS server role. Therefore each network component is installed with a TLS server key and server certificate. As in Org1 we only have one peer, we can inspect the material inside peers/peer0.org1.example.com/tls/.

Image for post
Image for post

And again we see the TLS server certificate server.crt is issued by TLS CA.

Image for post
Image for post

Network user plays a TLS client role. Therefore each network user is installed with a TLS client key and client certificate. We take Admin as an example here.

Image for post
Image for post

And the client certificate client.crt is also issued by TLS CA.

Image for post
Image for post

This is a diagram showing the relationship of crypto material generated with cryptogen. Note that the Identity CA for each organization is just a CA’s private key and a CA Certificate. There is no software currently running as a CA Server. You will not see a container running in any CA Servers generating this. This certificate issuance process is done by cryptogen software itself. Nevertheless, one can easily bring up a CA Server with CA’s private key and CA Certificate when identity for more entities is needed.

Image for post
Image for post
Crypto material generated using cryptogen tool in Test Network

Configuration File

We have seen the result of cryptogen. Now we take a look at the configuration files, which tell cryptogen how to generate the crypto material shown above.

The configuration for Test Network is kept inside organizations/cryptogen/. Inside these files we can specify the organization structure of the network (peer organization and orderer organization), and detail of entities within an organization. These configuration files are self-explanatory. For sake of completeness we put them here.

For orderer organization. The result is one orderer with name orderer.example.com. One Admin user is generated by default.

Image for post
Image for post
configuration for Orderer Organization in Test Network

For peer organization (here we see Org1). The result is one peer (Count 1 in Template) and the name is by default peer0.org1.example.com. One Admin user is generated by default. The Count 1 in Users means one additional user is generated as well, with a name User1 by default.

Image for post
Image for post
configuration for Org1, a PeerOrganization in Test Network

This matches all the entities we see above in Org1.

Use of cryptogen

In the script network.sh, the crypto material is generated in this command.

cryptogen generate --config=<configuration_file> --output=<output_directory>

This command is executed for all the three organizations: Org1, Org2 and Orderer Organization.

We can see how cryptogen works: with this command and configuration files, we can build the directory structure of crypto material shown above.

Discussion

Cryptogen by far is the simplest way for us to generate crypto material for typical network designs. No matter how many orderers we are to deploy (one for Solo, five for Raft, etc) in orderer organization, how many peers needed for each peer organization, or how many client users needed in each organization, we can specify the requirement in the configuration files and generate the material with a simple command.

This is also where the limitation lies. From Hyperledger Fabric documentation, we learn that cryptogen is good for generating crypto material for testing purposes, and would normally not be used in production operation. There are several considerations.

One is that the CA certificates are all generated as self-signed certificates. In real life we may ask for different settings. For example, we cannot use an existing CA as the MSP for our organization, or we cannot chained certificates.

Also it is predefined with organization type: orderer organizations and peer organizations are defined separately. If for example we need orderers owned by peer organizations, we have to use some workaround (refer to this article on this requirement).

Last but not the least, material is generated once based on configuration file, but in real life we may need more peers or more clients after deployment. Although it is still doable, as we can generate the new entities with CA’s private key, it is not as simple as running cryptogen on a configuration file.

If you are just creating for testing, the above should not be your concern, and cryptogen is good enough. Otherwise, you can take another approach: bring up a CA server and generate the crypto material for entities with this CA according to your desired network design. This is the second way to generate crypto material for a consortium network.

Generating Crypto Material using CA Server

We see some limitations in using cryptogen. To gain more flexibility and make it more practical, a better way is to bring up a CA Server, and we can generate crypto material for entities according to our network design. This involves more steps, but provides a standard way to either generate crypto material that cryptogen cannot generate, or for future use if more network components and users are needed.

Overview

A CA Server is a software to generate crypto material on request. Inside a CA Server there is a CA, represented by a private secret key and a CA Certificate. Any certificates signed and issued by this CA is verifiable with the CA Certificate (or CA’s public key). CA’s private secret key and CA Certificate is the essence of a CA, while a CA Server is just a software tool to perform this signing.

Technically speaking, once one gets hold the CA’s private secret key and CA Certificate, one can issue certificates. It can be done on a CA Server, or it can be done by common tools such as openssl. Therefore, always make a difference between a CA (representing the ownership of that private key) and a CA Server (where certificate is signed and issued).

There are many CA Server products in the market. Hyperledger Fabric comes with one, Fabric-CA Server. This whole process begins with bringing up a Fabric-CA Server, running as a docker container. We either use an existing CA for this CA Server, or generate a self-signed certificate as root CA. In the Test Network example, it is a self-signed certificate, and we then generate all crypto material from it.

To interact with Fabric-CA Server, we need another tool on our side. It can be either a Fabric-CA Client or Fabric SDKs. Fabric-CA Client is used in Test Network script. Note that Fabric-CA Client is running on our localhost, and it interacts with the Fabric-CA Server which is a running container. You can refer to a comprehensive explanation in fabric documentation (link).

Directory Structure and Content

As mentioned before, we need the same directory structure in order to match the docker-compose configuration. Therefore the result of using CA needs to be the same as the result after using cryptogen. This involves some name changing and file moving. Instead of repeating everything about the directory structure, here we highlight some difference on the content and process.

We first use ./network.sh script to bring up the minimum setup and observe the resulting directory structure and content. Note the option -a is required to bring up crypto material using CA instead of cryptogen.

cd test-network
./network.sh up -ca

The majority of steps are in fact executed in the script fabric-ca/registerEnroll.sh.

We see three CAs, one for each organization, up and running. As a comparison, no CA containers are running when we bring up the network using cryptogen.

Image for post
Image for post

Only the CA Certificate is here, which is a public information, but there is no private secret key (scroll up to the session on cryptogen and see a comparison). It is because the private secret key is in the CA Server. This is a more realistic way as it is the CA Server holding that secret, and it should not be available and visible outside the CA Server. We will see more detail about this CA Certificate when we dive into the Fabric-CA Server

The structure for entities (peer and users) are similar. Here we just bring out the certificate of the peer and both users. They are all issued by CA of Org1, with proper identifiers in OU.

Image for post
Image for post

We now take a look at the TLS. First the TLS CA. Interesting enough, we see the same certificate as CA.

Image for post
Image for post

In the previous session about cryptogen, we see a separate TLS CA for all TLS material. When using a CA, the script of Test Network generates all crypto material for TLS with the same CA. There is no strict requirement whether using one CA for each purpose or for both purposes.

Finally we see that only network component is equipped with TLS server certificates, and network users (e.g. Admin, User1) is not. It works fine as most setup only server authentication (one-way authentication) is required in TLS. To know more about TLS settings in Hyperledger Fabric, refer to the fabric documentation about this (link).

Image for post
Image for post

This is how it looks like in Test Network after launching it using CA Server.

Image for post
Image for post
Crypto material generated using Fabric CA Server in Test Network

Fabric-CA Server

We have seen the result. Now we can take a look on the CA Server side, and locate the material inside CA Server.

Our Fabric-CA Server is being brought up as a docker container. The docker compose file for it is ./docker/docker-compose-ca.yaml. It brings up all the three CA servers. We take a look at the one for Org1.

Image for post
Image for post
The Org1 portion of docker-compose-ca.yaml configuration file

Some observations in this configuration.

There is a predefined configuration file fabric-ca-server-config.yaml for Org1, stored in organizations/fabric-ca/org1. From the volumes (line 23–24) it is mapped to the container. Therefore this file exists in the container when fabric-ca-server is running. This is a practical way for managing configuration, as we can edit in localhost while the edited configuration file is directly in the container.

Several environment variables are set in the container. What is defined here is to override the configuration file.

Finally the command to bring up the Fabric CA Server is fabric-ca-server start with a bootstrap identity -b admin:adminpw. It will be used in the next step when we enroll a CA Admin with Fabric-CA Client.

Certain material is kept in /etc/hyperledger/fabric-ca-server when the Fabric-CA Server is running. We can examine the content in localhost, according to the mapping in the container (line 23–24).

Image for post
Image for post
Material inside Fabric-CA Server, being mapped to localhost for inspection.

Among them we will see

  • msp/: where the crypto material for CA of Org1 stored

We first take a look at the CA Certificate ca-cert.pem for Org1.

Image for post
Image for post

This is a self-signed certificate. Again, this is just one option. With Fabric-CA Server we can incorporate an existing certificate (with its private key).

Fabric-CA Server, as a server, supports TLS. Therefore a TLS server certificate is generated and kept as tls-cert.pem.

Image for post
Image for post

And this is just a TLS server certificate issued by CA of Org1.

The corresponding private keys are kept in msp/keystore/.

Image for post
Image for post

It does not take too much effort to correlate the two secret keys matching the CA Certificate for Org1 and TLS server certificate for the Fabric-CA Server.

The Process in Detail

We follow the process executed in network.sh script. Here is the overall flow of generating crypto material using CA Server. For an organization,

  1. Bring up the Fabric-CA Server which is used as the CA of that organization.

Proper configuration is provided whenever needed. And repeat this process for all organizations, as each of them has its own organization CA.

We will take Org1 as an example and take a look on the steps. And we will follow the instruction in network.sh and perform this process once. This helps us understand what happens behind.

Before, clean up what has been deployed with,

cd test-network
./network.sh down
# if you see permission error in removing some directories, perform the following
sudo rm -r organizations/fabric-ca/org1/msp/
sudo rm -r organizations/fabric-ca/org2/msp/
sudo rm -r organizations/fabric-ca/ordererOrg/msp/

Step 1: Fabric-CA Server

IMAGE_TAG=latest docker-compose -f docker/docker-compose-ca.yaml up -d

Check if the CAs are up and running.

Image for post
Image for post

Step 2: Enroll CA Admin

As we are using Fabric-CA Client to interact with the Fabric-CA Server, we first need to specify the Fabric-CA Client home directory.

mkdir -p organizations/peerOrganizations/org1.example.com/export PATH=$PATH:/home/ubuntu/fabric-samples/bin
export FABRIC_CA_CLIENT_HOME=${PWD}/organizations/peerOrganizations/org1.example.com/

Now we can enroll a CA Admin.

fabric-ca-client enroll -u https://admin:adminpw@localhost:7054 --caname ca-org1 --tls.certfiles ${PWD}/organizations/fabric-ca/org1/tls-cert.pem

In this enrollment,

  • bootstrap identity admin:adminpw is specified
Image for post
Image for post

Now our Fabric-CA Client has the credential of a CA Admin, and ready to perform the enrollment for each entity. For information this credential is kept in the home directory (msp/keystore and msp/signcerts).

Image for post
Image for post

And the certificate is issued by the CA of Org1.

Image for post
Image for post

Step 3: Register and Enroll Each Entity

For each entity (peer0, Admin and User), we will first register in the CA Server, and then enroll the entity to obtain the credential.

We need a config.yaml file to enable the OU identifiers, and keep the OU identifiers for each type of entity. They are peer, orderer, client and admin.

NodeOUs:
Enable: true
ClientOUIdentifier:
Certificate: cacerts/localhost-7054-ca-org1.pem
OrganizationalUnitIdentifier: client
PeerOUIdentifier:
Certificate: cacerts/localhost-7054-ca-org1.pem
OrganizationalUnitIdentifier: peer
AdminOUIdentifier:
Certificate: cacerts/localhost-7054-ca-org1.pem
OrganizationalUnitIdentifier: admin
OrdererOUIdentifier:
Certificate: cacerts/localhost-7054-ca-org1.pem
OrganizationalUnitIdentifier: orderer

Store this file under client home directory: organizations/peerOrganizations/org1.example.com/msp/

Here we just take peer0.org1.example.com as an example.

We first register this entity.

fabric-ca-client register --caname ca-org1 --id.name peer0 --id.secret peer0pw --id.type peer --id.attrs '"hf.Registrar.Roles=peer"' --tls.certfiles ${PWD}/organizations/fabric-ca/org1/tls-cert.pem

In this registration, besides the CA Name and TLS certificate of the CA Server, there are several items relevant to the identity of peer0.org1.example.com,

  • id.name: peer0, this is the name of entity
Image for post
Image for post

This effectively puts the information into the Fabric-CA Server database for later enrollment. Nothing is received after registration.

Then we can use the name (id.name) and secret (id.secret) to enrol the peer0.org1.example.com, with this command.

fabric-ca-client enroll -u https://peer0:peer0pw@localhost:7054 --caname ca-org1 -M ${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/msp --csr.hosts peer0.org1.example.com --tls.certfiles ${PWD}/organizations/fabric-ca/org1/tls-cert.pem

Besides the CA Name and TLS certificate of the CA Server, we need to specify

  • CA’s URL with entity’s name and secret, in the format as https://<name>:<secret>@<CA’s URL>
Image for post
Image for post

Now the crypto material for peer0.org1.example.com is stored in organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/msp/, as specified in the enroll command.

Image for post
Image for post

The directory is familiar, as it is the same directory structure we observe before. Here we just show the certificate (inside signcerts/).

Image for post
Image for post

Besides the subject and issuer, we also see some more information about the certificate of peer0.org1.example.com. Take a note on the X509v3 Subject Alternative Name with peer0.org1.example.com, and also the attributes specified in registration and enrollment.

We also need to enrol a TLS server cert.

fabric-ca-client enroll -u https://peer0:peer0pw@localhost:7054 --caname ca-org1 -M ${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls --enrollment.profile tls --csr.hosts peer0.org1.example.com --csr.hosts localhost --tls.certfiles ${PWD}/organizations/fabric-ca/org1/tls-cert.pem

The command is similar to enrol the credential for identity. The only difference is that the enrollment.profile is tls, and CSR hosts.

Image for post
Image for post

The result is stored in organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/, as specified in the enroll command.

Image for post
Image for post

Again we just see the TLS server certificate (in signcerts/).

Image for post
Image for post

Note the X509v3 Extended Key Usage. As we specify enrollment profile as TLS, we see TLS Web Server Authentication and TLS Web Client Authentication here.

This is for peer0.org1.example.com. The process is repeated for Admin and User1. Note that the network.sh script does not enrol the TLS client certificate for them.

Step 4: Move material to the right directory

The directory of identity is kept inside <entity>/msp/, and the structure is good for use in docker-compose files. However the directory of <entity>/tls/ is not the format. Therefore we need to move TLS CA (inside tlscacerts/), server cert (inside signcerts/) and server key (inside keystore/) out and fit them into the <entity>/tls/ with proper names.

Also copy the CA certificates, we can reconstruct the desired directory structure. This is good enough for bringing up containers with docker-compose.

You can read the remaining steps in network.sh and registerEnroll.sh script to complete the whole process.

Summary

In this article, we use Test Network to show the two different ways of generation crypto material for a consortium network. While cryptogen is a fast and easy way with minimum configuration to bring up a workable network, it has certain limitations and therefore is always considered for testing purposes. A more practical way for production is to bring up a CA server and generate the material according to one’s need in a consortium network. In the Test Network, we see how Fabric-CA Server is used as the CA Server, and how to use Fabric-CA Client to perform the whole process.

Happy to share what I learn on blockchain. Visit http://www.ledgertech.biz/kcarticles.html for my works. or reach me on https://www.linkedin.com/in/ktam1/.

Get the Medium app