Offline mirroring tool comparison for Openshift 4!
In the following post, we are going to talk on how to mirror the container base images locally first to an .tar file and later upload them to the offline registry on the Bastion Host.
Prerequisites
- OCPv4.10.X
- Redhat Enterprise Linux/Rocky Linux/Fedora
- Downloaded pull-secret.txt file
- podman package available to the OS
Step 1. Downloading oc-cli and oc-mirror-cli to the Bastion Host:
- Download oc-cli:
- Download oc-mirror-cli:
Note: The oc-mirror
its Technical Preview on stable-4.10
channel and on stable-4.11
channel its General Available.
Step 2. Creating the redhat-operator-index container
By creating the redhat-operator-index container we can query the available operator versions to download on the channel: stable-4.10.
Step 3. Usage of the redhat-operator-index container api
- List a specific operator by filtering on the name:
To download the grpcurl
, you will need to download the binary from grpcurl binary release.
Documenting the offline registry api:
In order to dig more information about an specific api:
Further in this stage we are going to use the following offline registry api’s:
- The full list of operators supported by the redhat-operator-index:
Step 4. Mirroring a specific list of container based images operator:
For the future comparission between the tools, we are going to consider the following list of operators: - local-storage-operator - odf-operator - mcg-operator - metallb-operator - kubernetes-nmstate-operator
- oc-cli mirroring process:
Make sure you are checking the following What to configure for pull-secret file
We are going to split the action of mirroring in two parts:
- Connected Host, where the host can reach the internet
-
Offline Host, where the host doesnt reach the internet BUT has a connection to an SFTP server.
- Connected Host actions:
Once opm-cli is made available to the host, you can proceed with creating the Offline registry on the Connected Host
Exporting the global variables:
Start the container base image prune
Pushing the pruned container based images to the local registry:
Downloading the container base images from the local registry localy:
Creating the archive of the container based images already downloaded localy:
At this state you need to move the .tar.gz file to the Offline Host.
- Offline Host actions:
After transfering the container base images from the Connected Host to the Offline Host, can proceed with creating the Offline registry on the Connected Host
Exporting the global variables:
Decompress the .tar.gz file that contains the mirrored container based images:
Uploading the container based images to the Offline Host local registry:
- oc-mirror-cli process:
Make sure you are checking the following What to configure for pull-secret file
Once the pull-secret file is properly configured, you will need to:
We are going to split the action of mirroring in two parts:
- Connected Host, where the host can reach the internet
-
Offline Host, where the host doesnt reach the internet BUT has a connection to an SFTP server.
- Connected Host actions:
Determine the container based images version:
Creating the imageset-config.yaml file used by oc-mirror:
To check the Image set configuration parameters for more details.
Downloading the container based images to the .tar file:
By setting the archiveSize: 2
, this will create a number of .tar files which limits the size to 2GiB.
At this state you need to move the .tar.gz file to the Offline Host.
- Offline Host actions:
Make sure that the binary oc-mirror is also available on the Offline Host.
Exporting the global variables:
Upload the container base images to the Offline Host registry:
Step 5. Mirroring tools comparison
In the initial phase we are going to consider the mirroring size .tar file for each tool.
- oc-cli mirroring:
Once the mirror has finished, the size of the tar file for the operator list used in the example above:
- oc-mirror-cli mirroring:
In order to describe the content of the mirror_seq1_000000.tar
As a conclusion, we can observe a 3GB difference between the size of the same container base images mirrored locally with the oc-mirror
and oc-cli
. This is a storage optimization of 10.35% in the benefit of the usage of oc-mirror
cli.
Step 6. How to use the container based images to your OCPv.4.10 cluster
Once the container based images are mirrored to the BastionHost offline registry, there is still required to perform a couple of steps until the OCPv4.10 cluster is able to make use of them, therefore in this subchapter we will going to focus on what is required to do and how it differentiates from the oc-mirror
and oc
cli.
- oc-cli upload the container based images :
Checking the content of the BastionHost
Offline registry content after mirroring upload:
Once the mirroring upload has finished, you can use the ICSP (ImageContentSourcePolicy) and CatalogSource files.
An successful mirror upload will terminate with the following message:
The content of the ImageContentSourcePolicy.yaml
:
NOTE: One of the aspect that should be changed for the ImageContentSourcePolicy.yaml
is the source path. The local/index/olm-mirror/redhat-operator-index
should be replace with the registry.redhat.io/redhat/redhat-operator-index
.
The content of the CatalogSource.yaml
:
- oc-mirror-cli upload the container based images :
Checking the content of the BastionHost
Offline registry content after mirror upload:
Once the mirroring upload has finished, you can use the ICSP (ImageContentSourcePolicy) and CatalogSource files.
An successful mirror upload will terminate with the following message:
NOTE: Please, note that the DNS server should resolve inbacrnrdl0100.offline.oxtechnix.lan
and INBACRNRDL0100.offline.oxtechnix.lan
.
The content of the ImageContentSourcePolicy.yaml
:
The content of the CatalogSource.yaml
:
Differences between oc-cli
and oc-mirror-cli
of the container base images upload:
-
For the
oc-cli
container based images all images are uploaded to theolm-mirror
namespace of the offline registry. -
For the
oc-mirror-cli
container based images are uploaded to theolm-mirror
namespace and inside this namespace has been defined other sub-namespacesodf4
,openshift4
,redhat
andrhel8
. -
The
ICSP.yaml
file generated using theoc-mirror-cli
its generating a more dynamic content for which no update is required.
Step 7. How to backtrack the content of .tar file container base images content
In the following subchapter we will try to compare the how to backtrack the image content of the .tar files using the two different cli’s.
- oc-mirror-cli:
Step 8. Automating the process with Ansible
One alternative to automate the mirroring process is to use the following playbook.
Step 9. Preparing the Offline registry for OCP upgrade
In this step we are going to discuss on how to prepare the Offline registry for OCP upgrade. The highlight in this step will be the process on how to use the oc adm catalog mirror
and oc-mirror
cli to prepare the required container base images
for minor or major OCP upgrade.
Step 10. Comparing container base images operators size
In this step we are going to analyze the size of a set container base images operator from different OCP versions.
In the mirroring process we used the following imageset-config.yaml
definition:
- Container Base Images Operators for OCPv4.8:
- Container Base Images Operators for OCPv4.10 if you do not mention a specific version and stream:
- Container Base Images Operators for OCPv4.10 if you mention a specific version and stream:
- Comparrison of the operators mirror .tar file size:
Operator Name | Redhat Operator Index | Size |
---|---|---|
local-storage-operator | v4.8 | 1.3G |
compliance-operator | v4.8 | 1.4G |
file-integrity-operator | v4.8 | 1.3G |
ocs-operator | v4.8 | 51G |
TOTAL .tar file bundle | v4.8 | 55G |
Operator Name | Redhat Operator Index | Size |
---|---|---|
local-storage-operator | v4.10 | 1.8G |
odf-csi-addons-operator | v4.10 | 3.7G |
file-integrity-operator | v4.10 | 1.6G |
ocs-operator | v4.10 | 22G |
mcg-operator | v4.10 | 25G |
odf-operator | v4.10 | 25G |
TOTAL .tar file bundle | v4.10 | 79.1G |
Operator Name | Redhat Operator Index | Operator version | Size |
---|---|---|---|
local-storage-operator | v4.10 | v4.10.4 | |
odf-csi-addons-operator | v4.10 | v4.10.4 | |
file-integrity-operator | v4.10 | v0.1.13 | |
ocs-operator | v4.10 | v4.10.4 | |
mcg-operator | v4.10 | v4.10.4 | |
odf-operator | v4.10 | v4.10.4 | |
TOTAL .tar file bundle | v4.10 | —————— | 26G |
For the OCPv4.10:
When do you run the following command grpcurl -plaintext INBACRNRDL0100.offline.oxtechnix.lan:50051 api.Registry.ListBundles | jq ' .packageName, .channelName, .bundlePath, .version'
you will obtain the full list of bundles shipped by a redhat operator index
for a specific OCP version (in this example its for OCPv4.10). If the administrator is not mentioning the specific version to mirror, then all the versions will be downloaded.
Here its a snip output from the command:
- ocs-operator:
In order to mirror a specific version, you will need to use the oc-mirror
cli and define the imageset-config.yaml
file as below:
In order to mirror all available versions, you will need to use the oc-mirror
cli and define the imageset-config.yaml
file as below:
Below we will compare the size of the bundle of ocs-operator
Operator Name | Channel version | Size |
---|---|---|
ocs-operator | v4.9.0 - v4.10.5 | 51G |
ocs-operator | v4.10.4 | 22G |
- odf-operator:
In order to mirror a specific version from a specific channel mentioned, you will need to use the oc-mirror
cli and define the imageset-config.yaml
file as below:
If you dont mention the parameter full
or full:true
(default value) this will overwite the minVersion
and maxVersion
values, therefore you will download all the odf-operator
images available in the stable-4.10
channel:
In order to mirror all available versions, you will need to use the oc-mirror
cli and define the imageset-config.yaml
file as below:
In order to mirror specific versions, you will need to use the oc-mirror
cli and define the imageset-config.yaml
file as below:
Below we will compare the size of the bundle of odf-operator
Operator Name | Channel version | Size | Note | Size compaction |
---|---|---|---|---|
odf-operator | v4.9.0 - v4.10.5 | 57G | all channel images | - |
odf-operator | v4.9.0 - v4.10.5 | 11G | specific version | 80% |
Operator Name | Channel version | Size | Note | Size compaction |
---|---|---|---|---|
odf-operator | v4.10.4 | 25G | all channel images | - |
odf-operator | v4.10.4 | 6.3G | a specific version | 75% |