Disconnected Installation - Installing | Installation and Configuration

Overview

Frequently, portions of a datacenter may not have access to the Internet, even via proxy servers. Installing OpenShift Enterprise in these environments is considered a disconnected installation.

An OpenShift Enterprise disconnected installation differs from a regular installation in two primary ways:

The OpenShift Enterprise software channels and repositories are not available via Red Hat’s content distribution network.
OpenShift Enterprise uses several containerized components. Normally, these images are pulled directly from Red Hat’s Docker registry. In a disconnected environment, this is not possible.

A disconnected installation ensures the OpenShift Enterprise software is made available to the relevant servers, then follows the same installation process as a standard connected installation. This topic additionally details how to manually download the container images and transport them onto the relevant servers.

Once installed, in order to use OpenShift Enterprise, you will need source code in a source control repository (for example, Git). This topic assumes that an internal Git repository is available that can host source code and this repository is accessible from the OpenShift Enterprise nodes. Installing the source control repository is outside the scope of this document.

Also, when building applications in OpenShift Enterprise, your build may have some external dependencies, such as a Maven Repository or Gem files for Ruby applications. For this reason, and because they might require certain tags, many of the Quickstart templates offered by OpenShift Enterprise may not work on a disconnected environment. However, while Red Hat container images try to reach out to external repositories by default, you can configure OpenShift Enterprise to use your own internal repositories. For the purposes of this document, we assume that such internal repositories already exist and are accessible from the OpenShift Enterprise nodes hosts. Installing such repositories is outside the scope of this document.

You can also have a Red Hat Satellite server that provides access to Red Hat content via an intranet or LAN. For environments with Satellite, you can synchronize the OpenShift Enterprise software onto the Satellite for use with the OpenShift Enterprise servers.

Red Hat Satellite 6.1 also introduces the ability to act as a Docker registry, and it can be used to host the OpenShift Enterprise containerized components. Doing so is outside of the scope of this document.

Prerequisites

This document assumes that you understand OpenShift Enterprise’s overall architecture and that you have already planned out what the topology of your environment will look like.

Required Software and Components

In order to pull down the required software repositories and container images, you will need a Red Hat Enterprise Linux (RHEL) 7 server with access to the Internet and at least 100GB of additional free space. All steps in this section should be performed on the Internet-connected server as the root system user.

Syncing Repositories

Before you sync with the required repositories, you may need to import the appropriate GPG key:

# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

If the key is not imported, the indicated package is deleted after syncing the repository.

To sync the required repositories:

Register the server with the Red Hat Customer Portal. You must use the login and password associated with the account that has access to the OpenShift Enterprise subscriptions:
```
# subscription-manager register
```

Attach to a subscription that provides OpenShift Enterprise channels. You can find the list of available subscriptions using:

# subscription-manager list --available

Then, find the pool ID for the subscription that provides OpenShift Enterprise, and attach it:

# subscription-manager attach --pool=<pool_id>
# subscription-manager repos --disable="*"
# subscription-manager repos \
    --enable="rhel-7-server-rpms" \
    --enable="rhel-7-server-extras-rpms" \
    --enable="rhel-7-server-ose-3.2-rpms"

The yum-utils command provides the reposync utility, which lets you mirror yum repositories, and createrepo can create a usable yum repository from a directory:
```
# yum -y install yum-utils createrepo docker git
```
You will need up to 110GB of free space in order to sync the software. Depending on how restrictive your organization’s policies are, you could re-connect this server to the disconnected LAN and use it as the repository server. You could use USB-connected storage and transport the software to another server that will act as the repository server. This topic covers these options.
Make a path to where you want to sync the software (either locally or on your USB or other device):
```
# mkdir -p </path/to/repos>
```

Sync the packages and create the repository for each of them. You will need to modify the command for the appropriate path you created above:

# for repo in \
rhel-7-server-rpms rhel-7-server-extras-rpms \
rhel-7-server-ose-3.2-rpms
do
  reposync --gpgcheck -lm --repoid=${repo} --download_path=/path/to/repos
  createrepo -v </path/to/repos/>${repo} -o </path/to/repos/>${repo}
done

Syncing Images

To sync the container images:

Start the Docker daemon:
```
# systemctl start docker
```

Pull all of the required OpenShift Enterprise containerized components:

# docker pull registry.access.redhat.com/openshift3/ose-haproxy-router:v3.2.1.31
# docker pull registry.access.redhat.com/openshift3/ose-deployer:v3.2.1.31
# docker pull registry.access.redhat.com/openshift3/ose-sti-builder:v3.2.1.31
# docker pull registry.access.redhat.com/openshift3/ose-docker-builder:v3.2.1.31
# docker pull registry.access.redhat.com/openshift3/ose-pod:v3.2.1.31
# docker pull registry.access.redhat.com/openshift3/ose-docker-registry:v3.2.1.31

Pull all of the required OpenShift Enterprise containerized components for the additional centralized log aggregation and metrics aggregation components:

# docker pull registry.access.redhat.com/openshift3/logging-deployment:3.2.1
# docker pull registry.access.redhat.com/openshift3/logging-elasticsearch:3.2.1
# docker pull registry.access.redhat.com/openshift3/logging-kibana:3.2.1
# docker pull registry.access.redhat.com/openshift3/logging-fluentd:3.2.1
# docker pull registry.access.redhat.com/openshift3/logging-auth-proxy:3.2.1
# docker pull registry.access.redhat.com/openshift3/metrics-deployer:3.2.1
# docker pull registry.access.redhat.com/openshift3/metrics-hawkular-metrics:3.2.1
# docker pull registry.access.redhat.com/openshift3/metrics-cassandra:3.2.1
# docker pull registry.access.redhat.com/openshift3/metrics-heapster:3.2.1

Pull the Red Hat-certified Source-to-Image (s2i) builder images that you intend to use in your OpenShift environment. You can pull the following images:
- jboss-eap70-openshift
- jboss-amq-62
- jboss-datagrid65-openshift
- jboss-decisionserver62-openshift
- jboss-eap64-openshift
- jboss-eap70-openshift
- jboss-webserver30-tomcat7-openshift
- jboss-webserver30-tomcat8-openshift
- mongodb
- mysql
- nodejs
- perl
- php
- postgresql
- python
- redhat-sso70-openshift
- ruby
  
  Make sure to indicate the correct tag specifying the desired version number. For example, to pull both the previous and latest version of the Tomcat image:
  # docker pull \ registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift:latest # docker pull \ registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift:1.1

Preparing Images for Export

Container images can be exported from a system by first saving them to a tarball and then transporting them:

Make and change into a repository home directory:

# mkdir </path/to/repos/images>
# cd </path/to/repos/images>

Export the OpenShift Enterprise containerized components:

# docker save -o ose3-images.tar \
    registry.access.redhat.com/openshift3/ose-haproxy-router \
    registry.access.redhat.com/openshift3/ose-deployer \
    registry.access.redhat.com/openshift3/ose-sti-builder \
    registry.access.redhat.com/openshift3/ose-docker-builder \
    registry.access.redhat.com/openshift3/ose-pod \
    registry.access.redhat.com/openshift3/ose-docker-registry

If you synchronized the metrics and log aggregation images, export:

# docker save -o ose3-logging-metrics-images.tar \
    registry.access.redhat.com/openshift3/logging-deployment \
    registry.access.redhat.com/openshift3/logging-elasticsearch \
    registry.access.redhat.com/openshift3/logging-kibana \
    registry.access.redhat.com/openshift3/logging-fluentd \
    registry.access.redhat.com/openshift3/logging-auth-proxy \
    registry.access.redhat.com/openshift3/metrics-deployer \
    registry.access.redhat.com/openshift3/metrics-hawkular-metrics \
    registry.access.redhat.com/openshift3/metrics-cassandra \
    registry.access.redhat.com/openshift3/metrics-heapster

Export the s2i builder images that you synced in the previous section. For example, if you synced only the Tomcat image:

# docker save -o ose3-builder-images.tar \
    registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift:latest \
    registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift:1.1

Repository Server

During the installation (and for later updates, should you so choose), you will need a webserver to host the repositories. RHEL 7 can provide the Apache webserver.

Option 1: Re-configuring as a Web server

If you can re-connect the server where you synchronized the software and images to your LAN, then you can simply install Apache on the server:

# yum install httpd

Skip to Placing the Software.

Option 2: Building a Repository Server

If you need to build a separate server to act as the repository server, install a new RHEL 7 system with at least 110GB of space. On this repository server during the installation, make sure you select the Basic Web Server option.

Placing the Software

If necessary, attach the external storage, and then copy the repository files into Apache’s root folder. Note that the below copy step (cp -a) should be substituted with move (mv) if you are repurposing the server you used to sync:
```
# cp -a /path/to/repos /var/www/html/
# chmod -R +r /var/www/html/repos
# restorecon -vR /var/www/html
```

Add the firewall rules:

# firewall-cmd --permanent --add-service=http
# firewall-cmd --reload

Enable and start Apache for the changes to take effect:
```
# systemctl enable httpd
# systemctl start httpd
```

OpenShift Enterprise Systems

Building Your Hosts

At this point you can perform the initial creation of the hosts that will be part of the OpenShift Enterprise environment. It is recommended to use the latest version of RHEL 7 and to perform a minimal installation. You will also want to pay attention to the other OpenShift Enterprise-specific prerequisites.

Once the hosts are initially built, the repositories can be set up.

Connecting the Repositories

On all of the relevant systems that will need OpenShift Enterprise software components, create the required repository definitions. Place the following text in the /etc/yum.repos.d/ose.repo file, replacing <server_IP> with the IP or host name of the Apache server hosting the software repositories:

[rhel-7-server-rpms]
name=rhel-7-server-rpms
baseurl=http://<server_IP>/repos/rhel-7-server-rpms
enabled=1
gpgcheck=0
[rhel-7-server-extras-rpms]
name=rhel-7-server-extras-rpms
baseurl=http://<server_IP>/repos/rhel-7-server-extras-rpms
enabled=1
gpgcheck=0
[rhel-7-server-ose-3.2-rpms]
name=rhel-7-server-ose-3.2-rpms
baseurl=http://<server_IP>/repos/rhel-7-server-ose-3.2-rpms
enabled=1
gpgcheck=0

Host Preparation

At this point, the systems are ready to continue to be prepared following the OpenShift Enterprise documentation.

Skip the section titled Registering the Hosts and start with Managing Packages.

Installing OpenShift Enterprise

Importing OpenShift Enterprise Containerized Components

To import the relevant components, securely copy the images from the connected host to the individual OpenShift Enterprise hosts:

# scp /var/www/html/repos/images/ose3-images.tar root@<openshift_host_name>:
# ssh root@<openshift_host_name> "docker load -i ose3-images.tar"

If you prefer, you could use wget on each OpenShift Enterprise host to fetch the tar file, and then perform the Docker import command locally. Perform the same steps for the metrics and logging images, if you synchronized them.

On the host that will act as an OpenShift Enterprise master, copy and import the builder images:

# scp /var/www/html/images/ose3-builder-images.tar root@<openshift_master_host_name>:
# ssh root@<openshift_master_host_name> "docker load -i ose3-builder-images.tar"

Running the OpenShift Enterprise Installer

You can now choose to follow the quick or advanced OpenShift Enterprise installation instructions in the documentation.

Creating the Internal Docker Registry

You now need to create the internal Docker registry.

Post-Installation Changes

In one of the previous steps, the s2i images were imported into the Docker daemon running on one of the OpenShift Enterprise master hosts. In a connected installation, these images would be pulled from Red Hat’s registry on demand. Since the Internet is not available to do this, the images must be made available in another Docker registry.

OpenShift Enterprise provides an internal registry for storing the images that are built as a result of the s2i process, but it can also be used to hold the s2i builder images. The following steps assume you did not customize the service IP subnet (172.30.0.0/16) or the Docker registry port (5000).

Re-tagging s2i Builder Images

On the master host where you imported the s2i builder images, obtain the service address of your Docker registry that you installed on the master:
```
# export REGISTRY=$(oc get service docker-registry -t '{{.spec.clusterIP}}{{"\n"}}')
```

Next, tag all of the builder images that you synced and exported before pushing them into the OpenShift Enterprise Docker registry. For example, if you synced and exported only the Tomcat image:

# docker tag \
registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift:1.1 \
$REGISTRY:5000/openshift/webserver30-tomcat7-openshift:1.1
# docker tag \
registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift:latest \
$REGISTRY:5000/openshift/webserver30-tomcat7-openshift:1.2
# docker tag \
registry.access.redhat.com/jboss-webserver-3/webserver30-tomcat7-openshift:latest \
$REGISTRY:5000/openshift/webserver30-tomcat7-openshift:latest

Creating an Administrative User

Pushing the container images into OpenShift Enterprise’s Docker registry requires a user with cluster-admin privileges. Because the default OpenShift Enterprise system administrator does not have a standard authorization token, they cannot be used to log in to the Docker registry.

To create an administrative user:

Create a new user account in the authentication system you are using with OpenShift Enterprise. For example, if you are using local htpasswd-based authentication:
```
# htpasswd -b /etc/openshift/openshift-passwd <admin_username> <password>
```
The external authentication system now has a user account, but a user must log in to OpenShift Enterprise before an account is created in the internal database. Log in to OpenShift Enterprise for this account to be created. This assumes you are using the self-signed certificates generated by OpenShift Enterprise during the installation:
```
# oc login --certificate-authority=/etc/origin/master/ca.crt \
    -u <admin_username> https://<openshift_master_host>:8443
```

Get the user’s authentication token:

# MYTOKEN=$(oc whoami -t)
# echo $MYTOKEN
iwo7hc4XilD2KOLL4V1O55ExH2VlPmLD-W2-JOd6Fko

Modifying the Security Policies

Using oc login switches to the new user. Switch back to the OpenShift Enterprise system administrator in order to make policy changes:
```
# oc login -u system:admin
```
In order to push images into the OpenShift Enterprise Docker registry, an account must have the image-builder security role. Add this to your OpenShift Enterprise administrative user:
```
# oadm policy add-role-to-user system:image-builder <admin_username>
```
Next, add the administrative role to the user in the openshift project. This allows the administrative user to edit the openshift project, and, in this case, push the container images:
```
# oadm policy add-role-to-user admin <admin_username> -n openshift
```

Editing the Image Stream Definitions

The openshift project is where all of the image streams for builder images are created by the installer. They are loaded by the installer from the /usr/share/openshift/examples directory. Change all of the definitions by deleting the image streams which had been loaded into OpenShift Enterprise’s database, then re-create them:

Delete the existing image streams:
```
# oc delete is -n openshift --all
```
Make a backup of the files in /usr/share/openshift/examples/ if you desire. Next, edit the file image-streams-rhel7.json in the /usr/share/openshift/examples/image-streams folder. You will find an image stream section for each of the builder images. Edit the spec stanza to point to your internal Docker registry.

For example, change:
"spec": { "dockerImageRepository": "registry.access.redhat.com/rhscl/mongodb-26-rhel7",
to:
"spec": { "dockerImageRepository": "172.30.69.44:5000/openshift/mongodb-26-rhel7",
In the above, the repository name was changed from rhscl to openshift. You will need to ensure the change, regardless of whether the repository is rhscl, openshift3, or another directory. Every definition should have the following format:
```
<registry_ip>:5000/openshift/<image_name>
```
Repeat this change for every image stream in the file. Ensure you use the correct IP address that you determined earlier. When you are finished, save and exit. Repeat the same process for the JBoss image streams in the /usr/share/openshift/examples/xpaas-streams/jboss-image-streams.json file.

Load the updated image stream definitions:

# oc create -f /usr/share/openshift/examples/image-streams/image-streams-rhel7.json -n openshift
# oc create -f /usr/share/openshift/examples/xpaas-streams/jboss-image-streams.json -n openshift

Loading the Container Images

At this point the system is ready to load the container images.

# docker login -u adminuser -e mailto:adminuser@abc.com \
   -p $MYTOKEN $REGISTRY:5000

Push the Docker images:

# docker push $REGISTRY:5000/openshift/webserver30-tomcat7-openshift:1.1
# docker push $REGISTRY:5000/openshift/webserver30-tomcat7-openshift:1.2
# docker push $REGISTRY:5000/openshift/webserver30-tomcat7-openshift:latest

Verify that all the image streams now have the tags populated:

# oc get imagestreams -n openshift
NAME                                 DOCKER REPO                                                      TAGS                                     UPDATED
jboss-webserver30-tomcat7-openshift  $REGISTRY/jboss-webserver-3/webserver30-jboss-tomcat7-openshift  1.1,1.1-2,1.1-6 + 2 more...              2 weeks ago
...

Installing a Router

At this point, the OpenShift Enterprise environment is almost ready for use. It is likely that you will want to install and configure a router.