OCP UPI install on AWS

bootstrap -1x i3.large
master - 3x m4.xlarge
worker - 3x m4.large

Throughout the install process you will have to use the output of `aws cloudformation describe-stacks` to fill the contents of the paramaters files.

Refer to the offical documentation for details

https://access.redhat.com/documentation/en-us/openshift_container_platform/4.1/html-single/installing/index#installing-on-user-provisioned-aws

create install config

mkdir install
openshift-install create install-config --dir=install

edit install config

Edit the install-config.yaml file to set the number of compute replicas to 0

vi ocp/install-config.yaml

compute:
- hyperthreading: Enabled
  name: worker
  platform: {}
  replicas: 0
  ```

#### backup the install-config.yaml
```sh
cp ocp/install-config.yaml .

create manifests

openshift-install create manifests --dir=ocp

remove the files that define the control plane and worker machinesets

rm -f ocp/openshift/99_openshift-cluster-api_master-machines-*.yaml
rm -f ocp/openshift/99_openshift-cluster-api_worker-machineset-*

create the ignition config files

openshift-install create ignition-configs --dir=ocp

get the infraID

I'll refer to the output as INFRAID for the rest of the document

jq -r .infraID ocp/metadata.json

deploy the stack

Edit the stack paramaters to meet your requirements

stack-parm.json

[
  {
    "ParameterKey": "VpcCidr",
    "ParameterValue": "10.0.0.0/16"
  },
  {
    "ParameterKey": "AvailabilityZoneCount",
    "ParameterValue": "1"
  },
  {
    "ParameterKey": "SubnetBits",
    "ParameterValue": "12"
  }
]

Get the stack cloudformations template

curl -O https://gitlab.com/zews79/upi-files/raw/master/stack/stack.yaml

Deploy the stack

aws cloudformation create-stack --stack-name INFRAID-stack  --template-body file://stack.yaml --parameters file://stack-parm.json

Check the status of the stack until is say STACK_COMPLETE

aws cloudformation describe-stacks --stack-name INFRAID

deploy the network

Edit the network paramaters to fit your environment.

net-param.json

[
  {
    "ParameterKey": "ClusterName",
    "ParameterValue": "CLUSTER_NAME"
  },
  {
    "ParameterKey": "InfrastructureName",
    "ParameterValue": "INFRAID"
  },
  {
    "ParameterKey": "HostedZoneId",
    "ParameterValue": "YOUR_HOSTED_ZONE_ID"
  },
  {
    "ParameterKey": "HostedZoneName",
    "ParameterValue": "YOUR_DOMAIN"
  },
  {
    "ParameterKey": "PublicSubnets",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "PrivateSubnets",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "VpcId",
    "ParameterValue": "vpc-<random_string>"
  }
]

Get the cloudformations network template

curl -O https://gitlab.com/zews79/upi-files/raw/master/net/network.yaml

Deploy the network stack

aws cloudformation create-stack --stack-name INFRAID-net --template-body file://network.yaml --parameters file://net-parm.json --capabilities CAPABILITY_NAMED_IAM

Check status until the deploy finishes

aws cloudformation describe-stacks --stack-name INFRAID-net

create security groups and roles

Create and edit the security paramaters file

sec-parm.json

[
  {
    "ParameterKey": "InfrastructureName",
    "ParameterValue": "mycluster-<random_string>"
  },
  {
    "ParameterKey": "VpcCidr",
    "ParameterValue": "10.0.0.0/16"
  },
  {
    "ParameterKey": "PrivateSubnets",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "VpcId",
    "ParameterValue": "vpc-<random_string>"
  }
]

Get the cloudformations security template

curl -O https://gitlab.com/zews79/upi-files/raw/master/sec/security.yaml

Deploy the security stack

aws cloudformation create-stack --stack-name INFRAID-sec --template-body file://security.yaml --parameters file://sec-parm.json --capabilities CAPABILITY_NAMED_IAM

Check the status until it completes

aws cloudformation describe-stacks --stack-name INFRAID-sec

create the bootstrap node

Create an s3 bucket

aws s3 mb s3://INFRAID-infra

Upload the bootstrap ignition file

aws s3 cp ocp/bootstrap.ign s3://INFRAID-infra/bootstrap.ign

Create and edit the bootstrap paramaters files

bootstrap-parm.json

[
  {
    "ParameterKey": "InfrastructureName",
    "ParameterValue": "mycluster-<random_string>"
  },
  {
    "ParameterKey": "RhcosAmi",
    "ParameterValue": "ami-<random_string>"
  },
  {
    "ParameterKey": "AllowedBootstrapSshCidr",
    "ParameterValue": "0.0.0.0/0"
  },
  {
    "ParameterKey": "PublicSubnet",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "MasterSecurityGroupId",
    "ParameterValue": "sg-<random_string>"
  },
  {
    "ParameterKey": "VpcId",
    "ParameterValue": "vpc-<random_string>"
  },
  {
    "ParameterKey": "BootstrapIgnitionLocation",
    "ParameterValue": "s3://<bucket_name>/bootstrap.ign"
  },
  {
    "ParameterKey": "AutoRegisterELB",
    "ParameterValue": "yes"
  },
  {
    "ParameterKey": "RegisterNlbIpTargetsLambdaArn",
    "ParameterValue": "arn:aws:lambda:<region>:<account_number>:function:<dns_stack_name>-RegisterNlbIpTargets-<random_string>"
  },
  {
    "ParameterKey": "ExternalApiTargetGroupArn",
    "ParameterValue": "arn:aws:elasticloadbalancing:<region>:<account_number>:targetgroup/<dns_stack_name>-Exter-<random_string>"
  },
  {
    "ParameterKey": "InternalApiTargetGroupArn",
    "ParameterValue": "arn:aws:elasticloadbalancing:<region>:<account_number>:targetgroup/<dns_stack_name>-Inter-<random_string>"
  },
  {
    "ParameterKey": "InternalServiceTargetGroupArn",
    "ParameterValue": "arn:aws:elasticloadbalancing:<region>:<account_number>:targetgroup/<dns_stack_name>-Inter-<random_string>"
  }
]

Get the bootstrap cloudformations template

curl -O https://gitlab.com/zews79/upi-files/raw/master/bootstrap/bootstrap.yaml

Deploy the bootstrap stacks

aws cloudformation create-stack --stack-name INFRAID-bootstrap --template-body file://bootstrap.yaml --parameters file://bootstrap-parm.json --capabilities CAPABILITY_NAMED_IAM

Check the status until it completes

aws cloudformation describe-stacks --stack-name INFRAID-bootstrap

create the control plane

Create and edit the control plane parameters files

control-plane-parm.json

[
  {
    "ParameterKey": "InfrastructureName",
    "ParameterValue": "mycluster-<random_string>"
  },
  {
    "ParameterKey": "RhcosAmi",
    "ParameterValue": "ami-<random_string>"
  },
  {
    "ParameterKey": "AutoRegisterDNS",
    "ParameterValue": "yes"
  },
  {
    "ParameterKey": "PrivateHostedZoneId",
    "ParameterValue": "<random_string>"
  },
  {
    "ParameterKey": "PrivateHostedZoneName",
    "ParameterValue": "mycluster.example.com"
  },
  {
    "ParameterKey": "Master0Subnet",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "Master1Subnet",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "Master2Subnet",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "MasterSecurityGroupId",
    "ParameterValue": "sg-<random_string>"
  },
  {
    "ParameterKey": "IgnitionLocation",
    "ParameterValue": "https://api-int.<cluster_name>.<domain_name>:22623/config/master"
  },
  {
    "ParameterKey": "CertificateAuthorities",
    "ParameterValue": "data:text/plain;charset=utf-8;base64,ABC...xYz=="
  },
  {
    "ParameterKey": "MasterInstanceProfileName",
    "ParameterValue": "<roles_stack>-MasterInstanceProfile-<random_string>"
  },
  {
    "ParameterKey": "MasterInstanceType",
    "ParameterValue": "m4.xlarge"
  },
  {
    "ParameterKey": "AutoRegisterELB",
    "ParameterValue": "yes"
  },
  {
    "ParameterKey": "RegisterNlbIpTargetsLambdaArn",
    "ParameterValue": "arn:aws:lambda:<region>:<account_number>:function:<dns_stack_name>-RegisterNlbIpTargets-<random_string>"
  },
  {
    "ParameterKey": "ExternalApiTargetGroupArn",
    "ParameterValue": "arn:aws:elasticloadbalancing:<region>:<account_number>:targetgroup/<dns_stack_name>-Exter-<random_string>"
  },
  {
    "ParameterKey": "InternalApiTargetGroupArn",
    "ParameterValue": "arn:aws:elasticloadbalancing:<region>:<account_number>:targetgroup/<dns_stack_name>-Inter-<random_string>"
  },
  {
    "ParameterKey": "InternalServiceTargetGroupArn",
    "ParameterValue": "arn:aws:elasticloadbalancing:<region>:<account_number>:targetgroup/<dns_stack_name>-Inter-<random_string>"
  }
]

Get the control plane cloudformation template

curl -O https://gitlab.com/zews79/upi-files/raw/master/control/control-plane.yaml

Deploy the control plane

aws cloudformation create-stack --stack-name INFRID-control --template-body file://control-plane.yaml --parameters file://control-plane-parm.json

Check the status until it completes

aws cloudformation describe-stacks --stack-name INFRAID-control

Initialize the bootstrap node

Run the openshift installer

openshift-install wait-for bootstrap-complete --dir=<installation_directory> --log-level debug

If the command exits without a FATAL warning, your production control plane has initialized.

Create worker nodes

Create and edit the control plane parameters files

workers-parm.json

[
  {
    "ParameterKey": "InfrastructureName",
    "ParameterValue": "mycluster-<random_string>"
  },
  {
    "ParameterKey": "RhcosAmi",
    "ParameterValue": "ami-<random_string>"
  },
  {
    "ParameterKey": "Subnet",
    "ParameterValue": "subnet-<random_string>"
  },
  {
    "ParameterKey": "WorkerSecurityGroupId",
    "ParameterValue": "sg-<random_string>"
  },
  {
    "ParameterKey": "IgnitionLocation",
    "ParameterValue": "https://api-int.<cluster_name>.<domain_name>:22623/config/worker"
  },
  {
    "ParameterKey": "CertificateAuthorities",
    "ParameterValue": ""
  },
  {
    "ParameterKey": "WorkerInstanceProfileName",
    "ParameterValue": ""
  },
  {
    "ParameterKey": "WorkerInstanceType",
    "ParameterValue": "m4.large"
  }
]

Get the worker cloudformation template

curl -O https://gitlab.com/zews79/upi-files/raw/master/worker/worker.yaml

Deploy the worker nodes

Run this command multiple times for as many worker nodes as you need. You'll need a minimum of 2. Change the stack name each time you run it.

aws cloudformation create-stack --stack-name INFRID-worker?? --template-body file://worker.yaml --parameters file://worker-parm.json

Check the status until it completes

aws cloudformation describe-stacks --stack-name INFAID-worker??

Access your cluster

Via command line

 export KUBECONFIG=<installation_directory>/auth/kubeconfig

 oc get nodes

Via web interface

Get the auto generated kubeadmin password

cat <installation_directory>auth/kubeadmin-password

Get the console route

oc get routes -n openshift-console | grep console-openshift | awk '{print $2}'

Using your web browser navigate to the route and use 'kubeadmin' as the login and the contents of 'kubeadmin-password' as your password.

OCP UPI install on AWS

Throughout the install process you will have to use the output of aws cloudformation describe-stacks to fill the contents of the paramaters files.

Refer to the offical documentation for details

create install config

edit install config

create manifests

remove the files that define the control plane and worker machinesets

create the ignition config files

get the infraID

deploy the stack

deploy the network

create security groups and roles

create the bootstrap node

create the control plane

Initialize the bootstrap node

Create worker nodes

Access your cluster

Throughout the install process you will have to use the output of `aws cloudformation describe-stacks` to fill the contents of the paramaters files.