Install delegates with custom certificates
This topic explains how to install Kubernetes, Docker, and Helm delegates with custom certificates.
The installation steps are different depending on your delegate version.
If your delegate with an immutable image type version is later than 81202 (image tag 23.10.81202), go to Install with custom certificates.
If your delegate with an immutable image type version is earlier than 81202 (image tag 23.10.81202), go to Install with custom truststore.
For information on delegate types, go to Delegate image types.
Install with custom certificates
Use the steps below to install custom certificates for a Docker, Kubernetes, or Helm delegate with an immutable image type version later than 23.10.81202.
Certificates must be PEM format.
- Docker delegate
- Kubernetes delegate
- Helm delegate
To install a Docker delegate with custom certificates, do the following:
- 
Prepare the custom cert file(s). 
- 
Mount the file(s) to the /opt/harness-delegate/ca-bundle/directory inside the delegate container.
- 
Start the delegate with the root user. Example: Mount custom certs from a folder docker run --cpus=1 -u root --memory=2g \
 -v PUT_YOUR_PATH_TO_FOLDER_OF_CUSTOM_CERTS:/opt/harness-delegate/ca-bundle \
 -e DELEGATE_NAME=PUT_YOUR_DELEGATE_NAME \
 -e NEXT_GEN="true" \
 -e DELEGATE_TYPE="DOCKER" \
 -e ACCOUNT_ID=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
 -e DELEGATE_TOKEN=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
 -e MANAGER_HOST_AND_PORT=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE harness/delegate:yy.mm.vernoExample: Mount a single custom cert or a CA bundle file docker run --cpus=1 -u root --memory=2g \
 -v PUT_YOUR_PATH_TO_CUSTOM_CERT:/opt/harness-delegate/ca-bundle/abc.pem \
 -e DELEGATE_NAME=PUT_YOUR_DELEGATE_NAME \
 -e NEXT_GEN="true" \
 -e DELEGATE_TYPE="DOCKER" \
 -e ACCOUNT_ID=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
 -e DELEGATE_TOKEN=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
 -e MANAGER_HOST_AND_PORT=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE harness/delegate:yy.mm.verno
To install a Kubernetes delegate with custom certificates, do the following:
- 
Create a Kubernetes secret with the custom cert file. kubectl create secret -n <YOUR_NAMESPACE> generic <YOUR_SECRET_NAME> --from-file custom-cert1=<certificate file name>noteYou can install multiple certificates by adding additional --from-filearguments. For example:kubectl create secret -n <YOUR_NAMESPACE> generic <YOUR_SECRET_NAME> \
 --from-file custom-cert1=site1cert.pem \
 --from-file custom-cert2=site2cert.pem \
 --from-file custom-cert3=site3cert.pem
- 
Modify the delegate manifest file to include a volume mount. - Add the following YAML under spec.template.spec.containers.
 volumeMounts:
 - mountPath: /opt/harness-delegate/ca-bundle/
 name: custom-certs
 readOnly: true- Add the following YAML under spec.template.spec. Replace<YOUR_SECRET_NAME>with the value you used when you created the secret in step 1.
 volumes:
 - name: custom-certs
 secret:
 secretName: <secret-name>
 defaultMode: 400
- Add the following YAML under 
- 
Set the security context to provide operator access to the mounted files. Add the following YAML under spec.template.spec.securityContext:
 fsGroup: 1001
- 
Use the root user. This is the default and might not require modification. Add the following YAML under spec.template.spec.containers.securityContext:
 allowPrivilegeEscalation: false
 runAsUser: 0
Kubernetes delegate with custom certificates YAML example
apiVersion: apps/v1
kind: Deployment
metadata:
   labels:
      harness.io/name: kubernetes-delegate
   name: kubernetes-delegate
   namespace: harness-delegate-ng
spec:
   replicas: 1
   minReadySeconds: 120
   selector:
      matchLabels:
         harness.io/name: kubernetes-delegate
   template:
      metadata:
         labels:
            harness.io/name: kubernetes-delegate
         annotations:
            prometheus.io/scrape: "true"
            prometheus.io/port: "3460"
            prometheus.io/path: "/api/metrics"
      spec:
         terminationGracePeriodSeconds: 600
         restartPolicy: Always
         securityContext:
            fsGroup: 1001
         containers:
            - image: harness/delegate:yy.mm.verno
              imagePullPolicy: Always
              name: delegate
              securityContext:
                 allowPrivilegeEscalation: false
                 runAsUser: 0
              ports:
                 - containerPort: 8080
              resources:
                 limits:
                    memory: "2048Mi"
                 requests:
                    cpu: "0.5"
                    memory: "2048Mi"
              livenessProbe:
                 httpGet:
                    path: /api/health
                    port: 3460
                    scheme: HTTP
                 initialDelaySeconds: 10
                 periodSeconds: 10
                 failureThreshold: 3
              startupProbe:
                 httpGet:
                    path: /api/health
                    port: 3460
                    scheme: HTTP
                 initialDelaySeconds: 30
                 periodSeconds: 10
                 failureThreshold: 15
              envFrom:
                 - secretRef:
                      name: kubernetes-delegate-account-token
              env:
                 - name: JAVA_OPTS
                   value: "-Xms64M"
                 - name: ACCOUNT_ID
                   value: PUT_YOUR_HARNESS_ACCOUNTID_HERE
                 - name: MANAGER_HOST_AND_PORT
                   value: PUT_YOUR_MANAGER_HOST_AND_PORT_HERE
                 - name: DELEGATE_NAME
                   value: kubernetes-delegate
                 - name: DELEGATE_TYPE
                   value: "KUBERNETES"
                 - name: DELEGATE_NAMESPACE
                   valueFrom:
                      fieldRef:
                         fieldPath: metadata.namespace
                 - name: INIT_SCRIPT
                   value: ""
                 - name: DELEGATE_DESCRIPTION
                   value: ""
                 - name: DELEGATE_TAGS
                   value: ""
                 - name: NEXT_GEN
                   value: "true"
              volumeMounts:
                 - mountPath: /opt/harness-delegate/ca-bundle/
                   name: custom-certs
                   readOnly: true
         volumes:
            - name: custom-certs
              secret:
                 secretName: mycerts
                 defaultMode: 400
Add self-signed certificates to delegate upgrader
For Kubernetes delegates, Harness supports self-signed certificates for delegate upgrader. For more information on delegate upgrades, go to Delegate automatic upgrades and expiration policy.
To add self-signed certificates for delegate upgrader, do the following:
- 
In the delegate YAML file, mount the certificates in /ca-bundle.
- 
Add the securityContextto the upgrader cron job.apiVersion: batch/v1
 kind: CronJob
 metadata:
 labels:
 harness.io/name: kubernetes-delegate-upgrader-job
 name: kubernetes-delegate-upgrader-job
 namespace: harness-delegate-ng
 spec:
 schedule: "0 */1 * * *"
 concurrencyPolicy: Forbid
 startingDeadlineSeconds: 20
 jobTemplate:
 spec:
 template:
 spec:
 serviceAccountName: upgrader-cronjob-sa
 restartPolicy: Never
 securityContext:
 fsGroup: 1001
 containers:
 - image: harness/upgrader:latest
 name: upgrader
 imagePullPolicy: Always
 envFrom:
 - secretRef:
 name: kubernetes-delegate-upgrader-token
 volumeMounts:
 - mountPath: /ca-bundle
 name: custom-certs
 readOnly: true
 volumes:
 - name: custom-certs
 secret:
 secretName: new-secret
 defaultMode: 400
- 
Create a Kubernetes secret with the custom cert file. kubectl create secret -n <YOUR_NAMESPACE> generic <YOUR_SECRET_NAME> --from-file custom-cert1=<certificate file name>noteYou can install multiple certificates by adding additional --from-filearguments. For example:kubectl create secret -n <YOUR_NAMESPACE> generic <YOUR_SECRET_NAME> \
 --from-file custom-cert1=site1cert.pem \
 --from-file custom-cert2=site2cert.pem \
 --from-file custom-cert3=site3cert.pem
- 
Run the following to set the delegateCustomCa.secretNamevariable when you install the Helm chart.--set delegateCustomCa.secretName=<SECRET_NAME>This adds your volume mount to the /opt/harness-delegate/ca-bundle/directory.
Add self-signed certificates to delegate upgrader
For Helm delegates, Harness supports self-signed certificates for delegate upgrader. For more information on delegate upgrades, go to Delegate automatic upgrades and expiration policy.
To add self-signed certificates for delegate upgrader, do the following:
- 
Create a Kubernetes secret with the custom cert file. kubectl create secret -n <namespace> generic <secret-name> --from-file custom-cert1=<certificate file name>
- 
Run the following to set the upgraderCustomCa.secretNamevariable when you install the Helm chart.--set upgraderCustomCa.secretName=<SECRET_NAME>This adds your volume mount to the /ca-bundledirectory.
Install with custom truststore
Harness Delegate ships with a Java Runtime Environment (JRE) that includes a default trusted certificate in its truststore located in the /opt/java/openjdk/lib/security/cacerts directory. This truststore uses multiple trusted certificates. You can limit the number you use based on your company's security protocols.
The JRE truststore must include the certificate that delegates require to establish trust with Harness (app.harness.io).
Command-line tools use truststore from the underlying Red Hat operating system.
Use the steps below to install custom certificates for a Docker or Kubernetes delegate with an immutable image type version earlier than 23.10.81202.
There are two aspects of custom certificates:
- 
A certificate for the delegate Java process, which makes connections to external systems. 
- 
A certificate for the OS itself. With this certificate, if another process, such as a shell script, is spawned, it can access custom certificates. 
In this topic, we will do the following:
- Create a custom truststore.
- Create a secret.
- Add a volume mount to the harness-delegate.yamlfile and provide it to the delegate Java process.
- Add a volume mount to the harness-delegate.yamlfile and configure the delegate container OS to have the certificates.
Harness recommends that you keep your existing Java KeyStore in place during the installation process. Updating the KeyStore might cause issues with your delegate.
For information on best practices for truststore creation, go to Java Keystore Best Practices.
Create a custom truststore
- 
Prepare the custom cert file(s). noteCertificates must be PEM format. 
- 
(Optional) Get a base truststore file from a running delegate instance. Kubernetes delegate kubectl cp -n harness-delegate-ng [pod name]:/opt/java/openjdk/lib/security/cacerts Path/to/destinationDocker delegate docker cp [container id]:/opt/java/openjdk/lib/security/cacerts Path/to/destination
- 
Import custom certs into the Java truststore. a. Split the certificates into individual files if the custom cert file contains multiple certificates. b. Run the keytool command below for each certificate file to import them. keytool -noprompt -import -trustcacerts -file [cer file] -alias [unique alias] -keystore [path to trust store] -storepass [password]c. Replace the password placeholder with the password you gave your truststore. d. Use a unique alias for all imports. 
Install truststore and custom certs
After you configure the truststore file and custom certificates, you're ready to install them in a Kubernetes or Docker delegate.
- Docker delegate
- Kubernetes delegate
- 
Mount the truststore file to the delegate container. 
- 
Mount the custom certificates to the /etc/pki/ca-trust/source/anchors/directory.
- 
Run the delegate container with the root user. 
- 
Add update-ca-trusttoINIT_SCRIPT.Example command docker run --cpus=1 --memory=2g -u root \
 -v PUT_YOUR_PATH_TO_CUSTOM_CERT:/etc/pki/ca-trust/source/anchors/ca1.pem \
 -v ... repeat for every custom cert ... \
 -v PUT_YOUR_PATH_TO_TRUSTSTORE:/cacerts/harness_trustStore.jks \
 -e JAVA_OPTS="... -Djavax.net.ssl.trustStore=/cacerts/harness_trustStore.jks -Djavax.net.ssl.trustStorePassword=password" \
 -e INIT_SCRIPT="update-ca-trust" \
 -e DELEGATE_NAME=PUT_YOUR_DELEGATE_NAME \
 -e NEXT_GEN="true" \
 -e DELEGATE_TYPE="DOCKER" \
 -e ACCOUNT_ID=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
 -e DELEGATE_TOKEN=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
 -e MANAGER_HOST_AND_PORT=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE harness/delegate:yy.mm.verno
- 
Use your custom truststore to create a secret. kubectl create secret -n harness-delegate-ng generic mysecret --from-file harness_trustStore.jks=harness_trustStore.jks
- 
Modify the delegate manifest file to include a volume mount. - 
Add the following YAML under spec.template.spec.containers.volumeMounts:
 - mountPath: /cacerts
 name: custom-truststore
 readOnly: true
- 
Add the following YAML under spec.template.spec. Replace<YOUR_SECRET_NAME>with the value you used when you created the secret in step 1.volumes:
 - name: custom-truststore
 secret:
 secretName: <secret-name>
 defaultMode: 400
 
- 
- 
Set the security context to provide operator access to the mounted files. Add the following YAML under spec.template.spec.securityContext:
 fsGroup: 1001
- 
Use the root user. This is the default and might not require modifications. Add the following YAML under spec.template.spec.containers.securityContext:
 allowPrivilegeEscalation: false
 runAsUser: 0
- 
Update the JAVA_OPTSenvironment variable with information about your custom truststore. Replace the password placeholder with the password you used in your truststore.- name: JAVA_OPTS
 value: "... -Djavax.net.ssl.trustStore=/cacerts/harness_trustStore.jks -Djavax.net.ssl.trustStorePassword=YOUR_PASSWORD"noteYou can omit the specification of the JAVA_OPTSenvironment variable if you mount the secret to the same location as the default truststore and give it the same name. The JVM then applies the change automatically.
Add custom certificates to the delegate pod
You can add certificates to the delegate pod so any command running on the pod has certificates installed.
This step isn't necessary if you don't intend to run commands directly on the pod that needs certificates to connect to external systems.
In this example, we'll use cert1.crt and cert2.crt files that have custom certificates.
- 
Mount the certificates to the delegate pod in the /etc/pki/ca-trust/source/anchors/directory.volumeMounts:
 - name: certs
 mountPath : "/etc/pki/ca-trust/source/anchors/cert1.crt"
 subPath: cert1.crt
 - name: certs
 mountPath : "/etc/pki/ca-trust/source/anchors/cert2.crt"
 subPath: cert2.crt
- 
Run update-ca-trustusingINIT_SCRIPT.- name: INIT_SCRIPT
 value: |-
 update-ca-trustnoteThe delegate must be the root user. securityContext:
 allowPrivilegeEscalation: false
 runAsUser: 0
Kubernetes delegate with truststore YAML example
The following example harness-delegate.yaml file includes the changes required to install a delegate with a custom certificate.
apiVersion: v1
kind: Namespace
metadata:
  name: harness-delegate-ng
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: harness-delegate-ng-cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: harness-delegate-ng
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: Secret
metadata:
  name: my-secret-account-token
  namespace: harness-delegate-ng
type: Opaque
data:
  ACCOUNT_SECRET: "XXXXXXXXXXXXXXXXXXXXXXXX"
---
# To learn how to proxy a delegate, go to [Configure delegate proxy settings](/docs/platform/delegates/manage-delegates/configure-delegate-proxy-settings/)
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    harness.io/name: my-secret
  name: my-secret
  namespace: harness-delegate-ng
spec:
  replicas: 1
  selector:
    matchLabels:
      harness.io/name: my-secret
  template:
    metadata:
      labels:
        harness.io/name: my-secret
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "3460"
        prometheus.io/path: "/api/metrics"
    spec:
      terminationGracePeriodSeconds: 600
      restartPolicy: Always
      securityContext:
        allowPrivilegeEscalation: false
        runAsUser: 0
      containers:
      - image: harness/delegate-immutable:22.07.75836.minimal
        imagePullPolicy: Always
        name: delegate
        ports:
          - containerPort: 8080
        resources:
          limits:
            cpu: "0.5"
            memory: "2048Mi"
          requests:
            cpu: "0.5"
            memory: "2048Mi"
        livenessProbe:
          httpGet:
            path: /api/health
            port: 3460
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          failureThreshold: 2
        startupProbe:
          httpGet:
            path: /api/health
            port: 3460
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 10
          failureThreshold: 15
        envFrom:
        - secretRef:
            name: my-secret-account-token
        env:
        - name: JAVA_OPTS
          value: "-Xms64M -Djavax.net.ssl.trustStore=/cacerts/harness_trustStore.jks -Djavax.net.ssl.trustStorePassword=mypassword"
        - name: ACCOUNT_ID
          value: XXXXXxxxxxx
        - name: MANAGER_HOST_AND_PORT
          value: https://qa.harness.io/gratis
        - name: DEPLOY_MODE
          value: KUBERNETES
        - name: DELEGATE_NAME
          value: my-secret
        - name: DELEGATE_TYPE
          value: "KUBERNETES"
        - name: DELEGATE_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INIT_SCRIPT
          value: |-
            update-ca-trust
        - name: DELEGATE_DESCRIPTION
          value: ""
        - name: DELEGATE_TAGS
          value: ""
        - name: NEXT_GEN
          value: "true"
        - name: CLIENT_TOOLS_DOWNLOAD_DISABLED
          value: "true"
        volumeMounts:
        - mountPath: /cacerts
          name: custom-keystore
          readOnly: true
        - name: certs
          mountPath : "/usr/local/share/ca-certificates/cert1.crt"
          subPath: cert1.crt
        - name: certs
          mountPath : "/usr/local/share/ca-certificates/cert2.crt"
          subPath: cert2.crt
      volumes:
      - name: custom-keystore
        secret:
          secretName: myjks
          defaultMode: 400
---
apiVersion: v1
kind: Service
metadata:
  name: delegate-service
  namespace: harness-delegate-ng
spec:
  type: ClusterIP
  selector:
    harness.io/name: my-secret
  ports:
    - port: 8080
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: upgrader-cronjob
  namespace: harness-delegate-ng
rules:
  - apiGroups: ["batch", "apps", "extensions"]
    resources: ["cronjobs"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["extensions", "apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: my-secret-upgrader-cronjob
  namespace: harness-delegate-ng
subjects:
  - kind: ServiceAccount
    name: upgrader-cronjob-sa
    namespace: harness-delegate-ng
roleRef:
  kind: Role
  name: upgrader-cronjob
  apiGroup: ""
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: upgrader-cronjob-sa
  namespace: harness-delegate-ng
---
apiVersion: v1
kind: Secret
metadata:
  name: my-secret-upgrader-token
  namespace: harness-delegate-ng
type: Opaque
data:
  UPGRADER_TOKEN: "XXXXXXXXXXXXXXXXXXXXXXXX"
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-secret-upgrader-config
  namespace: harness-delegate-ng
data:
  config.yaml: |
    mode: Delegate
    dryRun: false
    workloadName: my-secret
    namespace: harness-delegate-ng
    containerName: delegate
    delegateConfig:
      accountId: XXXXXXXXXXXXXXXXXXXXXXXX
      managerHost: https://qa.harness.io/gratis
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  labels:
    harness.io/name: my-secret-upgrader-job
  name: my-secret-upgrader-job
  namespace: harness-delegate-ng
spec:
  schedule: "0 */1 * * *"
  concurrencyPolicy: Forbid
  startingDeadlineSeconds: 20
  jobTemplate:
    spec:
      suspend: true
      template:
        spec:
          serviceAccountName: upgrader-cronjob-sa
          restartPolicy: Never
          containers:
          - image: us.gcr.io/qa-target/upgrader:1.0.0
            name: upgrader
            imagePullPolicy: Always
            envFrom:
            - secretRef:
                name: my-secret-upgrader-token
            volumeMounts:
              - name: config-volume
                mountPath: /etc/config
          volumes:
            - name: config-volume
              configMap:
                name: my-secret-upgrader-config