< img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=3131724&fmt=gif" />

Appendix: Advanced Configuration of KubeSphere Core

Helm Chart Options

Common Options

Option Default Value Description

adminPassword

""

string - Set a bootstrap password for the first admin user. Admins will need to reset the password upon login. If not set, the built-in default password P@88w0rd will be used.

hostname

"example.com"

string - Fully qualified domain name for the KubeSphere Server.

ingress.enabled

false

bool - If true, create the KubeSphere ingress.

ingress.tls.enabled

true

bool - If true, enable HTTPS for the KubeSphere ingress.

ingress.tls.source

"generation"

string - Source of ingress certificate, options: "generation, importation, letsEncrypt".

letsEncrypt.email

""

string - Email address to receive expiration reminders for certificates.

letsEncrypt.environment

"production"

string - Options: "staging, production". There are rate limit differences between certificates for test and production environments, see details in the Let’s Encrypt Documentation.

Advanced Options

Option Default Value Description

telemetry.enabled

true

bool - Whether to participate in the improvement program. If set to true, it indicates that KubeSphere is allowed to collect your data. For more information, please refer to Improvement Program.

internalTLS

false

bool - If true, enables internal TLS. Both console and apiserver will run HTTPS services.

ingress.ingressClassName

""

string - Optional ingress class to use for the ingress, options: "nginx, traefik".

ingress.secretName

"kubesphere-tls-certs"

string - Secrets containing TLS certificates used by the ingress.

extension.ingress.ingressClassName

""

ingressClassName for external ingress used by extensions.

extension.ingress.domainSuffix

""

Domain suffix for creating entry points for extensions; based on the external ingress address, it can be LB hostname address (e.g., xx.com), {node_ip}.nip.io, or internal DNS address (e.g., kse.local).

extension.ingress.httpPort

80

http port for extension ingress.

extension.ingress.httpsPort

443

https port for extension ingress.

certmanager.duration

2160h

string - Expiry time for certificates generated by cert-manager.

certmanager.renewBefore

360h

string - Time interval for cert-manager to renew certificates before expiry.

global.imageRegistry

registry.cn-beijing.aliyuncs.com

string - Set the global KubeSphere image registry address.

global.tag

"v4.1.1"

string - Set the global KubeSphere image registry tag.

apiserver.image.registry

""

string - Set ks-apiserver image registry address.

apiserver.image.repository

"kse/ks-apiserver"

string - Set ks-apiserver image name.

apiserver.image.tag

""

string - Set ks-apiserver image tag.

apiserver.nodePort

""

uint16 - Set NodePort port for ks-apiserver service.

console.image.registry

""

string - Set ks-console image registry address.

console.image.repository

"kse/ks-console"

string - Set ks-console image name.

console.image.tag

""

string - Set ks-console image tag.

console.nodePort

30880

uint16 - Set NodePort port for ks-console service.

controller.image.registry

""

string - Set ks-controller-manager image registry address.

controller.image.repository

"kse/ks-controller-manager"

string - Set ks-controller-manager image name.

controller.image.tag

""

string - Set ks-controller-manager image tag.

TLS Configuration

  1. Select SSL Configuration

    KubeSphere security configuration includes Ingress SSL Configuration and Internal SSL Configuration. The Ingress SSL Configuration supports three modes by default to enable SSL/TLS for secure access.

    1. Ingress SSL Configuration

      Configuration Helm Chart Option Cert-manager Required

      KubeSphere Generated TLS Certificates

      ingress.tls.source=generation

      No

      Let’s Encrypt

      ingress.tls.source=letsEncrypt

      Yes

      Import Existing Certificates

      ingress.tls.source=importation

      No

      • KubeSphere Generated TLS Certificates: Supports both cert-manager and Helm methods.

        • If cert-manager is already installed in the Kubernetes cluster, it is preferred to use cert-manager to generate certificates. KubeSphere uses cert-manager to issue and maintain certificates. KubeSphere generates its CA certificate, signs a certificate using that CA, and then manages the certificate with cert-manager.

        • If cert-manager is not installed, Helm is used to generate certificates. During the installation process with Helm, KubeSphere generates CA and TLS certificates based on the configured hostname. In this option, certificates do not support automatic expiration rotation.

      • Let’s Encrypt

        When using the Let’s Encrypt option, cert-manager must be utilized. In this scenario, cert-manager combines with a special issuer for Let’s Encrypt that performs all actions (including request and validation) necessary for getting a Let’s Encrypt issued cert. This configuration uses HTTP validation (HTTP-01), so the load balancer must have a public DNS record and be accessible from the internet.

      • Import Existing Certificates

        This option allows you to bring your own public- or private-CA signed certificate. KubeSphere will use that certificate to secure websocket and HTTPS traffic. In this case, you must upload this certificate (and associated key) as PEM-encoded files with the name tls.crt and tls.key. If you are using a private CA, you must also upload that certificate. This is due to the fact that this private CA may not be trusted by your nodes.

    2. Internal SSL Configuration

      After enabling internal SSL configuration, both Console UI and Apiserver in KubeSphere will use HTTPS. This configuration inherently supports cert-manager and helm generated certificates. When cert-manager is already installed in the Kubernetes cluster, it is preferred to use cert-manager to generate/manage certificates, and the DNS for certificates defaults to Console UI and Apiserver’s Service DNS within the Kubernetes cluster.

      Configuration Helm Chart Option Cert-manager Required

      Enable Internal SSL

      internalTLS=true

      No

  2. Install cert-manager

    If you are using your own certificate files (ingress.tls.source=importation), you can skip this step.

    Only when using KubeSphere-generated certificates (ingress.tls.source=generation) or Let’s Encrypt issued certificates (ingress.tls.source=letsEncrypt), you need to install cert-manager.

    # Add Jetstack Helm repository
    helm repo add jetstack https://charts.jetstack.io
    
    # Update local Helm Chart repository cache
    helm repo update
    
    # Install cert-manager Helm Chart
    helm install cert-manager jetstack/cert-manager -n cert-manager --create-namespace --set prometheus.enabled=false
    # Or
    kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/<VERSION>/cert-manager.yaml

    After installing cert-manager, check the running pods in the cert-manager namespace to verify that it has been deployed correctly:

    kubectl get pods --namespace cert-manager
  3. Based on your selected certificate option, enable SSL configuration for KubeSphere using Helm.

    1. Enable Ingress SSL Configuration

      • KubeSphere Generated Certificates

        helm upgrade --install -n kubesphere-system --create-namespace ks-core $chart --version $version \
        --set ingress.enabled=true \
        --set hostname=kubesphere.my.org
      • Let’s Encrypt

        This option uses cert-manager to automatically request and renew Let’s Encrypt certificates. Let’s Encrypt is free and a trusted CA, so it can provide valid certificates.

        helm upgrade --install -n kubesphere-system --create-namespace ks-core $chart --version $version \
        --set hostname=kubesphere.my.org \
        --set ingress.enabled=true \
        --set ingress.tls.source=letsEncrypt \
        --set letsEncrypt.email=me@example.org
      • Import External Certificates

        # Import external certificates
        kubectl create secret tls tls-ks-core-ingress --cert=tls.crt --key=tls.key -n kubesphere-system
        
        # Install KubeSphere
        helm upgrade --install -n kubesphere-system --create-namespace ks-core
        $chart --version $version \
        --set ingress.enabled=true \
        --set hostname=kubesphere.my.org \
        --set ingress.tls.source=importation
    2. Enable Internal SSL Configuration.

      helm upgrade --install -n kubesphere-system --create-namespace ks-core
      $chart --version $version \
      --set internalTLS=true

Configure the ratelimit Limiter

Once the limiter is enabled, it will independently limit requests for all users in KubeSphere, primarily supporting the following two methods:

  • Setting a rate limit for all users in KubeSphere, without support for setting individual rate limits for each user at the moment;

  • Setting a rate limit independently for each ServiceAccount in KubeSphere.

Enable the Limiter

Enabling the limiter means setting a rate limit for all users in KubeSphere.

  1. Modify the kubesphere-system configmap.

    kubectl -n kubesphere-system edit cm kubesphere-system

    Add the following content:

    rateLimit:
      enable: true   # Enable the limiter
      driver: memory # Memory mode
      QPS: 40.0      # Token recovery rate
      burst: 80      # Token bucket capacity
  2. Restart the ks-apiserver.

    kubectl -n kubesphere-system rollout restart deploy ks-apiserver

Set the ServiceAccount Limiter

Before setting, you need to enable the limiter as in the previous step. Then execute the following command to set the rate limit for ServiceAccount.

kubectl -n <Namespace> patch serviceaccounts.kubesphere.io <ServiceAccount> --type merge -p '{"metadata": {"annotations": {"kubesphere.io/ratelimiter-qps": "20.0", "kubesphere.io/ratelimiter-burst": "40"}}}'

Parameter Description

Option Default Value Description

rateLimit.enable

false

bool - Enable the limiter.

rateLimit.driver

memory

string - Limiter storage type, options: "memory".

rateLimit.QPS

5.0

float32 - Number of tokens recovered per second in the limiter token bucket algorithm.

rateLimit.burst

10

int - Maximum capacity of the token bucket in the limiter token bucket algorithm.

Note

The recommended QPS for token recovery rate should be half the burst capacity.

Receive the latest news, articles and updates from KubeSphere


Thanks for the feedback. If you have a specific question about how to use KubeSphere, ask it on Slack. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.