< img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=3131724&fmt=gif" />
🚀 KubeSphere v4.1.2 with Enhancements and Bug Fixes is available now. Read the release notes for v4.1.2 → close
  • English
    • English
    • 简体中文
    1. 1.Create a KubeSphere Cloud account and log in to the platform.
    2. 2.In the upper-right corner, click Console, and then click Lite Cluster on the left.
    3. 3.Create a K8s cluster with KubeSphere installed.
    4. 4.Access KubeSphere with the default account and password (admin/P@88w0rd).
  •  Star
  1. Docs / 
  2. Documentation / 
  3. Extension User Guide / 
  4. DevOps / 
  5. How to Use DevOps / 
  6. Pipelines / 
  7. Create a Pipeline Using a Jenkinsfile

Documentation

Last updated:
    Edit

    Create a Pipeline Using a Jenkinsfile

    A Jenkinsfile is a text file that contains the definition of a Jenkins pipeline and is checked into a source code control repository. As it stores the entire workflow as code, the Jenkinsfile forms the basis for code reviews and pipeline iterations. For more information, refer to the Jenkins Documentation.

    This document demonstrates how to create a pipeline based on a Jenkinsfile from a GitHub repository.

    Note

    DevOps supports creating two types of pipelines: pipelines created based on a Jenkinsfile in SCM as described in this document, and pipelines created through the graphical editing panel.

    The Jenkinsfile in SCM requires an internal Jenkinsfile in Source Control Management (SCM), meaning the Jenkinsfile must be part of the SCM. The DevOps system automatically builds the CI/CD pipeline based on the existing Jenkinsfile in the code repository. By defining workflows such as stage and step, specific build, test, and deployment requirements can be met.

    Prerequisites

    • DevOps must have been installed and enabled.

    • You have a Docker Hub account and a GitHub account.

    • A workspace, a DevOps project, and a user (e.g. project-regular) have been created, and the user has been invited to the DevOps project with the operator role. Refer to Role and Member Management.

    • A dedicated CI node has been set up to run pipelines. Refer to Set CI Nodes for Dependency Cache.

    • SonarQube has been installed and configured (optional). Refer to Integrate SonarQube into Pipelines. If you skip this, the SonarQube Analysis stage will be omitted.

    Pipeline Overview

    This example pipeline consists of the following stages:

    Note

    • Stage 1: Checkout SCM: Fetch the source code from the GitHub repository.

    • Stage 2: Unit Test: It will not proceed with the next stage until the test is passed.

    • Stage 3: SonarQube Analysis: The SonarQube code quality analysis.

    • Stage 4: Build and Push: Build an image based on the selected branches in Strategy Settings and push the SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER tag to Docker Hub, where $BUILD_NUMBER is the run ID of the record in the pipeline run records.

    • Stage 5: Push Latest: Tag the v4.1.0-sonarqube branch as latest and push it to Docker Hub.

    • Stage 6: Push with Tag: Generate a tag and release it to GitHub, which will be pushed to Docker Hub.

    Step 1: Create Credentials

    1. Log in to the KubeSphere web console as the project-regular user.

    2. Click Workspace Management and navigate to your DevOps project. Under DevOps Project Settings, create the following credentials in the Credentials page. For more information on creating credentials, refer to Credential Management.

      Note

      If your account or password contains special characters such as @ and $, errors may occur during pipeline runs due to unrecognized characters. In such cases, encode your account or password on a third-party website (e.g., urlencoder) and then copy and paste the encoded result as your credential information.
      Credential ID Type Where to use

      dockerhub-id

      Username and Password

      Docker Hub

      github-id

      Username and Password

      GitHub

    3. Create another credential for SonarQube (sonar-token) for Stage 3 (Code Analysis). Choose the credential type Access Token and enter the SonarQube token in the Token field. Refer to Create a SonarQube Token for the New Project. Click OK to complete the process.

    4. You also need to create a GitHub Personal Access Token (PAT) with the permissions shown in the following image. Then, in the DevOps project, use the generated token to create account credentials for GitHub authentication (e.g., github-token).

      github token scope

      Note

      To create a GitHub Personal Access Token, go to your GitHub account’s Settings, click Developer settings, select Personal access tokens, and then click Generate new token.

    5. Once created, you will see the credentials on the credentials page.

    Step 2: Modify the Jenkinsfile in your GitHub repository

    1. Log in to GitHub and fork all branches of the repository devops-maven-sample to your personal GitHub account.

    2. In your GitHub repository devops-maven-sample, switch to the v4.1.0-sonarqube branch and click on the file Jenkinsfile-online in the root directory.

    3. Click the edit icon on the right to edit the environment variables.

      Entry Value Description

      DOCKER_CREDENTIAL_ID

      dockerhub-id

      The name for your Docker Hub account in KubeSphere.

      GITHUB_CREDENTIAL_ID

      github-id

      The name for your GitHub account in KubeSphere to push tags to your GitHub repository.

      REGISTRY

      docker.io

      It defaults to docker.io, used as the address to push images.

      DOCKERHUB_NAMESPACE

      your-dockerhub-id

      Replace it with your Docker Hub account name or the organization name under that account.

      GITHUB_ACCOUNT

      your-github-id

      Replace it with your GitHub account name. For example, if your GitHub URL is https://github.com/kubesphere/, your GitHub account name is kubesphere or the organization name under that account.

      APP_NAME

      devops-maven-sample

      The application name.

      SONAR_CREDENTIAL_ID

      sonar-token

      The name for the SonarQube token in KubeSphere used for code quality checks.
      Note

      In the Jenkinsfile, the -o parameter for the mvn command enables offline mode. Relevant dependencies have been downloaded in this tutorial to save time and accommodate network disruptions in certain environments. Offline mode is enabled by default.

    4. After editing the environment variables, click Commit changes to update the file in the v4.1.0-sonarqube branch.

    Step 3: Create a Pipeline

    1. Log in to KubeSphere web console as the project-regular user.

    2. Click Workspace Management and navigate to your DevOps project. Click Pipelines and then click Create.

    3. In the pop-up dialog, name it jenkinsfile-in-scm.

    4. Under Pipeline Type, select Multi-branch Pipeline.

    5. Under Code Repository, choose a code repository and click Next to proceed.

      If there are no available code repositories, click Create a code repository below. For more information, see Import Code Repositories.

      1. In the Import Code Repository dialog, enter a custom code repository name and click Select a code repository.

      2. On the GitHub tab, select github-token from the Credential dropdown menu and click OK.

      3. In the GitHub list, select your GitHub account, and all repositories associated with that token will be listed on the right. Choose devops-maven-sample and click Select.

      4. Click OK to select your code repository.

    6. In Advanced Settings, check Delete outdated branches. In this tutorial, it is recommended to leave Branch Retention Period (days) and Maximum Branches at their default values.

      Delete outdated branches means that you will discard the branch record all together. The branch record includes console output, archived artifacts and other relevant metadata of specific branches. Fewer branches mean that you can save the disk space used by Jenkins. KubeSphere provides two options to determine when old branches are discarded:

      • Branch Retention Period (days). Branches that exceed the retention period are deleted.

      • Maximum Branches. The earliest branch is deleted when the number of branches exceeds the maximum number.

      Note

      Branch Retention Period (days) and Maximum Branches apply to branches at the same time. As long as a branch meets the condition of either field, it is deleted. For example, if you specify 2 as the retention period and 3 as the maximum number of branches, any branch that exceed either number is deleted. DevOps prepopulates these two fields with 7 and 5 by default respectively.

    7. In Strategy Settings, DevOps offers four strategies by default. You can delete Discover PRs from Forks, as this strategy will not be used in this example. For other strategies, no need to change the setting and you can use the default value directly.

      Note

      To enable Strategy Settings here, you should select GitHub as the code repository.

      As a Jenkins pipeline runs, the Pull Request (PR) submitted by developers will also be regarded as a separate branch.

      Discover Branches

      • Exclude branches field as PRs. The source branch is not scanned such as the origin’s master branch. These branches need to be merged.

      • Include only branches filed as PRs. Only scan the PR branch.

      • Include all branches. Pull all the branches from the repository origin.

      Discover PRs from Origin

      • Pull the code with the PR merged. A pipeline is created and runs based on the source code after the PR is merged into the target branch.

      • Pull the code at the point of the PR. A pipeline is created and runs based on the source code of the PR itself.

      • Create two pipelines respectively. Two pipelines are created, one is based on the source code after the PR is merged into the target branch, and the other is based on the source code of the PR itself.

    8. Scroll down to Script Path, set it to Jenkinsfile-online, which is the file name of Jenkinsfile in the example repository located in the root directory. The field specifies the Jenkinsfile path in the code repository. It indicates the repository’s root directory. If the file location changes, the script path also needs to be changed.

    9. In Scan Trigger, select Scan periodically and set the interval to 5 minutes. Click Create to finish.

      Note

      You can set a specific interval to allow pipelines to scan remote repositories, so that any code updates or new PRs can be detected based on the strategy you set in Strategy Settings.

    Step 4: Run the pipeline

    1. After a pipeline is created, click its name to go to its details page.

      Note

      • On the Pipelines list page, click more on the right of the pipeline, and select Copy to create a duplicate of that pipeline.

      • To simultaneously run multiple pipelines that do not include multibranch configurations , select these pipelines on the Pipelines list page and click Run to run them in bulk.

      • The Sync Status on the pipeline details page shows the synchronization result between KubeSphere and Jenkins. If the synchronization is successful, it will display Successful along with a green checkmark icon.

    2. Under Run Records, multiple branches are being scanned. Click Run on the right and the pipeline runs based on the behavioral strategy you set. Select v4.1.0-sonarqube from the drop-down list and add a tag number such as v0.0.2. Click OK to trigger a new run.

      Note

      • If you do not see any run records on this page, you need to refresh your browser manually or click More > Scan Repository.

      • The tag name is used to refer to the newly generated release and image in GitHub and Docker Hub. Existing tag names cannot be reused for the TAG_NAME field. Otherwise, the pipeline will not be running successfully.

    3. Wait for a while, click run records to view details.

      Note

      Activity failures may be caused by different factors. In this example, only the Jenkinsfile of the branch v4.1.0-sonarqube is changed as you edit the environment variables in the steps above. While, these variables in the v4.1.0 branch remain unchanged (namely, wrong GitHub and Docker Hub account). If you choose v4.1.0 branch to run, it will result in a failure. Other reasons for failures may be network issues, incorrect coding in the Jenkinsfile and so on.

      In the Run Logs tab on the run record details page, you can view detailed information of the logs to troubleshoot and resolve issues.

    4. If the pipeline reaches the Push with Tag stage, it will pause at this point and require a user with approval permissions to click Proceed.

      In a development or production environment, it requires someone who has higher permissions (for example, release manager) to review the pipeline, images, as well as the code analysis result. They have the authority to determine whether the pipeline can go to the next stage. In the Jenkinsfile, you use the section input to specify who reviews the pipeline. If you want to specify a user (for example, project-admin) to review it, you can add a field in the Jenkinsfile. If there are multiple users, you need to use commas to separate them as follows:

      input(id: 'release-image-with-tag', message: 'release image with tag?', submitter: 'project-admin,project-admin1')

    5. Log in to the KubeSphere web console as a user with pipeline approval permissions. Click Workspace Management and navigate to your DevOps project. Click the pipeline name to access its details page. Under the Run Records tab, click the record you want to review, then click Proceed to approve the pipeline.

      Note

      In KubeSphere, if you do not specify a reviewer, the user that can run a pipeline will be able to continue or terminate the pipeline. Additionally, the pipeline creator, users with the project administrator role, or any accounts specified by you also have the authority to continue or terminate the pipeline.

    Step 5: Check Pipeline Status

    1. Under the Pipeline tab in the run records, check the running status of the pipeline. The pipeline may take a few minutes to initialize when first created.

    2. Click the Run Logs tab to view the pipeline’s running logs. Click each stage to view detailed logs. Click View Full Logs to troubleshoot and resolve issues based on the logs, and you can also download the logs for further analysis.

    Step 6: Verify Results

    1. After a successful pipeline run, click Code Check to view the code analysis results provided by SonarQube. This page will be unavailable if SonarQube has not been configured beforehand.

    2. Following the definitions in the Jenkinsfile, the Docker image built by the pipeline has been successfully pushed to Docker Hub. In Docker Hub, you will see an image with the tag v0.0.2, specified before the pipeline runs.

    3. At the same time, a new tag and a new release have been generated in GitHub.

    Receive the latest news, articles and updates from KubeSphere


    Thanks for the feedback. If you have a specific question about how to use KubeSphere, ask it on Slack. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.

    Previous : Create a Pipeline Using Graphic Editing Panels
    What’s on this Page