Skip to main content

End-to-end SDK generation and publishing with Jenkins Pipelines

Supported SDK languages:

TypeScript /
Javascript
Java /
Kotlin
PythonC#GoPHP

This tutorial shows you how to use Jenkins Pipelines with liblab to automate the SDK generation and publishing process for an existing project. You'll learn how to configure CI/CD workflows that automatically generate and release SDKs to public package managers like npm, PyPI, NuGet, Maven, Go Packages, and Packagist.

Requirements

Ensure that you:

  • Have Docker installed, which will be used to run Jenkins.
  • Have an account on the package management platform(s) you want to deploy your SDKs to (ex. NPM, PyPi, Maven). This guide will help you with all of the other deployment details.
Starting from scratch

This guide doesn't cover setting up a new liblab SDK project.

If you don't have a liblab SDK project yet then you'll first want to follow one of our framework guides or general getting started guide.

Version Control Systems

When using Jenkins to control your CI/CD, you can choose to use GitHub, GitLab, or BitBucket.

This tutorial uses GitHub to host the necessary repos.

Structural Overview

Your completed project will consist of:

  • Your existing liblab project repository which we'll refer to as the "Control Repo"
  • One SDK repository per SDK language you want to publish which are your "SDK Repos"

When you complete the tutorial, the project repository structure will look like the example below.

Where each SDK repo will have the following organization:

Any change to the API spec or liblab configuration in the Control Repo will trigger SDK builds and create pull requests (PRs) in the respective SDK repositories.

Set up the Control Repo

First, you'll need to set up your Jenkins Pipeline. Run the following command from the root of your project:

Terminal
cd your-liblab-project
curl --remote-name "https://raw.githubusercontent.com/liblaber/control-repo-template/main/Jenkinsfile"

This command downloads the official liblab Jenkins file (Jenkinsfile) and places it in the root of your Control Repo. The Jenkinsfile is a pipeline configuration file for automating SDK generation and pull request creation when specific files or directories, such as liblab.config.json, spec/, hooks/, or customPlanModifiers/, are modified. It defines steps to validate environment variables, detect file changes, build SDKs using liblab, and create pull requests. You can customize it as needed.

This is what you should have so far:

Create SDK Repos

Your generated SDKs live in separate repos, one per SDK language.

Templates are available for each language. Click the links below and follow the instructions to create a new repository for each SDK language you want to publish.

Each liblab template includes a Jenkinsfile that automates the SDK build and deployment process to the package manager service.

This is what you should have so far:

Create liblab and GitHub tokens

To manage your CI/CD process with Jenkins, you must configure the following secrets to enable interactions with GitHub and access liblab features:

  1. A liblab token for authenticating the liblab CLI.
  2. A GitHub token to allow Jenkins and the liblab CLI to push to your SDK repositories.
Version Control Systems

If you're using Jenkins with GitLab or Bitbucket, use their respective tokens instead of the GitHub token to grant permissions to Jenkins and the liblab CLI.

Create a liblab token

Token expiration and secret management

By default, this token will expire after 90 days. You can increase this to up to 364 days by passing the --durationInDays parameter.

Read more on the liblab token command in the CLI documentation.

For guidance on setting up secrets in GitHub, refer to the GitHub secrets documentation.

To generate a liblab token, run the following command from your terminal:

liblab token create GITHUB_PR_TOKEN

You can replace GITHUB_PR_TOKEN with any name you prefer. If you're generating SDKs for multiple APIs, you can either share the token between them or create separate tokens for each API.

Token successfully generated, it will be valid for 90 days

-TOKEN-----------------------------------------
liblab_HYeJzYb5CO3t7JaW4I6ZWKNKbOGp01MqqYSTS6yG
-----------------------------------------------

Copy the token (e.g., liblab_HYeJzYb5CO3t7JaW4I6ZWKNKbOGp01MqqYSTS6yG) and save it somewhere safe. You'll use it to configure credentials in Jenkins.

Create a GitHub token

tip

It's helpful to create a reminder for yourself such as a calendar item to refresh your liblab and Github tokens just before they expire!

To enable the Jenkins pipeline to identify changes in your Control Repo, run the liblab CLI, and push changes to your SDKs repos, you'll need to generate a GitHub Fine-Grained Access Token with the right permissions. Follow these steps:

  1. Open GitHub's fine-grained token settings and click Generate new token.
  2. Name the token GITHUB_TOKEN and set its expiration.
  3. Under Repository Access, select **Only select repositories **. Choose the repos created in Step 1 and Step 2.
  4. Set the following repo permissions:
    • Commit Statuses: Read/Write.
    • Contents: Read/Write.
    • Metadata: Read.
    • Pull Requests: Read/Write.
    • Workflows: Read/Write.

Repository permissions

  1. Click Generate token.
CI/CD documentation

Access the Pull request creation documentation page for further information and detailed guidance on creating the token.

Once created, copy the GITHUB_TOKEN and save it in a secure location for use in the upcoming steps. For more details on how the GITHUB_TOKEN will be used to create PRs, access the liblab documentation.

Create package manager tokens

Depending on which SDK languages you're using, you'll need to create additional secrets.

To publish to npm, you will need an npm access token. You can generate a new access token from your user settings on npm.

  1. Select the Access Tokens tab, drop down the Generate New Token button, then select Granular Access Token.

    The access tokens page in npm settings

  2. Fill in all the required details for the token such as the name and expiry.

  3. Make sure that this token has read and write permission to publish packages. If the package already exists, you can scope this token to just that package, otherwise this token needs read and write for all packages.

  4. Once the token has been created, make a copy of it.

  5. In your TypeScript SDK repo, add this token as an Actions secret named NPM_TOKEN.

    The token set as an actions secret

note

You can learn more about creating npm tokens in the npm access tokens documentation.

At this point, you should have a complete setup.

Configure Jenkins

You can install Jenkins using different approaches and host it locally or in the cloud.

Run Jenkins using Docker composer

For this tutorial, Jenkins will run in a Docker and a Docker composer is used to define the configurations. The following code snippet shows the necessary code to use in the docker-compose.yml file.

docker-compose.yml
version: '3.8'

services:
jenkins:
image: jenkins/jenkins:lts
container_name: jenkins
ports:
- '8080:8080'
- '50000:50000'
volumes:
- ${HOME}/jenkins:/var/jenkins_home
privileged: true
user: root
Volume Configuration

Replace ${HOME} with an existing directory path on your machine to ensure Jenkins data is persisted locally.

In the code above, volume links the host machine's ${HOME}/jenkins directory to the container's /var/jenkins_home. This ensures Jenkins data, such as jobs and configurations, remains intact even if the container is stopped or removed. The privileged: true setting grants the container additional permissions. The user: root setting allows the container to run as the root user, enabling more control for system-level operations.

To start the Jenkins service, follow these steps:

  1. In a new directory, separate from the Control Repo, create a file named docker-compose.yml.

  2. Copy and paste the provided code into the file and save it.

  3. Open a terminal in the directory containing the docker-compose.yml file and run the following command:

    docker-compose up -d

    Your terminal should display output similar to this:

    $ docker-compose up -d
    [+] Running 1/1
    ✔ Container jenkins Started
  4. Once the Jenkins service is running, it will be accessible in your browser at http://localhost:8080.

  5. To log in, you need the Jenkins Admin password. Follow these steps to retrieve it:

    1. Access the Docker container terminal running Jenkins:

      docker exec -it jenkins bash
    2. Retrieve the Admin password:

      cat /var/jenkins_home/secrets/initialAdminPassword

    The password will be displayed in the terminal. Save it securely, as you'll need it to log in to Jenkins.

Run Jenkins and install the plugins

To get started with Jenkins, follow the steps:

  1. Access Jenkins through your browser using the address http://localhost:8080.

  2. Use the admin password from the previous step to unlock Jenkins. For future logins, you'll also need to use the admin user with the admin password.

    Jenkins unlok screen

    Additional Users

    You don't need to create additional users for this tutorial. You can perform all actions using the Admin user.

  3. Select the Install suggested plugins option. These are the essential plugins you must select:

    Select the suggested plugins installation

  4. Jenkins will download and install the selected plugins. Once the installation is complete, you'll be redirected to the Jenkins dashboard.

    Jenkins dashboard screen

To set up your Jenkins environment, install the three additional plugins:

  1. From the Jenkins dashboard, navigate to Manage Jenkins -> Plugins -> Available plugins.
  2. Search for and select the following plugins:
    • Docker plugin
    • Docker Pipeline
    • Basic Branch Build Strategies Plugin
  3. Once the plugins are selected, click Install.
  4. After the installation is complete, Jenkins will restart automatically.
note

During the restart process, the Jenkins container may stop temporarily. If you cannot access the dashboard at http://localhost:8080 and see a Page Not Found error, restart the Jenkins container by running:

docker-compose up -d

Add your credentials

Once you install the necessary plugins, add the LIBLAB_TOKEN and GITHUB_TOKEN as credentials in Jenkins. These credentials are used by Jenkins to interact with your GitHub repos (both the Control Repo and SDK repos) and to use the liblab CLI:

  1. From the Jenkins dashboard, go to Manage Jenkins -> Credentials.
  2. Under Stores scoped to Jenkins, select System.
  3. Under the Domain, choose Global credentials.
  4. Click + Add Credentials.
  5. Fill in the form for each of the credentials as described below:
  • Kind: Select the Secret text option.
  • Secret: Paste the LIBLAB_TOKEN from Step 3.
  • ID: Use ID equal LIBLAB_TOKEN.
  • Description: Add a meaningful description to help identify this credential later.

This is what you should have:

Jenkins credentials dashboard

Create a Jenkins pipeline

Now that your environment is configured, you can create a pipeline to manage your SDK projects' CI/CD process:

  1. From the Jenkins dashboard, click New Item.

  2. Enter a name for the pipeline, select Multibranch Pipeline, and click OK.

  3. On the pipeline configuration page, do the following:

    1. In the Display Name field, add a name for the pipeline.
    2. In the Description field, provide a meaningful description of the pipeline’s purpose.
    3. For Branch Sources, select GitHub as the source. A new configuration form will appear.
      1. For Credentials, choose the username and password (github-credentials) you added in Add your credentials.
      2. Paste the Control Repo URL into the Repository HTTPS URL field.
      3. (Optional) Configure branch strategies if needed. For this tutorial, you can leave the default settings.
    4. For Build Configuration, select by Jenkins for the Mode and add Jenkins for the Script Path field.
    5. For the Scan Repository Triggers, set Interval to 10 minutes to define how often Jenkins should scan the repo for changes.
    Webhooks and GitHub plugins

    Instead of setting a scan interval, you can configure GitHub Webhooks for automatic triggering or use GitHub plugins to enhance Jenkins-GitHub integration.

  4. Click Save.

Once saved, Jenkins will scan the repo for existing branches and pull requests. It will automatically trigger a build if it finds a Jenkinsfile.

Jenkins scanning the repo

Since your Control Repo has a Jenkinsfile, Jenkins will start the pipeline job. However, the pipeline will fail initially because the liblab.config.json file has not yet been updated with the SDK repos and organizations.

Jenkins pipeline console with error while building the SDKs and creating the pull request

Update the config file

Now that finished configuring Jenkins and all your repositories are set up, you'll need to update the liblab.config.json file. Make sure to configure the publishing information and update the settings for each SDK language you plan to publish.

  1. In the publishing section, set the githubOrg option to the name of your GitHub organization.
  2. In the languageOptions section:
    1. Update the githubRepoName for each SDK. This should be the repo name only (e.g., typescript-sdk), not the full GitHub URL (e.g., https://github.com/myorg/typescript-sdk).

    2. Set the sdkVersion to match your desired version number. This should be updated for each new release.

    3. Configure details for each package manager:

      • Set the npmName option to the name of your npm package.
      • Set the npmOrg option to the name of your npm organization.
Additional configuration options

There are many other options you can configure for SDK deployment but these are optional. Refer to the config file documentation to learn more.

The following code snippet shows an example of a liblab.config.json file, where SDKs for C#, Python, TypeScript, Go, PHP, and Java are configured for generation and publishing:

Example liblab.config.json configured for SDK deployment
{
"sdkName": "test-sdk",
"apiVersion": "1.0.0",
"apiName": "test-api",
"specFilePath": "spec.json",
"languages": [
"csharp",
"python",
"typescript",
"go",
"php",
"java"
],
"languageOptions": {
"csharp": {
"liblabVersion": "2",
"packageId": "Test.SDK",
"githubRepoName": "csharp-sdk",
"sdkVersion": "1.0.0"
},
"python": {
"liblabVersion": "2",
"pypiPackageName": "test-sdk",
"githubRepoName": "python-sdk",
"sdkVersion": "1.0.0"
},
"typescript": {
"npmName": "test-sdk",
"npmOrg": "myorg",
"githubRepoName": "typescript-sdk",
"sdkVersion": "1.0.0"
},
"java": {
"groupId": "com.myorg",
"artifactId": "test-sdk",
"homepage": "https://myorg.com",
"githubRepoName": "java-sdk",
"sdkVersion": "1.0.0",
"developers": [
{
"name": "John Doe",
"email": "[email protected]",
"organization": "My Organization",
"organizationUrl": "https://myorg.com"
}
]
},
"go": {
"goModuleName": "github.com/myorg/go-sdk",
"githubRepoName": "go-sdk",
"sdkVersion": "1.0.0",
},
"php": {
"packageName": "myorg/test-sdk",
"githubRepoName": "php-sdk",
"sdkVersion": "1.0.0"
},
},
"publishing": {
"githubOrg": "myorg"
}
}

Finally, commit and push your changes to your Control Repo. This will trigger the Jenkins pipeline to generate your SDKs and create PRs in your SDK repos.

Check the Jenkins pipeline

Once you pushes changes to the Control Repo, Jenkins will automatically scan the repo. If it detects changes, it will trigger a new build for the pipeline.

Triggering a New Build

If you want to manually force Jenkins to scan your repo for changes, go to the pipeline you created previously and click Scan Repository Now.

Scan repo now

To review the results of the latest Jenkins pipeline job:

  1. Navigate to the pipeline you created previously.
  2. Click Build History to view the results of previous builds.
  3. Select the build number of the most recent job to open its summary.
  4. Review the Pipeline Overview to inspect each pipeline stage. If errors arise, you can check the output of each stage to identify the problem.

The following GIF demonstrates how to inspect the results of your builds.

Sucessfull pipeline job gif

Merge SDK PRs

After Jenkins completes the pipeline, a pull request (PR) will be created automatically in each SDK repo by liblab.

To view the PRs, access each SDK repo on GitHub and navigate to the Pull Requests section to view the new PRs created by the liblab CLI.

New PR created by liblab CLI

note

If you don't see the PRs, review Jenkins' build history to check for any errors during the SDK generation process.

To finish, follow the steps:

  1. Open each PR to review the changes and ensure the SDK has been generated correctly.
  2. Verify that the modifications align with your expectations and meet project requirements.
  3. If everything is correct, approve the PR and merge it into the main branch.

Create a release in GitHub

To trigger the Action to publish your SDK to the package manager, you need to create a release in the SDK repo. Releases need a tag to tag the commit that you want to release.

  1. From each SDK repo, go to Releases and click Create a new release.

    The releases section

  2. Click Choose a tag, enter a tag (e.g., v1.0.0), and select Create new tag.

    note

    The tag should follow semantic versioning and typically includes a v prefix, like v1.0.0.

    Note on Versioning Go Packages

    It is important to note that the release tag version will be used as the version of the package in the Go Packages repository. Therefore, it is important to follow semantic versioning and ensure that the version in the liblab.config.json file corresponds to the version of the release tag.

  3. Give the release a title and description. The title can be the same as the tag.

  4. Click Publish release. This will trigger the GitHub Action to publish your SDK.

Confirm SDK publication

Once the publishing Action is done, check the package manager to confirm your SDK was published.

🎉🎉🎉   Congratulations! You have successfully generated and published your SDK!   🎉🎉🎉

Your setup is now fully automated to generate and publish SDKs with liblab and Jenkins pipelines, ensuring your latest API updates are quickly available to your users.

If you don’t see the SDK in the package manager, check the CI/CD > Pipelines or Jobs section in each SDK repo to troubleshoot any errors during the publishing process.