Skip to main content

End-to-end SDK generation and publishing with GitLab Pipelines

This tutorial includes the following SDK languages and versions:

TypeScript v1TypeScript v2Java v1Java v2Python v1Python v2C#GoPHP
✅✅✅✅✅✅✅✅✅

This tutorial will guide you through generating an SDK using liblab and automating the SDK publishing process using GitLab CI/CD pipelines. By combining liblab and GitLab's CI/CD features, you can manage the entire SDK lifecycle, from generation to publication on package managers such as npm, PyPI, and Maven.

By the end of this tutorial, you will have a GitLab CI/CD pipeline that automatically generates SDKs from your API specification and publishes them to package managers. The following diagram presents an overview of the flow:

Prerequisites​

To follow this tutorial, you’ll need the following:

Steps​

This tutorial covers the following steps:

  1. Create a control repo
  2. Create SDK repos
  3. Create liblab and GitLab tokens
  4. Create package manager tokens
  5. Update your spec and config file
  6. Merge SDK PRs
  7. Create a release in GitLab
  8. Confirm SDK publication

1. Create a control repo​

The first step is to create a control repo, which is used as the central repo to store your liblab config file, any hooks code, and the API spec.

API spec

You don't need to store the OpenAPI spec in the repo. You can link your OpenAPI spec from other repos directly to the liblab.config.json file.

For this tutorial, you can use the control-repo-template. Access the control repo documentation for more information. The control-repo-template provided by liblab contains:

FileDescription
.gitlab-ci.ymlIt automates the SDK generation process and creates pull requests for changes in specific files, such as liblab.config.json and directories like hooks and customPlanModifiers. It triggers on pushes to the main branch or manual web actions, using a Node.js environment to execute the build.
liblab.config.jsonAn example of liblab configuration file. It contains the configuration used by the liblab CLI to generate SDKs.
spec.jsonAn OpenAPI spec example based on the Petstore. The liblab.config.json file is refering this OpenAPI spec.
.github/workflowsDirectory with two GitHub Actions. One creates PRs for your SDKs when your API spec or config file changes. The second GitHub Action polls for liblab updates and creates PRs for your SDKs when there is a liblab update.

The control-repo-template is hosted in GitHub, therefore, you need to perform the following steps to use it with GitLab:

  1. Create a new repo in GitLab. In your GitLab dashboard:

    1. Click New project.
    2. Select Create blank project.
    3. Set the Project name and Project slug and select Visibility Level as Public.
    4. Click Create project.

  2. Clone the GitHub repo to your local machine:

git clone https://github.com/liblaber/control-repo-template
  1. Navigate to the cloned repo on your machine and add your new GitLab repo as a remote:
cd control-repo-template
git remote add gitlab https://gitlab.com/<your-gitlab-username>/<your-gitlab-repo>.git
  1. Push the control-repo-template to your GitLab repo:
git push gitlab main

This is what you should have on your GitLab project:

2. Create SDK repos​

Each SDK you create needs its own repo for storage and publication. These SDK repos should be created before you can publish your SDKs. You can create these repos manually or use one of the liblab templates.

Create new GitLab projects for each SDK language or use liblab's templates:

If you're using a liblab template, follow the steps outlined in Step 1 to use the templates with GitLab.

Each liblab template includes a GitLab CI automation file that automates the build and deployment process of the SDK to the package manager service.

note

Learn more about the SDK repos by accessing the SDK repos documentation.

After creating the repos, this is what you should have:

3. Create liblab and GitLab tokens​

The GitLab CI/CD pipeline used by the control repo that will generate SDKs requires two secrets:

  • A liblab token that authenticates the liblab CLI.
  • A GitLab token that allows the liblab CLI to push to your SDK repos.

3.1 Create a liblab token​

To create the liblab token, follow the steps below:

  1. Using the liblab CLI on your machine, run the following command to generate a liblab token:

    Terminal
    liblab token create GITLAB_CI_TOKEN

    You can replace GITLAB_CI_TOKEN with any name of your choice. If you're generating SDKs for multiple APIs, you can either use the same token for all of them or create separate tokens for each API.

    Terminal
    Token successfully generated, it will be valid for 90 days

    -TOKEN-----------------------------------------
    liblab_td7MY4WfsE6Df4BRfyhoma7uONeOrSWmvxtSRPDj
    -----------------------------------------------
    note

    By default, the token expires after 90 days. You can extend this to up to 364 days by passing the --durationInDays parameter.

    Learn more about the liblab token command in the CLI documentation.

  2. Copy the generated token. You should copy only the string liblab_... at the central line. The token will allow liblab to authenticate during the CI/CD pipeline run.

  3. Add the token to your Gitlab control-repo project. Access your GitLab project, navigate to Settings > CI/CD > Variables and click Add variable. For Key, use LIBLAB_TOKEN and paste the liblab token into the Value field. After, click Add variable.

add liblab token as variable

3.2 Create a GitLab token​

To enable the GitLab pipeline to run the liblab CLI and push changes to your SDKs repos, you need to generate a GitLab personal access token:

  1. In GitLab, click at your profile picture and select Preferences.

  2. Access the Access tokens tab from the sidebar.

  3. Select Add new token.

  4. Enter a name (e.g., liblab-token) and an expiration date based on your organization's security policy.

    tip

    It's helpful to create a calendar reminder to refresh the token before it expires.

  5. Select the following scopes:

    • api
    • read_repository
    • write_repository
    • read_user

  6. Select Create personal access token and copy the generated token. This token will be used to authenticate and push SDKs to your SDK repos.

  7. Add this token to your GitLab project's Settings > CI/CD > Variables as LIBLAB_GITLAB_TOKEN.

Control repo variables

This is what you should have set up so far:

4. Create package manager tokens​

Depending on what SDK language you are using, you will need to create additional secrets.

To publish to npm, you will need an npm access token. You can generate a new access token from your user settings on npm.

  1. Select the Access Tokens tab, drop down the Generate New Token button, then select Granular Access Token.

    The access tokens page in npm settings

  2. Fill in all the required details for the token such as the name and expiry.

  3. Make sure that this token has read and write permission to publish packages. If the package already exists, you can scope this token to just that package, otherwise this token needs read and write for all packages.

  4. Once the token has been created, make a copy of it.

  5. In your TypeScript SDK repo, add this token as an Actions secret named NPM_TOKEN.

    The token set as an actions secret

note

You can learn more about creating npm tokens in the npm access tokens documentation.

You should now have the following complete setup:

5. Update your spec and config file​

After creating and configuring your SDK repos, the next step is to update your control repo with your API spec and configure the liblab.config.json file.

Follow the steps below to update your API spec and trigger the release of new SDKs to the corresponding repos:

  1. Clone your control repo and open it in your preferred code editor.

  2. Replace the control repo API spec. The control repo includes a sample API spec (spec.json), which you can use to continue following this tutorial. However, you can replace the existing spec.json with your API spec.

    • If you have a local API spec, replace the spec.json file with your own.
    • If your API spec is hosted remotely:
      1. Update the specFilePath field in the liblab.config.json file with the URL of your remote API spec.
      2. Delete the existing spec.json file from the repo if you are using a remote spec.

  3. Open the liblab.config.json file in your control repo and update the following fields:

    • sdkName: Set the name of your SDK.
    • apiName: Set the name of your API.
    • apiVersion: Specify the API version you're working with.
    • languages: List the SDK languages you want to generate, such as TypeScript, Python, Go, etc.

  4. In the liblab.config.json, locate the languageOptions section and update the options for each SDK language you are generating:

    1. Remove any unused languages. If there are languages you don't need, remove their configuration.

    2. For each language, update the githubRepoName. liblab will use it to identify the GitLab project that matches the name of the corresponding SDK repo.

    3. Update the sdkVersion field to specify the version number for each SDK.

    4. Configure the specific fields for each package manager:

      • Set the npmName option to the name of your npm package.
      • Set the npmOrg option to the name of your npm organization.

    5. In the publishing section, set the githubOrg as the GitLab group from the corresponding SDK repo.

      tip

      You can configure additional optional settings. Access the liblab config file documentation to learn more.

    After finishing, your liblab.config.json file should look like this:

    liblab.config.json
    {
      "sdkName": "test-sdk",
      "apiVersion": "1.0.0",
      "apiName": "test-api",
      "specFilePath": "spec.json",
      "languages": [
        "csharp",
        "python",
        "typescript",
        "go",
        "php",
        "java"
      ],
      "languageOptions": {
        "csharp": {
          "liblabVersion": "2",
          "packageId": "Test.SDK",
          "githubRepoName": "csharp-sdk",
          "sdkVersion": "1.0.0"
        },
        "python": {
          "liblabVersion": "2",
          "pypiPackageName": "test-sdk",
          "githubRepoName": "python-sdk",
          "sdkVersion": "1.0.0"
        },
        "typescript": {
          "npmName": "test-sdk",
          "npmOrg": "myorg",
          "githubRepoName": "typescript-sdk",
          "sdkVersion": "1.0.0"
        },
        "java": {
          "groupId": "com.myorg",
          "artifactId": "test-sdk",
          "homepage": "https://myorg.com",
          "githubRepoName": "java-sdk",
          "sdkVersion": "1.0.0",
          "developers": [
            {
              "name": "John Doe",
              "email": "[email protected]",
              "organization": "My Organization",
              "organizationUrl": "https://myorg.com"
            }
          ]
        },
        "go": {
          "goModuleName": "github.com/myorg/go-sdk",  
          "githubRepoName": "go-sdk",
          "sdkVersion": "1.0.0",
        },
        "php": {
          "packageName": "myorg/test-sdk",
          "githubRepoName": "php-sdk",
          "sdkVersion": "1.0.0"
        },
      },
      "publishing": {
        "githubOrg": "myorg"
      }
    }

  5. Commit your changes to the liblab.config.json and API spec files.

  6. Push the changes to your control repo. This will trigger the pipeline that generates your SDKs and creates pull requests in your SDK repos.

6. Merge SDK PRs​

After the control repo pipeline is complete, a pull request (PR) will be automatically created in each SDK repo.

Go to Build > Pipelines or Build > Jobs to check the pipeline and job status in the control repo.

Log In to Complete the Job

When the pipeline starts a new job to build an SDK, the liblab CLI may require you to log in and confirm an authorization code. A link to the login page will be provided in the job logs within GitLab.

Running job

If you don't log in and confirm the code, the job may not be complete, and the SDK build will not be triggered.

After the job is completed successfully in the control repo, check the PRs generated by liblab in each SDK repo. To view the new PRs, access Code > Merge requests in each SDK repo.

New PR created by liblab CLI

note

If you don't see the PRs, check your control repo's CI/CD > Pipelines or Jobs section for any errors during the SDK generation process. You can also refer to the common errors section for troubleshooting.

To finish, follow the steps:

  1. Open each PR and review the changes to ensure the SDK was generated correctly.
  2. If everything looks correct, approve the PR and merge it into the main branch.

7. Create a release in GitLab​

You need to create a release in the SDK repo to trigger the Action to publish your SDK to the package manager. To create a release, follow the steps below:

  1. From each SDK repo, access Deploy > Releases and click Create a new release.

  2. On the release page:

    1. Provide a Tag name. The tag should be a version number using semantic versioning, and it is common to prefix the version with v, such as v1.0.0.
    Note on Versioning Go Packages

    It is important to note that the release tag version will be used as the version of the package in the Go Packages repo. Therefore, it is important to follow semantic versioning and ensure that the version in the liblab.config.json file corresponds to the version of the release tag.

    1. Give the release a Release title. The title can be the same as the tag.
    2. Add a Milestone. You can create one if necessary.
    3. Define the Release date and add the Release notes.

  3. Click Create release

This will trigger the pipeline to publish the SDK to the relevant package manager.

Release notes

8. Confirm SDK publication​

After completing the publishing process, check the package manager services related to the SDKs you have created, such as npm, PyPI, and Maven, to verify that your SDK has been successfully published.

🎉🎉🎉   Congratulations! You have successfully generated and published your SDK!   🎉🎉🎉

Your setup is now fully automated to generate and publish SDKs with liblab and GitLab CI/CD, ensuring your latest API updates are quickly available to your users.

If you don’t see the SDK in the package manager, check the CI/CD > Pipelines or Jobs section in each SDK repo to troubleshoot any errors during the publishing process.

Common errors​

If your repos are not configured correctly, you may see one of the following errors:

  • If the generate SDKs Action fails with this error:

    GitHub Actions output
    Error: Secrets LIBLAB_TOKEN and LIBLAB_GITLAB_TOKEN are required

    Then you haven't set up the LIBLAB_TOKEN or LIBLAB_GITLAB_TOKEN secrets correctly. Check that the tokens are set correctly, and that the tokens are valid.

Last updated on