Skip to main content

Pagination

Pagination is a technique for managing large datasets by breaking them into smaller, manageable chunks called "pages." Instead of loading all data at once, pagination allows you to retrieve data incrementally, improving performance and user experience.

liblab automatically handles pagination in generated SDKs, eliminating the need for manual query management. The SDK provides two simple patterns:

  • Iterable pagination: Automatically iterate through all pages using a simple for loop
  • Manual pagination: Fine-grained control with .next() to fetch pages on demand

Pagination types​

liblab supports two pagination strategies:

Pagination TypeLanguagesDescription
Offset-LimitTypeScript, Java, PythonUses numeric offset and limit parameters to navigate through data. Simple and widely supported.
CursorTypeScript, Java, PythonUses opaque cursor tokens to maintain position in dataset. Provides better consistency and performance for frequently-changing data.
Choosing a pagination type

Use offset-limit pagination for general use cases where data is relatively stable. Use cursor pagination when you need consistent results as data changes, better performance with large datasets, or the ability to resume pagination from saved positions.

How pagination works​

When you configure pagination for an endpoint in your OpenAPI spec or liblab config file, the generated SDK automatically:

  1. Manages pagination parameters: Handles offset/limit or cursor tokens internally
  2. Detects end of data: Knows when there are no more pages to fetch
  3. Provides simple iteration: Exposes clean APIs for both automatic and manual pagination
  4. Handles errors: Manages failed requests gracefully

You configure pagination once, and the SDK handles all the complexity.

Configuring pagination​

liblab offers two methods to configure pagination:

  1. Using the OpenAPI spec: Add the x-liblab-pagination annotation directly to your spec endpoint
  2. Using liblab.config.json: Configure pagination in the config file without modifying your spec

Both methods achieve the same result. Choose the approach that fits your workflow.

Configuration details

For complete configuration options and step-by-step setup, see:

Offset-Limit pagination​

Supported SDK languages:

TypeScript /
Javascript		Java /
Kotlin		PythonC#GoPHP
✅✅✅❌❌❌

Offset-limit pagination uses two numeric parameters to navigate through data:

  • Offset: The number of items to skip (starting position)
  • Limit: The maximum number of items to return per page

Configuration example​

openapi.yaml
paths:
  /pets:
    get:
      operationId: listPets
      x-liblab-pagination:
        type: offsetLimit
        inputFields:
          - name: offset
            in: query
            type: offset
          - name: limit
            in: query
            type: limit
        resultsArray:
          results: "$.pets"

Usage patterns​

The SDK provides two ways to use offset-limit pagination:

Iterable pagination​

Automatically iterate through all pages:

const sdk = new YourSdk({ baseUrl: 'https://api.example.com' });

// Automatically iterate through all pages
for await (const page of sdk.pets.listPets({ limit: 20, offset: 0 })) {
  console.log(`Page data:`, page.data);
}

Manual pagination​

Control when to fetch each page:

const sdk = new YourSdk({ baseUrl: 'https://api.example.com' });

const pages = sdk.pets.listPets({ limit: 20, offset: 0 });

// Fetch first page
const page1 = await pages.next();
console.log('First page:', page1.value.data);

// Fetch second page
const page2 = await pages.next();
console.log('Second page:', page2.value.data);

When to use offset-limit​

Offset-limit pagination is ideal when:

  • Your dataset is relatively stable (items aren't frequently added/removed)
  • You need simple numeric page navigation
  • You want broad language support (TypeScript, Java, Python)
  • Performance with large offsets is acceptable

Cursor pagination​

Supported SDK languages:

TypeScript /
Javascript		Java /
Kotlin		PythonC#GoPHP
✅✅✅❌❌❌

Cursor pagination uses opaque tokens instead of numeric offsets to maintain position in a dataset. Each response includes a cursor token that points to the next page.

Benefits​

  • Consistency: No duplicates or missing items when data changes during iteration
  • Performance: Maintains consistent speed regardless of position in dataset
  • Resume capability: Save and resume pagination from any cursor position
  • Reliability: Handles real-time data changes gracefully

Configuration example​

openapi.yaml
paths:
  /organizations:
    get:
      operationId: listOrganizations
      x-liblab-pagination:
        type: cursor
        inputFields:
          - name: page_token
            in: query
            type: cursor
        resultsArray:
          results: "$.items"
          nextCursor: "$.next_page_token"

Usage patterns​

Cursor pagination works identically to offset-limit from a code perspective:

Iterable pagination​

const sdk = new YourSdk({ baseUrl: 'https://api.example.com' });

// Automatically iterate through all pages using cursor tokens
for await (const page of sdk.organizations.listOrganizations()) {
  console.log(`Got ${page.data.length} organizations`);
}

Key points:

  • The SDK automatically manages cursor tokens
  • You never see or handle cursor values directly
  • Pagination ends when the API returns no next_page_token

Manual pagination​

const sdk = new YourSdk({ baseUrl: 'https://api.example.com' });

const pages = sdk.organizations.listOrganizations();

// Fetch first page
const page1 = await pages.next();
if (!page1.done) {
  console.log('First page:', page1.value.data);
}

// Fetch second page
const page2 = await pages.next();
if (!page2.done) {
  console.log('Second page:', page2.value.data);
}

Resume pagination​

A unique feature of cursor pagination is resuming from a saved position:

const sdk = new YourSdk({ baseUrl: 'https://api.example.com' });

// Resume from a previously saved cursor token
const savedCursor = 'eyJsYXN0X2lkIjoxMjM0fQ==';

for await (const page of sdk.organizations.listOrganizations({
  pageToken: savedCursor  // Resume from here
})) {
  console.log('Resumed page:', page.data);
}

This enables:

  • Pausing and resuming pagination across sessions
  • Implementing checkpointing for long-running operations
  • Graceful failure recovery

When to use cursor pagination​

Cursor pagination is ideal when:

  • Your dataset changes frequently (items added/removed in real-time)
  • You need consistent results across pagination requests
  • Performance at scale matters (large datasets with deep pagination)
  • You want to implement resume/checkpoint functionality
  • Your application requires reliable pagination

Best practices​

Configuration​

  1. Match your API's parameter names: Use the exact parameter names your API expects
  2. Set correct JSONPath: Ensure resultsArray.results points to the array in your response
  3. Test pagination: Verify pagination works with your API before releasing

Usage​

  1. Use iterable pagination by default: It's simpler and handles most use cases
  2. Use manual pagination for control: When you need to limit requests or implement custom logic
  3. Handle errors gracefully: Wrap pagination in try-catch blocks
  4. Consider page size: Balance between request count and data volume

Performance​

  1. Choose appropriate page sizes: Too small = many requests, too large = slow responses
  2. Use cursor pagination for large datasets: Better performance than offset-limit with deep pagination
  3. Implement resume logic: Save cursor positions for long-running operations

Examples and tutorials​

Last updated on