Vertesia Documentation

Release Notes Generation

This page explains how to generate release notes using LLM and Vertesia. It is split into three sections:

  1. Data Collection: how to collect information using the Recipe, Memory Pack, and the vertesia CLI.
  2. Generation: how to generate release notes using Interaction and a Memory Pack in Vertesia
  3. Integration: how to integrate into GitHub Actions to further automate the generation

After reading this page, you should be able to:

  1. Write a Recipe to interact with other CLIs, such as git and gh (GitHub)
  2. Build a Memory Pack for the generation of release notes
  3. Write an Interaction to generate release notes with the use of a Memory Pack
  4. Use the vertesia command line to run the actual execution
  5. Automate the generation of release notes in GitHub Actions

Prerequisites

You need to install the latest version of vertesia CLI to execute the Recipe and build the Memory Pack. Also, it requires using a TTY terminal, and Node.js version 18 or higher.

npm -g install @vertesia/cli

Step 1: Data Collection

Before interacting with an LLM for the generation, you need to collect data. This section explains how to collect information from GitHub Issues and GitHub pull requests using a script, and how to store the collected results as a data snapshot. It explains different instructions related to the data collection and the internal structure of the snapshot. It also mentions how to implement similar solutions for other project management tools.

First of all, you need to understand two terminologies related to this step: Recipe and Memory Pack.

  • A Recipe is a text document written in TypeScript that contains all the instructions for building the data snapshot. A Recipe can be executed by the command line interface vertesia. This is similar to Dockerfile for those who are familiar with Docker.
  • A Memory Pack is a data snapshot that presents an immutable context for Large Language Models. A Memory Pack is portable: it can be used for multiple interactions with LLMs.

1.1 Writing the Recipe

A Recipe is a TypeScript file used to build a Memory Pack. It can contain any JavaScript code, but it must use the built-in Memory Commands to create the Memory Pack, a TAR file. You will see some of the instructions in the sections below. For the complete reference, please read the README of the Git repository vertesia/memory.

1.1.1 Importing Commands

Before running any commands in the Recipe, you can import those commands from the JavaScript library @vertesia/memory-commands. In this tutorial, we are going to use the following commands:

  • copy — copy a file to the Memory Pack.
  • copyText — copy an inline text into the Memory Pack a file
  • exec — execute a shell command
  • tmpdir — create a temporary directory
  • vars — retrieve the variables specified by the user

You can import them as follows:

import {
    copy,
    copyText,
    exec,
    tmpdir,
    vars,
} from "@vertesia/memory-commands";

1.1.2 Extracting Input Parameters

Then, you need to extract information from the variables. Here the variables "start" and "end" represent the range of the release notes, where "start" stands for the earliest Git reference or commit SHA for this release, and "end" is the newest Git reference or commit SHA for this release:

const { start, end } = vars();

Then we create a temporary directory as the current working directory for the data collection process. We can add temporary files into this directory and package them as a Memory Pack.

const cwd = tmpdir();

1.1.3 Collecting Information From Git

We use the Memory Command exec to execute a bash command to extract information from git. It retrieves all the commits between the “start” and the “end” of the release and shows the title of each commit in a one-line format. Then, we extract the issues and pull requests from the commit message. If a message contains a word starting with a hashtag followed by a sequence of numbers, it will be found by our command. For example, we can find “123” from the message “Fix #123”. You can imagine other implementations, but what matters is the possibility of invoking any arbitrary shell command using the exec instruction. This can be very useful for building a data pipeline.

const hashtagIds = await exec(`git log ${start}..${end} --oneline | grep -o -E '#[0-9]+' | sed 's/#//'`) as string;

Then, we can use built-in JavaScript syntax to perform string manipulation. Here, we split the text by the newlines (\n), convert them into numbers, and finally, sort the numbers in ascending order.

const unsortedReferenceIds = new Set(`${hashtagIds}`.trim().split("\n").map(v => v.trim()).map(Number));
const referenceIds = Array.from(unsortedReferenceIds).sort((a, b) => a - b);

You can also extract the code difference between the beginning and end of the release to see what has been changed. This can be useful for the LLM to improve the description of the items in the release notes based on the actual code changes. It can also improve the categorization of the changes because you can see the file paths and the function signatures there.

await exec(`git diff --submodule=diff ${start}...${end} -- ':!docs/changelog/*' > ${cwd}/range_diff.txt`)
copy(`${cwd}/range_diff.txt`, "range_diff.txt");

1.1.4 Collecting Information From GitHub

Here, we use the GitHub CLI gh to retrieve the content of each reference. A reference can be either a pull request or an issue. Since we don’t know the type, we try one type and fall back to the other if needed. More precisely, we use the gh pr view command to retrieve the content. If it is empty, the reference is an issue rather than a pull request. So we would try again using the gh issue view command. Then, we use the Memory Command copyText to copy the content into a file in the Memory Pack. The reference is injected into a string using template laterals.

for (const reference of referenceIds) {
    let content = await exec(`gh pr view ${reference}`) as string;
    if (content) {
        copyText(content, `pull_requests/${reference}.txt`);
    } else {
        content = await exec(`gh issue view ${reference}`) as string;
        copyText(content, `issues/${reference}.txt`);
        issueIds.delete(`${reference}`);
    }
}

1.1.5 Collecting Information From Other Sources

Since we can run arbitrary commands using the Memory Command exec, you can interact with other sources easily. For example, you can fetch information from JIRA using the JIRA CLI; you can fetch information from GitLab using GitLab CLI; and many more. This approach works well when you don’t need to manipulate the response payload returned by the command line. For example, you don’t need to extract information from JSON or YAML.

1.1.6 Exporting Metadata

At the end of the file, you must export the Memory Pack Metadata. This is a JSON object which holds the properties to be used when interacting with LLMs. In our case, we are exporting two properties: the from_version and the release_version which maps to the variables start and end of the Memory Pack.

1.2 Building the Memory Pack

Now, you can use the vertesia memo build command to build your memory pack. You can specify the location of your memory pack, which can be local or remote. If you use the memory: prefix, it will be stored in Google Cloud under the bucket of your project. Otherwise, it will be stored on your local host.

vertesia memo build \
    --out "memory:release_notes/my_package-1_2_0" \
    --var-start "my_package/v1.1.0" \
    --var-end "my_package/v1.2.0" \
    recipes/release-notes.ts

Note that you need to use prefix memory: for the production use-cases. Otherwise, Vertesia Cloud cannot read the data and inject them into the prompts when interacting with LLMs.

When you run the command, you should see logs like these in your console:

Retrieving issues between my_package/v1.1.0 and my_package/v1.2.0...
Running: git log my_package/v1.1.0..my_package/v1.2.0 --oneline
Running: grep -o -E #[0-9]+
Running: sed s/#//
Command: git log my_package/v1.1.0..my_package/v1.2.0 --oneline | grep -o -E '#[0-9]+' | sed 's/#//' exited with status 0
Running: git log my_package/v1.1.0..my_package/v1.2.0 --oneline
Running: grep -o -E \([0-9]+\)
Running: sed s/[()]//g
Command: git log my_package/v1.1.0..my_package/v1.2.0 --oneline | grep -o -E '\([0-9]+\)' | sed 's/[()]//g' exited with status 0
Found 182 references
Processing reference #359
Running: gh pr view 359
Command: gh pr view 359 exited with status 0
Processing reference #474
Running: gh pr view 474
...

Once created, you can inspect the Memory Pack using tools that can manipulate a TAR file, such as the tar command or any GUI tools.

1.3 Recap

In "Step 1: Data Collection", we saw the first part of the generation of release notes. We saw how to create a small data pipeline to collect information using Recipe, a standalone TypeScript file with Memory Commands. Then, we saw how to build the Memory Pack using the vertesia command. In the next step, you will learn how to implement the interaction with LLM.

Step 2: Generation

This section explains how to generate release notes with Vertesia Studio and Memory Packs. It will explain the features of Interaction and Prompts in Studio; it will explain how to use a Memory Pack inside an Interaction; and finally, how to run the Interaction from the command line.

After reading this page, you should be able to:

  • Write an Interaction to generate release notes with the use of a Memory Pack
  • Use the vertesia command line to run the actual execution

2.1 Create An Interaction

An Interaction is a reusable function that can be invoked to perform a specific task related to large language models (LLMs). It contains one or multiple prompt segments, and uses a model for the execution. The prompts are used to provide context, instructions, and information related to input and output. They can be shared between different interactions for re-usability.

In the case of the generation of release notes, we can have two prompts: one system prompt for providing the context and a user prompt for describing the instructions.

2.1.1 Provide Context

You can use a system prompt to create the context of the generation. You can give a persona to the prompt as the initial information. This can be relevant background or scenario, which helps the model generate responses that align with the desired context. For example, you can mention your company's name and mission; you can require the LLM to play the role of a Release Manager, specializing in Software-as-a-Service (SaaS) products; you can specify the target audience; etc.

2.1.2 Provide Instructions

Then, you can use a user prompt to specify the instructions, the input data, and the output format for the generation.

For the instructions, you can specify the actions and logic you want the LLM to perform. For example, you can specify the sections of the release notes, e.g. “New Features”, “Major Changes”, “Improvement and Fixes”, based on labels of your project management tools (e.g. JIRA or GitHub Issues), code path, applications, the importance of the changes, etc. You can specify the structure inside a section and how each item is computed. You can also provide specific treatment for the LLM for content generation, such as how to handle the situation when one product change is matched to multiple pull requests, when one commit has multiple authors, etc.

Note that you can use Prompt Schema to inject variables into your prompt when choosing the content type as “JS Template”. This is useful for making the instructions accurate to your current task. For example, you can inject previous_version, release_version, to specify the range of the changes.

return `
## Instructions

You need to write release notes on release ${release_version}, highlighting
changes between the current version ${release_version} and the previous version
${previous_version}.

...
`

2.1.3 Provide Input Data

For the input data, you can specify all kinds of information related to the release notes, including the pull requests, the commit log, the code difference, or anything else you feel is relevant to the generation. Similar to the instructions, they can be defined in Prompt Schema and injected into a JS Template. These variables are injected during the execution as parameters. Here is what it looks like:

return `...

## Input Data

Here are the input data you can take into account for the content generation,
including Git commits, code differences, GitHub pull requests, and GitHub issues.

Here are the commits between the version "${previous_version}" and the version
"${release_version}", generated by the git-log command:

<commits>
${commits}
</commits>

Here is the code difference between the version "${previous_version}" version and
the version "${release_version}", generated by the git-diff command:

<code_diff>
${code_diff}
</code_diff>

...
`

But now imagine that you need to provide hundreds of commits, thousands of lines of code changes, and other pieces of information before running the Interaction. It is pretty challenging. You may also need to deal with data serialization issues if you run the command in your terminal. It's painful and not part of your business logic.

With the Memory Pack, it simplifies the data injection process. When running the Interaction from the vertesia CLI, you can use a specific entry @memory to provide the path of the Memory Pack to use, relative to your project, to provide all the pieces of information collected earlier. Then, you can map those entries to the prompts. For example, mapping all the entries of the Memory Pack using the expression "@": "@", or mapping the content of the commit log to the variable commits using the mapping "commits": "@content:commits.txt". We will see more details later on this page.

2.1.4 Provide Output Indicator

In the prompt, you can also specify the structure of the generated content. This can be related to the length of the description of each change, the metadata to be included (application, link of the ticket, author, date, ...), the styling (bold, italic), the structure of the content, or any other instructions for the output. Giving those details allows you to have more control over the format.

2.1.5 Publish Interaction

Once you are satisfied with the Interaction. You can publish it so that you can run the Interaction from the CLI.

2.2 Run Interaction

You can run your Interaction by full name. The full name is composed of an optional namespace, a required endpoint name, and an optional tag or version. Examples: name, namespace:name, namespace:name@version:

vertesia run [options] <interaction>

In the case of the release notes, you need to provide mappings as inline data via the option -d, --data. Here is an example:

mappings=$(cat << EOF
{
    "@memory": "release-notes/${GITHUB_RUN_ID}-${GITHUB_SHA::7}",
    "@": "@",
    "issues": "@content:issues/*",
    "pull_requests": "@content:pull_requests/*",
    "code_diff": "@content:range_diff.txt",
    "commits": "@content:commits.txt"
}
EOF
)

vertesia run GenerateReleaseNotes -d "$mappings"

In this example, we have:

  • @memory: the path of the Memory Pack (release-notes/${GITHUB_RUN_ID}-${GITHUB_SHA::7}). It is the location where the Memory Pack is stored. It is relative the root of the cloud storage bucket (Google Cloud Storage, Amazon S3) specified when you built your Memory Pack during the data collection process. Here, we use the GitHub Actions Run ID combined with the GitHub commit SHA to provide additional build time metadata for the Memory Pack.
  • @: the content mappings. It matches all the export attributes defined in the Memory Pack to the attributes with the same name, defined in the Parameters Schema of the Interaction.
  • issues: a mapping for the multi-value string field "issues" (string[]). This mapping extracts the content of each file located under the path issues/ of the Memory Pack, and map the content as a string for an element in the array.
  • code_diff: a mapping for the simple string field "code_diff" (string). This mapping extracts the content of the range_diff.txt file and assigns it to the variable code_diff.
  • And similar mappings for other fields

If you need to troubleshoot the execution, you can visit the "Runs" tab in Vertesia Studio. It provides detailed information about the input data, output result, the schema, the run status, etc.

2.3 Recap

In this section, we saw the second part of the generation of release notes: the use of LLM. We saw how to create an Interaction using Prompts with context, instructions, input data, and output indicators. We also saw how to run the Interaction using the vertesia command line. In particular, how to run the execution with a Memory Pack.

Step 3: GitHub Integration

In this section, we will discuss how to integrate the generation of release notes into GitHub Actions, a popular CI solution. We will see how to set up vertesia CLI in GitHub Actions. Then, we will discuss how to use it to build the Memory Pack and run the Interaction to generate the content.

3.1 Generating API Key

You need to create an API key in your Vertesia project (https://cloud.vertesia.io) before using Vertesia's service in GitHub Actions.

  • Go to the page "Settings" on the left sidebar, then go to the tab "API Keys"
  • Create a new API key, such as "Release Notes Generation", with the role executor.

More precisely, the role executor gives the API permission to upload Memory Packs and execute Interactions. You can also use a more permissive role, but the role must be authorized to perform these two operations. The type should be "Secret Key (sk)" and be meant to be used in a private location, not from a browser or any public location that exposes the secret.

3.2 Automate Generation with GitHub Actions

Once the API key is created, the next step is implementing the GitHub Actions to automate the generation. In this section, we will go through the most important actions. These code blocks are code snippets of a GitHub Workflow file.

3.2.1 Prepare Input Parameters

We need two parameters for generating the release notes: the target tag and the previous tag. The target tag is the release target, a commit SHA or a Git reference (tag, branch) for which we want to write the release notes. The previous tag is the previous version that had been released, which serves as the reference for the comparison.

on:
  workflow_dispatch:
    inputs:
      target_tag:
        description: Target tag
        required: true
        type: string
      previous_tag:
        description: Previous tag
        required: true
        type: string

3.2.2 Set up Vertesia CLI

You need to set up Node.js to your CI environment. This is a prerequisite for installing the vertesia CLI.

- uses: actions/setup-node@v4
  with:
    node-version: 22

Once done, you can install the vertesia CLI globally:

- run: npm install -g @vertesia/cli

Then, you need to add a profile to the vertesia so that you can interact with your project. As of 27 Jan 2025, we don't support initializing a new profile from a headless browser. Therefore, you have to initialize it manually by adding the following structure into the ~/.vertesia/profiles.json file:

cat <<EOF >> ~/.vertesia/profiles.json
{
    "default": "github-actions",
    "profiles": [
        {
            "name": "github-actions",
            "config_url": "https://cloud.vertesia.io/cli",
            "account": "${VT_ACCOUNT}",
            "project": "${VT_PROJECT}",
            "studio_server_url": "https://studio-server-production.api.vertesia.io",
            "zeno_server_url": "https://zeno-server-production.api.vertesia.io",
            "apikey": "${VT_API_KEY}"
        }
    ]
}
EOF

where:

  • VT_ACCOUNT is the account ID in Vertesia
  • VT_PROJECT is the project ID in Vertesia
  • VT_API_KEY is the API key generated in Vertesia

You can find those pieces of information in your Vertesia project.

The API key is a sensitive information. There are several ways to inject it securely into your GitHub Actions. You can use the GitHub Secrets to store it as a repository secret or an organization secret, then you can use it as an environment variable. Or, you can rely on third party services to store the sensitive information, such as using Google Secret Manager, Amazon Secret Manager, Amazon Systems Manager Paramter Store, etc. Here is an example using secrets in GitHub:

- name: Set up vertesia CLI
  env:
    vertesia_api_key: ${{ secrets.VT_API_KEY }}
  run: ...

Once done, you can use the following action to verify if the profile is correctly recognized by the CLI:

- run: vertesia profiles

And you are expected to see an active profile "github-actions" being created:

github-actions ✔

3.2.3 Gather information

Now you are ready to build the Memory Pack, using the vertesia memo build command we mentioned in the previous tutorial.

- name: Gather information
  run: |
    vertesia memo build \
        --out "memory:release-notes/${GITHUB_RUN_ID}-${GITHUB_SHA::7}" \
        --var-start "${{ env.PREVIOUS_TAG }}" \
        --var-end "${{ env.TARGET_TAG }}" \
        examples/api-doc-gen/recipes/release-notes.ts

where:

  • the Memory Pack will be uploaded to your Vertesia bucket under the path release-notes/ followed by the GitHub Run ID and the commit SHA.
  • the start and end variables define the range of the release notes

3.2.4 Run Interaction

Once the Memory Pack is uploaded to the Google Cloud, you can use it to run the Interaction to generate the release notes. Here is an example:

- name: Generate release notes
  run: |
    mappings=$(cat << EOF
    {
        "@memory": "release-notes/${GITHUB_RUN_ID}-${GITHUB_SHA::7}",
        "@": "@",
        "lang": "${lang}",
        "issues": "@content:issues/*",
        "pull_requests": "@content:pull_requests/*",
        "code_diff": "@content:range_diff.txt",
        "commits": "@content:commits.txt"
    }
    EOF
    )

    vertesia run GenerateReleaseNotes -d "$mappings"

The generated content will be printed in the GitHub workflow. It's up to you to handle the logic. For example, you may want to store the result into a file, and then create a pull-request to a human review before publishing it. Or you may want to store the generated result directly as part of the "release" on top of an existing tag on GitHub.

3.3 Recap

In this section, we saw how to generate the API key from Vertesia UI. Then, we saw how to use vertesia CLI to build the Memory Pack and run the Interaction to generate the content in GitHub Actions.

Conclusion

Thanks for spending time to read the whole tutorial! In this page, we went through the 3 parts of the release notes generation, including the data collection, document generation, and the integration to GitHub Actions. We saw how to create a small data pipeline to collect information using Recipe, a standalone TypeScript file with Memory Commands. Then, we saw how to build the Memory Pack using the vertesia command. We saw how to create an Interaction and run it using the vertesia CLI with a Memory Pack. Finally, we saw how to generate the API key from Vertesia UI. Then, we saw how to integrate the whole process into GitHub Actions.

Was this page helpful?