-
Notifications
You must be signed in to change notification settings - Fork 11
CLOUDP-295785 - Calculate next version and release notes script #193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
MaciejKaras
wants to merge
30
commits into
master
Choose a base branch
from
maciejk/ar-versioning
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
52ba697
WIP: changelog and versioning methods
MaciejKaras 04bff85
WIP: generate_changelog func
MaciejKaras 4f4e2d8
Working release notes generation
MaciejKaras 1ea4dd7
Added tests for release notes generation
MaciejKaras 0df481a
Release with breaking change test
MaciejKaras 79916bb
Added more releases
MaciejKaras 9a664a0
Added release branch test cases
MaciejKaras 4ebf3ab
Get the previous version based on current HEAD
MaciejKaras b55b748
Added tests, gitgraph, docs and cmd input
MaciejKaras 0977ac5
Add main method in versioning.py
MaciejKaras 9b37d49
Move main method to calculate_next_version.py
MaciejKaras 54707c1
Optimize imports
MaciejKaras 55bcbc0
Lint fix
MaciejKaras 9b176e3
Add changelog entry frontmatter text
MaciejKaras 2cacd12
Added frontmatter validation
MaciejKaras ab8a217
Script for generating changelog file
MaciejKaras 36bd5db
Review fixes
MaciejKaras 0f98136
Review fixes v2
MaciejKaras e0078a7
Review fixes v3
MaciejKaras 1fc658d
Review fixes v4
MaciejKaras da1849e
Using ChangeEntry type
MaciejKaras a2c1bcd
Making release a module
MaciejKaras 5a5018b
Fixing other kind of change issue + missing tests
MaciejKaras e33b344
Adding quotes to error message variables
MaciejKaras 471bb5c
remove venv from .gitignore
MaciejKaras 0d02f6f
fix unit tests
MaciejKaras 6ca498b
Update scripts/release/create_changelog.py
MaciejKaras 2f9df2c
Update scripts/release/changelog.py
MaciejKaras 6dd61e2
Update scripts/release/release_notes.py
MaciejKaras eb14e62
Review fixes 1
MaciejKaras File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Makes 'release' a Python package. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
import argparse | ||
import pathlib | ||
|
||
from git import Repo | ||
|
||
from scripts.release.changelog import ( | ||
DEFAULT_CHANGELOG_PATH, | ||
DEFAULT_INITIAL_GIT_TAG_VERSION, | ||
) | ||
from scripts.release.release_notes import calculate_next_version_with_changelog | ||
|
||
if __name__ == "__main__": | ||
parser = argparse.ArgumentParser( | ||
description="Calculate the next version based on the changes since the previous version tag.", | ||
formatter_class=argparse.RawTextHelpFormatter, | ||
) | ||
parser.add_argument( | ||
"-p", | ||
"--path", | ||
default=".", | ||
metavar="", | ||
action="store", | ||
type=pathlib.Path, | ||
help="Path to the Git repository. Default is the current directory '.'", | ||
) | ||
parser.add_argument( | ||
"-c", | ||
"--changelog-path", | ||
default=DEFAULT_CHANGELOG_PATH, | ||
metavar="", | ||
action="store", | ||
type=str, | ||
help=f"Path to the changelog directory relative to the repository root. Default is '{DEFAULT_CHANGELOG_PATH}'", | ||
) | ||
parser.add_argument( | ||
"-s", | ||
"--initial-commit-sha", | ||
metavar="", | ||
action="store", | ||
type=str, | ||
help="SHA of the initial commit to start from if no previous version tag is found.", | ||
) | ||
parser.add_argument( | ||
"-v", | ||
"--initial-version", | ||
default=DEFAULT_INITIAL_GIT_TAG_VERSION, | ||
metavar="", | ||
action="store", | ||
type=str, | ||
help=f"Version to use if no previous version tag is found. Default is '{DEFAULT_INITIAL_GIT_TAG_VERSION}'", | ||
) | ||
args = parser.parse_args() | ||
|
||
repo = Repo(args.path) | ||
|
||
version, _ = calculate_next_version_with_changelog( | ||
repo, args.changelog_path, args.initial_commit_sha, args.initial_version | ||
) | ||
|
||
print(version) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,159 @@ | ||
import datetime | ||
import os | ||
import re | ||
from enum import StrEnum | ||
|
||
import frontmatter | ||
from git import Commit, Repo | ||
|
||
DEFAULT_CHANGELOG_PATH = "changelog/" | ||
DEFAULT_INITIAL_GIT_TAG_VERSION = "1.0.0" | ||
FILENAME_DATE_FORMAT = "%Y%m%d" | ||
FRONTMATTER_DATE_FORMAT = "%Y-%m-%d" | ||
MAX_TITLE_LENGTH = 50 | ||
|
||
PRELUDE_ENTRIES = ["prelude"] | ||
BREAKING_CHANGE_ENTRIES = ["breaking", "major"] | ||
FEATURE_ENTRIES = ["feat", "feature"] | ||
BUGFIX_ENTRIES = ["fix", "bugfix", "hotfix", "patch"] | ||
|
||
|
||
class ChangeKind(StrEnum): | ||
PRELUDE = "prelude" | ||
BREAKING = "breaking" | ||
FEATURE = "feature" | ||
FIX = "fix" | ||
OTHER = "other" | ||
|
||
|
||
class ChangeEntry: | ||
def __init__(self, date: datetime, kind: ChangeKind, title: str, contents: str): | ||
self.date = date | ||
self.kind = kind | ||
self.title = title | ||
self.contents = contents | ||
|
||
|
||
def get_changelog_entries( | ||
base_commit: Commit, | ||
repo: Repo, | ||
changelog_sub_path: str, | ||
) -> list[ChangeEntry]: | ||
changelog = [] | ||
|
||
# Compare base commit with current working tree | ||
diff_index = base_commit.diff(other=repo.head.commit, paths=changelog_sub_path) | ||
|
||
# No changes since the previous version | ||
if not diff_index: | ||
return changelog | ||
|
||
# Traverse added Diff objects only (change type 'A' for added files) | ||
for diff_item in diff_index.iter_change_type("A"): | ||
file_path = diff_item.b_path | ||
|
||
change_entry = extract_changelog_entry(repo.working_dir, file_path) | ||
changelog.append(change_entry) | ||
|
||
return changelog | ||
|
||
|
||
def extract_changelog_entry(working_dir: str, file_path: str) -> ChangeEntry: | ||
file_name = os.path.basename(file_path) | ||
date, kind = extract_date_and_kind_from_file_name(file_name) | ||
|
||
abs_file_path = os.path.join(working_dir, file_path) | ||
with open(abs_file_path, "r") as file: | ||
file_content = file.read() | ||
|
||
change_entry = extract_changelog_entry_from_contents(file_content) | ||
|
||
if change_entry.date != date: | ||
raise Exception( | ||
f"{file_name} - date in front matter '{change_entry.date}' does not match date extracted from file name '{date}'" | ||
) | ||
|
||
if change_entry.kind != kind: | ||
raise Exception( | ||
f"{file_name} - kind in front matter '{change_entry.kind}' does not match kind extracted from file name '{kind}'" | ||
) | ||
|
||
return change_entry | ||
|
||
|
||
def extract_date_and_kind_from_file_name(file_name: str) -> (datetime, ChangeKind): | ||
match = re.match(r"(\d{8})_([a-zA-Z]+)_(.+)\.md", file_name) | ||
if not match: | ||
raise Exception(f"{file_name} - doesn't match expected pattern") | ||
|
||
date_str, kind_str, _ = match.groups() | ||
try: | ||
date = parse_change_date(date_str, FILENAME_DATE_FORMAT) | ||
except Exception as e: | ||
raise Exception(f"{file_name} - {e}") | ||
|
||
kind = get_change_kind(kind_str) | ||
|
||
return date, kind | ||
|
||
|
||
def parse_change_date(date_str: str, date_format: str) -> datetime: | ||
try: | ||
date = datetime.datetime.strptime(date_str, date_format).date() | ||
except Exception: | ||
raise Exception(f"date '{date_str}' is not in the expected format {date_format}") | ||
|
||
return date | ||
|
||
|
||
def get_change_kind(kind_str: str) -> ChangeKind: | ||
if kind_str.lower() in PRELUDE_ENTRIES: | ||
return ChangeKind.PRELUDE | ||
if kind_str.lower() in BREAKING_CHANGE_ENTRIES: | ||
return ChangeKind.BREAKING | ||
elif kind_str.lower() in FEATURE_ENTRIES: | ||
return ChangeKind.FEATURE | ||
elif kind_str.lower() in BUGFIX_ENTRIES: | ||
return ChangeKind.FIX | ||
return ChangeKind.OTHER | ||
|
||
|
||
def extract_changelog_entry_from_contents(file_contents: str) -> ChangeEntry: | ||
data = frontmatter.loads(file_contents) | ||
|
||
kind = get_change_kind(str(data["kind"])) | ||
date = parse_change_date(str(data["date"]), FRONTMATTER_DATE_FORMAT) | ||
## Add newline to contents so the Markdown file also contains a newline at the end | ||
contents = data.content + "\n" | ||
|
||
return ChangeEntry(date=date, title=str(data["title"]), kind=kind, contents=contents) | ||
|
||
|
||
def get_changelog_filename(title: str, kind: ChangeKind, date: datetime) -> str: | ||
sanitized_title = sanitize_title(title) | ||
filename_date = datetime.datetime.strftime(date, FILENAME_DATE_FORMAT) | ||
|
||
return f"{filename_date}_{kind}_{sanitized_title}.md" | ||
|
||
|
||
def sanitize_title(title: str) -> str: | ||
# Only keep alphanumeric characters, dashes, underscores and spaces | ||
regex = re.compile("[^a-zA-Z0-9-_ ]+") | ||
title = regex.sub("", title) | ||
|
||
# Replace multiple dashes, underscores and spaces with underscores | ||
regex_underscore = re.compile("[-_ ]+") | ||
title = regex_underscore.sub(" ", title).strip() | ||
|
||
# Lowercase and split by space | ||
words = [word.lower() for word in title.split(" ")] | ||
|
||
result = words[0] | ||
|
||
for word in words[1:]: | ||
if len(result) + len("_") + len(word) <= MAX_TITLE_LENGTH: | ||
result = result + "_" + word | ||
else: | ||
break | ||
|
||
return result |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,138 @@ | ||
import datetime | ||
import unittest | ||
|
||
from scripts.release.changelog import ( | ||
MAX_TITLE_LENGTH, | ||
ChangeKind, | ||
extract_changelog_entry_from_contents, | ||
extract_date_and_kind_from_file_name, | ||
sanitize_title, | ||
) | ||
|
||
|
||
class TestExtractChangelogDataFromFileName(unittest.TestCase): | ||
def test_prelude(self): | ||
date, kind = extract_date_and_kind_from_file_name("20250502_prelude_release_notes.md") | ||
self.assertEqual(date, datetime.date(2025, 5, 2)) | ||
self.assertEqual(kind, ChangeKind.PRELUDE) | ||
|
||
def test_breaking_changes(self): | ||
date, kind = extract_date_and_kind_from_file_name("20250101_breaking_api_update.md") | ||
self.assertEqual(date, datetime.date(2025, 1, 1)) | ||
self.assertEqual(kind, ChangeKind.BREAKING) | ||
|
||
date, kind = extract_date_and_kind_from_file_name("20250508_breaking_remove_deprecated.md") | ||
self.assertEqual(date, datetime.date(2025, 5, 8)) | ||
self.assertEqual(kind, ChangeKind.BREAKING) | ||
|
||
date, kind = extract_date_and_kind_from_file_name("20250509_major_schema_change.md") | ||
self.assertEqual(date, datetime.date(2025, 5, 9)) | ||
self.assertEqual(kind, ChangeKind.BREAKING) | ||
|
||
def test_features(self): | ||
date, kind = extract_date_and_kind_from_file_name("20250509_feature_new_dashboard.md") | ||
self.assertEqual(date, datetime.date(2025, 5, 9)) | ||
self.assertEqual(kind, ChangeKind.FEATURE) | ||
|
||
date, kind = extract_date_and_kind_from_file_name("20250511_feat_add_metrics.md") | ||
self.assertEqual(date, datetime.date(2025, 5, 11)) | ||
self.assertEqual(kind, ChangeKind.FEATURE) | ||
|
||
def test_fixes(self): | ||
date, kind = extract_date_and_kind_from_file_name("20251210_fix_olm_missing_images.md") | ||
self.assertEqual(date, datetime.date(2025, 12, 10)) | ||
self.assertEqual(kind, ChangeKind.FIX) | ||
|
||
date, kind = extract_date_and_kind_from_file_name("20251010_bugfix_memory_leak.md") | ||
self.assertEqual(date, datetime.date(2025, 10, 10)) | ||
self.assertEqual(kind, ChangeKind.FIX) | ||
|
||
date, kind = extract_date_and_kind_from_file_name("20250302_hotfix_security_issue.md") | ||
self.assertEqual(date, datetime.date(2025, 3, 2)) | ||
self.assertEqual(kind, ChangeKind.FIX) | ||
|
||
date, kind = extract_date_and_kind_from_file_name("20250301_patch_typo_correction.md") | ||
self.assertEqual(date, datetime.date(2025, 3, 1)) | ||
self.assertEqual(kind, ChangeKind.FIX) | ||
|
||
def test_other(self): | ||
date, kind = extract_date_and_kind_from_file_name("20250520_docs_update_readme.md") | ||
self.assertEqual(date, datetime.date(2025, 5, 20)) | ||
self.assertEqual(kind, ChangeKind.OTHER) | ||
|
||
date, kind = extract_date_and_kind_from_file_name("20250610_refactor_codebase.md") | ||
self.assertEqual(date, datetime.date(2025, 6, 10)) | ||
self.assertEqual(kind, ChangeKind.OTHER) | ||
|
||
def test_invalid_date(self): | ||
with self.assertRaises(Exception) as context: | ||
extract_date_and_kind_from_file_name("20250640_refactor_codebase.md") | ||
self.assertEqual( | ||
str(context.exception), | ||
"20250640_refactor_codebase.md - date '20250640' is not in the expected format %Y%m%d", | ||
) | ||
|
||
def test_wrong_file_name_format_date(self): | ||
with self.assertRaises(Exception) as context: | ||
extract_date_and_kind_from_file_name("202yas_refactor_codebase.md") | ||
self.assertEqual(str(context.exception), "202yas_refactor_codebase.md - doesn't match expected pattern") | ||
|
||
def test_wrong_file_name_format_missing_title(self): | ||
with self.assertRaises(Exception) as context: | ||
extract_date_and_kind_from_file_name("20250620_change.md") | ||
self.assertEqual(str(context.exception), "20250620_change.md - doesn't match expected pattern") | ||
|
||
|
||
def test_strip_changelog_entry_frontmatter(): | ||
file_contents = """ | ||
--- | ||
title: This is my change | ||
kind: feature | ||
date: 2025-07-10 | ||
--- | ||
|
||
* **MongoDB**: public search preview release of MongoDB Search (Community Edition) is now available. | ||
* Added new property [spec.search](https://www.mongodb.com/docs/kubernetes/current/mongodb/specification/#spec-search) to enable MongoDB Search. | ||
""" | ||
|
||
change_entry = extract_changelog_entry_from_contents(file_contents) | ||
|
||
assert change_entry.title == "This is my change" | ||
assert change_entry.kind == ChangeKind.FEATURE | ||
assert change_entry.date == datetime.date(2025, 7, 10) | ||
assert ( | ||
change_entry.contents | ||
== """* **MongoDB**: public search preview release of MongoDB Search (Community Edition) is now available. | ||
* Added new property [spec.search](https://www.mongodb.com/docs/kubernetes/current/mongodb/specification/#spec-search) to enable MongoDB Search. | ||
""" | ||
) | ||
|
||
|
||
class TestSanitizeTitle(unittest.TestCase): | ||
def test_basic_case(self): | ||
self.assertEqual(sanitize_title("Simple Title"), "simple_title") | ||
|
||
def test_non_alphabetic_chars(self): | ||
self.assertEqual(sanitize_title("Title tha@t-_ contain's strange char&s!"), "title_that_contains_strange_chars") | ||
|
||
def test_with_numbers_and_dashes(self): | ||
self.assertEqual(sanitize_title("Title with 123 numbers to-go!"), "title_with_123_numbers_to_go") | ||
|
||
def test_mixed_case(self): | ||
self.assertEqual(sanitize_title("MiXeD CaSe TiTlE"), "mixed_case_title") | ||
|
||
def test_length_limit(self): | ||
long_title = "This is a very long title that should be truncated because it exceeds the maximum length" | ||
sanitized_title = sanitize_title(long_title) | ||
self.assertTrue(len(sanitized_title) <= MAX_TITLE_LENGTH) | ||
self.assertEqual(sanitized_title, "this_is_a_very_long_title_that_should_be_truncated") | ||
|
||
def test_leading_trailing_spaces(self): | ||
sanitized_title = sanitize_title(" Title with spaces ") | ||
self.assertEqual(sanitized_title, "title_with_spaces") | ||
|
||
def test_empty_title(self): | ||
self.assertEqual(sanitize_title(""), "") | ||
|
||
def test_only_non_alphabetic(self): | ||
self.assertEqual(sanitize_title("!@#"), "") |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please add a more detailed comment here or a docstring to the function explaining how this works? E.g. why we only getting added files? How do we account for modified changelog entries (e.g. added in one commit, modified in another)? Deleted?
This will be extremity helpful for the future us who will be maintaining the tool/making changes here.