Skip to content

Implement Base Topics Functionality #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 31 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6f43730
Implement Base Topics Functionality
tmiddlet2666 Jul 3, 2025
72584ca
Fix some CI/CD issues
tmiddlet2666 Jul 3, 2025
184dfc1
Remove duplication
tmiddlet2666 Jul 3, 2025
5949c92
Add DAPR Tests
tmiddlet2666 Jul 4, 2025
1af9588
Fic DAPR test
tmiddlet2666 Jul 4, 2025
e4609b9
Add dapr init
tmiddlet2666 Jul 4, 2025
b19b2b2
Add the CLI to show the caches
tmiddlet2666 Jul 4, 2025
b3203dd
Update cli address to 127.0.0.1
tmiddlet2666 Jul 4, 2025
e011048
Fix dapr test script
tmiddlet2666 Jul 4, 2025
11a755e
Add check to make sure coherence is up
tmiddlet2666 Jul 4, 2025
2abf3c1
More tests for topics
tmiddlet2666 Jul 4, 2025
f575dd2
Use current coherence-go-client in dapr build
tmiddlet2666 Jul 5, 2025
b89197d
Use current coherence-go-client in dapr build
tmiddlet2666 Jul 5, 2025
7d925f8
Use current coherence-go-client in dapr build
tmiddlet2666 Jul 5, 2025
b0d02d8
Use current coherence-go-client in dapr build
tmiddlet2666 Jul 5, 2025
630eab5
Add subscriber group commands
tmiddlet2666 Jul 7, 2025
7d63aa2
Minor changes to topics
tmiddlet2666 Jul 7, 2025
1d7d0f6
Fix issue with subscriber test
tmiddlet2666 Jul 7, 2025
6f25d5e
Add tls tests for dapr
tmiddlet2666 Jul 8, 2025
410b3dc
Correct tls test
tmiddlet2666 Jul 8, 2025
a5af2b7
Correct tls test
tmiddlet2666 Jul 8, 2025
6582e49
Correct tls test
tmiddlet2666 Jul 8, 2025
df0c78c
Correct tls test
tmiddlet2666 Jul 8, 2025
cfba191
Correct tls test
tmiddlet2666 Jul 8, 2025
e185600
Correct tls test
tmiddlet2666 Jul 8, 2025
63d2b41
Correct tls test
tmiddlet2666 Jul 8, 2025
595cc8e
Fix topics test
tmiddlet2666 Jul 8, 2025
587412e
More topic events updates, update dapr test
tmiddlet2666 Jul 10, 2025
df960fd
remove clone of components-contrib
tmiddlet2666 Jul 10, 2025
56693fc
Add initial subscriber event tests
tmiddlet2666 Jul 10, 2025
e87634d
initial subscriber events
tmiddlet2666 Jul 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/workflows/build-dapr-tls.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Copyright 2024, 2025 Oracle Corporation and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at
# https://oss.oracle.com/licenses/upl.

# ---------------------------------------------------------------------------
# Coherence Go Client GitHub Actions CI build DAPR TLS
# ---------------------------------------------------------------------------
name: CI DAPR TLS

on:
workflow_dispatch:
push:
branches:
- '*'
schedule:
# Every day at midnight
- cron: '0 0 * * *'

jobs:
build:
runs-on: ubuntu-22.04
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
strategy:
fail-fast: false
matrix:
coherenceVersion:
- 25.09-SNAPSHOT
- 25.03.1
go-version:
- 1.24.x

# Checkout the source, we need a depth of zero to fetch all of the history otherwise
# the copyright check cannot work out the date of the files from Git.
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Get Docker Images
shell: bash
run: |
docker pull gcr.io/distroless/java17-debian12

- name: Set up JDK 17 for Build
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'zulu'

- name: Cache Go Modules
uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-mods-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-mods-

- name: Cache Maven packages
uses: actions/cache@v4
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '${{ matrix.go-version }}'

- name: Test DAPR
env:
COH_VERSION: ${{ matrix.coherenceVersion }}
shell: bash
run: |
go install google.golang.org/grpc/cmd/[email protected]
COHERENCE_BASE_IMAGE=gcr.io/distroless/java17-debian12 PROFILES=,secure,jakarta,-javax COHERENCE_VERSION=$COH_VERSION make clean certs generate-proto generate-proto-v1 build-test-images test-cluster-startup test-dapr-tls
make test-cluster-shutdown

- uses: actions/upload-artifact@v4
if: failure()
with:
name: test-output-${{ matrix.go-version }}-${{ matrix.coherenceVersion }}
path: build/_output/test-logs
85 changes: 85 additions & 0 deletions .github/workflows/build-dapr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Copyright 2024, 2025 Oracle Corporation and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at
# https://oss.oracle.com/licenses/upl.

# ---------------------------------------------------------------------------
# Coherence Go Client GitHub Actions CI build DAPR
# ---------------------------------------------------------------------------
name: CI DAPR

on:
workflow_dispatch:
push:
branches:
- '*'
schedule:
# Every day at midnight
- cron: '0 0 * * *'

jobs:
build:
runs-on: ubuntu-22.04
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
strategy:
fail-fast: false
matrix:
coherenceVersion:
- 25.09-SNAPSHOT
- 25.03.1
go-version:
- 1.24.x

# Checkout the source, we need a depth of zero to fetch all of the history otherwise
# the copyright check cannot work out the date of the files from Git.
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Get Docker Images
shell: bash
run: |
docker pull gcr.io/distroless/java17-debian12

- name: Set up JDK 17 for Build
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'zulu'

- name: Cache Go Modules
uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-mods-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-mods-

- name: Cache Maven packages
uses: actions/cache@v4
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '${{ matrix.go-version }}'

- name: Test DAPR
env:
COH_VERSION: ${{ matrix.coherenceVersion }}
shell: bash
run: |
go install google.golang.org/grpc/cmd/[email protected]
COHERENCE_BASE_IMAGE=gcr.io/distroless/java17-debian12 PROFILES=,jakarta,-javax COHERENCE_VERSION=$COH_VERSION make clean generate-proto generate-proto-v1 build-test-images test-cluster-startup
make test-dapr
make test-cluster-shutdown

- uses: actions/upload-artifact@v4
if: failure()
with:
name: test-output-${{ matrix.go-version }}-${{ matrix.coherenceVersion }}
path: build/_output/test-logs
83 changes: 83 additions & 0 deletions .github/workflows/build-topics.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Copyright 2024, 2025 Oracle Corporation and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at
# https://oss.oracle.com/licenses/upl.

# ---------------------------------------------------------------------------
# Coherence Go Client GitHub Actions CI build Topics
# ---------------------------------------------------------------------------
name: CI Topics

on:
workflow_dispatch:
push:
branches:
- '*'
schedule:
# Every day at midnight
- cron: '0 0 * * *'

jobs:
build:
runs-on: ubuntu-22.04
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
strategy:
fail-fast: false
matrix:
coherenceVersion:
- 25.09-SNAPSHOT
go-version:
- 1.23.x
- 1.24.x

# Checkout the source, we need a depth of zero to fetch all of the history otherwise
# the copyright check cannot work out the date of the files from Git.
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Get Docker Images
shell: bash
run: |
docker pull gcr.io/distroless/java17-debian12

- name: Set up JDK 17 for Build
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'zulu'

- name: Cache Go Modules
uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-mods-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-mods-

- name: Cache Maven packages
uses: actions/cache@v4
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '${{ matrix.go-version }}'

- name: E2E Topics Tests
env:
COH_VERSION: ${{ matrix.coherenceVersion }}
shell: bash
run: |
go install google.golang.org/grpc/cmd/[email protected]
COHERENCE_BASE_IMAGE=gcr.io/distroless/java17-debian12 INCLUDE_LONG_RUNNING=true PROFILES=,jakarta,-javax COHERENCE_VERSION=$COH_VERSION make clean generate-proto generate-proto-v1 build-test-images test-e2e-standalone-topics

- uses: actions/upload-artifact@v4
if: failure()
with:
name: test-output-${{ matrix.go-version }}-${{ matrix.coherenceVersion }}
path: build/_output/test-logs
51 changes: 42 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@
# ----------------------------------------------------------------------------------------------------------------------

# This is the version of the coherence-go-client
VERSION ?=2.3.1
VERSION ?=2.4.0
CURRDIR := $(shell pwd)
USER_ID := $(shell echo "`id -u`:`id -g`")

override BUILD_BIN := $(CURRDIR)/bin
override PROTO_DIR := $(CURRDIR)/etc/proto
override PROTO_DIR := $(CURRDIR)/etc/proto
override PROTOV1_DIR := $(CURRDIR)/etc/proto-v1
override PROTOTOPICS_DIR := $(CURRDIR)/etc/topics

# ----------------------------------------------------------------------------------------------------------------------
# Set the location of various build tools
Expand All @@ -25,6 +26,8 @@ override PROTO_OUT := $(CURRDIR)/proto
override BUILD_TARGETS := $(BUILD_OUTPUT)/targets
override TEST_LOGS_DIR := $(BUILD_OUTPUT)/test-logs
override COVERAGE_DIR := $(BUILD_OUTPUT)/coverage
override DAPR_DIR := $(BUILD_OUTPUT)/dapr-test
override DAPR_TEST_DIR := $(CURRDIR)/test/dapr
override COPYRIGHT_JAR := glassfish-copyright-maven-plugin-2.4.jar
override BUILD_CERTS := $(CURRDIR)/test/utils/certs
override ENV_FILE := test/utils/.env
Expand Down Expand Up @@ -240,23 +243,30 @@ ifeq ($(SKIP_PROTO_GENERATION),true)
@echo "Skipping proto generation..."
else
mkdir -p $(PROTOV1_DIR) || true
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/proxy_service_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/25.03.1/prj/coherence-grpc/src/main/proto/proxy_service_messages_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/proxy_service_v1.proto https://raw.githubusercontent.com/oracle/coherence/25.03.1/prj/coherence-grpc/src/main/proto/proxy_service_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/common_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/25.03.1/prj/coherence-grpc/src/main/proto/common_messages_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/cache_service_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/25.03.1/prj/coherence-grpc/src/main/proto/cache_service_messages_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/queue_service_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/25.03.1/prj/coherence-grpc/src/main/proto/queue_service_messages_v1.proto
mkdir -p $(PROTOTOPICS_DIR) || true
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/proxy_service_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/refs/heads/main/prj/coherence-grpc/src/main/proto/proxy_service_messages_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/proxy_service_v1.proto https://raw.githubusercontent.com/oracle/coherence/refs/heads/main/prj/coherence-grpc/src/main/proto/proxy_service_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/common_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/refs/heads/main/prj/coherence-grpc/src/main/proto/common_messages_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/cache_service_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/refs/heads/main/prj/coherence-grpc/src/main/proto/cache_service_messages_v1.proto
curl $(CURL_AUTH) -o $(PROTOV1_DIR)/queue_service_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/refs/heads/main/prj/coherence-grpc/src/main/proto/queue_service_messages_v1.proto
curl $(CURL_AUTH) -o $(PROTOTOPICS_DIR)/topic_service_messages_v1.proto https://raw.githubusercontent.com/oracle/coherence/refs/heads/main/prj/coherence-grpc/src/main/proto/topic_service_messages_v1.proto
echo "" >> $(PROTOV1_DIR)/proxy_service_messages_v1.proto
echo "" >> $(PROTOV1_DIR)/proxy_service_v1.proto
echo "" >> $(PROTOV1_DIR)/common_messages_v1.proto
echo "" >> $(PROTOV1_DIR)/cache_service_messages_v1.proto
echo "" >> $(PROTOV1_DIR)/queue_service_messages_v1.proto
echo "" >> $(PROTOTOPICS_DIR)/topic_service_messages_v1.proto
echo 'option go_package = "github.com/oracle/coherence-go-client/proto/v1";' >> $(PROTOV1_DIR)/proxy_service_messages_v1.proto
echo 'option go_package = "github.com/oracle/coherence-go-client/proto/v1";' >> $(PROTOV1_DIR)/proxy_service_v1.proto
echo 'option go_package = "github.com/oracle/coherence-go-client/proto/v1";' >> $(PROTOV1_DIR)/common_messages_v1.proto
echo 'option go_package = "github.com/oracle/coherence-go-client/proto/v1";' >> $(PROTOV1_DIR)/cache_service_messages_v1.proto
echo 'option go_package = "github.com/oracle/coherence-go-client/proto/v1";' >> $(PROTOV1_DIR)/queue_service_messages_v1.proto
mkdir ./proto/v1 || true
$(TOOLS_BIN)/protoc --proto_path=./etc/proto-v1 --go_out=./proto/v1 --go_opt=paths=source_relative --go-grpc_out=./proto/v1 --go-grpc_opt=paths=source_relative etc/proto-v1/proxy_service_messages_v1.proto etc/proto-v1/proxy_service_v1.proto etc/proto-v1/common_messages_v1.proto etc/proto-v1/cache_service_messages_v1.proto etc/proto-v1/queue_service_messages_v1.proto
echo 'option go_package = "github.com/oracle/coherence-go-client/proto/v1/topics";' >> $(PROTOTOPICS_DIR)/topic_service_messages_v1.proto
mkdir ./proto/v1 ./proto/topics || true
$(TOOLS_BIN)/protoc --proto_path=./etc/proto-v1 --go_out=./proto/v1 --go_opt=paths=source_relative --go-grpc_out=./proto/v1 --go-grpc_opt=paths=source_relative etc/proto-v1/proxy_service_messages_v1.proto etc/proto-v1/proxy_service_v1.proto etc/proto-v1/common_messages_v1.proto etc/proto-v1/cache_service_messages_v1.proto etc/proto-v1/queue_service_messages_v1.proto
$(TOOLS_BIN)/protoc --proto_path=./etc/topics --proto_path=./etc/proto-v1 --go_out=./proto/topics --go_opt=paths=source_relative --go-grpc_out=./proto/topics --go-grpc_opt=paths=source_relative etc/topics/topic_service_messages_v1.proto
cat proto/topics/topic_service_messages_v1.pb.go | sed 's,^\tv1 "github.com/oracle/coherence-go-client/proto/v1",\tv1 "github.com/oracle/coherence-go-client/v2/proto/v1",' > /tmp/proto-queues
mv /tmp/proto-queues proto/topics/topic_service_messages_v1.pb.go
endif

# ----------------------------------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -434,6 +444,15 @@ test-e2e-standalone-queues: test-clean test gotestsum $(BUILD_PROPS) ## Run e2e
-- $(GO_TEST_FLAGS) -v -coverprofile=$(COVERAGE_DIR)/cover-functional-queues.out -v ./e2e/queues/... -coverpkg=github.com/oracle/coherence-go-client/v2/coherence/...
go tool cover -func=$(COVERAGE_DIR)/cover-functional-queues.out | grep -v '0.0%'

# ----------------------------------------------------------------------------------------------------------------------
# Executes the Go end to end tests for standalone Coherence with Topics
# ----------------------------------------------------------------------------------------------------------------------
.PHONY: test-e2e-standalone-topics
test-e2e-standalone-topics: test-clean test gotestsum $(BUILD_PROPS) ## Run e2e tests with Coherence queues
cd test && CGO_ENABLED=0 $(GOTESTSUM) --format testname --junitfile $(TEST_LOGS_DIR)/go-client-test-queues.xml \
-- $(GO_TEST_FLAGS) -v -coverprofile=$(COVERAGE_DIR)/cover-functional-topics.out -v ./e2e/topics/... -coverpkg=github.com/oracle/coherence-go-client/v2/coherence/...
go tool cover -func=$(COVERAGE_DIR)/cover-functional-topics.out | grep -v '0.0%'

# ----------------------------------------------------------------------------------------------------------------------
# Executes the Go end to end tests for gRPC v1 tests
# ----------------------------------------------------------------------------------------------------------------------
Expand All @@ -450,6 +469,20 @@ test-v1-base: test-clean test gotestsum $(BUILD_PROPS) ## Run e2e tests with Coh
test-examples: test-clean gotestsum $(BUILD_PROPS) ## Run examples tests with Coherence
./scripts/run-test-examples.sh

# ----------------------------------------------------------------------------------------------------------------------
# Executes the tests for DAPR
# ----------------------------------------------------------------------------------------------------------------------
.PHONY: test-dapr
test-dapr: gotestsum $(BUILD_PROPS) ## Run dapr tests with Coherence
./scripts/run-test-dapr.sh $(DAPR_TEST_DIR) $(DAPR_DIR)

# ----------------------------------------------------------------------------------------------------------------------
# Executes the tests for DAPR TLS
# ----------------------------------------------------------------------------------------------------------------------
.PHONY: test-dapr-tls
test-dapr-tls: gotestsum $(BUILD_PROPS) ## Run dapr tests with Coherence with TLS
./scripts/run-test-dapr.sh $(DAPR_TEST_DIR) $(DAPR_DIR) true

# ----------------------------------------------------------------------------------------------------------------------
# Startup cluster members via docker compose
# ----------------------------------------------------------------------------------------------------------------------
Expand Down
Loading
Loading