Skip to content

Add end-to-end tests that run the plugin on a real server with real player bots #34

Description

@milanmalhotra

Why

The MockBukkit suite is good at what it does, but it mocks the Bukkit API in-process. Whole classes of behaviour are therefore untestable today, and every one of them is a place where a regression reaches server owners before it reaches us:

Untested surface Why MockBukkit cannot reach it
The AnvilGUI rename prompt (HomeActionsGui:333) Needs NMS. applyRename was factored out (HomeActionsGui:264) precisely so the logic could be tested without the prompt. The prompt itself has never run under test.
Real inventory clicks HomesGuiClickTest drives clicks through mocked inventory events. Packet-level slot clicks, pagination past the first page, and the right-click that opens HomeActionsGui are not exercised.
Real teleports TeleportSafetyUtil, cross-dimension moves, the delay countdown, cancel-on-move, and player_teleport_attempts surviving an actual restart.
LuckPerms group limits LuckPerms is a provided-scope soft dependency (CreateHome:138 guards on isPluginEnabled). It is never loaded in tests, so the groups branch of maxHomesType is only covered by its fallback.
The shaded jar itself AnvilGUI relocation, plugin.yml permission children, the SQLite file being created under plugins/SetHomesTwo/database/. mvn verify proves the shade runs, not that the artifact loads.

We already treat the local Paper server as the final check for anything NMS-backed. This issue is about making that check automatic.

Options considered

1. Plugwright (recommended)

https://github.com/drownek/paper-e2e-test, MIT, formerly Paperwright, formerly paper-e2e-test.

A Gradle plugin that downloads a Paper server, boots it with our jar, and joins real Mineflayer bots that click GUIs and read chat like actual players. Supports MC 1.8 through 1.21.11. Tests are JS or TS with a Playwright-shaped API:

test('clicking a home in the GUI teleports the player', async ({ player }) => {
  player.chat('/create-home base');
  player.chat('/homes');

  const gui = await player.gui({ title: 'Your homes' });
  await gui.locator(i => i.getDisplayName().includes('base')).click();

  await expect(player).toHaveReceivedMessage('Teleported to base');
});

What it gives us that matters here:

  • player.gui({ title }) matches on our configurable inventoryTitle, and gui.locator(predicate) finds items by display name, lore, or material, which is exactly how HomesGui renders homes.
  • createPlayer() spawns a second independent bot, which is what /go-player-home, /delete-player-home and /move-player-home need.
  • player.makeOp() and player.deOp(), plus server.execute(cmd), cover the permission matrix without a permissions plugin.
  • downloadPlugins { url(...) } can pull a real LuckPerms jar so the groups branch actually runs.
  • writeFiles stages files into the run directory before boot, so a test run can start from a chosen config.yml (delay: 0 for fast paths, cancelOnMove: true for the cancel test, maxHomeEnabled: true for limits).
  • player.bot exposes the raw Mineflayer bot, so anything the wrapper does not cover is still reachable: bot.simpleClick.rightMouse(slot), bot.entity.position, bot.game.dimension.
  • An official GitHub Action, drownek/plugwright-action@v1.

Cost: it is a Gradle plugin and this is a Maven project. See the spikes below.

2. Hand-rolled Mineflayer harness

Boot Paper from a script, join a Mineflayer bot, assert directly. Mineflayer is the mature piece here (7.3k stars, MC 1.8 to 1.21.11) and gives us simpleClick.leftMouse and simpleClick.rightMouse, the windowOpen event, messagestr, bot.entity.position and bot.game.dimension. Driven from Maven via failsafe or exec, so no Gradle enters the tree.

Full control, no build-tool mismatch, but we would be rebuilding server lifecycle, jar staging, retries and matchers, all of which Plugwright already has. A reasonable fallback if the Gradle bridge turns out to be worse than it looks.

3. SpigotTester

https://github.com/jwdeveloper/SpigotTester. A JUnit-alike that runs inside the live server and creates simulated players via addPlayer(name).

Real server, so NMS and plugin loading are real, but the players are simulated again, so it does not reach packet-level clicks. That is most of what we are trying to cover. 9 stars, no documented CI story. Not recommended.

4. WatchWolf

Starts real servers and real clients. Its documentation says integration runs are slow enough that a self-hosted runner is recommended for CI. Heavier than this project warrants.

5. Load smoke test (worth having regardless of the above)

https://github.com/FN-FAL913/minecraft-plugin-runtime-test is a GitHub Action that boots Paper across versions and fails if the plugin does not initialise. Not E2E, but it is cheap, it catches a broken shade or a bad plugin.yml, and it can land before any of the above.

Spikes to run first

Two things need answering before the work is scoped properly. Both are throwaway probes.

  1. Maven bridge. Plugwright's npm package publishes no bin, so Gradle is doing the orchestration and there is no standalone CLI. The likely bridge is a small build.gradle.kts used only for E2E, with useExternalPluginsOnly.set(true) so it does not chase a jar task, plus writeFiles staging our Maven-built jar into plugins/. writeFiles is documented for inline text and local source files; whether it copies a binary is unverified. If it does not, the alternatives are a downloadPlugins URL pointing at a local file server, or option 2.
  2. The anvil rename. This is the highest-risk piece and the one with the most to gain. Mineflayer's anvil.combine(item, name) is built around bot.openAnvil(anvilBlock), an anvil block in the world. AnvilGUI opens a server-side anvil window with no block behind it. It may come down to writing a raw name_item packet and clicking the output slot. If that cannot be made to work, the rename prompt stays manually verified and the rest of the suite proceeds without it.

Proposed first slice

Deliberately thin. The point of the first pass is to prove the harness works end to end in CI, not to cover the surface. Everything else becomes follow-up issues.

  • The plugin loads on a real Paper server with no errors in the log, and creates its data folder and SQLite database.
  • /create-home base reports success and the home persists.
  • /home base teleports the player, verified against bot.entity.position.
  • /homes opens an inventory whose title matches inventoryTitle, containing an item named for the home.
  • Left-clicking that item teleports the player.
  • A player without sh2.create-home is refused.

Follow-ups once the harness is trusted: GUI pagination past 45 entries, right-click into HomeActionsGui and each of its actions, the anvil rename, teleport delay and cancel-on-move, the dimension blacklist, LuckPerms group limits, restart persistence of pending teleports, and the EssentialsX and SetHomes v1 importers.

CI

E2E runs nightly on a schedule plus workflow_dispatch, in its own workflow, not in tests.yml.

Rationale: tests.yml runs on every push and stays fast. E2E pays roughly 20 seconds of server boot before the first assertion and is inherently flakier than in-process tests. A red nightly is cheaper to absorb than a blocked pull request. If the suite proves stable it can be promoted to run on pull requests into master, so it gates releases without slowing dev.

Open questions

  • Which Minecraft version to pin the E2E server to. The local test server is Paper 1.21.4 and plugin.yml declares api-version: 1.21, while the pom compiles against spigot-api 26.2. Plugwright lists 1.21, 1.21.9 and 1.21.11; 1.21.4 specifically is unconfirmed.
  • Whether a Gradle file in a Maven repository is acceptable, or whether option 2 is preferable on those grounds alone.
  • Whether the E2E suite lives in this repository or a sibling.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions