Feature/database storage support#148
Feature/database storage support#148bedge117 wants to merge 34 commits intoNighterDevelopment:mainfrom
Conversation
- Implement SpawnerStorage interface for abstracted storage layer - Add DatabaseManager with HikariCP connection pool for MariaDB - Create SpawnerDatabaseHandler implementing storage interface with dirty tracking and async batch saves - Add YamlToDatabaseMigration for one-time migration from YAML to DB - Update SpawnerFileHandler to implement SpawnerStorage interface - Add server_name config for cross-server spawner identification - Configure shadow plugin to shade HikariCP and MariaDB JDBC driver - Storage mode configurable via database.mode (YAML or DATABASE) Default behavior unchanged - YAML mode remains the default.
Added location check in createSmartSpawner() and createSmartItemSpawner() before creating new spawner data. If a spawner already exists at that location, the code now: - Stacks with existing spawner if same entity/item type - Updates last interacted player - Prevents duplicate entries in spawners map This fixes ghost spawner entries where the physical block could be removed but orphaned data remained in the YAML.
Added database storage support and migration from YAML.
Added cross-server selection functionality to the ListSubCommand. Implemented methods for server selection GUI and cross-server checks.
Mark spawner as modified for database save after updating stack size.
Refactor event handling for world selection and spawner management. Added support for remote servers and improved code readability.
Added targetServer field and updated constructors to handle it.
Added targetServer field and updated constructors to initialize it. Implemented isRemoteServer method to check for remote server.
Added support for cross-server world selection with target server name.
This class manages database connections using HikariCP, specifically for MariaDB, and includes methods for initializing the connection pool, creating necessary tables, and managing connections.
Added spawnerManager to handle spawner modifications.
Added spawnerManager to handle spawner modifications.
Added database configuration settings for spawner data storage.
Implements a listener for handling server selection clicks in the GUI.
Database fixes: - Fix MariaDB driver not found (explicitly set relocated driver class) - Fix stack size not persisting (add markSpawnerModified calls in GUI/physical stacking) - Fix NPE on spawner break/explosion in database mode (use spawnerManager instead of spawnerFileHandler) Cross-server feature: - Add server selection GUI for viewing spawners across multiple servers - Add CrossServerSpawnerData for remote server spawner display - Add back button to world selection when cross-server mode enabled - Support remote world selection with async database queries
This interface defines the operations for managing spawner data storage, including initialization, loading, and saving spawners.
Added an enumeration for storage modes including YAML and DATABASE.
- Add SQLite as alternative to MySQL/MariaDB for local database storage - Update StorageMode enum: YAML, MYSQL, SQLITE (removed DATABASE) - Rename config section from standalone: to sql: for clarity - Add sqlite.file config option for SQLite database filename - SQLite uses single connection pool with WAL journal mode - Cross-server sync only available for MYSQL mode (SQLite is local-only) - Add xerial sqlite-jdbc driver with proper relocation
Resolves merge conflicts by keeping SQLite storage support: - StorageMode: YAML, MYSQL, SQLITE (replaces DATABASE) - Config: sql: section (replaces standalone:) - DatabaseManager: supports both MySQL and SQLite - Cross-server sync only available for MYSQL mode
- YamlToDatabaseMigration now supports both MySQL and SQLite syntax - New SqliteToMySqlMigration class for migrating from SQLite to MariaDB - Migrations run automatically on startup when target mode is detected - Source files are renamed with .migrated suffix after successful migration
Added database.migrate_from_local option (default: true) that controls automatic data migration on startup. When enabled and using MYSQL or SQLITE mode: - Migrates spawners_data.yml to target database - For MYSQL mode, also migrates spawners.db (SQLite) to MySQL - Files are renamed with .migrated suffix after successful migration Config includes detailed comments explaining the migration behavior.
Automatically migrates old config keys when upgrading: - database.standalone.* -> database.sql.* - database.mode: DATABASE -> MYSQL New keys like migrate_from_local and sqlite.file are added automatically by the existing ConfigUpdater mechanism.
Cross-server spawner list improvements: - Filter (all/active/inactive) now works for remote server spawners - Sort (default/stack size asc/desc) now works for remote server spawners - Filter/sort buttons added to remote spawner list GUI Remote spawner management: - View Spawner Info: Shows spawner stats from database via chat - Edit Stack Size: Opens stack editor, saves changes to database async - Remove Spawner: Deletes from database (physical block syncs on target server refresh) - Teleport remains disabled (cannot teleport cross-server) Technical changes: - Added getCrossServerSpawnersAsync with filter/sort parameters - Added getRemoteSpawnerByIdAsync for fetching single spawner data - Added updateRemoteSpawnerStackSizeAsync for remote stack changes - Added deleteRemoteSpawnerAsync for remote spawner removal - New RemoteAdminStackerHolder for remote stack editing context
There was a problem hiding this comment.
Pull request overview
This PR introduces a pluggable storage layer for spawner persistence, adding database-backed storage (MySQL/MariaDB + SQLite) alongside the existing YAML file storage, and extends the /smartspawner list GUI to support cross-server spawner browsing/management when using MySQL.
Changes:
- Added
SpawnerStorageabstraction andStorageMode(YAML/MYSQL/SQLITE), with DB implementations via HikariCP. - Implemented automatic migrations (YAML→DB and SQLite→MySQL) and config auto-migration for renamed keys.
- Extended list/management/admin-stacker GUIs to support cross-server browsing and limited remote management actions.
Reviewed changes
Copilot reviewed 31 out of 32 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| core/src/main/resources/config.yml | Adds database configuration section (mode, sqlite/sql pool, migration flags). |
| core/src/main/java/github/nighter/smartspawner/updates/ConfigUpdater.java | Migrates renamed config paths and legacy DATABASE mode to MYSQL. |
| core/src/main/java/github/nighter/smartspawner/spawner/interactions/stack/SpawnerStackHandler.java | Marks stacked spawners as modified for persistence. |
| core/src/main/java/github/nighter/smartspawner/spawner/interactions/place/SpawnerPlaceListener.java | Prevents duplicate/ghost spawners by reusing existing location entries. |
| core/src/main/java/github/nighter/smartspawner/spawner/interactions/destroy/SpawnerExplosionListener.java | Routes delete tracking through SpawnerManager (storage-agnostic). |
| core/src/main/java/github/nighter/smartspawner/spawner/interactions/destroy/SpawnerBreakListener.java | Routes delete tracking through SpawnerManager (storage-agnostic). |
| core/src/main/java/github/nighter/smartspawner/spawner/gui/stacker/SpawnerStackerHandler.java | Marks stack changes as modified for DB/YAML flush. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/storage/StorageMode.java | Defines supported storage modes. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/storage/SpawnerStorage.java | Introduces storage interface for load/save/delete/flush operations. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/database/YamlToDatabaseMigration.java | Implements YAML→DB one-time migration. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/database/SqliteToMySqlMigration.java | Implements SQLite→MySQL one-time migration. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/database/SpawnerDatabaseHandler.java | Implements DB-backed SpawnerStorage + cross-server query/update helpers. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/database/DatabaseManager.java | Adds HikariCP setup and table/index creation for MySQL/SQLite. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/WorldEventHandler.java | Switches world save/load/unload flows to use SpawnerStorage. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/SpawnerManager.java | Uses SpawnerStorage for persistence operations. |
| core/src/main/java/github/nighter/smartspawner/spawner/data/SpawnerFileHandler.java | Makes YAML handler implement SpawnerStorage. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/worldselection/WorldSelectionHolder.java | Adds optional target-server context for cross-server world selection. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/serverselection/ServerSelectionHolder.java | Adds holder for server selection GUI. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/serverselection/ServerSelectionHandler.java | Handles clicks in server selection GUI and opens per-server world selection. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/management/SpawnerManagementHolder.java | Adds optional target-server context for management GUI. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/management/SpawnerManagementHandler.java | Adds remote management flows (info/stack edit/remove) for MySQL cross-server mode. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/management/SpawnerManagementGUI.java | Disables teleport for remote spawners; adds remote-action UI variants. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/list/SpawnerListHolder.java | Adds optional target-server context for list GUI. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/list/SpawnerListGUI.java | Adds remote navigation and management opening for cross-server browsing. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/adminstacker/RemoteAdminStackerHolder.java | Adds holder state for editing remote stack sizes. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/adminstacker/AdminStackerUI.java | Adds remote admin-stacker GUI rendering. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/adminstacker/AdminStackerHandler.java | Adds click handling for remote admin-stacker and DB update call. |
| core/src/main/java/github/nighter/smartspawner/commands/list/gui/CrossServerSpawnerData.java | Adds lightweight DTO for remote spawner list/management. |
| core/src/main/java/github/nighter/smartspawner/commands/list/ListSubCommand.java | Adds cross-server enablement checks and async DB-backed server/world/spawner listing. |
| core/src/main/java/github/nighter/smartspawner/SmartSpawner.java | Initializes storage mode, migrations, and shutdown flow for DB/YAML. |
| core/build.gradle | Adds Shadow shading/relocation for HikariCP + MariaDB + SQLite JDBC. |
| build.gradle | Registers Shadow plugin version for subprojects. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Fetch spawner data from database and display info | ||
| SpawnerDatabaseHandler dbHandler = getDbHandler(); | ||
| if (dbHandler == null) { | ||
| messageService.sendMessage(player, "database_error"); |
There was a problem hiding this comment.
messageService.sendMessage(player, "database_error") references a message key that does not exist in the shipped language files, which will show players a “Missing message key” warning. Add a database_error entry to messages.yml (and other locales) or reuse an existing generic failure key (e.g., action_failed).
| messageService.sendMessage(player, "database_error"); | |
| messageService.sendMessage(player, "action_failed"); |
| private void saveRemoteStackChanges(Player player, RemoteAdminStackerHolder holder) { | ||
| SpawnerStorage storage = plugin.getSpawnerStorage(); | ||
| if (!(storage instanceof SpawnerDatabaseHandler dbHandler)) { | ||
| messageService.sendMessage(player, "database_error"); |
There was a problem hiding this comment.
messageService.sendMessage(player, "database_error") references a message key that is missing from the language files, so players will see a “Missing message key” warning. Add the key to messages.yml (and other locales) or switch to an existing generic error message key.
| messageService.sendMessage(player, "database_error"); | |
| player.sendMessage("§cRemote stack changes require database storage. Please enable database mode."); |
| // For remote servers, query async | ||
| SpawnerDatabaseHandler dbHandler = getDbHandler(); | ||
| if (dbHandler == null) { | ||
| messageService.sendMessage(player, "database_error"); |
There was a problem hiding this comment.
messageService.sendMessage(player, "database_error") uses a message key that isn't present in the bundled language files, which will emit warnings and display a fallback “Missing message key” to players. Add database_error to messages.yml (and other locales) or reuse an existing generic error key like action_failed.
| messageService.sendMessage(player, "database_error"); | |
| messageService.sendMessage(player, "action_failed"); |
| // Rename the YAML file to prevent re-migration | ||
| if (failedCount == 0 || migratedCount > 0) { | ||
| File migratedFile = new File(plugin.getDataFolder(), YAML_FILE_NAME + MIGRATED_FILE_SUFFIX); | ||
| if (yamlFile.renameTo(migratedFile)) { | ||
| logger.info("YAML file renamed to " + YAML_FILE_NAME + MIGRATED_FILE_SUFFIX); | ||
| } else { | ||
| logger.warning("Failed to rename YAML file. Manual cleanup may be required."); | ||
| } |
There was a problem hiding this comment.
The migration renames spawners_data.yml even when there were migration failures (condition failedCount == 0 || migratedCount > 0). If any spawners fail to migrate, renaming prevents automatic re-migration on the next startup and can leave data only in the renamed YAML file. Rename the YAML file only when the migration fully succeeds (e.g., failedCount == 0), or keep the original file and log actionable instructions for manual recovery.
| // Rename the YAML file to prevent re-migration | |
| if (failedCount == 0 || migratedCount > 0) { | |
| File migratedFile = new File(plugin.getDataFolder(), YAML_FILE_NAME + MIGRATED_FILE_SUFFIX); | |
| if (yamlFile.renameTo(migratedFile)) { | |
| logger.info("YAML file renamed to " + YAML_FILE_NAME + MIGRATED_FILE_SUFFIX); | |
| } else { | |
| logger.warning("Failed to rename YAML file. Manual cleanup may be required."); | |
| } | |
| // Rename the YAML file to prevent re-migration only when migration fully succeeds | |
| if (failedCount == 0) { | |
| File migratedFile = new File(plugin.getDataFolder(), YAML_FILE_NAME + MIGRATED_FILE_SUFFIX); | |
| if (yamlFile.renameTo(migratedFile)) { | |
| logger.info("YAML file renamed to " + YAML_FILE_NAME + MIGRATED_FILE_SUFFIX); | |
| } else { | |
| logger.warning("Failed to rename YAML file after successful migration. Manual cleanup may be required."); | |
| } | |
| } else { | |
| logger.warning("Some spawners failed to migrate (" + failedCount + " failed). " | |
| + "The original YAML file has been kept as " + YAML_FILE_NAME + ". " | |
| + "Please check the server logs, resolve the issues, and rerun the migration or " | |
| + "manually migrate the remaining spawners from the YAML file if necessary."); |
| logger.info("Migration completed. Total: " + totalSpawners + ", Migrated: " + migratedCount + ", Failed: " + failedCount); | ||
|
|
||
| // Rename the SQLite file to prevent re-migration | ||
| if (failedCount == 0 || migratedCount > 0) { |
There was a problem hiding this comment.
The migration renames the SQLite DB file even when there were migration failures (condition failedCount == 0 || migratedCount > 0). With partial failures, the source DB gets renamed and the plugin won’t re-attempt migration automatically, which can strand records in the renamed file. Only rename when failedCount == 0 (or provide an explicit config flag to allow renaming on partial success).
| if (failedCount == 0 || migratedCount > 0) { | |
| if (failedCount == 0) { |
|
|
||
| private void setupMySQLDataSource(HikariConfig config) { | ||
| // JDBC URL for MariaDB/MySQL | ||
| String jdbcUrl = String.format("jdbc:mariadb://%s:%d/%s?useSSL=false&allowPublicKeyRetrieval=true&serverTimezone=UTC", |
There was a problem hiding this comment.
The JDBC URL hard-codes useSSL=false. For cross-server/database setups this can send credentials and data unencrypted over the network. Consider making SSL configurable (defaulting to enabled), and avoid forcing it off in code.
| String jdbcUrl = String.format("jdbc:mariadb://%s:%d/%s?useSSL=false&allowPublicKeyRetrieval=true&serverTimezone=UTC", | |
| String jdbcUrl = String.format("jdbc:mariadb://%s:%d/%s?allowPublicKeyRetrieval=true&serverTimezone=UTC", |
| if (event.getSlot() == 53 && currentPage < totalPages) { | ||
| // Next page | ||
| listSubCommand.openSpawnerListGUI(player, worldName, currentPage + 1, currentFilter, currentSort); | ||
| if (isRemote) { | ||
| listSubCommand.openSpawnerListGUIForServer(player, targetServer, worldName, currentPage + 1); | ||
| } else { |
There was a problem hiding this comment.
Remote pagination drops the current filter/sort because it calls the overload that uses defaults. This can reset the list ordering when navigating pages. Pass currentFilter/currentSort into the remote openSpawnerListGUIForServer(...) call here as well.
| if (event.getSlot() == 45 && currentPage > 1) { | ||
| // Previous page | ||
| listSubCommand.openSpawnerListGUI(player, worldName, currentPage - 1, currentFilter, currentSort); | ||
| if (isRemote) { | ||
| listSubCommand.openSpawnerListGUIForServer(player, targetServer, worldName, currentPage - 1); | ||
| } else { |
There was a problem hiding this comment.
Remote pagination drops the current filter/sort because it calls the overload that uses defaults. This means going to the next/previous page can unexpectedly reset the user’s current view. Pass currentFilter/currentSort into the remote openSpawnerListGUIForServer(...) call.
| // Save user preference when they change filter (only for local) | ||
| if (!isRemote) { | ||
| listSubCommand.saveUserPreference(player, worldName, nextFilter, currentSort); | ||
| } |
There was a problem hiding this comment.
The new remote-list logic avoids saving filter/sort preferences when the user clicks filter/sort, but onInventoryClose in this class still calls saveUserPreference(...) unconditionally. That means remote browsing will still persist preferences (and may overwrite local ones for the same world name). Consider guarding preference saves consistently for remote lists or including targetServer in the preference key.
| Map<String, String> placeholders = new HashMap<>(); | ||
| placeholders.put("old", String.valueOf(originalSize)); | ||
| placeholders.put("new", String.valueOf(newStackSize)); |
There was a problem hiding this comment.
The contents of this container are never accessed.
| Map<String, String> placeholders = new HashMap<>(); | |
| placeholders.put("old", String.valueOf(originalSize)); | |
| placeholders.put("new", String.valueOf(newStackSize)); |
|
Hi @bedge117 , big thanks for this. I just wanted to know if the features are properly tested so I don’t have to test it all over again |
|
@ptthanh02 , I tested on several test servers and identified a few issues in the process. My original issues have been corrected. The yamltosql conversion incorrectly assigned spawner stacks to 1, which I addressed. I had to modify the spawner interactions to update the Database. Originally, it was only detecting placement and not destroy. I also had to modify explosion (which I had disabled so missed originally). Now, all interactions, and spawner state is correctly saving persistent data. I opted to run everything async due to latency concerns. After testing, I deployed to my live servers. I have not had any reported issues. |
This PR adds database storage as an alternative to YAML file storage, providing better performance for large servers and enabling cross-server spawner management.
Features
Two New Storage Modes:
Cross-Server Spawner Management (MariaDB only):
Automatic Data Migration:
Config Auto-Update:
Configuration
database:
mode: YAML # YAML, MYSQL, or SQLITE
server_name: "server1"
sync_across_servers: false
migrate_from_local: true
Technical Details