Integrate with databus.openenergyplatform.org as backend service#1906
Draft
Integrate with databus.openenergyplatform.org as backend service#1906
Conversation
- make it more modular - prepare htmx based partial template loading
jh-RLI
changed the title
Contributor
|
Is openergy.databus.org correct? Or do you reference https://databus.openenergyplatform.org/ ? |
Contributor
Author
|
Yes sorry there was a energy.databus some time ago and im always confusing it now :D https://databus.openenergyplatform.org/ is the correct one. |
jh-RLI
changed the title
Contributor
Author
|
i enhanced the description above a bit. |
Contributor
Author
|
Not the best idea to add all these additionall issues here ... but this PR will implement the initial databus integration it will not stay open until it is fully integrated. |
…n be imported everywhere as partial using htmx) # 1348
- provide functionality to register a oep dataset on the databus using the databusclient (requires the server to be setup for server to server communication) - The oemetadata provide key metadata for the registration process: Title, Description, License. The table name is the artifact name - The use must provide a version, but we will provide a default like "initial" or "1" or a datetimestamp - Additionally there is functionality to create the moss entry also
…egister a oep table on the databus #1348
- Add databus app - new settings DATABUS_URI_BASE, MOSS_URL
… htmx request was send. This behavior needs to be adapted a now there can be multiple htmx request be send this would lead to a loop #1348
…on for local testing
- the databus python client is no longer supported as its package decencies are conflicting with oeplatform
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of the discussion
The Databus instance for the OEP (https://databus.openenergyplatform.org/) has been operational for some time, though currently, it is primarily known and utilized by related projects and their maintainers. With updates like the recent release of OEMetadata v2.0, and with regard to enhanding the search and adding features like dataset collections, we are actively working to integrate the Databus with the Open Energy Platform (OEP), bringing together these two core services. Additionally, another service is added MOSS (https://moss.openenergyplatform.org/) but it is currently empty. This service aims at consuming additional (to the databus) metadata. Once integrated, it will store the OEMetadata. Through this integration, OEMetadata will be added as a MOSS metadata layer extension on the Databus, making data descriptions richer and more standardized.
The Databus itself will not store any actual data but will act as a metadata registry, creating and managing metadata records for data distributions (which can be tables on the OEP or external distributions with available link).
This PR introduces functionality to register datasets hosted on the OEP directly with the Databus. Moving forward, the Databus will become an integral component of the OEP’s backend, handling advanced data organization and metadata management. This includes features such as creating collections across multiple tables, managing data groups, and harmonizing topics with Databus groups. By offloading data management and search functionalities to the Databus, the OEP core software can focus on its primary objectives, with Databus providing a robust foundation for metadata handling and discoverability.
This ongoing development will progress gradually, with step-by-step migrations, ensuring a seamless transition of data management tasks to the Databus backend and enhancing the overall user experience on the OEP.
Type of change (CHANGELOG.md)
Features
Workflow checklist
Automation
Closes #1348
Additional
Closes #1822
Closes #1666
Closes #1883
Closes #1882
PR-Assignee
Reviewer