Skip to content

Dump n load #1588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Dump n load #1588

wants to merge 9 commits into from

Conversation

louispt1
Copy link
Contributor

PR for MVP:

This PR:

  • Includes the work from @noracato for dumping and loading scenarios as JSON.
    • This includes API endpoints for dump and load, and inspect routes.
  • Expands functionality to allow single or multiple dump/load functionality for scenarios.
    • Currently, you can specify multiple scenario ids separated by commas to dump those scenarios. In the future I would like to expand this to include a dropdown with preset filters for scenario dumps like featured scenarios or scenarios belonging to a certain user.
    • You can dump a single scenario from the show view, or multiple from the index view. You upload scenarios via the index view.
  • Post upload, you are redirected to a page with the links to the uploaded scenarios in ETModel, and sets the active api id to the first scenario of the bunch that you've uploaded (I think this makes it easy to see which scenarios you've uploaded in the index view).
  • Spec has been added for the dump and load sub-models.

Outstanding issues:

  • Scenario sets are not yet implemented (especially featured would be nice)
  • Maybe we need more internal documentation on this feature? I think how it is currently implemented is fairly logical at least.
  • I set the etmodel urls in config/settings/.yml for each environment. Is that ok? Kyra had requested that after importing, it would be helpful to have a page with the links to the front end for the scenarios you imported.

Closes #1569, #1570, #1571

@louispt1 louispt1 requested a review from noracato May 20, 2025 12:25
Copy link
Member

@noracato noracato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! Just a few comments on the first few files if you want to continue working on this tomorrow! There are also a few questions for you in there to either research yourself if you think that's fun and you have time. Thought that could be a new thing we can try out if you're in to that. Otherwise we can discuss them Thursday, no prob :)

load_resource except: %i[show create destroy]
load_and_authorize_resource class: Scenario, only: %i[index show destroy]
load_resource except: %i[show create destroy dump]
load_and_authorize_resource class: Scenario, only: %i[index show create destroy dump]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
load_and_authorize_resource class: Scenario, only: %i[index show create destroy dump]
load_and_authorize_resource class: Scenario, only: %i[index show destroy dump]

Adding create here will create the scenario twice.

Comment on lines 73 to 97
file = params.permit(:dump)[:dump]
unless file&.respond_to?(:path)
redirect_back fallback_location: root_path, alert: 'No file provided'
return
end

raw_data = JSON.parse(File.read(file.path))
data_array = raw_data.is_a?(Array) ? raw_data : [raw_data]

@scenarios = data_array.map { |sd| ScenarioPacker::Load.new(sd.with_indifferent_access).scenario }

if @scenarios.one?
new_id = @scenarios.first.id
redirect_to inspect_scenario_path(
id: new_id,
api_scenario_id: new_id
),
notice: 'Scenario created'
else
api_id = @scenarios.first.id
@api_scenario = Scenario.find(api_id)
@scenario = Scenario::Editable.new(@api_scenario)
render :load_results
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember: we should try to make methods be not more than 12 lines. So we can smell a possible refactor here!

The question as always: which logic belongs where?

Small hint: The ScenarioPacker can currently Load one scenario from a parsed JSON. So his job is loading stuff from a parsed JSON.

Second hint: we can decide to use before hooks to validate incoming request data.

Comment on lines 107 to 112
filename =
if payload.one?
"#{scenarios.first.id}-dump.json"
else
"#{ids.join('-')}-dump.json"
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
filename =
if payload.one?
"#{scenarios.first.id}-dump.json"
else
"#{ids.join('-')}-dump.json"
end
filename = "#{ids.join('-')}-dump.json"

If there was only one in the ids array, the join would just return the id.

Comment on lines 101 to 102
type = params[:type] || 'user_input'
ids = ScenarioSetFetcher.fetch(type: type, params: params)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. It can be dangerous to just forward raw params further into the code, how can we do this differently?
  2. Who should now default keywords? The service class, the view, or the controller, or is it shared knowledge?

@louispt1 louispt1 changed the title Dump n load enhancements Dump n load May 30, 2025
@louispt1
Copy link
Contributor Author

I tried to spend some time addressing code smells @noracato - please let me know if you have more feedback or if you want to go over anything together

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement scenario dump and import process
2 participants