Skip to content

VIP input maps handling #658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Albireo-B wants to merge 3 commits into
base: develop
Choose a base branch
from
Open

VIP input maps handling #658

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a0c88ec
New [Inputs] : Handle lists of inputs maps from API
Apr 10, 2026
6d3934b
Fix [Tests] : Update input test files
Apr 10, 2026
6b298e6
Update [Execution] : Add InputValues Json deserializer and fix some m…
Apr 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 69 additions & 44 deletions vip-api/src/main/java/fr/insalyon/creatis/vip/api/business/ExecutionBusiness.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import java.util.Map;
import java.util.Map.Entry;
import java.util.function.Supplier;
import java.util.stream.Collectors;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
Expand Down Expand Up @@ -174,18 +175,21 @@ private Execution getExecutionFromSimulation(Simulation s, boolean summarize) th
for (InOutData iod : inputs) {
String key = iod.getProcessor();
String value = iod.getPath();

((List<Object>) e.getInputValues().computeIfAbsent(key, k -> new ArrayList<>())).add(value);
}
// retrieves results directory
List<Object> resDirList = (List<Object>) e.getInputValues().get(RESULTS_DIRECTORY_PARAM_NAME);
if (resDirList == null) {
resDirList = new ArrayList<>();
}
if (!resDirList.isEmpty()) {
e.setResultsLocation(resDirList);
e.getInputValues().remove(RESULTS_DIRECTORY_PARAM_NAME);
for (Map<String, Object> inputMap : e.getInputValues()) {
Copy link
Copy Markdown
Contributor

@axlbonnet axlbonnet Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think e.getInputValues() is empty at this point.
Anyway we have to solve the workflows-db issue before (see the moteurlite PR).

Copy link
Copy Markdown
Author

@Albireo-B Albireo-B Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some null checks were introduced by the latest PR

((List<Object>) inputMap.computeIfAbsent(key, k -> new ArrayList<>())).add(value);

// retrieves results directory
List<Object> resDirList = (List<Object>) inputMap.get(RESULTS_DIRECTORY_PARAM_NAME);
if (resDirList == null) {
resDirList = new ArrayList<>();
}
if (!resDirList.isEmpty()) {
e.setResultsLocation(resDirList);
inputMap.remove(RESULTS_DIRECTORY_PARAM_NAME);
}
}
}

List<InOutData> outputs = workflowBusiness.getOutputData(s.getID(), userFolder);
for (InOutData iod : outputs) {
String key = iod.getProcessor();
Expand Down Expand Up @@ -315,25 +319,35 @@ public void updateExecution(Execution execution) throws VipException {
}

public String initExecution(Execution execution) throws VipException {
Map<String, String> inputMap = new HashMap<>();
List<Map<String, String>> inputMaps = new ArrayList<>();
Object resultsLocation = execution.getResultsLocation();
boolean isInputMapList = execution.getInputValues().size() > 1;
for (Map<String, Object> inputValuesMap : execution.getInputValues()) {
Map<String, String> inputMap = new HashMap<>();
for (Entry<String, Object> restInput : inputValuesMap.entrySet()) {
if (isInputMapList && restInput.getValue() instanceof List) {
throw new VipException(
"Parameter '" + restInput.getKey() + "' contains a list, it should only have a single value when providing a list of input maps.");
}

for (Entry<String,Object> restInput : execution.getInputValues().entrySet()) {
inputMap.put(
restInput.getKey(),
handleRestParameter(restInput.getKey(), restInput.getValue()));
}
inputMap.put(
restInput.getKey(),
handleRestParameter(restInput.getKey(), restInput.getValue()));
}
Copy link
Copy Markdown
Contributor

@axlbonnet axlbonnet Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there are several inputs objets, then we must verify all inputs have only 1 element.

Copy link
Copy Markdown
Author

@Albireo-B Albireo-B Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an exception throw


// We handle resultsLocation the same as others restInputs, since it can either be a String or a List<String>
Object resultsLocation = execution.getResultsLocation();
if (resultsLocation != null) {
inputMap.put(
CoreConstants.RESULTS_DIRECTORY_PARAM_NAME,
handleRestParameter(CoreConstants.RESULTS_DIRECTORY_PARAM_NAME, resultsLocation));
// We handle resultsLocation the same as others restInputs, since it can either be a String or a List<String>
if (resultsLocation != null) {
inputMap.put(
CoreConstants.RESULTS_DIRECTORY_PARAM_NAME,
handleRestParameter(CoreConstants.RESULTS_DIRECTORY_PARAM_NAME, resultsLocation));
}

inputMaps.add(inputMap);
}

checkInputExecNameIsValid(execution.getName());
return initExecution(
execution.getPipelineIdentifier(), inputMap, execution.getTimeout(),
execution.getPipelineIdentifier(), inputMaps, execution.getTimeout(),
execution.getName(), execution.getStudyIdentifier());
}

Expand Down Expand Up @@ -378,7 +392,7 @@ private void checkInputExecNameIsValid(String input) throws VipException {
}

private String initExecution(String pipelineId,
Map<String, String> inputValues,
List<Map<String, String>> inputValues,
Integer timeoutInSeconds,
String executionName,
String studyId) throws VipException {
Expand All @@ -400,40 +414,51 @@ private String initExecution(String pipelineId,
if (pp.isReturnedValue()) {
continue;
}

List<Map<String, String>> mapsWithoutKey = inputValues.stream()
.filter(inputMap -> !inputMap.containsKey(pp.getName()))
.toList();

// ok if input is present
if (inputValues.get(pp.getName()) != null) {
continue;
}
// then ok if input has a default value (and we set it)
if (pp.getDefaultValue() != null) {
inputValues.put(pp.getName(), pp.getDefaultValue().toString());
if (mapsWithoutKey.isEmpty()) {
continue;
}
// then ok if it is optional
if (pp.isOptional()) {
continue;

for (Map<String, String> inputMap : mapsWithoutKey) {
// then ok if input has a default value (and we set it)
if (pp.getDefaultValue() != null) {
inputMap.put(pp.getName(), pp.getDefaultValue().toString());
continue;
}
// then ok if it is optional
if (pp.isOptional()) {
continue;
}
Copy link
Copy Markdown
Contributor

@axlbonnet axlbonnet Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it does not handle the case if some input object have a value and some not.

Copy link
Copy Markdown
Author

@Albireo-B Albireo-B Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, modified to handle only missing key values

Copy link
Copy Markdown
Contributor

@axlbonnet axlbonnet May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it more logic to do the test before the loop, like:

if (pp.getDefaultValue() != null) {
  mapsWithoutKey.stream.forEach(...)
  continue;
}
 // then ok if it is optional
if (pp.isOptional()) {
  continue;
}
// error : pp is an empty input with no default value and it is not optional
logger.error("Error initialising {}, missing {} parameter", pipelineId, pp.getName());
throw new VipException(ApiError.INPUT_FIELD_MISSING, pp.getName());

Copy link
Copy Markdown
Author

@Albireo-B Albireo-B May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it makes sense


// error : pp is an empty input with no default value and it is not optional
logger.error("Error initialising {}, missing {} parameter", pipelineId, pp.getName());
throw new VipException(ApiError.INPUT_FIELD_MISSING, pp.getName());
}
// error : pp is an empty input with no default value and it is not optional
logger.error("Error initialising {}, missing {} parameter", pipelineId, pp.getName());
throw new VipException(ApiError.INPUT_FIELD_MISSING, pp.getName());
}

// fill in overriddenInputs from explicit inputs
Map<String, String> overriddenInputs = p.getOverriddenInputs();
if (overriddenInputs != null) {
for (String key : overriddenInputs.keySet()) {
String value = overriddenInputs.get(key);
if (inputValues.containsKey(value)) {
inputValues.put(key, inputValues.get(value));
} else {
logger.error("Error initialising {}, missing {} parameter", pipelineId, value);
throw new VipException(ApiError.INPUT_FIELD_MISSING, value);
for (Map<String, String> inputMap : inputValues) {
if (inputMap.containsKey(value)) {
inputMap.put(key, inputMap.get(value));
} else {
logger.error("Error initialising {}, missing {} parameter", pipelineId, value);
throw new VipException(ApiError.INPUT_FIELD_MISSING, value);
}
}
}
}

boolean inputsContainsResultsDirectoryInput = inputValues
.containsKey(CoreConstants.RESULTS_DIRECTORY_PARAM_NAME);
boolean inputsContainsResultsDirectoryInput = inputValues.stream()
.allMatch(inputMap -> inputMap.containsKey(CoreConstants.RESULTS_DIRECTORY_PARAM_NAME));
boolean pipelineHasResultsDirectoryInput = p.getParameters().stream()
.anyMatch(param -> param.getName().equals(CoreConstants.RESULTS_DIRECTORY_PARAM_NAME));

Expand Down
14 changes: 10 additions & 4 deletions vip-api/src/main/java/fr/insalyon/creatis/vip/api/model/Execution.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
package fr.insalyon.creatis.vip.api.model;

import java.util.*;

import jakarta.validation.constraints.NotNull;

import com.fasterxml.jackson.databind.annotation.JsonDeserialize;

import fr.insalyon.creatis.vip.api.model.serializing.InputValuesDeserializer;

public class Execution {

private String identifier;
Expand All @@ -13,7 +18,8 @@ public class Execution {
private int timeout;
private ExecutionStatus status;
@NotNull
private Map<String, java.lang.Object> inputValues;
@JsonDeserialize(using = InputValuesDeserializer.class)
private List<Map<String, Object>> inputValues;
Copy link
Copy Markdown
Contributor

@axlbonnet axlbonnet Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to not break the API with this change, and support an object as before, and a list.
I see 2 solutions, but there may be others:

  • have inputValues as Object and have methods in Execution that help manage the different cases.
  • implement specific Execution Jackson Serializer and Deserializer.

Copy link
Copy Markdown
Author

@Albireo-B Albireo-B Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created InputValuesDeserializer which handles deserializing to List<Map<String,Object>>

private Map<String, List<java.lang.Object>> returnedFiles;

// optional arguments
Expand All @@ -25,7 +31,7 @@ public class Execution {
private Map<Integer, Map<String, Object>> jobs; // jobId -> status

public Execution() {
inputValues = new HashMap<>();
inputValues = new ArrayList<>();
returnedFiles = new HashMap<>();
jobs = new HashMap<>();
}
Expand Down Expand Up @@ -94,11 +100,11 @@ public void setStatus(ExecutionStatus status) {
this.status = status;
}

public Map<String, java.lang.Object> getInputValues() {
public List<Map<String, Object>> getInputValues() {
return inputValues;
}

public void setInputValues(Map<String, java.lang.Object> inputValues) {
public void setInputValues(List<Map<String, Object>> inputValues) {
this.inputValues = inputValues;
}

Expand Down
43 changes: 43 additions & 0 deletions .../src/main/java/fr/insalyon/creatis/vip/api/model/serializing/InputValuesDeserializer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package fr.insalyon.creatis.vip.api.model.serializing;

import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.deser.std.StdDeserializer;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;

public class InputValuesDeserializer extends StdDeserializer<List<Map<String, Object>>> {

public InputValuesDeserializer() {
super(List.class);
}

@Override
public List<Map<String, Object>> deserialize(JsonParser p, DeserializationContext ctx) throws IOException {
JavaType mapType = ctx.getTypeFactory().constructMapType(Map.class, String.class, Object.class);
// Array of dictionaries
if (p.currentToken() == JsonToken.START_ARRAY) {
List<Map<String, Object>> list = new ArrayList<>();
p.nextToken();
while (p.currentToken() != JsonToken.END_ARRAY) {
list.add(ctx.readValue(p, mapType));
p.nextToken();
}

return list;
// Unique dictionary
} else if (p.currentToken() == JsonToken.START_OBJECT) {
Map<String, Object> map = ctx.readValue(p, mapType);
return Collections.singletonList(map);
} else {
throw ctx.wrongTokenException(p, List.class, JsonToken.START_ARRAY,
"InputValues must be a dictionary or an array of dictionaries");
}
}
}
20 changes: 11 additions & 9 deletions vip-api/src/test/java/fr/insalyon/creatis/vip/api/data/ExecutionTestUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,12 @@ public class ExecutionTestUtils {
new GregorianCalendar(2016, 9, 2).getTime(),
"Exec test 1", SimulationStatus.Running.toString(), "engine 1", null);
execution1 = getExecution(simulation1, ExecutionStatus.RUNNING);
execution1.setInputValues(new HashMap<String,Object>() {{
put("param 1", "value 1");
put("param 2", "42");
}}
);
List<Map<String, Object>> parametersMaps = new ArrayList<>();
Map<String, Object> map = new HashMap<>();
map.put("param 1", "value 1");
map.put("param 2", "42");
parametersMaps.add(map);
execution1.setInputValues(parametersMaps);
execution1.clearReturnedFiles();

simulation1InData = Arrays.asList(
Expand All @@ -47,10 +48,11 @@ public class ExecutionTestUtils {
new GregorianCalendar(2016, 4, 29).getTime(),
"Exec test 2", SimulationStatus.Completed.toString(), "engine 1", null);
execution2 = getExecution(simulation2, ExecutionStatus.FINISHED);
execution2.setInputValues(new HashMap<String,Object>() {{
put("param2-1", "5.3");
}}
);
parametersMaps = new ArrayList<>();
parametersMaps.add(new HashMap<>() {{
put("param2-1", "5.3");
}});
execution2.setInputValues(parametersMaps);
execution2.setReturnedFiles(new HashMap<String,List<Object>>() {{
put("param2-res", Collections.singletonList("/vip/Home/testFile1.xml"));
}});
Expand Down
4 changes: 3 additions & 1 deletion ...rc/test/java/fr/insalyon/creatis/vip/api/rest/itest/processing/ExecutionControllerIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,9 @@ public void testInitBoutiquesExecution() throws Exception
expectedParams.put("testTextInput", List.of("best test text value"));
expectedParams.put("testFlagInput", List.of("false"));
expectedParams.put("results-directory", List.of("lfn:" + ServerMockConfig.TEST_USERS_ROOT + "/" + baseUser1.getFolder()));
String expectedInputs = workflowExecutionBusiness.getParametersAsJSONInput(expectedParams);
List<Map<String, List<String>>> paramsList = new ArrayList<>();
paramsList.add(expectedParams);
String expectedInputs = workflowExecutionBusiness.getParametersAsJSONInput(paramsList);
Assertions.assertEquals(expectedInputs, inputs);

// verify created workflow
Expand Down
10 changes: 6 additions & 4 deletions vip-api/src/test/resources/jsonObjects/execution1-name-updated.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
{
"name" : "Exec test 1 - modified",
"pipelineIdentifier" : "application 1/4.2",
"inputValues" : {
"param 1" : "test text",
"param 2" : "/path/test"
}
"inputValues" : [
{
"param 1" : "test text",
"param 2" : "/path/test"
}
]
}
10 changes: 6 additions & 4 deletions vip-api/src/test/resources/jsonObjects/execution1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
{
"name" : "Exec test 1",
"pipelineIdentifier" : "test application/4.2",
"inputValues" : {
"testFileInput" : "/vip/Home/path/to/input.in",
"testTextInput" : "best test text value"
}
"inputValues" : [
{
"testFileInput" : "/vip/Home/path/to/input.in",
"testTextInput" : "best test text value"
}
]
}
11 changes: 7 additions & 4 deletions ...n/src/main/java/fr/insalyon/creatis/vip/application/server/business/WorkflowBusiness.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,15 +139,18 @@ protected InputFileParser getInputFileParser(String currentUserFolder) {
return null;
}

public synchronized String launch(User user, List<String> groups, Map<String, String> parametersMap,
String appName, String version, String simulationName) throws VipException {
public synchronized String launch(User user, List<String> groups, List<Map<String, String>> parametersMaps,
String appName, String version, String simulationName) throws VipException {
Workflow workflow = null;

try {
checkVIPCapacities(user, simulationName);

AppVersion appVersion = appVersionBusiness.getVersion(appName, version);
Map<String, List<String>> parameters = getParameters(appVersion.getDescriptor(), parametersMap, user, groups);
List<Map<String, List<String>>> parametersMapList = new ArrayList<>();
for (Map<String, String> parameterMap : parametersMaps) {
parametersMapList.add(getParameters(appVersion.getDescriptor(), parameterMap, user, groups));
}

List<Resource> resources = resourceBusiness.getAvailableForExecution(user, appVersion);
if (resources.isEmpty()) {
Expand All @@ -159,7 +162,7 @@ public synchronized String launch(User user, List<String> groups, Map<String, St

appVersion.getSettings().put(ApplicationConstants.DEFAULT_EXECUTOR_GASW, resource.getType().toString());
try {
workflow = workflowExecutionBusiness.launch(engine.getEndpoint(), appVersion, user, simulationName, parameters, resource.getConfiguration());
workflow = workflowExecutionBusiness.launch(engine.getEndpoint(), appVersion, user, simulationName, parametersMapList, resource.getConfiguration());
} catch (Exception e) {
String mailSubject = "[VIP] Warn: Workflow submission failed!!";
String mailContent = "An error occured while submitting a workflow";
Expand Down
4 changes: 2 additions & 2 deletions ...n/java/fr/insalyon/creatis/vip/application/server/business/WorkflowExecutionBusiness.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ public WorkflowExecutionBusiness(Server server, WorkflowEngineInstantiator engin
}

public Workflow launch(String engineEndpoint, AppVersion appVersion, User user, String simulationName,
Map<String, List<String>> parameters, String executorConfig) throws VipException {
List<Map<String, List<String>>> parameters, String executorConfig) throws VipException {

try {
String workflowContent = appVersion.getDescriptor();
Expand Down Expand Up @@ -66,7 +66,7 @@ public void kill(String engineEndpoint, String simulationID) throws VipException
engine.kill(engineEndpoint, simulationID);
}

public String getParametersAsJSONInput(Map<String, List<String>> parameters) throws VipException {
public String getParametersAsJSONInput(List<Map<String, List<String>>> parameters) throws VipException {
try {
ObjectMapper mapper = new ObjectMapper();

Expand Down
4 changes: 3 additions & 1 deletion ...ion/src/main/java/fr/insalyon/creatis/vip/application/server/rpc/WorkflowServiceImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,8 +165,10 @@ public void launchSimulation(Map<String, String> parametersMap,
logger.info("received param {} : {}", p.getKey(), p.getValue());
}
fillInOverriddenInputs(parametersMap, applicationName, applicationVersion);
List<Map<String, String>> parametersMaps = new ArrayList<>();
parametersMaps.add(parametersMap);
String simulationID = workflowBusiness.launch(user, groups,
parametersMap, applicationName, applicationVersion, simulationName);
parametersMaps, applicationName, applicationVersion, simulationName);

trace(logger, "Simulation '" + simulationName + "' launched with ID '" + simulationID + "'.");
}
Expand Down
Loading