Downloading big files can run out of memory

[juanluisrp]

With large attachments having the content of the files as a byte array in memory can cause problems running out of memory.

core-geonetwork/services/src/main/java/org/fao/geonet/api/records/attachments/AttachmentsApi.java

Lines 251 to 285 in b670c52

	public HttpEntity<byte[]> getResource(
	@Parameter(description = "The metadata UUID", required = true, example = "43d7c186-2187-4bcd-8843-41e575a5ef56") @PathVariable String metadataUuid,
	@Parameter(description = "The resource identifier (ie. filename)", required = true) @PathVariable String resourceId,
	@Parameter(description = "Use approved version or not", example = "true") @RequestParam(required = false, defaultValue = "true") Boolean approved,
	@Parameter(description = "Size (only applies to images). From 1px to 2048px.", example = "200") @RequestParam(required = false) Integer size,
	@Parameter(hidden = true) HttpServletRequest request) throws Exception {
	ServiceContext context = ApiUtils.createServiceContext(request);
	try (Store.ResourceHolder file = store.getResource(context, metadataUuid, resourceId, approved)) {
	ApiUtils.canViewRecord(metadataUuid, request);
	MultiValueMap<String, String> headers = new HttpHeaders();
	headers.add("Content-Disposition", "inline; filename=\"" + file.getMetadata().getFilename() + "\"");
	headers.add("Cache-Control", "no-cache");
	String contentType = getFileContentType(file.getPath());
	headers.add("Content-Type", contentType);
	if (contentType.startsWith("image/") && size != null) {
	if (size >= MIN_IMAGE_SIZE && size <= MAX_IMAGE_SIZE) {
	BufferedImage image = ImageIO.read(file.getPath().toFile());
	BufferedImage resized = ImageUtil.resize(image, size);
	ByteArrayOutputStream output = new ByteArrayOutputStream();
	ImageIO.write(resized, "png", output);
	output.flush();
	byte[] imagesB = output.toByteArray();
	output.close();
	return new HttpEntity<>(imagesB, headers);
	} else {
	throw new IllegalArgumentException(String.format(
	"Image can only be resized from %d to %d. You requested %d.",
	MIN_IMAGE_SIZE, MAX_IMAGE_SIZE, size));
	}
	} else {
	return new HttpEntity<>(Files.readAllBytes(file.getPath()), headers);
	}

This should be written to the servlet output stream directly instead of loading the file in memory.

[fxprunayre]

I did some changes in the FME store implementation to stream response (from FME to client) - maybe that can be used here too
https://github.com/geonetwork/core-geonetwork/pull/6688/files#diff-02af4158bb9acdbe0f63adb7a37d891a2d364afce173b894b5bdc302511fd930R276 - not an expert on this.

[juanluisrp]

Yes, that's exactly what is needed, although you don't have to call close() on the response output stream: the servlet container will call it.

[fxprunayre]

I also noticed that we create temporary files

core-geonetwork/core/src/main/java/org/fao/geonet/api/records/attachments/S3Store.java

Line 263 in b670c52

path = Files.createTempFile("", getFilename(object.getKey()));

- and when you have 10Go files - it can take times before something is returned to the client ...

[juanluisrp]

Also the temporary resized image isn't deleted after using it.

[manubl2B]

Hello, I'm sorry but it's very problematic with big files. the solution seems to be found and is it possible to correct it for next release ?
Thank you very much !