Issue with image rendering

[pwalczysko]

Observing an issue of slow service when images are being rendered.

docker-compose run locally on M1 (first description of the general narrative):

• touch "foo&sleepOpenBytes=10000.fake"
• omero import foo\&sleepOpenBytes=10000.fake <- import 120 times in a loop
• open an OMERO.figure with the 120 images
• slide the rnd setting slider in OMERO.figure
• Go back to webclient and click refresh
• Observe 5 minutes wait before the webclient is back

jstacks will be delivered as I progress with them - see #174 (comment)

cc @jburel @joshmoore @will-moore @sbesson

[pwalczysko]

In order to be able to take jstacks inside the docker, need (from inside the docker as root)

docker exec --user 0 -i docker-example-omero-omeroserver-1 bash
sudo dnf install java-11-openjdk-devel.aarch64

[pwalczysko]

In order to take jstacks and store them (for now) inside the docker, develop a bash script

collect.sh

#!/bin/sh
while true
do
  echo "taking jstack"
  docker exec --user 0 -i docker-example-omero-omeroserver-1 bash  < test2.sh
  sleep 30
done

where test2.sh is

sudo -u omero-server jstack $(sudo -u omero-server /opt/omero/server/OMERO.server/bin/omero admin ice server pid Blitz-0) > /tmp/jstack-ro${n}-$(date -Iseconds).txt

[pwalczysko]

Docker M1 narrative - corresponding jstacks see

idr0-slot3:/uod/idr/scratch/jstacks-petr/jstacks-M1-docker-20240130-first

OMERO.server 5.6.10
OS Rockylinux9
db.poolsize=10

11.56 Start taking jstacks
11.56 Open an image in OMERO.figure
11.57 Refresh webclient - comes back immediately
11.57 Change rnd settings on that one image in figure
11.58 Refresh webclient - comes back immediately
11.59 Challenge the server with url opening 120 images in figure
11.59 Refresh webclient - comes back after 10 seconds
12.01 Change the rnd in figure on the 120 images
12.01 Refresh webclient
12.02 Webclient still did not come back
12.03 Webclient still did not come back
12.03 the webclient has vanished from the browser
12.04 the webclient is getting reloaded - slowly
12.04 the figure tab is still working on the rendering, aprox half of the panels done
12.05 very slowly, the webclient has the lhp almost rendered
12.06 webclient came back
12.06 around 90% of the panels in the figure are rendered, the rest still spinners
12.08 stop taking jstacks

[pwalczysko]

ome-training-3 VM narrative - corresponding jstacks see

idr0-slot3:/uod/idr/scratch/jstacks-petr/jstacks-ome-train-3-VM-poolsize60-20243001

OMERO.server 5.6.8
OS CentOS7
db.poolsize=60

16.41 start taking jstacks
16.43 open one image in OMERO.figure and change rnd on it
16.44 Refresh webclient - returns immediately
16.49 challenge server with url of 120 images in OMERO.figure https://ome-training-3.openmicroscopy.org/figure/new/?image=58007&image=58008&image=58009&image=58010&image=58011&image=58012&image=58013&image=58014&image=58015&image=58016&image=58017&image=58018&image=58019&image=58020&image=58021&image=58022&image=58023&image=58024&image=58025&image=58026&image=58027&image=58028&image=58029&image=58030&image=58031&image=58032&image=58033&image=58034&image=58035&image=58036&image=58037&image=58038&image=58039&image=58040&image=58041&image=58042&image=58043&image=58044&image=58045&image=58046&image=58047&image=58048&image=58049&image=58050&image=58051&image=58052&image=58053&image=58054&image=58055&image=58056&image=58057&image=58058&image=58059&image=58060&image=58061&image=58062&image=58063&image=58064&image=58065&image=58066&image=58067&image=58068&image=58069&image=58070&image=58071&image=58072&image=58073&image=58074&image=58075&image=58076&image=58077&image=58078&image=58079&image=58080&image=58081&image=58082&image=58083&image=58084&image=58085&image=58086&image=58087&image=58088&image=58089&image=58090&image=58091&image=58092&image=58093&image=58094&image=58095&image=58096&image=58097&image=58098&image=58099&image=58100&image=58101&image=58102&image=58103&image=58104&image=58105&image=58106&image=58107&image=58108&image=58109&image=58110&image=58111&image=58112&image=58113&image=58114&image=58115&image=58116&image=58117&image=58118&image=58119&image=58120&image=58121&image=58122&image=58123&image=58124&image=58125&image=58126&_=1706633063371
16.49 refresh webclient
16.49 webclient comes back after 10 seconds but the single folders are not resposive very fast 
16.50 change rnd settings in figure
16.50 refresh webclient
16.51 webclient still did not return
16.52 webclient still did not return
16.54 webclient still did not return
16.55 webclient returned
16.56 stop taking jstacks

cc @joshmoore

[joshmoore]

The hangs certainly make sense from the jstacks, but neither in them, nor locally in my own, am I seeing the type of deadlock that I was looking for. :/

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/omero-duplicate-weird-issues/68565/20