Reduce latency of ICON frontend

[FlorenceBerterottiere]

Hello,

It seems that the frontend of ICON (especially the "data" tab) is quite slow to open. Would there be a way to load only the latest experiments, not the whole history?

Thanks a lot! :)

[beitler]

Hi Flo,

you mention the 'data tab is slow to open'. I'd be interested in more detail which UI interactions exactly incur the delay. I suspect the following interactions to be slow:

• Initial startup of the webapp: When the webapp starts, it loads all past jobs (via the scheduler.get_scheduled_jobs method call). This is of course a growing list of jobs as more and more jobs are executed. As discussed in another context, I think we should only load the most recent jobs on startup and allow the user to paginate through older jobs.
  --> Workaround: For now, you want to make sure that the page is fully loaded on initial startup before navigating to other views.

• In the data page, selecting a job with many data points: In case of a long-running real-time job, or a regular parameter scan with many data points, the data can grow to sizes where the serialization-I/O-deserialization chain adds noticeable delay, before the charts get rendered on the data page. I think the main issue here is the way that the serialization is adding significant overhead to the encoding of the time series data into json. We will have to look into this.
  --> Workaround: For now, you could change the max_transfer_bytes parameter to a smaller value, like 4_000_000:
  

  icon/src/icon/server/api/experiment_data_controller.py

  Lines 25 to 27 in 649eb59

  	async def get_experiment_data_by_job_id(
  	self, job_id: int, max_transfer_bytes: int = 50_000_000
  	) -> dict[str, Any]:

[FlorenceBerterottiere]

Hi Anton,

Thanks for your input!
I was referring to the initial startup of the webapp when loading the data page. Just wanted to flag that as you guys discussed it would be great if only the latest experiments would be loaded!

Yesterday the rest of the UI was slower than usual too, maybe just a network issue. I'll come back with a better description of those instances when they incur again! :)