Merge pull request #1382 from square/wenli/improve-visualizer

Improve visualizer usability

Author: wenli-cai

gradle/libs.versions.toml

@@ -98,7 +98,6 @@ turbine = "1.0.0"
 vanniktech-publish = "0.32.0"
 
 [plugins]
-
 kotlin-jvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" }
 
 compose-compiler = { id = "org.jetbrains.kotlin.plugin.compose", version.ref = "kotlin" }

workflow-trace-viewer/README.md

@@ -1,6 +1,8 @@
 # Workflow Trace Viewer
 
-A Compose for Desktop app that can be used to view Workflow traces.
+A Compose for Desktop application that visualizes and debugs Workflow execution traces. This tool
+helps developers understand the hierarchical structure and execution flow of their Workflow
+applications by providing both file-based and live streaming trace visualization.
 
 ## Running
 
@@ -10,20 +12,75 @@ It can be run via Gradle using:
 ./gradlew :workflow-trace-viewer:run
 ```
 
-By Default, the app will be in file parsing mode, where you are able to select a previously recorded workflow trace file for it to visualize the data.
+## Usage Guide
 
-By hitting the bottom switch, you are able to toggle to live stream mode, where data is directly pulled from the emulator into the visualizer. A connection can only happen once. If there needs to be rerecording of the trace, the emulator must first be restarted, and then the app must be restarted as well. This is due to the fact that any open socket will consume all render pass data, meaning there is nothing to read from the emulator.
+By default, the app will be in file parsing mode, where you are able to select a previously recorded
+workflow trace file for it to visualize the data. Once the workflow tree is rendered
+in [File](#file-mode) or [Live](#live-mode) mode, you can switch frames (
+see [Terminology](#Terminology)) to see different events that fired. All nodes are color coded based
+on what had happened during this frame, and a text diff will show the specific changes. You can open
+the right node panel and left click a box get a more detailed view of the specific node, or right
+click to expand/collapse a specific node's children.
 
-It is ***important*** to run the emulator first before toggling to live mode.
+<img src="https://github.com/square/workflow-kotlin/raw/wenli/improve-visualizer/workflow-trace-viewer/docs/demo.gif" width="320" alt="Demo" />
+
+### File Mode
+
+Once a file of the live data is saved, it can easily be uploaded to retrace the steps taken during
+the live session. Currently, text/json files that are saved from recordings only contain raw data,
+meaning it is simply a list of lists of node renderings.
+
+<img src="https://github.com/square/workflow-kotlin/raw/wenli/improve-visualizer/workflow-trace-viewer/docs/file_mode.gif" width="320" alt="File Mode" />
+
+### Live Mode
+
+By hitting the bottom switch, you are able to toggle to live stream mode, where data is directly
+pulled from the emulator into the visualizer. To do so:
+
+- Start the app (on any device)
+- Start the app, and toggle the switch to enter Live mode
+- Select the desired device
+
+Once in Live mode, frames will appear as you interact with the app. You may also save the current
+data into a file saved in `~/Downloads` to be used later (this action will take some time, so it may
+not appear immediately)
+
+Render pass data is passively stored in a buffer before being sent to the visualizer, so you do not
+need to immediately open/run the app to "catch" everything. However, since the the buffer has
+limited size, it's strongly recommended to avoid interacting with the app — beyond starting it —
+before Live mode has been triggered; this helps to avoid losing data.
+
+<img src="https://github.com/square/workflow-kotlin/raw/wenli/improve-visualizer/workflow-trace-viewer/docs/live_mode.gif" width="320" alt="Live Mode" />
+
+### Note
+
+A connection can only happen once. There is currently no support for a recording of the trace data
+due to the fact that an open socket will consume all render pass data when a connection begins. To
+restart the recording:
+
+- (optional) Save the current trace
+- Switch out of Live mode
+- Restart the app
+- Switch back to Live mode, and the
 
 ### Terminology
 
-**Trace**: A trace is a file — made up of frames — that contains the execution history of a Workflow. It includes information about render passes, how states have changed within workflows, and the specific props being passed through. The data collected to generate these should be in chronological order, and allows developers to step through the process easily.
+`Trace`: A trace is a file — made up of frames — that contains the execution history of a Workflow.
+It includes information about render passes, how states have changed within workflows, and the
+specific props being passed through.
 
-**Frame**: Essentially a "snapshot" of the current "state" of the whole Workflow tree. It contains relevant information about the changes in workflow states and how props are passed throughout.
+`Frame`: Essentially a "snapshot" of the current "state" of the whole Workflow tree. It contains
+relevant information about the changes in workflow states and how props are passed throughout.
 
-- Note that "snapshot" and "state" are different from `snapshotState` and `State`, which are idiomatic to the Workflow library.
+- Note that "snapshot" and "state" are different from `snapshotState` and `State`, which are
+  idiomatic to the Workflow library.
 
 ### External Libraries
 
-[FileKit](https://github.com/vinceglb/FileKit) is an external library made to apply file operations on Kotlin and KMP projects. It's purpose in this app is to allow developers to upload their own json trace files. The motivation for its use is to quickly implement a file picker.
+[FileKit](https://github.com/vinceglb/FileKit) is an external library made to apply file operations
+on Kotlin and KMP projects. This simplified the development process of allowing file selection
+
+## Future
+
+This app can be integrated into the process of anyone working with Workflow, so it's highly
+encouraged for anyone to make improvements that makes their life a little easier using this app.

workflow-trace-viewer/api/workflow-trace-viewer.api

@@ -12,21 +12,19 @@ public final class com/squareup/workflow1/traceviewer/MainKt {
 public final class com/squareup/workflow1/traceviewer/ui/ComposableSingletons$WorkflowInfoPanelKt {
 	public static final field INSTANCE Lcom/squareup/workflow1/traceviewer/ui/ComposableSingletons$WorkflowInfoPanelKt;
 	public fun <init> ()V
-	public final fun getLambda$-1066151803$wf1_workflow_trace_viewer ()Lkotlin/jvm/functions/Function3;
 	public final fun getLambda$-1653175968$wf1_workflow_trace_viewer ()Lkotlin/jvm/functions/Function3;
 }
 
-public final class com/squareup/workflow1/traceviewer/util/ComposableSingletons$UploadFileKt {
-	public static final field INSTANCE Lcom/squareup/workflow1/traceviewer/util/ComposableSingletons$UploadFileKt;
+public final class com/squareup/workflow1/traceviewer/ui/control/ComposableSingletons$SearchBoxKt {
+	public static final field INSTANCE Lcom/squareup/workflow1/traceviewer/ui/control/ComposableSingletons$SearchBoxKt;
 	public fun <init> ()V
-	public final fun getLambda$-1046372460$wf1_workflow_trace_viewer ()Lkotlin/jvm/functions/Function3;
+	public final fun getLambda$-1622160509$wf1_workflow_trace_viewer ()Lkotlin/jvm/functions/Function2;
+	public final fun getLambda$673874721$wf1_workflow_trace_viewer ()Lkotlin/jvm/functions/Function2;
 }
 
-public final class com/squareup/workflow1/traceviewer/util/JsonParserKt {
-	public static final field ROOT_ID Ljava/lang/String;
-}
-
-public final class com/squareup/workflow1/traceviewer/util/UploadFileKt {
-	public static final fun UploadFile (Lkotlin/jvm/functions/Function1;Landroidx/compose/ui/Modifier;Landroidx/compose/runtime/Composer;II)V
+public final class com/squareup/workflow1/traceviewer/ui/control/ComposableSingletons$UploadFileKt {
+	public static final field INSTANCE Lcom/squareup/workflow1/traceviewer/ui/control/ComposableSingletons$UploadFileKt;
+	public fun <init> ()V
+	public final fun getLambda$-1248702605$wf1_workflow_trace_viewer ()Lkotlin/jvm/functions/Function3;
 }
 

workflow-trace-viewer/build.gradle.kts

@@ -15,6 +15,7 @@ kotlin {
         implementation(compose.runtime)
         implementation(compose.foundation)
         implementation(compose.material)
+        implementation(compose.material3)
         implementation(compose.ui)
         implementation(compose.components.resources)
         implementation(compose.components.uiToolingPreview)
@@ -25,6 +26,7 @@ kotlin {
         implementation(compose.materialIconsExtended)
         implementation(libs.squareup.moshi.kotlin)
         implementation(libs.filekit.dialogs.compose)
+        implementation(libs.java.diff.utils)
       }
     }
     jvmTest {

workflow-trace-viewer/src/jvmMain/kotlin/com/squareup/workflow1/traceviewer/App.kt

@@ -1,27 +1,38 @@
 package com.squareup.workflow1.traceviewer
 
+import androidx.compose.foundation.layout.Arrangement
 import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.padding
 import androidx.compose.runtime.Composable
 import androidx.compose.runtime.LaunchedEffect
 import androidx.compose.runtime.getValue
 import androidx.compose.runtime.mutableFloatStateOf
 import androidx.compose.runtime.mutableIntStateOf
 import androidx.compose.runtime.mutableStateListOf
+import androidx.compose.runtime.mutableStateMapOf
 import androidx.compose.runtime.mutableStateOf
 import androidx.compose.runtime.remember
 import androidx.compose.runtime.setValue
 import androidx.compose.runtime.snapshotFlow
+import androidx.compose.runtime.snapshots.SnapshotStateMap
 import androidx.compose.ui.Alignment
 import androidx.compose.ui.Modifier
 import androidx.compose.ui.geometry.Offset
+import androidx.compose.ui.layout.onSizeChanged
+import androidx.compose.ui.unit.IntSize
+import androidx.compose.ui.unit.dp
 import com.squareup.workflow1.traceviewer.model.Node
 import com.squareup.workflow1.traceviewer.model.NodeUpdate
-import com.squareup.workflow1.traceviewer.ui.FrameSelectTab
 import com.squareup.workflow1.traceviewer.ui.RightInfoPanel
-import com.squareup.workflow1.traceviewer.ui.TraceModeToggleSwitch
-import com.squareup.workflow1.traceviewer.util.RenderTrace
+import com.squareup.workflow1.traceviewer.ui.control.DisplayDevices
+import com.squareup.workflow1.traceviewer.ui.control.FileDump
+import com.squareup.workflow1.traceviewer.ui.control.FrameNavigator
+import com.squareup.workflow1.traceviewer.ui.control.SearchBox
+import com.squareup.workflow1.traceviewer.ui.control.TraceModeToggleSwitch
+import com.squareup.workflow1.traceviewer.ui.control.UploadFile
 import com.squareup.workflow1.traceviewer.util.SandboxBackground
-import com.squareup.workflow1.traceviewer.util.UploadFile
+import com.squareup.workflow1.traceviewer.util.parser.RenderTrace
 import io.github.vinceglb.filekit.PlatformFile
 
 /**
@@ -31,14 +42,20 @@ import io.github.vinceglb.filekit.PlatformFile
 internal fun App(
   modifier: Modifier = Modifier
 ) {
+  var appWindowSize by remember { mutableStateOf(IntSize(0, 0)) }
   var selectedNode by remember { mutableStateOf<NodeUpdate?>(null) }
-  val workflowFrames = remember { mutableStateListOf<Node>() }
+  var frameSize by remember { mutableIntStateOf(0) }
+  var rawRenderPass by remember { mutableStateOf("") }
   var frameIndex by remember { mutableIntStateOf(0) }
   val sandboxState = remember { SandboxState() }
+  val nodeLocations = remember { mutableStateListOf<SnapshotStateMap<Node, Offset>>() }
 
   // Default to File mode, and can be toggled to be in Live mode.
+  var active by remember { mutableStateOf(false) }
   var traceMode by remember { mutableStateOf<TraceMode>(TraceMode.File(null)) }
   var selectedTraceFile by remember { mutableStateOf<PlatformFile?>(null) }
+  // frameIndex is set to -1 when app is in Live Mode, so we increment it by one to avoid off-by-one errors
+  val frameInd = if (traceMode is TraceMode.Live) frameIndex + 1 else frameIndex
 
   LaunchedEffect(sandboxState) {
     snapshotFlow { frameIndex }.collect {
@@ -47,47 +64,83 @@ internal fun App(
   }
 
   Box(
-    modifier = modifier
+    modifier = modifier.onSizeChanged {
+      appWindowSize = it
+    }
   ) {
     fun resetStates() {
       selectedTraceFile = null
       selectedNode = null
       frameIndex = 0
-      workflowFrames.clear()
+      frameSize = 0
+      rawRenderPass = ""
+      active = false
+      nodeLocations.clear()
     }
 
     // Main content
     SandboxBackground(
+      appWindowSize = appWindowSize,
       sandboxState = sandboxState,
     ) {
       // if there is not a file selected and trace mode is live, then don't render anything.
-      val readyForFileTrace = traceMode is TraceMode.File && selectedTraceFile != null
-      val readyForLiveTrace = traceMode is TraceMode.Live
+      val readyForFileTrace = TraceMode.validateFileMode(traceMode)
+      val readyForLiveTrace = TraceMode.validateLiveMode(traceMode)
+
       if (readyForFileTrace || readyForLiveTrace) {
+        active = true
         RenderTrace(
           traceSource = traceMode,
           frameInd = frameIndex,
-          onFileParse = { workflowFrames.addAll(it) },
-          onNodeSelect = { node, prevNode ->
-            selectedNode = NodeUpdate(node, prevNode)
-          },
-          onNewFrame = { frameIndex += 1 }
+          onFileParse = { frameSize += it },
+          onNodeSelect = { selectedNode = it },
+          onNewFrame = { frameIndex += 1 },
+          onNewData = { rawRenderPass += "$it," },
+          storeNodeLocation = { node, loc -> nodeLocations[frameInd] += (node to loc) }
         )
       }
     }
 
-    FrameSelectTab(
-      frames = workflowFrames,
-      currentIndex = frameIndex,
-      onIndexChange = { frameIndex = it },
-      modifier = Modifier.align(Alignment.TopCenter)
-    )
-
-    RightInfoPanel(
-      selectedNode = selectedNode,
+    Column(
       modifier = Modifier
-        .align(Alignment.TopEnd)
-    )
+        .align(Alignment.TopCenter)
+        .padding(top = 8.dp),
+      verticalArrangement = Arrangement.spacedBy(8.dp),
+      horizontalAlignment = Alignment.CenterHorizontally
+    ) {
+      if (active) {
+        // Frames that appear in composition may not happen sequentially, so when the current frame
+        // locations is null, that means we've skipped frames and need to fill all the intermediate
+        // ones. e.g. Frame 1 to Frame 10
+        if (nodeLocations.getOrNull(frameInd) == null) {
+          // frameSize has not been updated yet, so on the first frame, frameSize = nodeLocations.size = 0,
+          // and it will append a new map
+          while (nodeLocations.size <= frameSize) {
+            nodeLocations += mutableStateMapOf()
+          }
+        }
+
+        val frameNodeLocations = nodeLocations[frameInd]
+        SearchBox(
+          nodes = frameNodeLocations.keys.toList(),
+          onSearch = { name ->
+            sandboxState.scale = 1f
+            val node = frameNodeLocations.keys.first { it.name == name }
+            val newX = (sandboxState.offset.x - frameNodeLocations.getValue(node).x
+              + appWindowSize.width / 2)
+            val newY = (sandboxState.offset.y - frameNodeLocations.getValue(node).y
+              + appWindowSize.height / 2)
+            sandboxState.offset = Offset(x = newX, y = newY)
+          },
+        )
+
+        FrameNavigator(
+          totalFrames = frameSize,
+          currentIndex = frameIndex,
+          onIndexChange = { frameIndex = it },
+        )
+      }
+    }
 
     TraceModeToggleSwitch(
       onToggle = {
@@ -96,13 +149,12 @@ internal fun App(
           frameIndex = 0
           TraceMode.File(null)
         } else {
-          // TODO: TraceRecorder needs to be able to take in multiple clients if this is the case
           /*
           We set the frame to -1 here since we always increment it during Live mode as the list of
           frames get populated, so we avoid off by one when indexing into the frames.
            */
           frameIndex = -1
-          TraceMode.Live
+          TraceMode.Live()
         }
       },
       traceMode = traceMode,
@@ -120,6 +172,26 @@ internal fun App(
         modifier = Modifier.align(Alignment.BottomStart)
       )
     }
+
+    if (traceMode is TraceMode.Live && (traceMode as TraceMode.Live).device == null) {
+      DisplayDevices(
+        onDeviceSelect = { selectedDevice ->
+          traceMode = TraceMode.Live(selectedDevice)
+        },
+        devices = listDevices(),
+        modifier = Modifier.align(Alignment.Center)
+      )
+
+      FileDump(
+        trace = rawRenderPass,
+        modifier = Modifier.align(Alignment.BottomStart)
+      )
+    }
+
+    RightInfoPanel(
+      selectedNode = selectedNode,
+      modifier = Modifier.align(Alignment.TopEnd)
+    )
   }
 }
 
@@ -134,5 +206,25 @@ internal class SandboxState {
 
 internal sealed interface TraceMode {
   data class File(val file: PlatformFile?) : TraceMode
-  data object Live : TraceMode
+  data class Live(val device: String? = null) : TraceMode
+
+  companion object {
+    fun validateLiveMode(traceMode: TraceMode): Boolean {
+      return traceMode is Live && traceMode.device != null
+    }
+
+    fun validateFileMode(traceMode: TraceMode): Boolean {
+      return traceMode is File && traceMode.file != null
+    }
+  }
+}
+
+/**
+ * Allows users to select from multiple devices that are currently running.
+ */
+private fun listDevices(): List<String> {
+  val process = ProcessBuilder("adb", "devices", "-l").start()
+  process.waitFor()
+  // We drop the header "List of devices attached"
+  return process.inputStream.bufferedReader().readLines().drop(1).dropLast(1)
 }