Question about HoloPart input requirements after SAMesh segmentation

[marceljhuber]

I'm using SAMesh to segment my GLB file, which generates:

A directory with 5 types of .npy files
Multiple PNG images
A model_segmented.glb file

Question: Does HoloPart only need the model_segmented.glb as input, or do I also need to run the provided code with the corresponding mask.npy files?
If mask files are required, which specific mask files should I use?

[Bocchi-NotLikeCodeVersion]

I'm using SAMesh to segment my GLB file, which generates:

A directory with 5 types of .npy files Multiple PNG images A model_segmented.glb file

Question: Does HoloPart only need the model_segmented.glb as input, or do I also need to run the provided code with the corresponding mask.npy files? If mask files are required, which specific mask files should I use?

May I ask if you have solved it? I also encountered the same problem.

[iambrc]

I think the labels for each face in the .glb model generated by SAMesh might be stored in face2label.json, rather than in the other mask.npy files. I tried using the following code to get the input needed for Holopart:

import trimesh
import numpy as np
import json

mesh = trimesh.load("segmented.glb", force="mesh")

# Load face labels from the JSON file
with open("face2label.json", 'r') as f:
    face_labels_dict = json.load(f)

# Create a mask array from the dictionary of face labels
# The dictionary maps string face indices to integer labels.
num_faces = mesh.faces.shape[0]
mask_npy = np.zeros(num_faces, dtype=int)
for face_idx_str, label in face_labels_dict.items():
    face_idx = int(face_idx_str)
    if face_idx < num_faces:
        mask_npy[face_idx] = label

mesh_parts = []
for part_id in np.unique(mask_npy):
    # Create a boolean mask for the current part ID
    face_mask = (mask_npy == part_id)
    # Get the face indices for the current part
    face_indices = np.where(face_mask)[0]
    
    # If there are no faces for this part, skip it
    if len(face_indices) == 0:
        continue
        
    # Extract the submesh using the face indices
    mesh_part = mesh.submesh([face_indices], append=True)
    mesh_parts.append(mesh_part)

# Export the collected parts as a single scene
trimesh.Scene(mesh_parts).export("input_mesh.glb")