Feature suggestion: avoid failing on non-VecGeom-supported solids outside of GPU regions (tessellations)

[wdconinc]

In our EIC/ePIC geometry we have some tessellated solids, but they fall outside of the Ecal regions where we want to offload to the GPU with AdePT (they are in the Hcal and in the interaction region beampipe; our only regions to offload to AdePT are the barrel and endcap Ecals). We are running inside DD4hep (yeah... not easily reproducible at this point since in development, combination of DD4hep PR#1240 and #1606, but hoping to stabilize this soon). We are not using (and haven't really tested) the surface geometry model (ADEPT_USE_SURF), but happy to switch if that's a way to avoid this.

We encounter the following error:

Geant4AdePTPhysics INFO  AdePT: configured for 4 thread(s)
Geant4AdePTPhysics INFO  Registered AdePT tracking manager for e-/e+/gamma
Geant4AdePTPhysics INFO    MillionsOfTrackSlots: 10.0, MillionsOfHitSlots: 4.0
Geant4AdePTPhysics INFO    GPU region: EcalBarrelRegion
Geant4AdePTPhysics INFO    GPU region: EcalEndcapPRegion
Geant4AdePTPhysics INFO    GPU region: EcalEndcapNRegion
PhysicsList      INFO  ConstructDecays 0x2e194b30
 NUM OF THREADS ACCORDING TO G4: 4
Loading geometry via G4VG
status: Converting Geant4 geometry
info: VolumeTree size is 11.835 MBytes
info: navigation table size is 633.44 MBytes
Reading in covfie file for magnetic field: 
No magnetic field file provided!
Visiting geometry ...
Visiting geometry done

=== Woodcock tracking summary (host) ===
KineticEnergyLimit = 0 [G4 units]
Total WDT roots found: 0
Regions with WDT: 0
  (none)
=== End Woodcock tracking summary ===

=== AsyncAdePTTransport: initializing geometry and physics
CUDA Device stack limit: 16384

**************************************************** 
*  A runtime error has occured :                     
*    runtime error: Attempted to copy UnplacedTessellated to GPU.  This is not yet supported.: 0 failed at /scratch/wdconinc/spack/stage/spack-stage-vecgeom-2.0.0-rd2jcacpn2lvkfvmey3o45ltferq76se/spack-src/source/UnplacedTessellated.cpp:150
*  the program will have to be terminated - sorry.   
**************************************************** 

Based on a glace at AsyncAdePTTransport::InitializeGeometry, the entire geometry is copied to the GPU:

  cudaManager.LoadGeometry(world);
  auto world_dev = cudaManager.Synchronize();

and it may be possible to take the world geometry and traverse before passing off to cudaManager (or to provide a filter lambda to the LoadGeometry call and let VecGeom handle it). Those non-GPU region volumes that are unsupported could be turned into their bounding box or so.

Note: we have some non-Geant4 solids as well that are not supported (or possible to be supported) by VecGeom, so those would need a similar treatment.

[SeverinDiederichs]

That is an interesting request!
The tesselated solid will be available in VecGeom soon, so for that one, I think it would be less work to have it arrive in VecGeom than adding a workaround in AdePT.

Which other non-Geant4 solids do you have that are possibly not supported by VecGeom?

One thing would be important:
The unsupported volumes must not only be not in the GPU region, but also not touching the GPU region. Because in the Navigation, when exiting / entering the GPU region, it still calculates the actual NavigationState outside of the GPU, so it interacts with volumes on that touch the GPU region (although no particles are tracked within them).
Can you confirm that this is the case for EIC/ePIC setup?

[wdconinc]

That is an interesting request!
The tesselated solid will be available in VecGeom soon, so for that one, I think it would be less work to have it arrive in VecGeom than adding a workaround in AdePT.

Which other non-Geant4 solids do you have that are possibly not supported by VecGeom?

We have an Geant4 solid interface to Open Cascade which allows us to use BREP-described CAD volumes from STEP files inside Geant4 without needing any tessellation or approximation (the generality is at cost of computational expense).

One thing would be important:
The unsupported volumes must not only be not in the GPU region, but also not touching the GPU region. Because in the Navigation, when exiting / entering the GPU region, it still calculates the actual NavigationState outside of the GPU, so it interacts with volumes on that touch the GPU region (although no particles are tracked within them).
Can you confirm that this is the case for EIC/ePIC setup?

The respective volumes are all inside other volumes that are enveloping detectors that are at least one detector subsystem removed from our Ecals so this is indeed satisfied. It would be interesting to turn it into a criterion that can be consistently tested for though.