Windows-Specific Cluster Failures When Estimating Behavior–Bipartite Interaction Effects in RSiena

[petershan1119]

bipEffects <- set_effect(bipEffects, avInAltDist2, depvar = "cos_align_r", covar1 = "analys")

bipEffects <- includeEffects(bipEffects, outdeg, interaction1 = "analys", name = "cos_align_r")

When estimating behavior–network interaction effects—specifically avInAltDist2 and outdeg for the behavioral variable (cos_align_r) conditioned on the bipartite network (analys)—I consistently encounter failures on Windows when using the useCluster option. The estimation terminates with the error:

bipResults <- siena07(bipModel, data = bipData, effects = bipEffects, useCluster = TRUE, nbrNodes = 8, verbose = TRUE)

Error in unserialize(node$con): error reading from connection

Notably, this problem does not arise when other objective functions are included, nor does it occur when useCluster = FALSE, although serial execution substantially increases runtime. In addition, the same specification runs successfully on macOS under otherwise comparable conditions.

Do you have any insight into why this combination of effects appears to trigger cluster failures specifically on Windows, or whether this might reflect platform-specific limitations in RSiena’s parallelization strategy for behavior–bipartite interaction terms? Thanks!

[HenrikBengtsson]

Error in unserialize(node$con): error reading from connection

This suggests that the parallel worker terminated abruptly (aka crashed; died; ...). This can happen when you run out of memory, but also for other reasons. Each parallel workers ("node", "core") consumes memory, so the more parallel workers you use, the greater the risk is to run out of memory.

Looking at the source code for siena07(), the reason for it might work better on non-Windows systems, is because there the function can use "forked" parallelization, which can be more memory efficient. Forked parallelization is not supported on Windows.

In your example, you use nbrNodes = 8. Start with nbrOfNodes = 2 and see if that works. If it does, try with more.

[petershan1119]

On the hardware side, I don’t think this is a memory or CPU constraint. My workstation laptop has 128GB of RAM and an 8-core CPU, and the failure persists even when I set nbrNodes = 2. That makes me suspect the issue is more specific to the Windows/PSOCK setup (or how the objects for these particular effects are serialized across workers), rather than a resource limitation per se.

In your experience, is there a viable workaround on Windows—e.g., changes to the cluster type or initialization, or any RSiena-specific options that avoid the unserialize(node$con) failure for these behavior–network interaction effects? Or is the most practical solution simply to run this specification on a non-Windows machine?

If helpful, I’m happy to share a minimal reproducible example (or details on the specific effects combination) that reliably triggers the error on Windows but not on macOS.

[HenrikBengtsson]

I haven't verified that it works, but to get better error message, you might be able to use:

library(future)
cl <- makeClusterFuture()
plan(multisession, workers = 2)

bipResults <- siena07(bipModel, data = bipData, effects = bipEffects, cl = cl, verbose = TRUE)

That should give two parallel workers just like nbrOfNodes = 2 does, but when it fails, there will be more information in the error message.

[petershan1119]

Thank you—yes, I tried it exactly as you suggested, but unfortunately I ran into the same failure again.

[petershan1119]

>   if (exists("cl")) try(parallel::stopCluster(cl), silent = TRUE)
> 
>   plan(multisession, workers = 2)
> 
>   cl <- makeClusterFuture()
> 
>   bipResults <- siena07(bipModel, data = bipData, effects = bipEffects, cl = cl, verbose = TRUE)

Stochastic approximation algorithm.
Initial value for gain parameter = 0.2828427.
Reduction factor for gain parameter = 0.5.
Start of the algorithm.
Observed function values are 
  1.     236.00000   2.     356.00000   3.     462.00000   4.     344.00000 
  5.     456.00000   6.     466.00000   7.     502.00000   8.    5432.00000 
  9. -228735.99239  10.   68106.00000  11.   39473.19963  12.    1367.45159 
 13.    2722.67941  14.    3762.00000  15.    6042.00000  16.     262.78589 
 17.     756.00000  18.    6611.59400  19.   29834.00000  20.     989.89925 
 21.     210.00000  22.     126.00000  23.     256.00000  24.     364.00000 
 25.     336.00000  26.     404.00000  27.     392.00000  28.    1621.00000 
 29.  -51469.92035  30.    8938.00000  31.    9520.73875  32.     147.62472 
 33.    3935.74511  34.    5844.00000  35.    4196.00000  36.     237.87714 
 37.     800.00000  38.    3279.11319  39.    7068.00000  40.     185.75337 
 41.     883.00000  42.     958.00000  43.    1070.00000  44.    1475.00000 
 45.    1048.00000  46.     931.00000  47.     980.00000  48.   20371.00000 
 49.  242526.00000  50.   61041.69258  51.   85483.48066  52.     909.31516 
 53.   26004.00000  54.   -5174.50151  55.    5436.00000  56.     180.00000 
 57.     210.00000  58.     243.00000  59.     195.00000  60.     178.00000 
 61.     147.00000  62.     152.00000  63.      62.91946  64.    3579.96677 
 65.    2919.42836  66.     382.20505  67.    1099.97062  68.     143.30053 
 69.   10461.30204  70.     858.27176
Current parameter values:
 1.53872967  2.32047209  3.01101123  2.24229785  2.97192411  3.03706931 
 3.27159204 -0.72940864  0.00000000  0.00000000  0.00000000  0.00000000 
 0.00000000  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000 
 0.00000000  0.00000000  1.36935214  0.82213245  1.66902007  2.37258825 
 2.19018169  2.63316906  2.55499482 -1.28352438  0.00000000  0.00000000 
 0.00000000  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000 
 0.00000000  0.00000000  0.00000000  0.00000000  5.73439015  6.22140098 
 6.94867048  9.57852894  6.80581397  6.04607708  6.36425749 -1.05250266 
 0.00000000  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000 
 0.00000000  1.34273144  1.60733349  1.94086409  1.42481891  1.35334260 
 1.18065469  1.32583605  0.02598636  0.00000000  0.00000000  0.00000000 
 0.00000000  0.00000000  0.00000000  0.00000000
error: Future (<unnamed-156>) of class MultisessionFuture interrupted (pid 696) [future <unnamed-156> (5beedc1ad74a30fd2bba964ae95bccff-156); on 5beedc1ad74a30fd2bba964ae95bccff@LAPTOP-BQ4757LV<25168> at 2026-01-30 10:11:33.537799]

[HenrikBengtsson]

The error: Future (<unnamed-156>) of class MultisessionFuture interrupted (pid 696) ... means that the parallel worker was interrupted by something. Here interrupt means that something (the operating system?) signaled an "interrupt" to the worker asking it to interrupt the processing. When running sequential, a user that signal an interrupt by pressing Ctrl-C or click "STOP" in RStudio, but when running in parallel, that's not easy for the user to do. So, there's something on the system that triggers that interrupt.

The Futureverse and cl <- future::makeClusterFuture() handles interrupts and other types of errors, which is why we get a much more informative error message. When using cl <- parallel::makeCluster() - what you used indirectly before - that is not handled. We just get an obscure error message that the connection is now broken.

Another benefit from using Futureverse here is that we also get the all that output that is produced on the parallel worker. With parallel::makeCluster() that is completely lost. Search the source code for those messages, it looks like it gets to:

rsiena/R/robmon.r

Line 245 in ef7a58e

WriteOutTheta(z)

but then it's interrupted somewhere after that.

That's all I can say and help with here. The maintainer of RSiena might be able to say more.

If helpful, I’m happy to share a minimal reproducible example (or details on the specific effects combination) that reliably triggers the error on Windows but not on macOS.

That is always useful when troubleshooting. It'll help others to confirm the problem and it increases the chance for the maintainer to dive into the problem and try to fix it.