Intermittent failure to start up

[McSherry]

Summary

We've observed that Fastino occasionally fails to initialise properly on power-up.

Our suspicion is that the bitstream is not being read correctly because the power-up sequencing for operating the FPGA in SPI master mode is not adhered to. This requirement is that $V_{CC}$ and $V_{CC\_SPI}$ are energised at least 0.25 ms before $V_{PP\_2V5}$.¹

Measurements

We first suspected a failure of the PLLs to lock, so we measured the clock output from two Kasli-SoC masters. This was perfectly fine (N-side as at the Fastino end of the ribbon cable, while under failure, is the blue trace in image below):²

What we also tested was adding three registers to the FPGA design. These were all set up to toggle every cycle and their outputs are shown as traces D0, D1, and D2. They were:

• D0, a register attached directly to the output from one of the iCE40's global input buffers, which should be clocked directly from the ~35.71 MHz that comes from Kasli-SoC with no intervening PLLs
• D1, a register attached to the ~47.62 MHz SPI clock generated from one of the iCE40's PLLs
• D2, a register attached to the 250 MHz clock generated from the iCE40's second PLL

Under a PLL misconfiguration/misuse (or a PLL-specific failure-to-initialise that Greg has said is a known issue with early-revision iCE40HXs), I'd expect D0 to be toggling while D1 and D2 remain steady. Because D0 also remains steady, this looks a lot more like a complete failure to initialise the FPGA. Since we know the SPI master power supply sequencing isn't being adhered to, it seems likely that this is a failure to read the bitstream.

Footnotes

1. Lattice DS02029-4.0, table 4.3 ↩

2. The top four logic traces aren't great representations of the true waveform. The logic probe was attached to vacant EEM connectors on Kasli-SoC mainly to verify that there was clock output on all expected connectors under failure. The probe didn't use a local ground and likely there was some mistriggering from the LVDS signal levels. ↩

[gkasprow]

@marmeladapk isn't it related to the issues we observed on some Fastinos? W thought that it was PLL issue..

[marmeladapk]

What we observed was connected to lack of proper initialization of some structures inside the FPGA. AFAIK since fixes in ARTIQ we haven't observed these issues.

[jordens]

IIRC I've seen some Fastinos with very early "engineering sample" silicon from mid 2012 that may have behaved differently.

[gkasprow]

That's why I asked about datecode

[McSherry]

The datecodes are A1461R81 and A5271R15. Apologies, forgot to copy them over from our chat.

[gkasprow]

[gkasprow]

Now 3V3 starts first, VCC/VCCPLL and VPP_2V5 wake up later.

[gkasprow]

The plan is to add:

• An RCD circuit that will delay 2V5 supply buck
• A low RDSON PMOS that will turn 3.3V FPGA and logic supply as the last rail

[gkasprow]

Here is the fixed power supply; update: 100nF->1uF; 10k -> 33k