IPU6 driver not working on Jasper Lake tablets

[jwrdegoede]

I'm trying to get the IPU6 driver to work on a Jasper Lake Juno tablet:
https://junocomputers.com/us/product/juno-tablet/

Unfortunately the jsl_penguin_peak branches of the various IPU6 driver stack components have not been updated for quite a while.

After cherry-picking various fixes to build against newer kernels from ipu6-drivers/master into the jsl_penguin_peak, I have ended up with an ipu6-drivers jsl_penguin_peak branch which shows no differences from the last merged master commit: https://github.com/jwrdegoede/ipu6-drivers/commits/jsl_penguin_peak

So I believe that Jasper Lake should work with the latest ipu6-drivers/master branch and I have been trying to get the GC5035 sensor on the Juno tablet to work using the latest ipu6-drivers/master + 6.4.0 .

For ipu6-camera-bins and ipu6-camera-hal I did use the jsl_penguin_peak branches, which are out of sync with the main branches, please update these if possible.

At first I used the config/linux/ipu6ep/gcss/graph_settings_gc5035.xml file from intel/ipu6-camera-hal#49 but that causes a segfault inside ipu6-camera-hal, so then I used config/ubuntu/jsl/gcss/graph_settings_OV13858_YHCE_JSLP.xml as a base instead and adjusted that for the gc5035
graph_settings_gc5035.xml.txt.

With this using gst-launch icamerasrc does try to start streaming, but then things fail with the following message in dmesg when using dyndbg:

[   49.513130] intel-ipu6-isys intel-ipu6-isys0: ---------------------------
[   49.513133] intel-ipu6-isys intel-ipu6-isys0: IPU_FW_ISYS_STREAM_CFG_DATA
[   49.513135] intel-ipu6-isys intel-ipu6-isys0: ---------------------------
[   49.513137] intel-ipu6-isys intel-ipu6-isys0: Source 0
[   49.513139] intel-ipu6-isys intel-ipu6-isys0: VC 0
[   49.513142] intel-ipu6-isys intel-ipu6-isys0: Nof input pins 1
[   49.513144] intel-ipu6-isys intel-ipu6-isys0: Nof output pins 1
[   49.513146] intel-ipu6-isys intel-ipu6-isys0: Input pin 0
[   49.513149] intel-ipu6-isys intel-ipu6-isys0: Mipi data type 0x2b
[   49.513151] intel-ipu6-isys intel-ipu6-isys0: Mipi store mode 0
[   49.513153] intel-ipu6-isys intel-ipu6-isys0: Bits per pixel 0
[   49.513155] intel-ipu6-isys intel-ipu6-isys0: Mapped data type 0x40
[   49.513158] intel-ipu6-isys intel-ipu6-isys0: Input res width 2592
[   49.513160] intel-ipu6-isys intel-ipu6-isys0: Input res height 1944
[   49.513163] intel-ipu6-isys intel-ipu6-isys0: mipi decompression 0
[   49.513165] intel-ipu6-isys intel-ipu6-isys0: capture_mode 0
[   49.513167] intel-ipu6-isys intel-ipu6-isys0: Crop info
[   49.513169] intel-ipu6-isys intel-ipu6-isys0: Crop.top_offset 0
[   49.513171] intel-ipu6-isys intel-ipu6-isys0: Crop.left_offset 0
[   49.513173] intel-ipu6-isys intel-ipu6-isys0: Crop.bottom_offset 1944
[   49.513175] intel-ipu6-isys intel-ipu6-isys0: Crop.right_offset 2592
[   49.513177] intel-ipu6-isys intel-ipu6-isys0: ----------------
[   49.513180] intel-ipu6-isys intel-ipu6-isys0: Output pin 0
[   49.513182] intel-ipu6-isys intel-ipu6-isys0: Output input pin id 0
[   49.513184] intel-ipu6-isys intel-ipu6-isys0: Output res width 2592
[   49.513186] intel-ipu6-isys intel-ipu6-isys0: Output res height 1944
[   49.513188] intel-ipu6-isys intel-ipu6-isys0: Stride 5632
[   49.513190] intel-ipu6-isys intel-ipu6-isys0: Pin type 3
[   49.513193] intel-ipu6-isys intel-ipu6-isys0: Payload 10960896
[   49.513195] intel-ipu6-isys intel-ipu6-isys0: Ft 24
[   49.513197] intel-ipu6-isys intel-ipu6-isys0: Watermar in lines 0
[   49.513199] intel-ipu6-isys intel-ipu6-isys0: Send irq 1
[   49.513201] intel-ipu6-isys intel-ipu6-isys0: Reserve compression 1
[   49.513204] intel-ipu6-isys intel-ipu6-isys0: snoopable 0
[   49.513206] intel-ipu6-isys intel-ipu6-isys0: error_handling_enable 0
[   49.513208] intel-ipu6-isys intel-ipu6-isys0: sensor type 1
[   49.513210] intel-ipu6-isys intel-ipu6-isys0: ----------------
[   49.513212] intel-ipu6-isys intel-ipu6-isys0: Isl_use 0
[   49.513214] intel-ipu6-isys intel-ipu6-isys0: stream sensor_type 0
[   49.513285] intel-ipu6-isys intel-ipu6-isys0: send_token: STREAM_OPEN
[   49.513727] intel-ipu6-isys intel-ipu6-isys0: hostlib: error resp 00 STREAM_OPEN_DONE, stream 0, error 7, details 8, timestamp 0x0000000000000000, pin 0
[   49.514258] intel-ipu6-isys intel-ipu6-isys0: stream open error: 7

I'm happy to try any patches or hints you may have to fix this.

Cc: @junocomp @liang1wang

[hao-yao]

Thanks @jwrdegoede .

IPU6 on JSL (ipu6se) is a very different version that many processing functions were striped, so it needs different graph descriptor and graph configs. I think GC5035 should use a simillar graph configs to OV13858.

Branch jsl_penguin_peak was for a cancelled project and I remembered it only supports kernel v5.10 and we have not maintained it for a long time.

[jwrdegoede]

I think GC5035 should use a simillar graph configs to OV13858.

Ok, that is what I have done after just copying over the GC5035 graph config for ipu6ep failed.

Branch jsl_penguin_peak was for a cancelled project and I remembered it only supports kernel v5.10 and we have not maintained it for a long time.

Right, but the regular branch of ipu6-camera-bins does not have jsl binaries. I see that there is a chrome_jsl_branch now, I guess I should use that?

[jwrdegoede]

Right, but the regular branch of ipu6-camera-bins does not have jsl binaries. I see that there is a chrome_jsl_branch now, I guess I should use that?

But there is no matching ipu6-camera-hal branch ? Should I just use the main ipu6-camera-hal branch with those binaries ?

[hao-yao]

But there is no matching ipu6-camera-hal branch ? Should I just use the main ipu6-camera-hal branch with those binaries ?

Tag Penguin_peak_20220805 in all 4 repos should make JSL devices (with ov13858 sensor) work.

We only have graph settings and tuning file for IPU6EP. Usually we need to generate a set of graph settings and tuning file for IPU6SE + gc5035 by a tuning tool. But I can't access to that tool, so we have to change graph_settings xml manually.

Seems IPU6SE can only use csi_be and isa_video for video pipe, maybe we can copy and change ov13858 graph settings and change the resolution for gc5035. But I think it is too tricky.

[jwrdegoede]

Tag Penguin_peak_20220805 in all 4 repos should make JSL devices (with ov13858 sensor) work.

Right, so that tag matches the latest commits in the jsl_penguin_peak branches, so that is what I have already been using.

[hao-yao]

stream start should be called after qbuf but the log shows that isys firmware didn't got any buffer when received stream start. Maybe we can check the output of gstreamer/libcamhal? We can enable log by setting cameraDebug env to 65535 when starting gstreamer pipeline.

[jwrdegoede]

Hmm, I wonder if ipu6-drivers maybe used to delay the start-stream until buffers were queued in the kernel and if the older hal / ipi6-camera-bins from Penguin_peak_20220805 rely on this?

As mentioned in the original description I'm using a newer kernel:

After cherry-picking various fixes to build against newer kernels from ipu6-drivers/master into the jsl_penguin_peak, I have ended up with an ipu6-drivers jsl_penguin_peak branch which shows no differences from the last merged master commit: https://github.com/jwrdegoede/ipu6-drivers/commits/jsl_penguin_peak

So I believe that Jasper Lake should work with the latest ipu6-drivers/master branch and I have been trying to get the GC5035 sensor on the Juno tablet to work using the latest ipu6-drivers/master + 6.4.0 .

So I just tried the following:

export cameraDebug=65535
gst-launch-1.0 icamerasrc buffer-count=7 ! video/x-raw,format=NV12,width=1280,height=720 ! videoconvert ! xvimagesink

Here are the syslog/journal log messages this has generated:
log.txt

From a quick peek CamHAL does seem to think that the buffers are queued before the streamon error:

Aug 14 11:51:10 fedora cameraHal[2412]: [08-14 11:51:10.513] CamHAL[LV2] processPendingBuffers: buffers in device:3

[hao-yao]

Hmm, I wonder if ipu6-drivers maybe used to delay the start-stream until buffers were queued in the kernel and if the older hal / ipi6-camera-bins from Penguin_peak_20220805 rely on this?

As far as I know, when stream starts, IPU kernel driver will set config data -> send STREAM_OPEN -> queue buffer and send STREAM_START_AND_CAPTURE to FW. But seems it failed at sending STREAM_OPEN. Maybe the config data is wrong for ipu6se? I didn't try JSL devices for a long time so maybe there is a regression between jsl_penguin_peak branch and master. I can later attach a script to capture a raw image from IPU isys and we can check if there is something wrong without libcamhal.

[jwrdegoede]

I can later attach a script to capture a raw image from IPU isys and we can check if there is something wrong without libcamhal.

If you can attach such a script that would be great.

[hao-yao]

By the way, @jwrdegoede could you attach a full dmesg output? and please also check your /lib/firmware/intel/ipu6se_fw.bin checksum?

[jwrdegoede]

By the way, @jwrdegoede could you attach a full dmesg output?

The journal log from 2 comments up also contains all dmesg output.

? and please also check your /lib/firmware/intel/ipu6se_fw.bin checksum?

[root@fedora ~]# sha256sum /lib/firmware/intel/ipu6se_fw.bin 
8e6cbab206806e46b787b30a2b183bfff56e730a0ca134579910bb24db51e882  /lib/firmware/intel/ipu6se_fw.bin

(this fw comes from the jsl_penguin_peak branch of ipu6-camera-bins)

[hao-yao]

The journal log from 2 comments up also contains all dmesg output.

I didn't find some expected outputs in that log so I was asking for a full one. So that means some log was not actually printed.

8e6cbab206806e46b787b30a2b183bfff56e730a0ca134579910bb24db51e882  /lib/firmware/intel/ipu6se_fw.bin

This is the latest ipu6se firmware.

According to the log, the ipu6se firmware was not receiving expected configuration data. I will check what was wrong later.

[junocomp]

Any updates on this?

[hao-yao]

Sorry.
@jwrdegoede Could you check if this repo helps? https://github.com/hao-yao/ipu6-sensor-guide/tree/main/reference/yavta-tools
I uploaded a yavta tool and a script for another imx390 sensor there.

[junocomp]

Any updates on this?