Verdin IMX8M Plus, RAW Camera Sensor Support

mzvis · June 17, 2021, 3:02pm

Hello,

I was wondering about the steps that would be necessary to integrate a monochrome RAW image sensor with the Verdin IMX8M Plus?

In theory, as the image is mono (8bit or 10bit), there would not be any processing (de-beyering) necessary.
But recently I read that the ISI of the IMX8 family does not support RAW8 formats, even if they bypass the format conversion (CSC). See here:

github.com

alliedvision/documentation/blob/master/Toradex_Apalis.md

# Toradex Apalis iMX8 restrictions for CSI-2 cameras

## Unsupported pixel formats
Apalis iMX8 doesn't support all pixel formats of CSI-2 cameras.
RAW formats (bayer and mono) are right-shifted by 2 bits, 
so that every pixel loses 2 bits and cannot be used. This is a restriction of the board.

## Workaround
As a workaround, the Allied Vision driver sets the camera to YUV when the pixel format 
V4L2_PIX_FMT_GREY is selected by the user.
The driver then uses the ISI of Apalis iMX8 to convert the image to Mono8.
This is done by using a multiplane YUV format internally and providing only the first plane 
(the luma component) to the user space. The color components are discarded.

Because the camera is streaming with YUV, only half of the bandwidth of RAW8 can be achieved.   
For the Bayer pixel formats, no workaround is available.

For more information on the issue, see:   
https://community.nxp.com/thread/519316   
https://community.nxp.com/docs/DOC-345149

This file has been truncated. show original

(the link is on the Toradex Apalis, but afaik the IMX8M Plus has the same ISI)

In theory one could use the ISP for this purpose, but there I cannot find any reference code on how to setup and use the ISP on the Verdin IMX8 Plus.
Also it seems like and overkill, as for monochrome there is anyways no processing required.

What would you do for interfacing RAW8/RAW10 monochrome camera with this board?

Thank you for your advice

kevin.tx · June 21, 2021, 1:48pm

Dear @mzvis,

Thank you for contacting Toradex by using our community.

We are currently looking into your question.

As you correctly stated, there is no need of using the ISP on a mono image. We are looking at the situation with the RAW8 format. Unfortunately the openly available information about the ISP is very limited.

Our team is currently working on some reference code samples to provide for setting up the ISP. This is still work in progress.

I will reach out to you again as soon as possible.

Best Regards
Kevin

jaski.tx · August 9, 2021, 8:04pm

Hi @mzvis

Thanks for your patience. Just to provide you an update. We are still working on this issue.
Did anything change from your side?
Which sensor do you intend to use?
What is your application?

Best regards,
Jaski

tgsell · January 12, 2022, 1:22pm

Hi @jaski.tx

I would be very much interested in a solution as well. I am using an OV9281 camera from OmniVision.

Is there any update on this topic?

Best regards
TG

kevin.tx · January 17, 2022, 7:11pm

Hi @tgsell,

unfortunately, there is no update. Thank you for your patience.

Is your setup/requirements the same or similar?

Best Regards
Kevin

tgsell · January 18, 2022, 1:46pm

Hi @kevin.tx

It is exactly the same requirement. We would like to directly use the ISP of the iMX8M Plus to process the RAW8 OV9281 output, as the ISI is only supporting 16bit.

You were mentioning that you are working on some reference code half a year ago. What is the status there? Would be great to get some support here.

Thanks and best regards,
TG

kevin.tx · February 2, 2022, 10:19am

Hi @tgsell ,

thank you for confirming the requirements.

The team is aware and we’re giving our best to provide a solution as soon as possible. We are waiting for certain components to be finished by our partners on our end. So as these become available we will be able to update you guys here.

Best Regards
Kevin