Framos imx662 low light/Infrared

Technical Support
, , ,
1

Hi again,

I was wondering if anyone had any joy with getting the imx662 sensor working well in low light conditions/infrared condition.

We’ve tried playing with some of the settings using the vvext binary ( like setting saturation to 0 ) but we’re still not getting the quality we’d of expected there’s lots more noise than we’d expect.

We’ve experimented with another camera sensor before this and it seemed to “just work” at night, this feels like we’re missing some sort of configuration, but at this point in time are not sure what.

We’ve raised a support ticket with Framos already but the support is a little slow at the moment, so thought we’d ask here too in case anyone else has had experienced similar challenges and can uhh enlighten us :wink:

Thanks in advance

Alan

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer? :+1: :-1:

IMX662 Low Light Performance Optimization

Hi Alan,

The IMX662 sensor has specific characteristics that can be optimized for low light and infrared conditions. Let me provide some guidance based on the available documentation.

IMX662 Sensor Specifications

The FSM:GO IMX662 sensor has the following key characteristics (1):

  • Sensor: IMX662 from Sony®
  • Resolution: 2MP CMOS Image Sensor with 1/2.8" optical format
  • Pixel size: 2.9μm x 2.9μm
  • Shutter type: CMOS Rolling Shutter
  • Focus Type: Fixed focus

Low Light Performance Considerations

Based on image sensor selection principles for embedded applications, several factors affect low light performance (2):

High Signal-to-Noise Ratio (SNR)

For low light conditions, cameras with large pixel sizes and better SNR are recommended (2). The IMX662’s 2.9μm x 2.9μm pixel size should provide reasonable low light performance, but optimization may be needed.

NIR (Near Infrared) Sensitivity

Applications requiring 24/7 operation in challenging lighting conditions benefit from NIR lighting assistance (2). The sensor should be designed to be highly sensitive to the NIR spectrum for night vision capabilities.

Camera Usage and Testing

You can test and optimize the camera using GStreamer pipelines (3). To view video output and experiment with settings:

# gst-launch-1.0 -v v4l2src device=/dev/video2 ! video/x-raw ! videoconvert ! waylandsink

(3)

Hardware Setup Verification

Ensure your hardware setup is correct (4)(1):

  • Verify the flat ribbon cable connections (these are fragile and can cause issues)
  • Confirm proper connection to the MIPI-CSI Camera Interface
  • Check that the device tree overlays are properly applied

Additional Recommendations

  1. Exposure Settings: Experiment with longer exposure times for low light conditions
  2. Gain Control: Adjust sensor gain settings through V4L2 controls
  3. Lighting Assistance: Consider adding NIR illumination for consistent night performance
  4. Pixel Format: Test different pixel formats that might be better suited for your low light application

The documentation shows that the IMX662 is available in the Peripheral Database (5) as a tested device, indicating it should work well with Toradex hardware when properly configured.

Since you’ve already contacted Framos support, I’d recommend also experimenting with the V4L2 controls available for exposure, gain, and other sensor parameters while waiting for their response.

Best regards, Toradex AI Assistant.

3

Hello @alan01252,

From my limited understanding, low-light conditions are something that may require some custom ISP tuning.
Framos would for sure know better as they are the specialists on this topic.

One thing that I can think about is maybe reducing the framerate that you capture the video to allow for a longer exposure time. Maybe capturing at 25-30 fps could lead to better low-light video.

Best Regards,
Bruno

4

Thanks @bruno.tx

We’ve discovered that you need to enable high conversion gain registry value, this isn’t supported in the Framos drivers available from github ( at time of writing ) and therefore not available in the toradex yocto wrapper either.

We’ve extended the driver to enable to toggle this on which does appear to of improved things, we’re still in the process of experimenting more at the moment.

We’ve also raised with Framos support for clearHDR mode, which appears to be available in the latest commit ( not the one used with available in the toradex wrapper ) but that doesn’t seem to “just work” either :slight_smile:

Who knew cameras could be so much fun :wink:

5

Hello @alan01252,

Sorry for the delay here.

How has this problem been evolving with Framos?
Do you want me to check with them on this?

Best Regards,
Bruno

6

Hi @bruno.tx

It’s been evolving :slight_smile:

Things we’ve found that might help others:

  • Framos opensource driver doesn’t support the register for LCG/HCG we’ve added that in that does improve things ( switching the register to HCG )
  • Framos opensource driver supports ClearHDR, but we couldn’t get it to work, it dropped down to 1fps and provided a rolling image. Framos haven’t come back to us on this thing specifically yet ( and to be fair I haven’t chased )
  • Framos opensource driver performs re-tries on failed i2c writes, but doesn’t appear to do on i2c reads, we noticed that could cause the ISP to crash and hang, we’ve added a patch to fix that in our yocto code
  • We had to set the FPS to 10 else the exposure settings didn’t get passed through from the ISP→ driver and we got poor results. We’re currently unsure whether this is something to do with the ISP tuning? There’s some sort of NXP provided tool to generate XML that the isp then uses. We don’t really have enough experience to know how much the existing XML has been “tuned” to the 662 sensor.

We also noticed a few hardware things that were limiting us, the lens we were using had a higher F number and therefore wasn’t giving us the performance we were hoping for in low light conditions.

We’re getting there :slight_smile:

Thanks for checking up on this :slight_smile: it’s appreciated!

7

Hello @alan01252,

Thanks for the update.

It could be. To me this looks like some sort of range that was defined that just triggered when you got down to 10 fps.
However, this is speculation.
The xml should be a tuning for no specific condition, just a starting point that works on normal development conditions.
It is likely that Framos could have or create a tuning for your low-light conditions.

However, maybe looking at some of the hardware configurations, such as the lens choice, first could be better.
Proper ISP tuning can be quite hard to do and therefore can be expensive.

Best Regards,
Bruno

1 Like