Connect parallel camera interface with the iMX6ULL WB IT and Colibri Evaluation Board

Hi all,
we are trying to get a parallel camera (MT9P006) running with the Colibri iMX6ULL WB on the Colibri Evaluation Board Rev. 3.2 (currently running the “Toradex Embedded Linux Demo with LXDE” image). Any tips / info on getting the interface to run on this combination? It appears to me that the pin-muxing would need to be changed (since some of the interfaces standard pins are used by the wireless module and the modules parallel interfce differs from the colibro standard interface pins)
I’m still starting out on embedded linux, so please forgive me if this is obvious.

Thanks in advance!

Hello @m.kl and Welcome to the Toradex Community!

For your parallel camera interface, you can look into our Analogue Camera Adapter. You would need similar pin-muxing as here. For the Colibri iMX6ULL module with Wifi feature, you need to use the alternative pins for Camera Input which are described in the section 5.21 Camera Interface of the Datasheet of the SoM.

Additionally you will need a driver for you parallel camera which can be found here.

Best regards,
Jaski

Thanks for the info, it helped me get going :slight_smile: Will hopefully be finally able to test the whole thing today!

The driver for the MT9P006 is also included in the mainline kernel as driver for MT9P031, which is the version without bayer filter, btw.

You are welcome. Thanks for the feedback.

Let us know about your results.

Best regards,
Jaski

Turns out the whole thing doesn’t work so far. My hardware seems to be fine (atleast it has not exploded so far, didn’t have the chance to properly check function with a oscilloscope yet), but there are several problems. I was hoping you could maybe help me with those aswell. (I’m sorry if i should have started a new thread for those)

I followed this guide: Patching Kernel in OpenEmbedded to add the mentioned camera driver to the toradex_4.9-2.3.x-imx-next kernel and modify the imx6ull-colibri-eval-v3.dtsi file of the devicetree to include the camera and its pin muxing, and turned off some PWMs and UARTs to avoid sharing pins (see below) (the pin muxing was based on toradex documentation, the cameras entry is based on an entry from a phytec (camera module manufacturer) devicetree, the csi part was done from examples). Then I took this image https://docs.toradex.com/106394-colibri-imx6ull-linux-lxde-image-tezi.tar and replaced the respective files for the eval board with the new ones and flashed it with toradex easy installer.
The driver seems to work, it is blocking 0x48 on I2C0 (or I2C1 in devicetree), which is where the camera should be. The devicetree seems to be the cause of problems though: dmseg | grep CLK finds an error message, that “imx6ul-pinctrl 20e0000.iomux: pin MX6UL_PAD_CSI_PIXCLK already requested by 20e0000.iomux; cannot claim for 0-0048” and “imx6ul-pinctrl 20e0000.iomux: could not request pin 118 (MX6UL_PAD_CSI_PIXCLK) from group csi1grp on device 20e0000.iomux”
Any idea, what could be wrong / causing this?

Also the camera driver is returning “mt9p031 0-0048: Error applying setting, reversing things back” and “mt9p031: probe of 0-0048 failed with error -22” (seems to mean invalid value)

changed imx6ull-colibri-eval-v3.dtsi:

/*
 * Copyright 2017 Toradex AG
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 */

#include <dt-bindings/input/input.h>

/ {
	chosen {
		bootargs = "console=ttymxc0,115200";
	};

	clocks {
		/* fixed crystal dedicated to mcp251x */
		clk16m: clk@1 {
			compatible = "fixed-clock";
			reg=<1>;
			#clock-cells = <0>;
			clock-frequency = <16000000>;
			clock-output-names = "clk16m";
		};
	};

	gpio-keys {
		compatible = "gpio-keys";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_snvs_gpiokeys>;

		power {
			label = "Wake-Up";
			gpios = <&gpio5 1 GPIO_ACTIVE_HIGH>;
			linux,code = <KEY_WAKEUP>;
			debounce-interval = <10>;
			gpio-key,wakeup;
		};
	};

	extcon_usbc_det: usbc_det {
		compatible = "linux,extcon-usb-gpio";
		debounce = <25>;
		id-gpio = <&gpio5 2 GPIO_ACTIVE_HIGH>;
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_snvs_usbc_det>;
	};

	reg_3v3: regulator-3v3 {
		compatible = "regulator-fixed";
		regulator-name = "3.3V";
		regulator-min-microvolt = <3300000>;
		regulator-max-microvolt = <3300000>;
		regulator-always-on;
	};

	reg_5v0: regulator-5v0 {
		compatible = "regulator-fixed";
		regulator-name = "5V";
		regulator-min-microvolt = <5000000>;
		regulator-max-microvolt = <5000000>;
	};

	reg_usbh_vbus: regulator-usbh-vbus {
		compatible = "regulator-fixed";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_usbh1_reg>;
		regulator-name = "VCC_USB[1-4]";
		regulator-min-microvolt = <5000000>;
		regulator-max-microvolt = <5000000>;
		gpio = <&gpio1 2 GPIO_ACTIVE_LOW>;
		vin-supply = <&reg_5v0>;
	};
};

&bl {
	brightness-levels = <0 4 8 16 32 64 128 255>;
	default-brightness-level = <6>;
	pwms = <&pwm4 0 5000000 1>;
	status = "okay";
};

&adc1 {
	status = "okay";
};

&pxp {
	status = "okay";
};

&ecspi1 {
	status = "okay";

	mcp251x0: mcp251x@0 {
		compatible = "microchip,mcp2515";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_can_int>;
		reg = <0>;
		clocks = <&clk16m>;
		interrupt-parent = <&gpio2>;
		interrupts = <4 IRQ_TYPE_EDGE_FALLING>;
		spi-max-frequency = <10000000>;
		status = "okay";
	};

	spidev0: spidev@0 {
		compatible = "toradex,evalspi";
		reg = <0>;
		spi-max-frequency = <23000000>;
		status = "disabled";
	};
};

#if 0
&flexcan1 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_flexcan1>;
	status = "okay";
};

&flexcan2 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_flexcan2>;
	status = "okay";
};
#endif

&i2c1 {
	pinctrl-names = "default", "gpio";
	pinctrl-0 = <&pinctrl_i2c1>;
	pinctrl-1 = <&pinctrl_i2c1_gpio>;
	sda-gpios = <&gpio1 29 GPIO_ACTIVE_LOW>;
	scl-gpios = <&gpio1 28 GPIO_ACTIVE_LOW>;
	status = "okay";

/* the PCAPs use SODIMM 28/30, also used for PWM<B>, PWM<C>, aka pwm5, pwm6.
   so if you enable one of the PCAP controllers disable the pwms */
	/* Atmel maxtouch controller */
	atmel_mxt_ts: atmel_mxt_ts@4a {
		compatible = "atmel,maxtouch";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_gpiotouch>;
		reg = <0x4a>;
		interrupt-parent = <&gpio4>;
		interrupts = <16 IRQ_TYPE_EDGE_FALLING>; /* SODIMM 28 */
		reset-gpios = <&gpio2 5 GPIO_ACTIVE_HIGH>; /* SODIMM 30 */
		status = "disabled";
	};

	touch: touchrevf0710a@10 {
		compatible = "touchrevolution,fusion-f0710a";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_gpiotouch>;
		reg = <0x10>;
			/* SODIMM 28, Pen down interrupt */
		gpios = <&gpio4 16 GPIO_ACTIVE_HIGH
			/* SODIMM 30, Reset interrupt */
			 &gpio2 5 GPIO_ACTIVE_LOW
			>;
		status = "disabled";
	};

	/* M41T0M6 real time clock on carrier board */
	rtc: m41t0m6@68 {
		compatible = "st,m41t0";
		reg = <0x68>;
	};

	/* MT9P006 (MT9P031 with color filter) camera module on adapter board */
	mt9p031@48 {		
		/*aptina, mt9p006?*/
		compatible = "aptina,mt9p031";

		/*same as I2C address, got error when changed*/
		reg = <0x48>;

		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_csi1>;
		reset-gpios = <&gpio4 28 GPIO_ACTIVE_LOW>;

		status = "okay";

		clocks = <&clks IMX6UL_CLK_CSI>;
              	clock-names = "csi_mclk"; 

                assigned-clocks =
                      	<&clks IMX6UL_CLK_CSI_SEL>,
                      	<&clks IMX6UL_CLK_CSI>; 
		
		assigned-clock-parents =
                      	<&clks IMX6UL_CLK_PLL2_PFD2>;

              	assigned-clock-rates =
                      	<0>, <50000000>; 


		port {
			mt9p031_1: endpoint {
                              input-clock-frequency = <50000000>;
                              pixel-clock-frequency = <50000000>;
                              pclk-sample = <0>;
                              remote-endpoint = <&csi1_ep>;
                              bus-width = <8>;
                              hsync-active = <1>;
                              vsync-active = <1>;
			};
		};
	};
};

&lcdif {
	display = <&display0>;
	status = "okay";

	display0: lcd-display {
		bits-per-pixel = <16>;
		bus-width = <18>;

		display-timings {
			native-mode = <&timing_vga>;

			/* Standard VGA timing */
			timing_vga: 640x480 {
				clock-frequency = <25175000>;
				hactive = <640>;
				vactive = <480>;
				hback-porch = <40>;
				hfront-porch = <24>;
				vback-porch = <32>;
				vfront-porch = <11>;
				hsync-len = <96>;
				vsync-len = <2>;

				de-active = <1>;
				hsync-active = <0>;
				vsync-active = <0>;
				pixelclk-active = <0>;
			};

			/* WVGA Timing, e.g. EDT ET070080DH6 */
			timing_wvga: 800x480 {
				clock-frequency = <33260000>;
				hactive = <800>;
				vactive = <480>;
				hback-porch = <216>;
				hfront-porch = <40>;
				vback-porch = <35>;
				vfront-porch = <10>;
				hsync-len = <128>;
				vsync-len = <2>;

				de-active = <1>;
				hsync-active = <0>;
				vsync-active = <0>;
				pixelclk-active = <0>;
			};
			/* WVGA Timing, TouchRevolution Fusion 7" */
			timing_wvga2: 800x480pixclkact {
				clock-frequency = <33260000>;
				hactive = <800>;
				vactive = <480>;
				hback-porch = <216>;
				hfront-porch = <40>;
				vback-porch = <35>;
				vfront-porch = <10>;
				hsync-len = <128>;
				vsync-len = <2>;

				de-active = <1>;
				hsync-active = <0>;
				vsync-active = <0>;
				pixelclk-active = <1>;
			};
			/* Standard SVGA timing */
			timing_svga: 800x600 {
				clock-frequency = <40000000>;
				hactive = <800>;
				vactive = <600>;
				hback-porch = <88>;
				hfront-porch = <40>;
				vback-porch = <23>;
				vfront-porch = <1>;
				hsync-len = <128>;
				vsync-len = <4>;

				de-active = <1>;
				hsync-active = <1>;
				vsync-active = <1>;
				pixelclk-active = <0>;
			};
			/* TouchRevolution Fusion 10"/CLAA101NC05 10.1 inch */
			timing_wsvga: 1024x600 {
				clock-frequency = <48000000>;
				hactive = <1024>;
				vactive = <600>;
				hback-porch = <104>;
				hfront-porch = <43>;
				vback-porch = <24>;
				vfront-porch = <20>;
				hsync-len = <5>;
				vsync-len = <5>;

				de-active = <1>;
				hsync-active = <0>;
				vsync-active = <0>;
				pixelclk-active = <0>;
			};
			/* Standard XGA timing */
			timing_xga: 1024x768 {
				clock-frequency = <65000000>;
				hactive = <1024>;
				vactive = <768>;
				hback-porch = <160>;
				hfront-porch = <24>;
				vback-porch = <29>;
				vfront-porch = <3>;
				hsync-len = <136>;
				vsync-len = <6>;

				de-active = <1>;
				hsync-active = <0>;
				vsync-active = <0>;
				pixelclk-active = <0>;
			};
		};
	};
};

/* the PCAPs use SODIMM 28/30, also used for PWM<B>, PWM<C>, aka pwm5, pwm6.
   so if you enable one of the PCAP controllers disable the pwms */
/* PWM <A> */
&pwm4 {
	status = "disabled";
};

/* PWM <B> */
&pwm5 {
	status = "disabled";
};

/* PWM <C> */
&pwm6 {
	status = "disabled";
};

/* PWM <D> */
&pwm7 {
	status = "disabled";
};

&uart1 {
	status = "disabled";
};

&uart2 {
	status = "disabled";
};

&uart5 {
	status = "disabled";
};

&usbotg1 {
	extcon = <&extcon_usbc_det &extcon_usbc_det>;
	vbus-supply = <&reg_usbh_vbus>;
	status = "okay";
};

&usbotg2 {
	vbus-supply = <&reg_usbh_vbus>;
	status = "okay";
};

/* The define SD_1_8 allows to use the SD interface at a higher speed mode
 * if the card supports it. For this the signaling voltage is switched from
 * 3.3V to 1.8V under the usdhc1's drivers control.
 */
/* #define SD_1_8 */
&usdhc1 {
#ifdef SD_1_8
	pinctrl-names = "default", "state_100mhz", "state_200mhz", "sleep";
	pinctrl-0 = <&pinctrl_usdhc1 &pinctrl_snvs_cd_usdhc1>;
	pinctrl-1 = <&pinctrl_usdhc1_100mhz &pinctrl_snvs_cd_usdhc1>;
	pinctrl-2 = <&pinctrl_usdhc1_200mhz &pinctrl_snvs_cd_usdhc1>;
	pinctrl-3 = <&pinctrl_usdhc1 &pinctrl_snvs_cd_usdhc1_sleep>;
	vqmmc-supply = <&reg_sd1_vmmc>;
#else
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_usdhc1 &pinctrl_snvs_cd_usdhc1>;
	no-1-8-v;
#endif
	cd-gpios = <&gpio5 0 GPIO_ACTIVE_LOW>;
	disable-wp;
	enable-sdio-wakeup;
	keep-power-in-suspend;
	status = "okay";
	vmmc-supply = <&reg_3v3>;
};

&iomuxc {
	imx6ull-eval-v3 {
		pinctrl_gpiotouch: touchgpios {
			fsl,pins = <
				MX6UL_PAD_NAND_DQS__GPIO4_IO16		0x74
				MX6UL_PAD_ENET1_TX_EN__GPIO2_IO05	0x14
			>;
		};

		/*Reset-GPIO for camera*/
		/*MX6UL_PAD_CSI_DATA07__GPIO4_IO28 defined in gpio4-grp in mainline imx6ull-colibri.dtsi*/
		pinctrl_gpio4: gpio4-grp {
			fsl,pins = <
				MX6UL_PAD_CSI_DATA07__GPIO4_IO28	0x10b0 /* SODIMM 65 */
			>;
		};
				
		pinctrl_csi1: csi1grp {
			fsl,pins = <

                		MX6UL_PAD_CSI_PIXCLK__CSI_PIXCLK           0x000010B0	/* SODIMM  96 */
				MX6UL_PAD_CSI_MCLK__CSI_MCLK               0x000010B0	/* SODIMM  75 */
                		MX6UL_PAD_GPIO1_IO08__CSI_VSYNC            0x000010B0	/* SODIMM   4 */
                		MX6UL_PAD_GPIO1_IO09__CSI_HSYNC            0x000010B0	/* SODIMM   2 */
               			MX6UL_PAD_LCD_DATA18__CSI_DATA10           0x000010B0	/* SODIMM  29 */
                		MX6UL_PAD_LCD_DATA19__CSI_DATA11           0x000010B0	/* SODIMM  37 */
                		MX6UL_PAD_UART1_CTS_B__CSI_DATA04          0x000010B0	/* SODIMM  25 */
                		MX6UL_PAD_UART1_RTS_B__CSI_DATA05          0x000010B0	/* SODIMM  27 */
                		MX6UL_PAD_UART1_RX_DATA__CSI_DATA03        0x000010B0	/* SODIMM  35 */
                		MX6UL_PAD_UART1_TX_DATA__CSI_DATA02        0x000010B0	/* SODIMM  33 */
                		MX6UL_PAD_UART2_CTS_B__CSI_DATA08          0x000010B0	/* SODIMM  32 */
                		MX6UL_PAD_UART2_RTS_B__CSI_DATA09          0x000010B0	/* SODIMM  34 */
                		MX6UL_PAD_UART2_RX_DATA__CSI_DATA07        0x000010B0	/* SODIMM  38 */
                		MX6UL_PAD_UART2_TX_DATA__CSI_DATA06        0x000010B0	/* SODIMM  36 */
                		MX6UL_PAD_UART3_RX_DATA__CSI_DATA00        0x000010B0	/* SODIMM  77 */
                		MX6UL_PAD_UART3_TX_DATA__CSI_DATA01        0x000010B0	/* SODIMM 135 */
			>;
		};
	};
};

&csi {
	status = "okay";
	port {
		csi1_ep: endpoint {
			remote-endpoint = <&mt9p031_1>;
		};
	};
};

HI

MX6UL_PAD_CSI_PIXCLK already requested

You have a Pin conflict, since this PIN is already used for some other function. Actually It is declared as GPIO in imx6ull-colibri.dtsi.

You need to resolve this Pin Conflict by removing the PIN from the node pinctrl_hog_2 or disable the Node, where this Pin is used.

Best regards,
Jaski

Hi

Thank you for your explanation, I have started trying that… now the pin conflict part seems resolved, but the system isn’t able to boot every time (sometimes it freezes while booting or gets a kernel panik) which didn’t happen before commenting out those lines ( in hoggrp1:

MX6UL_PAD_UART3_RX_DATA__GPIO1_IO25 ;
MX6UL_PAD_UART3_TX_DATA__GPIO1_IO24

in hoggrp2:
MX6UL_PAD_CSI_MCLK__GPIO4_IO17 ;
MX6UL_PAD_CSI_PIXCLK__GPIO4_IO18)
Any idea what might cause that / how to avoid it?

Also I wanted to ask what the preferred way to connect a (for now) 8bit camera interface (raw bayer) is… i believe i read bits 2-9 somewhere, but i can’t find it anymore and 0-7 would ofc be more obvious

Could you share the kernel panic log in a text file?

Sorry but I’m still new to embedded linux… what would be the best way to produce that kind of log?

Do you have access to the console output? If yes, then just copy the text shown on the console output just before and during crash and share it in a text file.

Could you also share the complete dmesg.log in a text file?

Thanks.

Hi, I’m sorry for moving the goal posts, but following a different example I replaced

         assigned-clocks =
                           <&clks IMX6UL_CLK_CSI_SEL>,
                           <&clks IMX6UL_CLK_CSI>; 
         
         assigned-clock-parents =
                           <&clks IMX6UL_CLK_PLL2_PFD2>;
 
         assigned-clock-rates =
                           <0>, <50000000>; 

with

	      csi_id = <0>;
	      mclk = <50000000>;
          mclk_source = <0>;

which stopped the Kernel Panic / failed boot attempts (will still try to get the logs at the next opportunity though). Now my new problem is, that i.mx6s_csi (what seems to be the driver of the camera interface of the SoM) is complaining about the cameras mbus of 0x3011, which v4l2 should know as MEDIA_BUS_FMT_SGRBG12_1X12 according to https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/subdev-formats.html .

"unknown mbus: 0x3011
mx6s-csi 21c4000.csi: mbus (0x00003011) invalid."

Do you have an idea what could be done here?

HI @m.kl

I have never seen this error before.

Could you share the complete dmesg.log and all the changes to devicetree files ( git diff )?

Thanks and best regards,
Jaski

Here you (hopefully) go. Thank you for your patience.

Git Diff of Linux-Toradex and Dmesg .log
(The 2 lines in the log with ---------- at the start are my inputs)

HI @m.kl

Thanks for sharing the dmesg.log and changes in devicetree files. I will try to reproduce this with our parallel camera module.

Please Note, the parallel camera feature was never brought up and tested on iMX6ULL since the SoM is not very useful for high end graphics. May I know what is your application?

Best regards,
Jaski

HI @jaski.tx

I think I found out what the issue is… the Camera driver (mt9p031.c in drivers/media/i2c) lists one supported mbus format, 0x3011, which the CSI-Module driver (mx6s_capture.c in drivers/media/platform/mxc/capture) doesn’t know. After changing the format in the camera driver (MEDIA_BUS_FMT_SGRBG12_1X12) to one listed in the CSI-driver (MEDIA_BUS_FMT_SBGGR8_1X8) it doesn’t complain anymore. I guess i should probably try to add the cameras actual format to the CSI-driver instead, to avoid confusion with other software though.

Trying to take a picture with any application still doesn’t work though, because “timeout when wait for SOF”, which seems to originate from mx6s_capture.c line 795. I’ve added a log file with that, maybe you can understand more of this and help me find the cause of that new problem.

log file line that starts with -------- is my input

Regarding my application, it is (somewhat abstracted) to monitor the remaining capacity of a space by taking a photo up to every 10 sec (or even less frequently), calculating a measure for the difference to the last picture, maybe also image segmentation if possible, and then uploading the findings and image to another device via Wifi. Also monitoring interrupts from two additional sensors

it seems that trying to take a picture turns the camera off… until the v4l2-ctl line to read an image it is outputting things that look normal, but afterwards it stops outputting

Hi, I copied a workaround for the timeout issue from the camera module manufacturers CSI driver version (which sadly uses a different kernel version), now that has stopped. Instead I am getting

VIDIOC_STREAMON: Invalid argument

(or vidioc_streamon returned -1 (invalid argument))

Do you have an idea what might be causing that?

These are my changed driver files for the camera and CSI.

Which kernel did you use?

You can also try the mainline kernel (5.4) with the corresponding device-tree. This should work too.

Best regards,
Jaski