Camera access in browser

I’ve been following guide and trying to access my camera in browser, but the browser does not see it. I have tried different settings (e.g. using device_cgroup_rules, volumes), all my attempts failed.
I am able to see camera preview when I set /dev/video4, so my camera is working.

In my opinion there are two possible sources of the problem: something wrong in my docker-compose file, browser does not support camera access.

What am I doing wrong?

My docker-compose file

version: "2.4"
services:
  weston:
    image: torizon/arm64v8-debian-weston-vivante:latest
    network_mode: host
    environment:
      - ACCEPT_FSL_EULA=1
    devices:
      - /dev/video4:/dev/video4
    volumes:
      - type: bind
        source: /tmp
        target: /tmp
      - type: bind  
        source: /run/udev
        target: /run/udev
      - type: bind
        source: /dev
        target: /dev
    cap_add:
      - CAP_SYS_TTY_CONFIG
    # Add device access rights through cgroup...
    device_cgroup_rules:
      # ... for tty0
      - 'c 4:0 rmw'
      # ... for tty7
      - 'c 4:7 rmw'
      # ... for /dev/input devices
      - 'c 13:* rmw'
      - 'c 199:* rmw'
      # ... for /dev/dri devices
      - 'c 226:* rmw'
      - 'c 81:* rmw'
  wayland:
    depends_on:
      - weston
    image: torizon/arm64v8-debian-kiosk-mode-browser:latest
    security_opt:
      - seccomp:unconfined
    command: --browser-mode https://google.com
    shm_size: 256mb
    devices:
      - /dev/video4:/dev/video4
    volumes:
       - /tmp:/tmp
       - /var/run/dbus:/var/run/dbus
       - /sys:/sys
       - /dev:/dev
    device_cgroup_rules:
      # ... for /dev/input devices
      - 'c 13:* rmw'
      - 'c 199:* rmw'
      # ... for /dev/dri devices
      - 'c 226:* rmw'
      - 'c 81:* rmw'

Console output

Creating torizon_weston_1 ... done
Creating torizon_wayland_1 ... done
Attaching to torizon_weston_1, torizon_wayland_1
weston_1   | NXP EULA has already been accepted.
weston_1   | Switching to VT 7
weston_1   | NXP EULA has already been accepted.
weston_1   | touch: cannot touch '/tmp/nxp-eula-accepted': Permission denied
weston_1   | Date: 2020-06-17 UTC
weston_1   | [09:21:58.616] weston 5.0.0
weston_1   |                https://wayland.freedesktop.org
weston_1   |                Bug reports to: https://gitlab.freedesktop.org/wayland/weston/issues/
weston_1   |                Build: unknown (not built from git or tarball)
weston_1   | [09:21:58.616] Command line: /usr/bin/weston --current-mode
weston_1   | [09:21:58.616] OS: Linux, 4.14.170-4.0.0-devel+git.fb6b17a22634, #1-TorizonCore SMP PREEMPT Mon May 11 14:38:24 UTC 2020, aarch64
weston_1   | [09:21:58.617] Using config file '/etc/xdg/weston/weston.ini'
weston_1   | [09:21:58.617] Output repaint window is 16 ms maximum.
weston_1   | [09:21:58.617] Loading module '/usr/lib/aarch64-linux-gnu/libweston-5/drm-backend.so'
weston_1   | [09:21:58.619] initializing drm backend
weston_1   | [09:21:58.623] using /dev/dri/card0
weston_1   | [09:21:58.624] DRM: supports universal planes
weston_1   | [09:21:58.624] DRM: supports atomic modesetting
weston_1   | [09:21:58.624] DRM: does not support picture aspect ratio
weston_1   | [09:21:58.624] Loading module '/usr/lib/aarch64-linux-gnu/libweston-5/gl-renderer.so'
weston_1   | [09:21:58.636] EGL client extensions: EGL_EXT_client_extensions
weston_1   |                EGL_EXT_platform_base EGL_KHR_platform_wayland
weston_1   |                EGL_EXT_platform_wayland EGL_KHR_platform_gbm
weston_1   | [09:21:58.641] warning: neither EGL_EXT_swap_buffers_with_damage or EGL_KHR_swap_buffers_with_damage is supported. Performance could be affected.
weston_1   | [09:21:58.641] EGL_KHR_surfaceless_context available
weston_1   | [09:21:58.647] EGL version: 1.5
weston_1   | [09:21:58.648] EGL vendor: Vivante Corporation
weston_1   | [09:21:58.648] EGL client APIs: OpenGL_ES
weston_1   | [09:21:58.648] EGL extensions: EGL_KHR_fence_sync EGL_KHR_reusable_sync
weston_1   |                EGL_KHR_wait_sync EGL_KHR_image EGL_KHR_image_base
weston_1   |                EGL_KHR_image_pixmap EGL_KHR_gl_texture_2D_image
weston_1   |                EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image
weston_1   |                EGL_EXT_image_dma_buf_import
weston_1   |                EGL_EXT_image_dma_buf_import_modifiers EGL_KHR_lock_surface
weston_1   |                EGL_KHR_create_context EGL_KHR_surfaceless_context
weston_1   |                EGL_EXT_buffer_age EGL_ANDROID_native_fence_sync
weston_1   |                EGL_WL_bind_wayland_display
weston_1   |                EGL_WL_create_wayland_buffer_from_image EGL_KHR_partial_update
weston_1   | [09:21:58.648] GL version: OpenGL ES 3.2 V6.2.4.p4.190076
weston_1   | [09:21:58.648] GLSL version: OpenGL ES GLSL ES 3.20
weston_1   | [09:21:58.648] GL vendor: Vivante Corporation
weston_1   | [09:21:58.648] GL renderer: Vivante GC7000XSVX
weston_1   | [09:21:58.648] GL extensions: GL_OES_vertex_type_10_10_10_2
weston_1   |                GL_OES_vertex_half_float GL_OES_element_index_uint
weston_1   |                GL_OES_mapbuffer GL_OES_vertex_array_object
weston_1   |                GL_OES_compressed_ETC1_RGB8_texture
weston_1   |                GL_OES_compressed_paletted_texture GL_OES_texture_npot
weston_1   |                GL_OES_rgb8_rgba8 GL_OES_depth_texture
weston_1   |                GL_OES_depth_texture_cube_map GL_OES_depth24 GL_OES_depth32
weston_1   |                GL_OES_packed_depth_stencil GL_OES_fbo_render_mipmap
weston_1   |                GL_OES_get_program_binary GL_OES_fragment_precision_high
weston_1   |                GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_sync
weston_1   |                GL_OES_texture_stencil8 GL_OES_shader_image_atomic
weston_1   |                GL_OES_sample_variables GL_OES_sample_shading
weston_1   |                GL_OES_texture_storage_multisample_2d_array
weston_1   |                GL_OES_shader_multisample_interpolation
weston_1   |                GL_OES_required_internalformat GL_OES_surfaceless_context
weston_1   |                GL_OES_copy_image GL_OES_draw_buffers_indexed
weston_1   |                GL_OES_geometry_shader GL_OES_geometry_point_size
weston_1   |                GL_OES_gpu_shader5 GL_OES_shader_io_blocks
weston_1   |                GL_OES_texture_border_clamp GL_OES_texture_buffer
weston_1   |                GL_OES_tessellation_shader GL_OES_tessellation_point_size
weston_1   |                GL_OES_texture_cube_map_array GL_OES_draw_elements_base_vertex
weston_1   |                GL_OES_texture_half_float GL_OES_texture_float
weston_1   |                GL_OES_primitive_bounding_box
weston_1   |                GL_KHR_texture_compression_astc_ldr
weston_1   |                GL_KHR_blend_equation_advanced GL_KHR_debug GL_KHR_robustness
weston_1   |                GL_EXT_texture_type_2_10_10_10_REV
weston_1   |                GL_EXT_texture_filter_anisotropic
weston_1   |                GL_EXT_texture_compression_dxt1 GL_EXT_texture_format_BGRA8888
weston_1   |                GL_EXT_texture_compression_s3tc GL_EXT_read_format_bgra
weston_1   |                GL_EXT_multi_draw_arrays GL_EXT_frag_depth
weston_1   |                GL_EXT_discard_framebuffer GL_EXT_blend_minmax
weston_1   |                GL_EXT_multisampled_render_to_texture
weston_1   |                GL_EXT_color_buffer_half_float GL_EXT_color_buffer_float
weston_1   |                GL_EXT_robustness GL_EXT_texture_sRGB_decode
weston_1   |                GL_EXT_draw_buffers_indexed GL_EXT_texture_border_clamp
weston_1   |                GL_EXT_texture_buffer GL_EXT_tessellation_shader
weston_1   |                GL_EXT_tessellation_point_size GL_EXT_geometry_shader
weston_1   |                GL_EXT_geometry_point_size GL_EXT_copy_image
weston_1   |                GL_EXT_texture_cube_map_array GL_EXT_gpu_shader5
weston_1   |                GL_EXT_shader_io_blocks GL_EXT_shader_implicit_conversions
weston_1   |                GL_EXT_multi_draw_indirect GL_EXT_draw_elements_base_vertex
weston_1   |                GL_EXT_texture_rg GL_EXT_primitive_bounding_box GL_EXT_sRGB
weston_1   |                GL_ANDROID_extension_pack_es31a GL_VIV_direct_texture
weston_1   | [09:21:58.648] GL ES 2 renderer features:
weston_1   |                read-back format: BGRA
weston_1   |                wl_shm sub-image to texture: yes
weston_1   |                EGL Wayland extension: yes
weston_1   | [09:21:58.659] event2  - LITEON Technology USB Multimedia Keyboard: is tagged by udev as: Keyboard
weston_1   | [09:21:58.659] event2  - LITEON Technology USB Multimedia Keyboard: device is a keyboard
weston_1   | [09:21:58.663] event1  - Logitech USB Optical Mouse: is tagged by udev as: Mouse
weston_1   | [09:21:58.663] event1  - Logitech USB Optical Mouse: device is a pointer
weston_1   | [09:21:58.664] event0  - sc-powerkey: is tagged by udev as: Keyboard
weston_1   | [09:21:58.664] event0  - sc-powerkey: device is a keyboard
weston_1   | [09:21:58.677] event3  - colibri-vf50-ts: is tagged by udev as: Touchscreen
weston_1   | [09:21:58.677] event3  - colibri-vf50-ts: device is a touch device
weston_1   | [09:21:58.688] Touchscreen - colibri-vf50-ts - /sys/devices/platform/vf50-touchscreen/input/input3/event3
weston_1   | [09:21:58.688] input device event3 has no enabled output associated (none named), skipping calibration for now.
weston_1   | [09:21:58.714] DRM: head 'HDMI-A-1' found, connector 146 is connected, EDID make 'PHL', model 'PHL 234E5', serial 'UHB142400574'
weston_1   | [09:21:58.714] Registered plugin API 'weston_drm_output_api_v1' of size 24
weston_1   | [09:21:58.725] Chosen EGL config details:
weston_1   |                RGBA bits: 8 8 8 0
weston_1   |                swap interval range: 1 - 60
weston_1   | [09:21:58.725] No backlight control for output 'HDMI-A-1'
weston_1   | [09:21:58.725] Output HDMI-A-1 (crtc 36) video modes:
weston_1   |                1920x1080@60.0, preferred, current, 148.5 MHz
weston_1   |                1920x1080@59.9, 148.4 MHz
weston_1   |                1920x1080@50.0, 148.5 MHz
weston_1   |                1680x1050@59.9, 119.0 MHz
weston_1   |                1280x1024@75.0, 135.0 MHz
weston_1   |                1280x1024@60.0, 108.0 MHz
weston_1   |                1440x900@75.0, 136.8 MHz
weston_1   |                1440x900@59.9, 88.8 MHz
weston_1   |                1280x720@60.0, 74.2 MHz
weston_1   |                1280x720@59.9, 74.2 MHz
weston_1   |                1280x720@50.0, 74.2 MHz
weston_1   |                1024x768@75.0, 78.8 MHz
weston_1   |                1024x768@60.0, 65.0 MHz
weston_1   |                800x600@75.0, 49.5 MHz
weston_1   |                800x600@60.3, 40.0 MHz
weston_1   |                720x576@50.0, 27.0 MHz
weston_1   |                720x480@60.0, 27.0 MHz
weston_1   |                720x480@59.9, 27.0 MHz
weston_1   |                640x480@75.0, 31.5 MHz
weston_1   |                640x480@72.8, 31.5 MHz
weston_1   |                640x480@66.7, 30.2 MHz
weston_1   |                640x480@60.0, 25.2 MHz
weston_1   |                640x480@59.9, 25.2 MHz
weston_1   |                720x400@70.1, 28.3 MHz
weston_1   | [09:21:58.725] associating input device event2 with output HDMI-A-1 (none by udev)
weston_1   | [09:21:58.725] associating input device event1 with output HDMI-A-1 (none by udev)
weston_1   | [09:21:58.725] associating input device event0 with output HDMI-A-1 (none by udev)
weston_1   | [09:21:58.726] associating input device event3 with output HDMI-A-1 (none by udev)
weston_1   | [09:21:58.726] Output 'HDMI-A-1' enabled with head(s) HDMI-A-1
weston_1   | [09:21:58.726] Compositor capabilities:
weston_1   |                arbitrary surface rotation: yes
weston_1   |                screen capture uses y-flip: yes
weston_1   |                presentation clock: CLOCK_MONOTONIC, id 1
weston_1   |                presentation clock resolution: 0.000000001 s
weston_1   | [09:21:58.727] Loading module '/usr/lib/aarch64-linux-gnu/weston/desktop-shell.so'
weston_1   | [09:21:58.729] launching '/usr/lib/aarch64-linux-gnu/weston-keyboard'
weston_1   | [09:21:58.736] Loading module '/usr/lib/aarch64-linux-gnu/libweston-5/xwayland.so'
weston_1   | [09:21:58.755] Registered plugin API 'weston_xwayland_v1' of size 32
weston_1   | [09:21:58.756] Registered plugin API 'weston_xwayland_surface_v1' of size 16
weston_1   | [09:21:58.756] xserver listening on display :0
weston_1   | [09:21:58.756] launching '/usr/lib/aarch64-linux-gnu/weston-desktop-shell'
weston_1   | could not load cursor 'dnd-move'
weston_1   | could not load cursor 'dnd-move'
weston_1   | could not load cursor 'dnd-copy'
weston_1   | could not load cursor 'dnd-copy'
weston_1   | could not load cursor 'dnd-none'
weston_1   | could not load cursor 'dnd-none'
weston_1   | [09:22:01.599] Spawned Xwayland server, pid 26
weston_1   | glamor: 'wl_drm' not supported
weston_1   | Missing Wayland requirements for glamor GBM backend
weston_1   | Failed to initialize glamor, falling back to sw
weston_1   | [09:22:01.893] xfixes version: 5.0
weston_1   | [09:22:01.947] created wm, root 908
wayland_1  | [1:34:0617/092202.800694:ERROR:bus.cc(393)] Failed to connect to the bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
wayland_1  | [1:218:0617/092203.285383:ERROR:bus.cc(393)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
wayland_1  | [1:223:0617/092203.350775:ERROR:bus.cc(393)] Failed to connect to the bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
wayland_1  | [1:223:0617/092203.351334:ERROR:bus.cc(393)] Failed to connect to the bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
wayland_1  | [1:223:0617/092203.351907:ERROR:bus.cc(393)] Failed to connect to the bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
wayland_1  | [1:223:0617/092203.353864:ERROR:bus.cc(393)] Failed to connect to the bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
wayland_1  | [1:223:0617/092203.355680:ERROR:bus.cc(393)] Failed to connect to the bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
wayland_1  | Error creating gpu
wayland_1  | [44:44:0617/092204.416237:ERROR:sandbox_linux.cc(369)] InitializeSandbox() called with multiple threads in process gpu-process.
wayland_1  | [456:1:0617/092218.768361:ERROR:child_process_sandbox_support_impl_linux.cc(81)] FontService unique font name matching request did not receive a response.
wayland_1  | [456:1:0617/092218.770040:ERROR:child_process_sandbox_support_impl_linux.cc(81)] FontService unique font name matching request did not receive a response.
mayland_1  | 0m3 2[6425.61:716:20469]17 /(0n9u2l2l1)8.:7 7m7x8c7_7i:sEiR_RcOaRp:tcuhriel_do_pperno, ceNsos _rseamnodtbeo xp_asdu pfpoourntd_!i
 pl_linux.cc(81)] FontService unique font name matching request did not receive a response.
wayland_1  | [456:1:0617/092218.782307:ERROR:child_process_sandbox_support_impl_linux.cc(81)] FontService unique font name matching request did not receive a response.
[ 3262.212981] (null): mxc_isi_capture_open, No remote pad found!
[ 3262.218961] (null): mxc_isi_capture_open, No remote pad found!
[ 3262.224879] (null): mxc_isi_capture_open, No remote pad found!
[ 3264.848055] (null): mxc_isi_capture_open, No remote pad found!
[ 3264.855067] (null): mxc_isi_capture_open, No remote pad found!
[ 3264.860989] (null): mxc_isi_capture_open, No remote pad found!
[ 3264.866912] (null): mxc_isi_capture_open, No remote pad found!

Greetings @AlexeyZ,

Can you provide more detail on how exactly you were trying to test the camera in the browser? i.e. What did you do in the browser after you launched your containers?

Just to be clear I believe this isn’t something we have tested/verified ourselves yet so I’m not completely sure if the support is all there for this to work. So I need to do some research and get back to you. But it’d be helpful to know your steps so we’re on the same page.

On initial review I’d assume you’ll need to do a bit of a custom container here that has both the browser you want to use as well as the gstreamer utilities needed to access the camera properly. Then it comes down as you said “whether the browser has proper camera access support”, which I will check on and get back to you when I have a clear answer.

By the way since you’re using browser, may I ask whether your project requires proper browser hardware acceleration? (i.e. webGL) Cause if you do then please keep in mind that accelerated browser support is something we are still working on. However if you can provide any feedback/requirements you have related to this, it would be greatly appreciated.

Best Regards,
Jeremias

Hello @jeremias.tx

For testing camera access I used such websites: webcamtests, browserleaks (WebRTC Media Devices). Moreover I tried gstreamer + WebRTC demo where the call of navigator.mediaDevices.getUserMedia({"video": true}) throws NotFoundError: Requested device not found.
The idea of my application is based on streaming video from my camera + simultaneous periodical taking photos, stream from the camera should be displayed in a html page together with some business things and another application in background sometimes takes photos. WebGL isn’t supposed to be used in the app.

Jeremias, you said that for proper camera access in the browser I need to use an additional utility, I thought that the browser can handle cameras on its own. Initially I tried to stream video from the camera via gstreamer and then show it in the browser, but the browser could not show streams: vp8enc + webmmux ERR_INVALID_HTTP_RESPONSE, theoraenc + oggmux ERR_INVALID_HTTP_RESPONSE. If you know demos/applications with such requirements as mine, I would be glad to know about them. At the moment I’m trying integration with mjpg-streamer.

Best Regards,
Alexey

Hi @AlexeyZ,

I did some further investigation on my side. So I used your docker-compose but with a USB camera instead. With a USB camera the browser was able to just detect and use it with no other changes.

This leads me to think there are 2 possible issues here. Either there is a permission issue with the browser accessing the MIPI CSI device or there is an issue with the output format of the OV5640 not being supported by the browser.

You can test for the first case by running your containers with full privileges with the privileged flag. If this still doesn’t work then it’s more likely the 2nd issue of incompatible format. In which case you may need to configure the camera to output a format that is compatible or if there is no compatible format then add a way to convert to a compatible format.

Best Regards,
Jeremias

Hello @jeremias.tx

I’ve just tried privileged flag, it didn’t help. It seems the problem is in the format incompatibility.
I started using MJPEG-Streamer, it provides both as taking photos and getting MJPEG stream which is supported by the browser.

Best Regards,
Alexey

Glad you were able to find a solution, and also we appreciate the feedback!