Linux NTFS/exFat: doesn't work

Technical Support
, , , ,
1

Hi!

For a usb related feature, we would like to support other filesystem formats than FAT (and ext* of course).

However, I was not able to get it working. I tested with different USB drives, from different vendors. The filesystem formatting was done on a Windows 10 computer.

NTFS:

First of all, i tried to enable ntfs directly in the kernel with the following params:

CONFIG_NTFS_FS=y
CONFIG_NTFS_RW=y

My second attempt was to include the ntfs-3g-ntfsprogs package. (ntfs-3g driver via FUSE)

Both results into an access error:

root@colibri-imx6:/media# ls -l
drwxrwx---    2 root     disk         16384 Jan  1  1970 mmcblk0p1
drwxrwx---    2 root     disk         32768 Jan  1  1970 mmcblk1p1
drwx------    1 root     root          4096 Mar 29 10:12 sda1
root@colibri-imx6:/media# cd sda1/
root@colibri-imx6:/media/sda1# touch test.txt
touch: test.txt: Permission denied
root@colibri-imx6:/media/sda1#

Also sda1 should be mounted with root:disk not root:root.

exFAT:
I enabled exFAT support by including the fuse-exfat package (exFAT via FUSE). However, this results into an error:

root@colibri-imx6:/media# ls -l
ls: ./sda1: Transport endpoint is not connected
drwxrwx---    2 root     disk         16384 Jan  1  1970 mmcblk0p1
drwxrwx---    2 root     disk         32768 Jan  1  1970 mmcblk1p1
root@colibri-imx6:/media# cd sda1
-sh: cd: sda1: Not a directory
root@colibri-imx6:/media#

However, if I unmount and remount the device, it works:

root@colibri-imx6:/media# umount sda1
root@colibri-imx6:/media# mount -t exfat /dev/sda1 ./sda1/
FUSE exfat 1.2.3
WARN: volume was not unmounted cleanly.
root@colibri-imx6:/media# cd sda1/
root@colibri-imx6:/media/sda1# ls
System Volume Information
root@colibri-imx6:/media/sda1# touch test.txt
root@colibri-imx6:/media/sda1# ls
System Volume Information  test.txt
root@colibri-imx6:/media/sda1#

Any suggestions about this topic?

BR Christian

2

Hi @cheesi

I assume the device has been mounted automatically the first time. This has probably done by udev. There is somewhere a udev rule that mounts your device with root:root. So you probably just need to adjust that udev rule or add your own. Check out this Wiki article: udev - ArchWiki

When you mount it the second time on console, you are mounting the usb drive and not udev that’s probably the reason it is then mounted with root:disk.

Best Regards,
Philippe

3

Hi @philippe.tx !

Your are mixing some things up here.

First of all: I never mounted NTFS manually on the console. It is automounted (as you mentioned by udev) with root:root. FAT32 usb drives, however, are automounted with root:disk.
Nevertheless, this doesn’t change anything about the “Permission denied” error. As you can see from the log above, I’m running as a root user and still get the same message. The reason why I need the drive to be mounted with root:disk is, because the final application is running with it’s own user, but in the group disk.

Now to the second part (exFAT): the automount didn’t work (“./sda1: Transport endpoint is not connected”), but a manual remount would solve that problem. However, this isn’t a usable process for us. We are not able to determine, if the drive has an exFAT formatting, so we couldn’t handle this special case. The formatting can be anything from FAT16/32 to NTFS and exFAT.

BR Christian

4

Hi @cheesi

I quickly checked where that stuff happens on our regular BSP. I found in /etc/udev/rules.d/automount.rules the following udev rules:

SUBSYSTEM=="block", ACTION=="add"    RUN+="/etc/udev/scripts/mount.sh"
SUBSYSTEM=="block", ACTION=="remove" RUN+="/etc/udev/scripts/mount.sh"
SUBSYSTEM=="block", ACTION=="change", ENV{DISK_MEDIA_CHANGE}=="1" RUN+="/etc/udev/scripts/mount.sh"

From there you see that all the magic is happening in /etc/udev/scripts/mount.sh. I think from there you can apply those specific settings that are probably needed for exFAT and NTFS.

Best Regards,
Philippe

5

Hi @philippe.tx

I see. This could solve the exFAT problem with some modifications in the mount.sh. Even thought, a double mount seems quite pathetic. (And to be honest, this shouldn’t even be necessary). Nevertheless, I will give it a try, even if it is just symptomatic treatment.

However, this will not solve the problem with NTFS. Any suggestions about that? I will try to also mount it with root:disk, but I don’t think this will change anything about the “Permission denied” error, as I am already running as root.

BR Christian

6

Hi @philippe.tx

I just gave it try. As presumed, adding ntfs to the mount.sh script didn’t changed anything. I still get the same error.
According to exFAT, I gained some new information: the mount of the exfat drive within the mount.sh script works without any problem at first. I was able to verify this by adding a sleep command after the successful mount in the mount.sh script. After that delay, the mount point becomes unusable. So the umount/mount workaround won’t work here. My first guess was, that maybe there is an other udev rule “crashing” (changing) the mount point. However, I looked through all the rules and couldn’t find one.

At one point, I saw the following change within the /media directory:
First: drwxrwx--- 1 root disk 32768 Jan 1 1970 sda1
Then: drwxr-xr-x 2 root root 1024 Apr 5 19:49 sda1
Finally: ls: ./sda1: Transport endpoint is not connected

Our opinion about that?

BR Christian

7

hi @cheesi

Sorry for the delayed answer. I mounted the problem with NTFS as following:

You need to mount it with the following command:
ntfs-3g /dev/sda1 /media/NTFS

Then you are able to write the FileSystem too.

Best regards,
Jaski

8

Hi @jaski.tx !

I already added the ntfs-3g-ntfsprogs package to our image, but there is no command ntfs-3g. Now I have also tried mounting it with mount -t ntfs-3g as mentioned in the NTFS-3G manual (https://www.tuxera.com/community/ntfs-3g-manual/), however this results into an error: mount: /media/sda1: unknown filesystem type 'ntfs-3g'.

This looks quite strange, as according to the recipe (ntfs-3g-ntfsprogs_2017.3.23.bb\ntfs-3g-ntfsprogs\recipes-filesystems\meta-filesystems - meta-openembedded - Collection of OpenEmbedded layers) there should also be a binary at /sbin/mount.ntfs-3g and a link at /sbin/mount.ntfs, however I don’t have neither of them.

The ntfs-3g-ntfsprogs package, however, is definitely installed:

root@colibri-imx6:/sbin# opkg list-installed | grep ntfs
ntfs-3g-ntfsprogs - 2017.3.23-r0
root@colibri-imx6:/sbin#

I am at my wit’s end…

9

Hmm, this is strange. I used a regular LXDE 2.8b6 image and then just compiled the ntfs-3g seperately as ipk pacakge. I think, you need to install ntfs-3g_X package too. Here you can find the compiled packages.

10

Hi @jaski.tx
I downloaded the packages and installed them on my testing device:

root@colibri-im6:~# wget https://share.toradex.com/iva52n2jtrafr8k?direct
--2019-04-12 16:51:50--  https://share.toradex.com/iva52n2jtrafr8k?direct
Resolving share.toradex.com... 178.32.63.175
Connecting to share.toradex.com|178.32.63.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 189730 (185K) [application/zip]
Saving to: ‘iva52n2jtrafr8k?direct’

iva52n2jtrafr8k?dir 100%[===================>] 185.28K   499KB/s    in 0.4s

2019-04-12 16:51:51 (499 KB/s) - ‘iva52n2jtrafr8k?direct’ saved [189730/189730]

root@colibri-im6:~# unzip iva52n2jtrafr8k\?direct
Archive:  iva52n2jtrafr8k?direct
  inflating: ntfs-3g_2017.3.23-r0_armv7at2hf-neon.ipk
  inflating: libntfs-3g88_2017.3.23-r0_armv7at2hf-neon.ipk
root@colibri-im6:~# ls
iva52n2jtrafr8k?direct
libntfs-3g88_2017.3.23-r0_armv7at2hf-neon.ipk
ntfs-3g_2017.3.23-r0_armv7at2hf-neon.ipk
root@colibri-im6:~# opkg install libntfs-3g88_2017.3.23-r0_armv7at2hf-neon.ipk                                                                                                                                                                                                                                                                     
Installing libntfs-3g88 (2017.3.23) on root
Configuring libntfs-3g88.
/sbin/ldconfig: /usr/lib/libOpenVG.so is not a symbolic link

root@colibri-im6:~# opkg install ntfs-3g_2017.3.23-r0_armv7at2hf-neon.ipk
Installing ntfs-3g (2017.3.23) on root
Configuring ntfs-3g.

And inserted my NTFS formatted USB drive:

root@colibri-im6:~# cd /media/
root@colibri-im6:/media# ls
ls: ./sda1: Transport endpoint is not connected
mmcblk0p1  mmcblk1p1

Now I’m getting the same error as with the exFAT fuse driver…
It seems like - even though it added the ntfs-3g-ntfsprogs package - it wasn’t fully included/installed. And so of course, mount used the kernel driver. Another thing, which will need some investigation.

Back to error message: It seems a little bit, like fuse is broken?

Could you please try it with a regular console image? Cause’ basically, that’s what we use (with some additional packages and files).

11

And so of course, mount used the kernel driver. Another thing, which will need some investigation.

Yes, I think too that the kernel NTFS driver must be deactivated before using ntsfs-3g. I will try to reproduce this, next week on a console Image.

Best regards and have a nice weekend,
Jaski

12

Hi @jaski.tx !

I will try to reproduce this, next week on a console Image.

Any updates according to this?

BR Christian

13

Hi @cheesi

Not yet. I will get back to you this week.

Best regards,
Jaski

15

Dear @cheesi ,

Thank you for your patience!

Unfortunately, we don’t have the expertise with NTFS on Linux.
Also, as we no longer support Windows at Toradex, we also don’t have the NTFS support on our modules. But if it is critical to your application, we can try to find a way around.

Thanks and Regards,
Janani.

16

hi @developer0916
Thanks for your Input.

@cheesi, could you try this out? Thanks

Best regards,
Jaski

17

I was able to use exfat and NTFS.

Unlike filesystems that are mounted by the kernel, FUSE keeps processes remaining while it is mounted.
udev kills such long-lived processes.
This causes an error in the mount.

Below is some helpful information.

To avoid this, I created a shell that launches the FUSE mount as a systemd service.

Environment:
Colibri-iMX6ULL 512MB
EVA Board 3.2A
Linux BSP 3.0.4

Add recipe
local.conf
IMAGE_INSTALL_append = " fuse-exfat ntfs-3g"

add exfat and ntfs case
/etc/udev/scripts/mount.sh

--- ./a	2020-10-24 09:36:42.000000000 +0900
+++ ./b	2020-10-27 13:59:32.434785087 +0900
@@ -20,6 +20,13 @@
 automount() {	
 	name="`basename "$DEVNAME"`"
 
+	if [ "$ID_FS_TYPE" = "exfat" ] || [ "$ID_FS_TYPE" = "ntfs" ];
+	then
+		systemctl start fuse-mount@$name
+		logger "$ID_FS_TYPE mount $DEVNAME"
+		return 0
+	fi
+
 	! test -d "/media/$name" && mkdir -p "/media/$name"
 	# Silent util-linux's version of mounting auto
 	if [ "x`readlink $MOUNT`" = "x/bin/mount.util-linux" ] ;
@@ -79,6 +86,12 @@
 
 
 if [ "$ACTION" = "remove" ] || [ "$ACTION" = "change" ] && [ -x "$UMOUNT" ] && [ -n "$DEVNAME" ]; then
+	if [ "$ID_FS_TYPE" = "exfat" ] || [ "$ID_FS_TYPE" = "ntfs" ];
+	then
+		systemctl stop fuse-mount@$name
+		logger "$ID_FS_TYPE unmount $DEVNAME"
+		exit 0
+	fi
 	for mnt in `cat /proc/mounts | grep "$DEVNAME" | cut -f 2 -d " " `
 	do
 		$UMOUNT $mnt

add fuse-mount@.service /etc/systemd/system/fuse-mount@.service

[Unit]
 Description=Mount FUSE  %i
 
 [Service]
 Type=oneshot
 RemainAfterExit=true
 ExecStart=/etc/udev/scripts/fuse-mount.sh add %i
 ExecStop=/etc/udev/scripts/fuse-mount.sh remove %i

add fuse-mount.sh /etc/udev/scripts/fuse-mount.sh

 #!/bin/sh
 
 MOUNT="/bin/mount"
 UMOUNT="/bin/umount"
 
 ACTION=$1
 DEVBASE=$2
 DEVNAME="/dev/${DEVBASE}"
 
 do_mount()
 {
         name="`basename "$DEVNAME"`"
 
         ! test -d "/media/$name" && mkdir -p "/media/$name"
         # Silent util-linux's version of mounting auto
         if [ "x`readlink $MOUNT`" = "x/bin/mount.util-linux" ] ;
         then
                 MOUNT="$MOUNT -o silent"
         fi
 
         if ! $MOUNT -t auto $DEVNAME "/media/$name"
         then
                 #logger "mount.sh/automount" "$MOUNT -t auto $DEVNAME \"/media/$name\" failed!"
                 rm_dir "/media/$name"
         else
                 logger "mount.sh/automount" "Auto-mount of [/media/$name] successful"
                 touch "/tmp/.automount-$name"
         fi
 }
 do_unmount()
 {
         for mnt in `cat /proc/mounts | grep "$DEVNAME" | cut -f 2 -d " " `
         do
                 $UMOUNT $mnt
         done
 
         # Remove empty directories from auto-mounter
         name="`basename "$DEVNAME"`"
         test -e "/tmp/.automount-$name" && rm_dir "/media/$name"
 }
 
 rm_dir() {
         # We do not want to rm -r populated directories
         if test "`find "$1" | wc -l | tr -d " "`" -lt 2 -a -d "$1"
         then
                 ! test -z "$1" && rm -r "$1"
         else
                 logger "mount.sh/automount" "Not removing non-empty directory [$1]"
         fi
 }
 
 
 case "${ACTION}" in
     add)
         do_mount
         ;;
     remove)
         do_unmount
         ;;
     *)
         ;;
 esac

Best regads.