How to make partitions on SLC NAND flash which uses ubifs?

We are using Toradex colibri VF50 module with latest kernel v2.6.1beta. It has
SLC NAND Flash with 100k write cycles. It uses ubifs. It has 4 MTD partitions
out of which one is for ubi. On this partition, we are using ubifs. The ubifs
has 3 logical volumes - kernel, dtb and rootfs. We have our applications and
configuration files on rootfs.

The problem with NAND flash is that when we write a file to it and close the
file, and if the power is cut while the application is still running, it
doesn’t synchronize the file and the entire content of the file gets deleted
when checked on restarting the system. We have already tried using fsync()
after writing to file so that it synchronizes the file as soon as something
is written to it but still there is no surety that on sudden power cuts,
file system will not get corrupted. To solve this problem, we have planned
to do partitioning of file system so that the important system related
content remains in read-only partition and nothing happens to it even when
the file system on some other partition gets corrupted.

We want to partition the flash as follows:
1. One partition for system files mounted as read only
2. One partition for configuration files which need to be read/written at
runtime, mounted initially as read only. For this partition, we are
planning to mount it as read-write during run time when we need to write
to configuration files, and immediately after writing, we will re-mount
it as read only.
3. One partition /sys/log mounted as read-write to write my log files.

So kindly guide us how to go about making these partitions(ubi volumes) providing us some useful links to refer to or some other better option than partitioning to solve the above issue.

While I think it is a good practice to use separate file system for data/configuration and the rootfs, I don’t think it is solving your data integrity issue… Just creating a second file system instance won’t change the behavior of the file system… This answer consists of two parts:

Ensure data integrity

Ensuring data integrity is not as easy as it seems. Especially nowadays file system optimize a lot and try to delay writes as much as possible. Using fsync() after writing data to the file should make sure data hit the NAND flash, but there is still a window in which the file could end up being empty (what happens if power gets lost during fsync?). Especially, it does not guarantee that the old data are still there. The common approach is to create a new, temporary file, write data, fsync() that file, and then do the renaming and a final fsync on the directory. Refer also to this LWN article for more info.

Creating independent UBIFS instances

You can partition the NAND flash on different levels:

  • Create separate MTD partitions
  • Create separate UBI volumes

Creating separate MTD partitions splits up the NAND flash in physical separated areas. Our BSP by default uses separate areas for the boot block and boot loader only, all the rest is in a single MTD partition (called ubi, see also the output of the mtdparts command in U-Boot). Having only one big partition allows to reserve blocks for bad block management only once. A NAND device makes guarantees that not more than a certain amount of blocks will turn bad within the guaranteed life cycle (e.g. ~5% or 20 blocks in the VF50 case). However, this could be clustered, so if you partition the device on a physical level, you will have to reserve 20 blocks for each partition…

Create separate UBI volumes does not have that draw back: UBI reserves blocks for bad block management for the whole device, and can remap them dynamically between UBI volumes. However, in case there is a bug in the UBI layer, it could be that one file system corrupts the other instance… I would not assume that since UBI has been tested quite extensively.

Creating a UBI volume can be done in Linux or U-Boot. Our update/write scripts use the U-Boot ubi create command (see flash_blk.scr). You can alter those scripts, or create the partitions manually. However, to change volume layout, you first need to delete the existing volumes using ubi remove. E.g. this create a data partition of 32MiB and allocates the rest to a root partition and write the root file system to it:

run setupdate
run prepare_ubi
ubi remove rootfs
ubi create data 0x2000000
ubi create rootfs 0 dynamic
run update_rootfs

To make sure the volume gets mounted on every boot, extend /etc/fstab a line like this:

ubi0:data            /mnt                 ubifs      defaults

You said that “Just creating a second file system instance won’t change the behavior of the file system”. By that do you mean I should create separate volumes and also use the “atomic file update” technique ?
Won’t “creating separate volumes” ensure that if one volume(data/configuration) gets corrupted due to unclean shutdowns during file write operations, it won’t affect any other volume and all other volumes will be mounted properly?
I am planning to create separate volumes just to ensure this.
And then in the separate data/configuration volume, I will use “atomic file update” technique as you mentioned.
Is the combination of these two the best way to protect my data and the overall module from corruption?
Kindly suggest the best way.

Also, I can consider the option of using SD-Card. Though, the size of data that I need to write isn’t much (~50MB) but if other solutions don’t work, I will have to resort to SD-card only.
So please suggest the best solution.

Can you write up a full example? I think you skipped the mkimage step.

see https://www.denx.de/wiki/DULG/UBootScripts

From the steps above, how do you modify the scr file? I get this for output on a VF50

===== removing ubi =====
Error, no UBI device/partition selected!
===== create data partition 0x2000000 =====
Error, no UBI device/partition selected!
===== creating rootfs with 0 and dynamic =====
Error, no UBI device/partition selected!
===== updating rootfs =====

Script is: echo '===== removing ubi ====='
ubi remove rootfs
echo '===== create data partition 0x2000000 ====='
ubi create data 0x2000000
echo '===== creating rootfs with 0 and dynamic ====='
ubi create rootfs 0 dynamic
echo '===== updating rootfs ====='
run update_rootfs
echo '===== normal returns with update to u-boot ====='

It seems that the UBI partition has not been selected. This is usually done as part of the run prepare_ubi command, or you can just explicitly attach the UBI partition using ubi part ubi

What are you trying to achieve exactly?

We are trying to create a separate partition for data - we have been experiencing corruption which we believe is on power down. I have modified the colibri_vf.h and gotten a partition which I can see with mtdparts, but am struggling to mount it. We are currently on V2.4 and Qt 4.8; upgrading to V2.7 is on our list with this. Going to Qt 5.x (maybe 5.4 to stay consistent with partner companies) is next. Here is more of the script:

setenv create_bcb 'nand erase.part vf-bcb && writebcb 0x20000 0x20000'
setenv update_configblock 'tftpboot ${loadaddr} ${board_name}/configblock.bin && nand write ${loadaddr} 0x800 ${filesize}'

setenv prepare_rootfs 'ubi part ubi && ubi check rootfs || ubi create rootfs'

setenv update_uboot 'tftpboot ${loadaddr} ${board_name}/u-boot-nand.imx && nand erase.part u-boot && nand erase.part u-boot-env && nand write ${loadaddr} u-boot'
setenv update_rootfs 'tftpboot ${loadaddr} ${board_name}/ubifs.img && run prepare_rootfs && ubi write ${loadaddr} rootfs ${filesize}'

#try putting this here
echo '===== removing ubi ====='
ubi remove rootfs
echo '===== create data partition 0x2000000 ====='
ubi create data 0x2000000
echo '===== creating rootfs with 0 and dynamic ====='
ubi create rootfs 0 dynamic
echo '===== updating rootfs ====='
run update_rootfs
echo '===== normal returns with update to u-boot ====='
setenv update_new 'run update_uboot; run update_rootfs; reset'

The in the original answer are more a starting point and should work when manually entered in a U-Boot shell.

To create a flash script which always creates this partition you have to alter the flash_blk.scr file. E.g. by prepending the data partition creation in the prepare_rootfs variable.

However, prepare_rootfs is only executed when creating the UBI the first time. In development you will have to manually erase the ubi partition (e.g. nand erase.part ubi) so that the prepare_ubi step will actually recreate the UBI volumes.