File operation issue?

Hi,

We are using Colibri Vf61 with Qt and linux 2.7 for our application development. We are saving some parameters as Binary file and Operation log as Text File. We are facing following issue:

Randomly the File size of above mentioned binary file and Text file changes to zero on ‘power off’ and ‘power on’ of the system (without using shutdown or reboot). This scenario does not happen when we execute a shutdown procedure or reboot procedure. After a successful file write operation we had verified the size as valid,but on turn off and restart the size will be shown as zero and data will be lost.

The Attached File contains the Code we use to write the Binary File link text.

Please suggest a resolution for this issue,We feel some how the OS is removing the File data that is written from User Space.

Hi @aneesh.kn

Could you provide a complete application which is writing data to a file? Where are you saving this file? How often are you writing to the file?

Randomly the File size of above mentioned binary file and Text file changes to zero on ‘power off’ and ‘power on’ of the system (without using shutdown or reboot).
This can happen, when you poweroff the module without any controlled procedure. You can have data loss.

This scenario does not happen when we execute a shutdown procedure or reboot procedure.
By doing a software shutdown and reboot, the kernel takes take care to finish all the write operations before shutting down the system.

After a successful file write operation we had verified the size as valid,but on turn off and restart the size will be shown as zero and data will be lost.
How did you turn off? Where did you saved the data?

  1. The Writing process is done on press of save button on ui or after completion of certain application specific functions. Even if we provide the application without proper mechanical and electronics hardware it will not be operable
  2. The frequency of writing may vary from 6 seconds to 1 hour
  3. Is this issue observed expected?
  4. We have a power button in our equipment, Turnoff means making power button off without going through Shutdown operation.
  5. Will this be avoided if we use sql database to save Data?
  6. We cannot include a shutdown scenario as part of our application.
  7. Because our application equipment will be used in scenarios were no back up power is present.
  8. File size is seen proper before Turning off Only at restart we observe the size reduce to zero.

Please suggest a resolution

Regards

Dear @aneesh.kn, as Jaski states, unexpected shutdowns may lead to data loss. However, this looks like more an issue from the application and its file handling than OS file handling issues. At a controlled shutdown scenario, the processes (and its opened file descriptors, among other stuff) are handled in order to avoid unexpected data loss.

Do you have the same issue if you repeat the process with L3afPad, the included text editor in the image?
Have you tried using regular C functions instead of Qt functions?
Are you making sure the file is correctly saved and closed when you unexpectedly turn off the module?

  1. I Created a text file with L3afPad in desktop. Typed some contents . Re opened the text file to verify data.Powered off the machine and powered on. After power on The file size reduced to zero

  2. We are facing same issue for C functions also, we were using c functions for Operation log

  3. During power off we are not doing any file Operations.

Any updates?

This is a very typical behavior or a Linux file system. Data (file contents) are not journaled whereas metadata (like file creation) is. When you create a file, the file creation operation is written to the journal immediately, whereas data get stored in cache. When you turn off your system during this time, the file system remembers that you created a file, but not its conntent.

You can use the sync command to ask the operating system to write everything to the underlying storage device.

In your program you can call fdatasync on the file explicitly to make sure data is written out to the storage device.

If you want to make sure that you have either the new or the old file (and data) you need to use a little trick: First, write the new data to a temporary file in the same folder (typically a hidden file starting with a . is used). Then use fsync on this file to make sure the data (content) of the file are written to flash. In the end rename the newly written file to the file you are trying to replace. The rename system call makes sure that the target file will atomically replaced. See also the man page of rename(2).

See also this blog post: Don’t fear the fsync!

Dear Stefan,
Thanks for update.
1. We will do the process that is suggested and will get back
2. Can you clarify if this behaviour is as expected. If not is there something we should do in image build level to resolve?

Regards,

Dear Stefan, Thanks for update.
You are welcome.

We will do the process that is suggested and will get back.
Yeah, please do that.

Can you clarify if this behaviour is as expected. If not is there something we should do in image build level to resolve?
As Stefan.tx stated, this behaviour is normal for Linux. You cannot change this by doing changes to the image.

Best regards