If you don’t have a battery powered RTC your board will get current date/time from a time server at boot. It may happen that, for connectivity issues, it can’t get the date and sets it to the default (1/1/2006).
If you have a network time server in your LAN you can use it for sync, instead of relying on internet connection.
You can disable the service, but letting it active with no network should not be an issue.
But how do you sync your time? Do you have an external RTC or you keep the internal one powered?
I use an external RTC (like the toradex evaluation board).
It seems that sometimes the RTC value is read wrong at the start,
but the value in RTC is it correct because if I turn off and re-turn on the board VF61 the value is correct.
Is it possible to read a second time the value of RTC, if the first time is read wrong?
Can you check the output on debug serial when the sync fails? Do you have pull-ups on the I2C lines? Do you have other devices connected?
Your application may check if date is 1/1/2006 and relaunch rtcsync.exe again, but it would be better to understand what’s wrong with the i2c connection.