WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Files unreadable for some reason.  (Read 14686 times)

Offline neonix

  • Wiki Author
  • Sr. Member
  • *****
  • Posts: 392
Re: Files unreadable for some reason.
« Reply #15 on: July 08, 2019, 09:32:36 AM »
I have this kind of problems all the time, with my memory cards and pendrives. Why? Because crappy Chinese memory card readers, bad quality USB cables and the most important, it's not recommended to use journaling with flash technology. That's why I run my TC in LiveCD mode. If I want to archive somethings I'll copy it to pendrive.

There are some solutions, you can use NAS as a hard disk or force TC to do file system check from time to time. Other solution is to create 2 partitions, remaster TC that all data will be loaded from partition 1 by bootloader and after loading, don't use 1st partition. In this way partition 1 always by use as read-only and there is no risk because it won't be mounted and there will be not chance to corruption.

I destroyed my Debian/Ubuntu installation many times (no jurnaling). I use TC on memory card since 2009 and it corrupt many times. But since I load remastered version from the bootloader, I don't have any problems.
« Last Edit: July 08, 2019, 09:54:05 AM by neonix »

Offline atomant

  • Newbie
  • *
  • Posts: 30
Re: Files unreadable for some reason.
« Reply #16 on: July 08, 2019, 10:29:27 AM »
I just pull the plug, so to say. As far as i know corruption of the SD card should only occur if it is being written to when the power is removed. To my knowledge, this isn't happening (but of course i could be wrong). I'm using this system to play music from the internet, it is running YouTubeDL (python script), omxplayer and some self-written software to control what's happening. True, i'm not completely sure if either python or the player won't start writing files to the SD card but i don't see any sign of this (i see no remnant files lying around). My guess is that if any of these has the need to write some temporary files, it would do so to /tmp which is on the memory drive. Of course, this is a big guess again.
|-|4\/3 4 |\|1(3 |)4\|

Offline coreplayer2

  • Hero Member
  • *****
  • Posts: 3020
Re: Files unreadable for some reason.
« Reply #17 on: July 08, 2019, 12:17:08 PM »
Not all SD cards are created equal, some are of really dubious quality.  So try a high quality branded SD card

Offline atomant

  • Newbie
  • *
  • Posts: 30
Re: Files unreadable for some reason.
« Reply #18 on: July 08, 2019, 12:23:03 PM »
Not all SD cards are created equal, some are of really dubious quality.  So try a high quality branded SD card
On paper it's a SanDisk...do you have any recommendations perhaps?
|-|4\/3 4 |\|1(3 |)4\|

Offline coreplayer2

  • Hero Member
  • *****
  • Posts: 3020
Re: Files unreadable for some reason.
« Reply #19 on: July 08, 2019, 12:24:45 PM »
I would recommend a SanDisk without hesitation, but are sure it's an original?

Could be defective in any case

Offline atomant

  • Newbie
  • *
  • Posts: 30
Re: Files unreadable for some reason.
« Reply #20 on: July 08, 2019, 12:46:27 PM »
I would recommend a SanDisk without hesitation, but are sure it's an original?
No idea how i could be sure of that...didn't get it from some shady looking memory card dealer in a long raincoat.  :D As far as i can remember i bought it in a computer store, where they got it from i can't know...
|-|4\/3 4 |\|1(3 |)4\|

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11044
Re: Files unreadable for some reason.
« Reply #21 on: July 08, 2019, 01:10:03 PM »
Could there be a voltage spike when you pull power?

Internal, delayed block ordering could result from a write (so the SD card could be writing several seconds after Linux says the write is done), but you say there were no writes.
The only barriers that can stop you are the ones you create yourself.

Offline neonix

  • Wiki Author
  • Sr. Member
  • *****
  • Posts: 392
Re: Files unreadable for some reason.
« Reply #22 on: July 08, 2019, 01:37:11 PM »
SanDisk is the best brand. File system corruption occurs even on HDD (when you don't use journaling). You can use journaling on your flash device (with the risk it degenerate faster) or you can configure your system in way that everything will be in RAM and all devices will be unmounted.

Offline Greg Erskine

  • Sr. Member
  • ****
  • Posts: 404
Re: Files unreadable for some reason.
« Reply #23 on: July 08, 2019, 06:01:37 PM »
piCorePlayer based on piCore has hundreds, maybe thousands of installations. I prepared a "copy2fs" solution waiting for corruptions of SD cards. It never happened. We might be lucky that our application does little writing to the SD card.

When you use "copy2fs" you can set it up to dismount all disks. I actually removed the SD card.

I've been using piCore based RPi's for over 5 years and I can't remember having a random SD card corruption! Lucky?

I use mainly SanDisk.

Offline PDP-8

  • Hero Member
  • *****
  • Posts: 915
Re: Files unreadable for some reason.
« Reply #24 on: July 08, 2019, 06:08:16 PM »
Roberts designed TC to be nice to flash from the outset, so if you are getting corruption, there is a local problem.

The major issue (aside from counterfeits) is write-amplification.  That is, many small files being written all the time.  Wear leveling helps.  To help wear-leveling perform the best, use 4MB partition boundaries.

Even with TC, use a big card to have redundant cells.  My recommendation is to go no smaller than 32gb, even if you are only using a small fraction of that with the o/s and your files.

And if overall speed is not a concern, there are so-called "high endurance" cards available designed for long term writing (think dashcams, body-cams, surveillance cams), which tradeoff a bit of speed for more endurance.

One such is the Samsung "Pro Endurance".  Markets itself as "High Endurance for Video Monitoring".

Use of ext-2:  these days the wear-leveling techniques are good enough that you *can* use journaling filesystems without worrying too much.  Especially with an o/s like TinyCore.

But yeah, if one is using a 2gb card from 1998, then maybe ext2 is a good idea.  Thing is, this "advice" from the past can live well beyond its expiration date if one stays up with technology.  And it provides good fodder for click-bait sales of old stock, that marketers are well aware of.
That's a UNIX book! - cool  -- Garth

Offline PDP-8

  • Hero Member
  • *****
  • Posts: 915
Re: Files unreadable for some reason.
« Reply #25 on: July 08, 2019, 06:24:06 PM »
Want peace-of-mind?  Test with the F3 util:

Computer stores and online sales that seem reputable may be vulnerable to man-in-the-middle counterfeit substitutions.

Check yours - perhaps a single one if purchasing in a batch:

https://fight-flash-fraud.readthedocs.io/en/stable/

Here's are some examples.  Many on web video as well:

https://fight-flash-fraud.readthedocs.io/en/stable/introduction.html#examples

You can get it for compiling right here:  Note that most linux users will want to read the entirety of the above docs if they want the "extra applications" to work.

https://github.com/AltraMayor/f3/releases

Looks like F3 would make a *great* candidate for a F3.tcz package!
« Last Edit: July 08, 2019, 06:32:18 PM by PDP-8 »
That's a UNIX book! - cool  -- Garth

Offline atomant

  • Newbie
  • *
  • Posts: 30
Re: Files unreadable for some reason.
« Reply #26 on: July 09, 2019, 09:35:16 AM »
Could there be a voltage spike when you pull power?

Internal, delayed block ordering could result from a write (so the SD card could be writing several seconds after Linux says the write is done), but you say there were no writes.
Not that i can see but of course it's not impossible, can't see what happens inside the rPi's hardware. I see no spike on the power line with the multimeter.

To the others, card is 16Gb and i'm using Ext4 on the partition that's experiencing the problems. Theoretically, in normal operation, no writes to the card should ever occur, but as said before, i do not know this for sure.
|-|4\/3 4 |\|1(3 |)4\|

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11630
Re: Files unreadable for some reason.
« Reply #27 on: July 09, 2019, 10:10:57 AM »
Hi atomant
... I see no spike on the power line with the multimeter. ...
If it's a digital multimeter, you probably won't see a spike, its response is too slow. If it's an analog meter, you might be able to
spot something if it's blatant enough.
Try to isolate the problem. Start with Juanitos advice
Do you use "sync" and then "sudo exitcheck.sh" to shutdown piCore or just pull the power cord out?

If the corruption continues, make sure cables are as short as possible. Try another power supply.
Doesn't the Pi have a power status LED that lights when the power rail is questionable?

If the corruption stops, then it doesn't like you just cutting the power.

Offline atomant

  • Newbie
  • *
  • Posts: 30
Re: Files unreadable for some reason.
« Reply #28 on: July 11, 2019, 04:17:12 AM »
I hope it's not the power-cut because creating a "proper shutdown" is a pain in the ass...
|-|4\/3 4 |\|1(3 |)4\|

Offline atomant

  • Newbie
  • *
  • Posts: 30
Re: Files unreadable for some reason.
« Reply #29 on: July 11, 2019, 04:47:51 AM »
FYI: I used iotop to see if any writes occur during operation, apparently not.
|-|4\/3 4 |\|1(3 |)4\|