WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: "readonly mounted SD could still get corrupted" - why?  (Read 5070 times)

Offline xpector

  • Newbie
  • *
  • Posts: 16
"readonly mounted SD could still get corrupted" - why?
« on: August 30, 2017, 10:13:30 AM »
I read from time to time warnings that readonly mounted SD could still get corrupted (e.g. http://forum.tinycorelinux.net/index.php/topic,21181.0/topicseen.html#subject_132508, http://forum.tinycorelinux.net/index.php?topic=20808.0#subject_129917).

Searching web for possible reasons I only found so far (https://linux.die.net/man/8/mount): "...the system may still write to the device. For example, Ext3 or ext4 will replay its journal if the filesystem is dirty. To prevent this kind of write access, you may want to "ro,noload" or set the block device to read-only mode, see command blockdev(8 )"

Would changing rebuildfstab and remaster to have it always mounted ro,noload help then? Do you know more reasons for "read-only corruption" than abovementioned?

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11048
Re: "readonly mounted SD could still get corrupted" - why?
« Reply #1 on: August 30, 2017, 01:28:39 PM »
It's nothing to do with Linux, but how the SD card works internally. If the power is cut when it's arranging its internal affairs, it may corrupt itself, regardless of the software/OS.
The only barriers that can stop you are the ones you create yourself.


Offline seandepagnier

  • Newbie
  • *
  • Posts: 14
Re: "readonly mounted SD could still get corrupted" - why?
« Reply #3 on: August 30, 2017, 09:21:23 PM »
I managed to short a raspberry by accident and the sdcard still works, but any data written to it, is as if it is not there.  If you create a file, it is lost when the card is remounted.

I'm not  sure how to reproduce this, but there might be a way with some sdcards to "lock" them by applying high voltage or negative voltage or something, so they can never write again.

Offline patrikg

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 718
Re: "readonly mounted SD could still get corrupted" - why?
« Reply #4 on: August 31, 2017, 03:40:50 AM »
Sorry to hear that you short a raspberry by accident and the sdcard, to may suggest a solution of working a lot with raspberry pi and
compiling and so on.. Just try to start a nfs server and boot up the tiny core and root fs from that.

I have accomplished that.. and it's work great, just little slower.

I am not at home right now, i have written some text to make to remember all things that i have to do.

Like booting up the raspberry pi first with raspberry pi firmware.. on sd card... and then booting up to uboot and then boot to nfs.
I have not the latest raspberry... if you have you can skip the first thing... just boot from network.


Offline xpector

  • Newbie
  • *
  • Posts: 16
TMP_WRITE_PROTECT bit of SD card
« Reply #5 on: August 31, 2017, 04:48:38 AM »
It's nothing to do with Linux, but how the SD card works internally. If the power is cut when it's arranging its internal affairs, it may corrupt itself, regardless of the software/OS.
Do you think that setting TMP_WRITE_PROTECT could help?
https://github.com/BertoldVdb/sdtool
https://www.seanet.com/~karllunt/sdlocker.html

If a wrong power sequence (like a sudden power off) can drive the card's controller crazy (what happened to the old good brown-out detection?), I'd expect that the spec tell how to do it right.
SD "Physical Layer Simplified Specification", Chapter 6.4.1.5 Power Down and Power Cycle https://www.sdcard.org/downloads/pls/
Assuming that Raspi conforms to SD specification and also assuming for a moment that cards conform, too, would it be logical to trust TMP_WRITE_PROTECT bit?

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11048
Re: "readonly mounted SD could still get corrupted" - why?
« Reply #6 on: August 31, 2017, 12:49:36 PM »
I don't think TMP_WRITE_PROTECT would help; it would just block the OS from writing to the card. It would not protect against a shoddy controller itself.
The only barriers that can stop you are the ones you create yourself.

Offline tehfink

  • Newbie
  • *
  • Posts: 11
Re: "readonly mounted SD could still get corrupted" - why?
« Reply #7 on: September 04, 2017, 02:02:41 PM »
I am not at home right now, i have written some text to make to remember all things that i have to do.

Hi patrikg,
I'd be very interested in your notes, if you have a chance to post them here.
Thanks!

Offline patrikg

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 718
Re: "readonly mounted SD could still get corrupted" - why?
« Reply #8 on: September 04, 2017, 03:48:05 PM »
Sorry for this badly notes... but here some start, i have also done some makefile, to do some of this things auto. I am not any pro of programming and do this stuff's.
But if any one can use some of my stuff and get some ideas and get it working, i will be very happy.

My thoughts maybe learning to do some stuff. And maybe i get some hints from the masters that make this possible. And don't forget to celebrating Bela that made tc for Raspberry Pi.  And I am to little to busy right now, to make this for the latest picore. And I am maybe later on thinking of making chipcore. But i think i am need some help to do so.. and where i am to start.

Here my notes file

Code: (bash) [Select]
My config
PC with Arch linux ip 10.42.0.1 user patrik
Raspberry Pi Model B 256M.

On the PC
------------------------------------------
Prep
On arch on pc.
Install uboot-tools, nfs
pacman -S uboot-tools  # To make uboot images and env.
pacman -S nfs-utils    # To start nfs server.
pacman -S dosfstools   # To format partision for msdos and make msdos label
pacman -S unzip        # To unzip files.
pacman -S util-linux   # To losetup command

suco systemctl enable nfs-server.service
sudo systemctl enable rpcbind.service
sudo systemctl start nfs-server.service
sudo systemctl start rpcbind.service

Edit /etc/conf.d/nfs-server.conf
Add -V 2 to support Ver 2 of nfs to support uboot.
NFSD_OPTS='-V 2'

Make my picore project work directory.
mkdir picore

I set all privileges on that....large security risk...but for easyness for now then mount correct way.
sudo chmod 777 picore

Add share exports to /etc/exports add follow line.
/home/patrik/picore *(rw,sync,no_root_squash,no_subtree_check)

Update nfs shares.
sudo exportfs -arv

ON PI.
First you need uboot.bin so compile that on Raspberry PI.
---------------------------------------------------------
Edit config.txt to make extra ram and enable uart.
mount /mnt/mmcblk0p1/
sudo nano /mnt/mmcblk0p1/config.txt
gpu_mem=16
enable_uart=1

sudo reboot

To make uboot on pi you need some programs, load this, expand part 2 first if you running tc from scratch.
tce-load -wi compile-essentials.tcz binutils.tcz git.tcz nano.tcz coreutils.tcz bc.tcz
tce-load -wi compiletc.tcz coreutils.tcz binutils.tcz git.tcz nano.tcz bc.tcz

cd /mnt/mmcblk0p2/tce

git clone git://git.denx.de/u-boot.git
cd u-boot
make rpi_defconfig
make

Copy the file to the PC.
scp u-boot.bin patrik@10.42.0.1:/home/patrik/picore/uboot.bin

ON PC
Change to project dir.
cd /home/patrik/picore

Get the picore zip file.
wget http://tinycorelinux.net/7.x/armv6/releases/RPi/piCore-7.0.zip

Unzip the picore zip file.
unzip piCore-7.0.zip

Make mountpoint directory img.
mkdir img

Get start offset of first vfat partistion in img.
fdisk -l piCore-7.0.img
Device          Boot Start   End Sectors  Size Id Type
piCore-7.0.img1       8192 69631   61440   30M  c W95 FAT32 (LBA)
piCore-7.0.img2      69648 93119   23472 11.5M 83 Linux

Sectorsize=512 bytes
Startsector first part=8192

8192*512=4194304

Mount the vfat part to img directory
sudo mount -t vfat -o ro,loop,offset=$((8192*512)) piCore-7.0.img img
boot
Copy the kernel initrd cmdline files.
cp ./img/7.0.gz .
cp ./img/kernel4113.img .
cp ./img/cmdline.txt .

Unmount the img.
sudo umount img/

Make uboot bootable image files with kernel4113.img and 7.0.gz.
mkimage -A arm -O linux -T kernel -C none -a 0x01000000 -n Kernel -d kernel4113.img kernel.img
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x02100000 -n RAMDisk -d 7.0.gz ramdisk.img

Make sd image 25meg is enuff.
dd if=/dev/zero of=image.img bs=1M count=25

parted --script -a none -s image.img mktable msdos mkpart primary fat32 0 10M mkpart primary ext4 10M 100%

sudo losetup -o $((512*1)) /dev/loop0 image.img
mkdir sdimg
sudo mkfs.vfat /dev/loop0
sudo dosfslabel /dev/loop0 RASPBOOT
sudo mount /dev/loop0 sdimg

Get latest firmware unzip only boot.

svn checkout --config-option config:miscellany:use-commit-times=yes https://github.com/raspberrypi/firmware/trunk/boot .
find . -type d -name .svn -exec rm -rf {} \;
rm kernel.img
rm kernel7.img

wget https://github.com/raspberrypi/firmware/archive/master.zip
unzip -d . master.zip 'firmware-master/boot/*'
mv firmware-master/boot/ .
rmdir firmware-master/
rm ./boot/kernel.img
rm ./boot/kernel7.img
sudo cp -R ./boot/* sdimg/

Make uboot.env file.
Edit /etc/fw_env.config
uboot.env 0x0000 0x4000 0x4000

dd if=/dev/zero of=uboot.env bs=1K count=16

To run uboot-tools some problems with lock file solved like this.
sudo touch /var/lock/fw_printenv.lock
sudo chown patrik /var/lock/fw_printenv.lock

#Clear out default settings from env file
fw_printenv | cut -f1 -d'=' | fw_setenv -s -

fw_setenv stdin 'serial,usbkbd'
fw_setenv stdout 'serial,lcd'
fw_setenv stderr 'serial,lcd'
fw_setenv bootdelay '3'
fw_setenv bootcmd 'usb start;setexpr macaddr gsub ":" "" ${ethaddr};env set ethact sms0;env set autoload no;dhcp;nfs 0x02000000 ${serverip}:/home/patrik/picore/${macaddr}.img;source 0x02000000'

nfs \${fdt_addr_r} \${serverip}:\${rootpath}\${fdtfile}

Copy the u-boot.env file to sdimg
sudo cp uboot.env sdimg

Make very simple config.txt file
echo kernel=u-boot.bin > config.txt
echo gpu_mem=16 >> config.txt
echo enable_uart=1 >> config.txt
Copy the config.txt to sdimg
sudo cp config.txt sdimg

umount sdimg
sudo umount sdimg
sudo losetup -d /dev/loop0

copy image to sdcard my sd card is on /dev/sdd.
sudo dd if=image.img of=/dev/sdd bs=4M
sync

#Add in cmdline.txt for swedish keyboard and Europe/Stockholm time zon.
#tz=CET-1CEST,M3.5.0,M10.5.0/3
#kmap=qwerty/sv-latin1

Make bootscr.txt with bash script. run with macaddress.img as parameter
#!/bin/bash
boardargs="bcm2708.boardrev=0x2 bcm2708.serial=0x46d41c0b"
cmdlineargs=$(cat cmdline.txt | cut -f2- -d' ')
extraargs="tz=CET-1CEST,M3.5.0,M10.5.0/3 kmap=qwerty/sv-latin1 lang=sv_SE.UTF-8"
echo "env set cmdline '$boardargs $cmdlineargs $extraargs'" > bootscr.txt
echo "env set nfs_rampos 'env set kernel_addr_r \"0x01000000\";env set ramdisk_addr_r \"0x02100000\";env set fdt_addr_r \"0x00000100\"'" >> bootscr.txt
echo "env set nfs_filename 'env set kernelfile \"kernel.img\";env set ramdiskfile \"ramdisk.img\";env set fdtfile \"bcm2708-rpi-b.dtb\"'" >> bootscr.txt
echo "env set nfs_rootpath 'env set rootpath \"/home/patrik/picore/\"'" >> bootscr.txt
echo "env set nfs_bootargs 'env set bootargs \"\${cmdline} nfsmount=\${serverip}:\${rootpath} tce=nfs/tce\"'" >> bootscr.txt
echo "env set nfs_load 'nfs \${kernel_addr_r} \${serverip}:\${rootpath}\${kernelfile};nfs \${ramdisk_addr_r} \${serverip}:\${rootpath}\${ramdiskfile};nfs \${fdt_addr_r} \${serverip}:\${rootpath}\${fdtfile}'" >> bootscr.txt
echo "env set nfs_start 'run nfs_rampos nfs_filename nfs_rootpath nfs_bootargs nfs_load;bootm \${kernel_addr_r} \${ramdisk_addr_r} \${fdt_addr_r}'" >> bootscr.txt
echo "run nfs_start" >> bootscr.txt
mkimage -A arm -O linux -T script -C none -a 0x02000000 -n Script -d bootscr.txt $1

Here the makefile.

Code: (bash) [Select]
# Package need to do make.
# dosfstools
# unzip
# util-linux
# uboot-tools
WORKDIR=$(shell pwd)/
TEMPWORKDIR=$(WORKDIR)TEMP/

FIRMWAREURL=https://github.com/raspberrypi/firmware/archive/
FIRMWAREDIR=firmware-master/boot/
FIRMWARETARFILE=master.tar.gz

PICOREURL=http://tinycorelinux.net/8.x/armv6/releases/RPi/

PICOREZIPFILE=piCore-8.1.5.zip
PICOREIMGFILE=piCore-8.1.5.img
PICOREROOTFILE=8.1.5.gz
PICORECMDLINEFILE=cmdline.txt
PICOREKERNELFILE=kernel4439.img
PICORECONFIGFILE=config.txt

PARTFATBEGIN=$(shell LANG=C parted -s $(TEMPWORKDIR)$(PICOREIMGFILE) unit s print |  awk '/^Number/{p=1;next} p{p++} {if (p==2) printf("%i",$$2)}')
PARTFATEND=$(shell LANG=C parted -s $(TEMPWORKDIR)$(PICOREIMGFILE) unit s print |  awk '/^Number/{p=1;next} p{p++} {if (p==2) printf("%i",$$3)}')
PARTEXTBEGIN=$(shell LANG=C parted -s $(TEMPWORKDIR)$(PICOREIMGFILE) unit s print |  awk '/^Number/{p=1;next} p{p++} {if (p==3) printf("%i",$$2)}')
PARTEXTEND=$(shell LANG=C parted -s $(TEMPWORKDIR)$(PICOREIMGFILE) unit s print |  awk '/^Number/{p=1;next} p{p++} {if (p==3) printf("%i",$$3)}')
PARTFATBEGINOFFSET=$(shell echo $$(($(PARTFATBEGIN)*$(SECTORSIZE))))
PARTEXTBEGINOFFSET=$(shell echo $$(($(PARTEXTBEGIN)*$(SECTORSIZE))))
PARTFATENDOFFSET=$(shell echo $$(($(PARTFATEND)*$(SECTORSIZE))))
PARTEXTENDOFFSET=$(shell echo $$(($(PARTEXTEND)*$(SECTORSIZE))))

PICORELOOPDEV=/dev/loop0
PICORELOOPDIR=loop0/
SDIMGLOOPDEV=/dev/loop1
SDIMGLOOPDIR=loop1/

SECTORSIZE=512
SDIMAGESIZE=25
MB=1048576
SDFATPART=10
SDFATPARTSIZE=$(shell echo $$(($(MB)*$(SDFATPART)/$(SECTORSIZE))))
FATLABEL=RASBBOOT

PARTSTARTOFFSET=1
EXT4START=$(shell echo $$(($(SDFATPARTSIZE)+$(PARTSTARTOFFSET))))
SDIMAGEFILE=new_image.img

all: help

getpicorezipfile:
wget $(PICOREURL)$(PICOREZIPFILE) -O $(TEMPWORKDIR)$(PICOREZIPFILE)

help:
@echo "1. make prep"
@echo "2. make getimage"
@echo "3. make help $(WORKDIR)"

mktempdir:
mkdir -p $(TEMPWORKDIR)

unzippicore:
unzip $(TEMPWORKDIR)$(PICOREZIPFILE) $(PICOREIMGFILE) -d $(TEMPWORKDIR)

mksdimage:
dd if=/dev/zero of=$(TEMPWORKDIR)$(SDIMAGEFILE) bs=1M count=$(SDIMAGESIZE)

partsdimage:
parted --script -a none $(TEMPWORKDIR)$(SDIMAGEFILE) unit s mktable msdos mkpart primary fat32 $(PARTSTARTOFFSET) $(SDFATPARTSIZE) mkpart primary ext4 $(EXT4START) 100%

picorelsetup:
sudo losetup -r -o $(PARTFATBEGINOFFSET) $(PICORELOOPDEV) $(TEMPWORKDIR)$(PICOREIMGFILE)

sdlsetup:
sudo losetup -o $(SECTORSIZE) $(SDIMGLOOPDEV) $(TEMPWORKDIR)$(SDIMAGEFILE)

formatsdfat:
sudo mkfs.vfat $(SDIMGLOOPDEV)

labelsdfat:
sudo dosfslabel $(SDIMGLOOPDEV) $(FATLABEL)

mkpicoreloopdir:
mkdir -p $(TEMPWORKDIR)$(PICORELOOPDIR)

mksdloopdir:
mkdir -p $(TEMPWORKDIR)$(SDIMGLOOPDIR)

rmpicoreloopdir:
rmdir $(TEMPWORKDIR)$(PICORELOOPDIR)

rmsdloopdir:
rmdir $(TEMPWORKDIR)$(SDIMGLOOPDIR)

mountpicoreloop:
sudo mount $(PICORELOOPDEV) $(TEMPWORKDIR)$(PICORELOOPDIR)

mountsdloop:
sudo mount $(SDIMGLOOPDEV) $(TEMPWORKDIR)$(SDIMGLOOPDIR)

umountpicoreloop:
sudo umount $(TEMPWORKDIR)$(PICORELOOPDIR)

umountsdloop:
sudo umount $(TEMPWORKDIR)$(SDIMGLOOPDIR)

picoreulsetup:
sudo losetup -d $(PICORELOOPDEV)

sdulsetup:
sudo losetup -d $(SDIMGLOOPDEV)

downloadfirmware:
wget $(FIRMWAREURL)$(FIRMWARETARFILE) -O - | tar -zxvf - -C $(TEMPWORKDIR) $(FIRMWAREDIR)

downloadpicore:
wget $(PICOREURL)$(PICOREZIPFILE) -O $(TEMPWORKDIR)$(PICOREZIPFILE)

removekernelfiles:
rm $(TEMPWORKDIR)$(FIRMWAREDIR)kernel.img
rm $(TEMPWORKDIR)$(FIRMWAREDIR)kernel7.img

movefirmware:
sudo mv -t $(TEMPWORKDIR)$(SDIMGLOOPDIR) $(TEMPWORKDIR)$(FIRMWAREDIR)*

mksdimgconfig:
sudo sh -c 'echo kernel=u-boot.bin > $(TEMPWORKDIR)$(SDIMGLOOPDIR)$(PICORECONFIGFILE)'
sudo sh -c 'echo gpu_mem=16 >> $(TEMPWORKDIR)$(SDIMGLOOPDIR)$(PICORECONFIGFILE)'
sudo sh -c 'echo enable_uart=1 >> $(TEMPWORKDIR)$(SDIMGLOOPDIR)$(PICORECONFIGFILE)'

showpicoreparts:
@echo "Part 1 Begin $(PARTFATBEGINOFFSET)"
@echo "Part 1 End   $(PARTFATENDOFFSET)"
@echo "Part 2 Begin $(PARTEXTBEGINOFFSET)"
@echo "Part 2 End   $(PARTEXTENDOFFSET)"

copypicorefiles:
sudo cp $(TEMPWORKDIR)$(PICORELOOPDIR)$(PICOREKERNELFILE) $(TEMPWORKDIR)$(SDIMGLOOPDIR)$(PICOREKERNELFILE)
sudo cp $(TEMPWORKDIR)$(PICORELOOPDIR)$(PICOREROOTFILE) $(TEMPWORKDIR)$(SDIMGLOOPDIR)$(PICOREROOTFILE)
sudo cp $(TEMPWORKDIR)$(PICORELOOPDIR)$(PICORECMDLINEFILE) $(TEMPWORKDIR)$(SDIMGLOOPDIR)$(PICORECMDLINEFILE)

sdtest: mktempdir mksdimage partsdimage sdlsetup formatsdfat labelsdfat mksdloopdir mountsdloop mksdimgconfig

usdtest: umountsdloop sdulsetup rmsdloopdir

pitest: unzippicore picorelsetup mkpicoreloopdir mountpicoreloop copypicorefiles

upitest: umountpicoreloop picoreulsetup rmpicoreloopdir

firmware: mktempdir downloadfirmware removekernelfiles mksdloopdir movefirmware mksdimgconfig

Offline patrikg

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 718
Re: "readonly mounted SD could still get corrupted" - why?
« Reply #9 on: September 09, 2017, 05:16:17 AM »
Can't edit the old thread, but add the new file.... also mounted my old hd and get the new file.

Here it comes:

Code: (bash) [Select]
#!/bin/bash
org_kernelfile="kernel4415.img"
org_ramdiskfile="8.0beta4.gz"
macfile="b827ebd41c0b.img"

boardargs="bcm2708.boardrev=0x2 bcm2708.serial=0x46d41c0b"
#cmdlineargs=$(cat cmdline.txt | cut -f2- -d' ')
cmdlineargs=$(cat cmdline.txt)
extraargs="tz=CET-1CEST,M3.5.0,M10.5.0/3 kmap=qwerty/sv-latin1 lang=sv_SE showapps"
kernelfile="kernel.img"
ramdiskfile="ramdisk.img"
fdtfile="bcm2708-rpi-b84.dtb"
kernel_addr_r="0x01000000"
ramdisk_addr_r="0x02100000"
fdt_addr_r="0x00000100"
rootpath="/home/patrik/picore/"
echo "env set cmdline '$cmdlineargs $extraargs $boardargs'" > bootscr.txt
echo "env set nfs_rampos 'env set kernel_addr_r \"$kernel_addr_r\";env set ramdisk_addr_r \"$ramdisk_addr_r\";env set fdt_addr_r \"$fdt_addr_r\"'" >> bootscr.txt
echo "env set nfs_filename 'env set kernelfile \"$kernelfile\";env set ramdiskfile \"$ramdiskfile\";env set fdtfile \"$fdtfile\"'" >> bootscr.txt
echo "env set nfs_rootpath 'env set rootpath \"$rootpath\"'" >> bootscr.txt
echo "env set nfs_bootargs 'env set bootargs \"\${cmdline} nfsmount=\${serverip}:\${rootpath} tce=nfs/tce84\"'" >> bootscr.txt
echo "env set nfs_load 'nfs \${kernel_addr_r} \${serverip}:\${rootpath}\${kernelfile};nfs \${ramdisk_addr_r} \${serverip}:\${rootpath}\${ramdiskfile}'" >> bootscr.txt
echo "env set nfs_start 'run nfs_rampos nfs_rootpath nfs_filename fdt_setup nfs_bootargs nfs_load;bootm \${kernel_addr_r} \${ramdisk_addr_r} \${fdt_addr_r}'" >> bootscr.txt
echo "env set fdt_setup 'nfs \${fdt_addr_r} \${serverip}:\${rootpath}\${fdtfile};fdt addr \${fdt_addr_r};fdt boardsetup;fdt resize;fdt chosen'" >> bootscr.txt
echo "run nfs_start" >> bootscr.txt
mkimage -A arm -O linux -T script -C none -a 0x02000000 -n Script -d bootscr.txt $macfile
mkimage -A arm -O linux -T kernel -C none -a 0x01000000 -n Kernel -d $org_kernelfile $kernelfile
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x02100000 -n RAMDisk -d $org_ramdiskfile $ramdiskfile