Tiny Core Linux

Tiny Core Base => TCB Q&A Forum => Topic started by: bitpusher on October 15, 2013, 11:16:42 AM

Title: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 15, 2013, 11:16:42 AM
Hi there,

I am using TinyCore-5.0.1 and set up my TC according to the article from this post (http://forum.tinycorelinux.net/index.php/topic,15579.0.html), minus the kernel changes (until now).
My understanding was that this way I would be able to use UTF-8 for all filesystems with the exception of the boot-device.

However this is not (really) working for a network drive (all tested in urxvt, actually using bash):

- Umlaute in directory/file-names are displayed as question marks
- Creating/copying/deleting dircetories/files with Umlaute in their names works, as long as the complete name is typed
- Using wildcards with any command does not find files when the actual name contains Umlaute
- Strangely however command line completion with <TAB> works also for Umlaute

# Network share is mounted like this:

sudo mount -t cifs //diskstation/data /mnt/data -o username=****,password=****,iocharset=utf8,file_mode=0777,dir_mode=0777

# Locale seems to be OK:

tc@box:/mnt/sda1$ locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

tc@box:/mnt/sda1$ locale -a
C
de_DE.utf8
en_US.utf8
POSIX

# However when I look at /proc/mounts:
...
//diskstation/data /mnt/data cifs rw,mand,relatime,vers=1.0,sec=ntlmssp,cache=strict,unc=\134\134diskstation
\134daten,username=****,domain=DISKSTATION,uid=0,noforceuid,gid=0,noforcegid,addr=172.17.1.250,unix,
posixpaths,serverino,acl,rsize=1048576,wsize=1048576,actimeo=1 0 0

it does not look like "iocharset=utf8" is actually used?
But then again, why does it partly work?
Addition: In XWindows-programs e.g. the file dialog of the editor it does not work either, only there I get wrong characters instead of question marks.

Thanks in advance.


Title: Re: UTF-8 for filesystems still not working for me...
Post by: curaga on October 15, 2013, 12:48:11 PM
FLTK 1.1 apps (like editor), and busybox applets (like cp, mv, ls) do not support UTF-8.  Have you installed the GNU versions (coreutils and util-linux)?
Title: Re: UTF-8 for filesystems still not working for me...
Post by: theYinYeti on October 16, 2013, 04:27:14 AM
Hi bitpusher! Been there done that ;)
You may be interested in this other post:
http://yeti.selfip.net/cms/index.php/post/2013/05/31/Lightweight-Windows-Network-Neighbourhood-for-TinyCore-Linux

The thing is: cifs tools seem to have a strange way of handling locales… I managed to have a satisfying mount with a command like this:
Code: [Select]
export LANG="fr_FR.iso885915@euro"
sudo mount.cifs //server/share /path/to/mount_point -o iocharset=utf8
From my tests, the only working iocharset is "utf8" for the mount.cifs command; the only way to actually change the locale to use is with the LANG environment variable.

I hope this helps :)
Title: Re: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 16, 2013, 07:21:51 AM
FLTK 1.1 apps (like editor), and busybox applets (like cp, mv, ls) do not support UTF-8.  Have you installed the GNU versions (coreutils and util-linux)?

Thanks for the quick answer - and duh, I completely overlooked that things like ls are not internal to the shell.

Thus - as of yesterday (coreutils.tcz & util-linux not installed):

non ASCII characters in file/directory names indicated as question marks, Umlaute obviously not working for ls, cd and so on.
But "ls" gives color coded output.
tc@box:~$ which ls
/bin/ls

coreutils.tcz installed (via load app locally & open new urxvt _or_ OnBoot maintenance & Reboot):

Umlaute are now indicated correctly in file/directory names, Umlaute working for ls, cd and so on.
However "ls" does no longer use color coding for output.

tc@box:~$ which ls
/usr/local/bin/ls

tc@box:~$ which mount                 (reason for this, see below)
/bin/mount

additionally installed util_linux.tcz(via load app locally _or_ OnBoot maintenance & Reboot):

Mounting network shares no longer works:

tc@box:~$ sudo mount -t cifs //diskstation/data /mnt/data -o user=****,pass=****
mount: wrong fs type, bad option, bad superblock on //diskstation/daten,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)

tc@box:~$ which mount
/usr/local/bin/mount

Therefore I wonder for what I would really need util_linux.tcz for?
And is there a way to have the best of both worlds ls wise - or maybe I just have to tell the "new" ls to use color coding?









Title: Re: UTF-8 for filesystems still not working for me...
Post by: theYinYeti on October 16, 2013, 07:29:03 AM
ls --color
Title: Re: UTF-8 for filesystems still not working for me...
Post by: curaga on October 16, 2013, 07:47:41 AM
With GNU mount, you also need cifsutils to mount cifs filesystems, as the message mentions.
Title: Re: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 16, 2013, 09:15:20 AM
ls --color

Ah, yes. Could have found that out myself.

However when I tried to duplicate the behavior of TC's "original" ls (just use color as default) I ran into a problem:

The all knowing internet tells me to put an alias like alias ls="ls --color=always" into /home/tc/.bashrc - this works,
but only after I do a source .bashrc - right after reboot nobody seems to know about this alias.

I also tried putting source /home/tc/.bashrc into /home/tc/.X.d/shell (where I already have export SHELL=/usr/local/bin/bash) to no avail (but maybe this is the completely wrong approach anyway).
Title: Re: UTF-8 for filesystems still not working for me...
Post by: theYinYeti on October 16, 2013, 09:18:56 AM
Your approach is not wrong (either works) but TC is special in this regard: you have to ensure the file you create or modify is saved by the backup system. In other words, you have to put the file path (without the initial /) in the /opt/.filetool.lst file.
Title: Re: UTF-8 for filesystems still not working for me...
Post by: Rich on October 16, 2013, 09:27:40 AM
Hi bitpusher
Code: [Select]
The all knowing internet tells me to put an alias like alias ls="ls --color=always" into /home/tc/.bashrc ... I would have though you'd want to put that into  /home/tc/.profile
Title: Re: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 16, 2013, 09:43:51 AM
With GNU mount, you also need cifsutils to mount cifs filesystems, as the message mentions.

OK, this leaves me with two possibilities:

1. Keep TC's original mount which relies on filesystems.tcz and skip on util-linux.tcz
2. Install util-linux.tcz and then cifsutils.tcz to make GNU mount work for me - and also drop filesystems-3.8.10-tinycore.tcz from my list again.

I guess as long as I don't need any of the "other" filesystems covered by filesystems-3.8.10-tinycore.tcz 2. is the better option for me as it will give me the complete set of UTF-8-utilities on the command line , right?
Title: Re: UTF-8 for filesystems still not working for me...
Post by: Rich on October 16, 2013, 10:01:25 AM
Hi bitpusher
Isn't  filesystems  a dependency of  cifs-utils.tcz?
Title: Re: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 16, 2013, 11:04:37 AM
Hi Rich,
Code: [Select]
The all knowing internet tells me to put an alias like alias ls="ls --color=always" into /home/tc/.bashrc ... I would have though you'd want to put that into  /home/tc/.profile
I put the alias ls="ls --color=always" into /home/tc/.profile but this will not help with the color.
When I instead put source /home/tc/.bashrc into /home/tc/.profile this will work for the color but screws up the key mapping in the shell (home, end, Del not working).
However when I enter the source /home/tc/.bashrc on the command line it works like expected.
Confused. :o
Title: Re: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 16, 2013, 11:06:52 AM
Your approach is not wrong (either works) but TC is special in this regard: you have to ensure the file you create or modify is saved by the backup system. In other words, you have to put the file path (without the initial /) in the /opt/.filetool.lst file.

At the moment this is not my problem. Both my changes to /home/tc/.bashrc and /home/tc/.profile survive the boot - however the results are not quite what I intended - see other post.
Title: Re: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 16, 2013, 11:11:06 AM
Hi bitpusher
Isn't  filesystems  a dependency of  cifs-utils.tcz?

Hmm, this is what apps/dependencies and deletions/list dependencies tells me, too.
However when I go to apps/maintenance/OnBoot maintenance I see filesystems-3.8.10-tinycore.tcz on the left, not on the right as if it had not been/would not be loaded at boot.
Maybe this is just me not understanding the way things are displayed here?
Title: Re: UTF-8 for filesystems still not working for me...
Post by: gerald_clark on October 16, 2013, 11:45:02 AM
The onboot.lst only shows packages selected to load onboot.
It does not list their dependencies.
The packages are loaded by tce-load which loads the dependencies.
Title: Re: UTF-8 for filesystems still not working for me...
Post by: Rich on October 16, 2013, 03:54:08 PM
Hi bitpusher
Sorry, I meant  /home/tc/.ashrc. It contains some aliases including one for  ls
Title: Re: UTF-8 for filesystems still not working for me...
Post by: bitpusher on October 17, 2013, 11:53:22 AM
Hi bitpusher
Sorry, I meant  /home/tc/.ashrc. It contains some aliases including one for  ls

Yes, this finally did the trick.
Changed alias ls=“ls -p“ in to alias ls=“ls -p --color“ in /home/tc/.ashrc and it works!
Now if only somebody would teach the shell how not to lose cursor position when there are non ASCII characters present...

@gerald_clark
OK, understood why it is not in the OnBoot list.
Maybe it would be good if the "apps" thing would be a bit more communicative to the user - specially for newbies like me.
Maybe just a few bubbles/hints on the controls...
Title: Re: UTF-8 for filesystems still not working for me...
Post by: gerald_clark on October 17, 2013, 02:43:32 PM
Click the "Depends" tab when browsing apps and you will see the whole deps tree.
Why in the world would you want to clutter up the onboot panel with deps?  It would obscure the info you are trying to manage ( The apps you want to load  ).
The "Dependencies and Deletions" panel is very communicative if you want to explore your apps in more detail.