Tiny Core Linux
Tiny Core Extensions => TCE Bugs => Topic started by: GNUser on October 26, 2023, 02:42:41 PM
-
I'm experimenting with wayland on TCL14 x86_64.
Any shell script that relies on the yes command causes the havoc terminal emulator (with default configuration) to be littered with "Broken pipe" warnings. This includes the tce-load shell script:
$ tce-load -wil bmon
bmon.tcz.dep OK
Downloading: libconfuse.tcz
Connecting to gnuser.ddns.net (24.0.182.246:80)
saving to 'libconfuse.tcz'
libconfuse.tcz 100% |********************************| 24576 0:00:00 ETA
'libconfuse.tcz' saved
libconfuse.tcz: OK
yes: Broken pipe
Downloading: bmon.tcz
Connecting to gnuser.ddns.net (24.0.182.246:80)
saving to 'bmon.tcz'
bmon.tcz 100% |********************************| 49152 0:00:00 ETA
'bmon.tcz' saved
bmon.tcz: OK
yes: Broken pipe
I've never seen this issue when using tce-load in X session, regardless of the terminal emulator being used.
In the case of tce-load the warnings are cosmetic. However, the same issue causes some of my shell scripts (those that use yes to give encryption password to bcrypt) to hang indefinitely (with upward tick in CPU usage and laptop's fans coming on).
Any ideas what could be causing this "Broken pipe" warning related to yes (and how to fix it)?
-
The same thing happens with the weston terminal.
-
Good to know--so it isn't anything specific to havoc. I'll keep digging.
-
Have you google it??
Can it be like this ??
https://stackoverflow.com/questions/20573282/hudson-yes-standard-output-broken-pipe (https://stackoverflow.com/questions/20573282/hudson-yes-standard-output-broken-pipe)
That the command that yes should send the yes to, have exit before ??
So what command is being used in the script to send yes to ??
Some text from the stackoverflow:
You could also report the bug against the component that runs the pipeline. The bug is in not restoring SIGPIPE to the default handler after the child is forked. It is what programs expect when they are run in a terminal on POSIX systems. Though I don't know whether there is a standard way to do it for a java-based program. jvm probably raises an exception for every write error so not-dying on SIGPIPE is not a problem for a java program.
So maybe it's the error comes from the shell that handling the signal correct.
-
That the command that yes should send the yes to, have exit before ??
Of course, it is the only way for yes to exit :-)
@GNUser, busybox yes dies silently. Are You sure that exactly busybox yes is envoked? Can You ensure this with the full path call?
-
Hi patrikg. Yes, I have done an internet search.
Hi jazzbiker. I don't have coreutils loaded. The only yes on my system is the busybox applet.
-
That stackoverflow link does look like the most likely explanation. IOW, it's a bug in those two terminals, havoc and weston terminal.
-
IOW, it's a bug in those two terminals, havoc and weston terminal.
Darn. In that case, I created this trivial wrapper script for yes, which does away with the errors when using tce-load:
#!/bin/sh
exec /usr/bin/yes "$@" 2>/dev/null
But that doesn't solve scripts where yes is needed multiple times in same pipeline. Even with the wrapper script, something like this:
yes "password" | bcrypt somefile
seems to enter an endless loop, driving CPU usage way up. Workaround for this situation is this kludge:
i=1
while true; do
echo "password"
[ $i -eq 2 ] && break
i=$(( $i + 1 ))
done | bcrypt somefile
If I figure out how to properly fix the bug in the terminal emulators, I'll post it here.
-
Why not file a bug report against weston or havoc?
-
Hi GNUser
... But that doesn't solve scripts where yes is needed multiple times in same pipeline. Even with the wrapper script, something like this:
yes "password" | bcrypt somefile
seems to enter an endless loop, driving CPU usage way up. ...
What if you do it like this:
echo -e "password\npassword" | bcrypt somefile
-
Why not file a bug report against weston or havoc?
Good idea. Will do.
What if you do it like this:
echo -e "password\npassword" | bcrypt somefile
Wow. Yes, that works and is much more elegant. Thank you!
-
Darn. In that case, I created this trivial wrapper script for yes, which does away with the errors when using tce-load:
#!/bin/sh
exec /usr/bin/yes "$@" 2>/dev/null
But that doesn't solve scripts where yes is needed multiple times in same pipeline. Even with the wrapper script, something like this:
yes "password" | bcrypt somefile
seems to enter an endless loop, driving CPU usage way up. Workaround for this situation is this kludge:
i=1
while true; do
echo "password"
[ $i -eq 2 ] && break
i=$(( $i + 1 ))
done | bcrypt somefile
If I figure out how to properly fix the bug in the terminal emulators, I'll post it here.
Maybe trap instruction in the wrapper will help? Something like
#!/bin/sh
trap 'exit 0' SIGPIPE
exec /usr/bin/yes "$@" 2>/dev/null
-
Hi GNUser
You are welcome. Thanks for confirming it worked.
If you feel like satisfying my curiosity, what happens if you do this:
echo -e "password\npassword\npassword" | bcrypt somefile
I expect another Broken pipe message, but reality may differ from my expectations.
-
Hi jazzbiker. No luck with the trap. But it's not surprising because with the exec statement, the script no longer exists in RAM when /usr/bin/yes executes.
So I also tried your suggestion without exec, like this:
#!/bin/sh
trap 'exit 0' SIGPIPE
/usr/bin/yes "$@" 2>/dev/null
But no luck with that, either.
Hi Rich. The superfluous passwords do not cause a Broken pipe message, surprisingly:
$ echo -e "tinycore\ntinycore\ntinycore\ntinycore\ntinycore\ntinycore" | bcrypt test.txt
Encryption key:
Again:
$
So the problem does not seem to be due to yes not knowing when to stop.
I'm happy with the wrapper script to fix the cosmetic issue when using tce-load and Rich's workaround for bcrypt. Thank you all :)
-
Hi GNUser,
I see that the problem is solved, so maybe if it is still interesting for You and You have a spare minute You may try not use the wrapper for yes but try declaration of the SIGPIPE handler inside tce-load script.
Edit: maybe better to set ignore for the SIGPIPE:
trap '' SIGPIPE
Hi Rich. The superfluous passwords do not cause a Broken pipe message, surprisingly:
$ echo -e "tinycore\ntinycore\ntinycore\ntinycore\ntinycore\ntinycore" | bcrypt test.txt
Encryption key:
Again:
$
Probably the pipe is buffered and the string is small enough to be placed in the buffer as a whole.
-
Hi jazzbiker. yes-related "Broken pipe" messages still appear if I add this to /usr/bin/tce-load:
trap '' SIGPIPE
I have submitted a bug report to the havoc developer.
-
Hi Juanito. havoc developer says the pipe of interest is created by the shell, not the terminal emulator (see here (https://github.com/ii8/havoc/issues/50)). Moving blame from terminal emulator to shell helps explain why havoc and weston-terminal are both affected.
I use havoc + busybox ash 99.9% of the time. I tried havoc + bash as a test and problem did not go away.
Since all terminal emulators and shells seem to be affected, I think the problem may have something to do with how weston or labwc is being started. Can you suggest a way to start the graphical session different from this (which is what weston.tcz.info suggests):
sudo -- seatd -g staff -n /run/seatd.sock & XDG_RUNTIME_DIR=/run/user/$(id -u) weston
?
-
Hi!
I've made some experiments trying to reproduce the observed by GNUser behaviour of yes. As it was described in the link by patrikg it looks rather counterintuitive.
I'm testing in TC14 x86 under Xorg.
tc@box:~$ yes | lua -e 'print(io.read())'
y
tc@box:~$ yes no | lua -e 'print(io.read())'
no
tc@box:~$ trap '' SIGPIPE # ignore signal
tc@box:~$ yes | lua -e 'print(io.read())'
y
yes: Broken pipe
tc@box:~$ yes no | lua -e 'print(io.read())'
no
here the pipeline hangs, interrupted by Ctrl-C
^Ctc@box:~$ trap - SIGPIPE # restore default handler
tc@box:~$ yes | lua -e 'print(io.read())'
y
tc@box:~$ yes no | lua -e 'print(io.read())'
no
tc@box:~$
So looks like havoc envokes the shell with SIGPIPE ignored. The cure may be restoring the default handler.
@GNUser, You may try adding
trap - SIGPIPE
to tce-load or to yes wrapper.
Looks like it is havoc's fail cause it leaves the shell without the SIGPIPE handler. I can't guess why the same behaviour is exhibited by another wayland terminal emulator, looks like coincidence.
You can notice the changes in the yes behaviour when it is supplied with the output string.
-
You can notice the changes in the yes behaviour when it is supplied with the output string.
In fact the difference in the yes behaviour is between single-char and multi-char output:
tc@box:~$ trap '' SIGPIPE
tc@box:~$ yes | lua -e 'print(io.read())'
y
yes: Broken pipe
tc@box:~$ yes n | lua -e 'print(io.read())'
n
yes: Broken pipe
tc@box:~$ yes no | lua -e 'print(io.read())'
no
^Ctc@box:~$ # hung up and interrupted by Ctrl-C
-
Hi jazzbiker
Nice find. Works under TC10 x86 too:
tc@E310:~$ busybox yes | head -n 3
y
y
y
tc@E310:~$ trap '' SIGPIPE
tc@E310:~$ busybox yes | head -n 3
y
y
y
yes: Broken pipe
tc@E310:~$ trap - SIGPIPE
tc@E310:~$ busybox yes | head -n 3
y
y
y
tc@E310:~$
... Looks like it is havoc's fail cause it leaves the shell without the SIGPIPE handler. I can't guess why the same behaviour is exhibited by another wayland terminal emulator, looks like coincidence. ...
Maybe they share a common code base. That can happen when
someone forks a program to implement changes they want.
Or maybe it's being caused by something else they share in
common, Wayland itself.
-
Hi jazzbiker and Rich. Thank you. It's good to have more than one workaround.
But I confess I'm no closer to understanding the root problem.
Looks like it is havoc's fail cause it leaves the shell without the SIGPIPE handler.
I'm not sure it is havoc's fault. The developer tried hard but was unable to reproduce the issue. See here:
https://github.com/ii8/havoc/issues/50#issuecomment-1783859048
-
Hi GNUser
Quote from https://github.com/ii8/havoc/issues/50#issuecomment-1783907710
In havoc, ls | sort and all other pipelines work fine. Strangely, the problem seems limited to pipelines containing yes command. ...
Nothing strange about it:
yes "password" | bcrypt somefile
yes starts writing "password"s to the pipe (FIFO).
bcrypt reads the first "password" and then the second one to confirm.
bcrypt encrypts the file and then exits.
When bcrypt exits, the pipe gets destroyed.
When yes attempts the next write, it encounters a write error and terminates.
The way to stop yes is to terminate it with Ctrl-C, kill, break the pipe, etc.
The broken pipe is the normal way to terminate yes , not an error in this case.
-
Hi Rich. I can follow all of that and it makes perfect sense. What I'm trying to understand is why these "broken pipe" messages only appear in terminal emulators on TCL in wayland session. Here is the situation:
console on TCL: no error
terminal emulator on TCL in X session: no error
terminal emulator on other distro in wayland session (per havoc's developer): no error
terminal emulator on TCL in wayland session: error
-
When bcrypt exits, the pipe gets destroyed.
When yes attempts the next write, it encounters a write error and terminates.
That's not what I'm seeing. yes is not exiting and enters an infinite loop after bcrypt gets the two lines it needs, driving CPU usage up and turning on my laptop's fans. Pressing Control+c does not stop the infinite loop. Only way to stop the infinite loop is to close the terminal emulator window:
$ echo "this is a test" >test.txt
$ yes "password" | bcrypt test.txt
Encryption key:
Again:
^C^C^C^C^C^C^C^C^C^C^C^C^C^C[laptop fans start running here, I close terminal emulator window, cpu usage goes down, fans stop]
-
Hi GNUser
Forgot about the infinite loop case. No idea about that.
-
Hi GNUser
I see you added an example to your post.
This is interesting:
tc@E310:~$ trap '' SIGPIPE # Disable handler
tc@E310:~$ echo "this is a test" >test.txt
tc@E310:~$ busybox yes "password" | bcrypt test.txt
Encryption key:
Again:
^Ctc@E310:~$ # busybox yes enters infinite loop, Ctrl-C breaks us out.
tc@E310:~$
tc@E310:~$ echo "this is a test" >test.txt
tc@E310:~$ yes "password" | bcrypt test.txt
Encryption key:
Again:
yes: standard output: Broken pipe
tc@E310:~$ # GNU yes works.
tc@E310:~$
tc@E310:~$ trap - SIGPIPE # Enable handler
tc@E310:~$ echo "this is a test" >test.txt
tc@E310:~$ busybox yes "password" | bcrypt test.txt
Encryption key:
Again:
tc@E310:~$ # busybox yes works.
tc@E310:~$
tc@E310:~$ echo "this is a test" >test.txt
tc@E310:~$ yes "password" | bcrypt test.txt
Encryption key:
Again:
tc@E310:~$ # GNU yes still works.
This seems to suggest there may be a bug in busybox yes when
there is no signal handler for SIGPIPE.
If you use GNU yes , does the problem go away?
-
Hi Rich.
If you use GNU yes , does the problem go away?
No.
I think I figured it out. Juanito sees this error in weston-terminal because that terminal ignores SIGPIPE (see here (https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/253)). In my case, I am seeing the error in havoc terminal because the terminal emulator inherits its environment from my compositor, labwc (version 0.6.5), which has been rigged to ignore SIGPIPE since version 0.6.3 (see here (https://github.com/labwc/labwc/blob/master/NEWS.md)).
If yes is running in an environment where SIGPIPE has been disabled, it will either error out with "Broken pipe" or will enter an infinite loop, depending on the situation. Either way, it will not behave as expected.
-
Hi GNUser
So just to be clear, installing coreutils and executing the
following still results in an endless loop:
$ echo "this is a test" >test.txt
$ /usr/local/bin/yes "password" | bcrypt test.txt
-
No. In that scenario an error is displayed but no endless loop. It seems only busybox yes enters the endless loop when used with bcrypt .
bruno@x230:~$ tce-load -wil coreutils
coreutils.tcz.dep OK
Downloading: coreutils.tcz
Connecting to gnuser.ddns.net (24.0.182.246:80)
saving to 'coreutils.tcz'
coreutils.tcz 100% |********************************| 2304k 0:00:00 ETA
'coreutils.tcz' saved
coreutils.tcz: OK
yes: Broken pipe
bruno@x230:~$ echo "this is a test" >test.txt
bruno@x230:~$ /usr/local/bin/yes "password" | bcrypt test.txt
Encryption key:
Again:
/usr/local/bin/yes: standard output: Broken pipe
bruno@x230:~$
-
Hi GNUser
And that's the point I was trying to make. GNU yes prints
the error message after bcrypt completed successfully.
That's why I suggested it may be a busybox issue.
-
Hi Rich. Now I understand you 100%.
The missing piece for me was that we are dealing with an environment where SIGPIPE has has been disabled. My TCL14 instance with Xorg has an intact SIGPIPE and none of these yes shenanigans.
So it seems busybox yes sometimes errors-out and sometimes enters an endless loop in an environment where SIGPIPE is disabled. GNU yes seems to consistently error-out and not enter endless loop.
Thread may be marked as solved. Thank you all.
-
Hi GNUser
Marked as solved.
By the way, busybox yes does not appear to be aliased. So if
you extract the yes executable from coreutils , copy it to
/usr/local/bin/ and add it to your backup, it might fix it.
-
So it seems busybox yes sometimes errors-out and sometimes enters an endless loop in an environment where SIGPIPE is disabled. GNU yes seems to consistently error-out and not enter endless loop.
Not sometimes. If the string to output is single-byte it exits with error, but if the string to includes more than one byte, yes hangs up.
Aren't You in the mood to send the bug report to busybox?
-
Hi jazzbiker. Yes (pun intended), that is a more precise statement of the problem. Thank you :)
I'm a bit worn out by this at the moment. Maybe I'll find the inspiration later. If you'd like to give the busybox developers a bug report, please go ahead.
The developer of my compositor fixed the issue with this:
https://github.com/labwc/labwc/pull/1210
I have recompiled labwc with this fix and resubmitted it for the repo.
-
Since all terminal emulators and shells seem to be affected, I think the problem may have something to do with how weston or labwc is being started. Can you suggest a way to start the graphical session different from this (which is what weston.tcz.info suggests):
Weston used to use weston-launch to start, but this is deprecated and still had the error. I believe sway uses seatd-launch, but I haven’t tried it.
Gnome-session uses the mutter wayland compositor and doesn’t exhibit the error in gnome-terminal, but I haven’t tried it with havoc.
Perhaps it’s worth moving the XDG_RUNTIME_DIR statement to /etc/profile and/or .ashrc to see if that helps?
-
havoc does not have the broken pipe error with gnome-session