Minimizing Differences When Recompiling BusyBox for CorePure64(tinycore 14.x)

[emdi.hsn.2023]

I've encountered an issue while trying to replace the default BusyBox in CorePure64(Tinycore 14.x) with a recompiled version. Despite using the default config file `busybox-1.36.0_config.nosuid` and not enabling any additional features, the differences between my recompiled binary and the original are more extensive than expected.

To analyze the differences, I used the following commands:
```bash
objdump -D -b binary -m i386:x86-64 ./busybox1 > busybox1.asm
objdump -D -b binary -m i386:x86-64 ./busybox5 > busybox5.asm
diff ./busybox1.asm ./busybox5.asm
```

Although the file sizes remain the same, I observed over 20 discrepancies beyond the anticipated changes in the date string. Here’s how I compiled BusyBox in TinyCore 11.x, as per the instructions:

1. **Setup and Download:**
    ```bash
    tce-load -i compiletc sstrip
    wget https://www.busybox.net/downloads/busybox-1.36.0.tar.bz2
    cd busybox-1.36.0
    ```

2. **Applying Patches:**
    ```bash
    patch -Np1 -i ../busybox-1.27.1-wget-make-default-timeout-configurable.patch
    patch -Np1 -i ../busybox-1.29.3_root_path.patch
    patch -Np1 -i ../busybox-1.33.0_modprobe.patch
    patch -Np0 -i ../busybox-1.33.0_tc_depmod.patch
    ```

3. **Configuration and Compilation:**
    ```bash
    cp busybox-1.36.0_config.suid .config
    make oldconfig
    make CC="gcc -flto -mtune=generic -Os -pipe" CXX="g++ -flto -mtune=generic -Os -pipe -fno-exceptions -fno-rtti"
    sudo make CC="gcc -flto -mtune=generic -Os -pipe" CONFIG_PREFIX=/tmp/pkg install
    sudo mv /tmp/pkg/bin/busybox /tmp/pkg/bin/busybox.suid
    ```

4. **Recompilation with No SUID:**
    ```bash
    cp busybox-1.36.0_config.nosuid .config
    make oldconfig
    make CC="gcc -flto -mtune=generic -Os -pipe"
    sudo make CC="gcc -flto -mtune=generic -Os -pipe" CONFIG_PREFIX=/tmp/pkg install
    ```

Despite following these steps, the number of differences is puzzling. My question to the community: How can I compile BusyBox so that it more closely matches the original BusyBox included with CorePure64? Any tips or insights would be greatly appreciated!

Thank you!

[mocore]

what version !!!! of core

any thing in
 http://tinycorelinux.net/15.x/x86_64/release/src/busybox/compile_busybox
helpful ?

IF you are running 15.x
i guess this would be a good start ?

wget https://www.busybox.net/downloads/busybox-1.36.1.tar.bz2

looks like 14.x  uses busybox-1.36.0.tar.bz2
http://tinycorelinux.net/14.x/x86_64/release/src/busybox/compile_busybox

...

[emdi.hsn.2023]

@mocore

what version !!!! of core

I used http://tinycorelinux.net/14.x/x86_64/release/src/busybox/compile_busybox to compile the busybox.
I'm using TinyCore 14.x (x86_64) and followed the official guide to build BusyBox. However, when I compare the BusyBox binary from CorePure64 with my compiled version, I see many differences, even though I haven't enabled any new features.

How can I compile BusyBox to match the original BusyBox included with CorePure64 more closely? Any tips or insights would be greatly appreciated!

[Rich]

Hi emdi.hsn.2023
I'm not absolutely certain, but I think compilation might be
performed on a previous version of Tinycore.

Try compiling busybox-1.36.0 under TinyCore 13.x (x86_64)
and see if the results are any better.

[Juanito]

From: http://tinycorelinux.net/14.x/x86_64/release/src/busybox/compile_busybox

*** need to compile in tc-11.x as glibc >= 2.32 does not support rpc ***

 

[emdi.hsn.2023]

I successfully built BusyBox in Tinycore Linux version 11.x, following the instructions from [Tinycore's BusyBox compilation guide](http://tinycorelinux.net/14.x/x86_64/release/src/busybox/compile_busybox), although the guide is for version 14.x.

Issue Description:
I'm currently using the BusyBox included in [corepure64.gz from Tinycore 14.0](http://tinycorelinux.net/14.x/x86_64/archive/14.0/distribution_files/corepure64.gz). I aim to compile a BusyBox version closely matching the one in corepure64.gz, using the same `nosuid.config` as suggested in the compile guide.

Problem Encountered:
I compiled two versions of BusyBox, named `obusybox1` and `obusybox5`. To identify differences, I used hexdump and diff as follows:

Code: [Select]

$ hexdump -C obusybox1 > obusybox1.hex
$ hexdump -C obusybox5 > obusybox5.hex
$ diff -u obusybox1.hex obusybox5.hex
--- obusybox1.hex    2024-08-03 07:40:28.257532542 +0900
+++ obusybox5.hex    2024-08-03 07:40:37.221691421 +0900
@@ -30673,7 +30673,7 @@
 000791c0  25 73 27 00 6f 75 74 20  6f 66 20 6d 65 6d 6f 72  |%s'.out of memor|
 000791d0  79 00 42 75 73 79 42 6f  78 20 76 31 2e 33 36 2e  |y.BusyBox v1.36.|
 000791e0  30 20 28 32 30 32 34 2d  30 38 2d 30 31 20 31 37  |0 (2024-08-01 17|
-000791f0  3a 30 36 3a 30 37 20 55  54 43 29 00 2f 64 65 76  |:06:07 UTC)./dev|
+000791f0  3a 33 30 3a 35 30 20 55  54 43 29 00 2f 64 65 76  |:30:50 UTC)./dev|
 00079200  2f 74 74 79 00 4c 49 4e  45 53 00 43 4f 4c 55 4d  |/tty.LINES.COLUM|
 00079210  4e 53 00 64 75 6d 62 00  63 61 6e 27 74 20 6f 70  |NS.dumb.can't op|
 00079220  65 6e 20 27 25 73 27 00  63 61 6e 27 74 20 72 65  |en '%s'.can't re|

The differences are due to build time strings, which is acceptable. However, comparing `obusybox1` with the BusyBox in corepure64.gz shows significant differences:

Code: [Select]

$ diff -u obusybox1.hex busybox.hex | diffstat
busybox.hex |68071 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 file changed, 34023 insertions(+), 34048 deletions(-)

Question:
Why is there such a significant difference between `obusybox1` and the BusyBox in corepure64.gz, despite using the same configuration file?

[Rich]

Hi emdi.hsn.2023

... Why is there such a significant difference between `obusybox1` and the BusyBox in corepure64.gz, despite using the same configuration file?

Did you run sstrip on it. I see the notes indicate loading it, but don't
mention running it. This is what  file  returns on TC14:

Code: [Select]

tc@box:~$ file /bin/busybox
/bin/busybox: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), for GNU/Linux 4.19.10, dynamically linked, interpreter /lib/ld-linux-x86-64.so.2, no section header
tc@box:~$I'm pretty sure the  "no section header"  is the result of running  sstrip  on it.

As for the number of differences, I want to point out one more thing.
The diff utility compares line by line:

Code: [Select]

tc@E310:~$ diff --help
Usage: diff [OPTION]... FILES
Compare FILES line by line.
 ----- Snip -----You converted a binary, which is not line oriented, to a hex file, which
has fixed length strings.

The first 8 characters consist of the address, which line by line, will basically
be identical for both files.

The remaining characters on each line are the contents at those addresses.

Insert or remove 1 character of content, and every address from then on
gets its contents shifted, creating differences.

[emdi.hsn.2023]

Hi Rich,
Thank you for your reply 
Is there anyway to make the busybox with md5 which matches tinycore 14x's busybox?

I am running sstrip and it looks like below:

Code: [Select]

file ./b/busybox
./b/busybox: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), for GNU/Linux 4.19.10, dynamically linked, interpreter /lib/ld-linux-x86-64.so.2, no section header
I noticed that tinycore 14.x's busybox version is like "BusyBox v1.36.0 (2023-01-17 07:23:30 UTC)". "2023-01-17 07:23:30 UTC" is filled by SOURCE_DATE_EPOCH, if SOURCE_DATE_EPOCH is not present then current time will be used.
So, i set SOURCE_DATE_EPOCH with 1673948610 which is 2023-01-17 07:23:30 UTC.

following is my build script:

Dockerfile:

Code: [Select]

FROM tatsushid/tinycore:11.0-x86_64 AS busybox-compile-suite

# Install necessary packages
RUN tce-load -wic compiletc sstrip perl5
# advcomp is required for iso image creation
# advcomp

ENV SOURCE_DATE_EPOCH=1673948610

USER root
WORKDIR /build

# Add BusyBox source and patches
ADD src/busybox/ /build/

build-busybox.sh

Code: [Select]

#!/bin/sh

set -e

# Variables
BUSYBOX_VERSION=1.36.0

# cd /opt/build
echo "Extracting BusyBox..."
ls -ltr
tar -xjf busybox-${BUSYBOX_VERSION}.tar.bz2
cd busybox-${BUSYBOX_VERSION}

echo "Applying patches..."
patch -Np1 -i ../busybox-1.27.1-wget-make-default-timeout-configurable.patch
patch -Np1 -i ../busybox-1.29.3_root_path.patch
patch -Np1 -i ../busybox-1.33.0_modprobe.patch
patch -Np0 -i ../busybox-1.33.0_tc_depmod.patch
echo "Applying patches... Done"

# Function to build and package BusyBox
build_and_package() {
    config=$1
    output_dir=$2
    make clean
    cp -rf ../${config} .config
    echo "Building BusyBox with ${config} configuration..."
    make oldconfig
    make CC="gcc -flto -mtune=generic -Os -pipe" CXX="g++ -flto -mtune=generic -Os -pipe -fno-exceptions -fno-rtti"
    mkdir -p ${output_dir}
    make CONFIG_PREFIX=${output_dir} install
    sudo sstrip ${output_dir}/bin/busybox
}

build_and_package "busybox-${BUSYBOX_VERSION}_config_nosuid" "/new_pkg_nosuid"


build.sh

Code: [Select]

#!/usr/bin/env bash

set -e

echo -e "This script will build BusyBox for Tiny Core\n"

# Clean up any previous files
cleanup() {
    rm -rf ./new_pkg_nosuid
    rm -rf ./src
}

cleanup

# Download BusyBox source and patches
wget -r -np -nH --cut-dirs=3 -R "index.html*" http://tinycorelinux.net/14.x/x86_64/release/src/busybox/
rm -rf containerid

# Build Docker image and run the build script
docker build --progress=plain -t busybox-build .

echo -e "\nBuilding BusyBox no-SUID configurations...\n"
docker run --volume "$(pwd):/opt/build" --cidfile=containerid -u root busybox-build sh -c "/opt/build/build-busybox.sh"
echo -e "Building BusyBox no-SUID configurations... Done\n"

# Copy the resulting packages from the container
CONTAINER_ID=$(cat containerid)
mkdir ./new_pkg_nosuid
docker cp ${CONTAINER_ID}:/new_pkg_nosuid/bin/busybox ./new_pkg_nosuid/

docker rm -f ${CONTAINER_ID}
rm containerid


Am i missing anything? What do you think?

This time i used radiff2 -e bin.relocs.apply=true -AAA -CC ./a/tinycore14x/bin/busybox ./b/busybox
I use radiff2, a binary diffing tool from the Radare2 suite, with the -CC flag to compare two versions of a binary like busybox, the output focuses on the differences in calling conventions between the functions in each binary. It provides a detailed listing that shows how functions handle parameters and return values differently, indicating modifications in function signatures. The output typically includes the function names and addresses along with a structured presentation of these differences. This is crucial for understanding behavioral changes, ensuring compatibility, and identifying potential security implications of these differences in a binary's evolution. radiff2 helps in pinpointing specific changes and anomalies between two binaries, which is especially valuable for security analysis and version control.

I can see lots of new/unmatched things existing on the tinycore 14x's busybox which is not present in my complied version. I am sharing the diff in the attachment.
Am i missing any thing during compilation?

[Rich]

Hi emdi.hsn.2023
There's a lot of output in that attachment and it's too cryptic
for me to draw any conclusions from it.

The one thing I noticed is your  make ... install  command omits
CC=  and  CXX=  definitions as well as the  sudo  prefix.

Is your binary the same size as the TC14 busybox?

[emdi.hsn.2023]

Hi Rich,
Thank you for your reply 

The one thing I noticed is your  make ... install  command omits
CC=  and  CXX=  definitions as well as the  sudo  prefix.

This fix the my problem

Is your binary the same size as the TC14 busybox?

Previously no.
Now, yes. Same md5.

[Rich]

Hi emdi.hsn.2023

... This fix the my problem ...

Thank you for confirming that.

... Previously no.
Now, yes. Same md5.

I would already have considered matching the size a success.

Matching the MD5, as you realized, can be trickier due to issues
like embedded timestamps.

Congratulations.

[emdi.hsn.2023]

Hi Rich,
Thank you for your reply.

Matching the MD5, as you realized, can be trickier due to issues
like embedded timestamps.

by following i can mimic the embedded timestamps.
ENV SOURCE_DATE_EPOCH=1673948610

[mocore]

From: http://tinycorelinux.net/14.x/x86_64/release/src/busybox/compile_busybox

*** need to compile in tc-11.x as glibc >= 2.32 does not support rpc ***

rich mentions hear that ' --enable-obsolete-rpc'  https://forum.tinycorelinux.net/index.php/topic,26077.msg167555.html#msg167555

this appears relevant
wrt the rpc note in compile_busybox


OT : idk why i cant quote the above topic message ?

[Juanito]

—enable-obsolete-rpc is itself depreciated 🙂