WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: distributed p2p repo  (Read 2291 times)

Offline jazzbiker

  • Hero Member
  • *****
  • Posts: 934
distributed p2p repo
« on: November 25, 2020, 12:58:39 AM »
Hi, Core people!

I want to try start the discussion inside the TinyCore community on using some existing p2p technology for decentralized delivery of TinyCore repo content.

This is not an attempt to solve any existing repo problems, everything is working fine and fluent, except some infrequent excesses. This is proposal for experiment.

This is not an attempt to change anything inside the present TinyCore architecture - I consider it as an example of best practices, sanity and balance.

The point is to make parallel repo based on the modern content-adressing techniques opposing location-addressing techniques widely and dominantly used nowadays for software delivery infrastructure by totally all (as to my knowledge) Linux and not only Linux distributions.

I want to say a few words about the history of the conception, maybe not all are acquainted with it. The story started inside Plan9 project as venti server. Then git with its approach to objects and links between them and brilliant implementation of branching, based on this technique. Bittorrent protocol brought to the wide world working implementation of DHT, and working excellent. The next were full-featured network frameworks such as Freenet, i2p, ZeroNet, GNUnet, ipfs and maybe some else which I'm missing, due to lack of knowledge. All this wonder suites are tightly based on modern cryptography achievements. Lets don't forget about cryptocurrencies, which all utilize the same ideas.

So the point is that every data structure (single file as the basic example) has some unique immutable hash, which can be used with appropriate network infrastructure as the search key and can functionally replace the location of this data structure. It means that we don't know, were the data reside, but we can pull them from the network and can be sure, that this data are exactly what we really want being absolutely agnostic (or no, on demand) about were this data are located in physical sence.

Sorry for verbosity, now about the repo. The situation is the same for any distribution, but I will talk about TinyCore. There exists tinycorelinux.net repo and we all copy the information to our localhosts. It means that all of us have some subset of the central repo data, and in case of possibility presence can distribute this data to other TinyCore users on their requests. Nowadays we all are network connected, so there is nothing extraordinary in such an approach, I hope. I want to repeat, that this is not an attempt to repair something broken, it's simply relaxed following the flow and some logics, about what we can do.

Naked bittorrent is not the solution because NATed peers are not able to share data, so some existing alive network must be used. I've made an effort to try GNUnet, but failed to do something substantial. Seems that it has not yet wide enough pool of live nodes.The most study i've done at ipfs. It has enough live nodes, go binary is extremely portable (I've tried go realization) and is capable to perform all the tasks with the single binary controlled through CLI. Though it is a little bit huge :-) - something around 50 MB 8-0 But it can be easily packed into the extension and works fine. I've used precompiled binary.

Thanks for reading so many letters. If You see some errors, please correct me and I will be happy to make some experiments, which we can discuss, if You are intersted in this topic.

Have a nice Core!

Offline jazzbiker

  • Hero Member
  • *****
  • Posts: 934
Re: distributed p2p repo
« Reply #1 on: December 05, 2020, 08:52:46 AM »

I performed some basic testing. File TinyCore-11.1.iso was added to ipfs datastore on the NATed machine, which is running  24/7. Hash is
Code: [Select]
QmTDhchgWHp7UhXRXFARsUZkBYTAWKDihraorQTGmLLnMcFile size is 19M and it takes around 7 sec to get it by another ipfs peer inside the LAN, to which the sharing machine belongs. Being requested by the ipfs peer residing inside another NATed LAN (another ISP) it takes around 17 sec to get the whole file.
This is case when the single instance of data present in the network. If anybody will pin (ipfs operation name) this file in his own ipfs datastore, it will be interseting to test, wether it will affect the load time.


Offline mocore

  • Hero Member
  • *****
  • Posts: 513
  • ~.~
Re: distributed p2p repo
« Reply #2 on: May 22, 2023, 11:16:55 AM »


interesting !

i just heard of i2p via reading some random unrelated thread
in which the following comparison  is made
i2p is what you could refer to as security by design
VPNs are refered to as security by policy

all though myself id more often than not prefix "sec" with "in"   ;D