Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: Can this project leverages all the nodes to pull the images parallely ? #630

Open
sadathknorket opened this issue Nov 5, 2024 · 2 comments

Comments

@sadathknorket
Copy link

Let's say I have an heavy image 11GB , can this project help k8s to pull images across the node ?
If I have 3 nodes then each node contributing 3GB pull and sharing across the nodes so the time of pulling is also reduced by 3 times

@phillebaba
Copy link
Member

The answer is it depends. If the image consists of multiple layers where each layer is 1GB for example, then Containerd would pull each layer in parallel. If there is a single layer however then it would be pulled from a single node. I do not know how beneficial it would be to on the Spegel side split the request, as the data would have to be written in order anyways to the response. I just merged #576 which would make this possible.

There is currently work going on in Containerd to split large requests into smaller requests, which would probably see a large performance increase together with Spegel. We now support range requests to Spegel will be compatible with this feature. I think our best option is to wait for this to be merged.

containerd/containerd#10177

@sadathknorket
Copy link
Author

cool , thanks @phillebaba

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants