[feature] is it too complicated to parse and sort the list by domains?

[SplendidSwifter]

i have just used the input file feature with a comment and for convenience i listed two fedora iso files from the same domain. the result was one (the second link) was downloading at over 7MB/s and the other was below 300kB/s. once one file was done, the other download did not ramp up to 7MB/s, and it stayed at under 300kB/s.

is it possible that with a input file like

example.com/file1
example.com/file2
example.com/file3
example-bis.com/file4

dlm starts the first and fourth file downloads in parallel, but for links 1, 2, and 3, it downloads them in sequence (maybe even reusing the connection without a new handshake)?

[agourlay]

I am not sure I want dlm to take care of this kind of scheduling TBH.

In this situation the transfer speed should have increased naturally once the first file was done.

Is it difficult for you to sort the links on your side?

[SplendidSwifter]

you are right not to want dlm to not take care of this sort of thing as it would add complexity, and complexity breeds bugs.

the transfer should increase. but it does not. one unpleasant workaround is to kill dlm and run it again, which is quite an aggression against the server because dlm would request once more files that were already done only to notice i have them already and close the connection.

finally, to answer your question: it would not be difficult at all just more work. and really, what is the point of having a computer if i can do it myself?

so i fully agree with you. and at the same time, why bother doing dlm at all? there are several apps already doing the same thing with about the same issues and about the same results?

[agourlay]

Thanks for elaborating on the root issue 👍

I am going to keep this issue open to think about it more.

I might have an idea how to implement a max_concurrent_per_domain knob.

Regarding

the transfer should increase. but it does not. one unpleasant workaround is to kill dlm and run it again, which is quite an aggression against the server because dlm would request once more files that were already done only to notice i have them already and close the connection.

That is something I can actually fix!

The HEAD request can be avoided when the file name can be deduced from the link which is the most common case.