pytorch dataset: pass cache/prefetch to DataChain instances

[skshetry]

This should enable prefetching on to_pytorch API. We were not passing any settings to the DataChain instances created inside to_pytorch, due to which it was not using cache or prefetch.

I have refrained from setting workers, etc. because I believe that is better left to PytorchDataLoader for now.

I have found this PR to increase performance for torch-loader.py example by 20-25% when prefetch and cache is enabled. However, this was done on my machine and was not a scientific measurement.

Closes #631.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.67%. Comparing base (692c8dc) to head (e258e9f).
Report is 4 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #653   +/-   ##
=======================================
  Coverage   87.67%   87.67%           
=======================================
  Files         111      111           
  Lines       10601    10603    +2     
  Branches     1436     1436           
=======================================
+ Hits         9294     9296    +2     
  Misses        945      945           
  Partials      362      362           

Flag	Coverage Δ
datachain	87.61% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[shcheklein]

@skshetry could you clarify the scope for the #631 based on the previous meetings discussion? (does this PR close it for example or we had more things in mind?). Do we need more testing, research how it works or not?

[dreadatour]

Looks good to me! 👍

[skshetry]

@skshetry could you clarify the scope for the #631 based on the previous meetings discussion? (does this PR close it for example or we had more things in mind?). Do we need more testing, research how it works or not?

I was considering whether we need prefetching at all, given that pytorch handles some of this internally. However, in the example I mentioned earlier, a significant amount of time is spent downloading files.

(Even with prefetch_factor set, it still is going to download in sequence, downloading while the iterator gets consumed).

When running with cache=True, I see that the DataLoader processes the dataset (200 files) in under a second, compared to around 30 seconds without caching. So the potential is there for improvement. That said, I haven’t noticed any meaningful improvements with prefetch alone so far.

I’m currently experimenting with a dataset containing a larger number of files to test the hypothesis that prefetching improves performance.

While prefetching should theoretically enhance DataLoader performance, I haven’t yet observed any tangible gains (see related issue: #635).

---

prefetch + cache does help though on successive invocations due to caching, it is how prefetch is implemented at the moment anyway. So I think we should pass these configs to to_pytorch, and research performance issue separately.