parallelize gpu test

[josephleekl]

Before submitting

Please complete the following checklist when submitting a PR:

• All new features must include a unit test.
  If you've fixed a bug or added code that should be tested, add a test to the
  tests directory!

• All new functions and code must be clearly commented and documented.
  If you do make documentation changes, make sure that the docs build and
  render correctly by running make docs.

• Ensure that the test suite passes, by running make test.

• Add a new entry to the .github/CHANGELOG.md file, summarizing the
  change, and including a link back to the PR.

• Ensure that code is properly formatted by running make format.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.

---

Context:

Description of the Change:
Some quick test result:

Python Tests (lightning_kokkos, kokkos-4.5.00, model-CUDA)
24m22s -> 17m35s

Python Tests (lightning_gpu, cuda-12)
1h21m29s -> 2h12m34s

Python Tests (lightning_tensor, cuda-12)
36m15s -> 24m2s

Benefits:

Possible Drawbacks:

Related GitHub Issues:

[github-actions]

Hello. You may have forgotten to update the changelog!
Please edit .github/CHANGELOG.md with:

• A one-to-two sentence description of the change. You may include a small working example for new features.
• A link back to this PR.
• Your name (or GitHub username) in the contributors section.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.08%. Comparing base (290c986) to head (11a1c3f).

❗ There is a different number of reports uploaded between BASE (290c986) and HEAD (11a1c3f). Click for more details.

HEAD has 49 uploads less than BASE

Flag	BASE (290c986)	HEAD (11a1c3f)
	56	7

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1042      +/-   ##
==========================================
- Coverage   98.10%   90.08%   -8.02%     
==========================================
  Files         233      112     -121     
  Lines       39079    16990   -22089     
==========================================
- Hits        38339    15306   -23033     
- Misses        740     1684     +944     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[multiphaseCFD]

Thank you @josephleekl . Do we want to install pytest-xdist? How can we confiugre the maping between cpu cores and gpu devices or instances (MIG)?

[maliasadi]

Just a note here: benchmarks are required for this PR.

[josephleekl]

Thanks! pytest-xdist is already installed from requirements-dev.txt . We're just using the single GPU device, but might be possible that if we partition into MIG and map directly, it will be more performant.

I have posted some preliminary timings in the PR description; for now GPU tests seem to take longer... I'll investigate more when I have time!