Skip to content

Commit

Permalink
add docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jingxu10 committed Sep 17, 2023
1 parent 0c0f673 commit f63b63f
Show file tree
Hide file tree
Showing 8 changed files with 123 additions and 16 deletions.
3 changes: 2 additions & 1 deletion docs/source-fabric/fundamentals/launch.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,9 @@ This is essentially the same as running ``python path/to/your/script.py``, but i
itself and are expected to be parsed there.
Options:
--accelerator [cpu|gpu|cuda|mps|tpu]
--accelerator [cpu|gpu|cuda|mps|tpu|xpu]
The hardware accelerator to run on.
Install Lightning-XPU to enable ``xpu``.
--strategy [ddp|dp|deepspeed] Strategy for how to run across multiple
devices.
--devices TEXT Number of devices to run on (``int``), which
Expand Down
8 changes: 8 additions & 0 deletions docs/source-pytorch/common/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
../advanced/model_parallel
Train on single or multiple GPUs <../accelerators/gpu>
Train on single or multiple HPUs <../integrations/hpu/index>
Train on single or multiple XPUs <../integrations/xpu/index>
Train on single or multiple IPUs <../accelerators/ipu>
Train on single or multiple TPUs <../accelerators/tpu>
Train on MPS <../accelerators/mps>
Expand Down Expand Up @@ -168,6 +169,13 @@ How-to Guides
:col_css: col-md-4
:height: 180

.. displayitem::
:header: Train on single or multiple XPUs
:description: Train models faster with XPU accelerators
:button_link: ../integrations/xpu/index.html
:col_css: col-md-4
:height: 180

.. displayitem::
:header: Train on single or multiple IPUs
:description: Train models faster with IPU accelerators
Expand Down
7 changes: 7 additions & 0 deletions docs/source-pytorch/common_usecases.rst
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,13 @@ Customize and extend Lightning for things like custom hardware or distributed st
:button_link: integrations/hpu/index.html
:height: 100

.. displayitem::
:header: Train on single or multiple XPUs
:description: Train models faster with XPUs.
:col_css: col-md-12
:button_link: integrations/xpu/index.html
:height: 100

.. displayitem::
:header: Train on single or multiple IPUs
:description: Train models faster with IPUs.
Expand Down
5 changes: 5 additions & 0 deletions docs/source-pytorch/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,11 @@ def _load_py_module(name: str, location: str) -> ModuleType:
target_dir="docs/source-pytorch/integrations/hpu",
checkout="tags/1.0.0",
)
assist_local.AssistantCLI.pull_docs_files(
gh_user_repo="Lightning-AI/lightning-XPU",
target_dir="docs/source-pytorch/integrations/xpu",
checkout="tags/1.0.0",
)

if not _FAST_DOCS_DEV:
fetch_external_assets(
Expand Down
31 changes: 16 additions & 15 deletions docs/source-pytorch/extensions/accelerator.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Currently there are accelerators for:
- :doc:`TPU <../accelerators/tpu>`
- :doc:`IPU <../accelerators/ipu>`
- :doc:`HPU <../integrations/hpu/index>`
- :doc:`XPU <../integrations/xpu/index>`
- :doc:`MPS <../accelerators/mps>`

The Accelerator is part of the Strategy which manages communication across multiple devices (distributed communication).
Expand All @@ -32,16 +33,16 @@ Create a Custom Accelerator
.. warning:: This is an :ref:`experimental <versioning:Experimental API>` feature.

Here is how you create a new Accelerator.
Let's pretend we want to integrate the fictional XPU accelerator and we have access to its hardware through a library
``xpulib``.
Let's pretend we want to integrate the fictional YPU accelerator and we have access to its hardware through a library
``ypulib``.

.. code-block:: python
import xpulib
import ypulib
class XPUAccelerator(Accelerator):
"""Support for a hypothetical XPU, optimized for large-scale machine learning."""
class YPUAccelerator(Accelerator):
"""Support for a hypothetical YPU, optimized for large-scale machine learning."""
@staticmethod
def parse_devices(devices: Any) -> Any:
Expand All @@ -52,29 +53,29 @@ Let's pretend we want to integrate the fictional XPU accelerator and we have acc
@staticmethod
def get_parallel_devices(devices: Any) -> Any:
# Here, convert the device indices to actual device objects
return [torch.device("xpu", idx) for idx in devices]
return [torch.device("ypu", idx) for idx in devices]
@staticmethod
def auto_device_count() -> int:
# Return a value for auto-device selection when `Trainer(devices="auto")`
return xpulib.available_devices()
return ypulib.available_devices()
@staticmethod
def is_available() -> bool:
return xpulib.is_available()
return ypulib.is_available()
def get_device_stats(self, device: Union[str, torch.device]) -> Dict[str, Any]:
# Return optional device statistics for loggers
return {}
Finally, add the XPUAccelerator to the Trainer:
Finally, add the YPUAccelerator to the Trainer:

.. code-block:: python
from lightning.pytorch import Trainer
accelerator = XPUAccelerator()
accelerator = YPUAccelerator()
trainer = Trainer(accelerator=accelerator, devices=2)
Expand All @@ -90,28 +91,28 @@ If you wish to switch to a custom accelerator from the CLI without code changes,

.. code-block:: python
class XPUAccelerator(Accelerator):
class YPUAccelerator(Accelerator):
...
@classmethod
def register_accelerators(cls, accelerator_registry):
accelerator_registry.register(
"xpu",
"ypu",
cls,
description=f"XPU Accelerator - optimized for large-scale machine learning.",
description=f"YPU Accelerator - optimized for large-scale machine learning.",
)
Now, this is possible:

.. code-block:: python
trainer = Trainer(accelerator="xpu")
trainer = Trainer(accelerator="ypu")
Or if you are using the Lightning CLI, for example:

.. code-block:: bash
python train.py fit --trainer.accelerator=xpu --trainer.devices=2
python train.py fit --trainer.accelerator=ypu --trainer.devices=2
----------
Expand Down
8 changes: 8 additions & 0 deletions docs/source-pytorch/glossary/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
GPU <../accelerators/gpu>
Half precision <../common/precision>
HPU <../integrations/hpu/index>
XPU <../integrations/xpu/index>
Inference <../deploy/production_intermediate>
IPU <../accelerators/ipu>
Lightning CLI <../cli/lightning_cli>
Expand Down Expand Up @@ -159,6 +160,13 @@ Glossary
:button_link: ../integrations/hpu/index.html
:height: 100

.. displayitem::
:header: XPU
:description: Intel® Graphics Cards for faster training
:col_css: col-md-12
:button_link: ../integrations/xpu/index.html
:height: 100

.. displayitem::
:header: Inference
:description: Making predictions by applying a trained model to unlabeled examples
Expand Down
40 changes: 40 additions & 0 deletions docs/source-pytorch/integrations/xpu/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
.. _xpu:

Accelerator: XPU training
=========================

.. raw:: html

<div class="display-card-container">
<div class="row">

.. Add callout items below this line
.. displayitem::
:header: Basic
:description: Learn the basics of single and multi-XPU core training.
:col_css: col-md-4
:button_link: basic.html
:height: 150
:tag: basic

.. displayitem::
:header: Intermediate
:description: Enable state-of-the-art scaling with advanced mix-precision settings.
:col_css: col-md-4
:button_link: intermediate.html
:height: 150
:tag: intermediate

.. displayitem::
:header: Advanced
:description: Explore state-of-the-art scaling with additional advanced configurations.
:col_css: col-md-4
:button_link: advanced.html
:height: 150
:tag: advanced

.. raw:: html

</div>
</div>
37 changes: 37 additions & 0 deletions docs/source-pytorch/levels/advanced_level_23.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
:orphan:

######################
Level 19: Explore XPUs
######################

Explore Intel® Graphics Cards (XPU) for model scaling.

----

.. raw:: html

<div class="display-card-container">
<div class="row">

.. Add callout items below this line
.. displayitem::
:header: Train models on XPUs
:description: Learn the basics of single and multi-XPU core training.
:col_css: col-md-6
:button_link: ../integrations/xpu/basic.html
:height: 150
:tag: basic

.. displayitem::
:header: Optimize models training on XPUs
:description: Enable state-of-the-art scaling with advanced mixed-precision settings.
:col_css: col-md-6
:button_link: ../integrations/xpu/intermediate.html
:height: 150
:tag: intermediate

.. raw:: html

</div>
</div>

0 comments on commit f63b63f

Please sign in to comment.