Why Should I Use PyTorch Lightning?


Original Source Here

Why Should I Use PyTorch Lightning?

This article explains why PyTorch Lightning is good and how it reduces Deep Learning Boilerplate and increases the readability reproducibility and robustness of your code.

What is PyTorch Lightning?

Boilerplate is code that is often reimplemented with little to no functional variation. Deep Learning boilerplate makes deep learning code hard to read, reuse, reproduce, and debug.

PyTorch Lighting is a lightweight PyTorch wrapper for high-performance AI research that aims to abstract Deep Learning boilerplate while providing you full control and flexibility over your code. With Lightning, you scale your models not the boilerplate.

Many in the Deep Learning community are using PyTorch Lightning to take their projects to the next level. After seeing some of these amazing ecosystem projects like PyTorch Video, PyTorch Forecasting, PyTorch Tabular, Asteroid, PyTorch Geometric, and more you might be asking yourself “Should I use PyTorch Lightning?”

My answer is an emphatic yes and here are a few reasons why.

10 Reasons You Should use PyTorch Lightning for Your Next Deep Learning Project

1. Readability

Often developers ask me how can I make my PyTorch code more readable? Photo by Yaroslav Shuraev from Pexels

Lightning code is clearer to read because engineering code is abstracted away, and common functions such as training_steps, process_data are standardized. Those familiar with Lightning know exactly where to look to understand my code.

2. Robustness

Often Developers ask me how can I make my code more robust to common deep learning mistakes? Photo by Joey Kyber from Pexels

By abstracting out Boilerplate Lightning handles the tricky engineering preventing common mistakes while enabling access to all the flexibility of PyTorch when needed.

With Lightning, by default, you don’t need to worry about the boilerplate calls that a responsible for 80% of PyTorch bugs unless you need to.

3. Reproducibility

It’s no secret that there is a reproducibility crisis in the field of Deep Learning. Researchers often ask me “how can we ensure our research code is reproducible?”

I answer that the bottleneck to reproducibility in deep learning is that models are often thought of as just a graph of computations and weights.

In Lightning, all Model and Data Code is self-contained enabling and keeps track of components needed for reproducibility such as initializations, optimizers, loss functions, data transforms, and augmentations.

4. Portability

Often I’m asked how can I decouple Machine Learning Code from the Hardware it’s running on without a major refactor.

Lightning modules are hardware agnostic; if your code runs on a CPU, it will run on GPUs, TPUs, and clusters without requiring gradient accumulation or process rank management. You can even implement your own custom accelerators.

5. Out of the Box Best Practices

Check out all the ways lightning can take your PyTorch code to the next level.

Lightning enables your code to leverage state of the art best practices from Checkpointing, Early Stopping, LR Scheduling, and Mixed Precision to Stochastic Weight Averaging directly from the Trainer without requiring you to add a line of additional Deep Learning Code.

6. Community

Get involved check out the PyTorch Lightning Contribution Guidelines and get free swag.

PyTorch Lightning has a dedicated community with over 3.3K open source ecosystem projects, close to 500 open source contributors, and dozens of integrations with popular machine learning tools such as TensorBoard, CometML, Weights & Biases.

When you invest in coding with Lightning you can take solace in knowing that you are not alone.

7. Dedicated Support

PyTorch Lightning has an extremely active and strong community check out our slack to get started.

In addition to the community, the core Lightning Team provides dedicated on-call hours to help unblock, inspire and amplify anyone working on Lightning projects.

8. Content

Check out and subscribe to the new Lightning Developer Blog for more content updates.

In addition to supporting you, the Lightning Developer Advocacy team is dedicated to creating novel and interesting content to help share best practices for taking your code to the next level. In addition to relevant Dev Blog, YouTube, and Social Content the Lightning Team maintains several best practice ecosystem projects such as Flash, Transformers, and Bolts to help you take your work to the next level.

9. Testing

Each release is tested rigorously with every new PR on every supported version of PyTorch and Python, OS, multi GPUs, and even TPUs.

10. Grid

You can get started with grid.ai for free with just a GitHub or Google Account

Grid.AI enables you to scale training from your laptop to the cloud without having to modify a single line of code. While Grid supports all the classic Machine Learning Frameworks such as TensorFlow, Keras, PyTorch and more. Leveraging Lightning features such as Early Stopping, Integrated Logging, Automatic Checkpointing, and CLI enables you to make the traditional MLOps behind model training seem invisible.

Next Steps

If you have any questions about PyTorch Lightning feel free to reach out to me in the comments or on Twitter or LinkedIn. You can

About the Author

Aaron (Ari) Bornstein is an AI researcher with a passion for history, engaging with new technologies and computational medicine. As Head of Developer Advocacy at Grid.ai, he collaborates with the Machine Learning Community, to solve real-world problems with game-changing technologies that are then documented, open-sourced, and shared with the rest of the world.


Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: