I heard a rumor from Open Source Summit in Edinburgh about some way to dynamically add GPUs to and dynamically remove them from containers.

Lightning talk by Russell Neches is also relevant here: github: ryneches/SummonUndead (can only post two links, sorry)

Proof of concept by colleague: (haven’t tested myself). Disadvantages: not all state can be serialized. Advantages: convenient, almost transparent. The other option I can think of is cell-level execution - there is a cell magic that serializes all notebook state, sends it to a remote execution environment (some sort of batch queue here, hopefully so fast that it’s not visible to users), runs, re-serializes the state, brings it back, and re-loads the new state into the notebook. I’m working on an extension to make this easier (an incidentally, notebooks more like scripts): But this is not so different than interactive debugging on dedicated machines then batch running after code works, so it’s a model that is similar to what people are used to. It runs the notebook top to bottom, saves the output (maybe the state), and you can non-interactively examine those results an iterate. One option is to share GPUs by whole notebooks: make an system to submit whole notebooks to a queue. Or we can share GPUs at the levels of cells: try to send single cells to the GPU in some sort of batch/sharing system. We can share GPUs at the level of notebooks: have some sort of batch queue, entire notebooks are sent to the GPU to run. So, normal CPU-level multiprocess timesharing doesn’t work. GPUs can not be shared, so we have to redefine what sharing even means. It made me realize that we have to think far outside the box in order to tackle this problem. Eventually, one of my colleagues came up with the catchphrase: “GPUs can not be shared, but they must be shared”. For research maybe we could solve this by throwing money at it, but for a 500-person course that’s not possible. On a cluster, they are extremely expensive, and they can not be shared securely in a multi-tenant environment securely (if we’re wrong on this, please correct us).įor months people asked us for GPUs, but we couldn’t efficiently put them into Jupyter - the efficiency of interactive use would be too low. But some resources are different, in particular GPUs. Some resources are easy to share - CPUs can be easily oversubscribed, memory can be oversubscribed but not over-used. Your model’s output freeze ¶ LightningModule.This was originally going to be a lightning talk, but I can’t be there so I think that writing is more efficient. **kwargs ¶ ( Any) – Keyword arguments are also possible. *args ¶ ( Any) – Whatever you decide to pass into the forward method. If you need to control how often the optimizer steps, override the optimizer_step() hook.įorward ¶ LightningModule. If you use multiple optimizers, you will have to switch to ‘manual optimization’ mode and step them If you use, Lightning handles the closure function automatically for you. If you use 16-bit precision ( precision=16), Lightning will automatically handle the optimizer.

step() method automatically in case of automatic optimization. 'interval' (default “epoch”) in the scheduler configuration, Lightning will call If a learning rate scheduler is specified in configure_optimizers() with key step() automatically in case of automatic optimization. Self.log('metric_to_track', metric_val) in your LightningModule. Metrics can be made available to monitor by simply logging it using _shared_eval_step ( batch, batch_idx ) metrics =, ) cross_entropy ( y_hat, y ) return loss def validation_step ( self, batch, batch_idx ): loss, acc = self. model = model def training_step ( self, batch, batch_idx ): x, y = batch y_hat = self. LightningModule ): def _init_ ( self, model ): super (). From torchmetrics.functional import accuracy class ClassificationTask ( L.

Lightning scheduler api