# DALL-E Mini in a Terminal

Run the [DALL-E Mini model](https://github.com/borisdayma/dalle-mini) in a
terminal.

I've taken the upstream project's inference pipeline notebook and reimplemented
as a normal Python module.
Currently does not perform the 'optional' CLIP scoring and sorting.
And obviously this module runs headlessly;
images are saved locally.



## Usage

The project can be setup (into a virtual environment for all dependencies)
by running `make install`.

Download the latest DALL-E mini model artifacts to a local `artifacts` folder.
See [W&B for these downloads](https://wandb.ai/dalle-mini/dalle-mini/artifacts).
The **mega** model is tagged as `mega-1-fp16`,
while the **mini** model is tagged as `mini-1`.

Try running with
`python -m dalle-mini-terminal -a path/to/artifacts -- avocado toast` 
It will take a while though, even with the mini model.

```
$ time (source .venv/bin/activate; python -m dalle_mini_terminal --artifacts ./mini-1_v0_artifacts -- cats playing chess)

[...]

real    79m59.554s
user    85m35.281s
sys     0m17.885s
```



### CUDA

Install the proprietary nvidia driver,
as well as the `cuda` and `cudnn` packages.
Likely also necessary to reboot, in order to load the kernel modules.

On Arch Linux, cuda libraries and binaries install into `/opt`,
while the cudnn libraries install into `/usr/lib`.
Debian-based distributions often use `/usr/lib` for both cuda and cuddn.
The underlying Python modules assume that the entire toolchain lives in
`/usr/local/cuda-MAJOR.MINOR`.

In other words, if using Arch Linux, it's also necessary to run:

```
sudo ln -s /opt/cuda /usr/local/cuda-11.7
sudo ln -s /usr/include/cudnn*.h /usr/local/cuda-11.7/include
sudo ln -s /usr/lib/libcudnn*.so /usr/local/cuda-11.7/lib64/
sudo ln -s /usr/lib/libcudnn*.a /usr/local/cuda-11.7/lib64/
```

The project can then be setup with `make install-cuda`.
Running will eat a lot of VRAM though;
far more than my measly GTX 950 has to offer.



## To-Do

 + Factor logic and functionality (e.g. `print_version`) out of `__main__.py`
   into an `internals.py` file.
 + Figure out how to automate downloading W&B artifacts in a Makefile.
 + Experiment with re-writing the codebase under the assumption that there is
   no GPU/TPU.
   + e.g. is there a cost to `flax.jax_utils.replicate`?
     It doesn't do anything for a CPU workload.
   + or is there any benefit to parallelism (via `jax.pmap`) when there is just
     one compute unit?
     (i.e. `jax.device_count() == 1`)
 + Figure out how to reflect flavors (in the BSD sense) in `pyproject.toml`,
   so that this project can be `pipx` installable both with and without cuda.
   + [Maybe not an option?](https://github.com/python-poetry/poetry/issues/2613)
 + Figure out if `mypy` is an option with this dependency chain.



## Licensing

This is all derivative of the iPython/Jupyter notebook hosted at
[https://github.com/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb].
As such, I have reproduced the original license in this repository
(see LICENSE.txt).
The work is licensed under Apache 2.

See a list of the model's authors
[here](https://github.com/borisdayma/dalle-mini#authors--contributors).

Cite the model as:

```
@misc{Dayma_DALL·E_Mini_2021,
      author = {Dayma, Boris and Patil, Suraj and Cuenca, Pedro and Saifullah, Khalid and Abraham, Tanishq and Lê Khắc, Phúc and Melas, Luke and Ghosh, Ritobrata},
      doi = {10.5281/zenodo.5146400},
      month = {7},
      title = {DALL·E Mini},
      url = {https://github.com/borisdayma/dalle-mini},
      year = {2021}
}
```

Images generated by the model are one of:

 1. Public domain
 2. Property of the AI model
 3. Licensed as a derivative work of the model,
    which itself is licensed under Apache 2 (see above)