#DALL-E Mini in a Terminal

Generate an image from the DALL-E Mini model. All without leaving the terminal. A simplistic refactoring of the official project's inference pipeline notebook.

#Usage

Download the pretrained VQGAN and DALL-E mini models from here. and here.

Run make install to install into a virtualenv.

$ time (. .venv/bin/activate; python -m dalle_mini_terminal \
> --dalle ./mini-1_v0_artifacts \
> --vqgan ./vqgan_imagenet_f16_16384_artifacts \
> -- your prompt should go here)
Generating images with prompt: cats playing chess
WARNING:jax._src.lib.xla_bridge:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

real    6m0.490s
user    6m32.701s
sys     0m5.739s

Some notes:

• The first time dalle_mini_terminal runs, a list of words will be downloaded to ~/.cache/huggingface/hub/models--dalle-mini--dalle-mini. This is an unavoidable side-effect from importing dalle_mini.

Or run make build to build the container image.

$ time sudo docker run --rm --interactive --tty \
> --mount type=bind,src="$(pwd)/mini-1_v0_artifacts",dst=/dalle-artifacts \
> --mount type=bind,src="$(pwd)/vqgan_imagenet_f16_16384_artifacts",dst=/vqgan_artifacts \
> --mount type=bind,src="$(pwd)/output",dst=/output \
> dalle_mini_terminal \
> your prompt should go here
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.
Moving 0 files to the new cache system
0it [00:00, ?it/s]
Generating images with prompt: cats playing chess
WARNING:jax._src.lib.xla_bridge:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

real    5m52.257s
user    0m0.085s
sys     0m0.067s

Some notes:

• The Dockerfile does not download the models. There are terms and conditions associated with use of these models, and by downloading files from the associate portals you will be accepting them. The only download that I have pre-built into the image is a list of words that is unavoidably downloaded and cached as a side effect of importing the dalle_mini package. I won't be making any other exceptions.
• I can't seem to do anything about those warnings. They shouldn't bother anyone much, since they will just go to a log unless the container is running interactively (i.e. --tty --interactive).
• If for any reason you need to interact with the image beyond generating images, be sure to override the entrypoint (i.e. --entrypoint sh).

#CUDA

This is more work than it's worth. If it just works for you, congrats. It doesn't work for me either, if that's any consolation.

#Licensing

This is all derivative of the iPython/Jupyter notebook hosted at [https://github.com/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb]. As such, I have reproduced the original license in this repository (see LICENSE.txt). The work is licensed under Apache 2.

See a list of the model's authors here.

Cite the model as:

@misc{Dayma_DALL·E_Mini_2021,
      author = {Dayma, Boris and Patil, Suraj and Cuenca, Pedro and Saifullah, Khalid and Abraham, Tanishq and Lê Khắc, Phúc and Melas, Luke and Ghosh, Ritobrata},
      doi = {10.5281/zenodo.5146400},
      month = {7},
      title = {DALL·E Mini},
      url = {https://github.com/borisdayma/dalle-mini},
      year = {2021}
}

Images generated by the model are one of:

1. Public domain
2. Property of the AI model
3. Licensed as a derivative work of the model, which itself is licensed under Apache 2 (see above)