pAInt

About

• Built with Pytorch for Pytorch Summer Hackathon 2020, pAInt can be called as an AI augmented painting tool.
• The user can sketch anything he/she wants and let AI complete the art with best in class Artistic style transfers and styling with a custom image
• The experience also involves a gallery consisting of some of the art completed by fellow artists and a cart that allows the user to simply download artwork from the application
• Uses Pix2Pix model for image to image translation and Bicycle GAN and Cycle GAN to generate diffrent variations of the synthesised image. Neural Style transfer was used to give a customized artistic touch for the final output.
• Includes a canvas forked from JS Paint which resembles MS Paint from the windows 90s era.
• The app was built with a typescript backend and a VueJS front end. The Typescript backend calls Python scripts that handles the ML tasks with Pytorch.
• The repo contains two folders /paint/ and /paint-frontend/. These contain the code for backend and frontend respectively.
• The backend uses the folder /paint/wwwroot/ as the public or static directory and this is the place where the compiled distribution files of the frontend are placed.
• You may find the codes of Deep Learning models used within /paint/ml/ directory.
• You may find the modified code of JS Paint within the repo at /paint/wwwroot/paint/.

DEMO video

The site is currently down. So you cannot try the web application by yourself. But however, you can watch this youtube video which shows our DEMO of the application.

Pre-trained checkpoints

• Under ./paint/ml/image_synthesis, paste the checkpoints folder from here
• Under ./paint/ml/style_branch/custom_style, paste the models folder from here
• Under ./paint/ml/style_branch/style_gen, paste the checkpoints folder from here
• Under ./paint/ml/style_branch/variation_gen, paste the checkpoints folder from here

Usage

• Deploy the backend to a server with a unix based shell that supports "&&" to chain two commands together and has an NVIDIA GPU attached.
• You must have python or conda set up with all requirements as given in /paint/ml/conda_env.yml and /paint/ml/conda_env_spec.txt
• Note that this project uses GIT LFS. So you'll have to install GIT LFS and fetch all the LFS files

Authors

• RA Keerthan
• Hari R
• Subrahmanyam