MAKE SURE TO HAVE requirements.txt! This project needs it!
Run test on white computer!




Readme Creation:
================================================
Add examples of original image and VAE representation for explanation.


Create a section near top that showcases all features of project in concise/short manner




Acknowledgements:
Umar, ExplainAI, SD Repo



Shortcomings:
- GUI isn't very good
- For a given prompt, it seems quite deterministic. Most likely due to limited dataset


Notes:
- Took quite a while. Had several different implementations and adjusted countless times. Took around 6 months lol (check)
- Current Flask application is quite rigid. So although unlikely, if you want to use this repo to train your own model and uses difference image resolution, might need to ajust the HTML/CSS accordingly.



Perceptual Loss choosing:
show four images. Without, 0.33x, 1x, 3x scaler of percept loss.
Saved 4 images

Here are the prompts:


['Amidst the lively hum of a vibrant party, a man in green jeans sits focused on his laptop, the glow of the screen illuminating his face, surrounded by colorful lights and laughter.', 'Golden autumn light filters through windows as a woman in a long coat walks gracefully across a cozy, warmly lit wooden floor.', 'A young girl in a flowing autumn skirt, walking gracefully through a cozy house, warm golden light filtering through windows, fallen leaves scattered on wooden floors.', 'Golden autumn light filters through large classroom windows, casting a warm glow on a young guy in a crisp yellow shirt, intently working on a laptop amidst scattered notebooks and fallen leaves.', 'A cheerful girl in a bright yellow jacket chats on a sunlit beach, waves gently lapping the shore.']




There are "5042" words in the dataset. Only top 2500 (most common) is used. Can look at distributino in "1k_most common..."


If using REAL ESRGAN-
Due to deprecation, the following line 
from torchvision.transforms.functional_tensor import rgb_to_grayscale

needs to be changed to
from torchvision.transforms.functional import rgb_to_grayscale

in basicsr degradation.py file

located at
 
\venv\Lib\site-packages\basicsr\data\degradations.py


Initially, it seems like one can just downgrade their torchvision version to 0.14.0 - 0.16.0 but now the oldest version available is 0.17.0, which is not compatible.
Fortunately it's not troublesome. Just locate the deprecation file and adjust that import statement




Overall result is
Acceptable, have tested one 64x64 and 128x128 images. Comparing between the two, where the main difference was
scaling up the model (via hyperparameters like num channels, layers, etc.,) while keep others the same (epochs, lr, ...)
In my tests, 128x128 in general led to lower losses.
I suspect using higher image resolution such as 256x256, 512x512, or even higher ones would lead to even better results
Of course, at the cost of computaitonal cost









So say I want to explain to a person who doesn't know how diffusion models work and have no cd background.

How is the following analogy.


Say there's a person who is trying to throw a dart. The dartboard is approximately 1km away from them. In this case, the person is a model, and denoising process == throwing dart

At first, (during training), the person would be terrible at throwing darts. Wrong direction, angle, technique, etcs., But as the person trains, like the model, they get better and better.

In DDPM, the final model inferences T steps, based on the value used during training.
If you only gave the model a single step to denoise, it's akin to asking the person to throw bullseye on one throw. Although the man trained for a very long time, it's still nearly impossible to throw bulleyes when the initial condition is so far away from the target.

Hence, we use T steps, typically 1000. So in this case, it would be like this.
We start with a completely noisy image, T=1000, in the analogy, that would be the man standing 1km away from the board. Next, we denoise once, via passing noisy image through unet. We then subtract the predicted noise and add the majority of the noise back. Such that final result is that image is just slighly less noisy (T=999)

In the analogy, it's akin to the man throwing the dart, say, 500 meters. What we do is account that 500 meters (subtracting noise), then add 499 meters back (moving the dart back 499 meters, akin to adding back most of the noise). What happens now is that the dart is 999 meters away from the bulleyes. Slightly closer, but still a long way to go. Reason why we move back is this.

When we threw the dart 500 meters, along it's much closer to the target. We are still quite far off.