* Initial support for SDXL refiner
* Cleanup
* Add arg for converting Unet in float32 precision
* Setup scale factor with pipeline in CLI
* Update cli arg and future warning
* Bundle refiner unet if specified
* Update script for bundled refiner
- Also skip loading model if check_output_correctness is missing, since the model does not require inferencing at conversion time
* Flip skip_model_load bool
* Cleanup
* Support bundled UnetRefiner
* Add seperate refiner config value
- Includes unloading base unet when swapping to refiner
* Update readme for SDXL refiner
* Add condition for new SDXL coreml input features
* Revert pipeline interface change, add extra logging on pipe load
* Reset model_version after refiner conversion
* Reset model_version before refiner conversion but after pipe init
* Add refiner chunking
* Ensure unets are unloaded for reduceMemory true
* Handle missing UnetRefiner.mlmodelc on pipeline load
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Prewarm refiner on load, unload on complete
* Force cpu_and_gpu for VAE until it can be fixed
* Include output dtype of np.float32 for all conversions
* Allow a custom VAE to be converted.
* Revert hardcoded reduceMemory
* Fix merge
* Default chunking arg for --merge-chunks-in-pipeline-model when called from torch2coreml
---------
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Extend ControlNet image conversion with hole punching for InPaint
The inpainting ControlNet model expects the masked areas to be filled with -1. This never occured with the original conversion, as the pixel data was converted to values between [0;1], thus the inpainting CN model never had any effect (apart from maybe generating artifacts).
This commit uses the _previously discarded_ alpha channel of the source image to mark masked pixels: any non-opaque pixel is cleared out to -1.
For an end-user, this allows creating masks simply by erasing areas with Preview to transparent, saving the result as PNG, and feeding it to the ControlNet.
* Mask only fully transparent pixels
* add controlnet tentatively
* add controlnet in python code
* implement swift part
* support 8-bit quantization
* add controlnet unload when reduce memory
* remove irrelevant changes
* add more description about controlnet option in swift
* fix some for pr and update README
* pre-allocate zero shapedArray + make multi-controlnet faster
* Add random source that matches PyTorch
This added random source that matches PyTorch on CPU. In particular, it
matches: `torch.randn([], dtype=torch.float)` result.
PyTorch's RNG is a bit convoluted and not claimed to be version-stable
(will open a separate issue in PyTorch repo on this). However, the
current implementation on CPU is fairly straightforward^*.
1. If it is less than 16 elements, it uses Gaussian distribution sampled
from MT19937 for double + Box-Muller transformation.
2. If it is more than 16 (16 included), it first do uniform sampling
with whatever the resulting data type would be (in this case, torch.float),
and then apply Box-Muller transformation over 16-element segment at a
type, treating the first floating-point and the 8th as a pair, so on
so forth.
3. If it is not a multiple of 16, trace back from the end for 16
elements and redo step 2.
* Update with configuration available in SwiftDiffusionCLI
* Fix the RNG is not passed into pipelineConfig.
* Image2Image Encoder
Encoder
* Scheduler and pipeline
* fix scheduler
* cli
* remove CLI comment
* disable dpm multistep solver with image2image
* clamp initial timestamp
* Store timesteps in reverse order for consistency.
* Report actual number of steps.
* uint32
* PRComments
* remove old initializer
* pr comments
* change name and add error handling
also fix 512 hard coded
* Add fix for Jpegs
---------
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Synced to main branch and minimizes line changes
* Adds negative prompt argument to CLI
Co-authored-by: Wanaldino Antimonio <carlosmm31093@gmail.com>
Toby Roseman