sylvainlapeyrade
commited on
Commit
•
ec59ea9
1
Parent(s):
585e145
Update README.md
Browse files
README.md
CHANGED
@@ -16,33 +16,50 @@ language:
|
|
16 |
pipeline_tag: text-to-image
|
17 |
---
|
18 |
|
19 |
-
<!-- This model card has been generated automatically according to the information the training script had access to. You
|
20 |
-
should probably proofread and complete it, then remove this comment. -->
|
21 |
-
|
22 |
-
|
23 |
# LoRA text2image fine-tuning - sylvainlapeyrade/kanji2english
|
24 |
-
|
25 |
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
![img_3](./image_3.png)
|
30 |
|
|
|
31 |
|
32 |
-
|
33 |
-
## Intended uses & limitations
|
34 |
|
35 |
-
|
36 |
|
37 |
```python
|
38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
39 |
```
|
40 |
|
41 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
42 |
|
43 |
-
|
44 |
|
45 |
-
##
|
46 |
|
47 |
-
|
48 |
-
-->
|
|
|
16 |
pipeline_tag: text-to-image
|
17 |
---
|
18 |
|
|
|
|
|
|
|
|
|
19 |
# LoRA text2image fine-tuning - sylvainlapeyrade/kanji2english
|
20 |
+
<i>This is a model ran on only one epoch on Google Colab free version for proof of concepts purpose. Do not expect extraordinary resutls.</i>
|
21 |
|
22 |
+
These are LoRA adaptation weights for CompVis/stable-diffusion-v1-4. The weights were fine-tuned on the sylvainlapeyrade/kanji_english_meaning dataset to generate Kanji images based on English descriptions. This model provides a novel approach to visualizing the artistic representation of Kanji characters through a text-to-image generation process, using the powerful Stable Diffusion architecture enhanced with LoRA layers for fine-tuning.
|
23 |
+
|
24 |
+
## Model Description
|
|
|
25 |
|
26 |
+
This model uses LoRA (Low-Rank Adaptation) layers to fine-tune the Stable Diffusion v1-4 model specifically for the task of generating Kanji images from English text descriptions. The fine-tuning process was conducted using the sylvainlapeyrade/kanji_english_meaning dataset, which contains a collection of Kanji characters and their corresponding English meanings.
|
27 |
|
28 |
+
## How to Use
|
|
|
29 |
|
30 |
+
This model is intended to be used with the `diffusers` library. Here is an example of how to generate an image from text:
|
31 |
|
32 |
```python
|
33 |
+
from diffusers import LoraDiffusionPipeline
|
34 |
+
|
35 |
+
# Load the pipeline
|
36 |
+
pipe = LoraDiffusionPipeline.from_pretrained("sylvainlapeyrade/kanji2english")
|
37 |
+
|
38 |
+
# Generate an image
|
39 |
+
prompt = "a kanji meaning a Doge"
|
40 |
+
image = pipe(prompt).images[0]
|
41 |
+
|
42 |
+
# Save or display the image
|
43 |
+
image.save("kanji_doge.png")
|
44 |
```
|
45 |
|
46 |
+
## Limitations and Bias
|
47 |
+
|
48 |
+
This model has been trained specifically for generating Kanji characters from English text inputs with the <i>"a kanji meaning"</i> sequence and may not generalize well to other text-to-image tasks outside this scope. Additionally, the accuracy of the generated images may vary depending on the complexity and rarity of the Kanji.
|
49 |
+
|
50 |
+
## Examples
|
51 |
+
|
52 |
+
Below are some example images generated by the model:
|
53 |
+
|
54 |
+
![img_0](./image_0.png)
|
55 |
+
![img_1](./image_1.png)
|
56 |
+
![img_2](./image_2.png)
|
57 |
+
![img_3](./image_3.png)
|
58 |
+
|
59 |
+
## Acknowledgements
|
60 |
|
61 |
+
Special thanks to the creators of the KANJIDIC and KanjiVG projects for providing the data sources used during training.
|
62 |
|
63 |
+
## License
|
64 |
|
65 |
+
This model is released under the CreativeML Open RAIL-M license, which allows for non-commercial use, sharing, and adaptation with attribution.
|
|