Update README.md
Browse files
README.md
CHANGED
|
@@ -14,13 +14,20 @@ Given a current GUI observation and an action, Code2World predicts the next scre
|
|
| 14 |
|
| 15 |
## Quickstart
|
| 16 |
|
| 17 |
-
Below, we provide
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
The code of Code2World has been in the latest Hugging Face transformers and we advise you to build from source with command:
|
| 20 |
```
|
| 21 |
pip install transformers==4.57.0
|
| 22 |
```
|
| 23 |
|
|
|
|
| 24 |
```python
|
| 25 |
import torch
|
| 26 |
from PIL import Image
|
|
|
|
| 14 |
|
| 15 |
## Quickstart
|
| 16 |
|
| 17 |
+
Below, we provide the main demo script for running one example case to show how to use Code2World with 🤗 Transformers.
|
| 18 |
+
|
| 19 |
+
To keep the demo clear and reusable, it relies on the following components:
|
| 20 |
+
- `prompt_builder.py`: builds the text prompt from the task instruction and action.
|
| 21 |
+
- `visual_hint.py`: adds visual action hints (e.g. click circles or swipe arrows) to the input screenshot.
|
| 22 |
+
- `render_utils.py`: post-processes generated HTML, renders it into an image, and saves outputs.
|
| 23 |
+
|
| 24 |
|
| 25 |
The code of Code2World has been in the latest Hugging Face transformers and we advise you to build from source with command:
|
| 26 |
```
|
| 27 |
pip install transformers==4.57.0
|
| 28 |
```
|
| 29 |
|
| 30 |
+
|
| 31 |
```python
|
| 32 |
import torch
|
| 33 |
from PIL import Image
|