README.md

Quick Start

Install

1. Clone this repository and navigate to IVM folder

git clone https://github.com/2toinf/IVM.git
cd IVM

2. Install Package

conda create -n IVM python=3.10 -y
conda activate IVM
pip install -e .

Usage

from IVM import load
from PIL import Image

model = load(ckpt_path, low_gpu_memory = False) # Set `low_gpu_memory=True` if you don't have enough GPU Memory

image = Image.open("image/demo/robot.jpg")
instruction = "pick up the red cup"

result = model.forward_batch([image], [instruction])
'''
result content:
        'soft': heatmap
        'bbox': crop bbox
        'blur_image': rbg
        'highlight_image': rbg
        'cropped_blur_img': rgb
        'cropped_highlight_img': rgb
        'alpha_image': rgba
'''