MiniCPM

The MiniCPM class provides a wrapper for MiniCPM v2.6 that answers questions based on both images and videos.

class MiniCPM()

use_local · boolean · default: "True"

If True, inference call is run on the local VM, else offloaded onto GRID-Cortex. Defaults to True.

This model is currently not available via Cortex.

def run()

use_cortex · boolean · default: "false"

If True, offloads inference to GRID-Cortex else runs locally on the session VM. Defaults to False.

fallback_to_local · boolean · default: true

If True, falls back to local inference if cloud inference fails. Defaults to True.

def run()

media · np.ndarray

The input RGB image of shape $(M,N,3)$.

prompt · str · required

The question to answer about the media.

Returns · str

The response to the prompt.

Inference Call

from grid.model.perception.vlm.minicpm import MiniCPM
car = AirGenCar()

# We will be capturing an image from the AirGen simulator 
# and run model inference on it.

img =  car.getImage("front_center", "rgb").data

model = MiniCPM(use_local = True)
result = model.run(media=img, prompt=<prompt>)
print(result)

License

This code is licensed under the Apache 2.0 License. We have obtained official license from the company to offer this model on GRID.

Source

https://huggingface.co/openbmb/MiniCPM-V-2_6