Fix: AttributeError when `input_ids` is None during multimodal LLM training
#77
by
lyulumos
- opened
When training a multimodal language model, such as MiniGPT-4, the model utilizes inputs_embeds instead of input_ids. This is because the multimodal embeddings are aligned with the LLM's text space and are concatenated with the text embeddings, rendering input_ids unnecessary and thus None.
This leads to the following error:
AttributeError: 'NoneType' object has no attribute 'shape'
This commit addresses the issue by modifying the code to handle cases where input_ids is None, ensuring that the model can properly process the provided inputs_embeds without relying on input_ids.