www.verysane.ai/p/polly-wants-a-better-argument
1 correction found
Its successors (GPT-4V, Gemini, Claude) are invariably trained on paired text-image, text-audio, and text-video data.
This overstates the modalities used by the cited models. OpenAI’s GPT-4V documentation says it was trained on text-and-image data, and Anthropic introduced Claude 3’s vision as a new modality rather than claiming Claude was trained on text, audio, image, and video pairs.
Full reasoning
The sentence claims that GPT-4V, Gemini, and Claude are all trained on paired text-image, text-audio, and text-video data. That is too broad.
OpenAI's own GPT-4V system card says GPT-4V was trained on text and image data: it states that GPT-4V's training process was the same as GPT-4's, and that the pre-trained model used "a large dataset of text and image data." It describes GPT-4V as extending GPT-4 with image inputs, not as a model trained on audio and video pairs.
Anthropic's Claude 3 launch materials likewise describe vision as the new modality: Anthropic wrote that Claude 3 has "Strong vision capabilities" and that they were "particularly excited to provide this new modality." That contradicts the article's claim that "Claude" was invariably trained on paired text, image, audio, and video data.
Gemini is natively multimodal, but that does not make the blanket statement true for GPT-4V and Claude. Because the sentence says these successors are invariably trained on all of those paired modalities, a single counterexample is enough to make it incorrect, and GPT-4V is one.
2 sources
- GPT-4V(ision) System Card
"The pre-trained model was first trained to predict the next word in a document, using a large dataset of text and image data from the Internet as well as licensed sources of data."
- Introducing the next generation of Claude
Anthropic describes Claude 3 as having "Strong vision capabilities" and says, "We're particularly excited to provide this new modality to our enterprise customers."