InternVL3: The Open-Source VLM That Matched GPT-4o on Multimodal Benchmarks

Shanghai AI Lab released InternVL3 in April 2025, continuing what has become the most competitive open-source VLM series against GPT-4o and Claude 3.5 Sonnet. The 78B flagship achieved benchmark parity with frontier models on multimodal tasks — document understanding, chart analysis, mathematical reasoning, and visual question answering — while maintaining MIT license weights freely available to researchers and developers.

The result: for the first time in a sustained way, an open-source vision-language model was not just competitive with GPT-4o but was exceeding it on document-centric tasks.

What's New

InternVL3 introduces four significant changes over InternVL2:

InternViT-6B visual encoder: Upgraded from InternViT-300M to InternViT-6B — a 20× increase in the visual encoder's capacity. The 6B vision encoder significantly increases the model's ability to extract fine-grained detail from complex images: dense text, technical diagrams, charts with small labels, medical images.

Dynamic High-Resolution: An improved dynamic tiling system processes images at up to 4096×4096 effective resolution by splitting images into tiles and encoding each tile at full resolution. Complex documents with fine print are no longer bottlenecked by image downsampling.

Multimodal RL alignment: InternVL3 incorporates RLHF and RL reward modeling specifically on multimodal tasks — the model learns not just to answer visual questions but to reason about visual content in structured, reliable ways that match human judgment.

InternLM3-8B language backbone: Uses InternLM3-8B as the language model backbone for the mid-sized variant, significantly improving language understanding and instruction following across all VLM sizes.

Architecture

InternVL3 uses a three-part architecture:

InternViT-6B is a ViT architecture scaled specifically for visual understanding. The 6B parameter encoder with 40-tile dynamic resolution is the primary driver of document and chart benchmark improvements.

Pixel shuffle compression: After encoding, visual tokens are compressed using pixel shuffle — adjacent tokens are merged, reducing the sequence length the language model must process while preserving spatial information. This allows the 6B encoder to produce a manageable number of tokens for the language model despite high-resolution input.

Model sizes: InternVL3 ships in 1B, 2B, 4B, 8B, 14B, 38B, and 78B variants. The 78B uses InternLM-38B as its language backbone with InternViT-6B as the visual encoder.

Benchmarks

InternVL3-78B vs. frontier models (April 2025):

DocVQA 92.2% and OCRBench 88.6% show the document understanding gains from InternViT-6B directly. MathVista 72.2% (+8.4pt over GPT-4o 63.8%) demonstrates the RL alignment improvements for visual mathematical reasoning.

The 8B variant (InternVL3-8B) reaches 72.1% on MathVista — competitive with prior full-scale models at 8B inference cost.

Dynamic High-Resolution in Practice

InternVL3's tiling approach works as follows:

1. Input image is analyzed for aspect ratio and complexity
2. Image is split into up to 40 tiles (configurable based on budget)
3. Each tile is encoded by InternViT-6B at 448×448 resolution
4. Tokens from all tiles are assembled with position-aware ordering
5. Pixel shuffle compresses tokens before passing to the language model

An A4 document scanned at 300 DPI becomes roughly 40 tiles. Each tile captures a ~500px wide column of text at full resolution — enough to read 8pt font clearly. This is the mechanism behind the DocVQA and OCRBench improvements.

Charts benefit differently: a complex chart may have legend labels, axis ticks, and trend lines — all of which require separate tile-level detail. InternVL3's tiling allocates resolution budget across the chart's visual elements rather than averaging them into a blurry single encoding.

What It Means for Developers

Open-source GPT-4o replacement for document tasks: InternVL3-78B exceeds GPT-4o on DocVQA (92.2% vs 91.1%), MathVista (72.2% vs 63.8%), and OCRBench (88.6% vs 79.0%). For document-heavy applications with open-source requirements, InternVL3 is the practical answer.

8B tier is production-viable: InternVL3-8B matches previous frontier VLMs at 8B inference cost. Invoice processing, form extraction, chart reading — tasks that required 70B+ models in 2024 are feasible at 8B in 2025.

RL alignment for reliability: The multimodal RLHF means InternVL3 is more calibrated on visual reasoning tasks — fewer hallucinations, more consistent structure in responses. Important for production pipelines where reliability matters as much as raw accuracy.

Video understanding: Video-MME 72.5% (vs GPT-4o 71.9%) extends the parity to video tasks. Meeting transcription, video QA, temporal analysis with frame-level resolution.

Availability

InternVL3 is available across all major inference platforms:

• HuggingFace: OpenGVLab/InternVL3-78B and all smaller variants
• License: MIT — all sizes
• vLLM and SGLang: Both supported for production deployment
• API: Available via OpenRouter and various inference providers

Bottom Line

InternVL3 established frontier parity for open-source vision-language models. The combination of InternViT-6B, 40-tile dynamic resolution, and multimodal RL alignment produced a model that beats GPT-4o on document and visual math benchmarks — the specific tasks where VLM quality gaps have historically been most costly.

For developers choosing a VLM in 2025: InternVL3-78B is the open-source baseline, and InternVL3-8B is the practical production tier for most document and chart tasks.

Resources

• HuggingFace — InternVL3-78B — flagship model
• HuggingFace — InternVL3-8B — production tier
• GitHub — InternVL — codebase and deployment guides
• Technical Report (arXiv 2504.10479) — architecture and benchmarks

InternVL3 models are available on HuggingFace under MIT license. All sizes including commercial use.