DeepSeek AI has unveiled DeepSeek-OCR, a groundbreaking
approach to compressing long contexts via optical 2D mapping. This innovative system demonstrates that
vision-based compression can achieve remarkable efficiency in handling text-heavy documents, potentially
revolutionizing how large language models (LLMs) process extensive textual information.
The DeepSeek-OCR system consists of two primary components: DeepEncoder
and DeepSeek3B-MoE-A570M as the decoder. Together, they achieve an impressive 97%
OCR precision when compressing text at a ratio of less than 10× (meaning 10 text tokens
compressed into 1 vision token). Even at an aggressive 20× compression ratio, the system maintains
approximately 60% accuracy.
1. Exceptional Compression Ratios with High
Accuracy
The core innovation of DeepSeek-OCR lies in its ability to compress textual
information dramatically while maintaining high accuracy:
- 96%+ OCR precision at 9-10× compression ratio
- ~90% accuracy at 10-12× compression ratio
- ~60% accuracy at 20× compression ratio
These results demonstrate that compact language models can effectively decode
compressed visual representations, suggesting that larger LLMs could readily acquire similar
capabilities through appropriate pretraining design.
2. DeepEncoder: Low Activation, High
Efficiency
DeepEncoder represents a novel architecture that maintains
low activation memory and minimal vision tokens even with high-resolution inputs. Key features include:
- Serial connection of window attention and global attention encoder components
- 16× convolutional compressor that reduces vision tokens before entering dense global attention
- Ability to handle large images without GPU memory overflow
- Effective memory and token compression for optimal performance
3. State-of-the-Art Performance with Minimal
Tokens
On the OmniDocBench benchmark, DeepSeek-OCR achieves
remarkable efficiency:
- Surpasses GOT-OCR2.0 (which uses 256 tokens/page) using only 100 vision
tokens
- Outperforms MinerU2.0 (which averages 6000+ tokens per page) while utilizing fewer
than 800 vision tokens
- Achieves state-of-the-art performance among end-to-end models while using the fewest vision tokens
4. Massive Production Scalability
DeepSeek-OCR demonstrates exceptional real-world performance, capable of
generating training data for LLMs and VLMs at an unprecedented scale:
- 200,000+ pages per day with a single A100-40G GPU
- 33 million pages per day using 20 nodes (160 A100-40G GPUs)
- Practical deployment for large-scale document processing tasks
Vision Encoder Comparison
Current open-source vision-language models (VLMs) employ three main types of
vision encoders, each with distinct advantages and limitations:
- Dual-tower architecture (e.g., Vary): Offers controllable parameters but requires
complex dual image preprocessing
- Tile-based methods (e.g., InternVL2.0): Reduces activation memory but can result in
excessive fragmentation and numerous vision tokens
- Adaptive resolution encoding (e.g., Qwen2-VL): Handles diverse resolutions flexibly
but faces challenges with massive activation memory consumption
DeepEncoder addresses these limitations by combining the best aspects of each
approach while minimizing their drawbacks, achieving a balance between memory efficiency, token count,
and processing capability.
Multi-Resolution Support
DeepEncoder is designed to support multiple resolutions efficiently, enabling
it to process documents of varying sizes and complexities without sacrificing performance or requiring
excessive computational resources.
The MoE Decoder Architecture
The decoder component utilizes DeepSeek3B-MoE-A570M, a
mixture-of-experts architecture that provides efficient inference while maintaining high accuracy. This
design enables the model to specialize in different aspects of OCR tasks while sharing knowledge across
experts.
