DZGCoOp-Paper/sec/0_abstract.tex

\begin{abstract}
Prompt learning has become the prevailing paradigm for adapting vision-language models such as CLIP to few-shot image classification. However, existing multi-branch prompt methods fuse branch predictions with fixed equal weights at the logit level, which is redundant for samples that the frozen CLIP already classifies with high confidence yet unable to reconcile the large disagreements that arise on low-confidence samples. Moreover, parallel branches typically share identical objectives, data, and losses, leaving the division of labor among branches implicit, while prompt injection restricted to the encoder input layer fails to exploit the hierarchical representations of deep transformers. We propose DZGCoOp, a Dual-Branch Zero-Shot Guided Context Optimization framework. DZGCoOp first distills frozen-CLIP priors---LLM-enriched semantic anchors, zero-shot visual features, a zero-shot probability distribution, and an instance-level confidence---then learns an asymmetric dual-branch multimodal prompt network in which a strongly guided, hand-crafted-initialized branch preserves pre-trained knowledge while a weakly guided, randomly-initialized branch explores task-specific semantics. The two branches, coupled through shared deep prompt layers on both encoders, are fused per-instance with the zero-shot confidence, and zero-shot guidance losses align learned features and output distributions with the frozen priors. An epoch-weighted averaging strategy further consolidates parameter snapshots along the training trajectory. On 11 benchmarks, DZGCoOp attains 80.33\% average harmonic mean in base-to-novel generalization (+1.78\% over MaPLe), and it also achieves consistent gains in cross-domain (60.66\%) and cross-dataset (65.99\%) transfer. Our code is available at: \href{https://github.com/Rain-Bus/DZGCoOp}{https://github.com/Rain-Bus/DZGCoOp}.
\end{abstract}
	`\begin{abstract}`
	Prompt learning has become the prevailing paradigm for adapting vision-language models such as CLIP to few-shot image classification. However, existing multi-branch prompt methods fuse branch predictions with fixed equal weights at the logit level, which is redundant for samples that the frozen CLIP already classifies with high confidence yet unable to reconcile the large disagreements that arise on low-confidence samples. Moreover, parallel branches typically share identical objectives, data, and losses, leaving the division of labor among branches implicit, while prompt injection restricted to the encoder input layer fails to exploit the hierarchical representations of deep transformers. We propose DZGCoOp, a Dual-Branch Zero-Shot Guided Context Optimization framework. DZGCoOp first distills frozen-CLIP priors---LLM-enriched semantic anchors, zero-shot visual features, a zero-shot probability distribution, and an instance-level confidence---then learns an asymmetric dual-branch multimodal prompt network in which a strongly guided, hand-crafted-initialized branch preserves pre-trained knowledge while a weakly guided, randomly-initialized branch explores task-specific semantics. The two branches, coupled through shared deep prompt layers on both encoders, are fused per-instance with the zero-shot confidence, and zero-shot guidance losses align learned features and output distributions with the frozen priors. An epoch-weighted averaging strategy further consolidates parameter snapshots along the training trajectory. On 11 benchmarks, DZGCoOp attains 80.33\% average harmonic mean in base-to-novel generalization (+1.78\% over MaPLe), and it also achieves consistent gains in cross-domain (60.66\%) and cross-dataset (65.99\%) transfer. Our code is available at: \href{https://github.com/Rain-Bus/DZGCoOp}{https://github.com/Rain-Bus/DZGCoOp}.
	`\end{abstract}`