DZGCoOp paper: CVPR author-kit scaffold and full draft

Paper draft for the second work (thesis Ch.4 ZGDPE, renamed DZGCoOp):
abstract, intro, related work, method (zero-shot guidance module,
asymmetric dual-branch with deep prompts, confidence-guided dynamic
fusion, guidance losses + EWA), experiments (base-to-novel,
cross-domain, cross-dataset, ablations) and conclusion.

Compiles to 8 pages with tectonic (no errors, no overfull boxes).

sec/1_intro.tex

@@ -0,0 +1,21 @@
+\section{Introduction}
+\label{sec:intro}
+
+Vision-language models (VLMs) such as CLIP~\cite{radford2021learning} and ALIGN~\cite{jia2021scaling} learn aligned image-text embedding spaces from web-scale pairs and exhibit remarkable zero-shot classification: an image is categorized by comparing its embedding with class-name embeddings produced from hand-crafted templates such as ``a photo of a \{class\}''. Adapting such models to downstream tasks by full fine-tuning is expensive and prone to catastrophic forgetting, which has made \emph{prompt learning}~\cite{zhou2022learning} the dominant few-shot adaptation paradigm: only a small set of continuous context vectors is optimized while both encoders stay frozen.
+
+Despite steady progress, two structural limitations persist in current prompt learners.
+\textbf{(i) Sample-agnostic branch fusion.}
+Multi-branch or ensemble-style methods---including our prior MSGCoOp~\cite{wang2025msgcoop}, CoCoOp~\cite{zhou2022conditional}, and AAPL~\cite{kim2024aapl}---aggregate branch logits with equal weights. For samples that frozen CLIP already predicts with high confidence, the branches largely agree and equal-weight averaging wastes both computation and representational capacity; for samples with high zero-shot uncertainty, the branches disagree substantially, and a plain mean cannot integrate their complementary evidence. The fusion weight is moreover identical for every instance, ignoring how reliably CLIP ``knows'' each sample.
+\textbf{(ii) Implicit division of labor among branches.}
+Parallel branches are typically trained with the same objective, data, and losses. Regularizers such as the diversity loss of MSGCoOp only implicitly discourage branch similarity, so branches may still converge to near-identical semantics, and none of them is explicitly responsible for preserving pre-trained knowledge versus exploring task-specific ones.
+In addition, text-side prompt learning that inserts vectors only at the encoder input~\cite{zhou2022learning,yao2023visual} underuses the hierarchical representations of deep transformers, whereas VPT~\cite{jia2022visual} and MaPLe~\cite{khattak2023maple} show that injecting prompts across layers of both encoders yields finer-grained adaptation.
+
+This paper proposes \textbf{DZGCoOp} (\textbf{D}ual-Branch \textbf{Z}ero-Shot \textbf{G}uided \textbf{Co}ntext \textbf{Op}timization), which turns the frozen CLIP itself into a per-sample routing and supervision signal. A zero-shot guidance module first distills four priors from the frozen encoders: LLM-enriched semantic anchors~\cite{wang2025msgcoop,brown2020language}, zero-shot visual features, a zero-shot probability distribution, and an instance-level confidence $\alpha$ given by the maximum zero-shot probability. On top of these priors, a dual-branch multimodal learning module builds an asymmetric architecture: a \emph{strong branch}, initialized from a hand-crafted template and strongly regularized toward the semantic anchors, is conservative and preserves pre-trained knowledge; a \emph{weak branch}, randomly initialized and weakly regularized, explores task-specific semantics. Both text branches share deep prompt layers coupled with deep visual prompts following MaPLe~\cite{khattak2023maple}. The final logits fuse the two branches \emph{per instance} as $\alpha\,\mathbf{l}^{\mathrm{str}}+(1-\alpha)\,\mathbf{l}^{\mathrm{weak}}$, so reliable CLIP knowledge dominates on confident samples while exploratory representations take over on uncertain ones. Two guidance losses align learned features and output distributions with the zero-shot priors, and an epoch-weighted averaging (EWA) strategy consolidates parameter snapshots along the training trajectory with a Gaussian schedule.
+
+Our contributions are summarized as follows:
+\begin{itemize}
+    \item We propose a zero-shot guidance module that exploits the frozen CLIP encoders to produce LLM-enriched semantic anchors, zero-shot visual features, zero-shot probabilities, and instance-level confidence, providing stable priors and an adaptive fusion signal for prompt learning.
+    \item We design an asymmetric dual-branch multimodal prompt network with shared deep prompt layers, in which strong and weak branches assume explicit complementary roles and are fused dynamically by the instance-level zero-shot confidence.
+    \item We introduce zero-shot feature and probability guidance losses together with an epoch-weighted averaging strategy, aligning optimization with pre-trained knowledge and smoothing training stochasticity.
+    \item Extensive experiments show that DZGCoOp achieves 80.33\% average harmonic mean on 11 base-to-novel benchmarks (+1.78\% over MaPLe), 60.66\% average accuracy in cross-domain transfer, and 65.99\% in cross-dataset transfer, with ablations validating each component.
+\end{itemize}