DZGCoOp paper: CVPR author-kit scaffold and full draft
LaTeX Build / build (push) Canceled after 0s

Paper draft for the second work (thesis Ch.4 ZGDPE, renamed DZGCoOp):
abstract, intro, related work, method (zero-shot guidance module,
asymmetric dual-branch with deep prompts, confidence-guided dynamic
fusion, guidance losses + EWA), experiments (base-to-novel,
cross-domain, cross-dataset, ablations) and conclusion.

Compiles to 8 pages with tectonic (no errors, no overfull boxes).

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
Rain&Bus
2026-10-14 07:59:55 +08:00
co-authored by Claude Code
commit eacab7b7da
18 changed files with 2724 additions and 0 deletions
+11
View File
@@ -0,0 +1,11 @@
\section{Related Work}
\label{sec:relw}
\noindent\textbf{Prompt learning for vision-language models.}
CoOp~\cite{zhou2022learning} first replaces hand-crafted templates with learnable continuous context vectors for CLIP. CoCoOp~\cite{zhou2022conditional} conditions the prompt on each input image to improve generalization to unseen classes. ProGrad~\cite{zhu2023prompt} restricts prompt updates to directions consistent with the zero-shot CLIP gradient, and KgCoOp~\cite{yao2023visual} regularizes learnable prompts toward the ``a photo of a \{class\}'' embeddings to mitigate forgetting. Beyond the text side, VPT~\cite{jia2022visual} inserts learnable tokens into vision transformers, and MaPLe~\cite{khattak2023maple} jointly prompts both encoders across depth by mapping deep text prompts to deep visual prompts. AAPL~\cite{kim2024aapl} disentangles attribute-related tokens from augmentation noise. Our prior MSGCoOp~\cite{wang2025msgcoop} ensembles $N$ parallel context vectors with LLM-generated semantic anchors and a diversity regularizer. These methods either fuse branch outputs with fixed equal weights or regularize branches only implicitly, and none exploits CLIP's zero-shot prediction itself to decide \emph{per sample} how much each branch should contribute. DZGCoOp differs in using the frozen encoders to produce an instance-level confidence that routes between an explicitly conservative branch and an explicitly exploratory one, while inheriting the deep cross-modal prompting of MaPLe.
\noindent\textbf{Knowledge preservation with zero-shot priors.}
Preserving pre-trained knowledge during few-shot adaptation is a central concern of prompt learning. KgCoOp~\cite{yao2023visual} and ProGrad~\cite{zhu2023prompt} anchor optimization to hand-crafted template embeddings; MSGCoOp~\cite{wang2025msgcoop} enriches such anchors with LLM-generated class descriptions~\cite{brown2020language,achiam2023gpt}. Distillation-style approaches additionally constrain output distributions. DZGCoOp extends this line along two axes: the zero-shot prior is used not only as a feature-space anchor but also as a distribution-level teacher via a KL guidance loss, and the zero-shot confidence actively participates in inference-time branch fusion rather than only in training-time regularization.
\noindent\textbf{Weight averaging.}
Stochastic Weight Averaging (SWA)~\cite{izmailov2018averaging} averages checkpoints sampled in a late phase of training and finds flatter minima. Prompt learning, however, exhibits strong epoch-dependent behavior: models trained longer overfit the source classes and transfer worse~\cite{wang2025msgcoop}. Our epoch-weighted averaging (EWA) generalizes uniform averaging with a Gaussian weighting schedule whose center and width select the favorable phase per transfer regime, subsuming ``early-checkpoint'' selection without extra inference cost.