Rain&Bus
@@ -0,0 +1,3 @@
|
||||
\begin{abstract}
|
||||
Prompt learning has become the prevailing paradigm for adapting vision-language models such as CLIP to few-shot image classification. However, existing multi-branch prompt methods fuse branch predictions with fixed equal weights at the logit level, which is redundant for samples that the frozen CLIP already classifies with high confidence yet unable to reconcile the large disagreements that arise on low-confidence samples. Moreover, parallel branches typically share identical objectives, data, and losses, leaving the division of labor among branches implicit, while prompt injection restricted to the encoder input layer fails to exploit the hierarchical representations of deep transformers. We propose DZGCoOp, a Dual-Branch Zero-Shot Guided Context Optimization framework. DZGCoOp first distills frozen-CLIP priors---LLM-enriched semantic anchors, zero-shot visual features, a zero-shot probability distribution, and an instance-level confidence---then learns an asymmetric dual-branch multimodal prompt network in which a strongly guided, hand-crafted-initialized branch preserves pre-trained knowledge while a weakly guided, randomly-initialized branch explores task-specific semantics. The two branches, coupled through shared deep prompt layers on both encoders, are fused per-instance with the zero-shot confidence, and zero-shot guidance losses align learned features and output distributions with the frozen priors. An epoch-weighted averaging strategy further consolidates parameter snapshots along the training trajectory. On 11 benchmarks, DZGCoOp attains 80.33\% average harmonic mean in base-to-novel generalization (+1.78\% over MaPLe), and it also achieves consistent gains in cross-domain (60.66\%) and cross-dataset (65.99\%) transfer. Our code is available at: \href{https://github.com/Rain-Bus/DZGCoOp}{https://github.com/Rain-Bus/DZGCoOp}.
|
||||
\end{abstract}
|
||||
@@ -0,0 +1,21 @@
|
||||
\section{Introduction}
|
||||
\label{sec:intro}
|
||||
|
||||
Vision-language models (VLMs) such as CLIP~\cite{radford2021learning} and ALIGN~\cite{jia2021scaling} learn aligned image-text embedding spaces from web-scale pairs and exhibit remarkable zero-shot classification: an image is categorized by comparing its embedding with class-name embeddings produced from hand-crafted templates such as ``a photo of a \{class\}''. Adapting such models to downstream tasks by full fine-tuning is expensive and prone to catastrophic forgetting, which has made \emph{prompt learning}~\cite{zhou2022learning} the dominant few-shot adaptation paradigm: only a small set of continuous context vectors is optimized while both encoders stay frozen.
|
||||
|
||||
Despite steady progress, two structural limitations persist in current prompt learners.
|
||||
\textbf{(i) Sample-agnostic branch fusion.}
|
||||
Multi-branch or ensemble-style methods---including our prior MSGCoOp~\cite{wang2025msgcoop}, CoCoOp~\cite{zhou2022conditional}, and AAPL~\cite{kim2024aapl}---aggregate branch logits with equal weights. For samples that frozen CLIP already predicts with high confidence, the branches largely agree and equal-weight averaging wastes both computation and representational capacity; for samples with high zero-shot uncertainty, the branches disagree substantially, and a plain mean cannot integrate their complementary evidence. The fusion weight is moreover identical for every instance, ignoring how reliably CLIP ``knows'' each sample.
|
||||
\textbf{(ii) Implicit division of labor among branches.}
|
||||
Parallel branches are typically trained with the same objective, data, and losses. Regularizers such as the diversity loss of MSGCoOp only implicitly discourage branch similarity, so branches may still converge to near-identical semantics, and none of them is explicitly responsible for preserving pre-trained knowledge versus exploring task-specific ones.
|
||||
In addition, text-side prompt learning that inserts vectors only at the encoder input~\cite{zhou2022learning,yao2023visual} underuses the hierarchical representations of deep transformers, whereas VPT~\cite{jia2022visual} and MaPLe~\cite{khattak2023maple} show that injecting prompts across layers of both encoders yields finer-grained adaptation.
|
||||
|
||||
This paper proposes \textbf{DZGCoOp} (\textbf{D}ual-Branch \textbf{Z}ero-Shot \textbf{G}uided \textbf{Co}ntext \textbf{Op}timization), which turns the frozen CLIP itself into a per-sample routing and supervision signal. A zero-shot guidance module first distills four priors from the frozen encoders: LLM-enriched semantic anchors~\cite{wang2025msgcoop,brown2020language}, zero-shot visual features, a zero-shot probability distribution, and an instance-level confidence $\alpha$ given by the maximum zero-shot probability. On top of these priors, a dual-branch multimodal learning module builds an asymmetric architecture: a \emph{strong branch}, initialized from a hand-crafted template and strongly regularized toward the semantic anchors, is conservative and preserves pre-trained knowledge; a \emph{weak branch}, randomly initialized and weakly regularized, explores task-specific semantics. Both text branches share deep prompt layers coupled with deep visual prompts following MaPLe~\cite{khattak2023maple}. The final logits fuse the two branches \emph{per instance} as $\alpha\,\mathbf{l}^{\mathrm{str}}+(1-\alpha)\,\mathbf{l}^{\mathrm{weak}}$, so reliable CLIP knowledge dominates on confident samples while exploratory representations take over on uncertain ones. Two guidance losses align learned features and output distributions with the zero-shot priors, and an epoch-weighted averaging (EWA) strategy consolidates parameter snapshots along the training trajectory with a Gaussian schedule.
|
||||
|
||||
Our contributions are summarized as follows:
|
||||
\begin{itemize}
|
||||
\item We propose a zero-shot guidance module that exploits the frozen CLIP encoders to produce LLM-enriched semantic anchors, zero-shot visual features, zero-shot probabilities, and instance-level confidence, providing stable priors and an adaptive fusion signal for prompt learning.
|
||||
\item We design an asymmetric dual-branch multimodal prompt network with shared deep prompt layers, in which strong and weak branches assume explicit complementary roles and are fused dynamically by the instance-level zero-shot confidence.
|
||||
\item We introduce zero-shot feature and probability guidance losses together with an epoch-weighted averaging strategy, aligning optimization with pre-trained knowledge and smoothing training stochasticity.
|
||||
\item Extensive experiments show that DZGCoOp achieves 80.33\% average harmonic mean on 11 base-to-novel benchmarks (+1.78\% over MaPLe), 60.66\% average accuracy in cross-domain transfer, and 65.99\% in cross-dataset transfer, with ablations validating each component.
|
||||
\end{itemize}
|
||||
@@ -0,0 +1,11 @@
|
||||
\section{Related Work}
|
||||
\label{sec:relw}
|
||||
|
||||
\noindent\textbf{Prompt learning for vision-language models.}
|
||||
CoOp~\cite{zhou2022learning} first replaces hand-crafted templates with learnable continuous context vectors for CLIP. CoCoOp~\cite{zhou2022conditional} conditions the prompt on each input image to improve generalization to unseen classes. ProGrad~\cite{zhu2023prompt} restricts prompt updates to directions consistent with the zero-shot CLIP gradient, and KgCoOp~\cite{yao2023visual} regularizes learnable prompts toward the ``a photo of a \{class\}'' embeddings to mitigate forgetting. Beyond the text side, VPT~\cite{jia2022visual} inserts learnable tokens into vision transformers, and MaPLe~\cite{khattak2023maple} jointly prompts both encoders across depth by mapping deep text prompts to deep visual prompts. AAPL~\cite{kim2024aapl} disentangles attribute-related tokens from augmentation noise. Our prior MSGCoOp~\cite{wang2025msgcoop} ensembles $N$ parallel context vectors with LLM-generated semantic anchors and a diversity regularizer. These methods either fuse branch outputs with fixed equal weights or regularize branches only implicitly, and none exploits CLIP's zero-shot prediction itself to decide \emph{per sample} how much each branch should contribute. DZGCoOp differs in using the frozen encoders to produce an instance-level confidence that routes between an explicitly conservative branch and an explicitly exploratory one, while inheriting the deep cross-modal prompting of MaPLe.
|
||||
|
||||
\noindent\textbf{Knowledge preservation with zero-shot priors.}
|
||||
Preserving pre-trained knowledge during few-shot adaptation is a central concern of prompt learning. KgCoOp~\cite{yao2023visual} and ProGrad~\cite{zhu2023prompt} anchor optimization to hand-crafted template embeddings; MSGCoOp~\cite{wang2025msgcoop} enriches such anchors with LLM-generated class descriptions~\cite{brown2020language,achiam2023gpt}. Distillation-style approaches additionally constrain output distributions. DZGCoOp extends this line along two axes: the zero-shot prior is used not only as a feature-space anchor but also as a distribution-level teacher via a KL guidance loss, and the zero-shot confidence actively participates in inference-time branch fusion rather than only in training-time regularization.
|
||||
|
||||
\noindent\textbf{Weight averaging.}
|
||||
Stochastic Weight Averaging (SWA)~\cite{izmailov2018averaging} averages checkpoints sampled in a late phase of training and finds flatter minima. Prompt learning, however, exhibits strong epoch-dependent behavior: models trained longer overfit the source classes and transfer worse~\cite{wang2025msgcoop}. Our epoch-weighted averaging (EWA) generalizes uniform averaging with a Gaussian weighting schedule whose center and width select the favorable phase per transfer regime, subsuming ``early-checkpoint'' selection without extra inference cost.
|
||||
@@ -0,0 +1,120 @@
|
||||
\section{Method}
|
||||
\label{sec:method}
|
||||
|
||||
We propose DZGCoOp, a prompt learning framework that exploits the frozen CLIP's zero-shot capability to guide an asymmetric dual-branch multimodal prompt network. Fig.~\ref{fig:framework} illustrates the overall pipeline: a zero-shot guidance stage builds priors from the frozen encoders, a dual-branch learning stage extracts deep-prompted visual and textual features and fuses them per instance, and a loss-and-optimization stage aligns the fused prediction with both labels and zero-shot priors. Throughout training, only prompt-related parameters are optimized.
|
||||
|
||||
\begin{figure*}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.92\textwidth]{fig/dzg_arch.pdf}
|
||||
\caption{Architecture of DZGCoOp. \textbf{Blue}: the zero-shot guidance module distills LLM-enriched semantic anchors, zero-shot visual features, a zero-shot probability distribution, and an instance-level confidence $\alpha$ from the frozen CLIP encoders. \textbf{Green}: the dual-branch multimodal learning module couples a conservative strong branch and an exploratory weak branch through shared deep prompt layers on both encoders and fuses their logits with $\alpha$. \textbf{Purple \& orange}: guidance losses and the epoch-weighted averaging (EWA) strategy. Only prompt parameters are updated.}
|
||||
\label{fig:framework}
|
||||
\end{figure*}
|
||||
|
||||
\subsection{Preliminaries}
|
||||
\label{sec:prelim}
|
||||
|
||||
CLIP consists of a vision encoder $\phi(\cdot)$ and a text encoder $\theta(\cdot)$ trained contrastively on web-scale image-text pairs. For a task with $N_c$ classes $\{\mathrm{cls}_1,\dots,\mathrm{cls}_{N_c}\}$, zero-shot classification encodes the template ``a photo of a [CLASS]'' into text features $\mathbf{w}_i^{\mathrm{txt}}$ and predicts via cosine similarity between $\mathbf{w}^{\mathrm{img}}=\phi(I)$ and each $\mathbf{w}_i^{\mathrm{txt}}$. Prompt learning replaces the fixed template with learnable context vectors while keeping $\phi$ and $\theta$ frozen. Following MaPLe~\cite{khattak2023maple}, we insert prompts at both the input layer (\emph{shallow}) and intermediate layers (\emph{deep}) of both encoders.
|
||||
|
||||
\subsection{Zero-Shot Guidance Module}
|
||||
\label{sec:zsg}
|
||||
|
||||
The zero-shot guidance module distills four priors from the frozen CLIP, which are consumed later by the fusion rule and the guidance losses.
|
||||
|
||||
\noindent\textbf{Class-level priors.}
|
||||
Following MSGCoOp~\cite{wang2025msgcoop}, a large language model~\cite{achiam2023gpt} generates multiple visually grounded descriptions of each class covering appearance, color, shape, and texture. The descriptions of class $\mathrm{cls}_i$ form a set $D_i$; encoding them with the frozen text encoder and averaging yields the \emph{semantic anchor}
|
||||
\begin{equation}
|
||||
\mathbf{w}^{\mathrm{zst}}_i = \frac{1}{|D_i|} \sum_{d \in D_i} \theta(d),
|
||||
\label{eq:anchor}
|
||||
\end{equation}
|
||||
a consensus representation of the class in CLIP's semantic space. Since anchors are instance-independent, they are precomputed once before training.
|
||||
|
||||
\noindent\textbf{Instance-level priors.}
|
||||
For an input image $I$, the frozen vision encoder yields the zero-shot visual feature $\mathbf{w}^{\mathrm{zsv}} = \phi(I)$. Cosine similarities against all anchors form the zero-shot logits $l^{\mathrm{zs}}_i = \mathrm{sim}(\mathbf{w}^{\mathrm{zsv}}, \mathbf{w}^{\mathrm{zst}}_i)$, which after softmax with temperature $\tau$ give the zero-shot probability distribution
|
||||
\begin{equation}
|
||||
p^{\mathrm{zs}}_i = \frac{\exp(l^{\mathrm{zs}}_i / \tau)}{\sum_{j=1}^{N_c} \exp(l^{\mathrm{zs}}_j / \tau)}.
|
||||
\label{eq:pzs}
|
||||
\end{equation}
|
||||
The \emph{instance-level confidence} is the maximum zero-shot probability,
|
||||
\begin{equation}
|
||||
\alpha = \max_{i}\, p^{\mathrm{zs}}_i \in [1/N_c,\, 1],
|
||||
\label{eq:alpha}
|
||||
\end{equation}
|
||||
which measures how deterministically the frozen CLIP ``knows'' this sample: $\alpha\!\to\!1$ indicates a confident zero-shot prediction, whereas $\alpha\!\to\!1/N_c$ indicates a near-uniform distribution. On large-scale benchmarks such as ImageNet, the average confidence on correct classes lies around $0.6$--$0.8$, making $\alpha$ a reliable per-sample signal. All computations in this module use frozen parameters and incur no gradient cost.
|
||||
|
||||
\subsection{Dual-Branch Multimodal Learning Module}
|
||||
\label{sec:dual}
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.92\linewidth]{fig/dzg_fusion.pdf}
|
||||
\caption{Zero-shot-confidence-guided dynamic fusion. The instance-level confidence $\alpha$ extracted from the frozen CLIP adaptively balances the conservative strong branch and the exploratory weak branch.}
|
||||
\label{fig:fusion}
|
||||
\end{figure}
|
||||
|
||||
The learnable part of DZGCoOp is an asymmetric dual-branch network with deep cross-modal prompting.
|
||||
|
||||
\noindent\textbf{Deep cross-modal prompting.}
|
||||
On the vision side, shallow visual prompts $\mathbf{v}^{\mathrm{vis}}$ are prepended to the patch tokens, and deep visual prompts $\text{prompt}_{\text{layer}_i}^{\mathrm{vis}}$ are injected into intermediate transformer layers, yielding a prompted encoder $\tilde{\phi}(\cdot;\cdot)$ whose backbone stays frozen. On the text side, shallow contexts $\mathbf{v}^{\mathrm{str}}$ and $\mathbf{v}^{\mathrm{weak}}$ feed the strong and weak branches respectively, while the deep text prompts $\text{prompt}_{\text{layer}_i}^{\mathrm{text}}$ are \emph{shared} between the two branches. The visual pathway is single and shared: the visual feature $\mathbf{w}^{\mathrm{vis}} = \tilde{\phi}(I; V^{\mathrm{vis}})$ serves both branches.
|
||||
|
||||
\noindent\textbf{Asymmetric branch design.}
|
||||
The strong branch concatenates $\mathbf{v}^{\mathrm{str}}$ (initialized from the hand-crafted template ``a photo of a'') with the class token and encodes it as $\mathbf{w}^{\mathrm{str}}_i = \tilde{\theta}(\mathrm{cls}_i; V^{\mathrm{str}})$; the weak branch uses randomly initialized $\mathbf{v}^{\mathrm{weak}}$ and yields $\mathbf{w}^{\mathrm{weak}}_i = \tilde{\theta}(\mathrm{cls}_i; V^{\mathrm{weak}})$. The strong branch thus starts aligned with pre-trained knowledge and is trained under a strong anchoring constraint, so it \emph{preserves} zero-shot generalization; the weak branch starts from a random point under a weak constraint, so it \emph{explores} task-specific semantics that the template cannot express. Compared with implicit diversity regularization in multi-prompt ensembles~\cite{wang2025msgcoop}, this initialization-plus-guidance asymmetry assigns the two branches explicit and complementary roles.
|
||||
|
||||
\noindent\textbf{Confidence-guided dynamic fusion.}
|
||||
Each branch scores the image against its class features,
|
||||
\begin{equation}
|
||||
l^{\mathrm{str}}_i = \mathrm{sim}(\mathbf{w}^{\mathrm{vis}}, \mathbf{w}^{\mathrm{str}}_i), \quad
|
||||
l^{\mathrm{weak}}_i = \mathrm{sim}(\mathbf{w}^{\mathrm{vis}}, \mathbf{w}^{\mathrm{weak}}_i),
|
||||
\end{equation}
|
||||
and the final logit vector is an instance-level convex combination (Fig.~\ref{fig:fusion}):
|
||||
\begin{equation}
|
||||
\mathbf{l}^{\mathrm{fin}} = \alpha \cdot \mathbf{l}^{\mathrm{str}} + (1 - \alpha) \cdot \mathbf{l}^{\mathrm{weak}}.
|
||||
\label{eq:fusion}
|
||||
\end{equation}
|
||||
When CLIP is confident about a sample, the strong branch dominates and pre-training knowledge is preserved; when CLIP is uncertain, the weak branch contributes more of its task-specific evidence. Unlike equal-weight averaging, Eq.~\eqref{eq:fusion} requires no extra parameters and adapts the fusion weights to every sample. The fused logits are normalized by softmax into the final distribution $\mathbf{p}^{\mathrm{fin}}$.
|
||||
|
||||
\subsection{Guidance Losses and Optimization}
|
||||
\label{sec:loss}
|
||||
|
||||
\noindent\textbf{Objective.}
|
||||
The total loss combines classification and two zero-shot guidance terms:
|
||||
\begin{equation}
|
||||
\mathcal{L} = \mathcal{L}_{\mathrm{ce}} + \mathcal{L}_{\mathrm{zfg}} + \mathcal{L}_{\mathrm{zpg}}.
|
||||
\label{eq:total}
|
||||
\end{equation}
|
||||
$\mathcal{L}_{\mathrm{ce}}$ is the cross-entropy on the fused logits $\mathbf{l}^{\mathrm{fin}}$. Because $\mathbf{l}^{\mathrm{fin}}$ is the confidence-weighted combination of both branches, the classification gradient supervises them jointly.
|
||||
|
||||
\noindent\textbf{Zero-shot feature guidance.}
|
||||
To prevent the prompts from drifting away from CLIP's representation space, we align learned features with the zero-shot features via $L_1$ distances:
|
||||
\begin{equation}
|
||||
\mathcal{L}_{\mathrm{zfg}} = \lambda_{\mathrm{zvg}} \mathcal{L}_{\mathrm{zvg}} + \lambda_{\mathrm{sg}}^{\mathrm{str}} \mathcal{L}_{\mathrm{sg}}^{\mathrm{str}} + \lambda_{\mathrm{sg}}^{\mathrm{weak}} \mathcal{L}_{\mathrm{sg}}^{\mathrm{weak}},
|
||||
\label{eq:zfg}
|
||||
\end{equation}
|
||||
\begin{equation}
|
||||
\mathcal{L}_{\mathrm{zvg}} = \frac{1}{B} \sum_{j=1}^{B} \|\mathbf{w}_j^{\mathrm{vis}} - \mathbf{w}_j^{\mathrm{zsv}}\|_1, \quad
|
||||
\mathcal{L}_{\mathrm{sg}}^{\mathrm{b}} = \frac{1}{N_c} \sum_{i=1}^{N_c} \|\mathbf{w}_i^{\mathrm{b}} - \mathbf{w}_i^{\mathrm{zst}}\|_1,
|
||||
\end{equation}
|
||||
where $B$ is the batch size and $\mathrm{b}\in\{\mathrm{str},\mathrm{weak}\}$. $L_1$ constrains feature \emph{positions} dimension-wise rather than only directions. Mirroring the asymmetric design, the guidance weights satisfy $\lambda_{\mathrm{sg}}^{\mathrm{str}} > \lambda_{\mathrm{sg}}^{\mathrm{weak}}$ (with $\lambda_{\mathrm{zvg}}$ and $\lambda_{\mathrm{sg}}^{\mathrm{weak}}$ following the mature configuration of~\cite{yao2023visual,wang2025msgcoop}), so the strong branch is anchored tightly while the weak branch keeps freedom to explore.
|
||||
|
||||
\noindent\textbf{Zero-shot probability guidance.}
|
||||
At the distribution level, the zero-shot probabilities $\mathbf{p}^{\mathrm{zs}}$ act as a soft teacher through a KL divergence~\cite{kullback1951information}:
|
||||
\begin{equation}
|
||||
\mathcal{L}_{\mathrm{zpg}} = \frac{1}{B \cdot N_c} \sum_{j=1}^{B} \sum_{i=1}^{N_c} p_{j,i}^{\mathrm{zs}} \log\frac{p_{j,i}^{\mathrm{zs}}}{p_{j,i}^{\mathrm{fin}}}.
|
||||
\label{eq:zpg}
|
||||
\end{equation}
|
||||
Beyond one-hot supervision, $\mathbf{p}^{\mathrm{zs}}$ encodes CLIP's inter-class structure (semantically close classes receive similar probabilities), which regularizes few-shot training against overfitting and retains the ability to predict unseen classes.
|
||||
|
||||
\noindent\textbf{Epoch-weighted averaging.}
|
||||
Prompt learners conventionally use the last-epoch parameters, which are sensitive to stochastic fluctuation. As illustrated in Fig.~\ref{fig:ewa}, we snapshot the learnable parameters $\xi_t$ at every epoch and fuse them with a Gaussian schedule after training:
|
||||
\begin{equation}
|
||||
\xi_{\mathrm{final}} = \sum_{t=1}^{T} \epsilon_t \cdot \xi_t, \quad
|
||||
\epsilon_t \propto \exp\!\left(-\frac{(t - \mu)^2}{2\sigma^2}\right),
|
||||
\label{eq:ewa}
|
||||
\end{equation}
|
||||
where $T$ is the number of epochs and $(\mu,\sigma)$ place and shape the weighting window. For base-to-novel generalization, later snapshots are of higher quality, so the window centers on late epochs ($\mu{=}15$, $\sigma{=}1$); for cross-domain and cross-dataset transfer, early snapshots retain more generic pre-trained semantics, so the window shifts early and widens ($\mu{=}6$, $\sigma{=}10$). Unlike SWA~\cite{izmailov2018averaging}, which uniformly averages late-phase checkpoints and therefore hurts transfer when the model overfits the source dataset, EWA explicitly selects the favorable phase per task family. Inference uses $\xi_{\mathrm{final}}$ directly at no additional cost.
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.92\linewidth]{fig/dzg_ewa.pdf}
|
||||
\caption{Epoch-weighted averaging (EWA). Snapshots along the training trajectory are fused with a Gaussian weight schedule whose center $\mu$ and width $\sigma$ adapt to the transfer regime.}
|
||||
\label{fig:ewa}
|
||||
\end{figure}
|
||||
@@ -0,0 +1,206 @@
|
||||
\section{Experiments}
|
||||
\label{sec:exp}
|
||||
|
||||
\subsection{Setup}
|
||||
\label{sec:setup}
|
||||
|
||||
\noindent\textbf{Benchmarks and protocols.}
|
||||
We evaluate three transfer protocols.
|
||||
\emph{(i) Base-to-novel generalization}: following the standard splits, 11 datasets (ImageNet~\cite{deng2009imagenet}, Caltech101, OxfordPets, StanfordCars, Flowers102, Food101, FGVCAircraft, SUN397, DTD, EuroSAT, UCF101) are divided into base classes for few-shot training and novel classes for evaluation, reporting base accuracy (B), novel accuracy (N), and their harmonic mean (HM).
|
||||
\emph{(ii) Cross-domain transfer}: the model trained on the 1000 classes of ImageNet is evaluated on ImageNetV2~\cite{recht2019imagenet}, ImageNet-Sketch~\cite{wang2019learning}, ImageNet-A~\cite{hendrycks2021natural}, and ImageNet-R~\cite{hendrycks2021many}.
|
||||
\emph{(iii) Cross-dataset transfer}: the ImageNet-trained model is directly evaluated on the other 10 datasets.
|
||||
|
||||
\noindent\textbf{Implementation details.}
|
||||
We use CLIP ViT-B/16 as the backbone. All models are trained in the 16-shot setting with batch size 4 using SGD (learning rate 0.0025) for 20 epochs with cosine annealing. Shallow visual and textual prompts have 4 tokens each. Deep prompt depths follow the recommended configuration of MaPLe~\cite{khattak2023maple}: 9 layers for base-to-novel and 3 layers for cross-domain and cross-dataset transfer. Guidance weights are $\lambda_{\mathrm{zvg}}{=}8$ and $\lambda_{\mathrm{sg}}^{\mathrm{weak}}{=}8$, inherited from~\cite{yao2023visual,wang2025msgcoop}, and $\lambda_{\mathrm{sg}}^{\mathrm{str}}{=}24$ (3$\times$ the weak weight) selected by grid search to realize the conservative-vs-exploratory asymmetry. EWA uses $(\mu,\sigma){=}(15,1)$ for base-to-novel and $(6,10)$ otherwise. All results are averaged over 3 random seeds.
|
||||
|
||||
\begin{table*}[t]
|
||||
\centering
|
||||
\caption{\textbf{Base-to-novel generalization on 11 benchmarks (16-shot).}
|
||||
\textbf{Bold} marks the best result per column.
|
||||
\textcolor{MidnightBlue}{Blue} (resp. \textcolor{Bittersweet}{red}) numbers show absolute gains (resp. drops) of DZGCoOp over MaPLe.}
|
||||
\label{tab:b2n}
|
||||
\renewcommand{\arraystretch}{1.15}
|
||||
\addtolength{\tabcolsep}{-2pt}
|
||||
\scalebox{0.72}{
|
||||
\begin{tabular}{l ccc ccc ccc ccc ccc ccc ccc}
|
||||
\toprule
|
||||
\textbf{Method} & \multicolumn{3}{c}{Average} & \multicolumn{3}{c}{ImageNet} & \multicolumn{3}{c}{Caltech101} & \multicolumn{3}{c}{OxfordPets} & \multicolumn{3}{c}{StanfordCars} & \multicolumn{3}{c}{Flowers102} \\
|
||||
\cmidrule(lr){2-4}\cmidrule(lr){5-7}\cmidrule(lr){8-10}\cmidrule(lr){11-13}\cmidrule(lr){14-16}\cmidrule(lr){17-19}
|
||||
& B & N & HM & B & N & HM & B & N & HM & B & N & HM & B & N & HM & B & N & HM \\
|
||||
\midrule
|
||||
CLIP & 69.34 & 74.22 & 71.70 & 72.43 & 68.14 & 70.22 & 96.84 & 94.00 & 95.40 & 91.17 & 97.26 & 94.12 & 63.37 & 74.89 & 68.65 & 72.08 & \textbf{77.80} & 74.83 \\
|
||||
CoOp~\cite{zhou2022learning} & 82.69 & 63.22 & 71.66 & 76.47 & 67.88 & 71.92 & 98.00 & 89.81 & 93.73 & 93.67 & 95.29 & 94.47 & 78.12 & 60.40 & 68.13 & 97.60 & 59.67 & 74.06 \\
|
||||
CoCoOp~\cite{zhou2022conditional} & 80.47 & 71.69 & 75.83 & 75.98 & 70.43 & 73.10 & 97.96 & 93.81 & 95.84 & 95.20 & 97.69 & 96.43 & 70.49 & 73.59 & 72.01 & 94.87 & 71.75 & 81.71 \\
|
||||
AAPL~\cite{kim2024aapl} & 80.27 & 72.17 & 76.01 & 76.53 & 70.57 & 73.43 & 97.87 & 95.10 & 96.46 & 95.63 & 97.40 & 96.51 & 70.33 & 73.50 & 71.88 & 95.10 & 70.63 & 81.06 \\
|
||||
KgCoOp~\cite{yao2023visual} & 80.73 & 73.36 & 77.00 & 75.83 & 69.96 & 72.78 & 97.72 & 94.39 & 96.03 & 94.65 & 97.76 & 96.18 & 71.76 & \textbf{75.04} & 73.36 & 95.00 & 74.73 & 83.65 \\
|
||||
MSGCoOp~\cite{wang2025msgcoop} & 81.40 & 75.05 & 78.10 & 76.46 & 70.45 & 73.31 & 98.08 & \textbf{95.81} & \textbf{96.91} & 95.62 & \textbf{97.89} & \textbf{96.70} & 71.22 & 74.56 & 72.85 & 96.20 & 75.91 & 84.84 \\
|
||||
MaPLe~\cite{khattak2023maple} & 82.28 & 75.14 & 78.55 & 76.66 & 70.54 & 73.47 & 97.74 & 94.36 & 96.02 & 95.43 & 97.76 & 96.58 & 72.94 & 74.00 & 73.47 & 95.92 & 72.46 & 82.56 \\
|
||||
\rowcolor{tabhighlight}
|
||||
DZGCoOp & \textbf{84.52} & \textbf{76.54} & \textbf{80.33} & \textbf{77.80} & \textbf{70.74} & \textbf{74.11} & \textbf{98.30} & 95.05 & 96.65 & \textbf{95.67} & 97.61 & 96.63 & \textbf{78.35} & 74.85 & \textbf{76.56} & \textbf{97.69} & 77.14 & \textbf{86.21} \\
|
||||
& \textcolor{MidnightBlue}{{+2.24}} & \textcolor{MidnightBlue}{{+1.40}} & \textcolor{MidnightBlue}{{+1.78}} & \textcolor{MidnightBlue}{{+1.14}} & \textcolor{MidnightBlue}{{+0.20}} & \textcolor{MidnightBlue}{{+0.64}} & \textcolor{MidnightBlue}{{+0.56}} & \textcolor{MidnightBlue}{{+0.69}} & \textcolor{MidnightBlue}{{+0.63}} & \textcolor{MidnightBlue}{{+0.24}} & \textcolor{Bittersweet}{{-0.15}} & \textcolor{MidnightBlue}{{+0.05}} & \textcolor{MidnightBlue}{{+5.41}} & \textcolor{MidnightBlue}{{+0.85}} & \textcolor{MidnightBlue}{{+3.09}} & \textcolor{MidnightBlue}{{+1.77}} & \textcolor{MidnightBlue}{{+4.68}} & \textcolor{MidnightBlue}{{+3.65}} \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
}
|
||||
|
||||
\bigskip
|
||||
|
||||
\scalebox{0.72}{
|
||||
\begin{tabular}{l ccc ccc ccc ccc ccc ccc ccc}
|
||||
\toprule
|
||||
\textbf{Method} & \multicolumn{3}{c}{Food101} & \multicolumn{3}{c}{FGVCAircraft} & \multicolumn{3}{c}{SUN397} & \multicolumn{3}{c}{DTD} & \multicolumn{3}{c}{EuroSAT} & \multicolumn{3}{c}{UCF101} \\
|
||||
\cmidrule(lr){2-4}\cmidrule(lr){5-7}\cmidrule(lr){8-10}\cmidrule(lr){11-13}\cmidrule(lr){14-16}\cmidrule(lr){17-19}
|
||||
& B & N & HM & B & N & HM & B & N & HM & B & N & HM & B & N & HM & B & N & HM \\
|
||||
\midrule
|
||||
CLIP & 90.10 & 91.22 & 90.66 & 27.19 & 36.29 & 31.09 & 69.36 & 75.35 & 72.23 & 53.24 & 59.90 & 56.37 & 56.48 & 64.05 & 60.03 & 70.53 & 77.50 & 73.85 \\
|
||||
CoOp~\cite{zhou2022learning} & 88.33 & 82.26 & 85.19 & 40.44 & 22.30 & 28.75 & 80.60 & 65.89 & 72.51 & 79.44 & 41.18 & 54.24 & 92.19 & 54.74 & 68.69 & 84.69 & 56.05 & 67.46 \\
|
||||
CoCoOp~\cite{zhou2022conditional} & 90.70 & 91.29 & 90.99 & 33.41 & 23.71 & 27.74 & 79.74 & 76.86 & 78.27 & 77.01 & 56.00 & 64.85 & 87.49 & 60.04 & 71.21 & 82.33 & 73.45 & 77.64 \\
|
||||
AAPL~\cite{kim2024aapl} & 90.70 & 91.60 & 91.15 & 34.07 & 24.17 & 28.28 & 79.65 & 76.90 & 78.25 & 73.90 & 53.43 & 62.02 & 87.00 & 66.30 & 75.25 & 82.20 & 74.27 & 78.03 \\
|
||||
KgCoOp~\cite{yao2023visual} & 90.50 & 91.70 & 91.09 & 36.21 & 33.55 & 34.83 & 80.29 & 76.53 & 78.36 & 77.55 & 54.99 & 64.35 & 85.64 & 64.34 & 73.48 & 82.89 & 76.67 & 79.65 \\
|
||||
MSGCoOp~\cite{wang2025msgcoop} & \textbf{90.72} & 91.62 & 91.18 & 36.73 & 34.57 & 35.62 & 81.38 & 76.20 & 78.71 & 79.32 & 56.64 & 66.09 & 86.29 & 74.97 & 80.23 & 83.52 & 77.05 & 80.16 \\
|
||||
MaPLe~\cite{khattak2023maple} & 90.71 & \textbf{92.05} & \textbf{91.38} & 37.44 & 35.61 & 36.50 & 80.82 & 78.70 & 79.75 & 80.36 & 59.18 & 68.16 & \textbf{94.07} & 73.23 & 82.35 & 83.00 & 78.66 & 80.77 \\
|
||||
\rowcolor{tabhighlight}
|
||||
DZGCoOp & 90.54 & 91.52 & 91.03 & \textbf{43.48} & \textbf{37.07} & \textbf{40.02} & \textbf{82.62} & \textbf{79.01} & \textbf{80.77} & \textbf{83.72} & \textbf{60.02} & \textbf{69.92} & 94.05 & \textbf{78.35} & \textbf{85.49} & \textbf{87.49} & \textbf{80.51} & \textbf{83.85} \\
|
||||
& \textcolor{Bittersweet}{{-0.17}} & \textcolor{Bittersweet}{{-0.53}} & \textcolor{Bittersweet}{{-0.35}} & \textcolor{MidnightBlue}{{+6.04}} & \textcolor{MidnightBlue}{{+1.46}} & \textcolor{MidnightBlue}{{+3.52}} & \textcolor{MidnightBlue}{{+1.80}} & \textcolor{MidnightBlue}{{+0.31}} & \textcolor{MidnightBlue}{{+1.02}} & \textcolor{MidnightBlue}{{+3.36}} & \textcolor{MidnightBlue}{{+0.84}} & \textcolor{MidnightBlue}{{+1.76}} & \textcolor{Bittersweet}{{-0.02}} & \textcolor{MidnightBlue}{{+5.12}} & \textcolor{MidnightBlue}{{+3.14}} & \textcolor{MidnightBlue}{{+4.49}} & \textcolor{MidnightBlue}{{+1.85}} & \textcolor{MidnightBlue}{{+3.08}} \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
}
|
||||
\end{table*}
|
||||
|
||||
\subsection{Base-to-Novel Generalization}
|
||||
|
||||
Table~\ref{tab:b2n} reports the 16-shot results on the 11 benchmarks. DZGCoOp achieves the best averages of 84.52\% (B), 76.54\% (N), and 80.33\% (HM), outperforming MaPLe by $+2.24$/$+1.40$/$+1.78$ and MSGCoOp by $+3.12$/$+1.49$/$+2.23$. It obtains the best HM on 8 of 11 datasets, with the largest gains on fine-grained and special-domain sets: FGVCAircraft ($+3.52$ HM over MaPLe), whose subtle inter-class differences benefit from the weak branch's exploratory features, Flowers102 ($+3.65$), StanfordCars ($+3.09$), EuroSAT ($+3.14$), and UCF101 ($+3.08$). Meanwhile, the strong branch keeps base accuracy high (best on 9 of 11 datasets), confirming that the asymmetric design improves novel-class generalization without sacrificing base-class fitness.
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
\caption{Cross-domain transfer with 16-shot setting. The model is trained on ImageNet and evaluated on four shifted target domains.}
|
||||
\label{tab:xdo}
|
||||
\small
|
||||
\addtolength{\tabcolsep}{-2.pt}
|
||||
\renewcommand{\arraystretch}{1.06}
|
||||
\scalebox{0.90}{
|
||||
\begin{tabular}{lcccccc}
|
||||
\toprule
|
||||
& \multicolumn{1}{c}{Source} & \multicolumn{5}{c}{Target} \\ \cline{2-7}
|
||||
& ImageNet & V2 & Sketch & A & R & Avg. \\
|
||||
\midrule
|
||||
CLIP & 66.73 & 60.83 & 46.15 & 47.77 & 73.96 & 57.17 \\
|
||||
CoOp & 71.51 & 64.20 & 47.99 & 49.71 & 75.21 & 59.28 \\
|
||||
CoCoOp & 71.02 & 64.07 & 48.75 & 50.63 & 76.18 & 59.90 \\
|
||||
AAPL & 71.37 & 64.20 & 48.80 & 50.60 & 76.87 & 60.12 \\
|
||||
KgCoOp & 71.20 & 64.10 & 48.97 & 50.69 & 76.70 & 60.11 \\
|
||||
MSGCoOp & 71.59 & 64.54 & 49.24 & 50.90 & 76.94 & 60.41 \\
|
||||
MaPLe & 70.72 & 64.07 & 49.15 & 50.90 & 76.98 & 60.27 \\
|
||||
\rowcolor{tabhighlight}
|
||||
DZGCoOp & \textbf{71.66} & \textbf{64.77} & \textbf{49.43} & \textbf{51.37} & \textbf{77.08} & \textbf{60.66} \\
|
||||
\bottomrule
|
||||
\end{tabular}}
|
||||
\end{table}
|
||||
|
||||
\subsection{Cross-Domain Transfer}
|
||||
|
||||
Table~\ref{tab:xdo} shows robustness to distribution shift. DZGCoOp ranks first on \emph{all four} target domains and on average (60.66\%, $+0.25$ over MSGCoOp, $+0.39$ over MaPLe). The gain is most pronounced on the adversarial ImageNet-A ($51.37\%$, $+0.47$), where the zero-shot guidance module supplies stable priors for samples far from the training distribution. We attribute the overall robustness to two designs: the frozen zero-shot branch remains a reliable reference under shift, and the EWA schedule configured for transfer ($\mu{=}6$, $\sigma{=}10$) keeps early, more generic snapshots in the fused parameters, mitigating overfitting to the source domain.
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
\caption{Cross-dataset transfer with 16-shot setting. The model is trained on ImageNet and directly evaluated on 10 downstream datasets.}
|
||||
\label{tab:xda}
|
||||
\tabstyle{4pt}
|
||||
\scalebox{0.71}{
|
||||
\begin{tabular}{l cccccccc}
|
||||
\toprule
|
||||
\textbf{Dataset} & \rotbox{CLIP} & \rotbox{CoOp} & \rotbox{CoCoOp} & \rotbox{AAPL} & \rotbox{KgCoOp} & \rotbox{MSGCoOp} & \rotbox{MaPLe} & \rotbox{DZGCoOp} \\
|
||||
\midrule
|
||||
ImageNet & 66.72 & 71.51 & 71.02 & 71.37 & 70.66 & 71.59 & 70.72 & \cellcolor{tabhighlight}\textbf{71.66} \\
|
||||
Caltech101 & 92.94 & 93.70 & 94.43 & 94.17 & 93.92 & 93.29 & 93.53 & \cellcolor{tabhighlight}\textbf{94.69} \\
|
||||
OxfordPets & 89.07 & 89.14 & 90.14 & \textbf{90.73} & 89.83 & 87.90 & 90.49 & \cellcolor{tabhighlight}90.20 \\
|
||||
StanfordCars & 65.29 & 64.51 & 65.32 & 65.10 & 65.41 & 65.10 & \textbf{65.57} & \cellcolor{tabhighlight}65.51 \\
|
||||
Flowers102 & 71.30 & 68.71 & 71.88 & 71.67 & 70.01 & 69.87 & \textbf{72.23} & \cellcolor{tabhighlight}71.00 \\
|
||||
Food101 & 86.11 & 85.30 & 86.06 & 86.00 & \textbf{86.36} & 85.60 & 86.20 & \cellcolor{tabhighlight}\textbf{86.36} \\
|
||||
Aircraft & \textbf{24.87} & 18.47 & 22.94 & 23.03 & 22.51 & 19.14 & 24.74 & \cellcolor{tabhighlight}22.89 \\
|
||||
SUN397 & 62.62 & 64.15 & 67.36 & 66.80 & 66.16 & 65.14 & 67.01 & \cellcolor{tabhighlight}\textbf{68.02} \\
|
||||
DTD & 44.56 & 41.92 & 45.73 & 44.80 & 46.35 & 42.59 & \textbf{46.49} & \cellcolor{tabhighlight}46.34 \\
|
||||
EuroSAT & 47.69 & 46.39 & 45.37 & 41.83 & 46.04 & 45.59 & \textbf{48.06} & \cellcolor{tabhighlight}44.82 \\
|
||||
UCF101 & 66.77 & 66.55 & 68.21 & \textbf{69.30} & 68.50 & 67.51 & 68.69 & \cellcolor{tabhighlight}\textbf{70.08} \\
|
||||
\midrule
|
||||
Average & 65.12 & 63.88 & 65.74 & 65.34 & 65.51 & 64.17 & \textbf{66.30} & \cellcolor{tabhighlight}65.99 \\
|
||||
\bottomrule
|
||||
\end{tabular}}
|
||||
\end{table}
|
||||
|
||||
\subsection{Cross-Dataset Transfer}
|
||||
|
||||
Table~\ref{tab:xda} evaluates direct transfer of the ImageNet-trained model. DZGCoOp reaches 65.99\% on average, $+1.82$ over MSGCoOp and above CLIP, CoOp, CoCoOp, AAPL, and KgCoOp, though slightly below MaPLe ($-0.31$). It is the best on Caltech101 (94.69\%), SUN397 (68.02\%, $+1.01$ over MaPLe), and UCF101 (70.08\%, $+1.39$). The confidence-guided fusion helps most on datasets whose categories are well covered by CLIP's pre-training (e.g., scenes and actions), while the residual gap to MaPLe on EuroSAT and Aircraft reflects the difficulty of satellite and fine-grained domains far from ImageNet semantics, where the source-trained weak branch provides limited additional evidence.
|
||||
|
||||
\subsection{Ablation Studies}
|
||||
\label{sec:ablation}
|
||||
|
||||
All ablations use the 16-shot base-to-novel setting averaged over the 11 datasets unless stated otherwise.
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
\caption{Ablation of the zero-shot guidance module and losses.}
|
||||
\label{tab:abl_zg}
|
||||
\small
|
||||
\renewcommand{\arraystretch}{1.15}
|
||||
\scalebox{0.82}{
|
||||
\begin{tabular}{l ccc}
|
||||
\toprule
|
||||
Method & B & N & HM \\
|
||||
\midrule
|
||||
IVLP & 84.21 & 71.79 & 77.51 \\
|
||||
+ dual-branch + avg. fusion & 84.33 & 72.10 & 77.74 \\
|
||||
+ ZG losses & 84.38 & 76.38 & 80.18 \\
|
||||
\rowcolor{tabhighlight}
|
||||
+ dynamic fusion (full model) & \textbf{84.52} & \textbf{76.54} & \textbf{80.33} \\
|
||||
\bottomrule
|
||||
\end{tabular}}
|
||||
\end{table}
|
||||
|
||||
\noindent\textbf{Zero-shot guidance.}
|
||||
Table~\ref{tab:abl_zg} starts from IVLP (independent visual-language prompting, \emph{i.e.}, MaPLe's prompting backbone without its coupling)~\cite{khattak2023maple}. Adding the asymmetric dual-branch with plain average fusion yields only $+0.23$ HM; the zero-shot guidance (ZG) losses are the key driver, lifting novel accuracy by $+4.28$ and HM by $+2.44$. Replacing average fusion with the confidence-guided dynamic fusion contributes a further $+0.15$ HM on top.
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
\caption{Ablation of the dual-branch design.}
|
||||
\label{tab:abl_dual}
|
||||
\small
|
||||
\renewcommand{\arraystretch}{1.15}
|
||||
\scalebox{0.83}{
|
||||
\begin{tabular}{l ccc}
|
||||
\toprule
|
||||
Method & B & N & HM \\
|
||||
\midrule
|
||||
Single branch (weak only) & 83.59 & 75.10 & 79.12 \\
|
||||
Single branch (strong only) & 84.31 & 76.27 & 80.09 \\
|
||||
Symmetric dual (two strong) + avg. & \textbf{84.65} & 75.50 & 79.81 \\
|
||||
Asymmetric dual (strong + weak) + avg. & 84.38 & 76.38 & 80.18 \\
|
||||
\rowcolor{tabhighlight}
|
||||
Full (asym. dual + dynamic fusion) & 84.52 & \textbf{76.54} & \textbf{80.33} \\
|
||||
\bottomrule
|
||||
\end{tabular}}
|
||||
\end{table}
|
||||
|
||||
\noindent\textbf{Asymmetric dual-branch.}
|
||||
Table~\ref{tab:abl_dual} dissects the branch structure. A single strong branch (80.09\% HM) beats a single weak branch (79.12\%), confirming the value of template initialization. Two \emph{strong} branches with average fusion reach the highest base accuracy (84.65\%) but a lower HM (79.81\%) than the asymmetric pair (80.18\%)---their predictions are highly correlated, so the ensemble degenerates into redundancy. The asymmetric strong-weak pairing improves novel accuracy by $+0.88$, and dynamic fusion adds another $+0.15$ HM, validating that the value of a second branch comes from complementary roles and instance-adaptive weighting rather than from ensembling per se.
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
\caption{Ablation of the parameter fusion strategy.}
|
||||
\label{tab:abl_ewa}
|
||||
\small
|
||||
\renewcommand{\arraystretch}{1.15}
|
||||
\scalebox{0.92}{
|
||||
\begin{tabular}{l ccc}
|
||||
\toprule
|
||||
Method & B & N & HM \\
|
||||
\midrule
|
||||
Last-Epoch & 84.18 & 76.14 & 79.96 \\
|
||||
SWA~\cite{izmailov2018averaging} & 84.35 & 76.28 & 80.11 \\
|
||||
\rowcolor{tabhighlight}
|
||||
EWA (ours) & \textbf{84.52} & \textbf{76.54} & \textbf{80.33} \\
|
||||
\bottomrule
|
||||
\end{tabular}}
|
||||
\end{table}
|
||||
|
||||
\noindent\textbf{Parameter fusion strategy.}
|
||||
Table~\ref{tab:abl_ewa} compares the final-parameter selection. Uniform late averaging (SWA) gains $+0.15$ HM over the last epoch; EWA's Gaussian schedule centered at $\mu{=}15$ with $\sigma{=}1$ further reaches 80.33\% ($+0.37$ over last-epoch, $+0.22$ over SWA), showing that explicitly weighting the stable late phase smooths single-epoch fluctuation. In cross-domain and cross-dataset regimes, the schedule shifts early ($\mu{=}6$, $\sigma{=}10$), which is what preserves generic semantics in Tables~\ref{tab:xdo} and~\ref{tab:xda}.
|
||||
@@ -0,0 +1,6 @@
|
||||
\section{Conclusion}
|
||||
\label{sec:conclusion}
|
||||
|
||||
We presented DZGCoOp, a zero-shot guided dual-branch prompt learning framework for few-shot image classification. DZGCoOp distills LLM-enriched semantic anchors, zero-shot visual features, probability distributions, and an instance-level confidence from the frozen CLIP; learns an asymmetric strong-weak dual-branch network coupled through shared deep prompt layers; fuses the branches per instance with the zero-shot confidence; and regularizes optimization with feature- and distribution-level guidance losses plus an epoch-weighted averaging strategy. Experiments on base-to-novel, cross-domain, and cross-dataset transfer demonstrate consistent improvements over strong prompt learning baselines, and ablations quantify the contribution of each component. We believe that exploiting a frozen VLM's own predictive confidence to route between conservative and exploratory adaptation paths is a general recipe for knowledge-preserving few-shot learning, and we leave its extension to denser vision-language tasks as future work.
|
||||
|
||||
\noindent\textbf{Acknowledgments.} This work was supported by the National Natural Science Foundation of China (No. 62306320) and the Natural Science Foundation of Jiangsu Province (No. BK20231063).
|
||||
Reference in New Issue
Block a user