1、Learning to Self-Train for Semi-Supervised Few-Shot ClassificationXinzhe Li1 Qianru Sun2 Yaoyao Liu3 Shibao Zheng1 Qin Zhou4Tat-Seng Chua5Bernt Schiele61Shanghai Jiao Tong University 2Singapore Management University 3Tianjin University 4Alibaba Group5National University of Singapore 6Max Planck Inst

2、itute for Informatics, Saarland Informatics CampusAbstractFew-shot classification (FSC) is challenging due to the scarcity of labeled training data (e.g. only one labeled data point per class). Meta-learning has shown to achieve promising results by learning to initialize a classification model for

3、FSC. In this paper we propose a novel semi-supervised meta-learning method called learning to self-train (LST) that leverages unlabeled data and specifically meta- learns how to cherry-pick and label such unsupervised data to further improve performance. To this end, we train the LST model through a

4、 large number of semi-supervised few-shot tasks. On each task, we train a few-shot model to predict pseudo labels for unlabeled data, and then iterate the self-training steps on labeled and pseudo-labeled data with each step followed by fine-tuning. We additionally learn a soft weighting network (SW

5、N) to optimize the self-training weights of pseudo labels so that better ones can contribute more to gradient descent optimization. We evaluate our LST method on two ImageNet benchmarks for semi-supervised few-shot classification and achieve large improvements over the state-of-the-art method. Code

6、is at /xinzheli1217/learning-to-self-train.1 IntroductionTodays deep neural networks require large amounts of labeled data for supervised training and best performance 39, 8, 30. Their potential applications to the small-data regimes are thus limited. There has been growing interest in red

7、ucing the required amount of data, e.g. to only 1-shot 12. One of the most powerful methods is meta-learning that transfers the experience learned from similar tasks to the target task 3. Among different meta strategies, gradient descent based methods are particularly promising for todays neural net

8、works 3, 32, 27. Another intriguing idea is to additionally use unlabeled data. Semi-supervised learning using unlabeled data with a relatively small set of labeled ones has obtained good performance on standard datasets 21, 20. A classic, intuitive and simple method is e.g. self-training. It first

9、trains a supervised model with labeled data, and then enlarges the labeled set based on the most confident predictions (called pseudo labels) on unlabeled data 40, 35, 20. It can outperform regularization based methods 17, 6, 9, especially when labeled data is scarce.The focus of this paper is thus

10、on the semi-supervised few-shot classification (SSFSC) task. Specifi- cally, there are few labeled data and a much larger amount of unlabeled data for training classifiers. To tackle this problem, we propose a new SSFSC method called learning to self-train (LST) that successfully embeds a well-perfo

11、rming semi-supervised method, i.e. self-training, into the metaThis work was done during their internships mainly supervised by Qianru.Corresponding authors: .sg; .33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.gradient desce

12、nt paradigm. However, this is non-trivial, as directly applying self-training recursively may result in gradual drifts and thus adding noisy pseudo-labels 41. To address this issue, we propose both to meta-learn a soft weighting network (SWN) to automatically reduce the effect of noisy labels, as we

13、ll as to fine-tune the model with only labeled data after every self-training step.Specifically, our LST method consists of inner-loop self-training (for one task) and outer-loop meta-learning (over all tasks). LST meta-learns both to initialize a self-training model and how to cherry-pick from nois

14、y labels for each task. An inner loop starts from the meta-learned initialization by which a task-specific model can be fast adapted with few labeled data. Then, this model is used to predict pseudo labels, and labels are weighted by the meta-learned soft weighting network (SWN). Self-training consi

15、sts of re-training using weighted pseudo-labeled data and fine-tuning on few labeled data. In the outer loop, the performance of these meta-learners are evaluated via an independent validation set, and parameters are optimized using the corresponding validation loss.In summary, our LST method learns

16、 to accumulate self-supervising experience from SSFSC tasks in order to quickly adapt to a new few-shot task. Our contribution is three-fold. (i) A novel self- training strategy that prevents the model from drifting due to label noise and enables robust recursive training. (ii) A novel meta-learned

17、cherry-picking method that optimizes the weights of pseudo labels particularly for fast and efficient self-training. (iii) Extensive experiments on two versions of ImageNet benchmarks miniImageNet 36 and tieredImageNet 24, in which our method achieves top performance.2 Related worksFew-shot classifi

18、cation (FSC). Most FSC works are based on supervised learning. They can be roughly divided into four categories: (1) data augmentation based methods 15, 29, 37, 38 generate data or features in a conditional way for few-shot classes; (2) metric learning methods 36, 31, 33 learn a similarity space of

19、image features in which the classification should be efficient with few examples; (3) memory networks 18, 28, 22, 16 design special networks to record training “experience” from seen tasks, aiming to generalize that to the learning of unseen ones; and (4) gradient descent based methods 3, 4, 1, 23,

20、11, 7, 42, 32, 14 learn a meta-learner in the outer loop to initialize a base-learner for the inner loop that is then trained on a novel few-shot task. In our LST method, the outer-inner loop optimization is based on the gradient descent method. Different to previous works, we propose a novel meta-l

21、earner that assigns weights to pseudo-labeled data, particularly for semi-supervised few-shot learning.Semi-supervised learning (SSL). SSL methods aim to leverage unlabeled data to obtain decision boundaries that better fit the underlying data structure 20. The -Model applies a simple consistency re

22、gularization 9, e.g. by using dropout, adding noise and data augmentation, in which data is automatically “l(fā)abeled”. Mean Teacher is more stable version of the -Model by making use of a moving average technique 34. Visual Adversarial Training (VAT) regularizes the network against the adversarial per

23、turbation, and it has been shown to be an effective regularization 17. Another popular method is Entropy Minimization that uses a loss term to encourage low-entropy (more confident) predictions for unlabeled data, regardless of their real classes 6. Pseudo-labeling is a self supervised learning meth

24、od that relies on the predictions of unlabeled data, i.e. pseudo labels 2. It can outperform regularization based methods, especially when labeled data is scarce 20 as in our envisioned setting. We thus use this method in our inner loop training.Semi-supervised few-shot classification (SSFSC). Semi-

25、supervised learning on FSC tasks aims to improve the classification accuracy by adding a large number of unlabeled data in training. Ren et al. proposed three semi-supervised variants of ProtoNets 31, basically using Soft k-Means method to tune clustering centers with unlabeled data. A more recent w

26、ork used the transductive propagation network (TPN) 13 to propagate labels from labeled data to unlabeled ones, and meta-learned the key hyperparameters of TPN. Differently, we build our method based on the simple and classical self-training 40 and meta gradient descent method 3, 32 without requirin

27、g a new design of a semi-supervised network. Rohrbach et al. 25 proposed to further leverage external knowledge, such as the semantic attributes of categories, to solve not only few-shot but also zero-shot problems. Similarly, we expect further gains of our approach when using similar external knowl

28、edge in our future work.115WIYHS PEFIPMRK7I XVEMRMRK:EP 8IWXFigure 1: The pipeline of the proposed LST method on a single (2-class, 3-shot) task. The prototype of a class is the mean feature in the class, and SWN is the soft weighting network whose optimization procedure is given in Figure 2 and Sec

29、tion 4.2.3 Problem definition and denotationIn conventional few-shot classification (FSC), each task has a small set of labeled training data called support set S, and another set of unseen data for test, called query set Q. Following 24, we denote another set of unlabeled data as R to be used for s

30、emi-supervised learning (SSL). R may or maynot contain data of distracting classes (not included in S).Our method follows the uniform episodic formulation of meta-learning 36 that is different to traditional classification in three aspects. (1) Main phases are meta-train and meta-test (instead of tr

31、ain and test), each of which includes training (and self-training in our case) and test. (2) Samples in meta-train and meta-test are not datapoints but episodes (SSFSC tasks in our case). (3) Meta objective is not to classify unseen datapoints but to fast adapt the classifier on a new task. Lets det

32、ail thedenotations. Given a dataset Dfor meta-train, we first sample SSFSC tasks T from a distributionp(T ) such that each T has few samples from few classes, e.g. 5 classes and 1 sample per class. Thas a support set S plus an unlabeled set R (with a larger number of samples) to train a task-specifi

33、c SSFSC model, and a query set Q to compute a validation loss used to optimize meta-learners. For meta-test, given an unseen new dataset Dun, we sample a new SSFSC task uTn. “Unseen” means there is no overlap of image classes (including distracting classes) between meta-test and meta-traintasks . We

34、 first initialize a model and weight pseudo labels for this unseen task, then self-train themodel on Sun and Run. We evaluate the self-training performance on a query set Qun. If we have multiple unseen tasks, we report average accuracy as the final evaluation.4 Learning to self-train (LST)The compu

35、ting flow of applying LST to a single task is given in Figure 1. It contains pseudo-labeling unlabeled samples by a few-shot model pre-trained on the support set; cherry-picking pseudo-labeled samples by hard selection and soft weighting; re-training on picked “cherries”, followed by a fine- tuning

36、step; and the final test on a query set. On a meta-train task, final test acts as a validation to output a loss for optimizing meta-learned parameters of LST, as shown in Figure 2.4.1 Pseudo-labeling & cherry-picking unlabeled dataPseudo-labeling. This step deploys a supervised few-shot method to tr

37、ain a task-specific classifier on the support set S. Pseudo labels of the unlabeled set Rare then predicted by . Basically, we can use different methods to learn . We choose a top-performing one meta-transfer learning(MTL) 32 (for fair comparison we also evaluate this method as a component of other

38、semi-supervised methods 24, 13) that is based on simple and elegant gradient descent optimization 3. In the outer loop meta-learning, MTL learns scaling and shifting parameters ss to fast adapt a large-scale pre- trained network (e.g. for 64 classes and 600 images per class on miniImageNet 36) to a

39、new learning task. In the inner loop base-learning, MTL takes the last fully-connected layer as classifier and trains it with S.In the following, we detail the pseudo-labeling process on a taskT . Given the support setS, its loss is used to optimize the task-specific base-learner (classifier) by gra

40、dientdescent:t t1 t1 L S; ss, t1 ,(1)where t is the iteration index and t1, ., T .The initialization 0 is given by 0 which is meta- learned (see Section 4.2). Once trained, we feed T with unlabeled samplesR to get pseudo labels Y R as follows,Y R = fss,T (R),(2)where f indicates the classifier funct

41、ion with parameters T and feature extractor with parametersss (the frozen is omitted for simplicity).Cherry-picking. As directly applying self-training on pseudo labels Y R may result in gradual drifts due to label noises, we propose two countermeasures in our LST method. The first is to meta-learn

42、the SWN that automatically reweighs the data points to up-weight the more promising ones and down-weighs the less promising ones, i.e. learns to cherry-pick. Prior to this step we also perform hard selection to only use the most confident predictions 35. The second countermeasure is to fine-tune the

43、 model with only labeled data (in S) after every self-training step (see Section 4.2).Specifically, we refer to the confident scores of Y R to pick-up the top Z samples per class. Therefore, we have ZC samples from C classes in this pseudo-labeled dataset, namelyR p. Before feedingR pto re-training,

44、 we compute their soft weights by a meta-learned soft weighting network (SWN), inorder to reduce the effect of noisy labels. These weights should reflect the relations or distances between pseudo-labeled samples and the representations of C classes. We refer to a supervised method called RelationNet

45、s 33 which makes use of relations between support and query samples for traditional few-shot classification.First, we compute the prototype feature of each class by averaging the features of all its samples. In the 1-shot case, we use the unique sample feature as prototype. Then, given a pseudo-labe

46、led sample (xi, yi) p, Rwe concatenate its feature with C prototype features, then feed them to SWN. Theweight on the c-th class is as follows,w= fhf(x );Pfk ss(xc,k) i,(3)i,cswnssiKwhere c is the class index and c 1, ., C, k is the sample index in one class and k 1, ., K, xc,k S, and swn denotes th

47、e parameters of SWN whose optimization procedure is given in Section 4.2. Note that wi,c have been normalized over C classes through a softmax layer in SWN.4.2 Self-training on cherry-picked dataAs shown in Figure 2 (inner loop), our self-training contains two main stages. The first stage contains a

48、 few steps of re-training on the pseudo-labeled data Rp in conjunction with support set S, and the second are fine-tuning steps with only S.We first initialize the classifier parameters as 0 0, where 0 is meta-optimized by previous tasks in the outer loop. We then update 0 by gradient descent on Rp

49、and S. Assuming there are T iterations, re-training takes the first 1 m iterations and fine-tuning takes the rest m + 1 T . For t 1, ., m, we havet t1 5 t1 L S Rp; swn, ss, t1 ,(4)2XWHU ORRS,QSXW DQ HSLVRGH,QQHU ORRS5H WUDLQWXIT 5H WUDLQWXIT Q)LQHWXQHWXIT Q )LQHWXQHWXIT 8PHWD XSGDWHGHSORPHWDFigure 2

50、: Outer-loop and inner-loop training procedures in our LST method. The inner loop in thered box contains the m steps of re-training (withS and Rp) and T m steps of fine-tuning (with only S). In recursive training, the fine-tuned T replaces the initial MTL learned T (see Section 4.1) for the pseudo-l

51、abeling at the next stage.where is the base learning rate. L denotes the classification losses that are different for samples from different sets, as follows,L S R ; (5)swnss tp, , =Lce fswn,ss ,t(xi), yi), if (xi, yi) S,Lce wi f swn,ss ,t (xi), yi), if (xi, yi) Rp,c=1where Lce is the cross-entropy

52、loss. It is computed in a standard way on S. For a pseudo-labeled sample in Rp, its predictions are weighted by wi = wi,cC before going into the softmax layer.For t m + 1, ., T , t is fine-tuned on S ast t1 5t1 L(S; swn, ss, t1).(6)Iterating self-training using fine-tuned model. Conventional self-tr

53、aining often follows an iterative procedure, aiming to obtain a gradually enlarged labeled set 40, 35. Similarly, our method can beiterated once a fine-tuned model T is obtained, i.e. to use T to predict better pseudo labels on Rand re-train again. There are two scenarios: (1) the size of R is small

54、, e.g. 10 samples per class, so that self-training can only be repeated on the same data; and (2) that size is infinite (at least bigenough, e.g. 100 samples per class), we can split it into multiple subsets (e.g. 10 subsets and each one has 10 samples) and do the recursive learning each time on a n

55、ew subset. In this paper, we consider the second scenario. We also validate in experiments that first splitting subsets and then recursive training is better than using the whole set for one re-training round.Meta-optimizing swn, ss and 0. Gradient descent base methods typically use T to compute the

56、 validation loss on query setQused for optimizing meta-learner 32, 3. In this paper, we havemultiple meta-learners with the parameters swn, ss and 0. We propose to update them by thevalidation losses calculated at different self-training stages, aiming to optimize them particularly towards specific

57、purposes. ss and 0 work for feature extraction and final classification affecting on the whole self-training. We optimize them by the loss of the final model T . While, swn produces soft weights to refine the re-training steps, and its quality should be evaluated by re-trained classifier m. We thus use the loss of m to optimize it. Two optimization functions are as follows,swn=: swn 1 5swn L(Q; swn, ss, m),(7)ss, 0 =: ss, 0 2 5ss ,0 L(Q; swn, ss, T ),(8) where 1 and 2 are meta learning rates that are manually set in experiments.