[딥러닝/이미지 처리] EfficientNet 모델 개요 및 적용

EfficientNet 모델 개요

• EfficientNet is deep learning architecture designed by Google(first introduced in Tan and Le, 2019) to tackle the problem of scaling Neural Networks (deciding how to best increase model size and increase accuracy).
• Given that there is a tradeoff between efficiency and accuracy in scaling CNNs, the idea by Google is to provide better accuracy and improve the efficiency of the models by reducing the parameters and FLOPS (Floating Point Operations Per Second) manifold.
• EfficientNet,  is among the most efficient models (i.e. requiring least FLOPS for inference) that reaches State-of-the-Art accuracy on both imagenet and common image classification transfer learning tasks.
• The smallest base model is similar to MnasNet, which reached near-SOTA with a significantly smaller model.
• By introducing a heuristic way to scale the model, EfficientNet provides a family of models (B0 to B7) that represents a good combination of efficiency and accuracy on a variety of scales.
• Such a scaling heuristics (compound-scaling, details see Tan and Le, 2019) allows the efficiency-oriented base model (B0) to surpass models at every scale, while avoiding extensive grid-search of hyperparameters.

 

latest updates of various augmentation schemes and semi-supervised learning approaches are applied on the model to further improve the imagenet performance of the models.

These extensions of the model can be used by updating weights without changing model architecture.

 

모델 사용 권장 사항

resolution, depth, width 3가지에 대한 결정은 여러 요인의 영향을 받을 수 있다

• Resolution(Image Size): input image의 resolution은 8 또는 16의 배수여야 함 (특히 작은 모델에서 중요, 그래서, 모델 B0와 B1의 input resolution이 224 와 240로 권장되는 것). 그렇지 않으면, 몇몇 layer의 boundary에서의 zero-padding이 computation resourse를 많이 잡아먹게 된다.  

EfficientNet 각 모델별 권장 resolution

• Depth and width: EfficientNet의 building blocks는 8의 배수의 channel size를 필요로 한다

• Resource limit: depth와 width가 계속 증가할 수 있는 상황에서, 메모리 한계(하드웨어적인 문제)로 인한 resolution의 병목 현상이 일어난다면, 이런 상황에서는, resolution은 고정시키고 depth 또는 width를 넓히는 것이 낫다

As a result, the depth, width and resolution of each variant of the EfficientNet models are hand-picked and proven to produce good results, though they may be significantly off from the compound scaling formula

 

적용

input image의 shape은 (img_size, img_size, 3)이어야 하며, 범위는 [0, 255] 이내여야 한다

Normalization은 모델 내부에서 자동적으로 이루어진다

Replacing the top layer with custom layers allows using EfficientNet as a feature extractor in a transfer learning workflow.

기존 EfficientNet의 마지막 Dense layer는 1280개의 feature를 penultimate layer에 넣어 class 1000개에 대한 prediction이 되도록 설정되어있지만, 실제 적용에 있어서는 각자 사용하는 클래스 개수만큼으로 조정해주어야 한다. 

 

NUM_CLASSES = 3

# Load in pretrained effnet model and remove its head, replacing it with fully connected layer that gives 3 outputs 
def get_model(model_name='efficientnet-b0', NUM_CLASSES):
    model = EfficientNet.from_pretrained(model_name)

    del model._fc
    model._fc = nn.Linear(1280, NUM_CLASSES)
    
    return model

CIFAR-100(32x32 크기)와 같이 resolution이 작은 데이터셋에서는 overfitting 문제가 발생할 수 있다. 

 

Hence training from scratch requires very careful choice of hyperparameters and is difficult to find suitable regularization. It would also be much more demanding in resources.

 

부가 사항

EfficientNet은 아래와 같이, input 이미지의 크기를 맞춰주어야 한다.
Recommended Image Size보다 작은 이미지 데이터를 학습시키고 싶을 때



1) 이미지 Resize 진행



2) Conv/Pooling step을 조정



관련 내용 : https://github.com/lukemelas/EfficientNet-PyTorch/issues/42




위 내용에 따르면, 32 x 32 크기의 CIFAR 데이터셋을 기준으로 학습시킨 결과,



1) Efficientnet B0 모델 : 87.13% accuracy (B1, B3, B7 모델 사용 시, accuracy는 더 하락)



2) ResNet50: 94~96 % accuracy

 

Reference

Keras documentation: Image classification via fine-tuning with EfficientNet

 

EfficientNet: Scaling of Convolutional Neural Networks done right