Artist identification of fine art paintings is a challenging problem primarily handled by art historians with extensive training and expertise. Many previous works have explored this problem by explicitly defining classification features. I train Convolutional Neural Networks (CNNs) with the goal of identifying the artist of a painting as accurately and precisely as possible. My dataset consists of 300 paintings per artist from 57 well-known artists. I train a variety of models ranging from a simple CNN designed from scratch to a ResNet-18 network with transfer learning. My best network achieves significantly higher classification accuracy than prior work. Additionally, I perform multiple experiments to explore and understand the learned representation of our networks.
Artist identification is the task of identifying the artist of a painting given no other information about it. This is an important requirement for cataloguing art, especially as art is increasingly digitized. One of the most comprehensive datasets, Wikiart, has around 150,000 artworks by 2,500 artist.
This paper has two key contributions:
1 Train a neural network that significantly outperforms existing approaches for artist identification on a large and varied dataset,
2. Explore and visualize the learned feature representation for identifying artists.
A large dataset of art compiled by Kaggle that is based on the WikiArt dataset. I split this dataset into training, validation, and test sets using a 80-10-10 split per artist.
ResNet-18 with Transfer Learning –
I have tested different network from simple CNN, ResNet-18 Trained from Scratch, to ResNet-18 but starts with pre-trained weights from the ImageNet dataset.
ResNet-18 with Transfer Learning shows the best result.
All of my models and experiments are implemented with using Tensorflow and Keras.
I used to set up the ResNet-18 architecture and to obtain weights for ResNet-18 pre-trained on ImageNet. All experiments are performed on a machine with 8 vCPUs and an NVIDIA GTX1080 ti GPU, and 200GB storage capacity.
Implementation Details – I trained all of our models using an Adam update rule. I explored using SGD with momentum, but obtained better results with Adam. For the two networks trained from scratch, I started with the default Adam parameters of learning rate = 10−3 , β1 = 0.9, and β2 = 0.999. We observed the accuracy and loss for both the training and validation datasets over the training epochs and decreased the learning rate by a factor of 10 if improvement slows down significantly. I initialized the weights of the convolutional layers of our networks based on as the methodology from this work is best practice when initializing networks that use a recitified linear activation function. For training the ResNet with transfer learning, we first held the weights of the base ResNet constant and updated only the fully-connected layer for a few epochs. We performed this step using the same default Adam parameters described previously. After network performance stopped improving, we allowed weights throughout the entire network to change but lower the learning rate to 10−4 . This allows for some change throughout the entire network to better fit our dataset. I experimented with varying levels of L2 regularization on all networks but did not see significant changes on validation set performance so in the end we used no regularization during training.
I introduce the problem of artist identification and apply a variety of CNN architectures to it to maximize classification accuracy, which has not been done in prior work. Our dataset consists of 300 paintings per artist for 57 artists over a wide variety of styles and time periods. Our best network, based on ResNet-18 pre-trained on ImageNet with transfer learning, outperforms traditional feature-based approaches by a significant margin. This network also outperforms training CNNs from scratch, indicating that features learned from the ImageNet training data are very relevant for artist identification as well. When asked to predict artist, our network created a representation of the style of paintings. We verify this through a variety of experiments looking at the underlying representation and how the network makes artist predictions.
Finally, I am implementing Ionic mobile framework to capture Painting and send to tensorflow serving for identification and display the artist name with accuracy.
ROBOTIC TECHNOLOGY & VIRTUAL ASSISTANTS
Author: Om Prakash Sharma
Status: Work In Progress