DeepVisInterests : deep data analysis for topics of interest prediction

Deep data analysis for latent information prediction has become an increasingly important area of research. In order to predict users’ interests and other latent attributes, most of the existing solutions (works, studies) have used textual data and have obtained accurate results. However, little attention has been paid to visual data that have become increasingly popular in recent years. This paper addresses the problem of discovering the attributed interests and analyzing the performance of the automatic prediction by a comparison with the self-assessed topics of interests (topics of interests provided by the user in a proposed questionnaire) based on data analysis techniques applied to users’ visual data. We analyze the content of individual images and aggregate the image-level information to predict user-level interest distribution. Thus, we employ a Convolutional Neural Network architecture pre-trained on ImageNet dataset for feature extraction. The suggested system is based on the construction of users’ interests ontology in order to learn the semantic representation for the popular Topics of interests defined by social networks (e.g., Facebook). Our experiments show that the analysis enhances the overall prediction performance. We set forth a novel evaluation database to improve our framework’s robustness and enhance its ability to generalize to new user profiles. Our proposed framework has shown promising results. Our method yields a competitive accuracy of 0.80 compared with the state of the art techniques.


Introduction
The last decades have witnessed a boom in deep data analysis techniques with the huge amount of textual and visual data provided by various social networks (e.g., Facebook).Data analysis techniques are useful for enhancing prediction systems and achieving good results [1].These systems have achieved remarkable advances using deep learning models that performed better accuracy than the traditional machine learning methods.These advances have led to renewed efforts on social network analysis [4].Hence, social media data management and analysis (e.g, data extracted from Facebook) represent one of the major challenges faced by diverse applications [36].
To this end, our work examined data analysis techniques by combining convolutional neural network architectures and semantic knowledge representations in order to model the users' ontological profiles and predict their interest distributions.In this paper, we focused on Convolutional Neural Network (CNN) architectures, pre-trained on ImageNet dataset [16], which have become popular as a highly recommended feature descriptor in several computer vision areas [2].
In contrast to the approach proposed by by [36], that presents a novel approach to predict the emotions embedded in social images by combining image visual features and users' demographic information( gender, marital status, occupation), our work aims to predict the users' topics of interest through their shared images within 24 classes predefined by Facebook.Furthermore, the authors in [36] used the factor graph-based model to present not only visual features, but also temporal and social correlation functions.In our work, we used an ontology-based semantic model to display visual features obtained by the convolutional neural network architecture used in object detection.However, our proposed framework called DeepVisInterest performs the users' interest prediction task based on a deep neural approach for the ontology construction.A list of interest topics is illustrated in Table 1.In fact, these topics are predefined by Facebook to determine the most important topics of interest categorized on Facebook using various daily shared data.
We improved not only classification accuracy, but also the ability to handle sophisticated presentation attack conditions.Consequently, this would greatly hinder the effectiveness of users' interest prediction systems [7].The major contributions of this research are summarized as follows: • Developing a novel framework named DeepVisInterests that performs the users' interest prediction task using a pre-trained CNN on ImageNet.• Designing a new ontology using a set of deep visual features in order to learn the semantic representation for the popular topics of interests.• Constructing a novel database for assessing the generalization ability of the system.The remainder of this paper is structured as follows.In Section 2, we review the related works suggested in the literature.In Section 3, we illustrate our proposed database.In Section 4, we provide a comprehensive explanation of our suggested approach in further detail of each phase.In Section 5, we present the conducted experiments and summarize the key findings of the presented work.In Section 6 we discuss the obtained results of our designed database.We finally draw some conclusions in Section 7.

Relevant works
Several deep data analysis methods have been proposed in the literature.Due to the rapid development of deep learning techniques, data analysis has become more challenging to discover various latent user attributes, especially users' interests [5].In fact, the users' interest discovery process passes through two major phases, namely the modelling phase and the prediction phase.In fact, the efficiency of the modeling phase is crucial for predicting users' interests [14,15].

Users' interest modeling
In [24], a Latent Topic of user interest (LUI) model was proposed in order to manage the topics distribution of tweets which possess non-Gaussian characteristics.To evaluate their model, the authors employed two microblogs Weibo and Twitter to construct two datasets that contain 10 million tweets and 100 million tweets respectively and they obtained a correlation coefficient between topics of 64.0 % and 63.0% for each mentioned dataset. .
Another work [3] set forth a user interest model based on Latent Dirichlet Allocation to model forum topics and distinguish user's serious and unserious interest topics.To validate their model, the authors used a forum thread from Tianya and achieved an accuracy of 80.5% and 93.3% for serious and unserious users respectively.Also, Yang et al. [38] proposed a framework that integrates users' behavior on multiple aspects (opinions and preferences).The component infer numerical ratings on the multiple aspects when such ratings are missing or not explicitly presented.To construct the user model, they firstly used the LDA approach to cluster each aspect term into laten aspect.Secondly, the tensor factorization approach was applied to automatically extract weights of various aspects while calculating an overall numerical rating.Finally, a simple algorithm was used to compute the overall rating of an item based on both aspect weight.To validate their model, the authors made use of two real datasets: The Internet Movie Database (IMDB) containing 193,266 reviews written by 83,585 users from the Inter net Movie Database website and the Hotel review Database including 81,085 reviews from 879 users.
In [30], the authors exploited the users' social data for developing aspects based on a sentiment analysis framework.This framework applied the neighborhood-based CF algorithm, KL-divergence and multidimensional Euclidean Distance in order to model the users' social data.To evaluate their offered model, Musto et al. [30] utilized three different databases, namely Yelp with 11,537 reviews from 45,981 users, TripAdvisor with 3,954 reviews from 536,932 users and Amazon with 50,210 reviews from 826,773 users.

Users' interest prediction
Considerable research efforts have been made to exploit the information provided by social network posts using data analysis and deep learning techniques in users' interest prediction.In [41], the authors put forward a novel approach for the image and group-level label propagation for users' interest prediction.They employed the AlexNet architecture for deep visual feature extraction and image-level similarity to propagate the label information between images in order to disseminate the topics of interest-level knowledge for all users' images.To validate their approach, the authors used a novel database collected from Pinterest containing 6000 images of 300 users' accounts and they obtained an accuracy of 43.0%.
In [40], the authors recommended a method of assessing the following suggestions from social users based on categorical classification interests.This method relies on the convolutional neural network architectures and a hierarchical topics of interests categorization.To validate their approach, the authors used a database comprising 20,500 images collected from Pinterest and they achieved an overall precision of 39.9%.
A social image classification was presented in [11] using unsupervised learning algorithms for users' interest prediction.To test the validity of the proposed method, the authors used a novel database involving 800 social images gathered from Pinterest and they obtained an accuracy of 68.0%.
In [45], the authors used a Click-through rate (CTR) prediction system to estimate the probability of a user clicking on the recommended item.They captured the latent user interest behind the user behavior data.An extractor layer was proposed to identify temporal interests from history behavior sequence.To measure the originality and performance of the suggested system, the authors used both public and industrial datasets.They achieved a 20.7% performance improvement on CTR.
For both user click prediction and user interest modeling, the authors in [9] set forth two different deep learning-based frameworks, namely LSTMcp and LSTMip based on the long short-term memory (LSTM) network in order to learn latent features representing user interests.To assess their proposed approach, the authors gathered real-world data Table 2 illustrates the topics of interests used by the aforementioned related works.

Discussion
The performance of the users' interest prediction model requires a deep understanding of users' social data.Based on the state-of-the-art research, this prediction has been obtained by analyzing explicit social data.Previous studies have examined the performance of ontologies and deep learning techniques for objects detection by mining users' latent information.Such deeper understanding of users' social visual data led to more accurate prediction results.Modelling users' interests represents the starting point for interest prediction.The LDA approaches presents users' data with a multi-nominal distribution distribution of words and documents while TF-IDF considers the ratio of term frequency of each word to the total number of terms as a term-topic-probability.These methods achieve better accuracy in various research fields.One limitation of these methods, however, is data representation based on the bag-of-Words method.However, bag-of-Words is indeed a bad approach to understand sentences due to some weaknesses:: • They lose the ordering of words, • They ignore the semantics of the words, • They do not consider the semantic relations between words.
Concerning the users' interest prediction, the topic modeling method exploits social data in order to predict latent information about social users such as topics of interests.Several studies have been extensively conducted for predicting users' latent information.Their primary objective is to classify social data: textual or visual data into some classes, according to the area of prediction (sentiments, opinions, preferences, etc) in order to understand social users' behaviors.
Then, several attempts have been made to classify.Despite their outstanding performance, previous studies have some limitations, such as using Facebook as a instead of Pinterest.In fact, few works has been carried out using social visual data taken from a Facebook user profile in order to develop supervised deep learning for user interest prediction.Facebook is one of the most popular social networks for different users.It is an effective tool to share huge of data daily.Also, recent research efforts have been made to exploit social data by adopting deep learning approaches in this prediction and referring to the previous works that used not only traditional topic modeling methods (e.g.LDA), but also feature extraction methods (e.g.AlexNet architecture).

VisualDatabase: pictures and interest
Social network data are considered as sensitive data because they reflect the private life of social media users.Therefore, to build a social database, we require permission to access a social media user's photos.In this way, users' private lives will be respected.To this end, an application has been created [18] to allow users to voluntarily subscribe, and therefore, permit us to gather the data posted on social media sites (e.g., Facebook) accounts/pages.Moreover, only abstract images with natural scenes and neutral themes have been selected to avoid personal images/pictures of the user's family or friends.
Our database, called VisualDatabase, was created in March 2018.This database contains a set of multiple social images from 240 accounts, also known as social users.For each social user, 100 images are randomly selected from the "liked" and "shared" photos.The image database has a 320*320 pixel resolution, with multiple ethnic identities and locations (Africans) different age groups (varying from 15 to 60 years old).Figure 1 shows some image samples.Furthermore, the database presents the users' self-assessed interests based on a well-structured Big Interest questionnaire (BI) filled out voluntarily by each user.The result of the BI is a vector where each component indicates the disposition of a user with respect to the core topics presented in Table 1.The self-assessed traits are examined to be the validated user's' core interest.

Proposed approach
The problem of users interest prediction may be considered as an image classification problem.However, in constrat to the traditional image classification where the objective is to maximize classification performance at individual level, we are based more on learning the overall user-level image distribution.Our proposed framework, named DeepVisInterest, is illustrated in Fig. 2 and is based essentially on users' interest modeling phase followed by the prediction phase.
The main novelties of this work are as follow:: • Our work highlights a combination of convolutional neural network (CNN) object detection architectures and our ontology to model the 24 topics of interests predefined by Facebook.• 4 basic CNN architectures ( LeNet, AlexNet, VGG19 and GooglNet) are compared.
• To predict topics of interest based on social visual data extracted from Facebook • The new social visual database named "VisualDatabase" could contribute to the com- puter vision research community.To the best of our knowledge, this is the only public social image dataset available for research on the detection of topics of interest.

Phase 1: users' interest modeling
The ontology have been conceived to alllow for a common definition if concepts, entities, relationships, situations and events and consequently for common understanding and for promoting information exchange.In fact, we observe that ontologies have been used in deep learning techniques to address uncertainty in image classification based on object detection methods [10].Based on this observation, ontologies are exploited for a user interest model to classify users' images relevant to 24 topics that are already presented in Table 1, and predict the user's topics of interest.An overview of our constructed ontology is illustrated in Fig. 3.In our proposed ontology, 24 topics represent sub-concepts.We used 24 benchmark database to exhibit the 24 topics (each database contains approximately 200 images).
We took an unannotated image set as input.We also employed the pre-trained ImageNet CNN architecture for object detection in order to detect the top 5 ranked objects, among 1000 objects from ImageNet, that will be integrated in the ontology as end-concepts.The latter are related to sub-concepts by using the object property "is-a".Krizhevsky et al. [17] set forth a CNN architecture that has obtained the state-of-the-art performance in the challenging ImageNet classification task.We direct the reader to the original paper for a more detailed description of this architecture.In fact, ontology implementation features represent the number of concepts, data properties, object properties, individuals and axioms.
To build our ontology, we used the tool named Protégé version 4.3.This tool provides Resource Description Framework (RDF) schemas and XML scripts by using a web ontology.
The Protégé is an ontology development environment with a huge number of active users.This ontology has been extended with support for OWL (Web Ontology Language) and has become one of the leading owl tools [28].Our recommended ontology applies all the implementation features and the object properties are used to define the relationships between individuals.

Fig. 3 Users' interest ontology construction based on CNN architectures
However, this ontology should be evaluated from multiple perspectives to determine its quality prior to use or reuse.Metrics, defined in [26], are used to measure the quality of the ontology in different dimensions.

Users' interest ontology vocabulary
We define the basic metrics for the size of the users' interest ontology in various aspects.The size of our ontology is defined as follows: Let O be the users' interest ontology:

Structure of users' interest ontology
Structural metrics are the most immense examining metrics in the ontology presentation, exactly, cohesion metrics that measure the degree of relatedness between concepts.
Among the cohesion metrics, we find the relation-based structural complexity.In fact, for each r ∈ R we have some few structural metrics such as: For the users interests ontology, the "is-a" relation based structure metrics are:

Context of users' interest ontology
We focus on users' interest prediction.They are interested in the ontology as a perfect tool for modeling the semantics of topics of interests [21].
Let assume that a user U possesses n topics of interests UI 1 , ..., UI n which contain a set of concepts C i,1 , ..., C i,n and a set of attributes A j,1 , ..., A j,k .
An ontology for describing the semantics of a topic of interest consists of the following expressions: • EXP UI i which describes the functionality of the topic UI i , • EXP C i,n which defines the meaning of the parameter C i,j in the set of concepts, • EXP Y i,k which illustrates the meaning of the parameter Y i,k , in the set of attributes of each concept.

Semiotic metrics assessment
The quality of each ontology is defined across a set of semiotic metrics.These metrics assess the syntactic, semantic, pragmatic and social aspects of ontology quality [22].Then, we use Protégé-OWL-5.2 which is an open-source platform that provides tools to construct domain models and knowledge-based applications with ontology [35,37].Figure 3 illustrates the integration of CNN architecture outputs in our ontology.Each input image is represented by a vector containing a set of scores of the top 5 concepts among 1000 concepts from ImageNet [6].The ontology vectorization method is used to find each user's interest using OWL API.

Benchmarks
To model users' interests , we constructed our ontology based on twenty-four publicly available databases, that represent the twenty-four topics mentioned in Table 1, such as: Food Image database [32], Sport Event Database [23] and DeepFashion Database [25].
They are challenging databases, since they consist of various types of users' behaviors with different image qualities (high definition, average, and very low quality).In the following section, we will offer a detailed description of these databases.

Food image database
This database comprises 568 food images including sweets (e.g., ice cream, chocolate), savory (e.g., pistachios, sandwiches), processed (e.g., hamburger, French fries, potato chips, chocolate bars) and whole foods (e.g., vegetables and fruits) and beverages (e.g., coffee, orange juice).Images were selected from a commercially available database (Hemera Photo Objects).All images are color photographs with a resolution of 600 × 450 pixels.

Sport event database
The Sport Event database contains 600 images classified into 6 sport event categories, namely rowing, badminton, polo, bocce, snowboarding, croquet, sailing, and rock climbing.Images are divided into easy and medium according to the human subject judgement.Each image provides some information about the distance of the foreground objects.All images are color photographs with a resolution of 600 × 450 pixels.These images are collected from non-copyrighted sources on the internet.

DeepFashion database
The DeepFashionDB includes 800 different fashion images ranging from well-posed shop images to unconstrained consumer photos.All images are color photographs with a resolution of 600 × 450 pixels.

Phase 2: users' interest predicting
Our prediction phase implements the construction of a VisualDatabase to propagate topics of interest distribution from image-level distribution to user-level distribution.

Visual users' interest prediction
Visual users' interest prediction is based on the combination between the image-level and user-level (VUIP-IL/UL) methods.Figure 4 illustrates the main steps of our proposed method named VUIP-IL/UL.This method is based on 3 prominent steps.In the first step, the CNN architectures are pre-trained on the ImageNet database to extract deep visual features from VisualDatabase.These features display image objects with their probabilities.Thus, by inferring our user interest ontology (UIO) model, each image object is replaced by its corresponding super-concept.In the second step, the probability of occurrence-based scoring mechanisms are applied to obtain an image-level distribution.In the third step, we build a mapping matrix from the image-level distribution to the user-level distribution.

Scoring users' interests
To quantify the user's interests, we use a scoring function to determine the weights of each topic.This function is very powerful as the user's interest scores will be applied to determine the adopted interest distribution for each user's image and therefore, for each user.We focus the probability of occurrence-based scoring mechanisms.The topic score of each image i ∈ I posted by a given user u ∈ U , can be measured by the probability of occurrence of an object o ∈ O where an image is represented by a collection of objects O.

S(u, I )
where p oi ∈ P I , oc oi ∈ OC I .
Here, P I is a set of probabilities within each i ∈ I obtained by each image' object and OC I is a set of occurrences with object o ∈ O for the given image.Algorithm 1 demonstrates the detailed steps used in our prediction task.

Image-level users' interest distribution
After applying the feature extraction task, each image possesses 5 objects with their probabilities, namely (espresso, 0.08), (cup, 0.07), (dough, 0.06), (ladle, 0.05) and (sandal, 0.04).Using the Fact++ Reasoner and SPARQL query, we infer the users' interest ontology to obtain the super-class for each image object.We use the data property "has-Instance" in order to generate the super-class for each input image object presented as an ontology instance.This step applies both the Fact++ reasoner and the SPARQL query: SELECT For further details, see Algorithm 2. Accordingly, we describe two matrices G R n * 24 and G R n * 24 to be the affinity matrices between the twenty-four core topics of interest and the n shared images by a specific user as u U .

User-level interest distribution
According to the explained image-level distribution, each user u ∈ U possesses two weighted matrices G and G for n shared images in social networks (e.g., Facebook).At this level, we intend to generate the target user's interest distribution matrix based on the two scoring mechanisms: (a) as it first treats the matrix G, we define: Algorithm 2 Image-level for the users' interest distribution.
where S(u, t k )is the user's score u about the topic t with p i,k is the probability of image i for topic k and n is the number of shared images.(b) as the second mechanism treats the matrix G , we define for k= (1,24) and i=(1, n): where S(u, t k ) is the user's score u about topic t with p i,k as the probability of image i for topic k and n as the number of shared images.For further details, see Algorithm 3.
Algorithm 3 User-level for the users' interest distribution.

Experimental analysis and discussion
In this study, our aim is to investigate the impact of visual data on the users' interest prediction problem.To this end, we led an intensive experimental study by generating multiple models based on different image combinations.The obtained results are reported based on each combination.We use the publicly available implementation Caffe [13] to test our model.All our experiments are evaluated on a Linux X86-64 machine with 32 GRAM.

Correlational study between topics of interests
This study determines the dependency between various topics already mentioned in Table 1 using the Pearson Moment correlation.This coefficient is commonly represented by p(rho) with: where Cov is the covariance, σ x is the standard deviation of X and σ y is the standard deviation of Y .
As indicated in Fig. 5, firstly, we notice that the topic Food is high total positive correlated with the topic Drink with a value greater than 0.5.This high correlation means that the images containing objects belonging to the super-class Food, in our UIO ontology, may contain objects belonging to the super-class Drink or Family or People.Furthermore, the topics "Hobbies" and "activities" are very similar in term of image objects which can be categorized into two main classes.To this end, they are highly correlated, as shown in Fig. 5. Secondly, we observe that the topic Fashion is high negative correlated with the topic Technology with a value of −0.553 which means that the user who has Fashion as a topic of interest can never be interested in the topic of Technology.Finally, the topic Education, for example, is correlated with the topic Culture with a value of 0.033.

Image-level for users' interest distribution study
To illustrate this level, we propose a demonstration about the Family class.According to Fig. 6, we notice that this distribution is articulated around four topics, namely Outdoors, Drink, People and Food.These results validate the positive correlation between these topics and the fact that a user who is interested by the topic Family, he/she may share images related to these discovered topics.For the same class, Fig. 7 illustrates this distribution level using 100 shared images per user.In fact, with 100 images, we remark that the images distribution has become more detailed with the appearance of new topics with low scores  such as Places and Entertainement and the disappearance of the self-assessed topic for some users.This appearance and disappearance is explained by the diversity of images that can generate vectors with low scores for several topics.To conclude, we can assume that in social networks (e.g., Facebook), each user may share a set of images which can be related or not related to his/her self-assessed topic of interest.With 50 images, the distribution is generally very close to reality with the appearance of self-assessed topics with the most important score.With 100 images, this distribution has become more detailed with new topics assigned to those correlated with self-assessed topics for each user.

User-level for the users' interest distribution study
At this level, we attempt to define the confidentiality area of the users' shared images.For this reason, we used k shared images by each user within the 24 classes.According to each class, we obtained a confidential area which generated the target users' interest matrix  with a high score for the self-assessed topic.Tables 3 and 4 describes the variation of the accuracy measure for each class according to the number of shared images per user.This variation is explained by to the fact that each user's topics of interests distribution consists of three layers: Starting term with 5 images, Middle term with 10 images, Long term with 50 images, Very Long term with 75 images and Extreme term with 100 images.
• The starting term presents the sharing of the first 5 images that each user chooses.These images indicate their self-assessed topics of interest, • For the middle term, the same user may be influenced by other topics and can share some images that can interrupt our classification, which explains the decrease of the system performance from 0.85 to 0.75, • In a Long term, after being biased in the middle term, the user settles back into his/her self-assessed topic of interest and our system predicts the correct target class with 50 shared images to obtain an accuracy of 0.95, • In a Very Long term, the user keeps a stability with a slight disturbance of his/her dis- tribution obtained in the long term.For this reason, our system shows a slight decrease in performance with an accuracy of 0.80, • In a Extreme Long term, our system performance undergoes a very remarkable decrease with an accuracy of 0.65, which validates that beyond 50 images, the distribution of topics of interest encounters a disturbance by the diversity of the images, which negatively influences the self-assessed topic.
To better visualize this variation, Figs. 8, 9, 10 and 11 describe the Cumulative Match Characteristics (CMC) curves.We remark that some classes possess a high accuracy value for a specific number of images.Other classes present the stability of accuracy value for some images.In addition, diverse classes possess an accuracy of 0 for some images.For example, Lifestyle class obtains an accuracy of 0 for 5, 10, 50, 75, 100 shared images and News class has an accuracy of 0 for 5, 10, 50 images.In fact, users who have Lifestyle or News as a self-assessed class are more likely to have perturbations in their shared images that change the output class to any other target class.In this context, we try to discuss the reasons why some images are miss-classified over classes other than the self-assessed class, as shown in Fig. 12.
To assess our framework, we deliver a continuously growing set of pre-trained models with famous architectures for the Caffe framework [13].Our work highlights the depth of the CNN, which affects the convolutional layers' ability.Thus, we make use of four different CNN architectures illustrated in Table 5 in terms of performance accuracy.Thus, GoogleNet is the best architecture based on the idea of executing the layers in parallel with the inception  Based on Table 6 , we compared our method to other existing methods, based on the number of images taken from the used database as well as the number of classes defined to predict the users' profiles.However, the databases used in previous studies are not available.For that reason, we could not test our framework on their databases.Thus, we could not carry out a comparison between our method and other existing works based on accuracy measures.We can valorize our work by choosing Facebook as a source of data rather than other social networking sites, using not only 24000 shared images but also 24 classes predefined by Facebook.
Furthermore, The social data collection requires the implementation of a specific Facebook application [18] compared to other works [33,39,41] that use Pinterest as a social source applying the crawling method based on public APIs.Unlike previous studies, we applied in this work four CNN architectures for object recognition to enhance the feature extraction and classification modules.To evaluate the performance of our algorithm, we applied two different criteria 1) Precision and 2) Recall illustrated in Tables 7 and 8, respectively.We used the Cumulative Match Characteristics (CMC) curve to illustrate the evolution of interest prediction rate with the number of shared images and to compare this rate for each class.

Result's discussion
In this section, we will discuss our obtained results by evaluating the performance of each architecture used in the feature extraction phase.Each CNN architecture contains two separate modules, namely the feature extraction module and the classification module.The performance of a CNN architecture is related to the parameters of the feature extraction module, especially the number of layers and the size of filters.
AlexNet is one of the deep convolutional neural network to deal with complex scene classification tasks.AlexNet has 5 convolutional layers, 3 sub-sampling layers and 3 fully con-nected layers.It uses a set of filters whose sizes are 11*11, 5*5 and 3*3 for each convolutional layer, respectively.Furthermore, to achieve better performance, the complexity of convolutional neural networks is continually increasing with deeper architectures.The VGG'19 is much deeper than AlexNet with 19 layers including thirteen convolutional layers whose filter sizes are 3*3 and 3 fully-connected layers.The use of very small filter sizes captures a set of deep visual features extracted from image input, decreases the number of parameters and increases the number of filters.The increasing amount of filters will also augment the depth of the image  input and consequently the depth of the network which presents a key component for achieving good performance in the users' interest classification task based on the complex scene image classification.Given that VGG'19 is based on the filters simplicity and the depth of the network, GoogleNet is one of the first architectures that introduces the idea of layer execution in parallel with the inception module based on the concatenating operation at different scales.
GoogleNet uses 9 inception modules with a creative structuring of layers in order to improve performance and computationally efficiency.Hence, GoogleNet helps us to get better classification accuracy by extracting information about the very fine grain details in the volume.
After having discussed the efficiency of the feature extraction phase, we try to analyze the images that are misclassified over classes other than the self-assessed class.This misclassification is due to diverse adversarial attacks in the form of delicate perturbations to each input user's image that conduct our framework to predict incorrect class compared to the self-assessed class.However, the users who have self-assessed classes like Relationships(Family Relationshsips and Friends Relationships), News, Wellness, Lifestyle and Hobbies are more likely to have perturbations in their shared images that change the output class to any other target class.
Several examples of misclassified users are shown in Fig. 12.The labels above the selected images present the self-assessed classes for each group of users and the labels below the images are the target classes obtained by our framework.In the first group on the right, we see a set of images shared by users, who have News as a self-assessed class.The classification of these images is based on the objects belonging to each class obtained by a CNN architecture for objects recognition.These objects possess Fashion, Places or People as a super class in our UIO ontology.This inference presents a perturbation to predict other target classes other than the self-assessed class by the user.
In the second group in the middle, we illustrate some images shared by users who consider Relationships as a self-assessed class.The classification of these images considers Outdoors and People as target classes.This miss-classified is caused by the fact that the objects belong to different classes.These shared images have Outdoors or People as a super class in our UIO ontology.In fact, the topic Relationships reflects the importance of making friendships with people in order to have great time together.
In the third group on the left, we present multiple images shared by users who have Wellness as self-assessed topic.The fact that these shared images contain objects which have Outdoors or People, these users have Outdoors or People as target classes in our classification.

Conclusion
In this paper, we examined the correlation between the topics of interest in social networks (e.g., Facebook).We also defined the confidential area of a number of shared images to obtain the best user's interest distribution.A joint novel framework, named DeepVisInterest, was established to predict users' interests from visual data applying mainly the CNN architectures for feature extractor and classification modules.We introduced novel users' interest model to conceptualize and categorize the 24 topics of interests into semantic representation using the ontology.We systematically evaluated the proposed framework regarding our VisualDatabase which contains over 24000 images.Our system showed competitive results compared to other state of the art techniques.Our future works will put special focus on the combination of visual, textual and social data to improve big social data analysis and predict users' profiles.

•
the number of Root nodes with NRN r (O) = Root r (O), • the number of Leaf nodes with NLN r (O) = Leaf r (O), • the maximum length of simple path with MaxSP L r (O) = Max p∈path(O) (Lenght (P )), • the number of Isolated nodes as NI C r (O) = Root r (O) Leaf r (O)), • the total number of Reachable nodes from Roots with T NRNR r (O) = x∈Root r (O) Reachable r (O) and • the average number of Reachable nodes from Roots with AN RN R r (O) = T NRNR r (O) \ NRN r (O).

Fig. 4
Fig. 4 An illustration of VUIP-IL/UL method.Three steps are incorporated

Fig. 5
Fig. 5 Topic of interests correlation

Fig. 6
Fig. 6 Image-level distribution for family class with 50 images per user

Fig. 7
Fig. 7 Image-level distribution for family class with 100 images per user

Fig. 8
Fig. 8 Cumulative match characteristics (CMC) curve for activities, business, drink, education, entertainment and events classes

Fig. 9
Fig. 9 Cumulative Match Characteristics (CMC) curve for family, fashion, fitness, food, industry and news classes

Fig. 12
Fig. 12 Some adversarial examples of images: (a) Users who have "News" as a self-assessed class, (b) Users who have "Relationships" as a self-assessed class and (c) Users who have "Wellness" as a self-assessed class

Table 1
List of 24 topics of interest in social networks (e.g., Facebook)

Table 2
State of the art of users' interest prediction: topics of interest