See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/320225102

SMART E-COMMERCE FRAMEWORK: AUGMENTED REALITY, MOBILE AND

COGNITIVE APPLICATION

Conference Paper · May 2017

CITATIONS READS

0 584

3 authors, including:

Brandas Claudiu Otniel Didraga

West University of Timisoara West University of Timisoara
30 PUBLICATIONS   63 CITATIONS    11 PUBLICATIONS   31 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Artificial Intelligence and Virtual Reality in Smart Business Development View project

All content following this page was uploaded by Brandas Claudiu on 05 October 2017.

The user has requested enhancement of the downloaded file.

 Proceedings of the IE 2017 International Conference
www.conferenceie.ase.ro

SMART E-COMMERCE FRAMEWORK: AUGMENTED REALITY,

MOBILE AND COGNITIVE APPLICATION

Claudiu BRANDAS
West University of Timisoara
claudiu.brandas@e-uvt.ro
Otniel DIDRAGA
West University of Timisoara
otniel.didraga@e-uvt.ro
Dario HUMA
West University of Timisoara
dario.huma95@e-uvt.ro

Abstract. E-commerce systems adopted augmented reality and cognitive computing as a new
type of “smart” e-commerce. Mobile e-commerce applications using augmented reality and
cognitive computing functions allow better user interaction, customer satisfaction, shopping
value, quality, and purchasing decisions. Augmented reality (AR) combines computer-
generated items with real environment in an interaction between humans, and virtual reality.
In the same time, cognitive computing technologies like visual recognition, sentiment analysis,
environment analysis, and speech recognition offer great support for improving human
decision-making and natural interaction. In this paper, we present a framework for smart e-
commerce applications development using augmented reality, mobile technology, and
cognitive computing functions. Based on this framework, we developed a prototype of a smart
e-commerce mobile application.

Keywords: Augmented Reality, Mobile Applications, Cognitive Computing, Smart E-

Commerce.
JEL classification: M15, O3.

1. Introduction
Augmented reality (AR) is a technology with an incredibly fast development nowadays. It
combines computer-generated objects with real environment, allowing human interaction real-
time. This new smart technology was introduced and developed in the 1990s [1], [2].
The AR technology focused on using different displays, interfaces, and algorithms. There are
AR devices that connect to AR mobile apps [2]. AR research has aimed at technical evaluation
of devices, user perception and acceptance, task performance, developing app design
guidelines, or AR cognition [2].
AR apps have been present in marketing and retail only in the last decade [2]. This is a novelty,
making e-commerce “smart”. Studies on AR apps between 2006 and 2016 mostly investigated
convenience, perceived usefulness, perceived ease of use, customer satisfaction, shopping
value, personal emotions, system quality, and innovativeness [2].
Smart e-commerce development is one of the bases of Artificial Intelligence 2.0 [3].
Human-machine hybrid-augmented intelligence is a future research direction considering
enhanced decision-making abilities using the collaboration between machines and humans [3].

140
 Proceedings of the IE 2017 International Conference
www.conferenceie.ase.ro

Retailers and customers benefit from AR as a smart technology that improves customer
engagement and purchasing decisions. It also generates better customer and business value [4],
[5], [6], [7].
An AR system is different from a traditional on-line shopping system because it includes
interactivity and immersion [8]. User experience is a key factor in the success of an AR system,
and it includes evaluating users’ perceptions, thoughts, and feelings about the products and
services [8].
Augmented reality is similar to virtual reality (VR), but instead of replacing the physical world,
“AR enhances the physical reality by integrating virtual objects into the physical world. The
virtual objects become an equal part of the natural environment” [9].
VR in e-commerce has been used to provide consumers a new type of shopping experience by
interacting with virtual products [9]. VR was a way of user interaction, but new methods were
needed to support customer experience and provide more realistic product perspective in size
and interaction [9]. Thus, AR integrates computer-generated objects in real-world scenes,
including the virtual objects in the natural environment [9].
Simple computer augmented reality applications were not enough, so AR reached mobile
devices. Mobile Augmented Reality (MAR) emerged in the mid-1990s, by applying AR
settings to mobile devices, away from confined spaces or desktop computers [10]. MAR
combines “wireless communication, location-based computing and services, and augmented
reality to create an integrated environment” [10]. MAR app users point their smartphone or
tablet in the direction of a product they are interested in, and the screen output of the camera
augments the display of the real environment with additional information [10], [11].
Cognitive computing is a frontier already crossed in the evolution of computing. Cognitive
systems are complex and can assist people and organizations in decision making [12].
Cognitive computing combines machine learning, language processing, speech, vision, and
computer-human interaction [12]. Some of the most useful cognitive computing technologies
made available through IBM Watson are image/speech recognition, sentiment analysis, and
environment analysis [12].

2. Research methodology
Research problem
The main problem is to design a framework that uses augmented reality, mobile technology,
and cognitive functions to enhance a customer satisfaction when dealing with e-commerce.
Research design
The research is experimental, and it is based on the development of Smart Ecommerce
Framework (SEF). Based on this framework, we developed a smart e-commerce application
that uses augmented reality technologies and cognitive functions on an e-commerce
infrastructure.
The framework
In our approach, the Smart Ecommerce Framework (SEF) have three main layers (Table 1):
- Application Models: contains five models for smart e-commerce application
development as follows: User Profile Model (UPM); 3D Products Model (3D PM);
Environment Model (EM); User Interface Model (UIM); Application Infrastructure
Model (AIM).
- Augmented Reality Process: represents the AR functions implemented in Application
Models.
- Cognitive Functions: represent functions and algorithms of cognitive computing and
Artificial Intelligence for supporting cognitive features of Application Models.

141
 Proceedings of the IE 2017 International Conference
www.conferenceie.ase.ro

Table 1. Smart E-commerce Framework

Application Models Augmented Reality Process Cognitive Functions
User Profile Model - Decision-making buying
-
(UPM) assistant
- Create/Load/Update 3D
3D Products Model - Product recognition and
Product from 3D Products
(3D PM) selection
Library
Environment Model - Load Environment
- Environment recognition
(EM) information
- Show 3D Product
User Interface Model
- Show Environment -
(UIM)
information
- IBM Watson Visual
- Executable Mobile
Recognition Service [15]
Application Application (Unity 3D, and
- Google TensorFlow
Infrastructure Model Vuforia for Android OS)
Image Recognition [16]
(AIM) - E-Commerce Website and
- Microsoft Computer
Database
Vision [17]

Tools and materials

The app was developed in Unity 3D [13], and Vuforia [14] for Android OS.
Unity 3d is a cross-platform game engine [13].
Vuforia is an Augmented Reality Software Development Kit (SDK) used to develop mobile
augmented reality applications [14].
“IBM Watson Visual Recognition uses deep learning algorithms to analyze images for objects,
and other content” [15].
“TensorFlow is an open source software library for numerical computation using data flow
graphs” [16], and it can be used for image recognition.
With Microsoft Computer Vision, the user can “extract rich information from images to
categorize and process visual data” [17].
Applying the SEF, we developed the app and obtained the results that we will present and
discuss in the following section.

3. Results and Discussion

In this section, we will present the main results regarding smart e-commerce application
development through SEF.
In figure 1, we present the IBM Watson Visual Recognition used for implementation of Product
recognition as a Cognitive Function in the 3D Products Model (3D PM). When the camera
identifies an object from the e-commerce system, the app analyzes the object and returns or
creates the 3D product (object) that can be used in AR process. The user can get the product
and use it (virtually through the mobile camera AR process) where they want to place the real
product.
The following code is the JSON code resulted from an image recognition of a flat-screen TV
with IBM Watson Visual Recognition.

[{"classes": [{"class": "flat panel display", "score": 0.954,

"type_hierarchy": "/electronic device/video display/flat panel display"},
{"class": "video display", "score": 0.954}, {"class": "electronic device",
"score": 0.954}, {"class": "device", "score": 0.954}, {"class": "high-

142
 Proceedings of the IE 2017 International Conference
www.conferenceie.ase.ro

definition television", "score": 0.682, "type_hierarchy":

"/medium/telecommunication/ broadcasting/high-definition television"},
{"class": "television", "score": 0.701}, {"class": "broadcasting", "score":
0.701}, {"class": "telecommunication", "score": 0.701}, {"class": "medium",
"score": 0.701}, {"class": "purple color", "score": 0.959}, {"class": "pink
color", "score": 0.9}], "classifier_id": "default", "name": "default"}]

Figure 1. Image recognition with IBM Watson Visual Recognition

Figure 2 represents the virtual objects developing in Unity 3D [13].

143
 Proceedings of the IE 2017 International Conference
www.conferenceie.ase.ro

Figure 2. Mobile app development in Unity 3D

The Environment Model (EM) contains functions for reality environment scanning and offers
information about the impact of a product on the current environment based on Environment
recognition algorithms from Cognitive Functions. This information will be displayed in the
User Interface Model (UIM) through the AR Process.
Based on the User Interface Model (UIM) functions, the product is displayed as an augmented
reality object when scanning a certain object or mark from the real world (figure 3 and figure
4).

Figure 3. AR product display Figure 4. AR Mobile app print-screen

The User Profile Model (UPM) will implement Cognitive Functions for the decision-making
buying assistant. Based on user behavior in the e-commerce system (activity logs, products
ratings, etc.), product properties (from 3D Products Library) and environment impact, the
application will be a “smart” decision-making assistant.

5. Conclusions
This paper provides insight on the future of AR apps in marketing and retailing.
Cognitive computing technologies embedded in a mobile augmented reality app can help
improve customer decision-making and give feedback to the selling organization.
The proposed framework has three objectives. First of all, through the 3D Products Model (3D
PM), Augmented Reality Process (ARP), and Cognitive Functions (CF), we want the app to
offer the best customer support, so they get to know the product and “use” it in real time before
they buy it. We consider that the level of satisfying the customer expectations will be high.
Secondly, through the Environment Model and Cognitive Functions, we want to improve
support for integrating the product adequately and ecologically in the environment the product
will be used in. Thirdly, through the User Profile Model (UPM), the app offers powerful
decision-making support for customers.
Limitations. For the time being, integrating mobile technologies with AR and Cognitive
Computing into one app is very complex. Also the objects recognition is limited. Environment
scanning is quite limited as well.
Future research. Implementing a dialog model with the application through voice and speak
recognition included in Cognitive Functions. Implementation of IoT interaction functions for

144
 Proceedings of the IE 2017 International Conference
www.conferenceie.ase.ro

automated purchasing and delivery of products. Extending the application to access and “test”
products directly from stores - considering the technology used by Amazon Go [18].

References
[1] A.T. Azuma, “A Survey on Augmented Reality”, Presence, vol. 6, no. 4, pp. 355-385, 2007.
[2] A. Rese, D. Baier, A. Geyer-Schulz and S. Schreiber, “How Augmented Reality Apps are
Accepted by Consumers: A Comparative Analysis Using Scales and Opinions”,
Technological Forecasting and Social Change, 2016,
http://dx.doi.org/10.1016/j.techfore.2016.10.010.
[3] Y. Pan, “Heading toward Artificial Intelligence 2.0”, Engineering, vol. 2, no. 4, pp. 409-
413, December 2016.
[4] S.G. Dacko, “Enabling Smart Retail Settings via Mobile Augmented Reality Shopping
Apps”, Technological Forecasting and Social Change, 2016,
http://dx.doi.org/10.1016/j.techfore.2016.09.032 .
[5] E. Pantano, “Augmented Reality in Retailing of Local Products of Magna Grcia:
Consumer's Response”, International Journal of Management Cases, vol. 11, no. 2, pp. 206-
213, 2009.
[6] E. Pantano, “Innovation Drivers in Retail Industry”, International Journal of Information
Management, vol. 34, no. 3, pp. 344-350, 2014.
[7] E. Pantano and H. Timmermans, “What is Smart for Retailing?”, Procedia Environmental
Sciences, vol. 22, pp. 101-107, 2014.
[8] C.-H. Wang, Y.-C. Chiang and M.-J. Wang, “Evaluation of an Augmented Reality
Embedded On-line Shopping System”, Procedia Manufacturing, vol. 3, pp. 5624-5630,
2015.
[9] Y. Lu and S. Smith, “Augmented Reality E-Commerce: How the Technology Benefits
People's Lives”, Chapter 14 in I. Pavlidis, “Human Computer Interaction”, pp. 215-238,
ISBN 978-953-7619-19-0, October 2008, InTech Publishing.
[10] P.E. Kourouthanassis, C. Boletsis and G. Lekakos, “Demystifying the Design of Mobile
Augmented Reality Applications”, Multimedia Tools and Applications, vol. 74, pp. 1045-
1066, 2015.
[11] S.F. Liu and M.H. Lee, “Mobile Commerce System Integrated with Augmented Reality
and Interactive Multimedia”, Przeglad Elektrotechniczny (Electrical Review), vol. 88, no
9b, pp. 100-103, 2012.
[12] J.E. Kelly III and S. Hamm, Smart Machines: IBM's Watson and the Era of Cognitive
Computing. New York: Columbia Business School Publishing, 2013.
[13] Unity Technologies. Unity User Manual (5.6). Internet:
https://docs.unity3d.com/Manual/index.html [Mar. 15, 2017].
[14] PTC Inc. Augmented Reality Software Development Kit (SDK). Internet:
https://unity3d.com/ [Mar. 16, 2017].
[15] IBM. Visual Recognition - Watson Developer Cloud. Internet:
https://www.ibm.com/watson/developercloud/visual-recognition.html, [Mar. 14, 2017].
[16] Google. TensorFlow. Internet: https://www.tensorflow.org/tutorials/image_recognition,
[Mar. 14, 2017].
[17] Microsoft. Computer Vision API - Microsoft Cognitive Services. Internet:
https://www.microsoft.com/cognitive-services/en-us/computer-vision-api, [Mar. 14,
2017].
[18] Amazon. Amazon Go. Internet: https://www.amazon.com/b?node=16008589011, [Mar.
15, 2017]

145

View publication stats