Abstract

The increasing individualization of products reinforces the importance of decoupled factories in production processes. Artificial intelligence (AI) is a recognized technology for problem solving and accelerates automation by enabling systems to act independently. In the field of robotics, there are new deep learning approaches which make robotic control systems human independent. This work provides a literature overview of the current state of development methodologies, showing that there are only limited methods available for the development of artificial intelligent robots. We present a novel development methodology based on artificial intelligence, particularly deep reinforcement learning. The so-called Q-model can enable robots to learn specific tasks independently. In summary, we show how an AI-based methodology assists the development of autonomous robots along the product lifecycle.

Issue Section:
Research Papers
Keywords:
artificial intelligence, machine learning for engineering applications, manufacturing automation, virtual prototyping
Topics:
Artificial intelligence, Design, Robotics, Robots, Machine learning, Control systems

References

1.
Boesl
,
D. B. O.
,
Bode
,
M.
,
Liepert
,
B.
, and
Greisel
,
S.
,
2017
, “
Structured Megatrends Research As Foundation for Future-Oriented Research-Planning and R&D Roadmapping in Robotics
,”
2017 International Conference on Current Trends in Computer, Electrical, Electronics and Communication (CTCEEC)
,
Mysore
, pp.
24
–
30
.
2.
Bauernhansl
,
T.
, and
Vogel-Heuser
,
B.
,
2014
, “
Industrie 4.0 in Produktion, Automatisierung und logistik. Anwendung · Technologien · migration
”. 10.1007/978-3-658-04682-8
3.
Stone
,
P.
,
Brooks
,
R.
,
Brynjolfsson
,
E.
,
Calo
,
R.
,
Etzioni
,
O.
,
Hager
,
G.
,
Hirschberg
,
J.
,
Kalyanakrishnan
,
S.
,
Kamar
,
E.
,
Kraus
,
S.
,
Leyton-Brown
,
K.
,
Parkes
,
D.
,
Press
,
W.
,
Saxenian
,
A.
,
Shah
,
J.
,
Tambe
,
M.
, and
Teller
,
A.
,
2016
, “
Artificial Intelligence and Life in 2030
”.
One Hundred Year Study on Artificial Intelligence: Report of the 2015-2016 Study Panel, Stanford University
,
Stanford, CA
, September, http://ai100.stanford.edu/2016-report, Accessed June 28, 2019.
4.
Van der Meer
,
J.
,
Hassan
,
S.
,
Lofink
,
O.
,
Jones
,
R.
, and
Hagedorn
,
R.
,
2018
, “
Ai and Robotics Automation in Consumer-Driven Supply Chains
”. The Consumer Goods Forum – AI-Robotics-Report, https://www.theconsumergoodsforum.com/wp-content/uploads/2018/04/201805-CGF-AI-Robotics-Report-with-PA-Consulting.pdf, Accessed June 28, 2019.
5.
Torresen
,
J.
,
2018
, “
A Review of Future and Ethical Perspectives of Robotics and AI
,”
Front. Robotics AI
,
4
(
75
), pp.
1
–
10
. 10.3389/frobt.2017.00075
6.
Beer
,
M.
,
2018
, “
Artificial Intelligence, Robotics and ‘Autonomous’ Systems
,”
Eur. Group Ethics Sci. New Technol.
,
30
(
1
), pp.
1
–
20
. 10.2777/786515
7.
Perez
,
J. A.
,
Deligianni
,
F.
,
Ravì
,
D.
, and
Yang
,
G.
,
2018
,
Artificial Intelligence and Robotics, UK Artificial Intelligence and Robotics Network, Report No. 1, London
.
8.
Beer
,
J. M.
,
Fisk
,
A. D.
, and
Rogers
,
W. A.
,
2014
, “
Toward a Framework for Levels of Robot Autonomy in Human-Robot Interaction
,”
J. Hum.-Robot Interact.
,
3
(
2
), pp.
74
–
99
. 10.5898/JHRI.3.2.Beer
9.
Durst
,
P.
,
Gray
,
W.
, and
Trentini
,
M.
,
2013
, “
Development of a Non-Contextual Model for Determining the Autonomy Level of Intelligent Unmanned Systems
,”
SPIE Defense, Security, and Sensing
,
Baltimore, MD
.
10.
Bhagat
,
S.
,
Banerjee
,
H.
,
Ho Tse
,
Z. T.
, and
Ren
,
H.
,
2019
, “
Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges
,”
Robotics
,
8
(
1
), p.
4
. 10.3390/robotics8010004
11.
Levine
,
S.
,
Finn
,
C.
,
Darrell
,
T.
, and
Abbeel
,
P.
,
2015
, “
End-to-end Training of Deep Visuomotor Policies
,”
J. Mach. Learn. Res.
,
17
, pp.
1
–
40
.
12.
Kurrek
,
P.
,
Jocas
,
M.
,
Zoghlami
,
F.
,
Stoelen
,
M.
, and
Salehi
,
V.
,
2019
, “
AI Motion Control — A Generic Approach to Develop Control Policies for Robotic Manipulation Tasks
,”
Proceedings of the 22nd International Conference on Engineering Design (ICED19)
,
Delft
.
13.
Blessing
,
L.
, and
Chakrabarti
,
A.
,
2009
, “Design Research Methodology”,
Springer-Verlag
,
London
.
14.
Woolley
,
R.
,
Krüger
,
M.
, and
Kordenbrock
,
P.
,
2009
,
Systems Engineering Guidebook for Intelligent Transportation Systems
, 3,
U.S. Department of Transportation, Federal Highway Administration
, pp.
7
–
14
.
15.
VDI2206
,
2004
, “
Vdi 2206: Entwicklungsmethodik fuer mechatronische systeme
,” Verein Deutscher Ingenieure, Duesseldorf, Germany.
16.
Kralev
,
V.
, and
Kraleva
,
R.
,
2017
, “
Methods and Tools for Rapid Application Development
,”
Proceedings of the 3rd International Scientific and Practical Conference Methodology of Modern Research
,
Dubai
.
17.
Bass
,
L. J.
,
Weber
,
I.
, and
Zhu
,
L.
,
2015
,
SEI series in Software Engineering
,
Addison-Wesley
,
USA
, pp.
3
–
24
.
18.
Saravia
,
S.
,
Slen
,
E.
, and
Pendse
,
G.
,
2018
, “
Artificial Intelligence and Robotics: Industry Report and Investment Case
”. NASDAQ Global Information Services, March. https://indexes.nasdaqomx.com/docs/NQROBO%20Research.pdf
19.
Coombs
,
C.
,
Hislop
,
D.
,
Barnard
,
S.
, and
Taneva
,
S.
,
2017
,
Impact of Artificial Intelligence, Robotics and Automation Technologies on Work – Rapid Evidence Review, Chartered Institute of Personnel and Development, Report No. 7632, London
.
20.
Maldonado
,
A.
,
Alvarez
,
H.
, and
Beetz
,
M.
,
2012
, “
Improving Robot Manipulation Through Fingertip Perception
,”
2012 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Vilamoura
.
21.
Lenz
,
I.
,
Lee
,
H.
, and
Saxena
,
A.
,
2015
, “
Deep Learning for Detecting Robotic Grasps
,”
Int. J. Robotics Res.
,
34
(
4–5
), pp.
705
–
724
. 10.1177/0278364914549607
22.
Zoghlami
,
F.
,
Kurrek
,
P.
,
Jocas
,
M.
,
Masala
,
G.
, and
Salehi
,
V.
,
2019
, “
Usage Identification of Anomaly Detection in An Industrial Context
,”
Proceedings of the 22nd International Conference on Engineering Design (ICED19)
,
Delft
.
23.
Nguyen-Tuong
,
D.
, and
Peters
,
J.
,
2011
, “
Model Learning for Robot Control: a Survey
,”
Cogn. Process.
,
12
(
4
), pp.
319
–
340
. 10.1007/s10339-011-0404-1
24.
Srivastava
,
S.
,
Fang
,
E.
,
Riano
,
L.
,
Chitnis
,
R.
,
Russell
,
S.
, and
Abbeel
,
P.
,
2014
, “
Combined Task and Motion Planning Through An Extensible Planner-Independent Interface Layer
,”
2014 IEEE International Conference on Robotics and Automation (ICRA)
,
Hong Kong
.
25.
Samuel
,
A. L.
,
1959
, “
Some Studies in Machine Learning Using the Game of Checkers
,”
IBM. J. Res. Dev.
,
3
(
3
), pp.
210
–
229
. 10.1147/rd.33.0210
26.
Laskey
,
M.
,
Chuck
,
C.
,
Lee
,
J.
,
Mahler
,
J.
,
Krishnan
,
S.
,
Jamieson
,
K.
,
Dragan
,
A. D.
, and
Goldberg
,
K. Y.
,
2016
, “
Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations
,”
2017 IEEE International Conference on Robotics and Automation (ICRA)
,
Singapore
.
27.
Bohg
,
J.
,
2011
, “
Multi-Modal Scene Understanding for Robotic Grasping
,” Doctoral dissertation, KTH Royal Institute of Technology, KTH Publication Database.
28.
Li
,
Q.
,
Haschke
,
R.
, and
Ritter
,
H.
,
2015
, “
A Visuo-Tactile Control Framework for Manipulation and Exploration of Unknown Objects
,”
2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids)
,
Seoul
.
29.
Sasaki
,
K.
,
Noda
,
K.
, and
Ogata
,
T.
,
2016
, “
Visual Motor Integration of Robot’s Drawing Behavior Using Recurrent Neural Network
,”
Rob. Auton. Syst.
,
86
, pp.
184
–
195
. 10.1016/j.robot.2016.08.022
30.
Mar
,
T.
,
Tikhanoff
,
V.
,
Metta
,
G.
, and
Natale
,
L.
,
2015
, “
Multi-Model Approach Based on 3D Functional Features for Tool Affordance Learning in Robotics
,”
2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids)
,
Seoul
.
31.
Ghadirzadeh
,
A.
,
Maki
,
A.
,
Kragic
,
D.
, and
Björkman
,
M.
,
2017
, “
Deep Predictive Policy Training Using Reinforcement Learning
,”
2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Vancouver
.
32.
Kaelbling
,
L. P.
,
Littman
,
M. L.
, and
Moore
,
A. W.
,
1996
, “
Reinforcement Learning: A Survey
,”
J. Artif. Int. Res.
,
4
(
1
), pp.
237
–
285
.
33.
Sutton
,
R. S.
, and
Barto
,
A. G.
,
1998
,
Introduction to Reinforcement Learning
, 1st ed.,
MIT Press
,
Cambridge, MA
.
34.
Bousmalis
,
K.
,
Irpan
,
A.
,
Wohlhart
,
P.
,
Bai
,
Y.
,
Kelcey
,
M.
,
Kalakrishnan
,
M.
,
Downs
,
L.
,
Ibarz
,
J.
,
Pastor
,
P.
,
Konolige
,
K.
,
Levine
,
S.
, and
Vanhoucke
,
V.
,
2017
, “
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
,”
2018 IEEE International Conference on Robotics and Automation (ICRA)
,
Brisbane
.
35.
Rusu
,
A. A.
,
Vecerik
,
M.
,
Rothörl
,
T.
,
Heess
,
N.
,
Pascanu
,
R.
, and
Hadsell
,
R.
,
2016
, “
Sim-to-Real Robot Learning from Pixels with Progressive Nets
,”
1st Conference on Robot Learning (CoRL 2017)
,
Mountain View
.
36.
Jocas
,
M.
,
Kurrek
,
P.
,
Zoghlami
,
F.
,
Gianni
,
M.
, and
Salehi
,
V.
,
2019
, “
AI-Based Learning Approach With Consideration of Safety Criteria on Example of a Depalletization Robot
,”
Proceedings of the 22nd International Conference on Engineering Design (ICED19)
,
Delft
.
37.
Liu
,
X.
, and
Jin
,
Y.
,
2018
, “
Design of Transfer Reinforcement Learning Under Low Task Similarity
,”
Volume 2A: 44th Design Automation Conference of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Quebec City
.
38.
Sommerville
,
I.
,
2019
, “
Artificial Intelligence and Systems Engineering
.” https://pdfs.semanticscholar.org/63a6/8192ed57c629192d4e7058cf7fa0b08ef89c.pdf, Accessed June 28, 2019.
39.
Stoelen
,
M. F.
,
de Tejada
,
V. F.
,
Huete
,
A. J.
,
Balaguer
,
C.
, and
Bonsignorio
,
F. P.
,
2015
, “
Distributed and Adaptive Shared Control Systems: Methodology for the Replication of Experiments
,”
IEEE Rob. Autom. Mag.
,
22
(
4
), pp.
137
–
146
. 10.1109/MRA.2015.2460911
40.
Yang
,
T.-H.
, and
Lee
,
W.-P.
,
2013
, “
A Service-Oriented Framework for the Development of Home Robots
,”
Int. J. Adv. Robotic Systems
,
10
(
2
), p.
122
. 10.5772/55055
41.
Beetz
,
M.
,
Mösenlechner
,
L.
, and
Tenorth
,
M.
,
2010
, “
Cram-A Cognitive Robot Abstract Machine for Everyday Manipulation in Human Environments
,”
2010 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Taipei
.
42.
Tan
,
H.
,
2015
, “
A Behavior Generation Framework for Robots to Learn From Demonstrations
,”
2015 IEEE International Conference on Systems, Man, and Cybernetics
,
Kowloon
.
43.
Ertel
,
W.
,
Schneider
,
M.
,
Cubek
,
R.
, and
Tokic
,
M.
,
2009
, “
The Teaching-Box: A Universal Robot Learning Framework
,”
2009 International Conference on Advanced Robotics
,
Munich
.
44.
Beetz
,
M.
,
Tenorth
,
M.
, and
Winkler
,
J.
,
2015
, “
Open-Ease—A Knowledge Processing Service for Robots and Robotics/ai Researchers
,”
2015 IEEE International Conference on Robotics and Automation (ICRA)
,
Seattle
.
45.
Guerin
,
K. R.
,
Lea
,
C.
,
Paxton
,
C.
, and
Hager
,
G. D.
,
2015
, “
A Framework for End-User Instruction of a Robot Assistant for Manufacturing
,”
2015 IEEE International Conference on Robotics and Automation (ICRA)
,
Seattle
.
46.
Nicolescu
,
N. M.
, and
Matarić
,
M. J.
,
2002
, “
A Hierarchical Architecture for Behavior-Based Robots
,”
Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 1, AAMAS ’02
,
New York
.
47.
Takahashi
,
K.
,
Kim
,
K.
,
Ogata
,
T.
, and
Sugano
,
S.
,
2017
, “
Tool-Body Assimilation Model Considering Grasping Motion Through Deep Learning
,”
Rob. Auton. Syst.
,
91
, pp.
115
–
127
. 10.1016/j.robot.2017.01.002
48.
Johannsmeier
,
L.
, and
Haddadin
,
S.
,
2017
, “
A Hierarchical Human-Robot Interaction-Planning Framework for Task Allocation in Collaborative Industrial Assembly Processes
,”
IEEE Robotics Automation Lett.
,
2
(
1
), pp.
41
–
48
. 10.1109/LRA.2016.2535907
49.
Martinez-Hernandez
,
U.
,
Damianou
,
A.
,
Camilleri
,
D.
,
Boorman
,
L. W.
,
Lawrence
,
N.
, and
Prescott
,
T. J.
,
2016
, “
An Integrated Probabilistic Framework for Robot Perception, Learning and Memory
,”
2016 IEEE International Conference on Robotics and Biomimetics (ROBIO)
,
Qingdao
.
50.
Bates
,
T.
,
Ramirez-Amaro
,
K.
,
Inamura
,
T.
, and
Cheng
,
G.
,
2017
, “
On-line Simultaneous Learning and Recognition of Everyday Activities From Virtual Reality Performances
,”
2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Vancouver
.
51.
Gonzalez
,
A. G. C.
,
Alves
,
M. V. S.
,
Viana
,
G. S.
,
Carvalho
,
L. K.
, and
Basilio
,
J. C.
,
2018
, “
Supervisory Control-based Navigation Architecture: A New Framework for Autonomous Robots in Industry 4.0 Environments
,”
IEEE Trans. Ind. Informatics
,
14
(
4
), pp.
1732
–
1743
. 10.1109/TII.2017.2788079
52.
Kumar
,
S.
, and
Sahin
,
F.
,
2017
, “
A Framework for An Adaptive Human-Robot Collaboration Approach Through Perception-Based Real-Time Adjustments of Robot Behavior in Industry
,”
2017 12th System of Systems Engineering Conference (SoSE)
,
Waikoloa
.
53.
Soria Zurita
,
N. F.
,
Colby
,
M. K.
,
Tumer
,
I. Y.
,
Hoyle
,
C.
, and
Tumer
,
K.
,
2017
, “
Design of Complex Engineered Systems Using Multi-Agent Coordination
,”
ASME J. Comput. Inf. Sci. Eng.
,
18
(
1
), pp.
11
–
0
.
54.
Berg
,
L. P.
, and
Vance
,
J. M.
,
2016
, “
An Industry Case Study: Investigating Early Design Decision Making in Virtual Reality
,”
ASME J. Comput. Inf. Sci. Eng.
,
17
(
1
), pp.
11
–
0
. 10.1115/1.4034267
55.
Salehi
,
V.
, and
Wang
,
S.
,
2019
, “
Munich Agile MBSE Concept (MAGIC)
,”
Proceedings of the 22nd International Conference on Engineering Design (ICED19)
,
Delft
.
56.
Salehi
,
V.
,
2019
, “
Development of An Agile Concept for Mbse for Future Digital Products Through the Entire Life Cycle Management Called Munich Agle Mbse Concept (magic)
,”
Computer-Aided Design Appl.
,
17
(
1
), pp.
147
–
166
. 10.14733/cadaps.2020.147-166
57.
Adachi
,
S.
,
2018
, “
Model-Based Development and Artificial Intelligence
,”
Keihin Technical Review
,
6
(
2017
), pp.
2
–
5
.
58.
Maletzki
,
G.
,
2013
, “
Rapid Control Prototyping komplexer und flexibler robotersteuerungen auf basis des sbc-ansatzes
,” Doctoral dissertation, University of Rostock, Faculty of Computer Science and Electrical Engineering.
59.
Sorte
,
B.
,
Joshi
,
P.
, and
Jagtap
,
V.
,
2015
, “
Use of Artificial Intelligence in Software Development Life Cycle: A State of the Art Review
,”
Int. J. Technol. Manage.
,
3
(
1
), pp.
2309
–
4893
.
60.
Kumari
,
V.
, and
Kulkarni
,
S.
,
2018
, “
Use of Artificial Intelligence in Software Development Life Cycle: Requirements and Its Model
,”
Int. Res. J. Eng. Technol. (IRJET)
,
5
(
8
), pp.
398
–
403
.
61.
Miao
,
H.
,
Li
,
A.
,
Davis
,
L. S.
, and
Deshpande
,
A.
,
2017
, “
Modelhub: Deep Learning Lifecycle Management
,”
2017 IEEE 33rd International Conference on Data Engineering (ICDE)
,
San Diego, CA
.
62.
Miao
,
H.
,
Li
,
A.
,
Davis
,
L. S.
, and
Deshpande
,
A.
,
2017
, “
Towards Unified Data and Lifecycle Management for Deep Learning
,”
2017 IEEE 33rd International Conference on Data Engineering (ICDE)
,
San Diego
.
63.
Deatcu
,
C.
,
Freymann
,
B.
,
Schmidt
,
A.
, and
Pawletta
,
T.
,
2015
, “
Matlab/Simulink based Rapid Control Prototyping for Multivendor Robot Applications
”. https://doi.org/10.13140/RG.2.1.1823.3442
64.
Chen
,
L.-C.
,
Schwing
,
A. G.
,
Yuille
,
A. L.
, and
Urtasun
,
R.
,
2014
, “
Learning Deep Structured Models
,”
32nd International Conference on Machine Learning
,
Lille
.
65.
Cloutier
,
A.
, and
Yang
,
J.
,
2018
, “
Grasping Force Optimization Approaches for Anthropomorphic Hands
,”
ASME J. Mech. Rob.
,
10
(
1
), pp.
011004
. 10.1115/1.4038684
66.
Juliani
,
A.
,
Berges
,
V.
,
Vckay
,
E.
,
Gao
,
Y.
,
Henry
,
H.
,
Mattar
,
M.
, and
Lange
,
D.
,
2018
,
Unity: A General Platform for Intelligent Agents
.
67.
Quigley
,
M.
,
Gerkey
,
B. P.
,
Conley
,
K.
,
Faust
,
J.
,
Foote
,
T.
,
Leibs
,
J.
,
Berger
,
E.
,
Wheeler
,
R.
, and
Ng
,
A.
,
2009
, “
ROS: An Open-Source Robot Operating System
,” http://www.willowgarage.com/sites/default/files/icraoss09-ROS.pdf, Accessed June 28, 2019.
68.
Dhariwal
,
P.
,
Hesse
,
C.
,
Klimov
,
O.
,
Nichol
,
A.
,
Plappert
,
M.
,
Radford
,
A.
,
Schulman
,
J.
,
Sidor
,
S.
,
Wu
,
Y.
, and
Zhokhov
,
P.
,
2017
, “
Openai Baselines
.” https://github.com/openai/baselines
69.
Schulman
,
J.
,
Wolski
,
F.
,
Dhariwal
,
P.
,
Radford
,
A.
, and
Klimov
,
O.
,
2017
, “
Proximal Policy Optimization Algorithms
,”
ArXiv
,
abs/1707.06347
(
2017
), pp.
1
–
12
.
70.
Peng
,
X. B.
,
Andrychowicz
,
M.
,
Zaremba
,
W.
, and
Abbeel
,
P.
,
2017
, “
Sim-to-Real Transfer of Robotic Control With Dynamics Randomization
,”
2018 IEEE International Conference on Robotics and Automation (ICRA)
,
Brisbane
.
Copyright © 2020 by ASME
You do not currently have access to this content.