Optimization and Machine Learning

Machine learning and optimization techniques are revolutionizing our world. Other types of information technology have not progressed as rapidly in recent years, in terms of real impact. The aim of this book is to present some of the innovative techniques in the field of optimization and machine learning, and to demonstrate how to apply them in the fields of engineering.

Optimization and Machine Learning presents modern advances in the selection, configuration and engineering of algorithms that rely on machine learning and optimization. The first part of the book is dedicated to applications where optimization plays a major role, and the second part describes and implements several applications that are mainly based on machine learning techniques. The methods addressed in these chapters are compared against their competitors, and their effectiveness in their chosen field of application is illustrated. RACHID CHELOUAH has a PhD and a Doctorate of Sciences (Habilitation) from CY Cergy Paris University, France. His main research interests are data science optimization and artificial intelligence methods and their applications in various fields of IT engineering, health, energy and security.

PATRICK SIARRY is a Professor in automatics and informatics at Paris-East Creteil University, France. His main research interests are the design of stochastic global optimization heuristics and their applications in various engineering fields. He has coordinated several books in the field of optimization.

Introduction xi

Rachid CHELOUAH

PART 1 OPTIMIZATION 1

CHAPTER 1 VEHICLE ROUTING PROBLEMS WITH LOADING CONSTRAINTS: AN OVERVIEW OF VARIANTS AND SOLUTION METHODS 3

Ines SBAI and Saoussen KRICHEN

1.1 Introduction 3

1.2 The capacitated vehicle routing problem with two-dimensional loading constraints 5

1.2.1 Solution methods 6

1.2.2 Problem description 8

1.2.3 The 2L-CVRP variants 9

1.2.4 Computational analysis 10

1.3 The capacitated vehicle routing problem with three-dimensional loading constraints 11

1.3.1 Solution methods 11

1.3.2 Problem description 13

1.3.3 3L-CVRP variants 14

1.3.4 Computational analysis 16

1.4 Perspectives on future research 18

1.5 References 18

CHAPTER 2 MAS-AWARE APPROACH FOR QOS-BASED IOT WORKFLOW SCHEDULING IN FOG-CLOUD COMPUTING 25

Marwa MOKNI and Sonia YASSA

2.1 Introduction 26

2.2 Related works 27

2.3 Problem formulation 29

2.3.1 IoT-workflow modeling 31

2.3.2 Resources modeling 31

2.3.3 QoS-based workflow scheduling modeling 31

2.4 MAS-GA-based approach for IoT workflow scheduling 33

2.4.1 Architecture model 33

2.4.2 Multi-agent system model 34

2.4.3 MAS-based workflow scheduling process 35

2.5 GA-based workflow scheduling plan 38

2.5.1 Solution encoding 39

2.5.2 Fitness function 41

2.5.3 Mutation operator 41

2.6 Experimental study and analysis of the results 43

2.6.1 Experimental results 45

2.7 Conclusion 51

2.8 References 51

CHAPTER 3 SOLVING FEATURE SELECTION PROBLEMS BUILT ON POPULATION-BASED METAHEURISTIC ALGORITHMS 55

Mohamed SASSI

3.1 Introduction 56

3.2 Algorithm inspiration 57

3.2.1 Wolf pack hierarchy 57

3.2.2 The four phases of pack hunting 58

3.3 Mathematical modeling 59

3.3.1 Pack hierarchy 59

3.3.2 Four phases of hunt modeling 61

3.3.3 Research phase – exploration 64

3.3.4 Attack phase – exploitation 65

3.3.5 Grey wolf optimization algorithm pseudocode 66

3.4 Theoretical fundamentals of feature selection 67

3.4.1 Feature selection definition 67

3.4.2 Feature selection methods 68

3.4.3 Filter method 68

3.4.4 Wrapper method 69

3.4.5 Binary feature selection movement 69

3.4.6 Benefits of feature selection for machine learning classification algorithms 70

3.5 Mathematical modeling of the feature selection optimization problem 70

3.5.1 Optimization problem definition 71

3.5.2 Binary discrete search space 71

3.5.3 Objective functions for the feature selection 72

3.6 Adaptation of metaheuristics for optimization in a binary search space 76

3.6.1 Module 𝑀1 77

3.6.2 Module 𝑀2 78

3.7 Adaptation of the grey wolf algorithm to feature selection in a binary search space 81

3.7.1 First algorithm bGWO1 81

3.7.2 Second algorithm bGWO2 83

3.7.3 Algorithm 2: first approach of the binary GWO 84

3.7.4 Algorithm 3: second approach of the binary GWO 85

3.8 Experimental implementation of bGWO1 and bGWO2 and discussion 86

3.9 Conclusion 87

3.10 References 88

CHAPTER 4 SOLVING THE MIXED-MODEL ASSEMBLY LINE BALANCING PROBLEM BY USING A HYBRID REACTIVE GREEDY RANDOMIZED ADAPTIVE SEARCH PROCEDURE 91

Belkharroubi LAKHDAR and Khadidja YAHYAOUI

4.1 Introduction 92

4.2 Related works from the literature 95

4.3 Problem description and mathematical formulation 97

4.3.1 Problem description 97

4.3.2 Mathematical formulation 98

4.4 Basic greedy randomized adaptive search procedure 99

4.5 Reactive greedy randomized adaptive search procedure 100

4.6 Hybrid reactive greedy randomized adaptive search procedure for the mixed model assembly line balancing problem type-2 101

4.6.1 The proposed construction phase 102

4.6.2 The local search phase 106

4.7 Experimental examples 107

4.7.1 Results and discussion 111

4.8 Conclusion 115

4.9 References 116

PART 2 MACHINE LEARNING 119

CHAPTER 5 AN INTERACTIVE ATTENTION NETWORK WITH STACKED ENSEMBLE MACHINE LEARNING MODELS FOR RECOMMENDATIONS 121

Ahlem DRIF, SaadEddine SELMANI and Hocine CHERIFI

5.1 Introduction 122

5.2 Related work 124

5.2.1 Attention network mechanism in recommender systems 124

5.2.2 Stacked machine learning for optimization 125

5.3 Interactive personalized recommender 126

5.3.1 Notation 128

5.3.2 The interactive attention network recommender 129

5.3.3 The stacked content-based filtering recommender 134

5.4 Experimental settings 136

5.4.1 The datasets 136

5.4.2 Evaluation metrics 137

5.4.3 Baselines 139

5.5 Experiments and discussion 140

5.5.1 Hyperparameter analysis 140

5.5.2 Performance comparison with the baselines 143

5.6 Conclusion 146

5.7 References 146

CHAPTER 6 A COMPARISON OF MACHINE LEARNING AND DEEP LEARNING MODELS WITH ADVANCED WORD EMBEDDINGS: THE CASE OF INTERNAL AUDIT REPORTS 151

Gustavo FLEURY SOARES and Induraj PUDHUPATTU RAMAMURTHY

6.1 Introduction 152

6.2 Related work 154

6.2.1 Word embedding 156

6.2.2 Deep learning models 157

6.3 Experiments and evaluation 158

6.4 Conclusion and future work 163

6.5 References 165

CHAPTER 7 HYBRID APPROACH BASED ON MULTI-AGENT SYSTEM AND FUZZY LOGIC FOR MOBILE ROBOT AUTONOMOUS NAVIGATION 169

Khadidja YAHYAOUI

7.1 Introduction 170

7.2 Related works 171

7.2.1 Classical approaches 172

7.2.2 Advanced methods 173

7.3 Problem position 174

7.4 Developed control architecture 176

7.4.1 Agents description 177

7.5 Navigation principle by fuzzy logic 183

7.5.1 Fuzzy logic overview 183

7.5.2 Description of simulated robot 184

7.5.3 Strategy of navigation 185

7.5.4 Fuzzy controller agent 186

7.6 Simulation and results 194

7.7 Conclusion 196

7.8 References 196

CHAPTER 8 INTRUSION DETECTION WITH NEURAL NETWORKS: A TUTORIAL 201

Alvise DE’ FAVERI TRON

8.1 Introduction 201

8.1.1 Intrusion detection systems 201

8.1.2 Artificial neural networks 202

8.1.3 The NSL-KDD dataset 202

8.2 Dataset analysis 203

8.2.1 Dataset summary 203

8.2.2 Features 203

8.2.3 Binary feature distribution 204

8.2.4 Categorical features distribution 207

8.2.5 Numerical data distribution 211

8.2.6 Correlation matrix 212

8.3 Data preparation 213

8.3.1 Data cleaning 213

8.3.2 Categorical columns encoding 213

8.3.3 Normalization 214

8.4 Feature selection 217

8.4.1 Tree-based selection 217

8.4.2 Univariate selection 218

8.5 Model design 219

8.5.1 Project environment 219

8.5.2 Building the neural network 220

8.5.3 Learning hyperparameters 220

8.5.4 Epochs 220

8.5.5 Batch size 221

8.5.6 Dropout layers 221

8.5.7 Activation functions 222

8.6 Results comparison 222

8.6.1 Evaluation metrics 222

8.6.2 Preliminary models 223

8.6.3 Adding dropout 225

8.6.4 Adding more layers 226

8.6.5 Adding feature selection 227

8.7 Deployment in a network 228

8.7.1 Sensors 228

8.7.2 Model choice 229

8.7.3 Model deployment 229

8.7.4 Model adaptation 231

8.8 Future work 231

8.9 References 231

List of Authors 233

Index 235