Allgemein
Sozioinformatik
Ein neuer Blick auf Informatik und Gesellschaft. Einführung in die Modellierung und Analyse digitaler Technikfolgen.Welche Auswirkungen könnte es haben, wenn Technik in den Körper implantiert wird und sich Menschen zunehmend zu Cyborgs entwickeln? Wie kann es passieren, dass sich mazedonische Jugendliche in den Präsidentschaftswahlkampf der USA einmischen? Wann entstehen Filterblasen?In den letzten Jahren konnten viele gewollte und ungewollte Technikfolgen der digitalen Transformation beobachtet werden. Die in diesem Buch vorgestellte Sozioinformatik befasst sich mit der Modellierung und Analyse solcher Phänomene: Sie untersucht dafür die Folgen der informatischen Gestaltung unter interdisziplinären Aspekten, insbesondere denen der Verhaltensökonomie.Zentrales Hilfsmittel der Analyse ist der Aufbau eines visuellen Wirkungsgefüges, mit dem verschiedene Dynamiken und Technikfolgen in der digitalen Transformation abgeschätzt werden können. Damit wird erklärbar, wann man eine Filterblase erwarten kann, warum manche digitale Technik unsere Aufmerksamkeit so effektiv bindet, und warum Software dazu verführen kann, Einfluss auf Wahlen in einem anderen Land zu nehmen.Das Buch eignet sich als Grundlage für »Informatik und Gesellschaft« Vorlesungen in der Informatik, genauso als Grundlage für Seminare in den Gesellschaftswissenschaften oder zur Besprechung digitaler Phänomene in der Schule. Es bietet zudem eine neue Kommunikationsmethode, die im Journalismus, der Politik oder in der Beratung eingesetzt werden kann.Leseprobe (PDF-Link)
R für Dummies (3. Auflg.)
Wollen Sie auch die umfangreichen Möglichkeiten von R nutzen, um Ihre Daten zu analysieren, sind sich aber nicht sicher, ob Sie mit der Programmiersprache wirklich zurechtkommen? Keine Sorge - dieses Buch zeigt Ihnen, wie es geht - selbst wenn Sie keine Vorkenntnisse in der Programmierung oder Statistik haben. Andrie de Vries und Joris Meys zeigen Ihnen Schritt für Schritt und anhand zahlreicher Beispiele, was Sie alles mit R machen können und vor allem wie Sie es machen können. Von den Grundlagen und den ersten Skripten bis hin zu komplexen statistischen Analysen und der Erstellung aussagekräftiger Grafiken. Auch fortgeschrittenere Nutzer finden in diesem Buch viele Tipps und Tricks, die Ihnen die Datenauswertung erleichtern. Andrie de Vries ist Berater in einem Marktforschungsunternehmen und hat sich auf die statistische Auswertung von Umfragen spezialisiert. Joris Meys ist Statistiker und R-Programmierer an der Faculty of Bio-Engineering der University of Ghent. Er hat zahlreiche R-Packages entwickelt.Über die Autoren 7Einleitung 21TEIL I: SIND SIE BEREIT? 29Kapitel 1: R im Überblick 31Kapitel 2: R erkunden 37Kapitel 3: Die Grundlagen von R 53TEIL II: ARBEITEN MIT R67Kapitel 4: Erste Schritte mit Arithmetik 69Kapitel 5: Erste Schritte im Lesen und Schreiben 95Kapitel 6: Ihr erstes Date mit R 119Kapitel 7: Arbeiten in höheren Dimensionen 129TEIL III: PROGRAMMIEREN IN R163Kapitel 8: Mehr Fun mit Funktionen 165Kapitel 9: Die Ablauflogik kontrollieren 185Kapitel 10: Fehlersuche 205Kapitel 11: Hilfe erhalten 221TEIL IV: DATEN ZUM REDEN BRINGEN231Kapitel 12: Daten lesen und schreiben 233Kapitel 13: Mit Daten arbeiten 249Kapitel 14: Daten verdichten 283Kapitel 15: Differenzen und Relationen untersuchen 307TEIL V: MIT GRAFIKEN ARBEITEN333Kapitel 16: Mit den Basisfunktionen für Grafik arbeiten 335Kapitel 17: Rastergrafiken mit »lattice«351Kapitel 18: Grammatik für Grafik: »ggplot2« 369TEIL VI: DER TOP-TEN-TEIL385Kapitel 19: Zehnmal R statt Excel 387Kapitel 20: Zehn Tipps zum Arbeiten mit Packages 397Anhang A: R und RStudio installieren 403Anhang B: Das »rfordummies«-Paket 409Stichwortverzeichnis 413
Roboter in der Bildung
Wie Robotik das Lernen im digitalen Zeitalter bereichern kann.Der Bildungsbereich verändert sich durch die Einführung digitaler Technologien. Roboter sind die Brücke zwischen der digitalen und der physischen Welt und daher ein wesentliches Thema in und für die Bildung. Dies hat einen direkten Einfluss darauf, wie und was wir den Lernenden beibringen.Dieses Buch bietet eine Einführung in die Verwendung und den Einsatz von Robotern in der Bildung und hilft Forschern geeignete Soft- und Hardware zu entwickeln. Lehrer und Trainer erfahren, wie sie Roboter in ihrer Arbeit mit Schülern und Studenten einsetzen können. Es bietet eine Einführung in die einschlägigen Lehr- und Lerntheorien im Zusammenhang mit dem veränderten Lernen sowie praktische Ratschläge zum Einsatz von Robotern als Teil eines Lehrplans.Leseprobe (PDF-Link)
Cognitive Engineering for Next Generation Computing
The cognitive approach to the IoT provides connectivity to everyone and everything since IoT connected devices are known to increase rapidly. When the IoT is integrated with cognitive technology, performance is improved, and smart intelligence is obtained. Discussed in this book are different types of datasets with structured content based on cognitive systems. The IoT gathers the information from the real time datasets through the internet, where the IoT network connects with multiple devices.This book mainly concentrates on providing the best solutions to existing real-time issues in the cognitive domain. Healthcare-based, cloud-based and smart transportation-based applications in the cognitive domain are addressed. The data integrity and security aspects of the cognitive computing main are also thoroughly discussed along with validated results.KOLLA BHANU PRAKASH is Professor and Research Group Head for Artificial Intelligence and Data Science Research Group in CSE Department, K L University, Andhra Pradesh, India. He received his MSc and MPhil in Physics from Acharya Nagarjuna University and his ME and PhD in Computer Science & Engineering from Sathyabama University, Chennai, India. Dr. Prakash has 14+ years of experience working in academia, research, and teaching. He has published multiple SCI journal articles as well as been granted 5 patents.G. R. KANAGACHIDAMBARESAN received his BE degree in Electrical and Electronics Engineering from Anna University in 2010; ME in Pervasive Computing Technologies in Anna University in 2012, and his PhD in Anna University Chennai in 2017. He is currently an associate professor, Department of CSE, Veltech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology.SRIKANTH VEMURU is a professor in the Department of Computer Science and Engineering, K L University. He received his PhD degree from Acharya Nagarjuna University (ANU) in 2011. He has more than 17 years of academic experience and in the software industry, and has published more than over 60 research papers in SCI journals and flagship conferences.VAMSIDHAR ENIREDDY is an associate professor in CSE Department, K L University, Andhra Pradesh, India. He received his PhD from JNTU Kakinada, India. Dr. Enireddy has 17+years of experience working in academia, research, and teaching. He has authored over 28 research papers in various national and international journals and conferences as well as been granted 3 patents and 1 patent filed.Preface xviiAcknowledgments xix1 INTRODUCTION TO COGNITIVE COMPUTING 1Vamsidhar Enireddy, Sagar Imambi and C. Karthikeyan1.1 Introduction: Definition of Cognition, Cognitive Computing 11.2 Defining and Understanding Cognitive Computing 21.3 Cognitive Computing Evolution and Importance 61.4 Difference Between Cognitive Computing and Artificial Intelligence 81.5 The Elements of a Cognitive System 111.5.1 Infrastructure and Deployment Modalities 111.5.2 Data Access, Metadata, and Management Services 121.5.3 The Corpus, Taxonomies, and Data Catalogs 121.5.4 Data Analytics Services 121.5.5 Constant Machine Learning 131.5.6 Components of a Cognitive System 131.5.7 Building the Corpus 141.5.8 Corpus Administration Governing and Protection Factors 161.6 Ingesting Data Into Cognitive System 171.6.1 Leveraging Interior and Exterior Data Sources 171.6.2 Data Access and Feature Extraction 181.7 Analytics Services 191.8 Machine Learning 221.9 Machine Learning Process 241.9.1 Data Collection 241.9.2 Data Preparation 241.9.3 Choosing a Model 241.9.4 Training the Model 241.9.5 Evaluate the Model 251.9.6 Parameter Tuning 251.9.7 Make Predictions 251.10 Machine Learning Techniques 251.10.1 Supervised Learning 251.10.2 Unsupervised Learning 271.10.3 Reinforcement Learning 271.10.4 The Significant Challenges in Machine Learning 281.11 Hypothesis Space 301.11.1 Hypothesis Generation 311.11.2 Hypotheses Score 321.12 Developing a Cognitive Computing Application 321.13 Building a Health Care Application 351.13.1 Healthcare Ecosystem Constituents 351.13.2 Beginning With a Cognitive Healthcare Application 371.13.3 Characterize the Questions Asked by the Clients 371.13.4 Creating a Corpus and Ingesting the Content 381.13.5 Training the System 381.13.6 Applying Cognition to Develop Health and Wellness 391.13.7 Welltok 391.13.8 CaféWell Concierge in Action 411.14 Advantages of Cognitive Computing 421.15 Features of Cognitive Computing 431.16 Limitations of Cognitive Computing 441.17 Conclusion 47References 472 MACHINE LEARNING AND BIG DATA IN CYBER-PHYSICAL SYSTEM: METHODS, APPLICATIONS AND CHALLENGES 49Janmenjoy Nayak, P. Suresh Kumar, Dukka Karun Kumar Reddy, Bighnaraj Naik and Danilo Pelusi2.1 Introduction 502.2 Cyber-Physical System Architecture 522.3 Human-in-the-Loop Cyber-Physical Systems (HiLCPS) 532.4 Machine Learning Applications in CPS 552.4.1 K-Nearest Neighbors (K-NN) in CPS 552.4.2 Support Vector Machine (SVM) in CPS 582.4.3 Random Forest (RF) in CPS 612.4.4 Decision Trees (DT) in CPS 632.4.5 Linear Regression (LR) in CPS 652.4.6 Multi-Layer Perceptron (MLP) in CPS 662.4.7 Naive Bayes (NB) in CPS 702.5 Use of IoT in CPS 702.6 Use of Big Data in CPS 722.7 Critical Analysis 772.8 Conclusion 83References 843 HEMOSMART: A NON-INVASIVE DEVICE AND MOBILE APP FOR ANEMIA DETECTION 93J.A.D.C.A. Jayakody, E.A.G.A. Edirisinghe and S.Lokuliyana3.1 Introduction 943.1.1 Background 943.1.2 Research Objectives 963.1.3 Research Approach 973.1.4 Limitations 983.2 Literature Review 983.3 Methodology 1013.3.1 Methodological Approach 1013.3.1.1 Select an Appropriate Camera 1023.3.1.2 Design the Lighting System 1023.3.1.3 Design the Electronic Circuit 1043.3.1.4 Design the Prototype 1043.3.1.5 Collect Data and Develop the Algorithm 1043.3.1.6 Develop the Prototype 1063.3.1.7 Mobile Application Development 1063.3.1.8 Completed Device 1073.3.1.9 Methods of Data Collection 1093.3.2 Methods of Analysis 1093.4 Results 1103.4.1 Impact of Project Outcomes 1103.4.2 Results Obtained During the Methodology 1113.4.2.1 Select an Appropriate Camera 1113.4.2.2 Design the Lighting System 1123.5 Discussion 1123.6 Originality and Innovativeness of the Research 1163.6.1 Validation and Quality Control of Methods 1173.6.2 Cost-Effectiveness of the Research 1173.7 Conclusion 117References 1174 ADVANCED COGNITIVE MODELS AND ALGORITHMS 121J. Ramkumar, M. Baskar and B. Amutha4.1 Introduction 1224.2 Microsoft Azure Cognitive Model 1224.2.1 AI Services Broaden in Microsoft Azure 1254.3 IBM Watson Cognitive Analytics 1264.3.1 Cognitive Computing 1264.3.2 Defining Cognitive Computing via IBM Watson Interface 1274.3.2.1 Evolution of Systems Towards Cognitive Computing 1284.3.2.2 Main Aspects of IBM Watson 1294.3.2.3 Key Areas of IBM Watson 1304.3.3 IBM Watson Analytics 1304.3.3.1 IBM Watson Features 1314.3.3.2 IBM Watson DashDB 1314.4 Natural Language Modeling 1324.4.1 NLP Mainstream 1324.4.2 Natural Language Based on Cognitive Computation 1344.5 Representation of Knowledge Models 1344.6 Conclusion 137References 1385 IPARKING—SMART WAY TO AUTOMATE THE MANAGEMENT OF THE PARKING SYSTEM FOR A SMART CITY 141J.A.D.C.A. Jayakody, E.A.G.A. Edirisinghe, S.A.H.M. Karunanayaka, E.M.C.S. Ekanayake, H.K.T.M. Dikkumbura and L.A.I.M. Bandara5.1 Introduction 1425.2 Background & Literature Review 1445.2.1 Background 1445.2.2 Review of Literature 1455.3 Research Gap 1515.4 Research Problem 1515.5 Objectives 1535.6 Methodology 1545.6.1 Lot Availability and Occupancy Detection 1545.6.2 Error Analysis for GPS (Global Positioning System) 1555.6.3 Vehicle License Plate Detection System 1565.6.4 Analyze Differential Parking Behaviors and Pricing 1565.6.5 Targeted Digital Advertising 1575.6.6 Used Technologies 1575.6.7 Specific Tools and Libraries 1585.7 Testing and Evaluation 1595.8 Results 1615.9 Discussion 1625.10 Conclusion 164References 1656 COGNITIVE CYBER-PHYSICAL SYSTEM APPLICATIONS 167John A., Senthilkumar Mohan and D. Maria Manuel Vianny6.1 Introduction 1686.2 Properties of Cognitive Cyber-Physical System 1696.3 Components of Cognitive Cyber-Physical System 1706.4 Relationship Between Cyber-Physical System for Human–Robot 1716.5 Applications of Cognitive Cyber-Physical System 1726.5.1 Transportation 1726.5.2 Industrial Automation 1736.5.3 Healthcare and Biomedical 1766.5.4 Clinical Infrastructure 1786.5.5 Agriculture 1806.6 Case Study: Road Management System Using CPS 1816.6.1 Smart Accident Response System for Indian City 1826.7 Conclusion 184References 1857 COGNITIVE COMPUTING 189T Gunasekhar and Marella Surya Teja7.1 Introduction 1897.2 Evolution of Cognitive System 1917.3 Cognitive Computing Architecture 1937.3.1 Cognitive Computing and Internet of Things 1947.3.2 Cognitive Computing and Big Data Analysis 1977.3.3 Cognitive Computing and Cloud Computing 2007.4 Enabling Technologies in Cognitive Computing 2027.4.1 Cognitive Computing and Reinforcement Learning 2027.4.2 Cognitive Computive and Deep Learning 2047.4.2.1 Rational Method and Perceptual Method 2057.4.2.2 Cognitive Computing and Image Understanding 2077.5 Applications of Cognitive Computing 2097.5.1 Chatbots 2097.5.2 Sentiment Analysis 2107.5.3 Face Detection 2117.5.4 Risk Assessment 2117.6 Future of Cognitive Computing 2127.7 Conclusion 214References 2158 TOOLS USED FOR RESEARCH IN COGNITIVE ENGINEERING AND CYBER PHYSICAL SYSTEMS 219Ajita Seth8.1 Cyber Physical Systems 2198.2 Introduction: The Four Phases of Industrial Revolution 2208.3 System 2218.4 Autonomous Automobile System 2218.4.1 The Timeline 2228.5 Robotic System 2238.6 Mechatronics 225References 2289 ROLE OF RECENT TECHNOLOGIES IN COGNITIVE SYSTEMS 231V. Pradeep Kumar, L. Pallavi and Kolla Bhanu Prakash9.1 Introduction 2329.1.1 Definition and Scope of Cognitive Computing 2329.1.2 Architecture of Cognitive Computing 2339.1.3 Features and Limitations of Cognitive Systems 2349.2 Natural Language Processing for Cognitive Systems 2369.2.1 Role of NLP in Cognitive Systems 2369.2.2 Linguistic Analysis 2389.2.3 Example Applications Using NLP With Cognitive Systems 2409.3 Taxonomies and Ontologies of Knowledge Representation for Cognitive Systems 2419.3.1 Taxonomies and Ontologies and Their Importance in Knowledge Representation 2429.3.2 How to Represent Knowledge in Cognitive Systems? 2439.3.3 Methodologies Used for Knowledge Representation in Cognitive Systems 2479.4 Support of Cloud Computing for Cognitive Systems 2489.4.1 Importance of Shared Resources of Distributed Computing in Developing Cognitive Systems 2489.4.2 Fundamental Concepts of Cloud Used in Building Cognitive Systems 2499.5 Cognitive Analytics for Automatic Fraud Detection Using Machine Learning and Fuzzy Systems 2549.5.1 Role of Machine Learning Concepts in Building Cognitive Analytics 2559.5.2 Building Automated Patterns for Cognitive Analytics Using Fuzzy Systems 2559.6 Design of Cognitive System for Healthcare Monitoring in Detecting Diseases 2569.6.1 Role of Cognitive System in Building Clinical Decision System 2579.7 Advanced High Standard Applications Using Cognitive Computing 2599.8 Conclusion 262References 26310 QUANTUM META-HEURISTICS AND APPLICATIONS 265Kolla Bhanu Prakash10.1 Introduction 26510.2 What is Quantum Computing? 26710.3 Quantum Computing Challenges 26810.4 Meta-Heuristics and Quantum Meta-Heuristics Solution Approaches 27110.5 Quantum Meta-Heuristics Algorithms With Application Areas 27310.5.1 Quantum Meta-Heuristics Applications for Power Systems 27710.5.2 Quantum Meta-Heuristics Applications for Image Analysis 28110.5.3 Quantum Meta-Heuristics Applications for Big Data or Data Mining 28210.5.4 Quantum Meta-Heuristics Applications for Vehicular Trafficking 28510.5.5 Quantum Meta-Heuristics Applications for Cloud Computing 28610.5.6 Quantum Meta-Heuristics Applications for Bioenergy or Biomedical Systems 28710.5.7 Quantum Meta-Heuristics Applications for Cryptography or Cyber Security 28710.5.8 Quantum Meta-Heuristics Applications for Miscellaneous Domain 288References 29111 ENSURING SECURITY AND PRIVACY IN IOT FOR HEALTHCARE APPLICATIONS 299Anjali Yeole and D.R. Kalbande11.1 Introduction 29911.2 Need of IoT in Healthcare 30011.2.1 Available Internet of Things Devices for Healthcare 30111.3 Literature Survey on an IoT-Aware Architecture for Smart Healthcare Systems 30311.3.1 Cyber-Physical System (CPS) for e-Healthcare 30311.3.2 IoT-Enabled Healthcare With REST-Based Services 30411.3.3 Smart Hospital System 30411.3.4 Freescale Home Health Hub Reference Platform 30511.3.5 A Smart System Connecting e-Health Sensors and Cloud 30511.3.6 Customizing 6LoWPAN Networks Towards IoT-Based Ubiquitous Healthcare Systems 30511.4 IoT in Healthcare: Challenges and Issues 30611.4.1 Challenges of the Internet of Things for Healthcare 30611.4.2 IoT Interoperability Issues 30811.4.3 IoT Security Issues 30811.4.3.1 Security of IoT Sensors 30911.4.3.2 Security of Data Generated by Sensors 30911.4.3.3 LoWPAN Networks Healthcare Systems and its Attacks 30911.5 Proposed System: 6LoWPAN and COAP Protocol-Based IoT System for Medical Data Transfer by Preserving Privacy of Patient 31011.6 Conclusion 312References 31212 EMPOWERING SECURED OUTSOURCING IN CLOUD STORAGE THROUGH DATA INTEGRITY VERIFICATION 315C. Saranya Jothi, Carmel Mary Belinda and N. Rajkumar12.1 Introduction 31512.1.1 Confidentiality 31612.1.2 Availability 31612.1.3 Information Uprightness 31612.2 Literature Survey 31612.2.1 PDP 31612.2.1.1 Privacy-Preserving PDP Schemes 31712.2.1.2 Efficient PDP 31712.2.2 POR 31712.2.3 HAIL 31812.2.4 RACS 31812.2.5 FMSR 31812.3 System Design 31912.3.1 Design Considerations 31912.3.2 System Overview 32012.3.3 Workflow 32012.3.4 System Description 32112.3.4.1 System Encoding 32112.3.4.2 Decoding 32212.3.4.3 Repair and Check 32312.4 Implementation and Result Discussion 32412.4.1 Creating Containers 32412.4.2 File Chunking 32412.4.3 XORing Partitions 32612.4.4 Regeneration of File 32612.4.5 Reconstructing a Node 32712.4.6 Cloud Storage 32712.4.6.1 NC-Cloud 32712.4.6.2 Open Swift 32912.5 Performance 33012.6 Conclusion 332References 333Index 335
Wireless Network Simulation
Learn to run your own simulation by working with model analysis, mathematical background, simulation output data, and most importantly, a network simulator for wireless technology. This book introduces the best practices of simulator use, the techniques for analyzing simulations with artificial agents and the integration with other technologies such as Power Line Communications (PLC).Network simulation is a key technique used to test the future behavior of a network. It’s a vital development component for the development of 5G, IoT, wireless sensor networks, and many more. This book explains the scope and evolution of the technology that has led to the development of dynamic systems such as Internet of Things and fog computing.You'll focus on the ad hoc networks with stochastic behavior and dynamic nature, and the ns-3 simulator. These are useful open source tools for academics, researchers, students and engineers to deploy telecommunications experiments, proofs and new scenarios with a high degree of similarity with reality. You'll also benefit from a detailed explanation of the examples and the theoretical components needed to deploy wireless simulations or wired, if necessary.WHAT YOU’LL LEARN* Review best practices of simulator uses* Understand techniques for analyzing simulations with artificial agents* Apply simulation techniques and experiment design* Program on ns-3 simulator* Analyze simulation results* Create new modules or protocols for wired and wireless networksWHO THIS BOOK IS FORUndergraduate and postgraduate students, researchers and professors interested in network simulations. This book also includes theoretical components about simulation, which are useful for those interested in discrete event simulation DES, general theory of simulation, wireless simulation and ns-3 simulator.HENRY ZÁRATE CEBALLOS received his PhD in Engineering Computing and Systems and Masters Degree in Telecommunications from the National University of Colombia. Henry is currently a researcher with the TLÖN Group. Henry has worked extesensively with the Ns-2 and Ns-3 simulators and wireless distributed operative systems.JORGE ERNESTO PARRA AMARIS received his Masters Degree in Telecommunication from the National University of Colombia, and is an Electronics Engineer from the Colombian School of Engineering Julio Garavito. Jorge's Masters thesis proposed a unique algorithm which was validated through simulation using NS-3.HERNÁN JIMÉNEZ JIMÉNEZ received his postgraduate Masters in Telecommunications from the National University of Colombia. Hernán is currently a researcher at TLÖN Group.DIEGO ALEXIS ROMERO RINCÓN received his Masters in Electronics from the National University of Colombia and is currently a researcher with the TLÖN Group. Diego focused his Masters thesis on on the NS-3 simulator. Deigo is currently a lecturer at the National University of Colombia.OSCAR AGUDELO ROJAS is a systems engineer and lecturer at the National University of Colombia, where he also received his Masters degree in Telecommunications. His research work includes networks (wired and wireless), network coding, simulation (ns2-ns3) and parallel and distributed systems.JORGE EDUARDO ORTIZ TRIVIÑO received his PhD in Engineering Computing Systems and Masters Degrees in Telecommunications, Statistics, and Philosophy from the National University of Colombia. Jorge is currently a professor at the National University of Colombia, while also working as a network specialist.Chapter 1: Introduction to Simulation 3.- Chapter 2: Wireless and Ad hoc Networks.- Chapter 3: Design of Simulation Experiments.- Chapter 4: Network Simulation using NS3.- Chapter 5: Analyses of Results.- Chapter 6: Manet Simulation on NS3.- Chapter 7: Manets and PLC on NS3.-Appendix A: Basic Statistics.- Appendix B: NS3 Installation.- Appendix C: Mininet.- Appendix C: NS3-GYM: Openai Gym Integration.- Appendix E: Experiment.- Appendix F: PLC Code Experiment.-
Information Refinement Technologies for Crisis Informatics
Marc-André Kaufhold explores user expectations and design implications for the utilization of new media in crisis management and response. He develops a novel framework for information refinement, which integrates the event, organisational, societal, and technological perspectives of crises. Therefore, he reviews the state of the art on crisis informatics and empirically examines the use, potentials and barriers of both social media and mobile apps. Based on these insights, he designs and evaluates ICT concepts and artifacts with the aim to overcome the issues of information overload and quality in large-scale crises, concluding with practical and theoretical implications for technology adaptation and design.About the author:Marc-André Kaufhold is a postdoc at the Chair of Science and Technology for Peace and Security (PEASEC) in the Department of Computer Science at the Technical University of Darmstadt. His research focuses on the user-centred design and evaluation of mobile apps and social media technologies in the context of crisis and security research.Part I: Outline.- Part II: Theoretical and Empirical Findings.- Part III: Design and Evaluation Findings.- Part IV: Conclusion and Outlook.
Machine Learning for Healthcare Applications
When considering the idea of using machine learning in healthcare, it is a Herculean task to present the entire gamut of information in the field of intelligent systems. It is, therefore the objective of this book to keep the presentation narrow and intensive. This approach is distinct from others in that it presents detailed computer simulations for all models presented with explanations of the program code. It includes unique and distinctive chapters on disease diagnosis, telemedicine, medical imaging, smart health monitoring, social media healthcare, and machine learning for COVID-19. These chapters help develop a clear understanding of the working of an algorithm while strengthening logical thinking. In this environment, answering a single question may require accessing several data sources and calling on sophisticated analysis tools. While data integration is a dynamic research area in the database community, the specific needs of research have led to the development of numerous middleware systems that provide seamless data access in a result-driven environment.Since this book is intended to be useful to a wide audience, students, researchers and scientists from both academia and industry may all benefit from this material. It contains a comprehensive description of issues for healthcare data management and an overview of existing systems, making it appropriate for introductory and instructional purposes. Prerequisites are minimal; the readers are expected to have basic knowledge of machine learning.This book is divided into 22 real-time innovative chapters which provide a variety of application examples in different domains. These chapters illustrate why traditional approaches often fail to meet customers’ needs. The presented approaches provide a comprehensive overview of current technology. Each of these chapters, which are written by the main inventors of the presented systems, specifies requirements and provides a description of both the chosen approach and its implementation. Because of the self-contained nature of these chapters, they may be read in any order. Each of the chapters use various technical terms which involve expertise in machine learning and computer science.SACHI NANDAN MOHANTY received his PhD from IIT Kharagpur in 2015. He has recently joined as an associate professor in the Department of Computer Science & Engineering at ICFAI Foundation for Higher Education Hyderabad. His research areas include data mining, big data analysis, cognitive science, fuzzy decision making, brain-computer interface, and computational intelligence. He has published 20 SCI journal articles and has authored/edited 7 books.G. NALINIPRIYA is a professor in the Department of Information Technology, Anna University, Chennai where she also obtained her PhD. She has more than 23 years of experience in the field of teaching, industry and research and her interests include artificial intelligence, machine learning, data science and cloud security.OM PRAKASH JENA is an assistant professor in the Department of Computer Science, Ravenshaw University, Cuttack, Odisha. He has 10 years of teaching and research experience and has published several technical papers in international journals/conferences/edited books. His current research interests include pattern recognition, cryptography, network security, soft computing, data analytics and machine automation.ACHYUTH SARKAR received his PhD in Computer Science and Engineering from the National Institute of Technology, Arunachal Pradesh in 2019. He has teaching experience of more than 10 years.Preface xviiPART 1: INTRODUCTION TO INTELLIGENT HEALTHCARE SYSTEMS 11 INNOVATION ON MACHINE LEARNING IN HEALTHCARE SERVICES—AN INTRODUCTION 3Parthasarathi Pattnayak and Om Prakash Jena1.1 Introduction 31.2 Need for Change in Healthcare 51.3 Opportunities of Machine Learning in Healthcare 61.4 Healthcare Fraud 71.4.1 Sorts of Fraud in Healthcare 71.4.2 Clinical Service Providers 81.4.3 Clinical Resource Providers 81.4.4 Protection Policy Holders 81.4.5 Protection Policy Providers 91.5 Fraud Detection and Data Mining in Healthcare 91.5.1 Data Mining Supervised Methods 101.5.2 Data Mining Unsupervised Methods 101.6 Common Machine Learning Applications in Healthcare 101.6.1 Multimodal Machine Learning for Data Fusion in Medical Imaging 111.6.2 Machine Learning in Patient Risk Stratification 111.6.3 Machine Learning in Telemedicine 111.6.4 AI (ML) Application in Sedate Revelation 121.6.5 Neuroscience and Image Computing 121.6.6 Cloud Figuring Systems in Building AI-Based Healthcare 121.6.7 Applying Internet of Things and Machine-Learning for Personalized Healthcare 121.6.8 Machine Learning in Outbreak Prediction 131.7 Conclusion 13References 14PART 2: MACHINE LEARNING/DEEP LEARNING-BASED MODEL DEVELOPMENT 172 A FRAMEWORK FOR HEALTH STATUS ESTIMATION BASED ON DAILY LIFE ACTIVITIES DATA USING MACHINE LEARNING TECHNIQUES 19Tene Ramakrishnudu, T. Sai Prasen and V. Tharun Chakravarthy2.1 Introduction 192.1.1 Health Status of an Individual 192.1.2 Activities and Measures of an Individual 202.1.3 Traditional Approach to Predict Health Status 202.2 Background 202.3 Problem Statement 212.4 Proposed Architecture 222.4.1 Pre-Processing 222.4.2 Phase-I 232.4.3 Phase-II 232.4.4 Dataset Generation 232.4.4.1 Rules Collection 232.4.4.2 Feature Selection 242.4.4.3 Feature Reduction 242.4.4.4 Dataset Generation From Rules 242.4.4.5 Example 242.4.5 Pre-Processing 262.5 Experimental Results 272.5.1 Performance Metrics 272.5.1.1 Accuracy 272.5.1.2 Precision 282.5.1.3 Recall 282.5.1.4 F1-Score 302.6 Conclusion 31References 313 STUDY OF NEUROMARKETING WITH EEG SIGNALS AND MACHINE LEARNING TECHNIQUES 33S. Pal, P. Das, R. Sahu and S.R. Dash3.1 Introduction 343.1.1 Why BCI 343.1.2 Human–Computer Interfaces 343.1.3 What is EEG 353.1.4 History of EEG 353.1.5 About Neuromarketing 353.1.6 About Machine Learning 363.2 Literature Survey 363.3 Methodology 453.3.1 Bagging Decision Tree Classifier 453.3.2 Gaussian Naïve Bayes Classifier 453.3.3 Kernel Support Vector Machine (Sigmoid) 453.3.4 Random Decision Forest Classifier 463.4 System Setup & Design 463.4.1 Pre-Processing & Feature Extraction 473.4.1.1 Savitzky–Golay Filter 473.4.1.2 Discrete Wavelet Transform 483.4.2 Dataset Description 493.5 Result 493.5.1 Individual Result Analysis 493.5.2 Comparative Results Analysis 523.6 Conclusion 53References 544 AN EXPERT SYSTEM-BASED CLINICAL DECISION SUPPORT SYSTEM FOR HEPATITIS-B PREDICTION & DIAGNOSIS 57Niranjan Panigrahi, Ishan Ayus and Om Prakash Jena4.1 Introduction 574.2 Outline of Clinical DSS 594.2.1 Preliminaries 594.2.2 Types of Clinical DSS 604.2.3 Non-Knowledge-Based Decision Support System (NK-DSS) 604.2.4 Knowledge-Based Decision Support System (K-DSS) 624.2.5 Hybrid Decision Support System (H-DSS) 644.2.6 DSS Architecture 644.3 Background 654.4 Proposed Expert System-Based CDSS 654.4.1 Problem Description 654.4.2 Rules Set & Knowledge Base 664.4.3 Inference Engine 664.5 Implementation & Testing 664.6 Conclusion 73References 735 DEEP LEARNING ON SYMPTOMS IN DISEASE PREDICTION 77Sheikh Raul Islam, Rohit Sinha, Santi P. Maity and Ajoy Kumar Ray5.1 Introduction 775.2 Literature Review 785.3 Mathematical Models 795.3.1 Graphs and Related Terms 805.3.2 Deep Learning in Graph 805.3.3 Network Embedding 805.3.4 Graph Neural Network 815.3.5 Graph Convolution Network 825.4 Learning Representation From DSN 825.4.1 Description of the Proposed Model 835.4.2 Objective Function 845.5 Results and Discussion 845.5.1 Description of the Dataset 855.5.2 Training Progress 855.5.3 Performance Comparisons 865.6 Conclusions and Future Scope 86References 876 INTELLIGENT VISION-BASED SYSTEMS FOR PUBLIC SAFETY AND PROTECTION VIA MACHINE LEARNING TECHNIQUES 89Rajitha B.6.1 Introduction 896.1.1 Problems Intended in Video Surveillance Systems 906.1.2 Current Developments in This Area 916.1.3 Role of AI in Video Surveillance Systems 916.2 Public Safety and Video Surveillance Systems 926.2.1 Offline Crime Prevention 926.2.2 Crime Prevention and Identification via Apps 926.2.3 Crime Prevention and Identification via CCTV 926.3 Machine Learning for Public Safety 946.3.1 Abnormality Behavior Detection via Deep Learning 956.3.2 Video Analytics Methods for Accident Classification/Detection 976.3.3 Feature Selection and Fusion Methods 986.4 Securing the CCTV Data 996.4.1 Image/Video Security Challenges 996.4.2 Blockchain for Image/Video Security 996.5 Conclusion 99References 1007 SEMANTIC FRAMEWORK IN HEALTHCARE 103Sankar Pariserum Perumal, Ganapathy Sannasi, Selvi M. and Kannan Arputharaj7.1 Introduction 1037.2 Semantic Web Ontology 1047.3 Multi-Agent System in a Semantic Framework 1067.3.1 Existing Healthcare Semantic Frameworks 1077.3.1.1 AOIS 1077.3.1.2 SCKE 1087.3.1.3 MASE 1097.3.1.4 MET4 1107.3.2 Proposed Multi-Agent-Based Semantic Framework for Healthcare Instance Data 1117.3.2.1 Data Dictionary 1117.3.2.2 Mapping Database 1127.3.2.3 Decision Making Ontology 1137.3.2.4 STTL and SPARQL-Based RDF Transformation 1157.3.2.5 Query Optimizer Agent 1167.3.2.6 Semantic Web Services Ontology 1167.3.2.7 Web Application User Interface and Customer Agent 1167.3.2.8 Translation Agent 1177.3.2.9 RDF Translator 1177.4 Conclusion 118References 1198 DETECTION, PREDICTION & INTERVENTION OF ATTENTION DEFICIENCY IN THE BRAIN USING TDCS 121Pallabjyoti Kakoti, Rissnalin Syiemlieh and Eeshankur Saikia8.1 Introduction 1218.2 Materials & Methods 1238.2.1 Subjects and Experimental Design 1238.2.2 Data Pre-Processing & Statistical Analysis 1258.2.3 Extracting Singularity Spectrum from EEG 1268.3 Results & Discussion 1268.4 Conclusion 132Acknowledgement 133References 1339 DETECTION OF ONSET AND PROGRESSION OF OSTEOPOROSIS USING MACHINE LEARNING 137Shilpi Ruchi Kerketta and Debalina Ghosh9.1 Introduction 1379.1.1 Measurement Techniques of BMD 1389.1.2 Machine Learning Algorithms in Healthcare 1389.1.3 Organization of Chapter 1399.2 Microwave Characterization of Human Osseous Tissue 1399.2.1 Frequency-Domain Analysis of Human Wrist Sample 1409.2.2 Data Collection and Analysis 1419.3 Prediction Model of Osteoporosis Using Machine Learning Algorithms 1449.3.1 K-Nearest Neighbor (KNN) 1449.3.2 Decision Tree 1459.3.3 Random Forest 1459.4 Conclusion 148Acknowledgment 148References 14810 APPLICATIONS OF MACHINE LEARNING IN BIOMEDICAL TEXT PROCESSING AND FOOD INDUSTRY 151K. Paramesha, Gururaj H.L. and Om Prakash Jena10.1 Introduction 15210.2 Use Cases of AI and ML in Healthcare 15310.2.1 Speech Recognition (SR) 15310.2.2 Pharmacovigilance and Adverse Drug Effects (ADE) 15310.2.3 Clinical Imaging and Diagnostics 15310.2.4 Conversational AI in Healthcare 15410.3 Use Cases of AI and ML in Food Technology 15410.3.1 Assortment of Vegetables and Fruits 15410.3.2 Personal Hygiene 15410.3.3 Developing New Products 15510.3.4 Plant Leaf Disease Detection 15610.3.5 Face Recognition Systems for Domestic Cattle 15610.3.6 Cleaning Processing Equipment 15710.4 A Case Study: Sentiment Analysis of Drug Reviews 15810.4.1 Dataset 15910.4.2 Approaches for Sentiment Analysis on Drug Reviews 15910.4.3 BoW and TF-IDF Model 16010.4.4 Bi-LSTM Model 16010.4.4.1 Word Embedding 16010.4.5 Deep Learning Model 16110.5 Results and Analysis 16410.6 Conclusion 165References 16611 COMPARISON OF MOBILENET AND RESNET CNN ARCHITECTURES IN THE CNN-BASED SKIN CANCER CLASSIFIER MODEL 169Subasish Mohapatra, N.V.S. Abhishek, Dibyajit Bardhan, Anisha Ankita Ghosh and Shubhadarshinin Mohanty11.1 Introduction 16911.2 Our Skin Cancer Classifier Model 17111.3 Skin Cancer Classifier Model Results 17211.4 Hyperparameter Tuning and Performance 17411.4.1 Hyperparameter Tuning of MobileNet-Based CNN Model 17511.4.2 Hyperparameter Tuning of ResNet50-Based CNN Model 17511.4.3 Table Summary of Hyperparameter Tuning Results 17611.5 Comparative Analysis and Results 17611.5.1 Training and Validation Loss 17711.5.1.1 MobileNet 17711.5.1.2 ResNet50 17711.5.1.3 Inferences 17711.5.2 Training and Validation Categorical Accuracy 17811.5.2.1 MobileNet 17811.5.2.2 ResNet50 17811.5.2.3 Inferences 17811.5.3 Training and Validation Top 2 Accuracy 17911.5.3.1 MobileNet 17911.5.3.2 ResNet50 17911.5.3.3 Inferences 18011.5.4 Training and Validation Top 3 Accuracy 18011.5.4.1 MobileNet 18011.5.4.2 ResNet50 18011.5.4.3 Inferences 18111.5.5 Confusion Matrix 18111.5.5.1 MobileNet 18111.5.5.2 ResNet50 18111.5.5.3 Inferences 18211.5.6 Classification Report 18211.5.6.1 MobileNet 18211.5.6.2 ResNet50 18211.5.6.3 Inferences 18311.5.7 Last Epoch Results 18311.5.7.1 MobileNet 18311.5.7.2 ResNet50 18311.5.7.3 Inferences 18411.5.8 Best Epoch Results 18411.5.8.1 MobileNet 18411.5.8.2 ResNet50 18411.5.8.3 Inferences 18411.5.9 Overall Comparative Analysis 18411.6 Conclusion 185References 18512 DEEP LEARNING-BASED IMAGE CLASSIFIER FOR MALARIA CELL DETECTION 187Alok Negi, Krishan Kumar and Prachi Chauhan12.1 Introduction 18712.2 Related Work 18912.3 Proposed Work 19012.3.1 Dataset Description 19112.3.2 Data Pre-Processing and Augmentation 19112.3.3 CNN Architecture and Implementation 19212.4 Results and Evaluation 19412.5 Conclusion 196References 19713 PREDICTION OF CHEST DISEASES USING TRANSFER LEARNING 199S. Baghavathi Priya, M. Rajamanogaran and S. Subha13.1 Introduction 19913.2 Types of Diseases 20013.2.1 Pneumothorax 20013.2.2 Pneumonia 20013.2.3 Effusion 20013.2.4 Atelectasis 20113.2.5 Nodule and Mass 20213.2.6 Cardiomegaly 20213.2.7 Edema 20213.2.8 Lung Consolidation 20213.2.9 Pleural Thickening 20213.2.10 Infiltration 20213.2.11 Fibrosis 20313.2.12 Emphysema 20313.3 Diagnosis of Lung Diseases 20413.4 Materials and Methods 20413.4.1 Data Augmentation 20613.4.2 CNN Architecture 20613.4.3 Lung Disease Prediction Model 20713.5 Results and Discussions 20813.5.1 Implementation Results Using ROC Curve 20913.6 Conclusion 210References 21214 EARLY STAGE DETECTION OF LEUKEMIA USING ARTIFICIAL INTELLIGENCE 215Neha Agarwal and Piyush Agrawal14.1 Introduction 21514.1.1 Classification of Leukemia 21614.1.1.1 Acute Lymphocytic Leukemia 21614.1.1.2 Acute Myeloid Leukemia 21614.1.1.3 Chronic Lymphocytic Leukemia 21614.1.1.4 Chronic Myeloid Leukemia 21614.1.2 Diagnosis of Leukemia 21614.1.3 Acute and Chronic Stages of Leukemia 21714.1.4 The Role of AI in Leukemia Detection 21714.2 Literature Review 21914.3 Proposed Work 22014.3.1 Modules Involved in Proposed Methodology 22114.3.2 Flowchart 22214.3.3 Proposed Algorithm 22314.4 Conclusion and Future Aspects 223References 223PART 3: INTERNET OF MEDICAL THINGS (IOMT) FOR HEALTHCARE 22515 IOT APPLICATION IN INTERCONNECTED HOSPITALS 227Subhra Debdas, Chinmoy Kumar Panigrahi, Priyasmita Kundu, Sayantan Kundu and Ramanand Jha15.1 Introduction 22815.2 Networking Systems Using IoT 22915.3 What are Smart Hospitals? 23315.3.1 Environment of a Smart Hospital 23415.4 Assets 23615.4.1 Overview of Smart Hospital Assets 23615.4.2 Exigency of Automated Healthcare Center Assets 23915.5 Threats 24115.5.1 Emerging Vulnerabilities 24115.5.2 Threat Analysis 24415.6 Conclusion 246References 24616 REAL TIME HEALTH MONITORING USING IOT WITH INTEGRATION OF MACHINE LEARNING APPROACH 249K.G. Maheswari, G. Nalinipriya, C. Siva and A. Thilakesh Raj16.1 Introduction 25016.2 Related Work 25016.3 Existing Healthcare Monitoring System 25116.4 Methodology and Data Analysis 25116.5 Proposed System Architecture 25216.6 Machine Learning Approach 25216.6.1 Multiple Linear Regression Algorithm 25316.6.2 Random Forest Algorithm 25316.6.3 Support Vector Machine 25316.7 Work Flow of the Proposed System 25316.8 System Design of Health Monitoring System 25616.9 Use Case Diagram 25716.10 Conclusion 258References 259PART 4: MACHINE LEARNING APPLICATIONS FOR COVID-19 26117 SEMANTIC AND NLP-BASED RETRIEVAL FROM COVID-19 ONTOLOGY 263Ramar Kaladevi and Appavoo Revathi17.1 Introduction 26317.2 Related Work 26417.3 Proposed Retrieval System 26617.3.1 Why Ontology? 26617.3.2 Covid Ontology 26617.3.3 Information Retrieval From Ontology 26917.3.4 Query Formulation 27217.3.5 Retrieval From Knowledgebase 27217.4 Conclusion 273References 27318 SEMANTIC BEHAVIOR ANALYSIS OF COVID-19 PATIENTS: A COLLABORATIVE FRAMEWORK 277Amlan Mohanty, Debasish Kumar Mallick, Shantipriya Parida and Satya Ranjan Dash18.1 Introduction 27818.2 Related Work 28018.2.1 Semantic Analysis and Topic Discovery of Alcoholic Patients From Social Media Platforms 28018.2.2 Sentiment Analysis of Tweets From Twitter Handles of the People of Nepal in Response to the COVID-19 Pandemic 28018.2.3 Study of Sentiment Analysis and Analyzing Scientific Papers 28018.2.4 Informatics and COVID-19 Research 28118.2.5 COVID-19 Outbreak in the World and Twitter Sentiment Analysis 28118.2.6 LDA Topic Modeling on Twitter to Study Public Discourse and Sentiment During the Coronavirus Pandemic 28118.2.7 The First Decade of Research on Sentiment Analysis 28218.2.8 Detailed Survey on the Semantic Analysis Techniques for NLP 28218.2.9 Understanding Text Semantics With LSA 28218.2.10 Analyzing Suicidal Tendencies With Semantic Analysis Using Social Media 28318.2.11 Analyzing Public Opinion on BREXIT Using Sentiment Analysis 28318.2.12 Prediction of Indian Elections Using NLP and Decision Tree 28318.3 Methodology 28318.4 Conclusion 286References 28719 COMPARATIVE STUDY OF VARIOUS DATA MINING TECHNIQUES TOWARDS ANALYSIS AND PREDICTION OF GLOBAL COVID-19 DATASET 289Sachin Kamley19.1 Introduction 28919.2 Literature Review 29019.3 Materials and Methods 29219.3.1 Dataset Collection 29219.3.2 Support Vector Machine (SVM) 29219.3.3 Decision Tree (DT) 29419.3.4 K-Means Clustering 29419.3.5 Back Propagation Neural Network (BPNN) 29519.4 Experimental Results 29619.5 Conclusion and Future Scopes 305References 30620 AUTOMATED DIAGNOSIS OF COVID-19 USING REINFORCED LUNG SEGMENTATION AND CLASSIFICATION MODEL 309J. Shiny Duela and T. Illakiya20.1 Introduction 30920.2 Diagnosis of COVID-19 31020.2.1 Pre-Processing of Lung CT Image 31020.2.2 Lung CT Image Segmentation 31120.2.3 ROI Extraction 31120.2.4 Feature Extraction 31120.2.5 Classification 31120.3 Genetic Algorithm (GA) 31120.3.1 Operators of GA 31220.3.2 Applications of GA 31220.4 Related Works 31320.5 Challenges in GA 31420.6 Challenges in Lung CT Segmentation 31420.7 Proposed Diagnosis Framework 31420.7.1 Image Pre-Processing 31520.7.2 Proposed Image Segmentation Technique 31520.7.3 ROI Segmentation 31820.7.4 Feature Extraction 31820.7.5 Modified GA Classifier 31820.7.5.1 Gaussian Type—II Fuzzy in Classification 31820.7.5.2 Classifier Algorithm 31920.8 Result Discussion 31920.9 Conclusion 321References 321PART 5: CASE STUDIES OF APPLICATION AREAS OF MACHINE LEARNING IN HEALTHCARE SYSTEM 32321 FUTURE OF TELEMEDICINE WITH ML: BUILDING A TELEMEDICINE FRAMEWORK FOR LUNG SOUND DETECTION 325Sudhansu Shekhar Patra, Nitin S. Goje, Kamakhya Narain Singh, Kaish Q. Khan, Deepak Kumar, Madhavi and Kumar Ashutosh Sharma21.1 Introduction 32521.1.1 Monitoring the Remote Patient 32621.1.2 Intelligent Assistance for Patient Diagnosis 32621.1.3 Fasten Electronic Health Record Retrieval Process 32621.1.4 Collaboration Increases Among Healthcare Practitioners 32621.2 Related Work 32721.3 Strategic Model for Telemedicine 32821.4 Framework for Lung Sound Detection in Telemedicine 33021.4.1 Data Collection 33021.4.2 Pre-Processing of Data 33121.4.3 Feature Extraction 33121.4.3.1 MFCC 33121.4.3.2 Lung Sounds Using Multi Resolution DWT 33221.4.4 Classification 33421.4.4.1 Correlation Coefficient for Feature Selection (CFS) 33421.4.4.2 Symmetrical Uncertainty 33421.4.4.3 Gain Ratio 33521.4.4.4 Modified RF Classification Architecture 33521.5 Experimental Analysis 33521.6 Conclusion 340References 34022 A LIGHTWEIGHT CONVOLUTIONAL NEURAL NETWORK MODEL FOR TUBERCULOSIS BACILLI DETECTION FROM MICROSCOPIC SPUTUM SMEAR IMAGES 343Rani Oomman Panicker, S.J. Pawan, Jeny Rajan and M.K. Sabu22.1 Introduction 34322.2 Literature Review 34522.3 Proposed Work 34622.4 Experimental Results and Discussion 34922.5 Conclusion 350References 35023 ROLE OF MACHINE LEARNING AND TEXTURE FEATURES FOR THE DIAGNOSIS OF LARYNGEAL CANCER 353Vibhav Prakash Singh and Ashish Kumar Maurya23.1 Introduction 35323.2 Clinically Correlated Texture Features 35823.2.1 Texture-Based LBP Descriptors 35823.2.2 GLCM Features 35823.2.3 Statistical Features 35923.3 Machine Learning Techniques 35923.3.1 Support Vector Machine (SVM) 35923.3.2 k-NN (k-Nearest Neighbors) 36023.3.3 Random Forest (RF) 36123.3.4 Naïve Bayes 36123.4 Result Analysis and Discussions 36123.5 Conclusions 366References 36624 ANALYSIS OF MACHINE LEARNING TECHNOLOGIES FOR THE DETECTION OF DIABETIC RETINOPATHY 369Biswabijayee Chandra Sekhar Mohanty, Sonali Mishra and Sambit Kumar Mishra24.1 Introduction 36924.2 Related Work 37024.2.1 Pre-Processing of Image 37124.2.2 Diabetic Retinopathy Detection 37224.2.3 Grading of DR 37424.3 Dataset Used 37424.3.1 DIARETDB1 37424.3.2 Diabetic-Retinopathy-Detection Dataset 37624.4 Methodology Used 37724.4.1 Pre-Processing 37724.4.2 Segmentation 37724.4.3 Feature Extraction 37824.4.4 Classification 37824.5 Analysis of Results and Discussion 37924.6 Conclusion 380References 381Index 383
Digital Cities Roadmap
DIGITAL CITIES ROADMAPTHIS BOOK DETAILS APPLICATIONS OF TECHNOLOGY TO EFFICIENT DIGITAL CITY INFRASTRUCTURE AND ITS PLANNING, INCLUDING SMART BUILDINGS.Rapid urbanization, demographic changes, environmental changes, and new technologies are changing the views of urban leaders on sustainability, as well as creating and providing public services to tackle these new dynamics. Sustainable development is an objective by which the processes of planning, implementing projects, and development is aimed at meeting the needs of modern communities without compromising the potential of future generations. The advent of Smart Cities is the answer to these problems.Digital Cities Roadmap provides an in-depth analysis of design technologies that lay a solid foundation for sustainable buildings. The book also highlights smart automation technologies that help save energy, as well as various performance indicators needed to make construction easier. The book aims to create a strong research community, to have a deep understanding and the latest knowledge in the field of energy and comfort, to offer solid ideas in the nearby future for sustainable and resilient buildings. These buildings will help the city grow as a smart city. The smart city has also a focus on low energy consumption, renewable energy, and a small carbon footprint.AUDIENCEThe information provided in this book will be of value to researchers, academicians and industry professionals interested in IoT-based architecture and sustainable buildings, energy efficiency and various tools and methods used to develop green technologies for construction in smart cities. ARUN SOLANKI PhD is an assistant professor in the Department of Computer Science and Engineering, Gautam Buddha University, Greater Noida, India, where he has been working since 2009. His research interests span expert systems, machine learning, and search engines. He has published many research articles in international journals/conferences.ADARSH KUMAR PhD is an associate professor at the University of Petroleum & Energy Studies, Dehradun, India. His main research interests are cybersecurity, cryptography, network security, and ad-hoc networks. He has published 60+ research papers in reputed journals, conferences and workshops.ANAND NAYYAR PhD is currently working in the Graduate School, Duy Tan University, Da Nang, Vietnam. He is a certified professional with more than 75 Professional Certificates from CISCO, Microsoft, Oracle, Google, Beingcert, EXIN, GAQM, Cyberoam, and many more. He published more than 300 research articles in various national and international journals and conferences. He has authored, coauthored or edited about 30 books and has been granted two patents in the areas of Internet of Things and speech processing.Preface xix1 THE USE OF MACHINE LEARNING FOR SUSTAINABLE AND RESILIENT BUILDINGS 1Kuldeep Singh Kaswan and Jagjit Singh Dhatterwal1.1 Introduction of ML Sustainable Resilient Building 21.2 Related Works 21.3 Machine Learning 51.4 What is Resilience? 61.4.1 Sustainability and Resiliency Conditions 71.4.2 Paradigm and Challenges of Sustainability and Resilience 71.4.3 Perspectives of Local Community 91.5 Sustainability and Resilience of Engineered System 121.5.1 Resilience and Sustainable Development Framework for Decision-Making 131.5.2 Exposures and Disturbance Events 151.5.3 Quantification of Resilience 151.5.4 Quantification of Sustainability 161.6 Community and Quantification Metrics, Resilience and Sustainability Objectives 171.6.1 Definition of Quantification Metric 181.6.2 Considering and Community 191.7 Structure Engineering Dilemmas and Resilient Epcot 211.7.1 Dilation of Resilience Essence 211.7.2 Quality of Life 221.8 Development of Risk Informed Criteria for Building Design Hurricane Resilient on Building 271.9 Resilient Infrastructures Against Earthquake and Tsunami Multi-Hazard 281.10 Machine Learning With Smart Building 291.10.1 Smart Building Appliances 291.10.2 Intelligent Tools, Cameras and Electronic Controls in a Connected House (SRB) 291.10.3 Level if Clouds are the IoT Institute Level With SBs 311.10.4 Component of Smart Buildings (SB) 331.10.5 Machine Learning Tasks in Smart Building Environment 461.10.6 ML Tools and Services for Smart Building 471.10.7 Big Data Research Applications for SBs in Real-Time 511.10.8 Implementation of the ML Concept in the SB Context 511.11 Conclusion and Future Research 53References 582 FIRE HAZARD DETECTION AND PREDICTION BY MACHINE LEARNING TECHNIQUES IN SMART BUILDINGS (SBS) USING SENSORS AND UNMANNED AERIAL VEHICLES (UAVS) 63Sandhya Tarar and Namisha Bhasin2.1 Introduction 642.1.1 Bluetooth 652.1.2 Unmanned Aerial Vehicle 652.1.3 Sensors 652.1.4 Problem Description 672.2 Literature Review 682.3 Experimental Methods 712.3.1 Univariate Time-Series 732.3.1.1 Naïve Bayes 742.3.1.2 Simple Average 742.3.1.3 Moving Average 752.3.1.4 Simple Exponential Smoothing (SES) 762.3.1.5 Holt’s Linear Trend 762.3.1.6 Holt–Winters Method 762.3.1.7 Autoregressive Integrated Moving Average Model (ARIMA) 772.3.2 Multivariate Time-Series Prediction 802.3.2.1 Vector Autoregressive (VAR) 802.3.3 Hidden Markov Model (HMM) 812.3.4 Fuzzy Logic 852.4 Results 892.5 Conclusion and Future Work 89References 903 SUSTAINABLE INFRASTRUCTURE THEORIES AND MODELS 97Saurabh Jain, Keshav Kaushik, Deepak Kumar Sharma, Rajalakshmi Krishnamurthi and Adarsh Kumar3.1 Introduction to Data Fusion Approaches in Sustainable Infrastructure 983.1.1 The Need for Sustainable Infrastructure 983.1.2 Data Fusion 993.1.3 Different Types of Data Fusion Architecture 1003.1.3.1 Centralized Architecture 1003.1.3.2 Decentralized Architecture 1013.1.3.3 Distributed Architecture 1013.1.3.4 Hierarchical Architecture 1023.1.4 Smart Cities Application With Sustainable Infrastructures Based on Different Data Fusion Techniques 1023.2 Smart City Infrastructure Approaches 1043.2.1 Smart City Infrastructure 1043.2.2 Smart City IoT Deployments 1053.2.3 Smart City Control and Monitoring Centers 1063.2.4 Theory of Unified City Modeling for Smart Infrastructure 1083.2.5 Smart City Operational Modeling 1093.3 Theories and Models 1103.3.1 Sustainable Infrastructure Theories 1103.3.2 Sustainable Infrastructure Models 1123.4 Case Studies 1133.4.1 Case Studies-1: Web Browsing History Analysis 1133.4.1.1 Objective 1153.4.2 Case Study-2: Data Model for Group Construction in Student’s Industrial Placement 1173.5 Conclusion and Future Scope 121References 1224 BLOCKCHAIN FOR SUSTAINABLE SMART CITIES 127Iftikhar Ahmad, Syeda Warda Ashar, Umamma Khalid, Anmol Irfan and Wajeeha Khalil4.1 Introduction 1284.2 Smart City 1304.2.1 Overview of Smart City 1304.2.2 Evolution 1304.2.3 Smart City’s Sub Systems 1304.2.4 Domains of Smart City 1324.2.5 Challenges 1344.3 Blockchain 1364.3.1 Motivation 1374.3.2 The Birth of Blockchain 1374.3.3 System of Blockchain 1374.4 Use Cases of Smart City Implementing Blockchain 1384.4.1 Blockchain-Based Smart Economy 1384.4.1.1 Facilitating Faster and Cheaper International Payment 1394.4.1.2 Distributed Innovations in Financial Transactions 1394.4.1.3 Enhancing the Transparency of Supply/Global Commodity Chains 1404.4.1.4 Equity Crowd Funding 1414.4.2 Blockchain for Smart People 1414.4.2.1 Elections through Blockchain Technology 1414.4.2.2 Smart Contract 1434.4.2.3 Protecting Personal Data 1444.4.2.4 E-Health: Storing Health Records on Blockchain 1454.4.2.5 Intellectual Property Rights 1454.4.2.6 Digital Payments 1464.4.2.7 Other Use Cases 1464.4.3 Blockchain-Based Smart Governance 1474.4.3.1 Transparent Record Keeping and Tracking of Records 1474.4.3.2 Fraud Free Voting 1484.4.3.3 Decision Making 1504.4.4 Blockchain-Based Smart Transport 1504.4.4.1 Digitizing Driving License 1504.4.4.2 Smart Ride Sharing 1504.4.5 Blockchain-Based Smart Environment 1514.4.5.1 Social Plastic 1514.4.5.2 Energy 1524.4.5.3 Environmental Treaties 1524.4.5.4 Carbon Tax 1534.4.6 Blockchain-Based Smart Living 1534.4.6.1 Fighting Against Frauds and Discriminatory Policies and Practices 1544.4.6.2 Managing Change in Ownership 1544.4.6.3 Sustainable Buildings 1544.4.6.4 Other Use Cases 1554.5 Conclusion 156References 1565 CONTEXTUALIZING ELECTRONIC GOVERNANCE, SMART CITY GOVERNANCE AND SUSTAINABLE INFRASTRUCTURE IN INDIA: A STUDY AND FRAMEWORK 163Nitin K. Tyagi and Mukta Goyal5.1 Introduction 1645.2 Related Works 1665.2.1 Research Questions 1665.3 Related E-Governance Frameworks 1785.3.1 Smart City Features in India 1815.4 Proposed Smart Governance Framework 1815.5 Results Discussion 1855.5.1 Initial Stage 1855.5.2 Design, Development and Delivery Stage 1865.6 Conclusion 186References 1886 REVOLUTIONIZING GERIATRIC DESIGN IN DEVELOPING COUNTRIES: IOT-ENABLED SMART HOME DESIGN FOR THE ELDERLY 193Shubhi Sonal and Anupadma R.6.1 Introduction to Geriatric Design 1946.1.1 Aim, Objectives, and Methodology 1966.1.2 Organization of Chapter 1976.2 Background 1976.2.1 Development of Smart Homes 1976.2.2 Development of Smart Homes for Elderly 1986.2.3 Indian Scenario 2006.3 Need for Smart Homes: An Assessment of Requirements for the Elderly-Activity Mapping 2016.3.1 Geriatric Smart Home Design: The Indian Context 2026.3.2 Elderly Activity Mapping 2026.3.3 Framework for Smart Homes for Elderly People 2066.3.4 Architectural Interventions: Spatial Requirements for Daily Activities 2076.3.5 Architectural Interventions to Address Issues Faced by Elderly People 2086.4 Schematic Design for a Nesting Home: IoT-Enabled Smart Home for Elderly People 2086.4.1 IoT-Based Real Time Automation for Nesting Homes 2086.4.2 Technological Components of Elderly Smart Homes 2126.4.2.1 Sensors for Smart Home 2126.4.2.2 Health Monitoring System 2136.4.2.3 Network Devices 2136.4.2.4 Alerts 2146.5 Worldwide Elderly Smart Homes 2146.5.1 Challenges in Smart Elderly Homes 2156.6 Conclusion and Future Scope 216References 2167 SUSTAINABLE E-INFRASTRUCTURE FOR BLOCKCHAIN-BASED VOTING SYSTEM 221Mukta Goyal and Adarsh Kumar7.1 Introduction 2227.1.1 E-Voting Challenge 2247.2 Related Works 2247.3 System Design 2277.4 Experimentation 2307.4.1 Software Requirements 2307.4.2 Function Requirements 2307.4.2.1 Election Organizer 2317.4.2.2 Candidate Registration 2317.4.2.3 Voter Registration Process 2327.4.3 Common Functional Requirement for All Users 2337.4.3.1 Result Display 2337.4.4 Non-Function Requirements 2337.4.4.1 Performance Requirement 2337.4.4.2 Security Requirement 2337.4.4.3 Usability Requirement 2337.4.4.4 Availability Requirement 2347.4.5 Implementation Details 2347.5 Findings & Results 2377.5.1 Smart Contract Deployment 2417.6 Conclusion and Future Scope 242Acknowledgement 246References 2468 IMPACT OF IOT-ENABLED SMART CITIES: A SYSTEMATIC REVIEW AND CHALLENGES 253K. Rajkumar and U. Hariharan8.1 Introduction 2548.2 Recent Development in IoT Application for Modern City 2568.2.1 IoT Potential Smart City Approach 2578.2.2 Problems and Related Solutions in Modern Smart Cities Application 2598.3 Classification of IoT-Based Smart Cities 2628.3.1 Program Developers 2638.3.2 Network Type 2638.3.3 Activities of Standardization Bodies of Smart City 2638.3.4 Available Services 2698.3.5 Specification 2698.4 Impact of 5G Technology in IT, Big Data Analytics, and Cloud Computing 2708.4.1 IoT Five-Layer Architecture for Smart City Applications 2708.4.1.1 Sensing Layer (Get Information from Sensor) 2728.4.1.2 Network Layer (Access and Also Transmit Information) 2728.4.1.3 Data Storage and Analyzing 2738.4.1.4 Smart Cities Model (Smart Industry Model, Smart Healthcare Model, Smart Cities, Smart Agriculture Model) 2738.4.1.5 Application Layer (Dedicated Apps and Services) 2738.4.2 IoT Computing Paradigm for Smart City Application 2748.5 Research Advancement and Drawback on Smart Cities 2808.5.1 Integration of Cloud Computing in Smart Cities 2808.5.2 Integration of Applications 2818.5.3 System Security 2818.6 Summary of Smart Cities and Future Research Challenges and Their Guidelines 2828.7 Conclusion and Future Direction 287References 2889 INDOOR AIR QUALITY (IAQ) IN GREEN BUILDINGS, A PRE-REQUISITE TO HUMAN HEALTH AND WELL-BEING 293Ankita Banerjee, N.P. Melkania and Ayushi Nain9.1 Introduction 2949.2 Pollutants Responsible for Poor IAQ 2969.2.1 Volatile Organic Compounds (VOCs) 2969.2.2 Particulate Matter (PM) 2989.2.3 Asbestos 2999.2.4 Carbon Monoxide (CO) 2999.2.5 Environmental Tobacco Smoke (ETS) 3009.2.6 Biological Pollutants 3019.2.7 Lead (Pb) 3039.2.8 Nitrogen Dioxide (NO2) 3049.2.9 Ozone (O3) 3059.3 Health Impacts of Poor IAQ 3069.3.1 Sick Building Syndrome (SBS) 3069.3.2 Acute Impacts 3079.3.3 Chronic Impacts 3089.4 Strategies to Maintain a Healthy Indoor Environment in Green Buildings 3089.5 Conclusion and Future Scope 313References 31410 AN ERA OF INTERNET OF THINGS LEADS TO SMART CITIES INITIATIVES TOWARDS URBANIZATION 319Pooja Choudhary, Lava Bhargava, Ashok Kumar Suhag, Manju Choudhary and Satendra Singh10.1 Introduction: Emergence of a Smart City Concept 32010.2 Components of Smart City 32110.2.1 Smart Infrastructure 32310.2.2 Smart Building 32310.2.3 Smart Transportation 32510.2.4 Smart Energy 32610.2.5 Smart Health Care 32710.2.6 Smart Technology 32810.2.7 Smart Citizen 32910.2.8 Smart Governance 33010.2.9 Smart Education 33010.3 Role of IoT in Smart Cities 33110.3.1 Intent of IoT Adoption in Smart Cities 33310.3.2 IoT-Supported Communication Technologies 33310.4 Sectors, Services Related and Principal Issues for IoT Technologies 33610.5 Impact of Smart Cities 33610.5.1 Smart City Impact on Science and Technology 33610.5.2 Smart City Impact on Competitiveness 33910.5.3 Smart City Impact on Society 33910.5.4 Smart City Impact on Optimization and Management 33910.5.5 Smart City for Sustainable Development 34010.6 Key Applications of IoT in Smart Cities 34010.7 Challenges 34310.7.1 Smart City Design Challenges 34310.7.2 Challenges Raised by Smart Cities 34410.7.3 Challenges of IoT Technologies in Smart Cities 34410.8 Conclusion 346Acknowledgements 346References 34611 TRIP-I-PLAN: A MOBILE APPLICATION FOR TASK SCHEDULING IN SMART CITY’S SUSTAINABLE INFRASTRUCTURE 351Rajalakshmi Krishnamurthi, Dhanalekshmi Gopinathan and Adarsh Kumar11.1 Introduction 35211.2 Smart City and IoT 35411.3 Mobile Computing for Smart City 35711.4 Smart City and its Applications 36011.4.1 Traffic Monitoring 36011.4.2 Smart Lighting 36111.4.3 Air Quality Monitoring 36211.5 Smart Tourism in Smart City 36311.6 Mobile Computing-Based Smart Tourism 36611.7 Case Study: A Mobile Application for Trip Planner Task Scheduling in Smart City’s Sustainable Infrastructure 36811.7.1 System Interfaces and User Interfaces 37111.8 Experimentation and Results Discussion 37111.9 Conclusion and Future Scope 373References 37412 SMART HEALTH MONITORING FOR ELDERLY CARE IN INDOOR ENVIRONMENTS 379Sonia and Tushar Semwal12.1 Introduction 38012.2 Sensors 38212.2.1 Human Traits 38312.2.2 Sensors Description 38412.2.2.1 Passive Sensors 38512.2.2.2 Active Sensors 38612.2.3 Sensing Challenges 38712.3 Internet of Things and Connected Systems 38712.4 Applications 38912.5 Case Study 39212.5.1 Case 1 39212.5.2 Case 2 39312.5.3 Challenges Involved 39312.5.4 Possible Solution 39312.6 Conclusion 39512.7 Discussion 395References 39513 A COMPREHENSIVE STUDY OF IOT SECURITY RISKS IN BUILDING A SECURE SMART CITY 401Akansha Bhargava, Gauri Salunkhe, Sushant Bhargava and Prerna Goswami13.1 Introduction 40213.1.1 Organization of the Chapter 40413.2 Related Works 40513.3 Overview of IoT System in Smart Cities 40713.3.1 Physical Devices 40913.3.2 Connectivity 40913.3.3 Middleware 41013.3.4 Human Interaction 41013.4 IoT Security Prerequisite 41113.5 IoT Security Areas 41313.5.1 Anomaly Detection 41313.5.2 Host-Based IDS (HIDS) 41413.5.3 Network-Based IDS (NIDS) 41413.5.4 Malware Detection 41413.5.5 Ransomware Detection 41513.5.6 Intruder Detection 41513.5.7 Botnet Detection 41513.6 IoT Security Threats 41613.6.1 Passive Threats 41613.6.2 Active Threats 41713.7 Review of ML/DL Application in IoT Security 41813.7.1 Machine Learning Methods 42113.7.1.1 Decision Trees (DTs) 42113.7.1.2 K-Nearest Neighbor (KNN) 42313.7.1.3 Random Forest 42413.7.1.4 Principal Component Analysis (PCA) 42513.7.1.5 Naïve Bayes 42513.7.1.6 Support Vector Machines (SVM) 42513.7.2 Deep Learning Methods 42613.7.2.1 Convolutional Neural Networks (CNNs) 42713.7.2.2 Auto Encoder (AE) 42913.7.2.3 Recurrent Neural Networks (RNNs) 42913.7.2.4 Restricted Boltzmann Machines (RBMs) 43213.7.2.5 Deep Belief Networks (DBNs) 43313.7.2.6 Generative Adversarial Networks (GANs) 43313.8 Challenges 43413.8.1 IoT Dataset Unavailability 43413.8.2 Computational Complications 43413.8.3 Forensics Challenges 43513.9 Future Prospects 43613.9.1 Implementation of ML/DL With Edge Computing 43713.9.2 Integration of ML/DL With Blockchain 43813.9.3 Integration of ML/DL With Fog Computing 43913.10 Conclusion 439References 44014 ROLE OF SMART BUILDINGS IN SMART CITY—COMPONENTS, TECHNOLOGY, INDICATORS, CHALLENGES, FUTURE RESEARCH OPPORTUNITIES 449Tarana Singh, Arun Solanki and Sanjay Kumar Sharma14.1 Introduction 44914.1.1 Chapter Organization 45314.2 Literature Review 45314.3 Components of Smart Cities 45514.3.1 Smart Infrastructure 45514.3.2 Smart Parking Management 45614.3.3 Connected Charging Stations 45714.3.4 Smart Buildings and Properties 45714.3.5 Smart Garden and Sprinkler Systems 45714.3.6 Smart Heating and Ventilation 45714.3.7 Smart Industrial Environment 45814.3.8 Smart City Services 45814.3.9 Smart Energy Management 45814.3.10 Smart Water Management 45914.3.11 Smart Waste Management 45914.4 Characteristics of Smart Buildings 45914.4.1 Minimal Human Control 45914.4.2 Optimization 46014.4.3 Qualities 46014.4.4 Connected Systems 46014.4.5 Use of Sensors 46014.4.6 Automation 46114.4.7 Data 46114.5 Supporting Technology 46114.5.1 Big Data and IoT in Smart Cities 46114.5.2 Sensors 46214.5.3 5G Connectivity 46214.5.4 Geospatial Technology 46214.5.5 Robotics 46314.6 Key Performance Indicators of Smart City 46314.6.1 Smart Economy 46314.6.2 Smart Governance 46414.6.3 Smart Mobility 46414.6.4 Smart Environment 46414.6.5 Smart People 46414.6.6 Smart Living 46514.7 Challenges While Working for Smart City 46514.7.1 Retrofitting Existing Legacy City Infrastructure to Make it Smart 46514.7.2 Financing Smart Cities 46614.7.3 Availability of Master Plan or City Development Plan 46614.7.4 Financial Sustainability of ULBs 46614.7.5 Technical Constraints ULBs 46614.7.6 Three-Tier Governance 46714.7.7 Providing Clearances in a Timely Manner 46714.7.8 Dealing With a Multivendor Environment 46714.7.9 Capacity Building Program 46714.7.10 Reliability of Utility Services 46814.8 Future Research Opportunities in Smart City 46814.8.1 IoT Management 46814.8.2 Data Management 46914.8.3 Smart City Assessment Framework 46914.8.4 VANET Security 46914.8.5 Improving Photovoltaic Cells 46914.8.6 Smart City Enablers 47014.8.7 Information System Risks 47014.9 Conclusion 470References 47115 EFFECTS OF GREEN BUILDINGS ON THE ENVIRONMENT 477Ayushi Nain, Ankita Banerjee and N.P. Melkania15.1 Introduction 47815.2 Sustainability and the Building Industry 48015.2.1 Environmental Benefits 48115.2.2 Social Benefits 48315.2.3 Economic Benefits 48315.3 Goals of Green Buildings 48415.3.1 Green Design 48515.3.2 Energy Efficiency 48515.3.3 Water Efficiency 48715.3.4 Material Efficiency 48915.3.5 Improved Internal Environment and Air Quality 49015.3.6 Minimization of Wastes 49215.3.7 Operations and Maintenance Optimization 49215.4 Impacts of Classical Buildings that Green Buildings Seek to Rectify 49315.4.1 Energy Use in Buildings 49415.4.2 Green House Gas (GHG) Emissions 49415.4.3 Indoor Air Quality 49415.4.4 Building Water Use 49615.4.5 Use of Land and Consumption 49615.4.6 Construction Materials 49715.4.7 Construction and Demolition (C&D) Wastes 49815.5 Green Buildings in India 49815.6 Conclusion 503Acknowledgement 504Acronyms 504References 505Index 509
Datengetriebenes Konzept zur Luftqualitätsprädiktion
Enes Esatbeyoǧlu verfolgt den Ansatz, die NO2-Konzentrationen als Indikator für die Luftqualität mit einem Fahrzeug und zwei verschiedenen Sensoren (Referenz und miniaturisiert) zu messen und darauf basierend ein datengetriebenes Prädiktions- und Adaptionskonzept zu entwickeln. Die Datenerhebung erfolgt dabei auf einer vordefinierten Route zu verschiedenen Zeiten sowie Umwelt- und Verkehrsbedingungen. Für die Prädiktion der Luftqualität und Adaption der Sensordaten wendet er verschiedene Machine Learning Modelle an. Dabei untersucht er die Performantesten auf Robustheit, Generalisierbarkeit und Übertragbarkeit. Der Autor Enes Esatbeyoglu studierte Maschinenbau und Kraftfahrzeugtechnik. Anschließend hat er berufsbegleitend mit der vorliegenden Schrift an der OVGU Magdeburg promoviert und ist jetzt im Bereich der Konzeptvorentwicklung eines Automobilkonzerns tätig.
The Magic of Computer Science
We are living in the era of digital transformation. Computers are rapidly becoming the most important tool for companies, science, society, and indeed our everyday life. We all need a basic understanding of Computer Science to make sense of the world, to make decisions, and to improve our lives.Yet there are many misunderstandings about Computer Science. The reason is that it is a nascent discipline that has evolved rapidly and had to reinvent itself several times over the last 100 years – from the beginnings of scientific computing to the modern era of smartphones and the cloud.This book gives an intuitive introduction to the foundations and main concepts of Computer Science. It describes the basic ideas of solving problems with algorithms, modern data-driven approaches, and artificial intelligence (AI). It also provides many examples that require no background in technology.This book is directed toward teenagers who may wonder whether they should major in Computer Science, though it will also appeal to anyone who wants to immerse themselves in the art of Computer Science and modern information technology. Of course, not everyone must become a computer expert, but everyone should take advantage of and understand the innovations and advances of modern technology.
AI and IoT-Based Intelligent Automation in Robotics
The 24 chapters in this book provides a deep overview of robotics and the application of AI and IoT in robotics. It contains the exploration of AI and IoT based intelligent automation in robotics. The various algorithms and frameworks for robotics based on AI and IoT are presented, analyzed, and discussed. This book also provides insights on application of robotics in education, healthcare, defense and many other fields which utilize IoT and AI. It also introduces the idea of smart cities using robotics.ASHUTOSH KUMAR DUBEY received his PhD degree in Computer Science and Engineering from JK Lakshmipat University, Jaipur, Rajasthan, India. He is currently in the Department of Computer Science and Engineering, Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, India. His research areas are data mining, optimization, machine learning, cloud computing, artificial intelligence, big data, IoT and object-oriented programming.ABHISHEK KUMAR is a Doctorate in computer science from the University of Madras and more than 50 publications in reputed peer reviewed national and international journals, books & conferences. His research interests include artificial intelligence, image processing, computer vision, data mining, machine learning. S. RAKESH KUMAR received his M.E. degree in Computer Science and Engineering from Anna University Chennai in 2016. His main research areas are big data analytics, network security and cloud computing.N. GAYATHRI received her B. Tech as well as M. Tech. degree in Computer Science and Engineering from Thiagarajar College of Engineering, Madurai, India. Her research interests include cloud computing, big data analytics and network security.PASENJIT DAS PHD is an associate professor at Chitkara University, Himachal Pradesh, India. He has 15 + years’ experience in industry and academia and his research areas are data mining, machine learning and image processing.Preface xvii1 INTRODUCTION TO ROBOTICS 1Srinivas Kumar Palvadi, Pooja Dixit and Vishal Dutt1.1 Introduction 11.2 History and Evolution of Robots 31.3 Applications 61.4 Components Needed for a Robot 71.5 Robot Interaction and Navigation 101.5.1 Humanoid Robot 111.5.2 Control 111.5.3 Autonomy Levels 121.6 Conclusion 12References 132 TECHNIQUES IN ROBOTICS FOR AUTOMATION USING AI AND IOT 15Sandeep Kr. Sharma, N. Gayathri, S. Rakesh Kumar and Rajiv Kumar Modanval2.1 Introduction 162.2 Brief History of Robotics 162.3 Some General Terms 172.4 Requirements of AI and IoT for Robotic Automation 202.5 Role of AI and IoT in Robotics 212.6 Diagrammatic Representations of Some Robotic Systems 232.7 Algorithms Used in Robotics 252.8 Application of Robotics 272.9 Case Studies 302.9.1 Sophia 302.9.2 ASIMO 302.9.3 Cheetah Robot 302.9.4 IBM Watson 312.10 Conclusion 31References 313 ROBOTICS, AI AND IOT IN THE DEFENSE SECTOR 35Rajiv Kumar Modanval, S. Rakesh Kumar, N. Gayathri and Sandeep Kr. Sharma3.1 Introduction 363.2 How Robotics Plays an Important Role in the Defense Sector 363.3 Review of the World’s Current Robotics Capabilities in the Defense Sector 383.3.1 China 383.3.2 United State of America 393.3.3 Russia 403.3.4 India 413.4 Application Areas of Robotics in Warfare 433.4.1 Autonomous Drones 433.4.2 Autonomous Tanks and Vehicles 443.4.3 Autonomous Ships and Submarines 453.4.4 Humanoid Robot Soldiers 473.4.5 Armed Soldier Exoskeletons 483.5 Conclusion 503.6 Future Work 50References 504 ROBOTICS, AI AND IOT IN MEDICAL AND HEALTHCARE APPLICATIONS 53Pooja Dixit, Manju Payal, Nidhi Goyal and Vishal Dutt4.1 Introduction 534.1.1 Basics of AI 534.1.1.1 AI in Healthcare 544.1.1.2 Current Trends of AI in Healthcare 554.1.1.3 Limits of AI in Healthcare 564.1.2 Basics of Robotics 574.1.2.1 Robotics for Healthcare 574.1.3 Basics of IoT 594.1.3.1 IoT Scenarios in Healthcare 604.1.3.2 Requirements of Security 614.2 AI, Robotics and IoT: A Logical Combination 624.2.1 Artificial Intelligence and IoT in Healthcare 624.2.2 AI and Robotics 634.2.2.1 Limitation of Robotics in Medical Healthcare 664.2.3 IoT with Robotics 664.2.3.1 Overview of IoMRT 674.2.3.2 Challenges of IoT Deployment 694.3 Essence of AI, IoT, and Robotics in Healthcare 704.4 Future Applications of Robotics, AI, and IoT 714.5 Conclusion 72References 725 TOWARDS ANALYZING SKILL TRANSFER TO ROBOTS BASED ON SEMANTICALLY REPRESENTED ACTIVITIES OF HUMANS 75Devi.T, N. Deepa, S. Rakesh Kumar, R. Ganesan and N. Gayathri5.1 Introduction 765.2 Related Work 775.3 Overview of Proposed System 785.3.1 Visual Data Retrieval 795.3.2 Data Processing to Attain User Objective 805.3.3 Knowledge Base 825.3.4 Robot Attaining User Goal 835.4 Results and Discussion 835.5 Conclusion 85References 856 HEALTHCARE ROBOTS ENABLED WITH IOT AND ARTIFICIAL INTELLIGENCE FOR ELDERLY PATIENTS 87S. Porkodi and D. Kesavaraja6.1 Introduction 886.1.1 Past, Present, and Future 886.1.2 Internet of Things 886.1.3 Artificial Intelligence 896.1.4 Using Robotics to Enhance Healthcare Services 896.2 Existing Robots in Healthcare 906.3 Challenges in Implementation and Providing Potential Solutions 906.4 Robotic Solutions for Problems Facing the Elderly in Society 986.4.1 Solutions for Physical and Functional Challenges 986.4.2 Solutions for Cognitive Challenges 986.5 Healthcare Management 996.5.1 Internet of Things for Data Acquisition 996.5.2 Robotics for Healthcare Assistance and Medication Management 1026.5.3 Robotics for Psychological Issues 1036.6 Conclusion and Future Directions 103References 1047 ROBOTICS, AI, AND THE IOT IN DEFENSE SYSTEMS 109Manju Payal, Pooja Dixit, T.V.M. Sairam and Nidhi Goyal7.1 AI in Defense 1107.1.1 AI Terminology and Background 1107.1.2 Systematic Sensing Applications 1117.1.3 Overview of AI in Defense Systems 1127.2 Overview of IoT in Defense Systems 1147.2.1 Role of IoT in Defense 1167.2.2 Ministry of Defense Initiatives 1177.2.3 IoT Defense Policy Challenges 1177.3 Robotics in Defense 1187.3.1 Technical Challenges of Defense Robots 1207.4 AI, Robotics, and IoT in Defense: A Logical Mix in Context 1237.4.1 Combination of Robotics and IoT in Defense 1237.4.2 Combination of Robotics and AI in Defense 1247.5 Conclusion 126References 1278 TECHNIQUES OF ROBOTICS FOR AUTOMATION USING AI AND THE IOT 129Kapil Chauhan and Vishal Dutt8.1 Introduction 1308.2 Internet of Robotic Things Concept 1318.3 Definitions of Commonly Used Terms 1328.4 Procedures Used in Making a Robot 1338.4.1 Analyzing Tasks 1338.4.2 Designing Robots 1348.4.3 Computerized Reasoning 1348.4.4 Combining Ideas to Make a Robot 1348.4.5 Making a Robot 1348.4.6 Designing Interfaces with Different Frameworks or Robots 1348.5 IoRT Technologies 1358.6 Sensors and Actuators 1378.7 Component Selection and Designing Parts 1388.7.1 Robot and Controller Structure 1408.8 Process Automation 1418.8.1 Benefits of Process Automation 1418.8.2 Incorporating AI in Process Automation 1418.9 Robots and Robotic Automation 1428.10 Architecture of the Internet of Robotic Things 1428.10.1 Concepts of Open Architecture Platforms 1438.11 Basic Abilities 1438.11.1 Discernment Capacity 1438.11.2 Motion Capacity 1448.11.3 Manipulation Capacity 1448.12 More Elevated Level Capacities 1458.12.1 Decisional Self-Sufficiency 1458.12.2 Interaction Capacity 1458.12.3 Cognitive Capacity 1468.13 Conclusion 146References 1469 AN ARTIFICIAL INTELLIGENCE-BASED SMART TASK RESPONDER: ANDROID ROBOT FOR HUMAN INSTRUCTION USING LSTM TECHNIQUE 149T. Devi, N. Deepa, SP. Chokkalingam, N. Gayathri and S. Rakesh Kumar9.1 Introduction 1509.2 Literature Review 1529.3 Proposed System 1529.4 Results and Discussion 1579.5 Conclusion 161References 16210 AI, IOT AND ROBOTICS IN THE MEDICAL AND HEALTHCARE FIELD 165V. Kavidha, N. Gayathri and S. Rakesh Kumar10.1 Introduction 16510.2 A Survey of Robots and AI Used in the Health Sector 16710.2.1 Surgical Robots 16710.2.2 Exoskeletons 16810.2.3 Prosthetics 17010.2.4 Artificial Organs 17110.2.5 Pharmacy and Hospital Automation Robots 17210.2.6 Social Robots 17310.2.7 Big Data Analytics 17510.3 Sociotechnical Considerations 17610.3.1 Sociotechnical Influence 17610.3.2 Social Valence 17710.3.3 The Paradox of Evidence-Based Reasoning 17810.4 Legal Considerations 18010.4.1 Liability for Robotics, AI and IoT 18010.4.2 Liability for Physicians Using Robotics, AI and IoT 18110.4.3 Liability for Institutions Using Robotics, AI and IoT 18210.5 Regulating Robotics, AI and IoT as Medical Devices 18310.6 Conclusion 185References 18511 REAL-TIME MILD AND MODERATE COVID-19 HUMAN BODY TEMPERATURE DETECTION USING ARTIFICIAL INTELLIGENCE 189K. Logu, T. Devi, N. Deepa, S. Rakesh Kumar and N. Gayathri11.1 Introduction 19011.2 Contactless Temperature 19111.2.1 Bolometers (IR-Based) 19211.2.2 Thermopile Radiation Sensors (IR-Based) 19311.2.3 Fiber-Optic Pyrometers 19311.2.4 RGB Photocell 19411.2.5 3D Sensor 19511.3 Fever Detection Camera 19611.3.1 Facial Recognition 19711.3.2 Geometric Approach 19811.3.3 Holistic Approach 19811.3.4 Model-Based 19811.3.5 Vascular Network 19911.4 Simulation and Analysis 20011.5 Conclusion 203References 20312 DRONES IN SMART CITIES 205Manju Payal, Pooja Dixit and Vishal Dutt12.1 Introduction 20612.1.1 Overview of the Literature 20612.2 Utilization of UAVs for Wireless Network 20912.2.1 Use Cases for WN Using UAVs 20912.2.2 Classifications and Types of UAVs 21012.2.3 Deployment of UAVS Using IoT Networks 21312.2.4 IoT and 5G Sensor Technologies for UAVs 21412.3 Introduced Framework 21712.3.1 Architecture of UAV IoT 21712.3.2 Ground Control Station 21812.3.3 Data Links 21812.4 UAV IoT Applications 22312.4.1 UAV Traffic Management 22312.4.2 Situation Awareness 22312.4.3 Public Safety/Saving Lives 22512.5 Conclusion 227References 22713 UAVS IN AGRICULTURE 229DeepanshuSrivastava, S. RakeshKumar and N. Gayathri13.1 Introduction 23013.2 UAVs in Smart Farming and Take-Off Panel 23013.2.1 Overview of Systems 23013.3 Introduction to UGV Systems and Planning 23413.4 UAV-Hyperspectral for Agriculture 23613.5 UAV-Based Multisensors for Precision Agriculture 23913.6 Automation in Agriculture 24213.7 Conclusion 245References 24514 SEMI-AUTOMATED PARKING SYSTEM USING DSDV AND RFID 247Mayank Agrawal, Abhishek Kumar Rawat, Archana, SandhyaKatiyar and Sanjay Kumar14.1 Introduction 24714.2 Ad Hoc Network 24814.2.1 Destination-Sequenced Distance Vector (DSDV) Routing Protocol 24814.3 Radio Frequency Identification (RFID) 24914.4 Problem Identification 25014.5 Survey of the Literature 25014.6 PANet Architecture 25114.6.1 Approach for Semi-Automated System Using DSDV 25214.6.2 Tables for Parking Available/Occupied 25314.6.3 Algorithm for Detecting the Empty Slots 25514.6.4 Pseudo Code 25514.7 Conclusion 256References 25615 SURVEY OF VARIOUS TECHNOLOGIES INVOLVED IN VEHICLE-TO-VEHICLE COMMUNICATION 259Lisha Kamala K., Sini Anna Alex and Anita Kanavalli15.1 Introduction 25915.2 Survey of the Literature 26015.3 Brief Description of the Techniques 26215.3.1 ARM and Zigbee Technology 26215.3.2 VANET-Based Prototype 26215.3.2.1 Calculating Distance by Considering Parameters 26315.3.2.2 Calculating Speed by Considering Parameters 26315.3.3 Wi-Fi–Based Technology 26315.3.4 Li-Fi–Based Technique 26415.3.5 Real-Time Wireless System 26615.4 Various Technologies Involved in V2V Communication 26715.5 Results and Analysis 26715.6 Conclusion 268References 26816 SMART WHEELCHAIR 271Mekala Ajay, Pusapally Srinivas and Lupthavisha Netam16.1 Background 27116.2 System Overview 27516.3 Health-Monitoring System Using IoT 27516.4 Driver Circuit of Wheelchair Interfaced with Amazon Alexa 27616.5 MATLAB Simulations 27716.5.1 Obstacle Detection 27716.5.2 Implementing Path Planning Algorithms 27816.5.3 Differential Drive Robot for Path Following 28016.6 Conclusion 28216.7 Future Work 282Acknowledgment 283References 28317 DEFAULTER LIST USING FACIAL RECOGNITION 285Kavitha Esther, Akilindin S.H., Aswin S. and Anand P.17.1 Introduction 28617.2 System Analysis 28717.2.1 Problem Description 28717.2.2 Existing System 28717.2.3 Proposed System 28717.3 Implementation 28917.3.1 Image Pre-Processing 28917.3.2 Polygon Shape Family Pre-Processing 28917.3.3 Image Segmentation 28917.3.4 Threshold 28917.3.5 Edge Detection 29117.3.6 Region Growing Technique 29117.3.7 Background Subtraction 29117.3.8 Morphological Operations 29117.3.9 Object Detection 29217.4 Inputs and Outputs 29217.5 Conclusion 292References 29318 VISITOR/INTRUDER MONITORING SYSTEM USING MACHINE LEARNING 295G. Jenifa, S. Indu, C. Jeevitha and V. Kiruthika18.1 Introduction 29618.2 Machine Learning 29618.2.1 Machine Learning in Home Security 29718.3 System Design 29718.4 Haar-Cascade Classifier Algorithm 29818.4.1 Creating the Dataset 29818.4.2 Training the Model 29918.4.3 Recognizing the Face 29918.5 Components 29918.5.1 Raspberry Pi 29918.5.2 Web Camera 30018.6 Experimental Results 30018.7 Conclusion 302Acknowledgment 302References 30319 COMPARISON OF MACHINE LEARNING ALGORITHMS FOR AIR POLLUTION MONITORING SYSTEM 305Tushr Sethi and R. C. Thakur19.1 Introduction 30519.2 System Design 30619.3 Model Description and Architecture 30719.4 Dataset 30819.5 Models 31019.6 Line of Best Fit for the Dataset 31219.7 Feature Importance 31319.8 Comparisons 31519.9 Results 31819.10 Conclusion 318References 32120 A NOVEL APPROACH TOWARDS AUDIO WATERMARKING USING FFT AND CORDIC-BASED QR DECOMPOSITION 323Ankit Kumar, Astha Singh, Shiv Prakash and Vrijendra Singh20.1 Introduction and Related Work 32420.2 Proposed Methodology 32620.2.1 Fast Fourier Transform 32820.2.2 CORDIC-Based QR Decomposition 32920.2.3 Concept of Cyclic Codes 33120.2.4 Concept of Arnold’s Cat Map 33120.3 Algorithm Design 33120.4 Experiment Results 33420.5 Conclusion 337References 33821 PERFORMANCE OF DC-BIASED OPTICAL ORTHOGONAL FREQUENCY DIVISION MULTIPLEXING IN VISIBLE LIGHT COMMUNICATION 339S. Ponmalar and Shiny J.J.21.1 Introduction 34021.2 System Model 34121.2.1 Transmitter Block 34121.2.2 Receiver Block 34221.3 Proposed Method 34221.3.1 Simulation Parameters for OptSim 34321.3.2 Block Diagram of DCO-OFDM in OptSim 34321.4 Results and Discussion 34421.5 Conclusion 352References 35322 MICROCONTROLLER-BASED VARIABLE RATE SYRINGE PUMP FOR MICROFLUIDIC APPLICATION 355G. B. Tejashree, S. Swarnalatha, S. Pavithra, M. C. Jobin Christ and N. Ashwin Kumar22.1 Introduction 35622.2 Related Work 35722.3 Methodology 35822.3.1 Hardware Design 35922.3.2 Hardware Interface with Software 36022.3.3 Programming and Debugging 36122.4 Result 36222.5 Inference 36322.5.1 Viscosity (η) 36522.5.2 Time Taken 36522.5.3 Syringe Diameter 36622.5.4 Deviation 36622.6 Conclusion and Future Works 366References 36823 ANALYSIS OF EMOTION IN SPEECH SIGNAL PROCESSING AND REJECTION OF NOISE USING HMM 371S. Balasubramanian23.1 Introduction 37223.2 Existing Method 37323.3 Proposed Method 37423.3.1 Proposed Module Description 37523.3.2 MFCC 37623.3.3 Hidden Markov Models 37923.4 Conclusion 382References 38324 SECURING CLOUD DATA BY USING BLEND CRYPTOGRAPHY WITH AWS SERVICES 385Vanchhana Srivastava, Rohit Kumar Pathak and Arun Kumar24.1 Introduction 38524.1.1 AWS 38724.1.2 Quantum Cryptography 38824.1.3 ECDSA 38924.2 Background 38924.3 Proposed Technique 39224.3.1 How the System Works 39324.4 Results 39424.5 Conclusion 396References 396Index 399
Archives in the Digital Age
Archiving has become an increasingly complex process. The challenge is no longer how to store the data but how to store it intelligently, in order to exploit it over time, while maintaining its integrity and authenticity.Digital technologies bring about major transformations, not only in terms of the types of documents that are transferred to and stored in archives, in the behaviors and practices of the humanities and social sciences (digital humanities), but also in terms of the volume of data and the technological capacity for managing and preserving archives (Big Data). Archives in The Digital Age focuses on the impact of these various digital transformations on archives, and examines how the right to memory and the information of future generations is confronted with the right to be forgotten; a digital prerogative that guarantees individuals their private lives and freedoms. ABDERRAZAK MKADMI holds a PhD in Information and Communication Sciences from the University of Paris 8, France, and is a Research Professor at the Higher Institute of Documentation (Manouba University, Tunisia). Preface ixIntroduction xiCHAPTER 1. DIGITAL ARCHIVES: ELEMENTS OF DEFINITION 11.1. Key concepts of digital archives 11.1.1. Archives 11.1.2. Archive management 21.1.3. Archival management tools 41.1.4. Digital archives 71.2. Electronic Records Management 71.2.1. ERM: elements of definition 71.2.2. ERM: implementation steps 101.3. Records management 181.3.1. Structure of standard 15489 191.3.2. Content of the standard 201.3.3. Design and implementation of an RM project according to the standard 221.3.4. MoReq: the added value of RM 251.4. EDRMS: merging ERM and RM 261.5. ECM: the overall data management strategy 271.6. Conclusion 30CHAPTER 2. DIGITAL ARCHIVING: METHODS AND STRATEGIES 312.1. Introduction 312.2. Digital archiving: elements of definition 312.3. Digital archiving: the essential standards 342.3.1. NF Z 42-013/ISO 14641 standard 362.3.2. NF 461: electronic archiving system 382.3.3. OAIS (ISO 14721): Open Archival Information System 392.3.4. ISO 19905 (PDF/A) 422.3.5. ISO 30300, ISO 30301 and ISO 30302 series of standards 442.3.6. ISO 23081 442.4. Methodology for setting up a digital archiving process 462.4.1. Qualifying and classifying information 462.4.2. Classification scheme 472.4.3. Retention schedule or retention standard 512.4.4. Metadata 522.4.5. Archiving processes and procedures 552.5. Archiving of audiovisual documents 582.5.1. Definition of audiovisual archives 582.5.2. Treatment of audiovisual archives 602.5.3. Migration of audiovisual documents 622.5.4. Digital archiving of audiovisual documents 632.6. Email archiving 652.6.1. Email archiving and legislation 662.6.2. Why archive emails? 672.7. Conclusion 69CHAPTER 3. ARCHIVES IN THE AGE OF DIGITAL HUMANITIES 713.1. Introduction 713.2. History of the digital humanities 723.2.1. “Literary and Linguistic Computing”: 1940–1980 723.2.2. “Humanities computing”: 1980–1994 743.2.3. “Digital humanities”: since 1994 773.3. Definitions of the digital humanities 783.4. Archives in the age of the digital humanities 803.4.1. Digital archive platforms 813.4.2. Software managing digital archives 843.4.3. Digital humanities at the heart of long-term preservation 893.4.4. Digital humanities and the liberation of the humanities: access and accessibility 1073.5. Conclusion 112CHAPTER 4. DIGITAL ARCHIVING AND BIG DATA 1134.1. Introduction 1134.2. Definition of Big Data 1154.3. Big Data issues 1194.4. Big Data: challenges and areas of application 1204.5. Data archiving in the age of Big Data 1224.5.1. Management and archiving of Big Data 1224.5.2. Big Data technologies and tools 1254.5.3. Blockchain, the future of digital archiving of Big Data 1374.6. Conclusion 147CHAPTER 5. PRESERVATION OF ARCHIVES VERSUS THE RIGHT TO BE FORGOTTEN 1495.1. Introduction 1495.2. Forgetting 1505.3. The right to be forgotten 1505.3.1. Limits to the right to be forgotten 1505.3.2. European Directive on the protection of personal data 1515.3.3. General Data Protection Regulation 1535.3.4. The right to dereferencing: common criteria 1565.4. Effectiveness of the right to be forgotten 1565.4.1. Technical challenge of the effectiveness of the right to be forgotten 1575.4.2. Legal challenge of the effectiveness of the right to be forgotten 1605.5. The right to digital oblivion: a controversial subject 1635.6. Public archives versus the right to be forgotten 1655.6.1. Archives: exemptions from the right to be forgotten 1675.6.2. Online publication of archives and finding aids containing personal data 1685.6.3. Private digital archives and the right to be forgotten 1715.6.4. Web archiving and the right to be forgotten 1725.7. Google and the right to be forgotten 1735.8. Conclusion 178Conclusion 181List of Acronyms 185References 193Index 207
Data Center Handbook
DATA CENTER HANDBOOKWritten by 59 experts and reviewed by a seasoned technical advisory board, the Data Center Handbook is a thoroughly revised, one-stop resource that clearly explains the fundamentals, advanced technologies, and best practices used in planning, designing, building and operating a mission-critical, energy-efficient, sustainable data center. This handbook, in its second edition, covers anatomy, ecosystem and taxonomy of data centers that enable the Internet of Things and artificial intelligent ecosystems and encompass the following:SECTION 1: DATA CENTER OVERVIEW AND STRATEGIC PLANNING* Megatrends, the IoT, artificial intelligence, 5G network, cloud and edge computing* Strategic planning forces, location plan, and capacity planning * Green design & construction guidelines and best practices* Energy demand, conservation, and sustainability strategies* Data center financial analysis & risk managementSECTION 2: DATA CENTER TECHNOLOGIES* Software-defined environment* Computing, storage, network resource management* Wireless sensor networks in data centers* ASHRAE data center guidelines* Data center telecommunication cabling, BICSI and TIA 942* Rack-level and server-level cooling* Corrosion and contamination control* Energy saving technologies and server design* Microgrid and data centersSECTION 3: DATA CENTER DESIGN & CONSTRUCTION* Data center site selection* Architecture design: rack floor plan and facility layout* Mechanical design and cooling technologies* Electrical design and UPS* Fire protection* Structural design* Reliability engineering* Computational fluid dynamics* Project managementSECTION 4: DATA CENTER OPERATIONS TECHNOLOGIES* Benchmarking metrics and assessment* Data center infrastructure management* Data center air management* Disaster recovery and business continuity managementThe Data Center Handbook: Plan, Design, Build, and Operations of a Smart Data Center belongs on the bookshelves of any professionals who work in, with, or around a data center. HWAIYU GENG P.E. (Palo Alto, California, USA) is the founder and managing director at AmicaResearch.org promoting green planning, designing, building and operating of high-tech projects. He has over four decades of planning, engineering and management experience having worked with Westinghouse, Applied Materials, Hewlett Packard, Intel and Juniper Networks. He is a frequent speaker at international conferences. Mr. Geng, a patent holder, is also the editor/author of the IoT and Data Analytics Handbook, Manufacturing Engineering Handbook (2nd edition), and Semiconductor Manufacturing Handbook (2nd edition).ContibutorsChapter 1: Sustainable Data Center Strategic Planning, Design, Construction, and Operations with Emerging TechnologiesChapter 2: Global Data Center Energy Demand and Strategies to Conserve EnergyChapter 3 Energy and Sustainability in Data CentersChapter 4: Data Center Architecture and InfrastructureChapter 5 Cloud and Edge ComputingChapter 6: Financial Analysis, ROI and TCOChapter 7: Managing Data Center RiskChapter 8: Software Defined EnvironmentChapter 9: Computing, Storage, Networking Resource Management in Data CentersChapter 10: Wireless Sensor Networks to Improve Energy Efficiency in Data CentersChapter 11: ASHRAE Standards & Practices for Data CentersChapter 12: Data Center Telecommunications Cabling and TIA StandardsChapter 13: Air Side Economizer TechnologiesChapter 14: Rack-Level Cooling and Server-Level CoolingChapter 15: Corrosion (Contamination) Control for Mission Critical FacilitiesChapter 16: Rack PDU for Green Data CentersChapter 17: Fiber Cabling Fundamentals, Installation and MaintenanceChapter 18: Design of Energy Efficiency IT EquipmentChapter 19: Energy Saving Technologies of Servers in Data CentersChapter 20: Cyber-Security and Data CentersChapter 21: Consideration Of Microgrids For Data CentersChapter 22: Data Center Site Search and SelectionChapter 23: Architecture: Data Center Rack Floor Plan and Facility Layout DesignChapter 24: Mechanical Design in Data CentersChapter 25: Data Center Electrical DesignChapter 26: Electrical: Uninterruptible Power Supply SystemChapter 27: Structural Design in Data Centers: Natural Disaster ResilienceChapter 28: Fire Protection and Life Safety Design in Data CentersChapter 29: Reliability Engineering For Data Centers InfrastructuresChapter 30: Computational Fluid Dynamics for Data CentersChapter 31: Data Center Project ManagementChapter 32: Data Center Benchmark MetricsChapter 33: Data Center Infrastructure ManagementChapter 34: Data Center Air ManagementChapter 35: Energy Efficiency Assessment of Data Centers using Measurement and Management TechnologyChapter 36: Drive Data Center Management and Build Better AI with IT Devices as SensorsChapter 37: Preparing Data Centers for Natural Disasters and PandemicsIndex
Understanding Infrastructure Edge Computing
UNDERSTANDING INFRASTRUCTURE EDGE COMPUTINGA COMPREHENSIVE REVIEW OF THE KEY EMERGING TECHNOLOGIES THAT WILL DIRECTLY IMPACT AREAS OF COMPUTER TECHNOLOGY OVER THE NEXT FIVE YEARSInfrastructure edge computing is the model of data center and network infrastructure deployment which distributes a large number of physically small data centers around an area to deliver better performance and to enable new economical applications. It is vital for those operating at business or technical levels to be positioned to capitalize on the changes that will occur as a result of infrastructure edge computing.This book provides a thorough understanding of the growth of internet infrastructure from its inception to the emergence of infrastructure edge computing. Author Alex Marcham, an acknowledged leader in the field who coined the term ‘infrastructure edge computing,’ presents an accessible, accurate, and expansive view of the next generation of internet infrastructure. The book features illustrative examples of 5G mobile cellular networks, city-scale AI systems, self-driving cars, drones, industrial robots, and more—technologies that increase efficiency, save time and money, and improve safety. Covering state-of-the-art topics, this timely and authoritative book:* Presents a clear and accurate survey of the key emerging technologies that will impact data centers, 5G networks, artificial intelligence and cyber-physical systems, and other areas of computer technology* Explores how and why Internet infrastructure has evolved to where it stands today and where it needs to be in the near future* Covers a wide range of topics including distributed application workload operation, infrastructure and application security, and related technologies such as multi-access edge computing (MEC) and fog computing* Provides numerous use cases and examples of real-world applications which depend upon underlying edge infrastructureWritten for Information Technology practitioners, computer technology practitioners, and students, Understanding Infrastructure Edge Computing is essential reading for those looking to benefit from the coming changes in computer technology. ALEX MARCHAM has been working in infrastructure edge computing from the shaping of the market and establishment of the terminology and key concepts at numerous companies and open source projects which have been leading its development. Alex has been involved with most elements of infrastructure design and deployment as well as the architecture and development of the key use cases for this tier of Internet infrastructure.Preface xvAbout the Author xviiAcknowledgements xix1 INTRODUCTION 12 WHAT IS EDGE COMPUTING? 32.1 Overview 32.2 Defining the Terminology 32.3 Where Is the Edge? 42.3.1 A Tale of Many Edges 52.3.2 Infrastructure Edge 62.3.3 Device Edge 62.4 A Brief History 82.4.1 Third Act of the Internet 82.4.2 Network Regionalisation 102.4.3 CDNs and Early Examples 102.5 Why Edge Computing? 122.5.1 Latency 122.5.2 Data Gravity 132.5.3 Data Velocity 132.5.4 Transport Cost 142.5.5 Locality 142.6 Basic Edge Computing Operation 152.7 Summary 18References 183 INTRODUCTION TO NETWORK TECHNOLOGY 213.1 Overview 213.2 Structure of the Internet 213.2.1 1970s 223.2.2 1990s 223.2.3 2010s 233.2.4 2020s 233.2.5 Change over Time 233.3 The OSI Model 243.3.1 Layer 1 253.3.2 Layer 2 253.3.3 Layer 3 263.3.4 Layer 4 263.3.5 Layers 5, 6, and 7 273.4 Ethernet 283.5 IPv4 and IPv6 293.6 Routing and Switching 293.6.1 Routing 303.6.2 Routing Protocols 313.6.3 Routing Process 343.7 LAN, MAN, and WAN 413.8 Interconnection and Exchange 423.9 Fronthaul, Backhaul, and Midhaul 443.10 Last Mile or Access Networks 453.11 Network Transport and Transit 463.12 Serve Transit Fail (STF) Metric 483.13 Summary 51References 524 INTRODUCTION TO DATA CENTRE TECHNOLOGY 534.1 Overview 534.2 Physical Size and Design 534.3 Cooling and Power Efficiency 544.4 Airflow Design 564.5 Power Distribution 574.6 Redundancy and Resiliency 584.7 Environmental Control 614.8 Data Centre Network Design 614.9 Information Technology (IT) Equipment Capacity 654.10 Data Centre Operation 664.10.1 Notification 674.10.2 Security 674.10.3 Equipment Deployment 674.10.4 Service Offerings 684.10.5 Managed Colocation 684.11 Data Centre Deployment 694.11.1 Deployment Costing 694.11.2 Brownfield and Greenfield Sites 694.11.3 Other Factors 704.12 Summary 70References 705 INFRASTRUCTURE EDGE COMPUTING NETWORKS 715.1 Overview 715.2 Network Connectivity and Coverage Area 715.3 Network Topology 725.3.1 Full Mesh 745.3.2 Partial Mesh 745.3.3 Hub and Spoke 755.3.4 Ring 765.3.5 Tree 765.3.6 Optimal Topology 765.3.7 Inter-area Connectivity 775.4 Transmission Medium 785.4.1 Fibre 785.4.2 Copper 785.4.3 Wireless 795.5 Scaling and Tiered Network Architecture 805.6 Other Considerations 815.7 Summary 826 INFRASTRUCTURE EDGE DATA CENTRES 836.1 Overview 836.2 Physical Size and Design 836.2.1 Defining an Infrastructure Edge Data Centre 846.2.2 Size Categories 846.3 Heating and Cooling 1026.4 Airflow Design 1056.4.1 Traditional Designs 1076.4.2 Non-traditional Designs 1096.5 Power Distribution 1136.6 Redundancy and Resiliency 1146.6.1 Electrical Power Delivery and Generation 1166.6.2 Network Connectivity 1186.6.3 Cooling Systems 1206.6.4 Market Design 1226.6.5 Redundancy Certification 1246.6.6 Software Service Resiliency 1256.6.7 Physical Redundancy 1266.6.8 System Resiliency Example 1276.7 Environmental Control 1286.8 Data Centre Network Design 1316.9 Information Technology (IT) Equipment Capacity 1346.9.1 Operational Headroom 1356.10 Data Centre Operation 1356.10.1 Site Automation 1366.10.2 Single or Multi-tenant 1426.10.3 Neutral Host 1446.10.4 Network Operations Centre (NOC) 1456.11 Brownfield and Greenfield Sites 1476.12 Summary 1517 INTERCONNECTION AND EDGE EXCHANGE 1537.1 Overview 1537.2 Access or Last Mile Network Interconnection 1537.3 Backhaul and Midhaul Network Interconnection 1587.4 Internet Exchange 1607.5 Edge Exchange 1647.6 Interconnection Network Technology 1677.6.1 5G Networks 1687.6.2 4G Networks 1697.6.3 Cable Networks 1707.6.4 Fibre Networks 1727.6.5 Other Networks 1737.6.6 Meet Me Room (MMR) 1737.6.7 Cross Connection 1747.6.8 Virtual Cross Connection 1767.6.9 Interconnection as a Resource 1797.7 Peering 1807.8 Cloud On-ramps 1817.9 Beneficial Impact 1837.9.1 Latency 1837.9.2 Data Transport Cost 1847.9.3 Platform Benefit 1857.10 Alternatives to Interconnection 1867.11 Business Arrangements 1877.12 Summary 1888 INFRASTRUCTURE EDGE COMPUTING DEPLOYMENT 1898.1 Overview 1898.2 Physical Facilities 1898.3 Site Locations 1918.3.1 kW per kM2 1928.3.2 Customer Facility Selection 1938.3.3 Site Characteristics 1948.4 Coverage Areas 1958.5 Points of Interest 1978.6 Codes and Regulations 1988.7 Summary 2009 COMPUTING SYSTEMS AT THE INFRASTRUCTURE EDGE 2039.1 Overview 2039.2 What Is Suitable? 2039.3 Equipment Hardening 2049.4 Rack Densification 2059.4.1 Heterogenous Servers 2079.4.2 Processor Densification 2089.4.3 Supporting Equipment 2109.5 Parallel Accelerators 2119.5.1 Field Programmable Gate Arrays (FPGAs) 2139.5.2 Tensor Processing Units (TPUs) 2139.5.3 Graphics Processing Units (GPUs) 2149.5.4 Smart Network Interface Cards (NICs) 2159.5.5 Cryptographic Accelerators 2169.5.6 Other Accelerators 2179.5.7 FPGA, TPU, or GPU? 2179.6 Ideal Infrastructure 2189.6.1 Network Compute Utilisation 2189.7 Adapting Legacy Infrastructure 2219.8 Summary 221References 22210 MULTI-TIER DEVICE, DATA CENTRE, AND NETWORK RESOURCES 22310.1 Overview 22310.2 Multi-tier Resources 22310.3 Multi-tier Applications 22610.4 Core to Edge Applications 22810.5 Edge to Core Applications 23010.6 Infrastructure Edge and Device Edge Interoperation 23110.7 Summary 23411 DISTRIBUTED APPLICATION WORKLOAD OPERATION 23511.1 Overview 23511.2 Microservices 23511.3 Redundancy and Resiliency 23611.4 Multi-site Operation 23711.5 Workload Orchestration 23811.5.1 Processing Requirements 24011.5.2 Data Storage Requirements 24011.5.3 Network Performance Requirements 24111.5.4 Application Workload Cost Profile 24111.5.5 Redundancy and Resiliency Requirements 24211.5.6 Resource Marketplaces 24311.5.7 Workload Requirement Declaration 24311.6 Infrastructure Visibility 24411.7 Summary 24512 INFRASTRUCTURE AND APPLICATION SECURITY 24712.1 Overview 24712.2 Threat Modelling 24712.3 Physical Security 24912.4 Logical Security 25012.5 Common Security Issues 25112.5.1 Staff 25112.5.2 Visitors 25212.5.3 Network Attacks 25212.6 Application Security 25312.7 Security Policy 25412.8 Summary 25513 RELATED TECHNOLOGIES 25713.1 Overview 25713.2 Multi-access Edge Computing (MEC) 25713.3 Internet of Things (IoT) and Industrial Internet of Things (IIoT) 25813.4 Fog and Mist Computing 25913.5 Summary 260Reference 26014 USE CASE EXAMPLE: 5G 26114.1 Overview 26114.2 What Is 5G? 26114.2.1 5G New Radio (NR) 26214.2.2 5G Core Network (CN) 26314.3 5G at the Infrastructure Edge 26414.3.1 Benefits 26414.3.2 Architecture 26414.3.3 Considerations 26514.4 Summary 26615 USE CASE EXAMPLE: DISTRIBUTED AI 26715.1 Overview 26715.2 What Is AI? 26815.2.1 Machine Learning (ML) 26815.2.2 Deep Learning (DL) 26915.3 AI at the Infrastructure Edge 27015.3.1 Benefits 27015.3.2 Architecture 27115.3.3 Considerations 27215.4 Summary 27316 USE CASE EXAMPLE: CYBER-PHYSICAL SYSTEMS 27516.1 Overview 27516.2 What Are Cyber-physical Systems? 27516.2.1 Autonomous Vehicles 27616.2.2 Drones 27816.2.3 Robotics 28016.2.4 Other Use Cases 28016.3 Cyber-physical Systems at the Infrastructure Edge 28016.3.1 Benefits 28016.3.2 Architecture 28116.3.3 Considerations 28216.4 Summary 282Reference 28317 USE CASE EXAMPLE: PUBLIC OR PRIVATE CLOUD 28517.1 Overview 28517.2 What Is Cloud Computing? 28617.2.1 Public Clouds 28617.2.2 Private Clouds 28717.2.3 Hybrid Clouds 28717.2.4 Edge Cloud 28817.3 Cloud Computing at the Infrastructure Edge 28817.3.1 Benefits 28817.3.2 Architecture 28917.3.3 Considerations 29017.4 Summary 29018 OTHER INFRASTRUCTURE EDGE COMPUTING USE CASES 29118.1 Overview 29118.2 Near Premises Services 29118.3 Video Surveillance 29318.4 SD-WAN 29418.5 Security Services 29518.6 Video Conferencing 29618.7 Content Delivery 29718.8 Other Use Cases 29818.9 Summary 29919 END TO END: AN INFRASTRUCTURE EDGE PROJECT EXAMPLE 30119.1 Overview 30119.2 Defining Requirements 30119.2.1 Deciding on a Use Case 30219.2.2 Determining Deployment Locations 30419.2.3 Identifying Required Equipment 30619.2.4 Choosing an Infrastructure Edge Computing Network Operator 30719.2.5 Regional or National Data Centres 30719.3 Success Criteria 30719.4 Comparing Costs 30819.5 Alternative Options 30919.6 Initial Deployment 31019.7 Ongoing Operation 31119.7.1 SLA Breaches 31219.8 Project Conclusion 31219.9 Summary 31420 THE FUTURE OF INFRASTRUCTURE EDGE COMPUTING 31520.1 Overview 31520.2 Today and Tomorrow 31520.3 The Next Five Years 31620.4 The Next 10 Years 31620.5 Summary 31621 CONCLUSION 317Appendix A: Acronyms and Abbreviations 319Index 323
Inside the World of Computing
Computers and the Internet are an undeniable and inextricable part of our daily lives. This book is for those who wish to better understand how this came to be. It explores the technological bases of computers, networks, software and data management, leading to the development of four �pillars� on which the essential applications that have a strong impact on individuals and society are based: embedded systems, Artificial Intelligence, the Internet, image processing and vision.We will travel to the heart of major application areas: robotics, virtual reality, health, mobility, energy, the factory of the future, not forgetting the major questions that this �digitization� can raise. This book is the author�s testimony after fifty years spent in environments that are very open to new technologies. It offers perspectives on the evolution of the digital world that we live in. JEAN-LOIC DELHAYE has a PhD in Artificial Intelligence. He directed the Centre National Universitaire Sud de Calcul, France, before piloting partnerships and the valorization of research at the Centre Inria Rennes?Bretagne Altlantique, France. He has also been very active in national and European collaborations on high performance computing. Foreword xiJean-Pierre BANÂTREPreface xvAcknowledgments xxiCHAPTER 1. FROM THE CALCULATOR TO THE SUPERCOMPUTER 11.1. Introduction 11.2. Some important concepts 11.2.1. Information and data 11.2.2. Binary system 31.2.3. Coding 31.2.4. Algorithm 51.2.5. Program 71.3. Towards automation of calculations 71.3.1. Slide rule 71.3.2. The Pascaline 81.3.3. The Jacquard loom 91.3.4. Babbage’s machine 91.3.5. The first desktop calculators 101.3.6. Hollerith’s machine 111.4. The first programmable computers 121.4.1. Konrad Zuse’s machines 121.4.2. Colossus 131.4.3. ENIAC 131.5. Generations of computers 141.5.1. First generation: the transition to electronics 151.5.2. Second generation: the era of the transistor 171.5.3. Third generation: the era of integrated circuits 201.5.4. Fourth generation: the era of microprocessors 241.6. Supercomputers 281.6.1. Some fields of use 281.6.2. History of supercomputers 291.6.3. Towards exaflops 331.7. What about the future? 351.7.1. An energy and ecological challenge 351.7.2. Revolutions in sight? 36CHAPTER 2. COMPUTER NETWORKS AND THEIR APPLICATIONS 372.1. Introduction 372.2. A long history 382.3. Computer network infrastructure 422.3.1. Geographic coverage: from PAN to WAN 432.3.2. Communication media 442.3.3. Interconnection equipment and topologies 482.3.4. Two other characteristics of computer networks 522.3.5. Quality of service 542.4. Communication protocols and the Internet 552.4.1. The first protocols 552.4.2. The OSI model 562.4.3. The history of the Internet 572.4.4. The TCP/IP protocol 582.4.5. IP addressing 592.4.6. Management and use of the Internet 602.4.7. Evolving technologies 612.4.8. What future? 622.5. Applications 632.5.1. The World Wide Web 642.5.2. Cloud computing 672.5.3. The Internet of Things 682.5.4. Ubiquitous computing and spontaneous networks 722.6. Networks and security 742.6.1. Vulnerabilities 742.6.2. The protection of a network 762.6.3. Message encryption 762.6.4. Checking its security 77CHAPTER 3. SOFTWARE 793.1. Introduction 793.2. From algorithm to computer program 803.2.1. Programs and subprograms 823.2.2. Programming languages 833.3. Basic languages and operating systems 853.3.1. Basic languages 853.3.2. Operating system functions 863.3.3. A bit of history 883.3.4. Universal operating systems 913.3.5. Targeted operating systems 933.4. “High-level” programming and applications 963.4.1. Imperative languages 963.4.2. Functional languages 983.4.3. Object programming 993.4.4. Other programming languages 1003.4.5. The most used languages 1013.5. Software development 1023.5.1. Software categories 1023.5.2. Software quality 1033.5.3. Development methods 1043.5.4. Software engineering 1073.6. Software verification and validation 1073.6.1. Errors with sometimes tragic consequences 1073.6.2. Software testing 1093.6.3. Formal methods 1113.6.4. Software certification 1143.7. Legal protection and distribution of software 1153.7.1. Legal protection of software 1153.7.2. Licenses 1163.7.3. Free software and open source 1173.8. The software market 118CHAPTER 4. DATA: FROM BINARY ELEMENT TO INTELLIGENCE 1214.1. Introduction 1214.2. Data and information 1224.2.1. Digitization of data 1224.2.2. Data compression 1254.3. The structuring of data towards information 1254.3.1. Structured data 1264.3.2. Semi-structured data and the Web 1274.4. Files and their formats 1284.5. Databases 1294.5.1. The main characteristics 1294.5.2. DBMS models 1314.5.3. Database design 1334.5.4. Enterprise resource planning (ERP) systems 1334.5.5. Other types of databases 1344.5.6. Data protection in a DB 1374.6. Intelligence and Big Data 1374.7. Data ownership and Open Data 1414.7.1. Personal data 1414.7.2. Opening up public data: Open Data 142CHAPTER 5. TECHNOLOGY BUILDING BLOCKS 1455.1. Embedded systems 1455.1.1. Specific architectures 1465.1.2. Some fields of use 1475.2. Artificial intelligence (AI) 1505.2.1. A bit of history 1505.2.2. Intelligence or statistics? 1525.2.3. Important work around automatic learning 1525.2.4. A multiplication of applications 1545.2.5. The challenges of AI 1555.2.6. What about intelligence? 1565.3. The Internet 1575.3.1. Mobility 1575.3.2. Social networks 1585.3.3. The Internet of Things 1595.3.4. The Cloud 1595.3.5. Blockchain 1595.3.6. Vulnerabilities 1605.4. Image processing and vision 1605.4.1. A bit of history 1605.4.2. Image sources and their uses 1615.4.3. The digital image 1625.4.4. Image storage and compression 1635.4.5. Computing and images 1645.4.6. Some applications 1655.5. Conclusion 166CHAPTER 6. SOME AREAS OF APPLICATION 1676.1. Robots 1676.1.1. A bit of history 1686.1.2. Fields of use regarding robots today 1696.1.3. Communication in the world of robots 1736.1.4. Fear of robots 1746.1.5. Challenges for researchers 1756.2. Virtual reality and augmented reality 1756.2.1. A bit of history 1766.2.2. Hardware configurations of virtual reality 1776.2.3. Fields of use of virtual reality 1796.2.4. Augmented reality 1806.3. Health 1816.3.1. Health informatics 1826.3.2. Information technology at the service of our health 1846.4. The connected (and soon autonomous?) car 1856.4.1. Levels of autonomy 1866.4.2. Challenges associated with the autonomous car 1876.4.3. Advantages and disadvantages of the autonomous car 1886.5. The smart city 1896.5.1. Smart energy 1906.5.2. Smart buildings 1906.5.3. Smart infrastructure 1916.5.4. Smart governance 1926.5.5. Dangers 1936.6. Smart mobility 1936.7. The factory of the future 1956.7.1. Technologies 1956.7.2. Issues 1976.7.3. The place of the human 198CHAPTER 7. SOCIETAL ISSUES 1997.1. Security 1997.1.1. Specific characteristics 2007.1.2. Some great threats 2007.1.3. Acting to protect oneself 2027.2. The respect of private life 2027.2.1. Our personal data 2027.2.2. Uses of our data 2047.2.3. What about the future? 205x Inside the World of Computing7.3. Influence on social life 2067.3.1. The development of social ties 2067.3.2. Citizen participation 2077.3.3. The socialization of knowledge 2077.4. Dangers to democracy 2087.4.1. The liberation of speech 2087.4.2. Private life under surveillance 2087.4.3. Job insecurity 2097.4.4. The power of the big Internet firms 2097.5. The digital divide 2107.5.1. From division to exclusion 2107.5.2. Digital technology and education 2117.6. Mastering the use of artificial intelligence 2127.7. The intelligent prosthesis and the bionic man 2137.8. Transhumanism 2147.9. What kind of society for tomorrow? 215Bibliography 217Index 219
CRAN Recipes
Want to use the power of R sooner rather than later? Don’t have time to plow through wordy texts and online manuals? Use this book for quick, simple code to get your projects up and running. It includes code and examples applicable to many disciplines. Written in everyday language with a minimum of complexity, each chapter provides the building blocks you need to fit R’s astounding capabilities to your analytics, reporting, and visualization needs.CRAN Recipes recognizes how needless jargon and complexity get in your way. Busy professionals need simple examples and intuitive descriptions; side trips and meandering philosophical discussions are left for other books.Here R scripts are condensed, to the extent possible, to copy-paste-run format. Chapters and examples are structured to purpose rather than particular functions (e.g., “dirty data cleanup” rather than the R package name “janitor”). Everyday language eliminates the need to know functions/packages in advance.WHAT YOU WILL LEARN* Carry out input/output; visualizations; data munging; manipulations at the group level; and quick data exploration* Handle forecasting (multivariate, time series, logistic regression, Facebook’s Prophet, and others)* Use text analytics; sampling; financial analysis; and advanced pattern matching (regex)* Manipulate data using DPLYR: filter, sort, summarize, add new fields to datasets, and apply powerful IF functions* Create combinations or subsets of files using joins* Write efficient code using pipes to eliminate intermediate steps (MAGRITTR)* Work with string/character manipulation of all types (STRINGR)* Discover counts, patterns, and how to locate whole words* Do wild-card matching, extraction, and invert-match* Work with dates using LUBRIDATE* Fix dirty data; attractive formatting; bad habits to avoidWHO THIS BOOK IS FORProgrammers/data scientists with at least some prior exposure to R.WILLIAM A. YARBERRY, JR., CPA, CISA, is principal consultant, ICCM Consulting LLC, based in Houston, Texas. His practice is focused on IT governance, Sarbanes-Oxley compliance, security consulting, and business analytics for cost management. He was previously a senior manager with PricewaterhouseCoopers, responsible for telecom and network services in the Southwest region. Yarberry has more than 30 years’ experience in a variety of IT-related services, including application development, internal audit management, outsourcing administration, and Sarbanes-Oxley consulting.His books include The Effective CIO (co-authored), Computer Telephony Integration, $250K Consulting, DPLYR, 50,000 Random Numbers, Telecommunications Cost Management, and GDPR: A Short Primer. In addition, he has written over 20 professional articles on topics ranging from wireless security to change management. One of his articles, "Audit Rights in an Outsource Environment," received the Institute of Internal Auditors Outstanding Contributor Award.Prior to joining PricewaterhouseCoopers, Yarberry was director of telephony services for Enron Corporation. He was responsible for operations, planning, and architectural design for voice communications servers and related systems for more than 7,000 employees. Yarberry graduated Phi Beta Kappa in chemistry from the University of Tennessee and earned an MBA at the University of Memphis. He enjoys reading history, swimming, hiking, and spending time with family.1: DPLYR2: STRINGR3: Lubridate4: Regular Expressions: Introduction5: Typical Uses6: Some Simple Patterns7: Character Classes8: Elements of Regular Expressions9: The Magnificent Seven10: Regular Expressions in Stringr11: Unicode12: Tools for Development and Resources13: Regex Summary14: Recipes for Common R Tasks15: Data Structures16: Visualization17: Simple Prediction Methods18: Smorgasbord of Simple Statistical Tests19: Validation of Data20: Shortcuts and Miscellaneous21: ConclusionAppendices
Chance, Calculation and Life
Chance, Calculation and Life brings together 16 original papers from the colloquium of the same name, organized by the International Cultural Center of Cerisy in 2019. From mathematics to the humanities and biology, there are many concepts and questions related to chance. What are the different types of chance? Does chance correspond to a lack of knowledge about the causes of events, or is there a truly intrinsic and irreducible chance? Does chance preside over our decisions? Does it govern evolution? Is it at the origin of life? What part do chance and necessity play in biology? This book answers these fundamental questions by bringing together the clear and richly documented contributions of mathematicians, physicists, biologists and philosophers who make this book an incomparable tool for work and reflection. THIERRY GAUDIN is an engineer at MINES ParisTech and holds a doctorate in Information Sciences and Communication from Paris Nanterre University, France. He is a widely renowned expert in innovation policy and has worked with the OECD, the European Commission and the World Bank. MARIE-CHRISTINE MAUREL is Professor at Sorbonne University and a researcher at the Institute of Systematics, Evolution, Biodiversity, MNHN, Paris, France. JEAN-CHARLES POMEROL is Professor Emeritus at Sorbonne University, France. He is a specialist in Decision Support Systems and former project leader for information technology in the Engineering Sciences Department at the CNRS. He was formerly in charge of the Artificial Intelligence laboratory at UPMC, Paris, as well as the President of UPMC between 2006 and 2011. Preface xiThierry GAUDIN, Marie-Christine MAUREL, Jean-Charles POMEROLIntroduction xvThierry GAUDIN, Marie-Christine MAUREL, Jean-Charles POMEROLPART 1. RANDOMNESS IN ALL OF ITS ASPECTS 1CHAPTER 1. CLASSICAL, QUANTUM AND BIOLOGICAL RANDOMNESS AS RELATIVE UNPREDICTABILITY 3Cristian S. CALUDE and Giuseppe LONGO1.1. Introduction 31.1.1. Brief historical overview 41.1.2. Preliminary remarks 51.2. Randomness in classical dynamics 61.3. Quantum randomness 81.4. Randomness in biology 151.5. Random sequences: a theory invariant approach 211.6. Classical and quantum randomness revisited 241.6.1. Classical versus algorithmic randomness 241.6.2. Quantum versus algorithmic randomness 261.7. Conclusion and opening: toward a proper biological randomness 271.8. Acknowledgments 301.9. References 30CHAPTER 2. IN THE NAME OF CHANCE 37Gilles PAGÈS2.1. The birth of probabilities and games of chance 372.1.1. Solutions 382.1.2. To what end? 402.2. A very brief history of probabilities 412.3. Chance? What chance? 422.4. Prospective possibility 452.4.1. LLN + CLT + ENIAC = MC 452.4.2. Generating chance through numbers 462.4.3. Going back the other way 482.4.4. Prospective possibility as master of the world? 502.5. Appendix: Congruent generators, can prospective chance be periodic? 532.5.1. A little modulo n arithmetic 532.5.2. From erratic arithmetic to algorithmic randomness 562.5.3. And, the winner is... Mersenne Twister 623.. 602.6. References 61CHAPTER 3. CHANCE IN A FEW LANGUAGES 63Clarisse HERRENSCHMIDT3.1. Classical Sanskrit 643.2. Persian and Arabic 653.3. Ancient Greek 663.4. Russian 673.5. Latin 673.6. French 693.7. English 713.8. Dice, chance and the symbolic world 723.9. References 77CHAPTER 4. THE COLLECTIVE DETERMINISM OF QUANTUM RANDOMNESS 79François VANNUCCI4.1. True or false chance 794.2. Chance sneaks into uncertainty 814.3. The world of the infinitely small 824.4. A more figurative example 844.5. Einstein’s act of resistance 864.6. Schrödinger’s cat to neutrino oscillations 874.7. Chance versus the anthropic principle 904.8. And luck in life? 924.9. Chance and freedom 94CHAPTER 5. WAVE-PARTICLE CHAOS TO THE STABILITY OF LIVING 97Stéphane DOUADY5.1. Introduction 975.2. The chaos of the wave-particle 975.3. The stability of living things 1045.4. Conclusion 1075.5. Acknowledgments 1085.6. References 108CHAPTER 6. CHANCE IN COSMOLOGY: RANDOM AND TURBULENT CREATION OF MULTIPLE COSMOS 109Michel CASSÉ6.1. Is quantum cosmology oxymoronic? 1096.2. Between two realities – at the entrance and exit – is virtuality 1206.3. Who will sing the metamorphoses of this high vacuum? 1206.4. Loop lament 1216.5. The quantum vacuum exists, Casimir has met it 1226.6. The generosity of the quantum vacuum 1226.7. Landscapes 1266.8. The good works of Inflation 1286.9. Sub species aeternitatis 1296.10. The smiling vacuum 130CHAPTER 7. THE CHANCE IN DECISION: WHEN NEURONS FLIP A COIN 133Mathias PESSIGLIONE7.1. A very subjective utility 1337.2. A minimum rationality 1347.3. There is noise in the choices 1357.4. On the volatility of parameters 1377.5. When the brain wears rose-tinted glasses 1387.6. The neurons that take a vote 1407.7. The will to move an index finger 1427.8. Free will in debate 1437.9. The virtue of chance 1447.10. References 145CHAPTER 8. TO HAVE A SENSE OF LIFE: A POETIC RECONNAISSANCE 147Georges AMAR8.1. References 157CHAPTER 9. DIVINE CHANCE 159Bertrand VERGELY9.1. Thinking by chance 1599.2. Chance, need: why choose? 1609.3. When chance is not chance 1629.4. When chance comes from elsewhere 166CHAPTER 10. CHANCE AND THE CREATIVE PROCESS 169Ivan MAGRIN-CHAGNOLLEAU10.1. Introduction 16910.2. Chance 17010.3. Creation 17310.4. Chance in the artistic creative process 17610.5. An art of the present moment 17910.6. Conclusion 18110.7. References 182PART 2. RANDOMNESS, BIOLOGY AND EVOLUTION 185CHAPTER 11. EPIGENETICS, DNA AND CHROMATIN DYNAMICS: WHERE IS THE CHANCE AND WHERE IS THE NECESSITY? 187David SITBON and Jonathan B. WEITZMAN11.1. Introduction 18711.2. Random combinations 18711.3. Random alterations 18811.4. Beyond the gene 18911.5. Epigenetic variation 19011.6. Concluding remarks 19211.7. Acknowledgments 19311.8. References 193CHAPTER 12. WHEN ACQUIRED CHARACTERISTICS BECOME HERITABLE: THE LESSON OF GENOMES 197Bernard DUJON12.1. Introduction 19712.2. Horizontal genetic exchange in prokaryotes 19912.3. Two specificities of eukaryotes theoretically oppose horizontal gene transfer 20012.4. Criteria for genomic analysis 20112.5. Abundance of horizontal transfers in unicellular eukaryotes 20212.6. Remarkable horizontal genetic transfers in pluricellular eukaryotes 20312.7. Main mechanisms of horizontal genetic transfers 20412.8. Introgressions and limits to the concept of species 20712.9. Conclusion 20812.10. References 208CHAPTER 13. THE EVOLUTIONARY TRAJECTORIES OF ORGANISMS ARE NOT STOCHASTIC213Philippe GRANDCOLAS13.1. Evolution and stochasticity: a few metaphors 21313.2. The Gouldian metaphor of the “replay” of evolution 21413.3. The replay of evolution: what happened 21513.4. Evolutionary replay experiments 21713.5. Phylogenies versus experiments 21813.6. Stochasticity, evolution and extinction 21913.7. Conclusion 21913.8. References 220CHAPTER 14. EVOLUTION IN THE FACE OF CHANCE 221Amaury LAMBERT14.1. Introduction 22114.2. Waddington and the concept of canalization 22414.3. A stochastic model of Darwinian evolution 22814.3.1. Redundancy and neutral networks 22814.3.2. A toy model 22914.3.3. Mutation-selection algorithm 23114.4. Numerical results 23114.4.1. Canalization 23114.4.2. Target selection 23414.4.3. Neighborhood selection 23514.5. Discussion 23814.6. Acknowledgments 239CHAPTER 15. CHANCE, CONTINGENCY AND THE ORIGINS OF LIFE: SOME HISTORICAL ISSUES 241Antonio LAZCANO15.1. Acknowledgments 24615.2. References 246CHAPTER 16. CHANCE, COMPLEXITY AND THE IDEA OF A UNIVERSAL ETHICS 249Jean-Paul DELAHAYE16.1. Cosmic evolution and advances in computation 25016.2. Two notions of complexity 25116.3. Biological computations 25216.4. Energy and emergy 25316.5. What we hold onto 25416.6. Noah knew this already! 25416.7. Create, protect and collect 25516.8. An ethics of organized complexity 25516.9. Not so easy 25616.10. References 258List of Authors 261Index 265
PostgreSQL Query Optimization
Write optimized queries. This book helps you write queries that perform fast and deliver results on time. You will learn that query optimization is not a dark art practiced by a small, secretive cabal of sorcerers. Any motivated professional can learn to write efficient queries from the get-go and capably optimize existing queries. You will learn to look at the process of writing a query from the database engine’s point of view, and know how to think like the database optimizer.The book begins with a discussion of what a performant system is and progresses to measuring performance and setting performance goals. It introduces different classes of queries and optimization techniques suitable to each, such as the use of indexes and specific join algorithms. You will learn to read and understand query execution plans along with techniques for influencing those plans for better performance. The book also covers advanced topics such as the use of functions and procedures, dynamic SQL, and generated queries. All of these techniques are then used together to produce performant applications, avoiding the pitfalls of object-relational mappers.WHAT YOU WILL LEARN* Identify optimization goals in OLTP and OLAP systems* Read and understand PostgreSQL execution plans* Distinguish between short queries and long queries* Choose the right optimization technique for each query type* Identify indexes that will improve query performance* Optimize full table scans* Avoid the pitfalls of object-relational mapping systems* Optimize the entire application rather than just database queriesWHO THIS BOOK IS FORIT professionals working in PostgreSQL who want to develop performant and scalable applications, anyone whose job title contains the words “database developer” or “database administrator" or who is a backend developer charged with programming database calls, and system architects involved in the overall design of application systems running against a PostgreSQL databaseHENRIETTA DOMBROVSKAYA is a database researcher and developer with over 35 years of academic and industrial experience. She holds a PhD in computer science from the University of Saint Petersburg, Russia. At present, she is Associate Director of Databases at Braviant Holdings, Chicago, Illinois. She is an active member of the PostgreSQL community, a frequent speaker at the PostgreSQL conference, and a local organizer of the Chicago PostgreSQL User Group. Her research interests are tightly coupled with practice and are focused on developing efficient interactions between applications and databases. She is a winner of the “Technologist of the Year” 2019 award of the Illinois Technology Association.BORIS NOVIKOV is currently a professor in the Department of Informatics at National Research University Higher School of Economics in Saint Petersburg, Russia. He graduated from Leningrad University’s School of Mathematics and Mechanics. He has worked for Saint Petersburg University for a number of years and moved to his current position in January, 2019. His research interests are in a broad area of information management and include several aspects of design, development, and tuning of databases, applications, and database management systems. He also has interests in distributed scalable systems for stream processing and analytics.ANNA BAILLIEKOVA is Senior Data Engineer at Zendesk. Previously, she built ETL pipelines, data warehouse resources, and reporting tools as a team lead on the Division Operations team at Epic. She has also held analyst roles on a variety of political campaigns and at Greenberg Quinlan Rosner Research. She received her undergraduate degree cum laude with College Honors in political science and computer science from Knox College in Galesburg, Illinois. 1. Why Optimize?2. Theory - Yes, We Need It!3. Even More Theory Algorithms4. Understanding Execution Plans5. Short Queries and Indexes6. Long Queries and Full Scans7. Long Queries: Additional Techniques8. Optimizing Data Modification9. Design Matters10. Application Development and Performance11. Functions12. Dynamic SQL13. Avoiding the Pitfalls of Object-Relational Mapping14. More Complex Filtering and Search15. Ultimate Optimization Algorithm16. Conclusion
Pointers in C Programming
Gain a better understanding of pointers, from the basics of how pointers function at the machine level, to using them for a variety of common and advanced scenarios. This short contemporary guide book on pointers in C programming provides a resource for professionals and advanced students needing in-depth hands-on coverage of pointer basics and advanced features. It includes the latest versions of the C language, C20, C17, and C14.You’ll see how pointers are used to provide vital C features, such as strings, arrays, higher-order functions and polymorphic data structures. Along the way, you’ll cover how pointers can optimize a program to run faster or use less memory than it would otherwise.There are plenty of code examples in the book to emulate and adapt to meet your specific needs.WHAT YOU WILL LEARN* Work effectively with pointers in your C programming* Learn how to effectively manage dynamic memory* Program with strings and arrays* Create recursive data structures* Implement function pointersWHO THIS BOOK IS FORIntermediate to advanced level professional programmers, software developers, and advanced students or researchers. Prior experience with C programming is expected.Thomas Mailund is an associate professor in bioinformatics at Aarhus University, Denmark. He has a background in math and computer science, including experience programming and teaching in the C and R programming languages. For the last decade, his main focus has been on genetics and evolutionary studies, particularly comparative genomics, speciation, and gene flow between emerging species.1. Pointers and the random access memory model2. Memory management3. Strings and arrays4. Recursive data structures5. Function pointers
Responsible Data Science
EXPLORE THE MOST SERIOUS PREVALENT ETHICAL ISSUES IN DATA SCIENCE WITH THIS INSIGHTFUL NEW RESOURCEThe increasing popularity of data science has resulted in numerous well-publicized cases of bias, injustice, and discrimination. The widespread deployment of “Black box” algorithms that are difficult or impossible to understand and explain, even for their developers, is a primary source of these unanticipated harms, making modern techniques and methods for manipulating large data sets seem sinister, even dangerous. When put in the hands of authoritarian governments, these algorithms have enabled suppression of political dissent and persecution of minorities. To prevent these harms, data scientists everywhere must come to understand how the algorithms that they build and deploy may harm certain groups or be unfair.Responsible Data Science delivers a comprehensive, practical treatment of how to implement data science solutions in an even-handed and ethical manner that minimizes the risk of undue harm to vulnerable members of society. Both data science practitioners and managers of analytics teams will learn how to:* Improve model transparency, even for black box models* Diagnose bias and unfairness within models using multiple metrics* Audit projects to ensure fairness and minimize the possibility of unintended harmPerfect for data science practitioners, Responsible Data Science will also earn a spot on the bookshelves of technically inclined managers, software developers, and statisticians.GRANT FLEMING is a Data Scientist at Elder Research Inc. His professional focus is on machine learning for social science applications, model interpretability, civic technology, and building software tools for reproducible data science.PETER BRUCE is the Senior Learning Officer at Elder Research, Inc., author of several best-selling texts on data science, and Founder of the Institute for Statistics Education at Statistics.com, an Elder Research Company.Introduction xixPART I MOTIVATION FOR ETHICAL DATA SCIENCE AND BACKGROUND KNOWLEDGE 1CHAPTER 1 RESPONSIBLE DATA SCIENCE 3The Optum Disaster 4Jekyll and Hyde 5Eugenics 7Galton, Pearson, and Fisher 7Ties between Eugenics and Statistics 7Ethical Problems in Data Science Today 9Predictive Models 10From Explaining to Predicting 10Predictive Modeling 11Setting the Stage for Ethical Issues to Arise 12Classic Statistical Models 12Black-Box Methods 14Important Concepts in Predictive Modeling 19Feature Selection 19Model-Centric vs. Data-Centric Models 20Holdout Sample and Cross-Validation 20Overfitting 21Unsupervised Learning 22The Ethical Challenge of Black Boxes 23Two Opposing Forces 24Pressure for More Powerful AI 24Public Resistance and Anxiety 24Summary 25CHAPTER 2 BACKGROUND: MODELING AND THE BLACK-BOX ALGORITHM 27Assessing Model Performance 27Predicting Class Membership 28The Rare Class Problem 28Lift and Gains 28Area Under the Curve 29AUC vs. Lift (Gains) 31Predicting Numeric Values 32Goodness-of-Fit 32Holdout Sets and Cross-Validation 33Optimization and Loss Functions 34Intrinsically Interpretable Models vs. Black-Box Models 35Ethical Challenges with Interpretable Models 38Black-Box Models 39Ensembles 39Nearest Neighbors 41Clustering 41Association Rules 42Collaborative Filters 42Artificial Neural Nets and Deep Neural Nets 43Problems with Black-Box Predictive Models 45Problems with Unsupervised Algorithms 47Summary 48CHAPTER 3 THE WAYS AI GOES WRONG, AND THE LEGAL IMPLICATIONS 49AI and Intentional Consequences by Design 50Deepfakes 50Supporting State Surveillance and Suppression 51Behavioral Manipulation 52Automated Testing to Fine-Tune Targeting 53AI and Unintended Consequences 55Healthcare 56Finance 57Law Enforcement 58Technology 60The Legal and Regulatory Landscape around AI 61Ignorance Is No Defense: AI in the Context of Existing Law and Policy 63A Finger in the Dam: Data Rights, Data Privacy, and Consumer Protection Regulations 64Trends in Emerging Law and Policy Related to AI 66Summary 69PART II THE ETHICAL DATA SCIENCE PROCESS 71CHAPTER 4 THE RESPONSIBLE DATA SCIENCE FRAMEWORK 73Why We Keep Building Harmful AI 74Misguided Need for Cutting-Edge Models 74Excessive Focus on Predictive Performance 74Ease of Access and the Curse of Simplicity 76The Common Cause 76The Face Thieves 78An Anatomy of Modeling Harms 79The World: Context Matters for Modeling 80The Data: Representation Is Everything 83The Model: Garbage In, Danger Out 85Model Interpretability: Human Understanding for Superhuman Models 86Efforts Toward a More Responsible Data Science 89Principles Are the Focus 90Nonmaleficence 90Fairness 90Transparency 91Accountability 91Privacy 92Bridging the Gap Between Principles and Practice with the Responsible Data Science (RDS) Framework 92Justification 94Compilation 94Preparation 95Modeling 96Auditing 96Summary 97CHAPTER 5 MODEL INTERPRETABILITY: THE WHAT AND THE WHY 99The Sexist Résumé Screener 99The Necessity of Model Interpretability 101Connections Between Predictive Performance and Interpretability 103Uniting (High) Model Performance and Model Interpretability 105Categories of Interpretability Methods 107Global Methods 107Local Methods 113Real-World Successes of Interpretability Methods 113Facilitating Debugging and Audit 114Leveraging the Improved Performance of Black-Box Models 116Acquiring New Knowledge 116Addressing Critiques of Interpretability Methods 117Explanations Generated by Interpretability Methods Are Not Robust 118Explanations Generated by Interpretability Methods Are Low Fidelity 120The Forking Paths of Model Interpretability 121The Four-Measure Baseline 122Building Our Own Credit Scoring Model 124Using Train-Test Splits 125Feature Selection and Feature Engineering 125Baseline Models 127The Importance of Making Your Code Work for Everyone 129Execution Variability 129Addressing Execution Variability with Functionalized Code 130Stochastic Variability 130Addressing Stochastic Variability via Resampling 130Summary 133PART III EDS IN PRACTICE 135CHAPTER 6 BEGINNING A RESPONSIBLE DATA SCIENCE PROJECT 137How the Responsible Data Science Framework Addresses the Common Cause 138Datasets Used 140Regression Datasets—Communities and Crime 140Classification Datasets—COMPAS 140Common Elements Across Our Analyses 141Project Structure and Documentation 141Project Structure for the Responsible DataScience Framework: Everything in Its Place 142Documentation: The Responsible Thing to Do 145Beginning a Responsible Data Science Project 151Communities and Crime (Regression) 151Justification 151Compilation 154Identifying Protected Classes 157Preparation—Data Splitting and Feature Engineering 159Datasheets 161COMPAS (Classification) 164Justification 164Compilation 166Identifying Protected Classes 168Preparation 169Summary 172CHAPTER 7 AUDITING A RESPONSIBLE DATA SCIENCE PROJECT 173Fairness and Data Science in Practice 175The Many Different Conceptions of Fairness 175Different Forms of Fairness Are Trade-Offs with Each Other 177Quantifying Predictive Fairness Within a Data Science Project 179Mitigating Bias to Improve Fairness 185Preprocessing 185In-processing 186Postprocessing 186Classification Example: COMPAS 187Prework: Code Practices, Modeling, and Auditing 187Justification, Compilation, and Preparation Review 189Modeling 191Auditing 200Per-Group Metrics: Overall 200Per-Group Metrics: Error 202Fairness Metrics 204Interpreting Our Models: Why Are They Unfair? 207Analysis for Different Groups 209Bias Mitigation 214Preprocessing: Oversampling 214Postprocessing: Optimizing ThresholdsAutomatically 218Postprocessing: Optimizing Thresholds Manually 219Summary 223CHAPTER 8 AUDITING FOR NEURAL NETWORKS 225Why Neural Networks Merit Their Own Chapter 227Neural Networks Vary Greatly in Structure 227Neural Networks Treat Features Differently 229Neural Networks Repeat Themselves 231A More Impenetrable Black Box 232Baseline Methods 233Representation Methods 233Distillation Methods 234Intrinsic Methods 235Beginning a Responsible Neural Network Project 236Justification 236Moving Forward 239Compilation 239Tracking Experiments 241Preparation 244Modeling 245Auditing 247Per-Group Metrics: Overall 247Per-Group Metrics: Unusual Definitions of “False Positive” 248Fairness Metrics 249Interpreting Our Models: Why Are They Unfair? 252Bias Mitigation 253Wrap-Up 255Auditing Neural Networks for Natural Language Processing 258Identifying and Addressing Sources of Bias in NLP 258The Real World 259Data 260Models 261Model Interpretability 262Summary 262CHAPTER 9 CONCLUSION 265How Can We Do Better? 267The Responsible Data Science Framework 267Doing Better As Managers 269Doing Better As Practitioners 270A Better Future If We Can Keep It 271Index 273
Das Medium aus der Maschine
»Die Informatik entwirft drei sehr unterschiedliche Bilder von Computer: Maschine – Werkzeug – Medium. Wie können so gegensätzliche Vorstellungen im gleichen Artefakt einen technologischen Ausdruck finden? Zu welchen Widersprüchen führen so differierende Sichtweisen in der Forschungspraxis der Informatik? Welches sind die Konzepte, über die sie sich verbinden lassen? Und wie verändert sich das Gewicht der Bilder von Maschine, Werkzeug und Medium in der Entwicklungsgeschichte des Computers und der Informatik?«Aus der EinleitungUnveränderter NachdruckHeidi Schelhowe, Prof. Dr., ist Professorin für Digitale Medien in der Bildung in der Informatik an der Universität Bremen und leitet dort die Arbeitsgruppe "Digitale Medien in der Bildung" (dimeb).
Basiswissen Mobile App Testing
Grundlegende Methoden, Verfahren und Werkzeuge zum Testen von mobilen Applikationen.»Basiswissen Mobile App Testing« vermittelt die Grundlagen des Testens mobiler Apps und gibt einen fundierten Überblick über geeignete Testarten, Testmethoden, den Testprozess und das Testkonzept für mobile Anwendungen. Auch auf Qualitätskriterien, mobile App-Plattformen, Werkzeuge und die Automatisierung der Testausführung wird eingegangen. Viele Beispiele aus realen Kundenprojekten erleichtern die Umsetzung des Gelernten in die Praxis.Die Themen im Einzelnen:Geschäftliche & technische Faktoren, Herausforderungen & Risiken, Teststrategien für mobile AppsTests mit Bezug zur mobilen PlattformÜbliche Testarten und der Testprozess für mobile AppsMobile App-Plattformen, Werkzeuge und UmgebungenAutomatisierung der TestausführungDas Buch ist konform zum ISTQB®-Lehrplan »Certified Mobile Application Tester« und eignet sich mit vielen Beispielen und Übungen nicht nur bestens für die Prüfungsvorbereitung, sondern dient gleichzeitig als kompaktes Basiswerk zum Thema in der Praxis und an Hochschulen.Über die Autoren:Björn Lemke ist Managing Consultant bei der trendig technology services GmbH. Die Schwerpunkte seiner Arbeit sind Softwarequalitätssicherung, Integrated Technology and Operations (ITOps), IT-Service-Management (ITIL), Testmanagement, Testdatenmanagement, Testinfrastrukturmanagement sowie Mobile Application Testing in kleinen bis hin zu sehr grossen Projekten.Nils Röttger arbeitet bei der imbus AG in Möhrendorf als Berater, Projektleiter und Speaker und ist u. a. verantwortlich für die Ausbildung und den Bereich Mobile Testing. In seinen Vorträgen beschäftigt er sich immer wieder mit Themen wie exploratives Testen, Usability oder Ethik im Softwaretest.
Introducing .NET for Apache Spark
Get started using Apache Spark via C# or F# and the .NET for Apache Spark bindings. This book is an introduction to both Apache Spark and the .NET bindings. Readers new to Apache Spark will get up to speed quickly using Spark for data processing tasks performed against large and very large datasets. You will learn how to combine your knowledge of .NET with Apache Spark to bring massive computing power to bear by distributed processing of extremely large datasets across multiple servers.This book covers how to get a local instance of Apache Spark running on your developer machine and shows you how to create your first .NET program that uses the Microsoft .NET bindings for Apache Spark. Techniques shown in the book allow you to use Apache Spark to distribute your data processing tasks over multiple compute nodes. You will learn to process data using both batch mode and streaming mode so you can make the right choice depending on whether you are processing an existing dataset or are working against new records in micro-batches as they arrive. The goal of the book is leave you comfortable in bringing the power of Apache Spark to your favorite .NET language.WHAT YOU WILL LEARN* Install and configure Spark .NET on Windows, Linux, and macOS * Write Apache Spark programs in C# and F# using the .NET bindings* Access and invoke the Apache Spark APIs from .NET with the same high performance as Python, Scala, and R* Encapsulate functionality in user-defined functions* Transform and aggregate large datasets * Execute SQL queries against files through Apache Hive* Distribute processing of large datasets across multiple servers* Create your own batch, streaming, and machine learning programsWHO THIS BOOK IS FOR.NET developers who want to perform big data processing without having to migrate to Python, Scala, or R; and Apache Spark developers who want to run natively on .NET and take advantage of the C# and F# ecosystemsED ELLIOTT is a data engineer who has been working in IT for 20 years and has focused on data for the last 15 years. He uses Apache Spark at work and has been contributing to the Microsoft .NET for Apache Spark open source project since it was released in 2019. Ed has been blogging and writing since 2014 at his own blog as well as for SQL Server Central and Redgate. He has spoken at a number of events such as SQLBits, SQL Saturday, and the GroupBy conference.IntroductionPART I. GETTING STARTED1. Understanding Apache Spark2. Setting up Spark3. Programming with .NET for Apache SparkPART II. THE APIS4. User-Defined Functions5. The DataFrame API6. Spark SQL and Hive Tables7. Spark Machine Learning APIPART III. EXAMPLES8. Batch Mode Processing9. Structured Streaming10. Troubleshooting11. Delta LakePART IV. APPENDICESAppendix A. Running in the CloudAppendix B. Implementing .Net for Apache Spark Code
Becoming a Data Head
"TURN YOURSELF INTO A DATA HEAD. YOU'LL BECOME A MORE VALUABLE EMPLOYEE AND MAKE YOUR ORGANIZATION MORE SUCCESSFUL."Thomas H. Davenport, Research Fellow, Author of Competing on Analytics, Big Data @ Work, and The AI AdvantageYOU'VE HEARD THE HYPE AROUND DATA—NOW GET THE FACTS.In Becoming a Data Head: How to Think, Speak, and Understand Data Science, Statistics, and Machine Learning, award-winning data scientists Alex Gutman and Jordan Goldmeier pull back the curtain on data science and give you the language and tools necessary to talk and think critically about it.You'll learn how to:* Think statistically and understand the role variation plays in your life and decision making* Speak intelligently and ask the right questions about the statistics and results you encounter in the workplace* Understand what's really going on with machine learning, text analytics, deep learning, and artificial intelligence* Avoid common pitfalls when working with and interpreting dataBecoming a Data Head is a complete guide for data science in the workplace: covering everything from the personalities you’ll work with to the math behind the algorithms. The authors have spent years in data trenches and sought to create a fun, approachable, and eminently readable book. Anyone can become a Data Head—an active participant in data science, statistics, and machine learning. Whether you're a business professional, engineer, executive, or aspiring data scientist, this book is for you.ALEX J. GUTMAN, PHD, is a Data Scientist, Corporate Trainer, and Accredited Professional Statistician. His professional focus is on statistical and machine learning and he has extensive experience working as a Data Scientist for the Department of Defense and two Fortune 50 companies.JORDAN GOLDMEIER is a Data Scientist, author, speaker, and community leader. He is a seven-time recipient of the Microsoft Most Valuable Professional Award and he has taught analytics to members of the Pentagon and Fortune 500 companies.Acknowledgments xiiiForeword xxiiiIntroduction xxviiPART ONE THINKING LIKE A DATA HEADCHAPTER 1 WHAT IS THE PROBLEM? 3Questions a Data Head Should Ask 4Why Is This Problem Important? 4Who Does This Problem Affect? 6What If We Don’t Have the Right Data? 6When Is the Project Over? 7What If We Don’t Like the Results? 7Understanding Why Data Projects Fail 8Customer Perception 8Discussion 10Working on Problems That Matter 11Chapter Summary 11CHAPTER 2 WHAT IS DATA? 13Data vs. Information 13An Example Dataset 14Data Types 15How Data Is Collected and Structured 16Observational vs. Experimental Data 16Structured vs. Unstructured Data 17Basic Summary Statistics 18Chapter Summary 19CHAPTER 3 PREPARE TO THINK STATISTICALLY 21Ask Questions 22There Is Variation in All Things 23Scenario: Customer Perception (The Sequel) 24Case Study: Kidney-Cancer Rates 26Probabilities and Statistics 28Probability vs. Intuition 29Discovery with Statistics 31Chapter Summary 33PART TWO SPEAKING LIKE A DATA HEADCHAPTER 4 ARGUE WITH THE DATA 37What Would You Do? 38Missing Data Disaster 39Tell Me the Data Origin Story 43Who Collected the Data? 44How Was the Data Collected? 44Is the Data Representative? 45Is There Sampling Bias? 46What Did You Do with Outliers? 46What Data Am I Not Seeing? 47How Did You Deal with Missing Values? 47Can the Data Measure What You Want It to Measure? 48Argue with Data of All Sizes 48Chapter Summary 49CHAPTER 5 EXPLORE THE DATA 51Exploratory Data Analysis and You 52Embracing the Exploratory Mindset 52Questions to Guide You 53The Setup 53Can the Data Answer the Question? 54Set Expectations and Use Common Sense 54Do the Values Make Intuitive Sense? 54Watch Out: Outliers and Missing Values 58Did You Discover Any Relationships? 59Understanding Correlation 59Watch Out: Misinterpreting Correlation 60Watch Out: Correlation Does Not Imply Causation 62Did You Find New Opportunities in the Data? 63Chapter Summary 63CHAPTER 6 EXAMINE THE PROBABILITIES 65Take a Guess 66The Rules of the Game 66Notation 67Conditional Probability and Independent Events 69The Probability of Multiple Events 69Two Things That Happen Together 69One Thing or the Other 70Probability Thought Exercise 72Next Steps 73Be Careful Assuming Independence 74Don’t Fall for the Gambler’s Fallacy 74All Probabilities Are Conditional 75Don’t Swap Dependencies 76Bayes’ Theorem 76Ensure the Probabilities Have Meaning 79Calibration 80Rare Events Can, and Do, Happen 80Chapter Summary 81CHAPTER 7 CHALLENGE THE STATISTICS 83Quick Lessons on Inference 83Give Yourself Some Wiggle Room 84More Data, More Evidence 84Challenge the Status Quo 85Evidence to the Contrary 86Balance Decision Errors 88The Process of Statistical Inference 89The Questions You Should Ask to Challenge the Statistics 90What Is the Context for These Statistics? 90What Is the Sample Size? 91What Are You Testing? 92What Is the Null Hypothesis? 92Assuming Equivalence 93What Is the Significance Level? 93How Many Tests Are You Doing? 94Can I See the Confidence Intervals? 95Is This Practically Significant? 96Are You Assuming Causality? 96Chapter Summary 97PART THREE UNDERSTANDING THE DATA SCIENTIST’S TOOLBOXCHAPTER 8 SEARCH FOR HIDDEN GROUPS 101Unsupervised Learning 102Dimensionality Reduction 102Creating Composite Features 103Principal Component Analysis 105Principal Components in Athletic Ability 105PCA Summary 108Potential Traps 109Clustering 110k-Means Clustering 111Clustering Retail Locations 111Potential Traps 113Chapter Summary 114CHAPTER 9 UNDERSTAND THE REGRESSION MODEL 117Supervised Learning 117Linear Regression: What It Does 119Least Squares Regression: Not Just a Clever Name 120Linear Regression: What It Gives You 123Extending to Many Features 124Linear Regression: What Confusion It Causes 125Omitted Variables 125Multicollinearity 126Data Leakage 127Extrapolation Failures 128Many Relationships Aren’t Linear 128Are You Explaining or Predicting? 128Regression Performance 130Other Regression Models 131Chapter Summary 131CHAPTER 10 UNDERSTAND THE CLASSIFICATION MODEL 133Introduction to Classification 133What You’ll Learn 134Classification Problem Setup 135Logistic Regression 135Logistic Regression: So What? 138Decision Trees 139Ensemble Methods 142Random Forests 143Gradient Boosted Trees 143Interpretability of Ensemble Models 145Watch Out for Pitfalls 145Misapplication of the Problem 146Data Leakage 146Not Splitting Your Data 146Choosing the Right Decision Threshold 147Misunderstanding Accuracy 147Confusion Matrices 148Chapter Summary 150CHAPTER 11 UNDERSTAND TEXT ANALYTICS 151Expectations of Text Analytics 151How Text Becomes Numbers 153A Big Bag of Words 153N-Grams 157Word Embeddings 158Topic Modeling 160Text Classification 163Naïve Bayes 164Sentiment Analysis 166Practical Considerations When Working with Text 167Big Tech Has the Upper Hand 168Chapter Summary 169CHAPTER 12 CONCEPTUALIZE DEEP LEARNING 171Neural Networks 172How Are Neural Networks Like the Brain? 172A Simple Neural Network 173How a Neural Network Learns 174A Slightly More Complex Neural Network 175Applications of Deep Learning 178The Benefits of Deep Learning 179How Computers “See” Images 180Convolutional Neural Networks 182Deep Learning on Language and Sequences 183Deep Learning in Practice 185Do You Have Data? 185Is Your Data Structured? 186What Will the Network Look Like? 186Artificial Intelligence and You 187Big Tech Has the Upper Hand 188Ethics in Deep Learning 189Chapter Summary 190PART FOUR ENSURING SUCCESSCHAPTER 13 WATCH OUT FOR PITFALLS 193Biases and Weird Phenomena in Data 194Survivorship Bias 194Regression to the Mean 195Simpson’s Paradox 195Confirmation Bias 197Effort Bias (aka the “Sunk Cost Fallacy”) 197Algorithmic Bias 198Uncategorized Bias 198The Big List of Pitfalls 199Statistical and Machine Learning Pitfalls 199Project Pitfalls 200Chapter Summary 202CHAPTER 14 KNOW THE PEOPLE AND PERSONALITIES 203Seven Scenes of Communication Breakdowns 204The Postmortem 204Storytime 205The Telephone Game 206Into the Weeds 206The Reality Check 207The Takeover 207The Blowhard 208Data Personalities 208Data Enthusiasts 209Data Cynics 209Data Heads 209Chapter Summary 210CHAPTER 15 WHAT’S NEXT? 211Index 215