Allgemein
Pro Java Microservices with Quarkus and Kubernetes
Build and design microservices using Java and the Red Hat Quarkus Framework. This book will help you quickly get started with the features and concerns of a microservices architecture. It will introduce Docker and Kubernetes to help you deploy your microservices.You will be guided on how to install the appropriate tools to work properly. For those who are new to enterprise development using Quarkus, you will be introduced to its core principles and main features through a deep step-by-step tutorial. For experts, this book offers some recipes that illustrate how to split monoliths and implement microservices and deploy them as containers to Kubernetes.By the end of reading this book, you will have practical hands-on experience of building microservices using Quarkus and you will master deploying them to Kubernetes.WHAT YOU WILL LEARN* Work with Quarkus and GraalVM* Split a monolith using the domain-driven design approach* Implement the cloud and microservices patterns* Rethink the deployment process* Introduce containerization, Docker, and Kubernetes to your toolkit* Boost microservices efficiency and performance with Azure* Play with Quarkus and distributed application runtimesWHO THIS BOOK IS FORJava developers who want to build microservices using Red Hat Quarkus and who want to deploy them in Kubernetes.Nebrass Lamouchi is a senior software engineer at Microsoft, addicted to Java and cloud technologies. He was a NetBeans Dream Team member until December 2017. Nebrass was one of the happy four winners of the Oracle Groundbreaker Awards in May 2019. Since March 2013 he has also worked as a project leader at the OWASP Foundation on the Barbarus Project.Table of ContentsDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4What this book covers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Reader feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Getting started with Containerization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Introduction to containerization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Introducing Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Installation and first hands-on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Docker Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Diving into Docker Containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Meeting the Docker Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Achieving more with Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Containerization is not Docker only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42Introduction to the Monolithic architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Introduction to an actual situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Presenting the context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45How to solve these issues ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Coding the Monolithic application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Presenting our domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Coding the application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106Upgrading the Monolithic application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Implementing QuarkuShop Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Building and Running QuarkuShop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145Building & Deploying the Monolithic application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147Importing the Project in Azure DevOps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147Creating the CI/CD pipelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171Adding the anti-disasters layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173Implementing the Security Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173Implementing the Monitoring Layer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226Microservices Architecture Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228Microservices Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229Making the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230Splitting the Monolith: Bombarding the domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232What is Domain-Driven Design ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232Bombarding QuarkuShop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234Dependencies and Commons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234Entities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234Refactoring Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236Transactional Boundaries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238Applying DDD to the code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240Applying Bounded Contexts to Java Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240Locating & breaking the BC Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242Meeting the microservices concerns and patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250Cloud Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250What’s next? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257Getting started with Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259What is Kubernetes ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259Run Kubernetes locally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269Practical Summary & Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271Additional reading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272Implementing the Cloud Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275Bringing the Monolithic Universe to Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293Building the Kubernetized Microservices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295Creating the Commons Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295Implementing the Product µservice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299Implementing the Order µservice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306Implementing the Customer µservice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315Implementing the User µservice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318Flying all over the Sky with Quarkus and Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320Implementing the Circuit Breaker pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320Implementing the Log Aggregation pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324Implementing the Distributed Tracing pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334Implementing the API Gateway pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346Playing with Quarkus in Azure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347Bringing Dapr into the game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349Final words & thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351About the author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
State-of-the-Art Deep Learning Models in TensorFlow
Use TensorFlow 2.x in the Google Colab ecosystem to create state-of-the-art deep learning models guided by hands-on examples. The Colab ecosystem provides a free cloud service with easy access to on-demand GPU (and TPU) hardware acceleration for fast execution of the models you learn to build. This book teaches you state-of-the-art deep learning models in an applied manner with the only requirement being an Internet connection. The Colab ecosystem provides everything else that you need, including Python, TensorFlow 2.x, GPU and TPU support, and Jupyter Notebooks.The book begins with an example-driven approach to building input pipelines that feed all machine learning models. You will learn how to provision a workspace on the Colab ecosystem to enable construction of effective input pipelines in a step-by-step manner. From there, you will progress into data augmentation techniques and TensorFlow datasets to gain a deeper understanding of how to work with complex datasets. You will find coverage of Tensor Processing Units (TPUs) and transfer learning followed by state-of-the-art deep learning models, including autoencoders, generative adversarial networks, fast style transfer, object detection, and reinforcement learning.Author Dr. Paper provides all the applied math, programming, and concepts you need to master the content. Examples range from relatively simple to very complex when necessary. Examples are carefully explained, concise, accurate, and complete. Care is taken to walk you through each topic through clear examples written in Python that you can try out and experiment with in the Google Colab ecosystem in the comfort of your own home or office.WHAT YOU WILL LEARN* Take advantage of the built-in support of the Google Colab ecosystem* Work with TensorFlow data sets* Create input pipelines to feed state-of-the-art deep learning models* Create pipelined state-of-the-art deep learning models with clean and reliable Python code* Leverage pre-trained deep learning models to solve complex machine learning tasks* Create a simple environment to teach an intelligent agent to make automated decisionsWHO THIS BOOK IS FORReaders who want to learn the highly popular TensorFlow deep learning platform, those who wish to master the basics of state-of-the-art deep learning models, and those looking to build competency with a modern cloud service tool such as Google ColabDR. PAPER is a retired academic from the Utah State University (USU) Data Analytics and Management Information Systems department in the Huntsman School of Business. He has over 30 years of higher education teaching experience. At USU, he taught for 27 years in the classroom and distance education over satellite. He taught a variety of classes at the undergraduate, graduate, and doctorate levels, but he specializes in applied technology education.Dr. Paper has competency in several programming languages, but his focus is currently on deep learning with Python in the TensorFlow-Colab Ecosystem. He has published extensively on machine learning, including Apress books: Data Science Fundamentals for Python and MongoDB, Hands-on Scikit-Learn for Machine Learning Applications: Data Science Fundamentals with Python, and TensorFlow 2.x in the Colaboratory Cloud: An Introduction to Deep Learning on Google’s Cloud Service. He has also published more than 100 academic articles.Besides growing up in family businesses, Dr. Paper has worked for Texas Instruments, DLS, Inc., and the Phoenix Small Business Administration. He has performed IS consulting work for IBM, AT&T, Octel, the Utah Department of Transportation, and the Space Dynamics Laboratory. He has worked on research projects with several corporations, including Caterpillar, Fannie Mae, Comdisco, IBM, RayChem, Ralston Purina, and Monsanto. He maintains contacts in corporations such as Google, Micron, Oracle, and Goldman Sachs.1. Build TensorFlow Input Pipelines2. Increase the Diversity of your Dataset with Data Augmentation3. TensorFlow Datasets4. Deep Learning with TensorFlow Datasets5. Introduction to Tensor Processing Units6. Simple Transfer Learning with TensorFlow Hub7. Advanced Transfer Learning8. Stacked Autoencoders9. Convolutional and Variational Autoencoders10. Generative Adversarial Networks11. Progressive Growing Generative Adversarial Networks12. Fast Style Transfer13. Object Detection14. An Introduction to Reinforcement Learning
Artificial Intelligence Hardware Design
ARTIFICIAL INTELLIGENCE HARDWARE DESIGNLEARN FOUNDATIONAL AND ADVANCED TOPICS IN NEURAL PROCESSING UNIT DESIGN WITH REAL-WORLD EXAMPLES FROM LEADING VOICES IN THE FIELDIn Artificial Intelligence Hardware Design: Challenges and Solutions, distinguished researchers and authors Drs. Albert Chun Chen Liu and Oscar Ming Kin Law deliver a rigorous and practical treatment of the design applications of specific circuits and systems for accelerating neural network processing. Beginning with a discussion and explanation of neural networks and their developmental history, the book goes on to describe parallel architectures, streaming graphs for massive parallel computation, and convolution optimization. The authors offer readers an illustration of in-memory computation through Georgia Tech’s Neurocube and Stanford’s Tetris accelerator using the Hybrid Memory Cube, as well as near-memory architecture through the embedded eDRAM of the Institute of Computing Technology, the Chinese Academy of Science, and other institutions. Readers will also find a discussion of 3D neural processing techniques to support multiple layer neural networks, as well as information like:* A thorough introduction to neural networks and neural network development history, as well as Convolutional Neural Network (CNN) models* Explorations of various parallel architectures, including the Intel CPU, Nvidia GPU, Google TPU, and Microsoft NPU, emphasizing hardware and software integration for performance improvement* Discussions of streaming graph for massive parallel computation with the Blaize GSP and Graphcore IPU* An examination of how to optimize convolution with UCLA Deep Convolutional Neural Network accelerator filter decompositionPerfect for hardware and software engineers and firmware developers, Artificial Intelligence Hardware Design is an indispensable resource for anyone working with Neural Processing Units in either a hardware or software capacity. ALBERT CHUN CHEN LIU, PHD, is Chief Executive Officer of Kneron. He is Adjunct Associate Professor at National Tsing Hua University, National Chiao Tung University, and National Cheng Kung University. He has published over 15 IEEE papers and is an IEEE Senior Member. He is a recipient of the IBM Problem Solving Award based on the use of the EIP tool suite in 2007 and IEEE TCAS Darlington award in 2021.OSCAR MING KIN LAW, PHD, is the Director of Engineering at Kneron. He works on smart robot development and in-memory architecture for neural networks. He has over twenty years of experience in the semiconductor industry working with CPU, GPU, and mobile design. He has also published over 60 patents in various areas. Author Biographies xiPreface xiiiAcknowledgments xvTable of Figures xvii1 INTRODUCTION 11.1 Development History 21.2 Neural Network Models 41.3 Neural Network Classification 41.3.1 Supervised Learning 41.3.2 Semi-supervised Learning 51.3.3 Unsupervised Learning 61.4 Neural Network Framework 61.5 Neural Network Comparison 10Exercise 11References 122 DEEP LEARNING 132.1 Neural Network Layer 132.1.1 Convolutional Layer 132.1.2 Activation Layer 172.1.3 Pooling Layer 182.1.4 Normalization Layer 192.1.5 Dropout Layer 202.1.6 Fully Connected Layer 202.2 Deep Learning Challenges 22Exercise 22References 243 PARALLEL ARCHITECTURE 253.1 Intel Central Processing Unit (CPU) 253.1.1 Skylake Mesh Architecture 273.1.2 Intel Ultra Path Interconnect (UPI) 283.1.3 Sub Non-unified Memory Access Clustering (SNC) 293.1.4 Cache Hierarchy Changes 313.1.5 Single/Multiple Socket Parallel Processing 323.1.6 Advanced Vector Software Extension 333.1.7 Math Kernel Library for Deep Neural Network (MKL-DNN) 343.2 NVIDIA Graphics Processing Unit (GPU) 393.2.1 Tensor Core Architecture 413.2.2 Winograd Transform 443.2.3 Simultaneous Multithreading (SMT) 453.2.4 High Bandwidth Memory (HBM2) 463.2.5 NVLink2 Configuration 473.3 NVIDIA Deep Learning Accelerator (NVDLA) 493.3.1 Convolution Operation 503.3.2 Single Data Point Operation 503.3.3 Planar Data Operation 503.3.4 Multiplane Operation 503.3.5 Data Memory and Reshape Operations 513.3.6 System Configuration 513.3.7 External Interface 523.3.8 Software Design 523.4 Google Tensor Processing Unit (TPU) 533.4.1 System Architecture 533.4.2 Multiply–Accumulate (MAC) Systolic Array 553.4.3 New Brain Floating-Point Format 553.4.4 Performance Comparison 573.4.5 Cloud TPU Configuration 583.4.6 Cloud Software Architecture 603.5 Microsoft Catapult Fabric Accelerator 613.5.1 System Configuration 643.5.2 Catapult Fabric Architecture 653.5.3 Matrix-Vector Multiplier 653.5.4 Hierarchical Decode and Dispatch (HDD) 673.5.5 Sparse Matrix-Vector Multiplication 68Exercise 70References 714 STREAMING GRAPH THEORY 734.1 Blaize Graph Streaming Processor 734.1.1 Stream Graph Model 734.1.2 Depth First Scheduling Approach 754.1.3 Graph Streaming Processor Architecture 764.2 Graphcore Intelligence Processing Unit 794.2.1 Intelligence Processor Unit Architecture 794.2.2 Accumulating Matrix Product (AMP) Unit 794.2.3 Memory Architecture 794.2.4 Interconnect Architecture 794.2.5 Bulk Synchronous Parallel Model 81Exercise 83References 845 CONVOLUTION OPTIMIZATION 855.1 Deep Convolutional Neural Network Accelerator 855.1.1 System Architecture 865.1.2 Filter Decomposition 875.1.3 Streaming Architecture 905.1.3.1 Filter Weights Reuse 905.1.3.2 Input Channel Reuse 925.1.4 Pooling 925.1.4.1 Average Pooling 925.1.4.2 Max Pooling 935.1.5 Convolution Unit (CU) Engine 945.1.6 Accumulation (ACCU) Buffer 945.1.7 Model Compression 955.1.8 System Performance 955.2 Eyeriss Accelerator 975.2.1 Eyeriss System Architecture 975.2.2 2D Convolution to 1D Multiplication 985.2.3 Stationary Dataflow 995.2.3.1 Output Stationary 995.2.3.2 Weight Stationary 1015.2.3.3 Input Stationary 1015.2.4 Row Stationary (RS) Dataflow 1045.2.4.1 Filter Reuse 1045.2.4.2 Input Feature Maps Reuse 1065.2.4.3 Partial Sums Reuse 1065.2.5 Run-Length Compression (RLC) 1065.2.6 Global Buffer 1085.2.7 Processing Element Architecture 1085.2.8 Network-on- Chip (NoC) 1085.2.9 Eyeriss v2 System Architecture 1125.2.10 Hierarchical Mesh Network 1165.2.10.1 Input Activation HM-NoC 1185.2.10.2 Filter Weight HM-NoC 1185.2.10.3 Partial Sum HM-NoC 1195.2.11 Compressed Sparse Column Format 1205.2.12 Row Stationary Plus (RS+) Dataflow 1225.2.13 System Performance 123Exercise 125References 1256 IN-MEMORY COMPUTATION 1276.1 Neurocube Architecture 1276.1.1 Hybrid Memory Cube (HMC) 1276.1.2 Memory Centric Neural Computing (MCNC) 1306.1.3 Programmable Neurosequence Generator (PNG) 1316.1.4 System Performance 1326.2 Tetris Accelerator 1336.2.1 Memory Hierarchy 1336.2.2 In-Memory Accumulation 1336.2.3 Data Scheduling 1356.2.4 Neural Network Vaults Partition 1366.2.5 System Performance 1376.3 NeuroStream Accelerator 1386.3.1 System Architecture 1386.3.2 NeuroStream Coprocessor 1406.3.3 4D Tiling Mechanism 1406.3.4 System Performance 141Exercise 143References 1437 NEAR-MEMORY ARCHITECTURE 1457.1 DaDianNao Supercomputer 1457.1.1 Memory Configuration 1457.1.2 Neural Functional Unit (NFU) 1467.1.3 System Performance 1497.2 Cnvlutin Accelerator 1507.2.1 Basic Operation 1517.2.2 System Architecture 1517.2.3 Processing Order 1547.2.4 Zero-Free Neuron Array Format (ZFNAf) 1557.2.5 The Dispatcher 1557.2.6 Network Pruning 1577.2.7 System Performance 1577.2.8 Raw or Encoded Format (RoE) 1587.2.9 Vector Ineffectual Activation Identifier Format (VIAI) 1597.2.10 Ineffectual Activation Skipping 1597.2.11 Ineffectual Weight Skipping 161Exercise 161References 1618 NETWORK SPARSITY 1638.1 Energy Efficient Inference Engine (EIE) 1638.1.1 Leading Nonzero Detection (LNZD) Network 1638.1.2 Central Control Unit (CCU) 1648.1.3 Processing Element (PE) 1648.1.4 Deep Compression 1668.1.5 Sparse Matrix Computation 1678.1.6 System Performance 1698.2 Cambricon-X Accelerator 1698.2.1 Computation Unit 1718.2.2 Buffer Controller 1718.2.3 System Performance 1748.3 SCNN Accelerator 1758.3.1 SCNN PT-IS-CP-Dense Dataflow 1758.3.2 SCNN PT-IS-CP-Sparse Dataflow 1778.3.3 SCNN Tiled Architecture 1788.3.4 Processing Element Architecture 1798.3.5 Data Compression 1808.3.6 System Performance 1808.4 SeerNet Accelerator 1838.4.1 Low-Bit Quantization 1838.4.2 Efficient Quantization 1848.4.3 Quantized Convolution 1858.4.4 Inference Acceleration 1868.4.5 Sparsity-Mask Encoding 1868.4.6 System Performance 188Exercise 188References 1889 3D NEURAL PROCESSING 1919.1 3D Integrated Circuit Architecture 1919.2 Power Distribution Network 1939.3 3D Network Bridge 1959.3.1 3D Network-on-Chip 1959.3.2 Multiple-Channel High-Speed Link 1959.4 Power-Saving Techniques 1989.4.1 Power Gating 1989.4.2 Clock Gating 199Exercise 200References 201Appendix A: Neural Network Topology 203Index 205
Principles of Blockchain Systems
THIS BOOK IS THE FIRST TO PRESENT THE STATE OF THE ART AND PROVIDE TECHNICAL FOCUS ON THE LATEST ADVANCES IN THE FOUNDATIONS OF BLOCKCHAIN SYSTEMS. It is a collaborative work between specialists in cryptography, distributed systems, formal languages, and economics, and addresses hot topics in blockchains from a theoretical perspective: cryptographic primitives, consensus, formalization of blockchain properties, game theory applied to blockchains, and economical issues.This book reflects the expertise of the various authors, and is intended to benefit researchers, students, and engineers who seek an understanding of the theoretical foundations of blockchains.* Preface* Acknowledgments* Cryptographic Tools for Blockchains* A Consensus Taxonomy in the Blockchain Era* The Next 700 Smart Contract Languages* Formalization of Blockchain Properties* Adversarial Cross-Chain Commerce* Strategic Interactions in Blockchain: A Survey of Game-Theoretic Approaches* Bankruptcy Solutions as Reward Functions in Mining Pools* Tokens and ICOs: A Review of the Economic Literature* Editors’ Biographies
Data Science For Dummies
MONETIZE YOUR COMPANY’S DATA AND DATA SCIENCE EXPERTISE WITHOUT SPENDING A FORTUNE ON HIRING INDEPENDENT STRATEGY CONSULTANTS TO HELPWhat if there was one simple, clear process for ensuring that all your company’s data science projects achieve a high a return on investment? What if you could validate your ideas for future data science projects, and select the one idea that’s most prime for achieving profitability while also moving your company closer to its business vision? There is.INDUSTRY-ACCLAIMED DATA SCIENCE CONSULTANT, LILLIAN PIERSON, SHARES HER PROPRIETARY STAR FRAMEWORK – A SIMPLE, PROVEN PROCESS FOR LEADING PROFIT-FORMING DATA SCIENCE PROJECTS.Not sure what data science is yet? Don’t worry! Parts 1 and 2 of Data Science For Dummies will get all the bases covered for you. And if you’re already a data science expert? Then you really won’t want to miss the data science strategy and data monetization gems that are shared in Part 3 onward throughout this book.Data Science For Dummies demonstrates:* The only process you’ll ever need to lead profitable data science projects* Secret, reverse-engineered data monetization tactics that no one’s talking about* The shocking truth about how simple natural language processing can be* How to beat the crowd of data professionals by cultivating your own unique blend of data science expertise Whether you’re new to the data science field or already a decade in, you’re sure to learn something new and incredibly valuable from Data Science For Dummies. Discover how to generate massive business wins from your company’s data by picking up your copy today.LILLIAN PIERSON is the CEO of Data-Mania, where she supports data professionals in transforming into world-class leaders and entrepreneurs. She has trained well over one million individuals on the topics of AI and data science. Lillian has assisted global leaders in IT, government, media organizations, and nonprofits.INTRODUCTION 1About This Book 3Foolish Assumptions 3Icons Used in This Book 4Beyond the Book 4Where to Go from Here 4PART 1: GETTING STARTED WITH DATA SCIENCE 5CHAPTER 1: WRAPPING YOUR HEAD AROUND DATA SCIENCE 7Seeing Who Can Make Use of Data Science 8Inspecting the Pieces of the Data Science Puzzle 10Collecting, querying, and consuming data 11Applying mathematical modeling to data science tasks 12Deriving insights from statistical methods 12Coding, coding, coding — it’s just part of the game 13Applying data science to a subject area 13Communicating data insights 14Exploring Career Alternatives That Involve Data Science 15The data implementer 16The data leader 16The data entrepreneur 17CHAPTER 2: TAPPING INTO CRITICAL ASPECTS OF DATA ENGINEERING 19Defining Big Data and the Three Vs 19Grappling with data volume 21Handling data velocity 21Dealing with data variety 22Identifying Important Data Sources 23Grasping the Differences among Data Approaches 24Defining data science 25Defining machine learning engineering 26Defining data engineering 26Comparing machine learning engineers, data scientists, and data engineers 27Storing and Processing Data for Data Science 28Storing data and doing data science directly in the cloud 28Storing big data on-premise 32Processing big data in real-time 35PART 2: USING DATA SCIENCE TO EXTRACT MEANING FROM YOUR DATA 37CHAPTER 3: MACHINE LEARNING MEANS USING A MACHINE TO LEARN FROM DATA 39Defining Machine Learning and Its Processes 40Walking through the steps of the machine learning process 40Becoming familiar with machine learning terms 41Considering Learning Styles 42Learning with supervised algorithms 42Learning with unsupervised algorithms 43Learning with reinforcement 43Seeing What You Can Do 43Selecting algorithms based on function 44Using Spark to generate real-time big data analytics 48CHAPTER 4: MATH, PROBABILITY, AND STATISTICAL MODELING 51Exploring Probability and Inferential Statistics 52Probability distributions 53Conditional probability with Naïve Bayes 55Quantifying Correlation 56Calculating correlation with Pearson’s r 56Ranking variable-pairs using Spearman’s rank correlation 58Reducing Data Dimensionality with Linear Algebra 59Decomposing data to reduce dimensionality 59Reducing dimensionality with factor analysis 63Decreasing dimensionality and removing outliers with PCA 64Modeling Decisions with Multiple Criteria Decision-Making 65Turning to traditional MCDM 65Focusing on fuzzy MCDM 67Introducing Regression Methods 67Linear regression 67Logistic regression 69Ordinary least squares (OLS) regression methods 70Detecting Outliers 70Analyzing extreme values 70Detecting outliers with univariate analysis 71Detecting outliers with multivariate analysis 73Introducing Time Series Analysis 73Identifying patterns in time series 74Modeling univariate time series data 75CHAPTER 5: GROUPING YOUR WAY INTO ACCURATE PREDICTIONS 77Starting with Clustering Basics 78Getting to know clustering algorithms 79Examining clustering similarity metrics 81Identifying Clusters in Your Data 82Clustering with the k-means algorithm 82Estimating clusters with kernel density estimation (KDE) 84Clustering with hierarchical algorithms 84Dabbling in the DBScan neighborhood 87Categorizing Data with Decision Tree and Random Forest Algorithms 88Drawing a Line between Clustering and Classification 89Introducing instance-based learning classifiers 90Getting to know classification algorithms 90Making Sense of Data with Nearest Neighbor Analysis 93Classifying Data with Average Nearest Neighbor Algorithms 94Classifying with K-Nearest Neighbor Algorithms 97Understanding how the k-nearest neighbor algorithm works 98Knowing when to use the k-nearest neighbor algorithm 99Exploring common applications of k-nearest neighbour algorithms 100Solving Real-World Problems with Nearest Neighbor Algorithms 100Seeing k-nearest neighbor algorithms in action 101Seeing average nearest neighbor algorithms in action 101CHAPTER 6: CODING UP DATA INSIGHTS AND DECISION ENGINES 103Seeing Where Python and R Fit into Your Data Science Strategy 104Using Python for Data Science 104Sorting out the various Python data types 106Putting loops to good use in Python 109Having fun with functions 110Keeping cool with classes 112Checking out some useful Python libraries 114Using Open Source R for Data Science 120Comprehending R’s basic vocabulary 121Delving into functions and operators 124Iterating in R 127Observing how objects work 129Sorting out R’s popular statistical analysis packages 131Examining packages for visualizing, mapping, and graphing in R 133CHAPTER 7: GENERATING INSIGHTS WITH SOFTWARE APPLICATIONS 137Choosing the Best Tools for Your Data Science Strategy 138Getting a Handle on SQL and Relational Databases 139Investing Some Effort into Database Design 144Defining data types 144Designing constraints properly 145Normalizing your database 145Narrowing the Focus with SQL Functions 147Making Life Easier with Excel 151Using Excel to quickly get to know your data 152Reformatting and summarizing with PivotTables 157Automating Excel tasks with macros 158CHAPTER 8: TELLING POWERFUL STORIES WITH DATA 161Data Visualizations: The Big Three 162Data storytelling for decision makers 162Data showcasing for analysts 163Designing data art for activists 164Designing to Meet the Needs of Your Target Audience 164Step 1: Brainstorm (All about Eve) 165Step 2: Define the purpose 166Step 3: Choose the most functional visualization type for your purpose 166Picking the Most Appropriate Design Style 167Inducing a calculating, exacting response 167Eliciting a strong emotional response 168Selecting the Appropriate Data Graphic Type 170Standard chart graphics 171Comparative graphics 173Statistical plots 176Topology structures 179Spatial plots and maps 180Testing Data Graphics 183Adding Context 184Creating context with data 184Creating context with annotations 185Creating context with graphical elements 186PART 3: TAKING STOCK OF YOUR DATA SCIENCE CAPABILITIES 187CHAPTER 9: DEVELOPING YOUR BUSINESS ACUMEN 189Bridging the Business Gap 189Contrasting business acumen with subject matter expertise 190Defining business acumen 191Traversing the Business Landscape 192Seeing how data roles support the business in making money 192Leveling up your business acumen 195Fortifying your leadership skills 196Surveying Use Cases and Case Studies 197Documentation for data leaders 199Documentation for data implementers 202CHAPTER 10: IMPROVING OPERATIONS 205Establishing Essential Context for Operational Improvements Use Cases 206Exploring Ways That Data Science Is Used to Improve Operations 207Making major improvements to traditional manufacturing operations 208Optimizing business operations with data science 210An AI case study: Automated, personalized, and effective debt collection processes 211Gaining logistical efficiencies with better use of real-time data 216Another AI case study: Real-time optimized logistics routing 217Modernizing media and the press with data science and AI 222Generating content with the click of a button 222Yet another case study: Increasing content generation rates 224CHAPTER 11: MAKING MARKETING IMPROVEMENTS 229Exploring Popular Use Cases for Data Science in Marketing 229Turning Web Analytics into Dollars and Sense 232Getting acquainted with omnichannel analytics 233Mapping your channels 233Building analytics around channel performance 235Scoring your company’s channels 235Building Data Products That Increase Sales-and-Marketing ROI 238Increasing Profit Margins with Marketing Mix Modeling 239Collecting data on the four Ps 240Implementing marketing mix modeling 241Increasing profitability with MMM 243CHAPTER 12: ENABLING IMPROVED DECISION-MAKING 245Improving Decision-Making 245Barking Up the Business Intelligence Tree 247Using Data Analytics to Support Decision-Making 249Types of analytics 252Common challenges in analytics 252Data wrangling 253Increasing Profit Margins with Data Science 254Seeing which kinds of data are useful when using data science for decision support 255Directing improved decision-making for call center agents 257Discovering the tipping point where the old way stops working 262CHAPTER 13: DECREASING LENDING RISK AND FIGHTING FINANCIAL CRIMES 265Decreasing Lending Risk with Clustering and Classification 266Preventing Fraud Via Natural Language Processing (NLP) 267CHAPTER 14: MONETIZING DATA AND DATA SCIENCE EXPERTISE 275Setting the Tone for Data Monetization 275Monetizing Data Science Skills as a Service 278Data preparation services 279Model building services 280Selling Data Products 282Direct Monetization of Data Resources 283Coupling data resources with a service and selling it 283Making money with data partnerships 284Pricing Out Data Privacy 285PART 4: ASSESSING YOUR DATA SCIENCE OPTIONS 289CHAPTER 15: GATHERING IMPORTANT INFORMATION ABOUT YOUR COMPANY 291Unifying Your Data Science Team Under a Single Business Vision 292Framing Data Science around the Company’s Vision, Mission, and Values 294Taking Stock of Data Technologies 296Inventorying Your Company’s Data Resources 298Requesting your data dictionary and inventory 298Confirming what’s officially on file 300Unearthing data silos and data quality issues 300People-Mapping 303Requesting organizational charts 303Surveying the skillsets of relevant personnel 304Avoiding Classic Data Science Project Pitfalls 305Staying focused on the business, not on the tech 305Drafting best practices to protect your data science project 306Tuning In to Your Company’s Data Ethos 306Collecting the official data privacy policy 307Taking AI ethics into account 307Making Information-Gathering Efficient 308CHAPTER 16: NARROWING IN ON THE OPTIMAL DATA SCIENCE USE CASE 311Reviewing the Documentation 312Selecting Your Quick-Win Data Science Use Cases 313Zeroing in on the quick win 313Producing a POTI model 314Picking between Plug-and-Play Assessments 316Carrying out a data skill gap analysis for your company 317Assessing the ethics of your company’s AI projects and products 318Assessing data governance and data privacy policies 323CHAPTER 17: PLANNING FOR FUTURE DATA SCIENCE PROJECT SUCCESS 327Preparing an Implementation Plan 328Supporting Your Data Science Project Plan 335Analyzing your alternatives 335Interviewing intended users and designing accordingly 337POTI modeling the future state 338Executing On Your Data Science Project Plan 339CHAPTER 18: BLAZING A PATH TO DATA SCIENCE CAREER SUCCESS 341Navigating the Data Science Career Matrix 341Landing Your Data Scientist Dream Job 343Leaning into data science implementation 345Acing your accreditations 346Making the grade with coding bootcamps and data science career accelerators 348Networking and building authentic relationships 349Developing your own thought leadership in data science 350Building a public data science project portfolio 351Leading with Data Science 354Starting Up in Data Science 357Choosing a business model for your data science business 357Selecting a data science start-up revenue model 359Taking inspiration from Kam Lee’s success story 361Following in the footsteps of the data science entrepreneurs 364PART 5: THE PART OF TENS 367CHAPTER 19: TEN PHENOMENAL RESOURCES FOR OPEN DATA 369Digging Through data.gov 370Checking Out Canada Open Data 371Diving into data.gov.uk 372Checking Out US Census Bureau Data 373Accessing NASA Data 374Wrangling World Bank Data 375Getting to Know Knoema Data 376Queuing Up with Quandl Data 378Exploring Exversion Data 379Mapping OpenStreetMap Spatial Data 380CHAPTER 20: TEN FREE OR LOW-COST DATA SCIENCE TOOLS AND APPLICATIONS 381Scraping, Collecting, and Handling Data Tools 382Sourcing and aggregating image data with ImageQuilts 382Wrangling data with DataWrangler 383Data-Exploration Tools 384Getting up to speed in Gephi 384Machine learning with the WEKA suite 386Designing Data Visualizations 387Getting Shiny by RStudio 387Mapmaking and spatial data analytics with CARTO 388Talking about Tableau Public 390Using RAWGraphs for web-based data visualization 392Communicating with Infographics 393Making cool infographics with Infogram 394Making cool infographics with Piktochart 395Index 397
Data Modeling with SAP BW/4HANA 2.0
Gain practical guidance for implementing data models on the SAP BW/4HANA platform using modern modeling concepts. You will walk through the various modeling scenarios such as exposing HANA tables and views through BW/4HANA, creating virtual and hybrid data models, and integrating SAP and non-SAP data into a single data model.Data Modeling with SAP BW/4HANA 2.0 gives you the skills you need to use the new SAP BW/HANA features and objects, covers modern modelling concepts, and equips you with the practical knowledge of how to use the best of the HANA and BW/4HANA worlds.WHAT YOU WILL LEARN* Discover the new modeling features in SAP BW/4HANA* Combine SAP HANA and SAP BW/4HANA artifacts* Leverage virtualization when designing and building data models* Build hybrid data models combining InfoObject, OpenODS, and a field-based approach* Integrate SAP and non-SAP data into single modelWHO THIS BOOK IS FORBI consultants, architects, developers, and analysts working in the SAP BW/4HANA environment.Konrad Załęski graduated from Warsaw University of Technology getting his master’s degree in management. After finishing his studies, he continued his education and completed two postgraduate courses in management information systems (Microsoft) and integrated information systems (SAP).Konrad gained his experience by delivering business intelligence solutions for multiple global corporations. The majority of the projects from his portfolio were implemented on top of the SAP HANA, SAP BW/4HANA, and Microsoft BI platforms. He is also an active contributor and publisher in the SAP community.CHAPTER 1: Modeling objectsSUBTOPIC:1.1. Modern data modeling1.1.1. Overview of SAP BW4HANA modeling concepts1.2. Main SAP HANA Objects used for modelling1.2.1. Calculation View1.2.2. Table Function1.3. Main SAP BW4HANA Objects used for modelling1.3.1. InfoObject1.3.2. Open ODS1.3.3. Advanced DSO1.3.4. Composite Provider1.3.5. QueryCHAPTER 2: PUBLISHING EXTERNAL DATA IN BW4HANASUBTOPIC:2.1. Scenario2.2. Smart Data Acess (SDA)2.3. Creating virtual tables2.4. Publish HANA tables through BW4HANA2.5. Publish HANA views through BW4HANA2.6. Pass input parameters from Query to Calculation View2.7. Pass input parameters from Query to Table Function2.8. SummaryCHAPTER 3: CREATING VIRTUAL BW4HANA MODELSUBTOPIC:3.1. Scenario3.2. Create Master Data OpenODS views3.3. Create Transactional Data OpenODS views3.4. Create Composite Provider consuming OpenODS views3.5. Create Query3.6. SummaryCHAPTER 4. CONVERTING VIRTUAL STRUCTURES INTO PERSISTENT BW4HANA MODELSUBTOPIC:4.1. Scenario4.2. Materialize data model based on OpenODS4.3. SummaryCHAPTER 5: CREATING HYBRID DATA MODEL IN BW4HANASUBTOPIC:5.1. Scenario5.2. Create data model combining OpenODS, InfoObjects and raw fields5.3. SummaryCHAPTER 6: COMBINING SAP AND NON-SAP DATA INTO SINGLE DATA MODELSUBTOPIC:6.1. Scenario6.2. Create view combining SAP and non-SAP data6.3. Create OpenODS views for whole data set6.4. Create Composite Provider6.5. Join additional objects in Composite Provider6.6. Create calculated fields on Composite Provider6.7. Summary
Hands-on Azure Functions with C#
Build serverless solutions using Azure Functions. This book provides you with a deep understanding of Azure Functions so you can build highly scalable and reliable serverless applications.The book starts with an introduction to Azure Functions and demonstrates triggers and bindings with use cases. The process to build an OTP mailer with Queue Storage Trigger and SendGrid output binding is presented, and timer triggers and blob storage binding are covered. Creating custom binding for Azure Functions and building a serverless API using Azure Functions and Azure SQL are discussed. You will know how to build a serverless API using Azure Functions and Azure Cosmos DB, and you will go over enabling application insights and Azure Monitor. Storing function secrets in Azure Key Vault is discussed as well as authentication and authorization using Azure Active Directory. You will learn how to secure your serverless apps using API Management and deploy your Azure Functions using IDEs.Deploying your Azure Functions using CI/CD pipelines is demonstrated along with running Azure Functions in containers. You will learn how to leverage Azure Cognitive Services to build intelligent serverless apps. And the authors introduce you to Azure Durable functions and teach you how to integrate Azure Functions in the logic app workflow. They also discuss best practices and pitfalls to avoid while designing Azure Functions.After reading this book, you will be able to design and deploy Azure Functions and implement solutions to real-world business problems through serverless applications.WHAT WILL YOU LEARN* Monitor and secure Azure Functions* Build and deploy Azure Functions* Enable continuous integration/continuous deployment (CI/CD) DevOps strategies for Azure Functions* Run Azure Functions on Azure Kubernetes ClusterWHO THIS BOOK IS FORExperienced developers, cloud architects, and tech enthusiasts in AzureASHIRWAD SATAPATHI is working as a software developer with a leading IT firm and has expertise in building scalable applications with .NET Core. He has a deep understanding of building full-stack applications using .NET and Azure PaaS and serverless offerings. He is an active blogger in the C# Corner developer community. He was awarded the C# Corner MVP (September 2020) for his remarkable contributions to the developer community.ABHISHEK MISHRA is an architect with a leading software multinational company and has deep expertise in designing and building enterprise-grade Intelligent Azure and .NET-based architectures. He is an expert in .NET full stack, Azure (PaaS, IaaS, serverless), Infrastructure as Code, Azure Machine Learning, Intelligent Azure (Azure Bot Services and Cognitive Services), and Robotics Process Automation. He has a rich 15+ years of experience working across top organizations in the industry. He loves blogging and is an active blogger in the C# Corner developer community. He was awarded the C# Corner MVP (December 2018 and 2019) for his contributions to the developer community.CHAPTER 1: INTRODUCTION TO AZURE FUNCTIONSCHAPTER GOAL: INTRODUCTION TO AZURE FUNCTIONS AND TOPICS COVERED IN THE BOOKNO OF PAGES 6SUB -TOPICS1. Introduction to Azure Functions2. What is serverless ?3. Azure Web Job Vs Azure Functions4. Advantage and Disadvantages5. Hosting Plans for Azure Functions6. Use cases for Azure Functions7. SummaryCHAPTER 2: BUILD YOUR FIRST AZURE FUNCTIONSCHAPTER GOAL: TO SETUP THE ENVIRONMENT AND BUILD YOUR FIRST AZURE FUNCTIONS USING VARIOUS TOOLINGNO OF PAGES: 25SUB - TOPICS1. Building Azure function using Azure Portal2. Installation guide to setup the environment to run Azure function using Azure Functions Runtime Tool3. Build an Azure Functions using Azure Functions Runtime Tool4. Installation guide to setup the environment to run Azure Function in VSCode5. Build an Azure Functions using VSCode and debugging it6. Installation guide to setup the environment to run Azure Functions in Visual Studio 2019 community edition7. Build an Azure Function using Visual Studio 2019 and debugging it locally.8. SummaryCHAPTER 3: WHAT ARE TRIGGERS AND BINDINGS?CHAPTER GOAL: TO INTRODUCE THE CONCEPTS OF TRIGGERS AND BINDINGS AND THEIR USE CASESNO OF PAGES : 15SUB - TOPICS:1. What are Triggers and Bindings ?2. Different triggers and bindings available3. Use cases of some of the Triggers and Bindings4. Build a simple function using a Trigger and Binding using Azure Portal5. SummaryCHAPTER 4: BUILD AN OTP MAILER WITH QUEUE STORAGE TRIGGER AND SENDGRID OUTPUT BINDINGCHAPTER GOAL: TO INTRODUCE THE CONCEPTS QUEUE STORAGE TRIGGER AND SENDGRID OUTPUT BINDING AND THEIR USE CASES BY BUILDING A PROJECTNO OF PAGES: 20SUB - TOPICS:1. Getting Started with Queue Storage Trigger and Use Cases2. Build a sample application with Queue Storage Trigger3. Getting Started with SendGrid output binding and Use Cases4. Build a sample application with SendGrid output binding5. Create a OTP mailer with Queue Storage Trigger and SendGrid output binding6. SummaryCHAPTER 5: BUILD A REPORT GENERATOR WITH TIMER TRIGGER AND BLOB STORAGE BINDINGCHAPTER GOAL: TO INTRODUCE THE CONCEPTS TIMER TRIGGER AND BLOB STORAGE BINDING AND THEIR USE CASES BY BUILDING A PROJECTNO OF PAGES: 20SUB - TOPICS:1. Getting started with Timer Trigger and its use cases2. Build a sample application with Timer trigger3. Getting Started with Blob Storage Binding and its use cases4. Build a sample application with Blob Storage Binding5. Create a Report Generator application with timer trigger and blob storage binding6. SummaryCHAPTER 6: BUILD A TO-DO API WITH HTTP TRIGGER AND TABLE STORAGE BINDINGCHAPTER GOAL: TO INTRODUCE THE CONCEPTS HTTP TRIGGER AND TABLE STORAGE BINDING AND THEIR USE CASES BY BUILDING A PROJECTNO OF PAGES: 35SUB - TOPICS:1. Getting started with HTTP Trigger and its use cases2. Routing in HTTP Triggered Azure Functions3. Build a sample application with HTTP trigger4. Getting Started with Table Storage Binding and its use cases5. Build a sample application with Azure Table Storage Binding6. Create a To-Do API with HTTP trigger and Table Storage binding7. SummaryCHAPTER 7: CREATING CUSTOM BINDING FOR AZURE FUNCTIONCHAPTER GOAL: TO INTRODUCE THE CONCEPTS OF DEPENDENCY INJECTION AND CREATING A CUSTOM BINDING FOR A EXTERNAL SERVICE FOR YOU AZURE FUNCTIONNO OF PAGES: 20SUB - TOPICS:1. Introduction to custom binding2. Use cases for custom binding3. What is dependency injection4. Build a custom binding for Azure Functions5. SummaryCHAPTER 8: BUILDING A SERVERLESS API USING AZURE FUNCTIONS AND AZURE SQLCHAPTER GOAL: UNDERSTAND THE CONCEPTS AND WAYS TO CREATE AN AZURE FUNCTIONS TO CONNECT WITH A AZURE SQL DATABASE AND PERFORM CRUD OPERATIONSNo of pages: 40SUB - TOPICS:1. What is a Serverless API ?2. What is Azure SQL ?3. Create an Azure SQL DB instance and create a table4. Build a HTTP Triggered Azure Functions to perform CRUD operation on the Azure SQL DB using ADO.NET5. SummaryCHAPTER 9: BUILDING A SERVERLESS API USING AZURE FUNCTIONS AND AZURE COSMOS DBCHAPTER GOAL: UNDERSTAND THE CONCEPTS AND WAYS TO CREATE AN AZURE FUNCTIONS TO CONNECT WITH A AZURE COSMOS DB USING BINDINGS AND PERFORM CRUD OPERATIONS AND LEVERAGE THE COSMOS SDK TO INTERACT WITH COSMOS DBNo of pages: 40SUB - TOPICS:1. What is Azure Cosmos DB and its use cases2. Getting started with Azure Cosmos DB Triggers by building a sample application3. Getting started with Azure Cosmos DB Triggers by building a sample application4. Build a HTTP Triggered Azure Function to perform CRUD operation on the Azure Cosmos DB using ADO.NET5. Leverage the Azure Cosmos DB SDK to interact with Cosmos DB from Azure Function6. SummaryCHAPTER 10 : ENABLING APPLICATION INSIGHTS AND AZURE MONITORChapter Goal: Understanding the way to gather telemetry data from your Azure Function to analyze and monitor them.NO OF PAGES: 20SUB - TOPICS:1. Gather and process telemetry data from application insights2. Perform Diagnostics for Azure Functions3. Analyze trends using Azure Monitor and create alerts4. Restrict the number of scaling instances for function app5. SummaryCHAPTER 11: STORING FUNCTION SECRET IN AZURE KEY VAULTCHAPTER GOAL: INTRODUCING A SAFER WAY TO STORE APP SECRETS OF YOUR AZURE FUNCTIONSNO OF PAGES: 20SUB - TOPICS:1. What is Key Vault ?2. Creating a Key Vault in Azure Portal3. Storing keys and secret data in Key Vault4. Fetch app secrets from Azure Key Vault in your azure Function5. SummaryCHAPTER 12: AUTHENTICATION AND AUTHORIZATION USING AZURE ACTIVE DIRECTORYCHAPTER GOAL: A COMPREHENSIVE GUIDE TO ENABLING AAD BASED AUTHENTICATION FOR AZURE FUNCTIONNo of pages: 20SUB - TOPICS:1. Getting started with Azure Active Directory2. What is Authentication and Authorization ?3. Implement Authentication and Authorization for your azure Function using AAD4. SummaryCHAPTER 13: SECURING AZURE FUNCTIONS WITH API MANAGEMENTCHAPTER GOAL: TO UNDERSTAND, HOW TO SECURE YOUR SERVERLESS APIS USING API MANAGEMENTNo of pages: 20SUB - TOPICS:1. What is API Management2. Advantage and Use Cases3. Configure API Management for our Functions4. Demo5. SummaryCHAPTER 14: DEPLOYING YOUR AZURE FUNCTIONS USING IDESCHAPTER GOAL: TO HAVE A UNDERSTANDING ON CREATING RESOURCES IN AZURE TO DEPLOY AZURE FUNCTIONS USING VISUAL STUDIO AND VSCODENo of pages: 20SUB - TOPICS:1. How to deploy your Azure Function using Visual Studio 20192. How to deploy your Azure Function in a Deployment Slot using Visual Studio 20193. How to deploy your Azure Function using VSCode4. SummaryCHAPTER 15: DEPLOYING YOUR AZURE FUNCTIONS USING CI/CD PIPELINES USING AZURE DEVOPSChapter Goal: To understand, how to leverage Azure Devops to give deployments using CI/CD pipelines for your Azure FunctionsNO OF PAGES: 30SUB - TOPICS:1. Introduction to Azure Devops2. Creating a Repository for your Azure Function3. Building a build pipeline for Azure Function and enable CI4. Building a release pipeline for Azure Function and enable CD5. SummaryCHAPTER 16: RUNNING AZURE FUNCTIONS IN CONTAINERSCHAPTER GOAL: RUNNING AZURE FUNCTIONS ON AKSNO OF PAGES: 20SUB - TOPICS:1. Getting started with Containers and AKS2. What is Serverless AKS and KEDA ?3. Deploying your Azure Functions to AKS using KEDA4. Deploying your Azure Function to ACI5. SummaryCHAPTER 17: ADDING COGNITIVE CAPABILITIES TO YOUR AZURE FUNCTIONSCHAPTER GOAL: TO UNDERSTAND, HOW TO LEVERAGE AZURE COGNITIVE SERVICE TO BUILD INTELLIGENT SERVERLESS APPSNO OF PAGES: 30SUB - TOPICS:1. Getting started with Azure Cognitive Services2. Build a severless application to analyze feedbacks using sentiment analysis3. Build a serverless application to classify images using azure vision api4. SummaryCHAPTER 18: INTRODUCTION TO AZURE DURABLE FUNCTIONSCHAPTER GOAL: TO GIVE A BASIC UNDERSTANDING TO THE READER ON BUILDING STATEFUL FUNCTIONSNo of pages: 20SUB - TOPICS:1. Introduction to Azure Durable Functions and use cases2. Advantages and Disadvantages3. Application Patterns4. Build a sample application to demonstrate the capabilities of Azure Durable functions5. SummaryCHAPTER 19: INTEGRATING AZURE FUNCTIONS IN LOGIC APPS WORKFLOWCHAPTER GOAL: TO UNDERSTAND WAYS TO INTEGRATE AZURE FUNCTION IN A LOGIC APP WORKFLOWNO OF PAGES: 20SUB - TOPICS:1. Getting started with Azure Logic Apps2. Build a Serverless application integrating Azure Function in Logic App workflow3. SummaryCHAPTER 20: BEST PRACTICES AND PITFALLS TO AVOIDCHAPTER GOAL: DESIGNING AZURE FUNCTIONS IN A EFFICIENT WAYNO OF PAGES: 15SUB - TOPICS:1. Design Guidelines and Best Practices2. Pitfalls to avoid
Creating Wordpress Online Store and Wordpress Online Magazine
The objective of this work is to develop a Word Press Online Store with Different Ecommerce Plugins and Themes and Word Press Online Magazine with MH Magazine ThemeThe work consists of three parts:i. Part I: Building Personal Websie with online shop the sell Ebooks:The objective of this part is to develop a Ecommerce word press website with all commonly used Plug-ins.First I registered in some free webhost my domain http://hidaia-alassouli.000space.comThen I created the database and installed the word press package.I installed after that all important Plugins for my website. I tested different ecommerce plugin to sell ebooks .The report includes:1- Changing the wordpress theme.2- Creating the frontpage post and the other pages.3- Adding Gallery Plugin.4- Adding yoast.5- Submission the Site to Search Engine and Analyze your Website6- Adding Contact Form Plugin7- Using easyfiledownloads Plugin to sell ebooks8- Using WP-Ecommerce Plugin to sell ebooks9- Using WP Shopping Cart Plugin10- Using Woocommerce Plugin to sell my EbooksI ended up to build my ecommerce shop with woocommerce as it was the most efficient and comfortable.ii. Part II: Building Ecomerce website with mystile theme and woocommerce pluginThe objective of this part is to develop a Ecommerce website with mystyle theme and woocommerce plugin and other commonly used Plug-ins.First I registered in some free webhost my domain http://hedaya-alasooly.000space.comThen I created the database and installed the word press package.I installed after that all important Plugins for my website. The second part of report includes:1- Installing mystyle theme.2- Installing woocommerce plugin Plugin.3- Adding yoast seo Plugin.4- Submission the Site to Search Engine and Analyze your Websiteiii. Part III: Building Online magazine website with MH-Magazine themeThe objective of this part is to develop an online magazine website with MH Magazine theme and other commonly used Plug-ins.First I registered in some free webhost my domain http://anticorruption.000space.com.Then I created the database and installed the word press package.I installed after that all important Plugins for my website. The third part of report includes:1- Installing MH Magazine theme.2- Configuring MH Magazine theme.3- Adding yoast seo Plugin.4- Submission the Site to Search Engine and Analyze your WebsiteI am Dr. Hidaia Mahmoud Mohamed Alassouli. I completed my PhD degree in Electrical Engineering from Czech Technical University by February 2003, and my M. Sc. degree in Electrical Engineering from Bahrain University by June 1995. I completed also one study year of most important courses in telecommunication and computer engineering courses in Islamic university in Gaza. So, I covered most important subjects in Electrical Engineering, Computer Engineering and Telecommunications Engineering during my study. My nationality is Palestinian from gaza strip.I obtained a lot of certified courses in MCSE, SPSS, Cisco (CCNA), A+, Linux.I worked as Electrical, Telecommunicating and Computer Engineer in a lot of institutions. I worked also as a computer networking administrator.I had considerable undergraduate teaching experience in several types of courses in many universities. I handled teaching the most important subjects in Electrical and Telecommunication and Computer Engineering.I could publish a lot of papers a top-tier journals and conference proceedings, besides I published a lot of books in Publishing and Distribution houses.I wrote a lot of important Arabic articles on online news websites. I also have my own magazine website that I publish on it all my articles: http:// www.anticorruption.000space.comMy personal website: www.hidaia-alassouli.000space.comEmail: hidaia_alassouli@hotmail.com
8 Steps to Better Security
HARDEN YOUR BUSINESS AGAINST INTERNAL AND EXTERNAL CYBERSECURITY THREATS WITH A SINGLE ACCESSIBLE RESOURCE.In 8 Steps to Better Security: A Simple Cyber Resilience Guide for Business, cybersecurity researcher and writer Kim Crawley delivers a grounded and practical roadmap to cyber resilience in any organization. Offering you the lessons she learned while working for major tech companies like Sophos, AT&T, BlackBerry Cylance, Tripwire, and Venafi, Crawley condenses the essence of business cybersecurity into eight steps.Written to be accessible to non-technical businesspeople as well as security professionals, and with insights from other security industry leaders, this important book will walk you through how to:* Foster a strong security culture that extends from the custodial team to the C-suite* Build an effective security team, regardless of the size or nature of your business* Comply with regulatory requirements, including general data privacy rules and industry-specific legislation* Test your cybersecurity, including third-party penetration testing and internal red team specialistsPerfect for CISOs, security leaders, non-technical businesspeople, and managers at any level, 8 Steps to Better Security is also a must-have resource for companies of all sizes, and in all industries.KIM CRAWLEY focuses on researching and writing about cybersecurity issues. Her career has included work with Sophos, AT&T Cybersecurity, BlackBerry Cylance, Tripwire, and Venafi. She specializes in all matters red team, blue team, and purple team and is especially fascinated by malware, social engineering, and advanced persistent threats. She runs an online cybersecurity event called DisInfoSec.Foreword xiIntroduction xiiiCHAPTER 1: STEP 1: FOSTER A STRONG SECURITY CULTURE 1Kevin Mitnick, Human Hacker Extraordinaire 3The Importance of a Strong Security Culture 5Hackers Are the Bad Guys, Right? 6What is Security Culture? 7How to Foster a Strong Security Culture 9Security Leaders on Security Culture 12What Makes a Good CISO? 13The Biggest Mistakes Businesses Make When It Comes to Cybersecurity 14The Psychological Phases of a Cybersecurity Professional 15CHAPTER 2: STEP 2: BUILD A SECURITY TEAM 19Why Step 2 is Controversial 20How to Hire the Right Security Team. . .the Right Way 28Security Team Tips from Security Leaders 29The “Culture Fit”—Yuck! 30Cybersecurity Budgets 34Design Your Perfect Security Team 35CHAPTER 3: STEP 3: REGULATORY COMPLIANCE 39What Are Data Breaches, and Why Are They Bad? 40The Scary Truth Found in Data Breach Research 45An Introduction to Common Data Privacy Regulations 49The General Data Protection Regulation 49The California Consumer Privacy Act 50The Health Insurance Portability and Accountability Act 52The Gramm-Leach-Bliley Act 52Payment Card Industry Data Security Standard 53Governance, Risk Management, and Compliance 53More About Risk Management 54Threat Modeling 55CHAPTER 4: STEP 4: FREQUENT SECURITY TESTING 57What is Security Testing? 58Security Testing Types 58Security Audits 58Vulnerability Assessments Versus Penetration Testing 59Red Team Testing 61Bug Bounty Programs 61What’s Security Maturity? 63The Basics of Security Audits and Vulnerability Assessments 64Log Early, Log Often 66Prepare for Vulnerability Assessments and Security Audits 67A Concise Guide to Penetration Testing 69Penetration Testing Based on Network Knowledge 70Penetration Testing Based on Network Aspects 73Security Leaders on Security Maturity 76Security Testing is Crucial 78CHAPTER 5: STEP 5: SECURITY FRAMEWORK APPLICATION 79What is Incident Response? 80Preparation 80Identification or Analysis 82Containment, Mitigation, or Eradication 83Recovery 84Post-incident 86Your Computer Security Incident Response Team 86Cybersecurity Frameworks 89NIST Cybersecurity Framework 89Identify 90Protect 92Detect 95Respond 97Recover 99ISO 27000 Cybersecurity Frameworks 101CIS Controls 102COBIT Cybersecurity Framework 105Security Frameworks and Cloud Security 106CHAPTER 6: STEP 6: CONTROL YOUR DATA ASSETS 109The CIA Triad 110Access Control 112Patch Management 113Physical Security and Your Data 115Malware 116Cryptography Basics 119Bring Your Own Device and Working from Home 123Data Loss Prevention 124Managed Service Providers 126The Dark Web and Your Data 128Security Leaders on Cyber Defense 130Control Your Data 132CHAPTER 7: STEP 7: UNDERSTAND THE HUMAN FACTOR 133Social Engineering 134Phishing 139What Can NFTs and ABA Teach Us About Social Engineering? 141How to Prevent Social Engineering Attacks on Your Business 146UI and UX Design 147Internal Threats 148Hacktivism 152CHAPTER 8: STEP 8: BUILD REDUNDANCY AND RESILIENCE 155Understanding Data and Networks 156Building Capacity and Scalability with the Power of the Cloud 158Back It Up, Back It Up, Back It Up 161RAID 162What Ransomware Taught Business About Backups 164Business Continuity 167Disaster Recovery 168CHAPTER 9: AFTERWORD 173STEP 1 173The Most Notorious Cyberattacker Was Actually a Con Man 174A Strong Security Culture Requires All Hands on Deck 174Hackers Are the Good Guys, Actually 174What Is Security Culture? 175What Makes a Good CISO? 175The Psychological Phases of a Cybersecurity Professional 176Recommended Readings 177STEP 2 178Tackling the Cybersecurity Skills Gap Myth 178Take “Culture Fit” Out of Your Vocabulary 179Your Cybersecurity Budget 180Recommended Readings 180STEP 3 181Data Breaches 181Data Privacy Regulations 182Risk Management 183Recommended Readings 183STEP 4 184Security Audits 184Vulnerability Assessments 185Penetration Testing 185Bug Bounty Programs 185Recommended Reading 186STEP 5 187Incident Response 187Cybersecurity Frameworks 187Recommended Reading 188STEP 6 188The CIA Triad 188Access Control 189Patch Management 189Physical Security 189Malware 189Cryptography 190BYOD and Working from Home 190Data Loss Prevention 191Managed Service Providers 191Recommended Reading 191STEP 7 192Social Engineering 192UI and UX Design 193Internal Threats 193Recommended Readings 194STEP 8 194Cloud Networks 195Data Backups 195Business Continuity and Disaster Recovery 196Recommended Readings 196Keeping Your Business Cyber Secure 197Index 199
CompTIA Cloud+ Study Guide
In the newly revised Third Edition of CompTIA Cloud+ Study Guide: Exam CVO-003, expert IT Ben Piper delivers an industry leading resource for anyone preparing for the CompTIA Cloud+ certification and a career in cloud services. The book introduces candidates to the skills and the competencies critical for success in the field and on the exam.The book breaks down challenging cloud management concepts into intuitive and manageable topics, including cloud architecture and design, cloud security, deployment, operations and support, and cloud troubleshooting. It also offers practical study features, like Exam Essentials and challenging chapter review questions.Written in a concise and straightforward style that will be immediately familiar to the hundreds of thousands of readers who have successfully use other CompTIA study guides to further their careers in IT, the book offers:* Efficient and effective training for a powerful certification that opens new and lucrative career opportunities* Fully updated coverage for the new Cloud+ CV0-003 Exam that includes the latest in cloud architecture and design* Access to the Sybex online learning center, with chapter review questions, full-length practice exams, hundreds of electronic flashcards, and a glossary of key termsPerfect for everyone preparing for the CompTIA Cloud+ Exam CV0-003 certification, this book is an ideal resource for current and aspiring cloud services professionals seeking an efficient and up-to-date resource that will dramatically improve their ability to maintain, secure, and optimize cloud environments.ABOUT THE AUTHORBEN PIPER, is an IT consultant who has created more than 30 educational courses covering cloud technologies, networking, and automation. Learn more about Ben from his website at www.benpiper.com.Introduction xxiiiAssessment Test IviAnswers to Assessment Test lxxviiCHAPTER 1 INTRODUCING CLOUD COMPUTING CONFIGURATIONS AND DEPLOYMENTS 1Introducing Cloud Computing 4Virtualization 7Cloud Service Models 10Cloud Reference Designs and Delivery Models 14Introducing Cloud Concepts and Components 16Connecting the Cloud to the Outside World 18Deciding Whether to Move to the Cloud 18Selecting Cloud Compute Resources 18Hypervisor Affinity Rules 19Validating and Preparing for the Move to the Cloud 19Choosing Elements and Objects in the Cloud 20Internet of Things 21Machine Learning/Artificial Intelligence (AI) 21Creating and Validating a Cloud Deployment 22The Cloud Shared Resource Pooling Model 23Organizational Uses of the Cloud 27Scaling and Architecting Cloud Systems Based on Requirements 29Understanding Cloud Performance 29Delivering High Availability Operations 30Managing and Connecting to Your Cloud Resources 30Is My Data Safe? (Replication and Synchronization) 32Understanding Load Balancers 34Cloud Testing 35Verifying System Requirements 36Correct Scaling for Your Requirements 36Making Sure the Cloud Is Always Available 37Remote Management of VMs 39Monitoring Your Cloud Resources 41Writing It All Down (Documentation) 41Creating Baselines 41Shared Responsibility Model 42Summary 43Exam Essentials 43Written Lab 45Review Questions 46CHAPTER 2 CLOUD DEPLOYMENTS 51Executing a Cloud Deployment 58Understanding Deployment and Change Management 59Cloud Deployment Models 65Network Deployment Considerations 67Service Level Agreements 77Matching Data Center Resources to Cloud Resources 78What Are Available and Proposed Hardware Resources? 78Templates and Images 81Physical Resource High Availability 82Introducing Disaster Recovery 82Physical Hardware Performance Benchmarks 83Cost Savings When Using the Cloud 83Energy Savings in the Cloud 84Shared vs. Dedicated Hardware Resources in a Cloud Data Center 84Microservices 84Configuring and Deploying Storage 86Identifying Storage Configurations 86Storage Provisioning 90Storage Priorities: Understanding Storage Tiers 94Managing and Protecting Your Stored Data 95Storage Security Considerations 102Accessing Your Storage in the Cloud 105Performing a Server Migration 105Different Types of Server Migrations 106Addressing Application Portability 109Workload Migration Common Procedures 110Examining Infrastructure Capable of Supporting a Migration 110Managing User Identities and Roles 111RBAC: Identifying Users and What Their Roles Are 112What Happens When You Authenticate? 113Understanding Federation 113Single Sign-On Systems 113Understanding Infrastructure Services 114Summary 117Exam Essentials 118Written Lab 119Review Questions 120CHAPTER 3 SECURITY IN THE CLOUD 125Cloud Security Compliance and Configurations 128Establishing Your Company’s Security Policies 130Selecting and Applying the Security Policies to Your Cloud Operations 130Some Common Regulatory Requirements 130Encrypting Your Data 134Remote Access Protocols 139Automating Cloud Security 140Security Best Practices 141Access Control 144Accessing Cloud-Based Objects 144Cloud Service Models and Security 146Cloud Deployment Models and Security 147Role-Based Access Control 148Mandatory Access Control 148Discretionary Access Control 148Multifactor Authentication 149Single Sign-On 149Summary 149Exam Essentials 150Written Lab 151Review Questions 153CHAPTER 4 IMPLEMENTING CLOUD SECURITY 157Implementing Security in the Cloud 159Data Classification 159Segmenting Your Deployment 160Implementing Encryption 162Applying Multifactor Authentication 163Regulatory and Compliance Issues During Implementation 164Cloud Access Security Broker 165Automating Cloud Security 165Automation Tools 166Techniques for Implementing Cloud Security 168Security Services 170Summary 173Exam Essentials 174Written Lab 175Review Questions 177CHAPTER 5 MAINTAINING CLOUD OPERATIONS 183Applying Security Patches 187Patching Cloud Resources 187Patching Methodologies 189Patching Order of Operations and Dependencies 193Updating Cloud Elements 193Hotfix 193Patch 194Version Update 194Rollback 195Workflow Automation 195Continuous Integration and Continuous Deployment 196Virtualization Automation Tools and Activities 197Storage Operations 199Types of Backups 199Backup Targets 203Backup and Restore Operations 205Summary 206Exam Essentials 207Written Lab 209Review Questions 210CHAPTER 6 DISASTER RECOVERY, BUSINESS CONTINUITY, AND ONGOING MAINTENANCE 215Implementing a Disaster Recovery and Business Continuity Plan 216Service Provider Responsibilities and Capabilities 217Disaster Recovery Models and Techniques 219Business Continuity 225Establishing a Business Continuity Plan 225Establishing Service Level Agreements 227Cloud Maintenance 228Establishing Maintenance Windows 228Maintenance Interruptions to Operations 229Maintenance Automation Impact and Scope 229Common Maintenance Automation Tasks 229Summary 233Exam Essentials 234Written Lab 235Review Questions 236CHAPTER 7 CLOUD MANAGEMENT 241Cloud Metrics 244Monitoring Your Deployment 246Cloud Support Agreements 250Standard Cloud Maintenance Responsibilities 250Configuration Management Applications and Tools 251Change Management Processes 251Adding and Removing Cloud Resources 252Determining Usage Patterns 252Bursting 252Migrating Between Cloud Providers 252Scaling Resources to Meet Requirements 253Extending the Scope of the Cloud 256Understanding Application Life Cycles 256Organizational Changes 257Managing Account Provisioning 258Account Identification 258Authentication 259Authorization 259Lockout Policies 259Password Complexity 259Account Automation and Orchestration 260Summary 261Exam Essentials 262Written Lab 263Review Questions 264CHAPTER 8 CLOUD MANAGEMENT BASELINES, PERFORMANCE, AND SLAS 269Measuring Your Deployment Against the Baseline 272Object Tracking for Baseline Validation 273Applying Changes to the Cloud to Meet Baseline Requirements 277Changing Operations to Meet Expected Performance/Capacity Requirements 280Cloud Accounting, Chargeback, and Reporting 281Summary 284Exam Essentials 285Written Lab 286Review Questions 287Chapter 9 Troubleshooting 291Incident Management 294Incident Types 294Logging Incidents 298Prioritizing Incidents 298Preparation 299Templates 300Time Synchronization 301Workflow 301Troubleshooting Cloud Capacity Issues 301Capacity Boundaries in the Cloud 301Troubleshooting Automation and Orchestration 304Process and Workflow Issues 305Summary 307Exam Essentials 308Written Lab 309Review Questions 310CHAPTER 10 TROUBLESHOOTING NETWORKING AND SECURITY ISSUES AND UNDERSTANDING METHODOLOGIES 315Troubleshooting Cloud Networking Issues 317Identifying the Common Networking Issues in the Cloud 318Network Troubleshooting and Connectivity Tools 324Remote Access Tools 333Troubleshooting Security Issues 336Account Privilege Escalation 336Network Access Issues 337Authentication 337Authorization 337Federations 338Certificate Configuration Issues 338Device- Hardening Settings 338External Attacks 339Internal Attacks 339Maintain Sufficient Security Controls and Processes 339Network Access Tunneling and Encryption 340Troubleshooting Methodology 340Identifying the Problem 341Establishing a Theory 341Testing the Theory 343Creating and Implementing a Plan of Action 344Verifying the Resolution 344Documenting the Ordeal 344Summary 344Exam Essentials 345Written Lab 346Review Questions 347Index 375
C# Programming for Absolute Beginners
Get started using the C# programming language. Based on the author’s 15 years of experience teaching beginners, this book provides you with a step-by-step introduction to the principles of programming, or rather, how to think like a programmer. The task-solution approach will get you immersed, with minimum theory and maximum action.WHAT YOU WILL LEARN* Understand what programming is all about* Write simple, but non-trivial, programs* Become familiar with basic programming constructs such as statements, types, variables, conditions, and loops* Think like a programmer and combine these programming constructs in new ways* Get to know C# as a modern, mainstream programming language, and Visual Studio as one of the world’s most popular programming toolsWHO THIS BOOK IS FORThose with very little or no experience in computer programming, who know how to use a computer, install a program, and navigate the webRADEK VYSTAVĚL started programming at a young age and has been doing it ever since, as well as sharing it with others. Having worked as both a programming teacher in college and privately, as a programmer, Radek started writing books on programming because many of his students had no previous knowledge of programming and he had difficulty finding a suitable textbook for them. Throughout the years he has gained a unique insight into what resonates when teaching beginners. Chapter 1: Getting ReadyPart I: DataChapter 2: First ProgramChapter 3: OutputsChapter 4: VariablesChapter 5: ObjectsChapter 6: Object and ActionsChapter 7: More on ObjectsPart II. CalculationsChapter 8: InputsChapter 9: NumbersChapter 10: Economic CalculationsChapter 11: Calculations with Dates Chapter 12: Understanding Different Kinds of NumbersChapter 13: Accumulating ValuesPart III: ConditionalsChapter 14: Essential ToolsChapter 15: Starting with ConditionsChapter 16: Practical Conditions Chapter 17: Compound ConditionsChapter 18: Multiple ConditionsChapter 19: Advanced ConditionsPart IV: LoopsChapter 20: First LoopsChapter 21: Improving LoopsChapter 22: Number SeriesChapter 23: Unknown Number of RepetitionsChapter 24: Accumulating Intermediate ResultsChapter 25: Advanced Loops
Machine Learning Algorithms and Applications
MACHINE LEARNING ALGORITHMS is for current and ambitious machine learning specialists looking to implement solutions to real-world machine learning problems. It talks entirely about the various applications of machine and deep learning techniques, with each chapter dealing with a novel approach of machine learning architecture for a specific application, and then compares the results with previous algorithms.The book discusses many methods based in different fields, including statistics, pattern recognition, neural networks, artificial intelligence, sentiment analysis, control, and data mining, in order to present a unified treatment of machine learning problems and solutions. All learning algorithms are explained so that the user can easily move from the equations in the book to a computer program.METTU SRINIVAS PhD from the Indian Institute of Technology Hyderabad, and is currently an assistant professor in the Department of Computer Science and Engineering, NIT Warangal, India.G. SUCHARITHA PhD from KL University, Vijayawada and is currently an assistant professor in the Department of Electronics and Communication Engineering at ICFAI Foundation for Higher Education Hyderabad. ANJANNA MATTA PhD from the Indian Institute of Technology Hyderabad and is currently an assistant professor in the Department of Mathematics at ICFAI Foundation for Higher Education Hyderabad. PRASENJIT CHATTERJEE PhD is an associate professor in the Mechanical Engineering Department at MCKV Institute of Engineering, India. Acknowledgments xvPreface xviiPART 1: MACHINE LEARNING FOR INDUSTRIAL APPLICATIONS 11 A LEARNING-BASED VISUALIZATION APPLICATION FOR AIR QUALITY EVALUATION DURING COVID-19 PANDEMIC IN OPEN DATA CENTRIC SERVICES 3Priyank Jain and Gagandeep Kaur1.1 Introduction 41.1.1 Open Government Data Initiative 41.1.2 Air Quality 41.1.3 Impact of Lockdown on Air Quality 51.2 Literature Survey 51.3 Implementation Details 61.3.1 Proposed Methodology 71.3.2 System Specifications 81.3.3 Algorithms 81.3.4 Control Flow 101.4 Results and Discussions 111.5 Conclusion 21References 212 AUTOMATIC COUNTING AND CLASSIFICATION OF SILKWORM EGGS USING DEEP LEARNING 23Shreedhar Rangappa, Ajay A. and G. S. Rajanna2.1 Introduction 232.2 Conventional Silkworm Egg Detection Approaches 242.3 Proposed Method 252.3.1 Model Architecture 26.3.2 Foreground-Background Segmentation 282.3.3 Egg Location Predictor 302.3.4 Predicting Egg Class 312.4 Dataset Generation 352.5 Results 352.6 Conclusion 37Acknowledgment 38References 383 A WIND SPEED PREDICTION SYSTEM USING DEEP NEURAL NETWORKS 41Jaseena K. U. and Binsu C. Kovoor3.1 Introduction 423.2 Methodology 453.2.1 Deep Neural Networks 453.2.2 The Proposed Method 473.2.2.1 Data Acquisition 473.2.2.2 Data Pre-Processing 483.2.2.3 Model Selection and Training 503.2.2.4 Performance Evaluation 513.2.2.5 Visualization 513.3 Results and Discussions 523.3.1 Selection of Parameters 523.3.2 Comparison of Models 533.4 Conclusion 57References 574 RES-SE-NET: BOOSTING PERFORMANCE OF RESNETS BY ENHANCING BRIDGE CONNECTIONS 61Varshaneya V., S. Balasubramanian and Darshan Gera4.1 Introduction 614.2 Related Work 624.3 Preliminaries 634.3.1 ResNet 634.3.2 Squeeze-and-Excitation Block 644.4 Proposed Model 664.4.1 Effect of Bridge Connections in ResNet 664.4.2 Res-SE-Net: Proposed Architecture 674.5 Experiments 684.5.1 Datasets 684.5.2 Experimental Setup 684.6 Results 694.7 Conclusion 73References 745 HITTING THE SUCCESS NOTES OF DEEP LEARNING 77Sakshi Aggarwal, Navjot Singh and K.K. Mishra5.1 Genesis 785.2 The Big Picture: Artificial Neural Network 795.3 Delineating the Cornerstones 805.3.1 Artificial Neural Network vs. Machine Learning 805.3.2 Machine Learning vs. Deep Learning 815.3.3 Artificial Neural Network vs. Deep Learning 815.4 Deep Learning Architectures 825.4.1 Unsupervised Pre-Trained Networks 825.4.2 Convolutional Neural Networks 835.4.3 Recurrent Neural Networks 845.4.4 Recursive Neural Network 855.5 Why is CNN Preferred for Computer Vision Applications? 855.5.1 Convolutional Layer 865.5.2 Nonlinear Layer 865.5.3 Pooling Layer 875.5.4 Fully Connected Layer 875.6 Unravel Deep Learning in Medical Diagnostic Systems 895.7 Challenges and Future Expectations 945.8 Conclusion 94References 956 TWO-STAGE CREDIT SCORING MODEL BASED ON EVOLUTIONARY FEATURE SELECTION AND ENSEMBLE NEURAL NETWORKS 99Diwakar Tripathi, Damodar Reddy Edla, Annushree Bablani and Venkatanareshbabu Kuppili6.1 Introduction 1006.1.1 Motivation 1006.2 Literature Survey 1016.3 Proposed Model for Credit Scoring 1036.3.1 Stage-1: Feature Selection 1046.3.2 Proposed Criteria Function 1056.3.3 Stage-2: Ensemble Classifier 1066.4 Results and Discussion 1076.4.1 Experimental Datasets and Performance Measures 1076.4.2 Classification Results With Feature Selection 1086.5 Conclusion 112References 1137 ENHANCED BLOCK-BASED FEATURE AGGLOMERATION CLUSTERING FOR VIDEO SUMMARIZATION 117Sreeja M. U. and Binsu C. Kovoor7.1 Introduction 1187.2 Related Works 1197.3 Feature Agglomeration Clustering 1227.4 Proposed Methodology 1227.4.1 Pre-Processing 1237.4.2 Modified Block Clustering Using Feature Agglomeration Technique 1257.4.3 Post-Processing and Summary Generation 1277.5 Results and Analysis 1297.5.1 Experimental Setup and Data Sets Used 1297.5.2 Evaluation Metrics 1307.5.3 Evaluation 1317.6 Conclusion 138References 138PART 2: MACHINE LEARNING FOR HEALTHCARE SYSTEMS 1418 CARDIAC ARRHYTHMIA DETECTION AND CLASSIFICATION FROM ECG SIGNALS USING XGBOOST CLASSIFIER 143Saroj Kumar Pandeyz, Rekh Ram Janghel and Vaibhav Gupta8.1 Introduction 1438.2 Materials and Methods 1458.2.1 MIT-BIH Arrhythmia Database 1468.2.2 Signal Pre-Processing 1478.2.3 Feature Extraction 1478.2.4 Classification 1488.2.4.1 XGBoost Classifier 1488.2.4.2 AdaBoost Classifier 1498.3 Results and Discussion 1498.4 Conclusion 155References 1569 GSA-BASED APPROACH FOR GENE SELECTION FROM MICROARRAY GENE EXPRESSION DATA 159Pintu Kumar Ram and Pratyay Kuila9.1 Introduction 1599.2 Related Works 1619.3 An Overview of Gravitational Search Algorithm 1629.4 Proposed Model 1639.4.1 Pre-Processing 1639.4.2 Proposed GSA-Based Feature Selection 1649.5 Simulation Results 1669.5.1 Biological Analysis 1689.6 Conclusion 172References 172PART 3: MACHINE LEARNING FOR SECURITY SYSTEMS 17510 ON FUSION OF NIR AND VW INFORMATION FOR CROSS-SPECTRAL IRIS MATCHING 177Ritesh Vyas, Tirupathiraju Kanumuri, Gyanendra Sheoran and Pawan Dubey10.1 Introduction 17710.1.1 Related Works 17810.2 Preliminary Details 17910.2.1 Fusion 18110.3 Experiments and Results 18210.3.1 Databases 18210.3.2 Experimental Results 18210.3.2.1 Same Spectral Matchings 18310.3.2.2 Cross Spectral Matchings 18410.3.3 Feature-Level Fusion 18610.3.4 Score-Level Fusion 18910.4 Conclusions 190References 19011 FAKE SOCIAL MEDIA PROFILE DETECTION 193Umita Deepak Joshi, Vanshika, Ajay Pratap Singh, Tushar Rajesh Pahuja, Smita Naval and Gaurav Singal11.1 Introduction 19411.2 Related Work 19511.3 Methodology 19711.3.1 Dataset 19711.3.2 Pre-Processing 19811.3.3 Artificial Neural Network 19911.3.4 Random Forest 20211.3.5 Extreme Gradient Boost 20211.3.6 Long Short-Term Memory 20411.4 Experimental Results 20411.5 Conclusion and Future Work 207Acknowledgment 207References 20712 EXTRACTION OF THE FEATURES OF FINGERPRINTS USING CONVENTIONAL METHODS AND CONVOLUTIONAL NEURAL NETWORKS 211E. M. V. Naga Karthik and Madan Gopal12.1 Introduction 21212.2 Related Work 21312.3 Methods and Materials 21512.3.1 Feature Extraction Using SURF 21512.3.2 Feature Extraction Using Conventional Methods 21612.3.2.1 Local Orientation Estimation 21612.3.2.2 Singular Region Detection 21812.3.3 Proposed CNN Architecture 21912.3.4 Dataset 22112.3.5 Computational Environment 22112.4 Results 22212.4.1 Feature Extraction and Visualization 22312.5 Conclusion 226Acknowledgements 226References 22613 FACIAL EXPRESSION RECOGNITION USING FUSION OF DEEP LEARNING AND MULTIPLE FEATURES 229M. Srinivas, Sanjeev Saurav, Akshay Nayak and Murukessan A. P.13.1 Introduction 23013.2 Related Work 23213.3 Proposed Method 23513.3.1 Convolutional Neural Network 23613.3.1.1 Convolution Layer 23613.3.1.2 Pooling Layer 23713.3.1.3 ReLU Layer 23813.3.1.4 Fully Connected Layer 23813.3.2 Histogram of Gradient 23913.3.3 Facial Landmark Detection 24013.3.4 Support Vector Machine 24113.3.5 Model Merging and Learning 24213.4 Experimental Results 24213.4.1 Datasets 24213.5 Conclusion 245Acknowledgement 245References 245PART 4: MACHINE LEARNING FOR CLASSIFICATION AND INFORMATION RETRIEVAL SYSTEMS 24714 ANIMNET: AN ANIMAL CLASSIFICATION NETWORK USING DEEP LEARNING 249Kanak Manjari, Kriti Singhal, Madhushi Verma and Gaurav Singal14.1 Introduction 24914.1.1 Feature Extraction 25014.1.2 Artificial Neural Network 25014.1.3 Transfer Learning 25114.2 Related Work 25214.3 Proposed Methodology 25414.3.1 Dataset Preparation 25414.3.2 Training the Model 25414.4 Results 25814.4.1 Using Pre-Trained Networks 25914.4.2 Using AnimNet 25914.4.3 Test Analysis 26014.5 Conclusion 263References 26415 A HYBRID APPROACH FOR FEATURE EXTRACTION FROM REVIEWS TO PERFORM SENTIMENT ANALYSIS 267Alok Kumar and Renu Jain15.1 Introduction 26815.2 Related Work 26915.3 The Proposed System 27115.3.1 Feedback Collector 27215.3.2 Feedback Pre-Processor 27215.3.3 Feature Selector 27215.3.4 Feature Validator 27415.3.4.1 Removal of Terms From Tentative List of Features on the Basis of Syntactic Knowledge 27415.3.4.2 Removal of Least Significant Terms on the Basis of Contextual Knowledge 27615.3.4.3 Removal of Less Significant Terms on the Basis of Association With Sentiment Words 27715.3.4.4 Removal of Terms Having Similar Sense 27815.3.4.5 Removal of Terms Having Same Root 27915.3.4.6 Identification of Multi-Term Features 27915.3.4.7 Identification of Less Frequent Feature 27915.3.5 Feature Concluder 28115.4 Result Analysis 28215.5 Conclusion 286References 28616 SPARK-ENHANCED DEEP NEURAL NETWORK FRAMEWORK FOR MEDICAL PHRASE EMBEDDING 289Amol P. Bhopale and Ashish Tiwari16.1 Introduction 29016.2 Related Work 29116.3 Proposed Approach 29216.3.1 Phrase Extraction 29216.3.2 Corpus Annotation 29416.3.3 Phrase Embedding 29416.4 Experimental Setup 29716.4.1 Dataset Preparation 29716.4.2 Parameter Setting 29716.5 Results 29816.5.1 Phrase Extraction 29816.5.2 Phrase Embedding 29816.6 Conclusion 303References 30317 IMAGE ANONYMIZATION USING DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORK 305Ashish Undirwade and Sujit Das17.1 Introduction 30617.2 Background Information 31017.2.1 Black Box and White Box Attacks 31017.2.2 Model Inversion Attack 31117.2.3 Differential Privacy 31217.2.3.1 Definition 31217.2.4 Generative Adversarial Network 31317.2.5 Earth-Mover (EM) Distance/Wasserstein Metric 31617.2.6 Wasserstein GAN 31717.2.7 Improved Wasserstein GAN (WGAN-GP) 31717.2.8 KL Divergence and JS Divergence 31817.2.9 DCGAN 31917.3 Image Anonymization to Prevent Model Inversion Attack 31917.3.1 Algorithm 32117.3.2 Training 32217.3.3 Noise Amplifier 32317.3.4 Dataset 32417.3.5 Model Architecture 32417.3.6 Working 32517.3.7 Privacy Gain 32517.4 Results and Analysis 32617.5 Conclusion 328References 329Index 331
Fuzzy Intelligent Systems
FUZZY INTELLIGENT SYSTEMSA COMPREHENSIVE GUIDE TO EXPERT SYSTEMS AND FUZZY LOGIC THAT IS THE BACKBONE OF ARTIFICIAL INTELLIGENCE.The objective in writing the book is to foster advancements in the field and help disseminate results concerning recent applications and case studies in the areas of fuzzy logic, intelligent systems, and web-based applications among working professionals and those in education and research covering a broad cross section of technical disciplines. Fuzzy Intelligent Systems: Methodologies, Techniques, and Applications comprises state-of-the-art chapters detailing how expert systems are built and how the fuzzy logic resembling human reasoning, powers them. Engineers, both current and future, need systematic training in the analytic theory and rigorous design of fuzzy control systems to keep up with and advance the rapidly evolving field of applied control technologies. As a consequence, expert systems with fuzzy logic capabilities make for a more versatile and innovative handling of problems. This book showcases the combination of fuzzy logic and neural networks known as a neuro-fuzzy system, which results in a hybrid intelligent system by combining a human-like reasoning style of neural networks. AUDIENCEResearchers and students in computer science, Internet of Things, artificial intelligence, machine learning, big data analytics and information and communication technology-related fields. Students will gain a thorough understanding of fuzzy control systems theory by mastering its contents. E. CHANDRESEKARAN, PHD is a Professor of Mathematics at Veltech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Chennai India.R. ANANDAN, PHD is a IBMS/390 Mainframe professional, a Chartered Engineer from the Institution of Engineers in India and received a fellowship from Bose Science Society, India. He is currently a Professor in the Department of Computer Science and Engineering, School of Engineering, Vels Institute of Science, Technology & Advanced Studies (VISTAS), Chennai. G. SUSEENDRAN, PHD was an assistant professor in the Department of Information Technology, School of Computing Sciences, Vels Institute of Science, Technology & Advanced Studies (VISTAS), Chennai and passed away as this book was being prepared. S. BALAMURUGAN, PHD is the Director of Research and Development, Intelligent Research Consultancy Services(iRCS), Coimbatore, Tamilnadu, India. He is also Director of the Albert Einstein Engineering and Research Labs (AEER Labs), as well as Vice-Chairman, Renewable Energy Society of India(RESI), India. HANAA HACHIMI, PHD is an associate professor at the Ibn Tofail University, in the National School of Applied Sciences ENSA in Kenitra, Morocco. She is President of the Moroccan Society of Engineering Sciences and Technology (MSEST). Preface xiii1 FUZZY FRACTALS IN CERVICAL CANCER 1T. Sudha and G. Jayalalitha1.1 Introduction 21.1.1 Fuzzy Mathematics 21.1.1.1 Fuzzy Set 21.1.1.2 Fuzzy Logic 21.1.1.3 Fuzzy Matrix 31.1.2 Fractals 31.1.2.1 Fractal Geometry 41.1.3 Fuzzy Fractals 41.1.4 Cervical Cancer 51.2 Methods 71.2.1 Fuzzy Method 71.2.2 Sausage Method 111.3 Maximum Modulus Theorem 151.4 Results 181.4.1 Fuzzy Method 191.4.2 Sausage Method 201.5 Conclusion 21References 232 EMOTION DETECTION IN IOT-BASED E-LEARNING USING CONVOLUTION NEURAL NETWORK 27Latha Parthiban and S. Selvakumara Samy2.1 Introduction 282.2 Related Works 302.3 Proposed Methodology 312.3.1 Students Emotion Recognition Towards the Class 312.3.2 Eye Gaze-Based Student Engagement Recognition 312.3.3 Facial Head Movement-Based Student Engagement Recognition 342.4 Experimental Results 352.4.1 Convolutional Layer 352.4.2 ReLU Layer 352.4.3 Pooling Layer 362.4.4 Fully Connected Layer 362.5 Conclusions 42References 423 FUZZY QUOTIENT-3 CORDIAL LABELING OF SOME TREES OF DIAMETER 5—PART III 45P. Sumathi and J. Suresh Kumar3.1 Introduction 463.2 Related Work 463.3 Definition 473.4 Notations 473.5 Main Results 483.6 Conclusion 71References 714 CLASSIFYING FUZZY MULTI-CRITERION DECISION MAKING AND EVOLUTIONARY ALGORITHM73Kirti Seth and Ashish Seth4.1 Introduction 744.1.1 Classical Optimization Techniques 744.1.2 The Bio-Inspired Techniques Centered on Optimization 754.1.2.1 Swarm Intelligence 774.1.2.2 The Optimization on Ant Colony 784.1.2.3 Particle Swarm Optimization (PSO) 824.1.2.4 Summary of PSO 834.2 Multiple Criteria That is Used for Decision Making (MCDM) 834.2.1 WSM Method 864.2.2 WPM Method 864.2.3 Analytic Hierarchy Process (AHP) 874.2.4 TOPSIS 894.2.5 VIKOR 904.3 Conclusion 91References 915 FUZZY TRI-MAGIC LABELING OF ISOMORPHIC CATERPILLAR GRAPH J62,3,4 OF DIAMETER 5 93P. Sumathi and C. Monigeetha5.1 Introduction 935.2 Main Result 955.3 Conclusion 154References 1546 FUZZY TRI-MAGIC LABELING OF ISOMORPHIC CATERPILLAR GRAPH J6 2,3,5 OF DIAMETER 5 155P. Sumathi and C. Monigeetha6.1 Introduction 1556.2 Main Result 1576.3 Conclusion 215References 2157 CEASELESS RULE-BASED LEARNING METHODOLOGY FOR GENETIC FUZZY RULE-BASED SYSTEMS 217B. Siva Kumar Reddy, R. Balakrishna and R. Anandan7.1 Introduction 2187.1.1 Integration of Evolutionary Algorithms and Fuzzy Logic 2197.1.2 Fuzzy Logic-Aided Evolutionary Algorithm 2207.1.3 Adaptive Genetic Algorithm That Adapt Manage Criteria 2207.1.4 Genetic Algorithm With Fuzzified Genetic Operators 2207.1.5 Genetic Fuzzy Systems 2207.1.6 Genetic Learning Process 2237.2 Existing Technology and its Review 2237.2.1 Techniques for Rule-Based Understanding with Genetic Algorithm 2237.2.2 Strategy A: GA Primarily Based Optimization for Computerized Built FLC 2237.2.3 Strategy B: GA-Based Optimization of Manually Created FLC 2247.2.4 Methods of Hybridization for GFS 2257.2.4.1 The Michigan Strategy—Classifier System 2267.2.4.2 The Pittsburgh Method 2297.3 Research Design 2337.3.1 The Ceaseless Rule Learning Approach (CRL) 2337.3.2 Multistage Processes of Ceaseless Rule Learning 2347.3.3 Other Approaches of Genetic Rule Learning 2367.4 Findings or Result Discussion so for in the Area of GFS Hybridization 2377.5 Conclusion 239References 2408 USING FUZZY TECHNIQUE MANAGEMENT OF CONFIGURATION AND STATUS OF VM FOR TASK DISTRIBUTION IN CLOUD SYSTEM 243Yogesh Shukla, Pankaj Kumar Mishra and Ramakant Bhardwaj8.1 Introduction 2448.2 Literature Review 2448.3 Logic System for Fuzzy 2468.4 Proposed Algorithm 2488.4.1 Architecture of System 2488.4.2 Terminology of Model 2508.4.3 Algorithm Proposed 2528.4.4 Explanations of Proposed Algorithm 2548.5 Results of Simulation 2578.5.1 Cloud System Numerical Model 2578.5.2 Evaluation Terms Definition 2588.5.3 Environment Configurations Simulation 2598.5.4 Outcomes of Simulation 2598.6 Conclusion 260References 2669 THEOREMS ON FUZZY SOFT METRIC SPACES 269Qazi Aftab Kabir, Ramakant Bhardwaj and Ritu Shrivastava9.1 Introduction 2699.2 Preliminaries 2709.3 FSMS 2719.4 Main Results 2739.5 Fuzzy Soft Contractive Type Mappings and Admissible Mappings 278References 28210 SYNCHRONIZATION OF TIME-DELAY CHAOTIC SYSTEM WITH UNCERTAINTIES IN TERMS OF TAKAGI–SUGENO FUZZY SYSTEM 285Sathish Kumar Kumaravel, Suresh Rasappan and Kala Raja Mohan10.1 Introduction 28510.2 Statement of the Problem and Notions 28610.3 Main Result 29110.4 Numerical Illustration 30210.5 Conclusion 312References 31211 TRAPEZOIDAL FUZZY NUMBERS (TRFN) AND ITS APPLICATION IN SOLVING ASSIGNMENT PROBLEM BY HUNGARIAN METHOD: A NEW APPROACH 315Rahul Kar, A.K. Shaw and J. Mishra11.1 Introduction 31611.2 Preliminary 31711.2.1 Definition 31711.2.2 Some Arithmetic Operations of Trapezoidal Fuzzy Number 31811.3 Theoretical Part 31911.3.1 Mathematical Formulation of an Assignment Problem 31911.3.2 Method for Solving an Assignment Problem 32011.3.2.1 Enumeration Method 32011.3.2.2 Regular Simplex Method 32111.3.2.3 Transportation Method 32111.3.2.4 Hungarian Method 32111.3.3 Computational Processor of Hungarian Method (For Minimization Problem) 32311.4 Application With Discussion 32511.5 Conclusion and Further Work 331References 33212 THE CONNECTEDNESS OF FUZZY GRAPH AND THE RESOLVING NUMBER OF FUZZY DIGRAPH 335Mary Jiny D. and R. Shanmugapriya12.1 Introduction 33612.2 Definitions 33612.3 An Algorithm to Find the Super Resolving Matrix 34112.3.1 An Application on Resolving Matrix 34412.3.2 An Algorithm to Find the Fuzzy Connectedness Matrix 34712.4 An Application of the Connectedness of the Modified Fuzzy Graph in Rescuing Human Life From Fire Accident 34912.4.1 Algorithm to Find the Safest and Shortest Path Between Two Landmarks 35212.5 Resolving Number Fuzzy Graph and Fuzzy Digraph 35612.5.1 An Algorithm to Find the Resolving Set of a Fuzzy Digraph 36012.6 Conclusion 362References 36213 A NOTE ON FUZZY EDGE MAGIC TOTAL LABELING GRAPHS 365R. Shanmugapriya and P.K. Hemalatha13.1 Introduction 36513.2 Preliminaries 36613.3 Theorem 36713.3.1 Example 36813.4 Theorem 37013.4.1 Example 37113.4.1.1 Lemma 37413.4.1.2 Lemma 37413.4.1.3 Lemma 37413.5 Theorem 37413.5.1 Example as Shown in Figure 13.5 Star Graph S(1,9) is FEMT Labeling 37413.6 Theorem 37613.7 Theorem 37713.7.1 Example 37813.8 Theorem 38013.9 Theorem 38113.10 Application of Fuzzy Edge Magic Total Labeling 38313.11 Conclusion 385References 38514 THE SYNCHRONIZATION OF IMPULSIVE TIME-DELAY CHAOTIC SYSTEMS WITH UNCERTAINTIES IN TERMS OF TAKAGI–SUGENO FUZZY SYSTEM 387Balaji Dharmalingam, Suresh Rasappan, V. Vijayalakshmi and G. Suseendran14.1 Introduction 38714.2 Problem Description and Preliminaries 38914.2.1 Impulsive Differential Equations 38914.3 The T–S Fuzzy Model 39114.4 Designing of Fuzzy Impulsive Controllers 39314.5 Main Result 39414.6 Numerical Example 40014.7 Conclusion 410References 41015 THEOREMS ON SOFT FUZZY METRIC SPACES BY USING CONTROL FUNCTION 413Sneha A. Khandait, Chitra Singh, Ramakant Bhardwaj and Amit Kumar Mishra15.1 Introduction 41315.2 Preliminaries and Definition 41415.3 Main Results 41515.4 Conclusion 429References 42916 ON SOFT Α(γ,β)-CONTINUOUS FUNCTIONS IN SOFT TOPOLOGICAL SPACES 431N. Kalaivani, E. Chandrasekaran and K. Fayaz Ur Rahman16.1 Introduction 43216.2 Preliminaries 43216.2.1 Outline 43216.2.2 Soft αγ-Open Set 43216.2.3 Soft αγTi Spaces 43416.2.4 Soft (αγ, βs)-Continuous Functions 43616.3 Soft α(γ,β)-Continuous Functions in Soft Topological Spaces 43816.3.1 Outline 43816.3.2 Soft α(γ,β)-Continuous Functions 43816.3.3 Soft α(γ,β)-Open Functions 44416.3.4 Soft α(γ,β)-Closed Functions 44716.3.5 Soft α(γ,β)-Homeomorphism 45016.3.6 Soft (αγ, βs)-Contra Continuous Functions 45016.3.7 Soft α(γ,β)-Contra Continuous Functions 45516.4 Conclusion 459References 459Index 461
Beginning Azure Cognitive Services
Get started with Azure Cognitive Services and its APIs that expose machine learning as a service. This book introduces the suite of Azure Cognitive Services and helps you take advantage of the proven machine learning algorithms that have been developed by experts and made available through Cognitive Services, easily integrating those algorithms into your own applications without having to develop the algorithms from scratch. The book also shows you how to use the algorithms provided by Cognitive Services to accelerate data analysis and development within your organization.The authors begin by introducing the tools and describing the steps needed to invoke libraries to analyze structured and unstructured text, speech, and pictures, and you will learn to create interactive chatbots using the Cognitive Services libraries. Each chapter contains the information you need to implement artificial intelligence (AI) via Azure Cognitive Services in your personal and professional projects.The book also covers ethical considerations that are becoming increasingly of concern when using AI to drive decision making. You will be introduced to tools such as FairLearn and InterpretML that can help you detect bias and understand the results your models are generating.WHAT YOU WILL LEARN* Invoke the Cognitive Services APIs from a variety of languages and apps* Understand common design architectures for AI solutions in Azure* Decrease discrimination and bias when creating an AI-driven solution* Execute the examples within the book and learn how to extend those examples* Implement best practices for leveraging the Vision, Speech, and Language parts of the suite* Test Cognitive Services APIs via the Azure portal and using the Postman API tool* Execute AI from low-code and no-code platforms like Logic Apps and Microsoft’s Power PlatformWHO THIS BOOK IS FORTechnical professionals who are interested in implementing artificial intelligence (AI) in pre-existing apps, expanding their value and skill sets, or learning more about AI for personal projects; for programmers working in languages such as C# and Python; and for those using low- and no-code platforms such as Microsoft Power PlatformALICIA MONIZ is a Microsoft AI MVP, #KafkaOnAzure Evangelista, and active supporter of women in technology. She authors the blog HybridDataLakes.com, focused on cloud data learning resources, and produces content for the YouTube channel, #KafkaOnAzure. She is also on the Leadership Board for the Global AI community and is an organizer for Global AI Bootcamp - Houston Edition, a Microsoft AI-sponsored event. Alicia is active in the Microsoft User Group community and enjoys speaking on AI, SQLServer, #KafkaOnAzure, and personal branding for women in technology topics.MATT GORDON is a Microsoft Data Platform MVP and has worked with SQL Server since 2000. He is the leader of the Lexington, KY Data Technology Group and a frequent domestic and international community speaker. He is an IDERA ACE alumnus and 2021 Friend of Redgate. His original data professional role was in database development, which quickly evolved into query tuning work that further evolved into being a DBA in the healthcare realm. He has supported several critical systems utilizing SQL Server and managed dozens of 24/7/365 SQL Server implementations. Following several years as a consultant, Matt is now the Director of Data and Infrastructure for rev.io, where he is implementing data governance, DevOps, and performance improvements enterprise-wide.IDA BERGUM is a Microsoft Data Platform MVP, and is currently working for Avanade Norway as a Solution Architect for the Data & AI Market Unit as well as leading Avanade's Community of Practice for Power BI globally. Ida has been advising on, architecting, and implementing modern data platform and BI solutions on Azure since she started at Avanade in 2015. She aims at advising and delivering great analytics experiences and data smart solutions for clients while sharing those experiences with the community. Her spare time is spent contributing to Data Platform User Group Norway among others, posting to her Twitter account, and cross-country skiing whenever she gets the opportunity.MIA CHANG works as a Solutions Architect specializing in machine learning for a tech company in Berlin, Germany. She cooperates with the customers on their development projects and delivers architecture leveraging services on the cloud. Her machine learning journey started with her studies in the field of applied mathematics, and computer science focused on algorithm research of strategic board games. During her seven-year professional journey, she has worked as a data scientist, and she has been proficient in the life cycle of ML projects. Apart from the work, Mia is passionate about open source projects and has been awarded the Microsoft AI MVP from 2017 to 2020. Her first book Microsoft AI MVP Book was published in 2019. She also dedicates her time to organizing communities and speaks at meetups and conferences. In addition to tech-related hobbies, she loves gardening, hiking, and traveling.GINGER GRANT is a consultant who shares what she has learned while working with data technology clients by providing training to people around the world. As a Microsoft MVP in Data Platform and a Microsoft Certified Trainer, she is proficient in creating solutions using Power BI and the Microsoft Azure Data Stack components, including Databricks, Data Factory, Data Lakes, Data Analytics, Synapse, and Machine Learning. Ginger co-authored Exam Ref 70-774 Perform Cloud Data Science with Azure Machine Learning and has a number of current exam certifications. When not working, she maintains her blog and spends time on twitter.1. Introducing Cognitive Services2. Prerequisites and Tools3. Vision4. Language in Cognitive Services5. Speech Services6. Power Platform & Cognitive Services7. Chatbots8. Ethics in AI
3D-Konstruktionen mit Autodesk Inventor 2022
* 3D-MODELLIEREN VON DER SKIZZE ÜBER BAUTEILE UND BAUGRUPPEN BIS ZUR WERKZEICHNUNG UND ANIMATION* PRAXISNAHE ERLÄUTERUNG GRUNDLEGENDER BEFEHLE MIT VIELEN VERSCHIEDENEN KONSTRUKTIONSBEISPIELEN ZUM NACHBAUEN* EINFÜHRUNG IN DIE ILOGIC-PROGRAMMIERUMGEBUNGDieses Grundlagen- und Lehrbuch zeigt Ihnen anhand vieler einzelner Konstruktionsbeispiele die Möglichkeiten von Inventor 2022 und richtet sich insbesondere an Inventor-Neulinge, die Wert auf einen gründlichen praxisnahen Einstieg legen.Mit der Inventor-Testversion von der Autodesk-Webseite können Sie zügig eigene dreidimensionale Konstruktionen erstellen. Die wichtigsten Vorgehensweisen werden mit vielen einzelnen Beispielen erklärt. Für jedes Kapitel finden Sie Testfragen mit Lösungen im Anhang.ZAHLREICHE BEFEHLE WERDEN DETAILLIERT ERLÄUTERT, WIE Z.B.:* 2D-Skizzen mit Linie, Bogen, Kreis, Rechteck, Langloch, Polygon* Abhängigkeiten und Bemaßungen in der Skizze* Volumenkörper aus Skizzen mit Extrusion, Rotation, Sweeping und Lofting* Spezielle Modellierung mit Spirale, Prägen, Ableiten, Rippe und Aufkleben* Befehle mit neuen Eigenschaften-Dialogen* Import für AutoCAD-2D-Zeichnungen mit assoziativer Zuordnung* 3D-Modellierung mit Grundkörpern wie Quader, Zylinder, Kugel und Torus* Volumenkörper durch Features ergänzen: Bohrung, Fase, Rundung, Gewinde, Wandstärke, Flächenverjüngung etc.* Baugruppen durch Platzieren der Bauteile mit geometrischen Abhängigkeiten* Integration von Architektur-Dateien aus Revit, Export in die Fusion-Umgebung* Zeichnungsableitung mit Ansichten, Bemaßungen und Beschriftungen* Erstellen von Stücklisten und Positionsnummern* Gestaltung verschiedener Modellzustände innerhalb eines Bauteils oder einer Baugruppe* Animierte Präsentationen, fotorealistische Bilder mit verschiedenen Stilen* Einführung in die iLogic-Programmierung mit einfachen nützlichen BeispielenAlle Befehle werden umfangreich vorgestellt und können daher leicht für eigene Projekte eingesetzt werden. Insbesondere die Befehle mit den neuen Eigenschaften-Fenstern anstelle der alten Dialogfenster werden über passende Beispiele erschlossen.AUS DEM INHALT:* Installation und Benutzeroberfläche* Skizzenerstellung in 2D und 3D* 3D-Modellierung* Abhängigkeiten und Bemaßungen* AutoCAD-2D-Import* 3D-Modellierung* iFeatures, iMates, iParts, iAssemblies* Baugruppenerstellung und Gestaltung von Modellzuständen* Zeichnungs-Ableitung* Bemaßungen und Stile* Präsentationen* Module für Blech, Gestell, Welle, Schweißen, BIM-Export, Elektronik* iLogic-ProgrammierungDetlef Ridder hat bereits zahlreiche Bücher zu AutoCAD, Inventor, Revit und ArchiCAD veröffentlicht und gibt Schulungen zu diesen Programmen und im Bereich CNC.
In-/Near-Memory Computing
THIS BOOK PROVIDES A STRUCTURED INTRODUCTION OF THE KEY CONCEPTS AND TECHNIQUES THAT ENABLE IN-/NEAR-MEMORY COMPUTING. For decades, processing-in-memory or near-memory computing has been attracting growing interest due to its potential to break the memory wall. Near-memory computing moves compute logic near the memory, and thereby reduces data movement. Recent work has also shown that certain memories can morph themselves into compute units by exploiting the physical properties of the memory cells, enabling in-situ computing in the memory array. While in- and near-memory computing can circumvent overheads related to data movement, it comes at the cost of restricted flexibility of data representation and computation, design challenges of compute capable memories, and difficulty in system and software integration. Therefore, wide deployment of in-/near-memory computing cannot be accomplished without techniques that enable efficient mapping of data-intensive applications to such devices, without sacrificing accuracy or increasing hardware costs excessively. This book describes various memory substrates amenable to in- and near-memory computing, architectural approaches for designing efficient and reliable computing devices, and opportunities for in-/near-memory acceleration of different classes of applications.* Preface* Acknowledgments* Introduction* Technology Basics and Taxonomy* Computing with DRAMs* Computing with SRAMs* Computing with Non-Volatile Memories* Domain-Specific Accelerators* Programming Models* Closing Thoughts* Bibliography* Authors' Biographies
Microsoft Office für Senioren - Word, Excel und PowerPoint
Die leicht nachvollziehbare Anleitung für Einsteiger ohne Vorkenntnisse:- Grundlagenwissen mit praktischen Beispielen, Übungen und Tipps- Für die Versionen Microsoft 365 sowie Office 2019, 2016 & 2013Ob im Alltag, im Beruf oder für Ihr Hobby: Entdecken Sie die Möglichkeiten von Word, Excel und PowerPoint! Dieses Buch führt Einsteiger Schritt für Schritt in die Programme von Microsoft Office ein – Vorkenntnisse sind dabei nicht erforderlich. Lernen Sie, wie Sie mit dem Textverarbeitungsprogramm Word u. a. Briefe schreiben und Ihre Texte um Bilder und Tabellen ergänzen. Excel ist der Spezialist für Tabellen: Erstellen Sie im Handumdrehen Adresslisten oder gestalten Sie Ihre persönliche Finanzübersicht. Teil 3 zeigt Ihnen, wie Sie mit PowerPoint und seinen grafischen Fähigkeiten z. B. Fotoalben und individuelle Geburtstagsgrüße als effektvolle Bildschirmpräsentation kreieren, in denen auch Animationen nicht fehlen.Inge Baumeister kennt durch ihre langjährige Erfahrung als Dozentin die typischen Fragen von Anfängern und weiß, wie komplexe Sachverhalte einfach zu erklären sind. Wie in einem Kurs leitet sie ihre Leser durch die unzähligen Möglichkeiten von Word, Excel und PowerPoint und gibt Tipps, die auch Ihre Arbeit erleichtern.Aus dem InhaltWord:- Dokumente erstellen, speichern und drucken- Texte eingeben und formatieren- Rechtschreib- und Grammatikprüfung nutzen- Aufzählungen und Nummerierungen hinzufügen- Bilder und Tabellen einfügen- Das Seitenlayout einrichtenExcel:- Aufbau von Excel-Arbeitsmappen und Tabellen- Text, Zahlen und Datum in Tabellen eingeben- Inhalte übersichtlich gestalten und drucken- Summen und einfache Formeln berechnen- Adressen- und Geburtstagslisten gestalten- Ihre persönliche Finanzübersicht erstellenPowerPoint:- Folien mit Text und Bildelementen gestalten- Vorlagen nutzen und anpassen- Ein Fotoalbum als Präsentation erstellen- Folienübergänge einsetzen- Elemente mit Animationseffekten versehen- Die fertige Präsentation vorführen – u. v. m.!
Monetarisierung von technischen Daten
Die Monetarisierung von Daten ist per se ein sehr junges Thema, zu dem es nur sehr vereinzelt Fallbeispiele gibt. Es fehlt an einer Strategie bzw. einem Konzept, das Führungskräften den Weg in die Monetarisierung von Daten zeigt, insbesondere jenen, die die Digitale Transformation bzw. Industrie 4.0 für sich entdeckt haben oder davon bedroht sind. Weil Maschinendaten meist unstrukturiert und ohne Domänenwissen/Metadaten nicht verwertbar sind, birgt die Monetarisierung von Maschinendaten ein noch nicht abschließend bewertbares Potenzial. Um dieses Potenzial greifbar zu machen, werden in diesem Werk neben Beiträgen aus der Wissenschaft auch Praxisbeispiele aus der Industrie beschrieben. Anhand von unterschiedlichen Beispielen aus diversen Branchen kann der Leser bereits heute Teil einer zukünftigen Datenökonomie werden. Mehrwerte und Nutzen werden konkret beschrieben.
Foundations of Data Intensive Applications
PEEK “UNDER THE HOOD” OF BIG DATA ANALYTICSThe world of big data analytics grows ever more complex. And while many people can work superficially with specific frameworks, far fewer understand the fundamental principles of large-scale, distributed data processing systems and how they operate. In Foundations of Data Intensive Applications: Large Scale Data Analytics under the Hood, renowned big-data experts and computer scientists Drs. Supun Kamburugamuve and Saliya Ekanayake deliver a practical guide to applying the principles of big data to software development for optimal performance. The authors discuss foundational components of large-scale data systems and walk readers through the major software design decisions that define performance, application type, and usability. You???ll learn how to recognize problems in your applications resulting in performance and distributed operation issues, diagnose them, and effectively eliminate them by relying on the bedrock big data principles explained within. Moving beyond individual frameworks and APIs for data processing, this book unlocks the theoretical ideas that operate under the hood of every big data processing system. Ideal for data scientists, data architects, dev-ops engineers, and developers, Foundations of Data Intensive Applications: Large Scale Data Analytics under the Hood shows readers how to:* IDENTIFY THE FOUNDATIONS OF LARGE-SCALE, DISTRIBUTED DATA PROCESSING SYSTEMS##SINGLE_LINE##* MAKE MAJOR SOFTWARE DESIGN DECISIONS THAT OPTIMIZE PERFORMANCE##SINGLE_LINE##* DIAGNOSE PERFORMANCE PROBLEMS AND DISTRIBUTED OPERATION ISSUES##SINGLE_LINE##* UNDERSTAND STATE-OF-THE-ART RESEARCH IN BIG DATA##SINGLE_LINE##* EXPLAIN AND USE THE MAJOR BIG DATA FRAMEWORKS AND UNDERSTAND WHAT UNDERPINS THEM##SINGLE_LINE##* USE BIG DATA ANALYTICS IN THE REAL WORLD TO SOLVE PRACTICAL PROBLEMS##SINGLE_LINE##SUPUN KAMBURUGAMUVE, PHD, is a computer scientist researching and designing large scale data analytics tools. He received his doctorate in Computer Science from Indiana University, Bloomington and architected the data processing systems Twister2 and Cylon.SALIYA EKANAYAKE, PHD, is a Senior Software Engineer at Microsoft working in the intersection of scaling deep learning systems and parallel computing. He is also a research affiliate at Berkeley Lab. He received his doctorate in Computer Science from Indiana University, Bloomington. Introduction xxviiCHAPTER 1 DATA INTENSIVE APPLICATIONS 1Anatomy of a Data-Intensive Application 1A Histogram Example 2Program 2Process Management 3Communication 4Execution 5Data Structures 6Putting It Together 6Application 6Resource Management 6Messaging 7Data Structures 7Tasks and Execution 8Fault Tolerance 8Remote Execution 8Parallel Applications 9Serial Applications 9Lloyd’s K-Means Algorithm 9Parallelizing Algorithms 11Decomposition 11Task Assignment 12Orchestration 12Mapping 13K-MeansAlgorithm 13Parallel and Distributed Computing 15Memory Abstractions 16Shared Memory 16Distributed Memory 18Hybrid (Shared + Distributed) Memory 20Partitioned Global Address Space Memory 21Application Classes and Frameworks 22Parallel Interaction Patterns 22Pleasingly Parallel 23Dataflow 23Iterative 23Irregular 23Data Abstractions 24Data-IntensiveFrameworks 24Components 24Workflows 25An Example 25What Makes It Difficult? 26Developing Applications 27Concurrency 27Data Partitioning 28Debugging 28Diverse Environments 28Computer Networks 29Synchronization 29Thread Synchronization 29Data Synchronization 30Ordering of Events 31Faults 31Consensus 31Summary 32References 32CHAPTER 2 DATA AND STORAGE 35Storage Systems 35Storage for Distributed Systems 36Direct-Attached Storage 37Storage Area Network 37Network-Attached Storage 38DAS or SAN or NAS? 38Storage Abstractions 39Block Storage 39File Systems 40Object Storage 41Data Formats 41XML 42JSON 43CSV 44Apache Parquet 45Apache Avro 47Avro Data Definitions (Schema) 48Code Generation 49Without Code Generation 49Avro File 49Schema Evolution 49Protocol Buffers, Flat Buffers, and Thrift 50Data Replication 51Synchronous and Asynchronous Replication 52Single-Leader and Multileader Replication 52Data Locality 53Disadvantages of Replication 54Data Partitioning 54Vertical Partitioning 55Horizontal Partitioning (Sharding) 55Hybrid Partitioning 56Considerations for Partitioning 57NoSQL Databases 58Data Models 58Key-Value Databases 58Document Databases 59Wide Column Databases 59Graph Databases 59CAP Theorem 60Message Queuing 61Message Processing Guarantees 63Durability of Messages 64Acknowledgments 64Storage First Brokers and Transient Brokers 65Summary 66References 66CHAPTER 3 COMPUTING RESOURCES 69A Demonstration 71Computer Clusters 72Anatomy of a Computer Cluster 73Data Analytics in Clusters 74Dedicated Clusters 76Classic Parallel Systems 76Big Data Systems 77Shared Clusters 79OpenMPI on a Slurm Cluster 79Spark on a Yarn Cluster 80Distributed Application Life Cycle 80Life Cycle Steps 80Step 1: Preparation of the Job Package 81Step 2: Resource Acquisition 81Step 3: Distributing the Application (Job) Artifacts 81Step 4: Bootstrapping the Distributed Environment 82Step 5: Monitoring 82Step 6: Termination 83Computing Resources 83Data Centers 83Physical Machines 85Network 85Virtual Machines 87Containers 87Processor, Random Access Memory, and Cache 88Cache 89Multiple Processors in a Computer 90Nonuniform Memory Access 90Uniform Memory Access 91Hard Disk 92GPUs 92Mapping Resources to Applications 92Cluster Resource Managers 93Kubernetes 94Kubernetes Architecture 94Kubernetes Application Concepts 96Data-Intensive Applications on Kubernetes 96Slurm 98Yarn 99Job Scheduling 99Scheduling Policy 101Objective Functions 101Throughput and Latency 101Priorities 102Lowering Distance Among the Processes 102Data Locality 102Completion Deadline 102Algorithms 103First in First Out 103Gang Scheduling 103List Scheduling 103Backfill Scheduling 104Summary 104References 104CHAPTER 4 DATA STRUCTURES 107Virtual Memory 108Paging and TLB 109Cache 111The Need for Data Structures 112Cache and Memory Layout 112Memory Fragmentation 114Data Transfer 115Data Transfer Between Frameworks 115Cross-Language Data Transfer 115Object and Text Data 116Serialization 116Vectors and Matrices 1171D Vectors 118Matrices 118Row-Major and Column-Major Formats 119N-Dimensional Arrays/Tensors 122NumPy 123Memory Representation 125K-means with NumPy 126Sparse Matrices 127Table 128Table Formats 129Column Data Format 129Row Data Format 130Apache Arrow 130Arrow Data Format 131Primitive Types 131Variable-Length Data 132Arrow Serialization 133Arrow Example 133Pandas DataFrame 134Column vs. Row Tables 136Summary 136References 136CHAPTER 5 PROGRAMMING MODELS 139Introduction 139Parallel Programming Models 140Parallel Process Interaction 140Problem Decomposition 140Data Structures 140Data Structures and Operations 141Data Types 141Local Operations 143Distributed Operations 143Array 144Tensor 145Indexing 145Slicing 146Broadcasting 146Table 146Graph Data 148Message Passing Model 150Model 151Message Passing Frameworks 151Message Passing Interface 151Bulk Synchronous Parallel 153K-Means 154Distributed Data Model 157Eager Model 157Dataflow Model 158Data Frames, Datasets, and Tables 159Input and Output 160Task Graphs (Dataflow Graphs) 160Model 161User Program to Task Graph 161Tasks and Functions 162Source Task 162Compute Task 163Implicit vs. Explicit Parallel Models 163Remote Execution 163Components 164Batch Dataflow 165Data Abstractions 165Table Abstraction 165Matrix/Tensors 165Functions 166Source 166Compute 167Sink 168An Example 168Caching State 169Evaluation Strategy 170Lazy Evaluation 171Eager Evaluation 171Iterative Computations 172DOALL Parallel 172DOACROSS Parallel 172Pipeline Parallel 173Task Graph Models for Iterative Computations 173K-Means Algorithm 174Streaming Dataflow 176Data Abstractions 177Streams 177Distributed Operations 178Streaming Functions 178Sources 178Compute 179Sink 179An Example 179Windowing 180Windowing Strategies 181Operations on Windows 182Handling Late Events 182SQL 182Queries 183Summary 184References 184CHAPTER 6 MESSAGING 187Network Services 188TCP/IP 188RDMA 189Messaging for Data Analytics 189Anatomy of a Message 190Data Packing 190Protocol 191Message Types 192Control Messages 192External Data Sources 192Data Transfer Messages 192Distributed Operations 194How Are They Used? 194Task Graph 194Parallel Processes 195Anatomy of a Distributed Operation 198Data Abstractions 198Distributed Operation API 198Streaming and Batch Operations 199Streaming Operations 199Batch Operations 199Distributed Operations on Arrays 200Broadcast 200Reduce and AllReduce 201Gather and AllGather 202Scatter 203AllToAll 204Optimized Operations 204Broadcast 205Reduce 206AllReduce 206Gather and AllGather Collective Algorithms 208Scatter and AllToAll Collective Algorithms 208Distributed Operations on Tables 209Shuffle 209Partitioning Data 211Handling Large Data 212Fetch-Based Algorithm (Asynchronous Algorithm) 213Distributed Synchronization Algorithm 214GroupBy 214Aggregate 215Join 216Join Algorithms 219Distributed Joins 221Performance of Joins 223More Operations 223Advanced Topics 224Data Packing 224Memory Considerations 224Message Coalescing 224Compression 225Stragglers 225Nonblocking vs. Blocking Operations 225Blocking Operations 226Nonblocking Operations 226Summary 227References 227CHAPTER 7 PARALLEL TASKS 229CPUs 229Cache 229False Sharing 230Vectorization 231Threads and Processes 234Concurrency and Parallelism 234Context Switches and Scheduling 234Mutual Exclusion 235User-Level Threads 236Process Affinity 236NUMA-Aware Programming 237Accelerators 237Task Execution 238Scheduling 240Static Scheduling 240Dynamic Scheduling 240Loosely Synchronous and Asynchronous Execution 241Loosely Synchronous Parallel System 242Asynchronous Parallel System (Fully Distributed) 243Actor Model 244Actor 244Asynchronous Messages 244Actor Frameworks 245Execution Models 245Process Model 246Thread Model 246Remote Execution 246Tasks for Data Analytics 248SPMD and MPMD Execution 248Batch Tasks 249Data Partitions 249Operations 251Task Graph Scheduling 253Threads, CPU Cores, and Partitions 254Data Locality 255Execution 257Streaming Execution 257State 257Immutable Data 258State in Driver 258Distributed State 259Streaming Tasks 259Streams and Data Partitioning 260Partitions 260Operations 261Scheduling 262Uniform Resources 263Resource-Aware Scheduling 264Execution 264Dynamic Scaling 264Back Pressure (Flow Control) 265Rate-Based Flow Control 266Credit-Based Flow Control 266State 267Summary 268References 268CHAPTER 8 CASE STUDIES 271Apache Hadoop 271Programming Model 272Architecture 274Cluster Resource Management 275Apache Spark 275Programming Model 275RDD API 276SQL, DataFrames, and DataSets 277Architecture 278Resource Managers 278Task Schedulers 279Executors 279Communication Operations 280Apache Spark Streaming 280Apache Storm 282Programming Model 282Architecture 284Cluster Resource Managers 285Communication Operations 286Kafka Streams 286Programming Model 286Architecture 287PyTorch 288Programming Model 288Execution 292Cylon 295Programming Model 296Architecture 296Execution 297Communication Operations 298Rapids cuDF 298Programming Model 298Architecture 299Summary 300References 300CHAPTER 9 FAULT TOLERANCE 303Dependable Systems and Failures 303Fault Tolerance is Not Free 304Dependable Systems 305Failures 306Process Failures 306Network Failures 307Node Failures 307Byzantine Faults 307Failure Models 308Failure Detection 308Recovering from Faults 309Recovery Methods 310Stateless Programs 310Batch Systems 311Streaming Systems 311Processing Guarantees 311Role of Cluster Resource Managers 312Checkpointing 313State 313Consistent Global State 313Uncoordinated Checkpointing 314Coordinated Checkpointing 315Chandy-Lamport Algorithm 315Batch Systems 316When to Checkpoint? 317Snapshot Data 318Streaming Systems 319Case Study: Apache Storm 319Message Tracking 320Failure Recovery 321Case Study: Apache Flink 321Checkpointing 322Failure Recovery 324Batch Systems 324Iterative Programs 324Case Study: Apache Spark 325RDD Recomputing 326Checkpointing 326Recovery from Failures 327Summary 327References 327CHAPTER 10 PERFORMANCE AND PRODUCTIVITY 329Performance Metrics 329System Performance Metrics 330Parallel Performance Metrics 330Speedup 330Strong Scaling 331Weak Scaling 332Parallel Efficiency 332Amdahl’s Law 333Gustafson’s Law 334Throughput 334Latency 335Benchmarks 336LINPACK Benchmark 336NAS Parallel Benchmark 336BigDataBench 336TPC Benchmarks 337HiBench 337Performance Factors 337Memory 337Execution 338Distributed Operators 338Disk I/O 339Garbage Collection 339Finding Issues 342Serial Programs 342Profiling 342Scaling 343Strong Scaling 343Weak Scaling 344Debugging Distributed Applications 344Programming Languages 345C/C++ 346Java 346Memory Management 347Data Structures 348Interfacing with Python 348Python 350C/C++ Code integration 350Productivity 351Choice of Frameworks 351Operating Environment 353CPUs and GPUs 353Public Clouds 355Future of Data-Intensive Applications 358Summary 358References 359Index 361
The Official (ISC)2 CISSP CBK Reference
THE ONLY OFFICIAL, COMPREHENSIVE REFERENCE GUIDE TO THE CISSPThoroughly updated for 2021 and beyond, this is the authoritative common body of knowledge (CBK) from (ISC)2 for information security professionals charged with designing, engineering, implementing, and managing the overall information security program to protect organizations from increasingly sophisticated attacks. Vendor neutral and backed by (ISC)2, the CISSP credential meets the stringent requirements of ISO/IEC Standard 17024.This CBK covers the current eight domains of CISSP with the necessary depth to apply them to the daily practice of information security. Revised and updated by a team of subject matter experts, this comprehensive reference covers all of the more than 300 CISSP objectives and sub-objectives in a structured format with:* Common and good practices for each objective* Common vocabulary and definitions* References to widely accepted computing standards* Highlights of successful approaches through case studiesWhether you've earned your CISSP credential or are looking for a valuable resource to help advance your security career, this comprehensive guide offers everything you need to apply the knowledge of the most recognized body of influence in information security.Foreword xixIntroduction xxiDOMAIN 1: SECURITY AND RISK MANAGEMENT 1Understand, Adhere to, and Promote Professional Ethics 2(ISC)2 Code of Professional Ethics 2Organizational Code of Ethics 3Understand and Apply Security Concepts 4Confidentiality 4Integrity 5Availability 6Limitations of the CIA Triad 7Evaluate and Apply Security Governance Principles 8Alignment of the Security Function to Business Strategy, Goals, Mission, and Objectives 9Organizational Processes 10Organizational Roles and Responsibilities 14Security Control Frameworks 15Due Care and Due Diligence 22Determine Compliance and Other Requirements 23Legislative and Regulatory Requirements 23Industry Standards and Other Compliance Requirements 25Privacy Requirements 27Understand Legal and Regulatory Issues That Pertain to Information Security in a Holistic Context 28Cybercrimes and Data Breaches 28Licensing and Intellectual Property Requirements 36Import/Export Controls 39Transborder Data Flow 40Privacy 41Understand Requirements for Investigation Types 48Administrative 49Criminal 50Civil 52Regulatory 53Industry Standards 54Develop, Document, and Implement Security Policy, Standards, Procedures, and Guidelines 55Policies 55Standards 56Procedures 57Guidelines 57Identify, Analyze, and Prioritize Business Continuity Requirements 58Business Impact Analysis 59Develop and Document the Scope and the Plan 61Contribute to and Enforce Personnel Security Policies and Procedures 63Candidate Screening and Hiring 63Employment Agreements and Policies 64Onboarding, Transfers, and Termination Processes 65Vendor, Consultant, and Contractor Agreements and Controls 67Compliance Policy Requirements 67Privacy Policy Requirements 68Understand and Apply Risk Management Concepts 68Identify Threats and Vulnerabilities 68Risk Assessment 70Risk Response/Treatment 72Countermeasure Selection and Implementation 73Applicable Types of Controls 75Control Assessments 76Monitoring and Measurement 77Reporting 77Continuous Improvement 78Risk Frameworks 78Understand and Apply Threat Modeling Concepts and Methodologies 83Threat Modeling Concepts 84Threat Modeling Methodologies 85Apply Supply Chain Risk Management Concepts 88Risks Associated with Hardware, Software, and Services 88Third-Party Assessment and Monitoring 89Minimum Security Requirements 90Service-LevelRequirements 90Frameworks 91Establish and Maintain a Security Awareness, Education, and Training Program 92Methods and Techniques to Present Awareness and Training 93Periodic Content Reviews 94Program Effectiveness Evaluation 94Summary 95DOMAIN 2: ASSET SECURITY 97Identify and Classify Information and Assets 97Data Classification and Data Categorization 99Asset Classification 101Establish Information and Asset Handling Requirements 104Marking and Labeling 104Handling 105Storage 105Declassification 106Provision Resources Securely 108Information and Asset Ownership 108Asset Inventory 109Asset Management 112Manage Data Lifecycle 115Data Roles 116Data Collection 120Data Location 120Data Maintenance 121Data Retention 122Data Destruction 123Data Remanence 123Ensure Appropriate Asset Retention 127Determining Appropriate Records Retention 129Records Retention Best Practices 130Determine Data Security Controls and Compliance Requirements 131Data States 133Scoping and Tailoring 135Standards Selection 137Data Protection Methods 141Summary 144DOMAIN 3: SECURITY ARCHITECTURE AND ENGINEERING 147Research, Implement, and Manage Engineering Processes Using Secure Design Principles 149ISO/IEC 19249 150Threat Modeling 157Secure Defaults 160Fail Securely 161Separation of Duties 161Keep It Simple 162Trust, but Verify 162Zero Trust 163Privacy by Design 165Shared Responsibility 166Defense in Depth 167Understand the Fundamental Concepts of Security Models 168Primer on Common Model Components 168Information Flow Model 169Noninterference Model 169Bell–LaPadula Model 170Biba Integrity Model 172Clark–Wilson Model 173Brewer–Nash Model 173Take-Grant Model 175Select Controls Based Upon Systems Security Requirements 175Understand Security Capabilities of Information Systems 179Memory Protection 180Secure Cryptoprocessor 182Assess and Mitigate the Vulnerabilities of Security Architectures, Designs, and Solution Elements 187Client-Based Systems 187Server-Based Systems 189Database Systems 191Cryptographic Systems 194Industrial Control Systems 200Cloud-Based Systems 203Distributed Systems 207Internet of Things 208Microservices 212Containerization 214Serverless 215Embedded Systems 216High-Performance Computing Systems 219Edge Computing Systems 220Virtualized Systems 221Select and Determine Cryptographic Solutions 224Cryptography Basics 225Cryptographic Lifecycle 226Cryptographic Methods 229Public Key Infrastructure 243Key Management Practices 246Digital Signatures and Digital Certificates 250Nonrepudiation 252Integrity 253Understand Methods of Cryptanalytic Attacks 257Brute Force 258Ciphertext Only 260Known Plaintext 260Chosen Plaintext Attack 260Frequency Analysis 261Chosen Ciphertext 261Implementation Attacks 261Side-Channel Attacks 261Fault Injection 263Timing Attacks 263Man-in-the-Middle 263Pass the Hash 263Kerberos Exploitation 264Ransomware 264Apply Security Principles to Site and Facility Design 265Design Site and Facility Security Controls 265Wiring Closets/Intermediate Distribution Facilities 266Server Rooms/Data Centers 267Media Storage Facilities 268Evidence Storage 269Restricted and Work Area Security 270Utilities and Heating, Ventilation, and Air Conditioning 272Environmental Issues 275Fire Prevention, Detection, and Suppression 277Summary 281DOMAIN 4: COMMUNICATION AND NETWORK SECURITY 283Assess and Implement Secure Design Principles in Network Architectures 283Open System Interconnection and Transmission Control Protocol/Internet Protocol Models 285The OSI Reference Model 286The TCP/IP Reference Model 299Internet Protocol Networking 302Secure Protocols 311Implications of Multilayer Protocols 313Converged Protocols 315Microsegmentation 316Wireless Networks 319Cellular Networks 333Content Distribution Networks 334Secure Network Components 335Operation of Hardware 335Repeaters, Concentrators, and Amplifiers 341Hubs 341Bridges 342Switches 342Routers 343Gateways 343Proxies 343Transmission Media 345Network Access Control 352Endpoint Security 354Mobile Devices 355Implement Secure Communication Channels According to Design 357Voice 357Multimedia Collaboration 359Remote Access 365Data Communications 371Virtualized Networks 373Third-PartyConnectivity 374Summary 374DOMAIN 5: IDENTITY AND ACCESS MANAGEMENT 377Control Physical and Logical Access to Assets 378Access Control Definitions 378Information 379Systems 380Devices 381Facilities 383Applications 386Manage Identification and Authentication of People, Devices, and Services 387Identity Management Implementation 388Single/Multifactor Authentication 389Accountability 396Session Management 396Registration, Proofing, and Establishment of Identity 397Federated Identity Management 399Credential Management Systems 399Single Sign-On 400Just-In-Time 401Federated Identity with a Third-Party Service 401On Premises 402Cloud 403Hybrid 403Implement and Manage Authorization Mechanisms 404Role-Based Access Control 405Rule-Based Access Control 405Mandatory Access Control 406Discretionary Access Control 406Attribute-Based Access Control 407Risk-Based Access Control 408Manage the Identity and Access Provisioning Lifecycle 408Account Access Review 409Account Usage Review 411Provisioning and Deprovisioning 411Role Definition 412Privilege Escalation 413Implement Authentication Systems 414OpenID Connect/Open Authorization 414Security Assertion Markup Language 415Kerberos 416Remote Authentication Dial-In User Service/Terminal Access Controller Access Control System Plus 417Summary 418DOMAIN 6: SECURITY ASSESSMENT AND TESTING 419Design and Validate Assessment, Test, and Audit Strategies 420Internal 421External 422Third-Party 423Conduct Security Control Testing 423Vulnerability Assessment 423Penetration Testing 428Log Reviews 435Synthetic Transactions 435Code Review and Testing 436Misuse Case Testing 437Test Coverage Analysis 438Interface Testing 439Breach Attack Simulations 440Compliance Checks 441Collect Security Process Data 442Technical Controls and Processes 443Administrative Controls 443Account Management 444Management Review and Approval 445Management Reviews for Compliance 446Key Performance and Risk Indicators 447Backup Verification Data 450Training and Awareness 450Disaster Recovery and Business Continuity 451Analyze Test Output and Generate Report 452Typical Audit Report Contents 453Remediation 454Exception Handling 455Ethical Disclosure 456Conduct or Facilitate Security Audits 458Designing an Audit Program 458Internal Audits 459External Audits 460Third-Party Audits 460Summary 461DOMAIN 7: SECURITY OPERATIONS 463Understand and Comply with Investigations 464Evidence Collection and Handling 465Reporting and Documentation 467Investigative Techniques 469Digital Forensics Tools, Tactics, and Procedures 470Artifacts 475Conduct Logging and Monitoring Activities 478Intrusion Detection and Prevention 478Security Information and Event Management 480Continuous Monitoring 481Egress Monitoring 483Log Management 484Threat Intelligence 486User and Entity Behavior Analytics 488Perform Configuration Management 489Provisioning 490Asset Inventory 492Baselining 492Automation 493Apply Foundational Security Operations Concepts 494Need-to-Know/Least Privilege 494Separation of Duties and Responsibilities 495Privileged Account Management 496Job Rotation 498Service-LevelAgreements 498Apply Resource Protection 499Media Management 500Media Protection Techniques 501Conduct Incident Management 502Incident Management Plan 503Detection 505Response 506Mitigation 507Reporting 508Recovery 510Remediation 510Lessons Learned 511Operate and Maintain Detective and Preventative Measures 511Firewalls 512Intrusion Detection Systems and Intrusion Prevention Systems 514Whitelisting/Blacklisting 515Third-Party-Provided Security Services 515Sandboxing 517Honeypots/Honeynets 517Anti-malware 518Machine Learning and Artificial Intelligence Based Tools 518Implement and Support Patch and Vulnerability Management 519Patch Management 519Vulnerability Management 521Understand and Participate in Change Management Processes 522Implement Recovery Strategies 523Backup Storage Strategies 524Recovery Site Strategies 527Multiple Processing Sites 527System Resilience, High Availability, Quality of Service, and Fault Tolerance 528Implement Disaster Recovery Processes 529Response 529Personnel 530Communications 531Assessment 532Restoration 533Training and Awareness 534Lessons Learned 534Test Disaster Recovery Plans 535Read-through/Tabletop 536Walkthrough 536Simulation 537Parallel 537Full Interruption 537Participate in Business Continuity Planning and Exercises 538Implement and Manage Physical Security 539Perimeter Security Controls 541Internal Security Controls 543Address Personnel Safety and Security Concerns 545Travel 545Security Training and Awareness 546Emergency Management 546Duress 547Summary 548DOMAIN 8: SOFTWARE DEVELOPMENT SECURITY 549Understand and Integrate Security in the Software Development Life Cycle (SDLC) 550Development Methodologies 551Maturity Models 561Operation and Maintenance 567Change Management 568Integrated Product Team 571Identify and Apply Security Controls in Software Development Ecosystems 572Programming Languages 572Libraries 577Toolsets 578Integrated Development Environment 579Runtime 580Continuous Integration and Continuous Delivery 581Security Orchestration, Automation, and Response 583Software Configuration Management 585Code Repositories 586Application Security Testing 588Assess the Effectiveness of Software Security 590Auditing and Logging of Changes 590Risk Analysis and Mitigation 595Assess Security Impact of Acquired Software 599Commercial Off-the-Shelf 599Open Source 601Third-Party 602Managed Services (SaaS, IaaS, PaaS) 602Define and Apply Secure Coding Guidelines and Standards 604Security Weaknesses and Vulnerabilities at the Source-Code Level 605Security of Application Programming Interfaces 613API Security Best Practices 613Secure Coding Practices 618Software-Defined Security 621Summary 624Index 625
Data through Movement
WHEN YOU PICTURE HUMAN-DATA INTERACTIONS (HDI), WHAT COMES TO MIND? THE DATAFICATION OF MODERN LIFE, ALONG WITH OPEN DATA INITIATIVES ADVOCATING FOR TRANSPARENCY AND ACCESS TO CURRENT AND HISTORICAL DATASETS, HAS FUNDAMENTALLY TRANSFORMED WHEN, WHERE, AND HOW PEOPLE ENCOUNTER DATA. PEOPLE NOW RELY ON DATA TO MAKE DECISIONS, UNDERSTAND CURRENT EVENTS, AND INTERPRET THE WORLD. WE FREQUENTLY EMPLOY GRAPHS, MAPS, AND OTHER SPATIALIZED FORMS TO AID DATA INTERPRETATION, YET THE FAMILIARITY OF THESE DISPLAYS CAUSES US TO FORGET THAT EVEN BASIC REPRESENTATIONS ARE COMPLEX, CHALLENGING INSCRIPTIONS AND ARE NOT NEUTRAL; THEY ARE BASED ON REPRESENTATIONAL CHOICES THAT IMPACT HOW AND WHAT THEY COMMUNICATE. THIS BOOK DRAWS ON FRAMEWORKS FROM THE LEARNING SCIENCES, VISUALIZATION, AND HUMAN-COMPUTER INTERACTION TO EXPLORE EMBODIED HDI. THIS EXCITING SUB-FIELD OF INTERACTION DESIGN IS BASED ON THE PREMISE THAT EVERY DAY WE PRODUCE AND HAVE ACCESS TO QUINTILLIONS OF BYTES OF DATA, THE EXPLORATION AND ANALYSIS OF WHICH ARE NO LONGER CONFINED WITHIN THE WALLS OF RESEARCH LABORATORIES. THIS VOLUME EXAMINES HOW HUMANS INTERACT WITH THESE DATA IN INFORMAL (NOT WORK OR SCHOOL) ENVIRONMENTS, PARITCULARLY IN MUSEUMS.The first half of the book provides an overview of the multi-disciplinary, theoretical foundations of HDI (in particular, embodied cognition, conceptual metaphor theory, embodied interaction, and embodied learning) and reviews socio-technical theories relevant for designing HDI installations to support informal learning. The second half of the book describes strategies for engaging museum visitors with interactive data visualizations, presents methodologies that can inform the design of hand gestures and body movements for embodied installations, and discusses how HDI can facilitate people's sensemaking about data.This cross-disciplinary book is intended as a resource for students and early-career researchers in human-computer interaction and the learning sciences, as well as for more senior researchers and museum practitioners who want to quickly familiarize themselves with HDI.* Figure Credits List* Foreword by Niklas Elmqvist* Acknowledgments* Introduction* Understanding Human-Data Interaction* Theoretical Foundations: Embodiment* Background: Designing for Learning in Museums* Background: Visualizations to Support Learning* Designing Engaging Human-Data Interactions* Designing Hand Gestures and Body Movements for HDI* Embodiment and Sensemaking* Conclusion* Bibliography* Authors’ Biographies
Hyperreality
What is the similarity between battery chicken, iris scans, Facebook friends, and porn videos? They are all features of a technical system built to satisfy our desires and to suppress our fears. It is a so-called hyperreality, an improved version of natural reality, promising wealth, security, and belonging. However, behind the shiny appearance we can detect a few dangerous mechanisms. Increasingly our tools are controlling us, instead of the other way around, and we are steadily rebuilding the world into a machine with laws we are unable to change. What are the risks of this machine? How can we discern the illusions of hyperreality? With insights derived from Rene Girard and Jacques Ellul, among others, this book calls for a joyful spiritual life, in the midst of stubborn reality. Frank Mulder is a freelance journalist in the Netherlands writing for different magazines and newspapers. With his wife and four children he lives in a community with refugees in a poor neighborhood. He likes technological products like his bicycle, but he has no smartphone.
Computational Analysis and Deep Learning for Medical Care
The book details deep learning models like ANN, RNN, LSTM, in many industrial sectors such as transportation, healthcare, military, agriculture, with valid and effective results, which will help researchers find solutions to their deep learning research problems.We have entered the era of smart world devices, where robots or machines are being used in most applications to solve real-world problems. These smart machines/devices reduce the burden on doctors, which in turn make their lives easier and the lives of their patients better, thereby increasing patient longevity, which is the ultimate goal of computer vision. Therefore, the goal in writing this book is to attempt to provide complete information on reliable deep learning models required for e-healthcare applications. Ways in which deep learning can enhance healthcare images or text data for making useful decisions are discussed. Also presented are reliable deep learning models, such as neural networks, convolutional neural networks, backpropagation, and recurrent neural networks, which are increasingly being used in medical image processing, including for colorization of black and white X-ray images, automatic machine translation images, object classification in photographs/images (CT scans), character or useful generation (ECG), image caption generation, etc. Hence, reliable deep learning methods for the perception or production of better results are a necessity for highly effective e-healthcare applications. Currently, the most difficult data-related problem that needs to be solved concerns the rapid increase of data occurring each day via billions of smart devices. To address the growing amount of data in healthcare applications, challenges such as not having standard tools, efficient algorithms, and a sufficient number of skilled data scientists need to be overcome. Hence, there is growing interest in investigating deep learning models and their use in e-healthcare applications.AUDIENCEResearchers in artificial intelligence, big data, computer science, and electronic engineering, as well as industry engineers in transportation, healthcare, biomedicine, military, agriculture.AMIT KUMAR TYAGI is an assistant professor and senior researcher at Vellore Institute of Technology (VIT), Chennai Campus, India. He received his PhD in 2018 from Pondicherry Central University, India. He has published more than 8 patents in the area of deep learning, Internet of Things, cyber physical systems and computer vision.Preface xixPART I: DEEP LEARNING AND ITS MODELS 11 CNN: A REVIEW OF MODELS, APPLICATION OF IVD SEGMENTATION 3Leena Silvoster M. and R. Mathusoothana S. Kumar1.1 Introduction 41.2 Various CNN Models 41.2.1 LeNet-5 41.2.2 AlexNet 71.2.3 ZFNet 81.2.4 VGGNet 101.2.5 GoogLeNet 121.2.6 ResNet 161.2.7 ResNeXt 211.2.8 SE-ResNet 241.2.9 DenseNet 241.2.10 MobileNets 251.3 Application of CNN to IVD Detection 261.4 Comparison With State-of-the-Art Segmentation Approaches for Spine T2W Images 281.5 Conclusion 28References 332 LOCATION-AWARE KEYWORD QUERY SUGGESTION TECHNIQUES WITH ARTIFICIAL INTELLIGENCE PERSPECTIVE 35R. Ravinder Reddy, C. Vaishnavi, Ch. Mamatha and S. Ananthakumaran2.1 Introduction 362.2 Related Work 392.3 Artificial Intelligence Perspective 412.3.1 Keyword Query Suggestion 422.3.1.1 Random Walk–Based Approaches 422.3.1.2 Cluster-Based Approaches 422.3.1.3 Learning to Rank Approaches 432.3.2 User Preference From Log 432.3.3 Location-Aware Keyword Query Suggestion 442.3.4 Enhancement With AI Perspective 442.3.4.1 Case Study 452.4 Architecture 462.4.1 Distance Measures 472.5 Conclusion 49References 493 IDENTIFICATION OF A SUITABLE TRANSFER LEARNING ARCHITECTURE FOR CLASSIFICATION: A CASE STUDY WITH LIVER TUMORS 53B. Lakshmi Priya, K. Jayanthi, Biju Pottakkat and G. Ramkumar3.1 Introduction 543.2 Related Works 563.3 Convolutional Neural Networks 583.3.1 Feature Learning in CNNs 593.3.2 Classification in CNNs 603.4 Transfer Learning 613.4.1 AlexNet 613.4.2 GoogLeNet 623.4.3 Residual Networks 633.4.3.1 ResNet-18 653.4.3.2 ResNet-50 653.5 System Model 663.6 Results and Discussions 673.6.1 Dataset 673.6.2 Assessment of Transfer Learning Architectures 673.7 Conclusion 73References 744 OPTIMIZATION AND DEEP LEARNING-BASED CONTENT RETRIEVAL, INDEXING, AND METRIC LEARNING APPROACH FOR MEDICAL IMAGES 79Suresh Kumar K., Sundaresan S., Nishanth R. and Ananth Kumar T.4.1 Introduction 804.2 Related Works 824.3 Proposed Method 854.3.1 Input Dataset 864.3.2 Pre-Processing 864.3.3 Combination of DCNN and CFML 864.3.4 Fine Tuning and Optimization 884.3.5 Feature Extraction 894.3.6 Localization of Abnormalities in MRI and CT Scanned Images 904.4 Results and Discussion 924.4.1 Metric Learning 924.4.2 Comparison of the Various Models for Image Retrieval 924.4.3 Precision vs. Recall Parameters Estimation for the CBIR 934.4.4 Convolutional Neural Networks–Based Landmark Localization 964.5 Conclusion 104References 104PART II: APPLICATIONS OF DEEP LEARNING 1075 DEEP LEARNING FOR CLINICAL AND HEALTH INFORMATICS 109Amit Kumar Tyagi and Meghna Mannoj Nair5.1 Introduction 1105.1.1 Deep Learning Over Machine Learning 1115.2 Related Work 1135.3 Motivation 1155.4 Scope of the Work in Past, Present, and Future 1155.5 Deep Learning Tools, Methods Available for Clinical, and Health Informatics 1175.6 Deep Learning: Not-So-Near Future in Biomedical Imaging 1195.6.1 Types of Medical Imaging 1195.6.2 Use and Benefits of Medical Imaging 1205.7 Challenges Faced Toward Deep Learning Using Biomedical Imaging 1215.7.1 Deep Learning in Healthcare: Limitations and Challenges 1225.8 Open Research Issues and Future Research Directions Biomedical Imaging (Healthcare Informatics) 1245.9 Conclusion 127References 1276 BIOMEDICAL IMAGE SEGMENTATION BY DEEP LEARNING METHODS 131K. Anita Davamani, C.R. Rene Robin, S. Amudha and L. Jani Anbarasi6.1 Introduction 1326.2 Overview of Deep Learning Algorithms 1356.2.1 Deep Learning Classifier (DLC) 1366.2.2 Deep Learning Architecture 1376.3 Other Deep Learning Architecture 1396.3.1 Restricted Boltzmann Machine (RBM) 1396.3.2 Deep Learning Architecture Containing Autoencoders 1406.3.3 Sparse Coding Deep Learning Architecture 1416.3.4 Generative Adversarial Network (GAN) 1416.3.5 Recurrent Neural Network (RNN) 1416.4 Biomedical Image Segmentation 1456.4.1 Clinical Images 1466.4.2 X-Ray Imaging 1466.4.3 Computed Tomography (CT) 1476.4.4 Magnetic Resonance Imaging (MRI) 1476.4.5 Ultrasound Imaging (US) 1486.4.6 Optical Coherence Tomography (OCT) 1486.5 Conclusion 149References 1497 MULTI-LINGUAL HANDWRITTEN CHARACTER RECOGNITION USING DEEP LEARNING 155Giriraj Parihar, Ratnavel Rajalakshmi and Bhuvana J.7.1 Introduction 1567.2 Related Works 1577.3 Materials and Methods 1607.4 Experiments and Results 1617.4.1 Dataset Description 1627.4.1.1 Handwritten Math Symbols 1627.4.1.2 Bangla Handwritten Character Dataset 1627.4.1.3 Devanagari Handwritten Character Dataset 1627.4.2 Experimental Setup 1627.4.3 Hype-Parameters 1647.4.3.1 English Model 1647.4.3.2 Hindi Model 1657.4.3.3 Bangla Model 1657.4.3.4 Math Symbol Model 1657.4.3.5 Combined Model 1667.4.4 Results and Discussion 1677.4.4.1 Performance of Uni-Language Models 1677.4.4.2 Uni-Language Model on English Dataset 1687.4.4.3 Uni-Language Model on Hindi Dataset 1687.4.4.4 Uni-Language Model on Bangla Dataset 1697.4.4.5 Uni-Language Model on Math Symbol Dataset 1697.4.4.6 Performance of Multi-Lingual Model on Combined Dataset 1717.5 Conclusion 177References 1788 DISEASE DETECTION PLATFORM USING IMAGE PROCESSING THROUGH OPENCV 181Neetu Faujdar and Aparna Sinha8.1 Introduction 1828.1.1 Image Processing 1838.2 Problem Statement 1838.2.1 Cataract 1838.2.1.1 Causes 1848.2.1.2 Types of Cataracts 1848.2.1.3 Cataract Detection 1858.2.1.4 Treatment 1868.2.1.5 Prevention 1868.2.1.6 Methodology 1868.2.2 Eye Cancer 1928.2.2.1 Symptoms 1948.2.2.2 Causes of Retinoblastoma 1948.2.2.3 Phases 1958.2.2.4 Spreading of Cancer 1968.2.2.5 Diagnosis 1968.2.2.6 Treatment 1978.2.2.7 Methodology 1998.2.3 Skin Cancer (Melanoma) 2028.2.3.1 Signs and Symptoms 2038.2.3.2 Stages 2038.2.3.3 Causes of Melanoma 2048.2.3.4 Diagnosis 2048.2.3.5 Treatment 2058.2.3.6 Methodology 2068.2.3.7 Asymmetry 2078.2.3.8 Border 2088.2.3.9 Color 2088.2.3.10 Diameter Detection 2098.2.3.11 Calculating TDS (Total Dermoscopy Score) 2108.3 Conclusion 2108.4 Summary 212References 2129 COMPUTER-AIDED DIAGNOSIS OF LIVER FIBROSIS IN HEPATITIS PATIENTS USING CONVOLUTIONAL NEURAL NETWORK 217Aswathy S. U., Ajesh F., Shermin Shamsudheen and Jarin T.9.1 Introduction 2189.2 Overview of System 2199.3 Methodology 2199.3.1 Dataset 2209.3.2 Pre-Processing 2219.3.3 Feature Extraction 2219.3.4 Feature Selection and Normalization 2239.3.5 Classification Model 2259.4 Performance and Analysis 2279.5 Experimental Results 2329.6 Conclusion and Future Scope 232References 233PART III: FUTURE DEEP LEARNING MODELS 23710 LUNG CANCER PREDICTION IN DEEP LEARNING PERSPECTIVE 239Nikita Banerjee and Subhalaxmi Das10.1 Introduction 23910.2 Machine Learning and Its Application 24010.2.1 Machine Learning 24010.2.2 Different Machine Learning Techniques 24110.2.2.1 Decision Tree 24210.2.2.2 Support Vector Machine 24210.2.2.3 Random Forest 24210.2.2.4 K-Means Clustering 24210.3 Related Work 24310.4 Why Deep Learning on Top of Machine Learning? 24510.4.1 Deep Neural Network 24610.4.2 Deep Belief Network 24710.4.3 Convolutional Neural Network 24710.5 How is Deep Learning Used for Prediction of Lungs Cancer? 24810.5.1 Proposed Architecture 24810.5.1.1 Pre-Processing Block 25010.5.1.2 Segmentation 25010.5.1.3 Classification 25210.6 Conclusion 253References 25311 LESION DETECTION AND CLASSIFICATION FOR BREAST CANCER DIAGNOSIS BASED ON DEEP CNNS FROM DIGITAL MAMMOGRAPHIC DATA 257Diksha Rajpal, Sumita Mishra and Anil Kumar11.1 Introduction 25711.2 Background 25811.2.1 Methods of Diagnosis of Breast Cancer 25811.2.2 Types of Breast Cancer 26011.2.3 Breast Cancer Treatment Options 26111.2.4 Limitations and Risks of Diagnosis and Treatment Options 26211.2.4.1 Limitation of Diagnosis Methods 26211.2.4.2 Limitations of Treatment Plans 26311.2.5 Deep Learning Methods for Medical Image Analysis: Tumor Classification 26311.3 Methods 26511.3.1 Digital Repositories 26511.3.1.1 DDSM Database 26511.3.1.2 AMDI Database 26511.3.1.3 IRMA Database 26511.3.1.4 BreakHis Database 26511.3.1.5 MIAS Database 26611.3.2 Data Pre-Processing 26611.3.2.1 Advantages of Pre-Processing Images 26711.3.3 Convolutional Neural Networks (CNNs) 26811.3.3.1 Architecture of CNN 26911.3.4 Hyper-Parameters 27211.3.4.1 Number of Hidden Layers 27311.3.4.2 Dropout Rate 27311.3.4.3 Activation Function 27311.3.4.4 Learning Rate 27411.3.4.5 Number of Epochs 27411.3.4.6 Batch Size 27411.3.5 Techniques to Improve CNN Performance 27411.3.5.1 Hyper-Parameter Tuning 27411.3.5.2 Augmenting Images 27411.3.5.3 Managing Over-Fitting and Under-Fitting 27511.4 Application of Deep CNN for Mammography 27511.4.1 Lesion Detection and Localization 27511.4.2 Lesion Classification 27911.5 System Model and Results 28011.5.1 System Model 28011.5.2 System Flowchart 28111.5.2.1 MIAS Database 28111.5.2.2 Unannotated Images 28111.5.3 Results 28211.5.3.1 Distribution and Processing of Dataset 28211.5.3.2 Training of the Model 28311.5.3.3 Prediction of Unannotated Images 28611.6 Research Challenges and Discussion on Future Directions 28611.7 Conclusion 288References 28912 HEALTH PREDICTION ANALYTICS USING DEEP LEARNING METHODS AND APPLICATIONS 293Sapna Jain, M. Afshar Alam, Nevine Makrim Labib and Eiad Yafi12.1 Introduction 29412.2 Background 29812.3 Predictive Analytics 29912.4 Deep Learning Predictive Analysis Applications 30512.4.1 Deep Learning Application Model to Predict COVID-19 Infection 30512.4.2 Deep Transfer Learning for Mitigating the COVID-19 Pandemic 30812.4.3 Health Status Prediction for the Elderly Based on Machine Learning 30912.4.4 Deep Learning in Machine Health Monitoring 31112.5 Discussion 31912.6 Conclusion 320References 32113 AMBIENT-ASSISTED LIVING OF DISABLED ELDERLY IN AN INTELLIGENT HOME USING BEHAVIOR PREDICTION—A RELIABLE DEEP LEARNING PREDICTION SYSTEM 329Sophia S., Sridevi U.K., Boselin Prabhu S.R. and P. Thamaraiselvi13.1 Introduction 33013.2 Activities of Daily Living and Behavior Analysis 33113.3 Intelligent Home Architecture 33313.4 Methodology 33513.4.1 Record the Behaviors Using Sensor Data 33513.4.2 Classify Discrete Events and Relate the Events Using Data Analysis Algorithms 33513.4.3 Construct Behavior Dictionaries for Flexible Event Intervals Using Deep Learning Concepts 33513.4.4 Use the Dictionary in Modeling the Behavior Patterns Through Prediction Techniques 33613.4.5 Detection of Deviations From Expected Behaviors Aiding the Automated Elderly Monitoring Based on Decision Support Algorithm Systems 33613.5 Senior Analytics Care Model 33713.6 Results and Discussions 33813.7 Conclusion 341Nomenclature 341References 34214 EARLY DIAGNOSIS TOOL FOR ALZHEIMER’S DISEASE USING 3D SLICER 343V. Krishna Kumar, M.S. Geetha Devasena and G. Gopu14.1 Introduction 34414.2 Related Work 34514.3 Existing System 34714.4 Proposed System 34714.4.1 Usage of 3D Slicer 35014.5 Results and Discussion 35314.6 Conclusion 356References 356PART IV: DEEP LEARNING – IMPORTANCE AND CHALLENGES FOR OTHER SECTORS 36115 DEEP LEARNING FOR MEDICAL HEALTHCARE: ISSUES, CHALLENGES, AND OPPORTUNITIES 363Meenu Gupta, Akash Gupta and Gaganjot Kaur15.1 Introduction 36415.2 Related Work 36515.3 Development of Personalized Medicine Using Deep Learning: A New Revolution in Healthcare Industry 36715.3.1 Deep Feedforward Neural Network (DFF) 36715.3.2 Convolutional Neural Network 36715.3.3 Recurrent Neural Network (RNN) 36915.3.4 Long/Short-Term Memory (LSTM) 36915.3.5 Deep Belief Network (DBN) 37015.3.6 Autoencoder (AE) 37015.4 Deep Learning Applications in Precision Medicine 37015.4.1 Discovery of Biomarker and Classification of Patient 37015.4.2 Medical Imaging 37115.5 Deep Learning for Medical Imaging 37215.5.1 Medical Image Detection 37215.5.1.1 Pathology Detection 37215.5.1.2 Detection of Image Plane 37315.5.1.3 Anatomical Landmark Localization 37315.5.2 Medical Image Segmentation 37315.5.2.1 Supervised Algorithms 37415.5.2.2 Semi-Supervised Algorithms 37415.5.3 Medical Image Enhancement 37515.5.3.1 Two-Dimensional Super-Resolution Techniques 37515.5.3.2 Three-Dimensional Super-Resolution Techniques 37515.6 Drug Discovery and Development: A Promise Fulfilled by Deep Learning Technology 37515.6.1 Prediction of Drug Properties 37615.6.2 Prediction of Drug-Target Interaction 37715.7 Application Areas of Deep Learning in Healthcare 37715.7.1 Medical Chatbots 37715.7.2 Smart Health Records 37715.7.3 Cancer Diagnosis 37815.8 Privacy Issues Arising With the Usage of Deep Learning in Healthcare 37915.8.1 Private Data 37915.8.2 Privacy Attacks 38015.8.2.1 Evasion Attack 38015.8.2.2 White-Box Attack 38015.8.2.3 Black-Box Attack 38015.8.2.4 Poisoning Attack 38115.8.3 Privacy-Preserving Techniques 38115.8.3.1 Differential Privacy With Deep Learning 38115.8.3.2 Homomorphic Encryption (HE) on Deep Learning 38215.8.3.3 Secure Multiparty Computation on Deep Learning 38315.9 Challenges and Opportunities in Healthcare Using Deep Learning 38315.10 Conclusion and Future Scope 386References 38716 A PERSPECTIVE ANALYSIS OF REGULARIZATION AND OPTIMIZATION TECHNIQUES IN MACHINE LEARNING 393Ajeet K. Jain, PVRD Prasad Rao and K. Venkatesh Sharma16.1 Introduction 39416.1.1 Data Formats 39516.1.1.1 Structured Data 39516.1.1.2 Unstructured Data 39616.1.1.3 Semi-Structured Data 39616.1.2 Beginning With Learning Machines 39716.1.2.1 Perception 39716.1.2.2 Artificial Neural Network 39816.1.2.3 Deep Networks and Learning 39916.1.2.4 Model Selection, Over-Fitting, and Under-Fitting 40016.2 Regularization in Machine Learning 40216.2.1 Hamadard Conditions 40316.2.2 Tikhonov Generalized Regularization 40416.2.3 Ridge Regression 40616.2.4 Lasso—L1 Regularization 40616.2.5 Dropout as Regularization Feature 40716.2.6 Augmenting Dataset 40816.2.7 Early Stopping Criteria 40816.3 Convexity Principles 40916.3.1 Convex Sets 41016.3.1.1 Affine Set and Convex Functions 41116.3.1.2 Properties of Convex Functions 41116.3.2 Optimization and Role of Optimizer in ML 41316.3.2.1 Gradients-Descent Optimization Methods 41416.3.2.2 Non-Convexity of Cost Functions 41616.3.2.3 Basic Maths of SGD 41816.3.2.4 Saddle Points 41816.3.2.5 Gradient Pointing in the Wrong Direction 42016.3.2.6 Momentum-Based Optimization 42316.4 Conclusion and Discussion 424References 42517 DEEP LEARNING-BASED PREDICTION TECHNIQUES FOR MEDICAL CARE: OPPORTUNITIES AND CHALLENGES 429S. Subasree and N. K. Sakthivel17.1 Introduction 43017.2 Machine Learning and Deep Learning Framework 43117.2.1 Supervised Learning 43317.2.2 Unsupervised Learning 43317.2.3 Reinforcement Learning 43417.2.4 Deep Learning 43417.3 Challenges and Opportunities 43517.3.1 Literature Review 43517.4 Clinical Databases—Electronic Health Records 43617.5 Data Analytics Models—Classifiers and Clusters 43617.5.1 Criteria for Classification 43817.5.1.1 Probabilistic Classifier 43917.5.1.2 Support Vector Machines (SVMs) 43917.5.1.3 K-Nearest Neighbors 44017.5.2 Criteria for Clustering 44117.5.2.1 K-Means Clustering 44217.5.2.2 Mean Shift Clustering 44217.6 Deep Learning Approaches and Association Predictions 44417.6.1 G-HR: Gene Signature–Based HRF Cluster 44417.6.1.1 G-HR Procedure 44617.6.2 Deep Learning Approach and Association Predictions 44617.6.2.1 Deep Learning Approach 44617.6.2.2 Intelligent Human Disease-Gene Association Prediction Technique (IHDGAP) 44717.6.2.3 Convolution Neural Network 44717.6.2.4 Disease Semantic Similarity 44917.6.2.5 Computation of Scoring Matrix 45017.6.3 Identified Problem 45017.6.4 Deep Learning–Based Human Diseases Pattern Prediction Technique for High-Dimensional Human Diseases Datasets (ECNN-HDPT) 45117.6.5 Performance Analysis 45317.7 Conclusion 45717.8 Applications 458References 45918 MACHINE LEARNING AND DEEP LEARNING: OPEN ISSUES AND FUTURE RESEARCH DIRECTIONS FOR THE NEXT 10 YEARS 463Akshara Pramod, Harsh Sankar Naicker and Amit Kumar Tyagi18.1 Introduction 46418.1.1 Comparison Among Data Mining, Machine Learning, and Deep Learning 46518.1.2 Machine Learning 46518.1.2.1 Importance of Machine Learning in Present Business Scenario 46718.1.2.2 Applications of Machine Learning 46718.1.2.3 Machine Learning Methods Used in Current Era 46918.1.3 Deep Learning 47118.1.3.1 Applications of Deep Learning 47118.1.3.2 Deep Learning Techniques/Methods Used in Current Era 47318.2 Evolution of Machine Learning and Deep Learning 47518.3 The Forefront of Machine Learning Technology 47618.3.1 Deep Learning 47618.3.2 Reinforcement Learning 47718.3.3 Transfer Learning 47718.3.4 Adversarial Learning 47718.3.5 Dual Learning 47818.3.6 Distributed Machine Learning 47818.3.7 Meta Learning 47818.4 The Challenges Facing Machine Learning and Deep Learning 47818.4.1 Explainable Machine Learning 47918.4.2 Correlation and Causation 47918.4.3 Machine Understands the Known and is Aware of the Unknown 47918.4.4 People-Centric Machine Learning Evolution 48018.4.5 Explainability: Stems From Practical Needs and Evolves Constantly 48018.5 Possibilities With Machine Learning and Deep Learning 48118.5.1 Possibilities With Machine Learning 48118.5.1.1 Lightweight Machine Learning and Edge Computing 48118.5.1.2 Quantum Machine Learning 48218.5.1.3 Quantum Machine Learning Algorithms Based on Linear Algebra 48218.5.1.4 Quantum Reinforcement Learning 48318.5.1.5 Simple and Elegant Natural Laws 48318.5.1.6 Improvisational Learning 48418.5.1.7 Social Machine Learning 48518.5.2 Possibilities With Deep Learning 48518.5.2.1 Quantum Deep Learning 48518.6 Potential Limitations of Machine Learning and Deep Learning 48618.6.1 Machine Learning 48618.6.2 Deep Learning 48718.7 Conclusion 488Acknowledgement 489Contribution/Disclosure 489References 489Index 491
Cyber Threat Intelligence
Understand the process of setting up a successful cyber threat intelligence (CTI) practice within an established security team. This book shows you how threat information that has been collected, evaluated, and analyzed is a critical component in protecting your organization’s resources. Adopting an intelligence-led approach enables your organization to nimbly react to situations as they develop. Security controls and responses can then be applied as soon as they become available, enabling prevention rather than response.There are a lot of competing approaches and ways of working, but this book cuts through the confusion. Author Aaron Roberts introduces the best practices and methods for using CTI successfully. This book will help not only senior security professionals, but also those looking to break into the industry. You will learn the theories and mindset needed to be successful in CTI.This book covers the cybersecurity wild west, the merits and limitations of structured intelligence data, and how using structured intelligence data can, and should, be the standard practice for any intelligence team. You will understand your organizations’ risks, based on the industry and the adversaries you are most likely to face, the importance of open-source intelligence (OSINT) to any CTI practice, and discover the gaps that exist with your existing commercial solutions and where to plug those gaps, and much more.WHAT YOU WILL LEARN* Know the wide range of cybersecurity products and the risks and pitfalls aligned with blindly working with a vendor* Understand critical intelligence concepts such as the intelligence cycle, setting intelligence requirements, the diamond model, and how to apply intelligence to existing security information* Understand structured intelligence (STIX) and why it’s important, and aligning STIX to ATT&CK and how structured intelligence helps improve final intelligence reporting* Know how to approach CTI, depending on your budget* Prioritize areas when it comes to funding and the best approaches to incident response, requests for information, or ad hoc reporting* Critically evaluate services received from your existing vendors, including what they do well, what they don’t do well (or at all), how you can improve on this, the things you should consider moving in-house rather than outsourcing, and the benefits of finding and maintaining relationships with excellent vendorsWHO THIS BOOK IS FORSenior security leaders in charge of cybersecurity teams who are considering starting a threat intelligence team, those considering a career change into cyber threat intelligence (CTI) who want a better understanding of the main philosophies and ways of working in the industry, and security professionals with no prior intelligence experience but have technical proficiency in other areas (e.g., programming, security architecture, or engineering)AARON ROBERTS is an intelligence professional specializing in Cyber Threat Intelligence (CTI) and Open-Source Intelligence (OSINT). He is focused on building intelligence-led cyber capabilities in large enterprises and conducting online investigations and research. He has worked within several the public and private sectors as well as the British Military. As such he understands how intelligence can and should be utilized within a range of environments and the fundamental approach that businesses must take to get the maximum value out of their cyber threat intelligence program.CHAPTER 1: INTRODUCTIONThis chapter is designed to introduce the reader to me, why I’m knowledgeable on the subject and to set the expectations of what they’ll learn throughout the book.CHAPTER 2: THE CYBERSECURITY WILD WESTThis chapter discusses the wide-range of cybersecurity products and understanding the risks and pitfalls aligned with blindly working with a vendor. How to understand what you get for your money and how to get the most out of any commercial partnerships you enter into.CHAPTER 3: CYBER THREAT INTELLIGENCE – WHAT DOES IT EVEN MEAN?This chapter discusses critical intelligence concepts such as the intelligence cycle, setting intelligence requirements, the diamond model and how we apply intelligence to existing security information (by way of Mitre ATT&CK).CHAPTER 4: STRUCTURED INTELLIGENCE – WHAT’S THE POINT?This chapter builds on chapter 3, and discusses the benefits of adding structure to intelligence data. We’ll discuss STIX and why it’s important, aligning STIX to ATT&CK and how structured intelligence helps improve final intelligence reporting.CHAPTER 5: DETERMINING WHAT YOUR BUSINESS NEEDSThis chapter will look at how to approach CTI depending on your budget, the business itself (and its underlying sector/industry), what already exists within the organization and how you could expand and automate some aspects of the collection.CHAPTER 6: HOW CAN I IMPLEMENT THIS? (NO MATTER WHAT BUDGET YOU HAVE)This chapter will look at the main factors of CTI, accepting what gaps might exist (if you have no budget), and how you could potentially consider trying to fill them. We’ll discuss how to priorities areas when it comes to funding and the best approaches to incident response, requests for information or ad-hoc reporting.CHAPTER 7: THINGS TO CONSIDER WHEN IMPLEMENTING CTIThis chapter will look at an organizations footprint and understanding the risks associated with your organization—the gaps left by funding or vendor/IT black holes in your estate and staffing and resourcing.CHAPTER 8: THE IMPORTANCE OF OSINTOpen-Source Intelligence is a significant part of a successful CTI practice. This chapter will look at what OSINT is (and can be), what an analyst or investigator needs in terms of necessary tooling to succeed, how to create and maintain accounts for research purposes and what to do if you can’t immediately employ Human Intelligence (HUMINT) into your collection.CHAPTER 9: I ALREADY PAY FOR VENDOR X. SHOULD I BOTHER?This chapter is designed to assist the reader in critically evaluating the service they receive from their existing vendors. This includes what they do well, what they don’t do well (or at all), how you can improve on this, what things you should consider moving in-house rather than outsourcing, and the benefits of finding and maintaining relationships with excellent vendors.CHAPTER 10: SUMMARYThis chapter will summaries the main themes discussed in each chapter. The next steps that should be imperative to any organization, how the reader could follow up with me for any questions or comments, and if they can’t do anything today, what they should take away from the book to try and improve their CTI practice.CHAPTER 11: USEFUL RESOURCESThis chapter will list several useful resources the reader could investigate to help them on their way to set up a successful CTI team, broken down into sub-headings.