AR and VR Using the WebXR API

Gain an in-depth knowledge in immersive web development to create augmented reality (AR) and virtual reality (VR) applications inside web browsers using WebXR API, WebGL, Three.js, and A-Frame. This project-based book will provide the practice and portfolio content to make the most of what the futures of spatial computing and immersive technology have to offer.

Beginning with technical analysis of how web browsers function, the book covers programming languages such as WebGL, JavaScript, and HTML, with an eye on a complete understanding of the WebXR lifecycle. You'll then explore how contemporary web browsers work at the code level and see how to set up a local development server and use it with the Visual Studio Code IDE to create 3D animation in the WebGL programming language.

With a familiarity of the web-rendering pipeline in place, you’ll venture on to WebGL abstractions such as the Three.js JavaScript library and Mozilla’s A-Frame XR Framework, which use WebXR to create high-end visual effects. In the final projects of the book, you’ll create an augmented reality web session for an Android phone device, and create a VR scene in A-Frame (built on Three.js) to demo essential components of the WebXR API pertaining to user positioning and interaction.

Game engines have become common-place for the creation of mixed reality content. However, developers not interested in learning entirely new workflows may be better suited to work within a medium almost universally open to all—the web; AR and VR Using the WebXR API will show you the way.

WHAT YOU'LL LEARN

* Master the creation of virtual reality and augmented reality features for web page
* Prepare to work as an immersive web developer with a portfolio of projects in sought-after technologies
* Review the fundamentals of writing shaders in WebGL
* Experience the unity between client, server, and cloud architecture as it applies to location-based AR

WHO THIS BOOK IS FOR

Aspiring immersive web developers and developers already familiar with the fundamentals of web development who want to further explore topics such as spatial computing, computer vision, spatial anchors, and cloud-computing for multi-user social experiences. Rakesh Baruah is a writer and creator with 15 years of experience in new media, film, and television in New York City. After completing an MFA in screenwriting and directing for film from Columbia University, Rakesh joined the writers’ room of a hit, primetime, network drama as an assistant. The experience opened his eyes to the limits of television and the opportunities promised by 3D, immersive content. In 2016 he began a self-guided journey toward mixed reality design that has taken him through startups, bootcamps, the Microsoft offices, and many, many hours in front of a computer. He is the author of one previous book on virtual reality and the Unity Game Engine and has received an Nvidia-certified nanodegree in Computer Vision. He shares what he’s learned with you in a style and format designed specifically for the person who, in high school, preferred English class to Trigonometry. Chapter 1: Introduction

Sub-Topics:

- The Components of WebXR

• Pose tracking

• Camera data

• Scene understanding- Hit-Testing

- Anchors

- Point-clouds

- Surfaces

- light-estimation

• The webXR API

- WebGL 1 → WebGL 2 → WebGPU → WebVR → WebXR

- Khronos Group and WC3

• The webXR emulator

• Industry Standard XR Guidelines (Microsoft, Google)

Chapter 2: Setup

Sub-Topics:

- Types of Browsers and Requirements

- Chrome Canary

- Firefox Reality

- Oculus Browser

• How to setup a local server for development

- Node, Python, Servez

• Local machine setup

- Download and install Visual Studio Code

- Create GitHub account

• Playgrounds vs. Local development while executive the code in the book

Chapter 3: Introduction to Web Browsers

Sub-Topics:

• The Workings of the World Wide Web

- Client - Server relationship

- HTML, CSS, and Javascript

• The Workings of a Web Page

- The Document Object Model

- The Render Engine

- HTML5 and Canvas

• Toward Native Code in the Browser

- WebAssembly

Physics and the GPU

Chapter 4: A Brief Introduction to WebGL

Sub-Topics:

• The Big Picture of WebGL

- A Crash Course in 3D Graphics

- The Graphics Rendering → Rasterizing Pipeline

• The Difference Between WebGL and JavaScript

- CPU vs GPU

- Browser vs Native

- The Value of Many Threads

• The Components of a WebGL Application

- Vertex and Fragment Shaders

- Uniforms, Attributes, and Buffers- ViewMatrix, ProjectionMatrix, and the WebGL Matrix Math Library

• The Cognitive Dissonance between WebGL and Web Programmers

Chapter 5: WebXR Libraries -- Three.js

Sub-Topics:

• An Overview of JavaScript Libraries• Introduction to Three.js, which Makes Programming WebGL Easier

• Async and Await vs Promises in JavaScript

• JavaScript and the GPU

Chapter 6: WebXR Frameworks: Mozilla’s A-Frame

Sub-Topics:

• A-Frame makes Three.js easier

• The Components of A-Frame:

- Scenes, Cameras, Objects, Interactions

- Light and Shadows

- 3D Objects and the gLTF file format

Chapter 7: WebXR Engines: Babylon.js

Sub-Topics:

• TypeScript vs JavaScript

- Type safety

- Parallel Processing

• The Babylon.js Tools

- Playground

- Inspector

- Node Creator

• What Makes an Engine vs. a Framework?

- The role of physics in XR creation

Chapter 8: Web Augmented Reality in Chrome CanarySub-Topics:

• An Overview of the WebXR Features in the Chrome Canary XR API

- Hit-Testing

- Camera access

• Accessing Developer Features in Chrome

• The Browser and Hardware Connection

- Final Project: Client-Server database connection for data persistence in AR

Chapter 9: The Future of WebXR

Sub-Topics:

• Computer Vision and WebXR

- Facial Recognition and Filters

• Multi-user Interaction

- Spatial / Cloud Anchors

- Social XR

• Hand Gestures and Voice Commands

- Motion Tracking

• Cloud-Computing and Privacy- The ethical responsibility of the Immersive Web developer