In 2017, frog was asked to provide a better understanding of use cases and microservices for Edge Computing in the form of functional edge computing demonstrators across AR maintenance, Robotics and autonomous vehicles. I built the HoloLens app for the AR maintenance demo as well as the data screens in OpenGL and contributed to the autonomous vehicle demo. frog fielded a team of talented designers and technologists to bring the demonstrators to life across industrial, interaction and visual design as well as frontend, backend and custom PCB technology development.
- Technology: HoloLens, Unity, Vuforia, Microservices, Python, C++/OpenGL, Openframeworks, Arduino, Digital and Physical Prototyping and Production, Interactive Installation
The applications demonstrate five key aspects of Edge technology that can be leveraged independently or in combination to create meaningful and transformative application experiences.
-
Realtime data – Edge increases application responsiveness (i.e. through decreased latency) to be at or below average human perception of latency to create on-demand, real-time, mobile application experiences.
-
Compound Latency – Because Edge increases network responsiveness, it proves valuable in use-cases where variable (or human-based) instructions have to be executed in sequence.
-
Network Variance/Jitter – Edge increases network consistency and proves valuable in use-cases that are critically impacted when a traditional networks are under duress.
-
IoT Orchestration – When Edge technology is integrated with widespread local sensors, safer and more efficient automated IoT systems are enabled.
-
Virtualized Machine Infrastructure – Because Edge brings compute power closer to the user, computation and rendering can be shifted from the device to the network.
Equipping workers with rich, timely, and contextual information will help prevent critical and costly failure where human intervention is required. To meet the demands of collecting data across a network of distributed sensors (perhaps not even in the same geographic region), and then delivering that data in near real-time to a mobile device such as an augmented reality headset, requires network reliability and speed that only Edge technology can guarantee.
In this application, an industrial system is unbalanced and requires human intervention to fix it. Data sent to an augmented reality headset allows the wearer to know both what position the valve is and what varying target position the valve needs to match. However, the position of the valve is broadcasted to the headset via a wireless network, which is subject to delay caused by latency and network variance.
Engaging Edge via voice command (“Edge on”) diminishes any delay caused by the network, allowing the user to match the target position with much less difficulty, and therefore resolve the imbalance in the system.
The images above show some of the build process of the AR maintenance application while integrating on premises industrial sensors with a HoloLens headset and edge enabled microservices. The data readings would output network variance and jitter data to show additional viewers what was happening as the experience progressed.
In this application, the three cars on the track represent a larger network of connected autonomous vehicles. The cars travel at a user-specified speed, calculating the safest distance they must travel from the car in front of them based on the speed. By connecting the cars through the cloud, the safest speed becomes a function of how fast the vehicles can orchestrate changes amongst themselves from specific input commands. When vehicles are connected through a low latency and stable network (little to no network variance or jitter) like Edge, they are able to process given commands quickly and as expected. When high network variance is present, the vehicles need to account for this unreliability, which results in larger distances between the cars to maintain safety. Edge reduces network variance considerably, allowing the cars to travel closer together, increasing efficiency while still maintaining safety.
In this application, one robotic arm is enabled by standard cloud topology, while the other is enabled by Edge technology. A single ball sits on the plane beneath the arms. Each arm contains a camera and a spotlight, both aimed at the ball below. (The Edge enabled arm projects a magenta light, while the standard network topology enabled arm projects a white light.) Video output from each camera is shown on a corresponding screen. The user can move the ball at will using the provided control pad. The robotic arms are autonomous and will attempt to track the ball as it moves, using positional coordinates sent over the network to each arm which are subject to latency and network variance. With Edge, these factors are significantly reduced, enabling the precision and responsiveness expected of a robotic application (despite offloading instructions to a remote source). The performance of each arm can be seen by how closely the magenta (Edge) and white (non-Edge) spotlights align with the ball as it moves.
Additional shot of the AR installation form factor, data screen and HoloLens mounting apparatus.
Credits:
- Tech: AR HoloLens, Data Screens, Auto v2 – Charles Yust
- Tech: Microservices, Robotics and Auto Circuit Design – Anderson Miller
- Tech: Auto v1 – Jeff Ong
- Industrial Design and Mechanical Engineering – Scott Thibeault & Ryan Starling
- Visual Design – Seth Mach
- Creative Lead – Christine Todorovich
- Interaction Design – Andrew Haskin
- Program Oversight – Mark Freudenberg
- Program Management – Daniel Goodman
NOTICE – This project was completed while working at frog design. All rights to the project belong to frog design and the client.
More from this project
