By Manuel Protásio (PhD Candidate at the School of Law of the University of Minho)

▪

When Virtual Reality and Augmented Reality become ubiquitous in our most mundane actions and inter-personal relations, they will certainly bring many changes in how Law addresses human behavior.

The need for a coherent discussion regarding the potential cognitive effects of these technologies and, subsequently, the legal consequences that may be triggered by their effects is highly relevant and necessary to avoid possible misconceptions in courts and legal systems.

The use of these technologies may result in alterations of our cognitive functions, significant enough to be considered a type of an altered state of consciousness, amenable to different legal consequences. On that premise, it is important to realize that these technologies can have both positive^[1] and negative effects. ^[2] 

These technologies are built and defined with reference to the concept of reality. Such terminology is used to contrast actual reality.  Reality, as it is defined by the Oxford Dictionary, is “the state of things as they actually exist, as opposed to an idealistic or notional idea of them”.^[3] This reality, or the “the thing in itself” as Kant proposed, in the information age and especially in the light of technologies like Augmented and Virtual Reality, has become harder to ascertain, since the human model of perception^[4] is being exposed to more filter layers than it is used to.^[5]

The ontological dimension of reality has always shifted depending on the criteria and discourse used to define it. John Locke for instance, in his Essay on Human Understanding in 1690, describes reality as the knowledge that we convey on the objects that surround us. That knowledge – he states – comes from our observational Experience, which in turn comes from the external interaction of our senses with “sensible objects” followed by the internal operations of our mind.^[6] He describes these internal operations as being a cognitive reflective process on the perceived objects, which can be interpreted as employing meaning – or affections as he says- to those “sensible objects”. From this systematic process, sensible qualities are born, such as “Yellow, White, Heat, Cold, Soft, Hard, Bitter, Sweet”.   

Newton on the other hand adopts a more scientific and less subjective view of reality.^[7] In Opticks, in 1779, he portrays reality as a pure material conception beyond our subjective minds, composed by particles and atom units with different bodies, textures and forms that exist independently of our perception, as it was later argued by Locke. According to this idea, our perception of reality is nothing more than a subjective interpretation of the true and objective material reality that was given by God or Nature and that is only distinguishable based on its spatial and temporal determination.

Richard Rorty in his book Philosophy and the Mirror of Nature,^[8] where he asserted the need to abandon this dualistic view of reality where everything is divided into a mental or a physical model of perception. He defends instead a behaviorist and materialistically oriented philosophy regarding the concept of reality.^[9]

This ontological view is established on the notion that the difference between concepts like “number and tables, quarks and stars, lost socks and moral values” lies on the descriptive and normative value that we give to these objects.^[10] This societal – and individual – assessment is based on the importance -or the non-importance- that we give to every sort of object in the distinctive discourses they appear. This means that the ontological “status” we attribute to a certain reality is often based on our network of beliefs and not just on the intrinsic features of those objects. In my opinion, this ontological perspective of reality is the most coherent and perhaps the most realistic one.

The ambiguity of the ontological meaning of reality is of the utmost importance since technologies as Artificial Intelligence (AI), Augmented Reality (AR), and Virtual Reality (VR) are defined as being a true extension of our reality, as we know it. In fact, these technologies are within the scope of the term Extended Reality (ER). Furthermore, the ontological meaning of these technologies cannot be identified without stating the ambiguity of the concept of reality itself. Since the nature of the environments deployed by AR and VR depends on the descriptive and normative value we assign to them, these assessments must be, as much as possible, unambiguous to avoid prospect incoherent ontological status (descriptive) and legal frameworks (normative). 

The nature of AR derives from VR itself. Whereas VR-users experience an entire computer-generated reality, AR applications provide an interaction with the real environment surrounding the users. VR environments are entirely composed by synthetic elements, while an AR environment provides computer-generated elements overlaid in the real environment.

Since both technologies use virtual/digital elements, the main distinction lies in how the technology interacts with the real environment surrounding the users. In AR, the virtual information is combined with the real environment, whether by increasing it or unfolding it, while in VR the technology isolates the user from its real environment, by inserting him in an entirely computer-generated context. Hence, the terms Virtual and Augmented are used as a reference to the effect caused to the users and their relation with actual reality.     

Thus, it is debatable which technology creates a more immersive environment for the user. It depends on which criteria are used to define what qualifies as an immersive environment. To comprehend the plethora of impacts originated using technologies like AR and VR, a multi- and inter-disciplinary study must be conducted.

With investments up to 4 billion dollars in total and with companies such as Meta, Google and Sony developing and investing, market experts believe that in the next 5 years, the VR-market will worth more than 80 billion dollars and will become consumer-mainstream by 2025.^[11] Hence, since the use of the technology will increase, so will its legal relevance. Thus, there is a pressing need for legal actors to understand how this technology works before addressing it.

Virtual Reality technology generates an entire computer-based reality mostly through a headset device called Head-Mounted Display (HMD). The wider the image projected the more immersive is the experience for the user.

VR devices usually consist of three tracking apparatus: (1) Head Tracking; ^[12] (2) Eye Tracking^[13] and (3) Motion Tracking. ^{[14],[15]/[16]} A complete immersive virtual experience depends usually on the interoperability between these tracking devices, with emphasis on Motion Tracking, which is inherently connected with the optical display, and it is what allows the user to interact with the virtual world. The more refined are the tracking devices, the more immersive will the experience be. As the investment increases and the tracking devices become more mainstream, companies will have more means and less costs to generate the intended immersive effect of this technology, which will also create more opportunities for the technology to be applied in different markets.

The abovementioned tracking devices are essential not only for the purpose of creating an immersive experience for the user but also for health reasons since the lack of accuracy in representing reality can trigger conditions such as simulation disease.^[17]

Like VR, a great investment has been made for Augmented Reality technologies to enter the market in the next few years. Several surveys have been conducted in main business sectors, which show that 69% of the respondents believe that AR will become a mainstream technology in the next five years.^[18] 

This is a clear proof that both VR and AR are creating a new paradigm in the business industry and it is just a question of when they will become ubiquitous in our lives.

Some available information regarding one of Google’s pioneer projects on AR called the Tango Platform,^[19] provides some insight regarding the four technological apparatus that a standard AR device needs to generate its desired effects.

Like VR, AR needs a Sensor Tracking (1). However, on the contrary to VR, AR requires a Single-Camera Markerless or a Marker-Based device together with a Software Development Kit (SDK) (2); a Simultaneous Localization and Mapping (SLAM) (3) and finally and most important, a GPS-based Tracking device (4). ^[20] The interoperability of these devices is what enables the AR effect. The second element of AR; the Single-Camera Markerless device, which enables the platform to overlay the digital information in the environment surrounding the user, generating the user’s immersive experience, also marks the main difference between the two technologies. Additionally, the SLAM and the GPS-based Tracking devices are also relevant to the immersive experience of the user, since it allows the individual to interact in real-time with 3D models in his/her real environment and on his/her current location. 

Haptic feedback is definitely what will determine the next big step in the ER industry, since with this, the experiences provided will be capable of recreating at least three of the five biological senses needed to perceive and interact in the real environment, becoming a determinant factor for granting to the user an immersive experience.

Haptic feedback is the recreation, through technological means, of the sensation of touch.  Although mostly used in VR systems, this synthetic recreation of tactile sensation started with the use of diverse technological means, from electrical fields; pneumatic systems; mechanical pins activated by solenoid,^[21] to kinesthetic interfaces, or anything capable of causing the feeling of expansion and contraction. Mechanics aside, haptic and kinesthetic systems integrated in VR and AR pose some very interesting questions regarding how they will affect our biological perceptive systems, and consequently our behavior. These questions raise relevant legal issues regarding criminal and liability when, for example, actions made in the virtual or augmented environment have consequences in reality.

After explaining the technical aspects of AR and VR, some potential applications of these technologies will be mentioned.

VR has been around already for 30 years, but its applications are increasing as more investment is made. In terms of prospects, VR can be potentially used in every industry;^[22] Healthcare; Sports; Military; Entertainment; Scientific Research; Construction, and in Consumer Business and Marketing-Advertising practices.

Recent literature demonstrates how business practices are adapting to integrate Virtual Reality in their consumer-engagement methods. However, the effects of this application, in contrast with other possible uses of the technology, are the most controversial.^[23] Thus, Law will also face immense challenges regarding the deployment of VR in consumer engagement techniques.

---

[1] “Immersive AR and VR applications have the potential to change the connection between customers and companies because of the opportunity to deliver the most engaging, personalized, and useful experiences. Judicious selection of the appropriate tools and venues will provide innovative marketers with effective and valuable solutions.” Finn G. (2017), How Augmented Reality and Virtual Reality Are Changing Things for Marketers, MarketingProfs, available at: https://www.marketingprofs.com/articles/2017/32549/how-augmented-reality-and-virtual-reality-are-changing-things-for-marketers.

[2] “Two computer scientists-turned-ethicists are seriously considering the problematic ramifications of a technology that allows for real-world pop-ups: Keith Miller at the University of Missouri-St. Louis and Bo Brinkman at Miami University in Ohio (…) A very important question is who controls these augmentations,” Miller says. “It’s a huge responsibility to take over someone’s world — you could manipulate people. You could nudge them.” For now, this issue has remained largely undiscussed for the simple reason that the market isn’t saturated. Google Glass bombed, and nothing has yet stepped into that space. But Miller says it won’t take long — maybe a few years — for A.R. to become common, if not ubiquitous. Beyond construction work, augmented reality is being tossed around as a potential alternative to exposure therapy for patients, a way for doctors to practice surgical maneuvers before doing a procedure, and a tool for consumers to make better decisions. Will a headset be required in a decade?”, in Basu T. (2016), How to Get Lost in Augmented Reality, Inverse, available at: https://www.inverse.com/article/21706-augmented-reality-technology-ethics-advertising.

[3] Definition of reality by the Oxford Live Dictionary, available at: https://en.oxforddictionaries.com/definition/reality.

[4] For instance, in the cultural and linguistic process the model of human perception assigns meaning to the reality captured by our sensorial system.

[5] Youngman, P. (2009). We Are the Machine: The Computer, the Internet, and Information in Contemporary German Literature. Boydell & Brewer.

[6] Valor J-A., (2017), What Actually is Augmented Reality in Ontological and Linguistic View on Augmented Reality, in Ariso J-M. (ed.) (2017), Augmented Reality Reflections on Its Contribution to Knowledge Formation, Berlin Studies in Knowledge Research, 11, De Gruyter, p.p. 111 – 113.

[7] Ibid.pp. 116 – 117.

[8] Rorty, R. (2009) Philosophy and the mirror of nature. Thirtieth anniversary edition, edn. Princeton, N.J.: Princeton University Press (Princeton Classics Ser).

[9] Ibid, p.118.

[10] See Chapter 1.2.1.

[11] Barnes S., (2016), Understanding Virtual Reality in Marketing: Nature, Implications and Potential. Available at SSRN: https://ssrn.com/abstract=2909100.

[12] The movement tracked by the device is referred as pitch, yaw and roll. For better explanation, see the images available at Smithsonian National Air and Space Museum, How Things Fly, available at:  https://howthingsfly.si.edu/flight-dynamics/roll-pitch-and-yaw.

[13] Eye Tracking, as one intuitively might think, relates to the device used to track our eye movement. This may seem simple but can also be the trickiest part in building a VR technology, since it is probably the most invasive tracking device of the three. See a detailed explanation in Gobbetti E. and Scateni R. (1998), Virtual Reality: Past, Present, and Future, Center for Advanced Studies, Research and Development in Sardinia Cagliari, Italy, p. 13.

[14] Ibid, p. 14, on the information regarding the optical or non-optical tracking markers used in Motion Tracking.

[15] See a detailed explanation in Charara S. (2017), Explained: How does VR actually work?, Wearable, available at:  https://www.wareable.com/vr/how-does-vr-work-explained, and Virtual reality Society, What is Virtual Reality, available at: https://www.vrs.org.uk/virtual-reality/what-is-virtual-reality.html.

[16] On the non-optical markers used in Motion Tracking, e.g. gyroscopes, magnetometers or accelerometers, see available information in Levski Y. A Brief Guide to VR Motion Tracking Technology, Appreal, available at: https://appreal-vr.com/blog/virtual-reality-motion-tracking-how-it-works.

[17] This condition happens whenever our brain realizes that something is not real, meaning for example if there is a noticeable disconnection between what our eyes are seeing with what our body feels, hence the relevance of Motion and Eye Tracking devices. For more available information see Simulator Sickness, Wikipedia, available at:  https://en.wikipedia.org/wiki/Simulator_sickness.

[18] Jabil Augmented and Virtual Reality Trends Survey, available at https://www.jabil.com/ar-vr-trends.

[19] See Bardi J. (2017), SLAM, GPS, Multi-Camera? 6 Keys To Choosing An AR Solution, Marxent, available at: https://www.marxentlabs.com/markerless-augmented-reality-google-tango-slam-marxent/.

[20] Regarding GPS tracking applied to AR, see PokemonGo app. See more available information in Wikipedia, available at: https://en.wikipedia.org/wiki/Pok%C3%A9mon_Go.

[21] “Solenoid is the generic term for a coil of wire used as an electromagnet. It also refers to any device that converts electrical energy to mechanical energy using a solenoid. The device creates a magnetic field from electric current and uses the magnetic field to create linear motion.”. See more available information Nicholson J. (2018), How Does Solenoid Work, Sciencing, avaialbe at https://sciencing.com/a-solenoid-work-4567178.html.

[22] See more detailed information Virtual reality Society, Applications of Virtual Reality, available at https://www.vrs.org.uk/virtual-reality-applications/.

[23] Ibid.

Photo by Bradley Hook on Pexels.com.