eXtended meta-uni-omni-Verse (XV): Introduction, Taxonomy, and State-of-the-Art

The term “virtual reality” has been in use since 1938, but the recent explosion of jargon, confusing new terminology, and the hype around a plethora of related “realities” (augmented, mediated, mixed, digital, etc.) have made it necessary to bring some order to these realities. There are also many “gray” areas between and beyond these realities, making it desirable to embrace an overarching vision. Throughout the world, many, including us, have chosen to use the framework of extended reality (XR) as a unifying concept to interpolate between the realities and to eXtrapolate beyond them. Together with XR, there is also XI (extended intelligence) for which the IEEE has convened the Council on eXtended Intelligence (CXI). A related concept is the metaverse, i.e., shared VR, introduced in 1974 as “metavision” and “metaveillance” and recently popularized by industry. In this article, we propose eXtended meta-uni-omni-Verse (XV) as an overarching term, concept, and taxonomy for shared (social) XR across all of the “verses,” including the universe (physical reality, i.e., atoms), the metaverse (virtuality, i.e., bits), the omniverse and multiverse, etc. We also briefly outline the state-of-the-art in the various realities and verses covered by XV. Our main findings highlight the necessity for additional investigation into the potential and worth of XV technologies. This entails improving XV systems through standardization and integration efforts, broadening their uses, developing specialized solutions for those with disabilities and the elderly, and addressing the implications of XV to ensure its accessibility and benefits.

industry.In this article, we propose eXtended meta-uni-omni-Verse (XV) as an overarching term, concept, and taxonomy for shared (social) XR across all of the "verses," including the universe (physical reality, i.e., atoms), the metaverse (virtuality, i.e., bits), the omniverse and multiverse, etc.We also briefly outline the state-of-the-art in the various realities and verses covered by XV.Our main findings highlight the necessity for additional investigation into the potential and worth of XV technologies.This entails improving XV systems through standardization and integration efforts, broadening their uses, developing specialized solutions for those with disabilities and the elderly, and addressing the implications of XV to ensure its accessibility and benefits.

"REALITIES"
The term virtual reality was introduced in 1938 1 and has evolved as more than just a concept, into a technology and whole industry, along with more recent concepts like augmented reality, mixed reality, 2 mediated reality 3,4,5,6,7 and augmediated reality (augmented mediated reality). 8,9,10,11The three main building blocks of the realities can be illustrated as in Figure 1: physical reality P, virtual reality V, and augmented reality A.
Physical reality (i.e., nature) exists along the a axis, whereas virtual reality (i.e., technology such as audio, visual, haptic, etc.) exists along the virtuality axis b.Augmented reality exists along both axes.This defines a set of quadrants-a taxonomy of sorts-where the lower left quadrant refers to a "diminished reality" representing technologies such as dark sunglasses, welding glass, earplugs, etc., that deliberately reduce or attenuate (diminish) reality.A goal of eXtended Reality (XR) is to generalize between and beyond all the existing realities, i.e., as an overarching term (X is like a mathematical variable that can represent any of the realities R), and as a blending or extension to the whole set of all possible realities.
An example of XR is an eyeglass that allows us to extend vision to see in the ultraviolet, infrared, etc., as well as see radio waves and sound waves, including underwater sound waves (sonar), as shown in Figure 2.

"VERSES"
The work shown in Figure 2 is an example of what is known as "metavision" and FIGURE 1. Building blocks of the "realities," set upon the axes of reality, i.e., the universe as made of atoms a and the virtual world as made of bits b.FIGURE 2. Seeing sonar is an example of eXtended Reality (XR).XR subsumes all the other realities and also goes beyond them. 12,13,14,15metaveillance" (the sensing of sensors and the sensing of their capacity to sense), a concept that dates back to 1974, 16 illustrated in Figure 3, which is often regarded as the origin of the metaverse.17 In this work, light sources are waved around in a dark room, so a large number of people can see, in real time, the otherwise invisible metaveillance flux of a surveillance camera, due to video feedback from a wireless receiver connected to a powerful amplifier driving a light bulb.By using different kinds of very sensitive lock-in amplifiers, a variety of virtual phenomena were shared wirelessly across several spaces in which multiple participants shared the same "reality" and participated meaningfully in that reality, including for example gamification (shared-reality video games) 13 that combined traditional photography with real-time video.
Recently, there has been a great deal of interest in the metaverse, as well as various other verses such as the omniverse and multiverse, and their interplay with reality (the universe). 18,19,20,21he metaverse is a persistent, networked, connected, and shared virtual reality.The multiverse is a collection of various separate shared virtual realities (like the metaverse but not universal across all of them).The omniverse is the union of these various realities, i.e., the union of the metaverse and the multiverse. 18,19,20,21Moreover, "omniverse" is the term used by NVIDIA to refer to its metaverse platform.
When we examine what is now happening with the "verses," we can see a parallel with what happened with the realities, namely that there is a great deal of unnecessary confusion.
There is also a need to interpolate between the various verses as well as extrapolate beyond them.This same need that was met by XR for the realities can be now be met by XV for the verses.
Therefore, three main contributions of this article, in contrast to recent surveys focusing on metaverse definitions 22 and relevant technologies, 23 are as follows: 1) Introduction of XV as an overarching term, concept, and 3-D taxonomy for shared (social) XR across all of the "verses"; 2) Proposal of XV for shared, social, collaborative XR as extension into a third, sociality dimension; 3) Summary of state-of-the-art and future directions in the realities and verses in XV, with examples.

XV
We propose XV for shared, social, collaborative XR, i.e., to take the space of all realities in and beyond the a-b plane and extend them into a third, sociality dimension, as illustrated in

STATE OF THE ART
XV offers immersive and meaningful computational experiences, but its full potential is still being explored.In this section, we examine the arrangement of the b axis, briefly outlining some relevant use cases for consumer electronics and the possibilities of AI integration in XV.Indeed, this is just an overview of the many exciting  16 the sensing of sensors and their capacity to sense.Here we see a surveillance camera with an extended-reality "bug sweeper" consisting of a television receiver driving a light bulb.By way of video feedback, this results in an overlay of the camera's capacity to sense.The overlay can be seen by large numbers of observers without the need for special eyewear.The invention of Metavision/ Metaveillance is often regarded as the origin of the metaverse. 17One of the key open points of XV pertains to the arrangement of the b axis for the extensive computational world of virtuality, AI, machine learning, etc. that is emerging.Regarding the first aspect, there are many alternatives to be considered in devising those elements of the virtual and computational XV that replicate portions of the real world.They can range from static 3-D reconstructions of pieces of a to be visualized and explored in the XV, to analogs or digital twins.
IBM defines a digital twin as "a virtual model designed to accurately reflect a physical object" with no mention of "digital," i.e., this is what used to be called "analog."An analog is "a thing seen as comparable to another" [Oxford Languages], which is the basis of analog computers that simulate the real world.In 1938 when Artaud introduced virtual reality, through "The Theatre and Its Double" he presented theatre as that analog of the real world.Digital twins integrate real-time data from the physical world in order to synchronize the virtual replica with artificial intelligence and machine learning. 24,25Both the approaches have their strengths and shortcomings.In the case of static replicas, the interactivity may be sacrificed to facilitate reconstructing elements of the a as they were in a different time of the past or future, or another reality.Digital twins, in turn, are hampered by technological difficulties related to sensing the changes in a, but at the same time offer the possibility of a seemless transition between a and b while experiencing the XV.The term "digital" generates a great deal of confusion as we can have digital reality (e.g., a hundred-yearold ON/OFF switch), analog reality (a potentiometer), digital virtuality (a virtual switch), or analog virtuality (a virtual "slider" adjustment), which we often refer to as "undigital."In this sense, we prefer the simplicity of XV to the confusion of "digital twins" or anything else using the term "digital" ambiguously.
Another key technology for XV is blockchain 26 and other distributed ledgers to secure digital or "undigital" assets and ensure integrity, privacy, and reputation of users' content and transactions.
There are many different scenarios and usecases that can be enabled by the availability of a copy of the a axis, e.g., via consumer devices.For instance, a 3-D reconstruction can be used for cultural heritage to let visitors access and interact with locations and artifacts which no longer exist, are too delicate to handle, or  Playing together in a virtual world (0% real, 100% virtual, and 100% social), (center) icewater swimming together, wearing immersive XR eyewear to communicate safety information and share maps, overlays, warnings, etc., and (bottom) swimming together in a natural setting with no distractions from any phone, Internet, or other devices (100% real, 0% virtual, 100% social).Float tank picture at origin courtesy of Floatguru, Wikimedia Commons.
difficult to access.Education and training can benefit as well from a replica in different ways.This is the case, e.g., of first responders, who can practice in the XV with faithful reconstructions of typically harsh environments and required tools.Learners from both formal and nonformal contexts could also benefit from a more "cyber-physical" learning experience that can be enabled by digital twins or computational analogs, in terms of democratized access to institutions through the XV.The persistence of the XV can additionally empower the collaboration among students and teachers in unprecedented ways, by using a virtual (b) form of stigmergy to enable the sharing of knowledge, opinions, and ideas among millions of individuals.Replicas from a to b also enable "cyber-physical" systems.
AI can enhance the XV in many different ways.AI-powered chatbots and virtual assistants, for instance, can facilitate social interactions within XV, suggesting other users to connect with and even offering personalized tips and advice to help in building connections.From users' past behaviors, the XV system can provide customized recommendations for realworld activities, such as places or events where to meet and collaborate with others.Using AI algorithms to analyze users' behaviors, XV can be more engaging, with procedurally generated content tailored to individual users' preferences, personality traits, and skill sets.Applying AI to XV can help create a more immersive, social, and personalized experience-spanning from b to not only a but also g-which can keep users engaged and connected with the real and virtual worlds around them.

EXAMPLES
Some of the most exciting new developments in XV are among the simplest.In this section, some examples are provided.

Moveillance
The first example is represented by commonly available 360 capture cameras.A camera of this type can be worn on a "headup" display in a way that provides a "foot first" downwards facing view in front of the wearer, and works in a variety of situations, including outdoors, in the rain, and even underwater (Figure 6).
This forms a useful seeing aid for persons who need to be able to see their feet to walk and find their way but, for instance, have suffered a back injury that makes looking down difficult or problematic.This allows overlay of maps, compass, and other forms of wayfinding tools, in body-centered coordinates (egocentric vision 27 ) or map coordinates, e.g., North at the top (Figures 7 and 8).Interestingly, this also suggests a choice of first-person versus third-person orientation when choosing body-centered coordinates.Similarly, for teachers or professors with a shoulder injury who have difficulty, e.g., writing on a regular chalkboard, it provides a sort of existential document camera for use with a horizontally mounted chalkboard.
New artistic possibilities emerge as well (Figure 9), and, perhaps more importantly, it becomes possible to capture and share the everyday world as an environment map by simply walking around in it.

Freehicles TM
The second example concerns the relation between XV and vehicles for freedom of mobility.
There is an inherent social bias that "permits" inanimate objects to be "smart," but stigmatizes people who are "smart" (tech-equipped).In some FIGURE 6. Experimental "Unicorn Vision" system functions as a "bird's-eye-view" seeing aid for persons having a back injury, allowing a clear downwards-facing "foot first" view of the ground immediately beneath the wearer's feet, to be streamed live to a "headup" display.Left: live video feed from camera.Right: experimental apparatus (underwater).Thanks to Dan Bowman + SwimOP for the underwater picture.
sense, cars or vehicles in general, including micromobility devices, enjoy a certain "right" to "wear" computer vision systems that people often do not.About 15.6% of people (approximately 1.1 billion) have a disability. 28This is the world's largest minority, and the only minority into which anyone might end up becoming a member.About 3.5% of people have a visual impairment. 28oreover, the growing population of elderly suggests that these percentages are on the rise, and will increase greatly in the future.
In this context, the word "mobility" means being able to walk, as assisted by wheelchairs, but also walkers and other similar technologies; it also means knowing how to navigate, e.g., in the context of orientation and mobility (abbreviated O&M) within the community of the blind and visually impaired.Likewise "cane" means something that can help the infirm walk and support their weight, but also something that can help the blind feel their way.
Within this scenario, one may wish to develop a seeing and mobility aid that helps   persons with vision and mobility needs, specifically wayfinding, visual memory, and lower back issues that make it difficult to sit, thus precluding the use of a standard wheelchair or other seat-based mobility solutions.Our current research aims to create a stand-up mobility aid that can operate indoors or outdoors while offering the ability to vary spine pressure through a variety of modes of operation such as kneeling on both or one knee, standing, leaning, and supported in various positions and by various means that can be quickly and constantly changed during use.The mobility aid could also be equipped with numerous cameras, lidar, radar, and sonar in order to add wayfinding functionality.The mobility solution could allow for exercise during portions of a trip, on land, or water.
Previously a machine was invented to carry heavy objects for the disabled and follow behind a person.It used a proportional, integral, integral, integral, derivative (PIIID) controller, to encapsulate not just acceleration and its derivative and integral, but also to capture position and abasement and velocity. 29Presently we are developing a self-driving mobility aid that rides in front of a person to provide guidance (O&M) and/or physical support (mobility), and/or carrying cargo.Additionally, the ability to stand upon the vehicle and ride it is also an option, so that the user can ride it part of the way and get exercise walking or running some of the time.The Freehicle is a smart seeing aid that "sees" and "remembers" what it experiences, thus providing value in vision, memory, as well as physical support.
A very compact version of the Freehicle is a smart walker/rollator/cane/aid that transitions between a cane and a wheeled walker, and which can transform from modest support at times, to full support, as required at other times.This provides adaptive time-varying independence to people with disabilities.

CHALLENGES AND FUTURE DIRECTIONS
Looking toward the future, there are many challenges still to face and several promising directions to consider for the continued development and implementation of XV technology.First, there is a need for further exploration of the full potential of XV and enhancements of its value for humanity.This exploration will include ongoing research and development to refine the accuracy, speed, and reliability of XV systems.The need for standardization and integration is also crucial to ensure seamless collaboration and compatibility between different XV components, allowing for streamlined operations and enhanced overall performance.In addition, efforts will be required to expand the range of applications and contexts in which XV can be used, such as education, entertainment, healthcare, and assistive devices for those with disabilities.For example, there are efforts in progress for putting all of the technologies found in a selfdriving car into a smart walker. 30here is a great opportunity for the development of systems that provide O&M assistance while scanning the world to help in wayfinding, memory, physical support, guidance, exercise, physical activity, and independence for those with disabilities as well as the growing population of elderly (see Figure 10).The Strollator TM , for example, transitions between a motor-assist jogging stroller (long wheelbase for reduced danger of falls) and a rollator (short wheelbase).It transforms its shape or size by recognizing rough terrain and automatically lengthening the wheelbase when rough terrain is detected, and then retracting again for smooth terrain (such as indoors, where the shorter wheelbase affords increased maneuverability).Such systems can significantly enhance the quality of life for individuals who experience difficulties with mobility or sensory processing, and potentially open up new avenues for social and economic participation.
Finally, it will be important to address the ethical, social, and technological implications of XV and work to address many challenges that arise.This includes ongoing discussions about issues such as privacy, data ownership, and the potential impact of XV on employment and social relations.Additionally, efforts will be made to ensure that XV technology is accessible and beneficial to diverse populations, including those from historically marginalized communities.By proactively addressing these issues, we can work to ensure that XV technology is used in a way that maximizes its potential benefits while minimizing any potential harms.

CONCLUSION
XV represents a promising frontier with numerous possibilities for innovation and advancement.
Further aspects remain to be explored to unlock the full potential of XV and enhance its value for users.The reported examples represent the basis for systems that provide O&M assistance while scanning the world to help in wayfinding, memory, as well as physical support, guidance, exercise, physical activity, and independence to those with disabilities as well as the growing population of elderly.
These examples cover just a subset of the many scenarios XV will bring value to.They also suggest that, while continuing to advance in this exciting field, it is essential to remain mindful of the ethical, social, and technological implications of XV and work to address any challenges that arise.
Figures 4 and 5 for a 3-D nature-technology-human (a-b-g) taxonomy volume.

FIGURE 3 .
FIGURE 3. Metavision and Metaveillance,16 the sensing of sensors and their capacity to sense.Here we see a surveillance camera with an extended-reality "bug sweeper" consisting of a television receiver driving a light bulb.By way of video feedback, this results in an overlay of the camera's capacity to sense.The overlay can be seen by large numbers of observers without the need for special eyewear.The invention of Metavision/ Metaveillance is often regarded as the origin of the metaverse.17 possibilities and challenges presented by XV and related technologies.

FIGURE 5 .
FIGURE 5. Examples of XV continuum: (leftmost)Playing together in a virtual world (0% real, 100% virtual, and 100% social), (center) icewater swimming together, wearing immersive XR eyewear to communicate safety information and share maps, overlays, warnings, etc., and (bottom) swimming together in a natural setting with no distractions from any phone, Internet, or other devices (100% real, 0% virtual, 100% social).Float tank picture at origin courtesy of Floatguru, Wikimedia Commons.

FIGURE 8 .
FIGURE 8. Bird's-eye-view commonly available only to vehicles is now available to people, thanks to "Unicorn Vision."Here a pharmacy can be seen to the person's left, and York St. to the person's right, and Bike Share straight ahead.

FIGURE 9 .
FIGURE 9. Vision system of Fig. 6 has potential as a new form of visual art (circle portrait in spherical and compressed cylindrical coordinates), and perhaps more importantly, in capturing the real world by simply walking around in it.

FIGURE 10 .
FIGURE 10.The Strollator TM smart self-driving walker provides spinal support as well as wayfinding and vision assistance.