Entering the Metaverse

What comes after the Internet? This is not so much about replacement as evolution, at least in a Pokémon sense. What is the evolved form of the Internet? The Metaverse. The coming together - or collision - of the physical and digital worlds.I have written about this a lot in various forms over recent years, but I was inspired to address it again after speaking to fellow futurist Cathy Hackl for an episode of my podcast, Talk About Tomorrow. This episode rather accidentally became part of a trilogy of interviews on the subject of the metaverse, bookended by conversations with Steve Sinclair of Mojo Vision, maker of smart contact lenses, and John Keefe, co-founder and director of Draw and Code, one of the world's premier immersive content studios.I return to this topic again and again because I don't think I can stress enough just how important it is that all of us get our heads around the Metaverse concept. So here's a bit of an overview to compliment those three podcast episodes. I strongly recommend having a listen.

Technologies powering the Metaverse

There are multiple definitions of the Metaverse, but it was first laid out in Neal Stephenson's book, Snowcrash. Here, Stephenson described a virtual reality patterned after the real world. Somewhere that your avatar could walk around. Imagine Fortnite or more accurately, Population One - a virtual world projected into your vision through a set of goggles.Today, we talk about the Metaverse as the blending of the physical and digital worlds. What does that mean? Well, multiple technologies are shattering the sheet of glass - your phone or computer screen - that has historically held the two worlds apart. These include:

Voice Assistants

Alexa and Google Home allow us to interact with the internet, ecommerce and other digital sources of data (e.g. music streaming) with our voices rather than our fingers on a keyboard or touchscreen. In doing so, voice assistants have become one of the first and most prominent ways in which the digital world has started to bleed into the physical.

Internet of Things (IoT)

It now costs just a few pence to add intelligence to an every day object: a lightbulb, power socket, wristband etc. The result is that millions of objects are now being equipped with an internet connection and computing power. This allows them to do two things:

• feed information from the physical world back into the digital
• control the physical world from the digital

Again, this leads to the blurring of the lines between physical and digital, when information is common between the two and control can flow from one to the other.

Computer Vision

The rising power of computers combined with the ubiquity of cheap cameras means that machines are getting better and better at interpreting our physical world. They can recognise people, objects, locations and even emotions. The result is that computers and artificial intelligences need less and less explicit input in order to make things happen. For example, autonomous vehicles can 'see' obstructions.

AI

As ever, I use this term in its broadest rather than its most specific sense, to mean software that is configured to take on cognitive loads that would formerly have been shouldered by humans. AI is a critical part of the metaverse because it gives objects and virtual entities the smarts to usefully interact with us, either in the physical or digital realm. Information drawn from voice assistants, computer vision, or any number of sensors can be interpreted into action. And that action can be wrapped in an interface, be it a digital avatar or a change in the environment, that is meaningful to us.Some of this can be done by much simpler code than can be usefully called 'AI'. But the added power means that each interaction doesn't need to be coded manually. The system can learn, interpret, experiment and adapt.

VR

Virtual Reality allows us to experience digital content as if it were a physical environment. Even in the state of the art, it is a long way from perfect. For example, moving around an environment beyond ducks and lunges has to be done with a joystick rather than your legs. But it is nonetheless compelling. Games engage. Virtual cinemas allow us to escape the four walls in which we've been cooped up. And as John Keefe pointed out, people are even (finally) turning to VR as a collaboration tool.

AR

Augmented Reality, or Mixed Reality, is our primary interface with the Metaverse because it too blends the physical and digital worlds. If you have ever used a filter on Snapchat or Facebook Messenger, or played Pokémon Go, then you have experienced rudimentary AR. But it can be so much more. Some time in the next decade, probably in the next five years, we will begin the transition from handsets to headsets, giving us the opportunity to overlay digital items on to the physical world at any time. Virtual people, creatures, aliens, displays, interfaces and objects. There is a huge amount of design work to be done in order to create an experience that is natural, engaging and desirable. In many ways, this design challenge is much greater than the complexities of condensing the hardware. But I believe we will overcome it.

Impacts of the Metaverse

The Metaverse will be all pervasive. It will be our primary interface to just about every form of transaction and probably much more:

Society

Today, we worry about the amount of time people spend behind a screen, lost in a digital world. I suspect that within a few years, most people will spend ten hours a day in mixed reality. This will amplify today's difficulties but also help to resolve them. With the Metaverse you never leave the physical world, but you can twist it. You can repaint the world to meet your preferences, changing your surroundings and even the people who inhabit them.

Shopping

You will never have to ask "How much is that?" Your integral AI will know what you are looking at and seek out the answer. The shop may beam a virtual assistant into your field of view, so that everyone gets a personal shopper. Maybe it will negotiate behind the scenes with your own AI for discounts based on your loyalty.

Banks

Imagine a sixth sense for your spending and credit limit. A subtle colour overlay on products telling you what you can afford, or which is the best use of your limited funds. Imagine your credit score represented in three dimensions, a monument you need to rebuild.

Government services

A personal advisor for everyone, powered by AI? No more language barriers and hard to navigate websites, just conversations. The trade-off being that they might know so much more about you.

Property

Virtual tours available instantly, captured by the agent's glasses and streamed to yours. High definition capture of any issues in the home. Three dimensional guides to any DIY job, from assembling furniture to fixing a leaking tap.

Dating

Who is that? Are the available? Are they a match? This one is fraught with risk.

Ubiquitous technology

I could go on and on because the Metaverse will touch every aspect of life. It might be the lens through which we work and study. It will be so ubiquitous that we will rapidly start to assume that everyone can access it, as we have with the smartphone.The location of the access hardware is important here. Because it is on your head, interactions can be much more subtle. A headset can see what you see, hear what you hear, and answer the questions that those sensory inputs throw up before you even verbalise them. The AI that sits behind your AR experience, personalising your environment and picking up on you needs, will become very much a co-pilot. The shift to the Metaverse will mean the cognitive augmentation of every human who engages with it. That has implications for inequality, but also for health: imagine the help it could be to those suffering with dementia.In summary, whatever your field you should be contemplating what this shift will do to your sector. Because I am more certain than ever that it is coming. And more than ever that it will touch just about everything. 

Long read: 5G and Mixed Reality

I am taking part in two roundtable discussions this week, each focused on a specific technology or set of technologies that stands to have a major impact on our world over the next few years. Though I now spend more time talking about business and culture than I do technology, I remain convinced that it has been one of the biggest drivers of change in our world over recent years and will continue to play a primary role in shifting our future. Understanding the potential impact of these technologies – positive and negative – is therefore critical.Here then – in part as an exercise to organise my own thoughts in advance of these discussions – are my thoughts on two questions:

• What is the role of 5G in the future economic success (or otherwise) of the UK?
• What is the potential of AR and VR technologies, particularly in the property realm, and how/when will this potential be realised?

5G: poor marketing

I doubt I am the first to make this analogy, but 5G is a lot like HS2. OK, no-one mistakenly believes that 5G is responsible for causing pandemics. But each is a large-scale infrastructure project, the motivations for which are widely misunderstood.The naming of HS2 is one of the worst marketing decisions ever. Because HS2 is not about speed. It is about capacity. We just do not have the tracks available to move enough trains up and down the country – or across it for that matter. HS2 does not greatly accelerate the journey from London to Manchester (the most pertinent route to me). But it does put high speed trains on a separate track from local stopping services, allowing more of those to operate at a more consistent rhythm. It adds capacity.Most of the headlines about 5G have been about speed of one form or another. How fast you can download a film. The low latency connections that will apparently be critical to self-driving cars (something about which I am a little sceptical). It is understandable that this has been the message as this is what might sell it to individual users keen to be ahead of the curve. But really, 5G is also about capacity of one kind or another.

Increasing efficiency

The first kind of efficiency that matters, is spectral efficiency. There are only so many frequencies on which you can usefully carry information from point A to point B. A subset of these are suitable for mobile devices. With demand for connectivity growing constantly((OpenReach recorded 10 PetaBytes per hour flowing over its network in May, a large leap in a constantly growing figure driven by lockdown - https://www.ispreview.co.uk/index.php/2020/05/openreach-records-10-petabyte-peak-in-uk-internet-traffic.html)), the challenge is on to carry more and more data over the available spectrum. 5G uses new spectrum but it also makes much better use of it, topping out at 30 bits per second (bps) per Hertz (Hz) according to the CTIA ((https://www.telecompetitor.com/ctia-5g-will-provide-big-spectral-efficiency-gains/#:~:text=5G%20networks%20will%20provide%20major,%2DAdvanced%20technology%2C%20CTIA%20said.)) versus the 15 bps/Hz limit of 4G.This is closely connected to the second kind of efficiency: simultaneous connections. We keep adding more and more devices to the network. The compound annual growth rate of the number of smartphones averaged 93% from the early 80s through 2017. There are now an estimated 22bn connected devices – predicted to be 50bn by 2030. 4G is seriously limited in the number of devices it can connect in a given area, its so-called connection density. While 4G networks can connect around 2000 devices per square kilometre, 5G devices can theoretically connect up to a million. If we want to add more smart devices to the network (and some examples of where we might are below), then we need a more efficient network.Of course, all this data must go somewhere once it reaches the base station from your devices. This is called backhaul. As part of the 5G rollout, operators will be looking at how they deal with your data, introducing more computing at the edge of networks for example, and caching copies of popular content so your request doesn’t need to flow across the network.

Too much juice

Then there is the issue of energy efficiency. 4G networks consume a *lot* of juice. Speaking to LightReading((https://www.lightreading.com/asia-pacific/operators-starting-to-face-up-to-5g-power-cost-/d/d-id/755255)), Jake Saunders, the managing director at ABI Research, said a typical 4G cell might draw six kilowatts in power, rising to perhaps nine kilowatts at peak periods. Multiply that by the roughly 23,000 base stations in the UK((https://en.wikipedia.org/wiki/Telecommunications_towers_in_the_United_Kingdom)) and you get a draw on the grid somewhere between 138 and 207 megawatts.Huawei estimates a 5G base station might consume 300% to 350% of a 4G base station, once operating across all the available frequencies. This sounds bad, until you consider the alternative. Even if 4G could support the continuing growth in demand, its power consumption would be orders of magnitude greater than 5G. According to Orange((https://hellofuture.orange.com/en/5g-energy-efficiency-by-design/)), more efficient 5G technologies are expected to divide the energy consumption per gigabit transported by a factor of 10 compared to 4G once they reach maturity by 2025, and then by a factor of 20 by 2030 .So, if we want to keep expanding the number of devices on the network, and the richness of the media we consume, we need to move to 5G.

Slowing progress

The obvious rejoinder from some will be: do we really need these increases? Or should we stick with 4G? While I think there are technological avenues we ought to avoid (for example, some geo-engineering responses to climate change, certain weapons technologies), I don’t think there is a very strong argument for suppressing the development of general purpose technologies like networks. Just like roads and rail before them, networks are not without their risks and challenges. But they are the platform for other forms of progress: scientific, economic, medical and social. Stopping our move to 5G would not only undermine these efforts, it would place us at a global disadvantage.

Delays on the line

Which leads neatly to one of the issues under discussion at the roundtable event: the delay in our 5G rollout caused by the UK government’s decision to prevent Huawei supplying new equipment, and forcing operators to remove what they had already deployed.Personally, I have always been deeply sceptical about the specific threat presented by Huawei’s technology. The telecoms equipment industry, like all high-tech industries, has long and complex supply chains. There are many opportunities for hostile entities to try to interfere with the equipment before or when it is in place. And it is so complex that maintaining security is a constant battle. Do I believe that Chinese government agencies would try to access UK networks through Huawei’s technology? It seems likely. That’s what spies do. But is there any greater risk from us using Huawei’s technology to anyone else’s? That, I find hard to believe.I have been saying this publicly for a couple of years now, though until the last few months have had no actual relationship with Huawei aside from going to a phone launch about five years ago where I walked away with a free device (as did everyone else in the room). Huawei are paying me to be part of this roundtable event, as any other company would have to. But my opinions are fairly well documented long before this point and haven’t changed at all.

GDP impact

Whether or not you believe Huawei’s equipment presents a material risk to our network security, we now have another issue to contend with. Back in 2017, the UK government predicted that once rolled out, 5G would play a key role in 5-6% of UK GDP((https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/582640/FCCG_Interim_Report.pdf)). If its rollout is delayed, what effect will that have on the economy?Research commissioned by Huawei from Assembly Research has attempted to quantify that effect. It suggests the economic hit will be around £18.2bn. The delay could also “jeopardise £108bn of economic benefit to the UK and the creation of 350,000 jobs in regions outside London and the South-East over the next decade, putting at risk the Government's 'levelling up' agenda.”If you want to understand where those figures come from, then I recommend reading the research (not available to link to as I write this). But I have my own take on this, particularly with regards to where the economic benefits of 5G come from. For me there are two distinct drivers to look at, one of productivity, one of growth.

Hello? Can you hear me?

While our broadband providers have coped admirably through lockdown, I think most people would acknowledge now that our connectivity problems are far from solved. How many calls have you been on over the last few months where people’s video and audio were disrupted, or they dropped in and out of the conversation because of poor connectivity? People are still wandering around their houses trying to get a better signal or working from strange locations because their Wi-Fi or 4G just does not give them a reliable connection elsewhere.Lots of people have tried to quantify the cost of this poor connectivity. In January, Zen Internet suggested SMEs could be losing 72 minutes per day((https://www.computerweekly.com/news/252476441/Poor-UK-internet-connectivity-and-technology-cited-as-hampering-UK-SME-productivity)). Clearly, Zen has something to gain from such statements. But I do not think they are probably that far out. Even if Zen’s estimate is out by a factor of ten, that is half an hour of lost productivity each week, for each employee. That adds up.We need connectivity that stops being a question. You should never, ever, need to think about whether you are connected, only about the application.

Tech-driven growth

The question of growth is more interesting perhaps, but also more speculative. What future applications does 5G support that we do not have today? The simplest answer is just more: as noted above there are hard limits on the number of things we can connect via 4G. But the more fun answer comes from thinking about how you might use the specific characteristics of 5G: ubiquity, latency, bandwidth as required.The first obvious answer is things. Right now, connecting things to the network – particularly remote or mobile things – is still a bit fiddly. Imagine if you could take a thing out of its box and it just worked, with no configuration, for maybe five years before it needed recharging. Think of the applications in manufacturing, fitness, care, logistics, security, entertainment, toys. 5G combined with new battery technologies, passive energy harvesting, and low energy computing should finally open up the potential of the so-called Internet of Things market by stripping away the hassle and fuss and making it easier to deploy connected stuff in the field((Note, early 5G devices have been very power hungry because they are maintaining simultaneous connections to 3 and 4G networks to sustain coverage. This situation will improve dramatically with future generations of the technology and improved 5G coverage.)).The second answer is mixed reality. I have long argued that it is likely we will make a shift from smart phones to smart glasses or headsets. These devices will need ubiquitous rich connectivity, and low latency. Which makes for a neat segue to the second part of this very long blog post.

Living in mixed reality

Right now, we live in two discrete worlds: physical and digital. They touch in various places, like crossing points between dimensions. But they are fundamentally separate. We interact in different ways with these two worlds, one directly and the other indirectly, mediated by a mouse, keyboard, games controller or touchscreen. Or perhaps a voice assistant. We behave differently, communicating online sentiments and in styles we would not dream of replicating in the physical world.These two worlds are separate and distinct, but they are moving closer together. Sixty years ago, the digital world was alien, unintuitive, and required deep expertise to navigate. Those who visited the digital realm were like astronauts exploring an alien space. Today, we can feel our way through the digital world, a space that is more and more familiar. Tomorrow, I think the two worlds begin to truly merge.

Alternatives to dystopia

There are many dystopian visions for this future. Of humans separated from each other by screens spread across their field of vision. Of flashing pop-ups and garish neon interruptions to our field of vision. Notifications pinging endlessly in our ears. But while I think this is a risk in the interim stage, ultimately I can see a much more positive vision for our future cyborg selves.Human senses are incredibly plastic. We can learn to process all sorts of data coming through as sound, touch, taste, sight and smell. You can see this in the work on sensory substitution of the celebrity neuroscientist David Eagleman((https://www.eagleman.com/www/www/research/sensory-substitution)). Or you can see it in the way we learn not just to drive cars, but to sense them. After a while driving a car you stop being conscious of most of your actions. They become automatic. But for them to become automatic, your brain must be unconsciously processing sensory inputs: it’s the tone of the engine and the change in acceleration that tells you when to change up a gear, not the numbers on the rev counter.As we start to live in a permanent mixed reality environment, I think the same thing will happen. We’ll start to ignore the dashboard analogue: the pop-ups, notifications and formal interfaces, and start to rely on more subtle signals. I think the future of mixed reality is like having super powers, or what the original Book of the Future called ‘Extra Sensory Perception’.Imagine all the rich information to which a smart headset might have access translated into streams of sensory information that your brain can interpret subconsciously. Subtle hues of colour, vibrations, sounds, temperatures, electrical stimulation perhaps. A sixth sense for what is happening in your social circle or the stock market. Subtle indicators to point out bargains, risks, or potential dates.The digital world in this scenario becomes translucent: something that colours our interactions with the physical world but does not block them. I think it has the potential to make us more human, not less.There is a huge amount of technological development and interaction design to be done here. But I see this as a more likely outcome than the dystopian futures so often presented.VR, by contrast, will always be an escape from reality. Or at the least, a vehicle into other realities: someone else’s or perhaps our own future. In lockdown, all of those options sound more appealing than ever. But as an escape, they are only somewhere we could, or should, spend a small amount of time. Most of the time we have to face reality, however moderated.So, bringing this back to today and the topic of the roundtable event: how will these technologies, VR and AR, affect the property sector.Right now, I still think the advantage conferred by VR in exploring a property is marginal. It is a very solitary experience, which buying a property rarely is. That is not to say it cannot add value – particularly when the property is not yet built. But the core of the decision making is still likely to be made based on data, photographs, discussion, and a good measure of gut feel.AR has a much greater role to play, though perhaps less immediately. AR will be a critical tool for everyone in the property value chain. It will be used to capture images and plans of land, overlay existing properties with virtual remodels, highlight critical information from utilities to crime, inform contractors etc. We are just not there yet with the hardware, the interaction design, or frankly, the network.It is perhaps fitting that it is in a property industry context – a business that deals specifically with the built environment – that we have this conversation. But tomorrow’s world is one where physical and digital are largely indistinguishable.

Convergence is not (only) the future of gaming

Hideo Kojima is a gaming legend. His plans to integrate gaming, film, music and more, formed the basis of a quick interview I gave this morning on the sofa at BBC Breakfast.It’s not a new idea that these different media might converge. In some ways it is happening already: look at the integration across the Marvel Universe where comic stories weave in and out of games, TV shows and films. Or how film promotion now starts with experiential games seeded around the internet. People have long considered ways to make the cinema experience interactive — a group ‘choose your own adventure’. And the natural conclusion of high-end games is total immersion in an experience of cinematic reality via VR.But I don’t think this is what Kojima is suggesting. Rather, what I interpret from his few words, is that a single, multi-threaded narrative might be explored through multiple forms of media combined in a single entertainment package.This makes a lot of sense with the convergence of entertainment delivery on a small number of devices: phones, tablets and streaming boxes. With some caveats, and the support of some high-end servers in the background, these devices are capable of delivering anything from a simple page of text to a rich VR experience.Why not utilise this breadth of capability to engage us in many different ways? It’s certainly one answer. But I don’t think this is the biggest opportunity in the future of gaming.The largest single segment of the gaming market, following years of rapid growth, is mobile gaming. Within that, the largest phenomenon in recent years is Pokemon Go. Though limited, I think this AR experience points to what will be the most popular and pervasive form of gaming.

Lessons for tomorrow

Imagine real life, gamified through the overlay of the physical world with digital sights and sounds. Virtual places, people, objects and creatures that you can interact with as though they were real. We’ve acclimatised to people speaking to themselves on wireless headsets. People running around the streets chasing Pokemon seemed to generate a lot more smiles and good will than criticism and questioning. I think we’ll adapt to people playing in the streets in their own virtual world — eventually.The revenue streams are certainly there to drive such an industry. Imagine an advert you have to interact with to win a game. Imagine that advert is a virtual character with a rounded virtual intelligence. This is a far cry from today’s billboards: this is hyper-targeted, totally personalised, and fully interactive.Whether you like the sound of that or not, it’s coming.