AI and the Future of Interaction: Meta’s Vision with Quest 3 and Ray-Ban Meta Glasses

Ep14 AI and the Future of Interaction: Meta's Vision with Quest 3 and Ray-Ban Meta Glasses

Transcript

Title: "AI and the Future of Interaction: Meta’s Vision with Quest 3 and Ray-Ban Meta Glasses" Today, we're diving into Meta's innovative hardware, the Quest 3 and Ray-Ban Meta glasses, and their integration with AI technology. Host Intro: "Before we dive in, I'm Dalton. My background is a mix: of programming and data science, and insurance. Offline, you might find me running, building my side business, or lost in a good book. You can listen to the podcast in video or audio format on Spotify or YouTube and listen to the audio on Apple Podcast. Intro: (2 minutes) Briefly introduce the topic and explain that Meta has two hardware products available to the general public: Quest 3 and Ray-Ban Meta. Mention that both products are integrated with Llama 3 and will be upgraded to Llama 4, which focuses on agency and autonomous task management. Segment 1: Llama 4 and AI Agents (10 minutes) Explain Llama 4's focus on agency and how it will enable users to interact with AI agents for task management. Discuss the potential benefits of having personal AI assistants and how they can free up time for more important tasks. AI Agent Demo With @datingwithcarter and @cookwithmax Segment 2: Meta Quest 3 and Pass-Through Capabilities (10 minutes) Discuss the enhancements of Meta Quest 3, including pass-through capabilities and mixed reality features. Explain how users can interact with their surroundings while wearing the headset and the potential applications for gaming, work, and socializing. Segment 3: Meta Ray-Bans and POV Shots (8 minutes) Discuss the features of Meta Ray-Bans, including POV shots and video quality. Explain how Meta Ray-Bans can change the way we interact with others in public spaces and capture precious moments. Segment 4: Integrating Llama with Meta Ray-Bans (8 minutes) Discuss the potential integration of Llama with Meta Ray-Bans and how users can access information and AI-powered assistance on the go. Explain the potential benefits of having a personal AI assistant integrated with wearable technology. Conclusion: (2 minutes) Summarize the main points discussed in the episode and reiterate the potential of Meta's products to enhance our daily lives. Encourage listeners to share their thoughts and opinions on the topic. Additional Resources: Link to Mark Zuckerberg and Lex Friedman's interview with 3D spatial avatars Mark Zuckerberg: First Interview in the Metaverse | Lex Fridman Podcast #398 Link to Meta's news article on social profiles for AI characters https://about.fb.com/news/2023/09/social-profiles-for-metas-ai-characters/ Link to Meta's website for more information on Quest 3 and Ray-Ban Meta https://www.meta.com/quest https://www.meta.com/smart-glasses Meta Keynote Meta Connect 2023 I Full Keynote Okay so last week I wasn't able to discuss this element of meta but I am this week meta has two Hardware products sold to the General Public it is there Quest 3, Rayban Meta. Both of these items are integrated with llama 3 which is the new improved llama model the next iteration of llama llama 4 is going to be focusing on agency and what I mean is they will have models that will have certain tasks like you might have an agent that handles your personal schedule or one for your health or one for helping plan what meals you want and they have a good example of this in their keynote that they had 6 months ago where they are creating 28 AI agents with more on the way and then using the persona of famous people like Snoop Dogg for Dungeon navigation role-playing with your friends. This normalizes people using AI agents to help out with their daily lives this is in preparation for llama 4 release which would be a while from now that will allow these agents to act on their own and kind of have a routing agent say Alfred who handles your needs and say oh Alfred I need something for dinner tonight I've got this stuff in the fridge can you reach out to Max and see see what his thoughts are can I have three options. then the Alfred I will reach out to Max and your request and then Max will get back to Alfred and then Alfred will get back to you at the end of the day but what if you could start your day off with Alfred asking hey what are some things I need to get done today and you say oh I need I need to make a doctor's appointment I need to follow up with Sally and she has my charger and I don't know what I can cope with dinner for tonight with the materials I have. then Alfred replies sure no problem I'll reach out to my friends and I'll get back to you later today after work sound good do you want the notification and the information to be played to you on your way back from work? Yes. That kind of thing would be a great Improvement in somebody's daily life where you have this personal assistant handling your monotonous task that one you forget about two a AI agent could complete these tasks which frees up a lot of your time and gives you more time to spend with your family more time to work on your passion projects and more time to relax. this kind of seems farfetched but I don't think so because this is the route that Mark Zuckerberg is declared for their next version of llama and they've already launched 28 AI agents and they said in their keynote that there's more to come this is our first launch and we're so excited for what's next. currently these agents are available on Facebook and Instagram so I think we could demo some interaction with them I can't share my screen I'll have to do it on my phone because I can't message the AI agents on the computer you can only do that on mobile so we'll see from there and yeah it's going to be very interesting. The future of these AI agents is to help you interact in your day-to-day and what I see in the future is these AI agents acting independently on your behalf to handle tasks for you. I think that this is a actual possibility because of the push from meta with their initial 28 AI agents using famous people's personas with everything being integrated into Instagram and Facebook it normalizes the use of AI agents on their platform to help you solve problems and get you a little bit more comfortable with the transition as many people are not comfortable talking to AI nor are they doing so on the daily basis if you're listening to this podcast you're probably intrigued but think about your grandma your grandfather do you think you're grandparents are friends that have trouble sending emails are going to be enthusiastic or all in on these AI agents probably not but it definitely seems like they're marketing these agents I wouldn't say to kids but definitely not middle-aged adults young adults to tweens. The next thing I didn't get to talk about was meta's Quest 3 https://about.fb.com/news/2023/09/social-profiles-for-metas-ai-characters/ The next thing I want to talk about was Meadows Quest 3 that came out a couple months ago I would say and metis Quest 3 has a lot of enhancements but one of them is the pass through capabilities that are coming with Quest 3 so pass through would mean that when you have the headset on you're not completely immersed into VR you are somewhat immersed and you can choose your immersion by changing the pass through settings. and this is important because a lot of people feel more comfortable using these mixed reality headsets when you can see your surroundings. another cool feature of the mixed reality versus virtual reality is that you can interact with your current space so you can overlay board games or Sports shows sports games tournaments. you can have meetings with your friends on the couch or there's also video games that you can play with your friends at your house and you can use objects in your house as cover which is pretty cool. where the skits interesting or more interesting is they're coming out with these things called augments and these augments are going to be 3D Interactive objects that are anchored in a virtual space so when you use your headset and use pass through you can have you know your Instagram feed of you know your favorite friends on your kitchen counter so when you're going through your meal you can see the feet or your friends or you can see the weather or some cool achievements / trophies display arts or have videos playing as a family album but virtually. and these things kind of move towards living in mixed reality where mixed reality doesn't come as a item or feature that you use to attack attached to accomplishing tasks it is something that you routinely use to live your day to day and that also means working in VR and so I watch some videos and there there are some apps out for VR and I think apple is a little bit further ahead on this one but I don't think that long-term Apple wins this as apple is super expensive their product is heavy there isn't the same level of adoption never will be they don't have open source software meta's way ahead of Apple in this regard other big technology brands have committed to making Hardware using meta and developing apps on the App Store so overall I think Apple loses the headset battle so open source wins. that being said working in VR could be a great way to interact and do things virtually you can have a heightened level of emerging with the new technology that's coming out you can have hyper realistic 3D spatial avatars There is example of this with Mark Zuckerberg and Lex Friedman doing a interview with their 3D spatial avatars that is hyperlistic and you can't really tell that it's not reality and they couldn't tell either there's sometimes where you could tell but for the most part it was a very realistic conversation and those are things that you can't do when you are interacting with someone else in a 2d plane we live in 3D we should speak to others and interact with others in 3D. I'll link that interview in the show notes I think that if you listen to the show you're probably interested in that podcast but the same thing applies to users being able to work in VR integration with Microsoft 365 is coming soon later this year and so you'll be able to basically do what the majority of people will be completing in mixed reality and you can blow up your screens have many screens up and have an integrated with your mouse and keyboard in your computer and so it almost is these external monitors that you have which would cost thousands of dollars to set up instead you can use this headset to set up your external monitors and that would allow you to work on many screens without having to have a huge desk with monitors and a docking station all these other components that come together I mean my desk set up is thousands of dollars I have my docking station which is like 300 my computer monitor setup with my monitor stand which was $300 my monitors were 600 each my custom-made stand that I have which was like would stand up so I could put stuff underneath my computer monitors is I think 200 my mic is a couple hundred dollars my boom arm I think it's 100 and so this really adds up and maybe I couldn't do the mic and the the boom arm but the rest of the information that I presented to you could all be completed with just having a $500 headset and I can take it anywhere I can't take my desk up anywhere I have to have a separate setup for that which was I think $300 for two external fold down monitors that I take with me when I travel and so you have this opportunity to free up a lot of desk space and interact with your colleagues and a hyperistic way eventually which is coming soon they're just trying to be able to do the same thing and upscale on your phones but it's phones are getting better every year I don't think that it's that far out That brings us to our next topic meta Ray-Bans and one of the things that meta Ray-Bans does is it allows you to take POV shots of your life and you'll take videos of your friends or photos of your friends and the photos and video quality are pretty good actually meta Ray-Bans slowly change the way that we interact with others in public spaces or capture those moments that are precious and you don't have time to get your phone out and you just want to double tap on the side of your glasses. I think this is a Tipping Point coming soon where people become more comfortable with these devices on their face and interacting with these AI agents on social media the next step is integrating that all in to where llama is already integrated into meta Ray-Ban glasses and you can ask meta on the Fly questions like what's today's fun fact or how long is the Great Wall of China and it can respond fairly quickly but I think the next step is okay if you're looking at a shop can you pull up the menu and send to my phone or can you tell me the reviews and whether or you think I'd like it on the Fly those are things that would be useful when people might take it to have a hard take on getting more comfortable with it but if it saves you a lot of time and slowly adopted then what's the issue. The next thing is it would be allowing you to fully integrate with an AI system where you could give it tasks and then that's when llama could reach out to the AI agents and the AI agents could handle those to ask and then Lama could get back to you at the end of the day those are things that have a wow factor in my opinion and that's the route of which meta wants to take llama and these devices long term which I think is fascinating. In the world of social media there's no better take than having it on your face. Personalized assistance: AI-powered wearables will learn our habits, preferences, and goals, offering tailored support and guidance to enhance our well-being, productivity, and decision-making. Continuous health monitoring: Wearables will track our vital signs, detecting potential health issues before they become severe, and AI will provide personalized recommendations for prevention and treatment. Augmented cognition: AI-enhanced wearables will amplify our cognitive abilities, offering real-time information, insights, and suggestions to support our learning, memory, and problem-solving. Emotional intelligence and support: AI will analyze our emotional states, offering empathy, guidance, and strategies for emotional regulation, mental health, and relationships. Seamless connectivity: Wearables will enable effortless communication, allowing us to stay connected with loved ones, access information, and control our digital lives with mere thoughts or gestures. Autonomous task management: AI will assist with scheduling, organization, and task completion, freeing us from mundane responsibilities and enabling us to focus on creative pursuits and personal growth. Intimate feedback loops: Wearables will provide constant feedback on our physical and mental states, enabling us to refine our self-awareness, self-regulation, and self-improvement. Ethical considerations: As AI becomes more autonomous, we'll need to address concerns around privacy, consent, bias, and accountability, ensuring that our relationships with devices align with our values and humanity.