DeSciNYC

DeSciNYC

← Back to videos

Talk & transcript

When AI Eats Science with Siraj Raval

Siraj Raval

Transcript

Download text

Automatically generated transcript. May contain errors. Select a timestamp to watch that moment on YouTube.

0:19

the

0:32

okay everybody uh take a seat take a

0:45

seat intermission intermission all right quiet quiet quiet quiet okay yay okay everyone um take a seat there we have some more chairs outside if you want to grab a chair um we have a couple more feel free to um do a couple more and Ben can you go in the back um Ben okay good um awesome so thank you guys all for coming um [Music] oh all right thank you guys all for

1:26

coming thank you guys all for coming out uh I know it's a little bit of a tight space we had a little change in plans last minute uh with the venue but it all worked out oh keep keep the S go back one bed um uh if you'd like to uh you can follow us on our Instagram and our Twitter account here uh we you can find all the latest uh things we post um we have our photographer we'll post all our pictures there so if you want to get your picture uh if you want to be on the picture you can uh do that um if you want to tag us tonight that's always appreciated it helps to impre more people into the community so you can take out your phones now and take a picture don't be shy take

2:07

a picture follow it I know some people already follow it but maybe some people don't already follow it so you can um follow it did everyone get it did everyone get it did everyone get it did everyone get it okay there's still a couple more people um also I'll just say like before we're going to start doing these dinners beforehand and you can talk to the speaker so uh just keep an eye out for that on the uh on the Luma and so if you want to just chat more and also get to know more people it was a really fun dinner today and last time so uh keep an eye out for that okay so everyone have that so take a picture tag us tag something on social media tonight thank you thank you actually do it okay next

2:47

slide um so dite NYC has probably like the 5,000 numb is a little small I think it's closer to 6,000 now and uh people have come to one of our events um we've had Founders we have people from uh the science Community we have uh crypto people we have random people everyone in and everyone in between and uh we have like lots and lots of people as you can see here um we've had Brian Johnson we've had uh Miche 11 from toughs we've had uh Jean her from Einstein on F replacement so we do a lot of like Cutting Edge stuff uh ideally like 50% of the people should you should like and 50% of the people should think for any

3:30

speaker I hope that 50% of the people really really like them and I hope that uh 50% of the people really really hate them I want decisive people and people that are doing edgy and controversial things so that's sort of like what we a to go for um a lot of people are upset with it but whatever all right next slide um the next the next one is going to be on uh the science of dating this is a really interesting one uh with Dan crass he ran a website and he scheduled uh 2 almost 3,000 in uh First Dates between people and uh I've seen the talk and he sent me the slides it looks really good um if you want to RSVP now you can do so here uh it's a QR code um

4:11

and uh it's a really interesting talk around uh what makes people like each other and what he learned from it he has like all these very interesting stats and um insights into it and uh I'm excited for it I think it'll be I think it'll be a really interesting talk um so you can RSVP here this will be we're sort of um taking a little Hiatus um for uh Christmas and New Year's so we will return uh in January all right next slide oh okay uh this is out of order next slide um so um what is decentralized science so um decentralized why did we start dite NYC

4:51

and what is dite NYC and what is what is D to begin with um so decentralized science is thinking about how to use the block blockchain to improve science so uh how many people here are like blockchain people like like the blockchain a lot okay and then we have the other half who is like the science people and um like you know you've heard of defi which is around like decentralized Finance which is about like opening up Finance to people that were unbankable unbanked or unbankable and uh you know opening up and democratizing finance and uh you know that's great and all but the issue at the end of the day is that uh like you're only sort of at the you're only

5:32

sort of Shifting tokens from one wallet to another right you're not actually like creating value and uh what what I'm really excited about decentralized science is like you're putting actual information on chain so that people can begin to do science and uh participate in Trials participate in clinical studies um and they can uh upload things that are working for them maybe they've seen an archive paper or a biox archive paper that is really working well and they want to comment on it uh now this is a way for creating a university that's like on the blockchain that anyone can contribute to and that um anyone is rewarded for and uh this is very interesting so uh right now we're

6:14

in we're in a world in which science is progressing so slowly and uh you know nothing seems to be happening and uh part of one we'll see how AI can fix that and decentralized science tries to gamify science and make it interesting for anyone to participate um secondly science is really inaccurate people today think like quiet in the back uh people today think that uh like science is like the gold standard for everything and like peer review is like the best thing in the entire world but this actually is not the case like right like the peer review has suffers from a lot of things people don't care people aren't incentivized to doing the right way you see a lot of top universities like Stanford uh that like the even the

6:54

top people are lying in their papers and people aren't keeping track of like where the data is coming from and inaccuracies that exist but if you put all that data on the blockchain it enables it so that there's many eyes on it many AI eyes on it and allowing uh more accurate data so I think this is probably one of the more important things um then uh right now science is a little bit too closed off right so like a lot of the things that are happening uh are allowing for uh like only people that are PhD students to participate or like professors but this is is a very small group of people and does not reflect uh the general population of the world so

7:35

uh decentralized science hopes to open that up and then lastly uh the cost structure so science is expensive now but it doesn't have to be like there are ways to make science cheaper and decentralized science is about how to how to do that so um we wanted to take all these ideas in which deepai sucks today and try and make them come on in uh and try and make them into like a fun game thing that we that you can do so we started this uh company called uh Stadium science you can go to the next slide um and our idea here is to make like science as as engaging as like a sport so that people want to participate in science uh people have an incentive to participate in science and uh you

8:16

know you want to make science and research almost like addictive and almost like a game so we have this concept of like like a social network for scientists where people are contributing data into a like a pool and uh the way it works is is uh people we have like uh a bunch of players who are wearing an aura ring and the the goal is to try and bet to see who's going to be the best sleeper each individual night so you're trying to create like a Marketplace for almost like horse racing but instead of for horses it's for people and if you like squint your eyes a little bit on this it's like a prediction Market or a betting platform to incentivize people to live healthy so you're setting up like an optimization where you're saying like we want to bet

8:57

on who's going to be the most healthy person person and uh if you are correct and if you're correct if you look at their BMI their age their blood work what New Tropics they're taking anything like that that's all will be that's all available um you're and you can accurately predict who's going to be the most healthy so you're coming up with a model either me a mental model or an AI model on what is working well and then you guess correctly you're going to be rewarded for guessing that correctly so it's really so you're incentivizing people to a uh upload their dating create these like large data sets that anyone can participate in and then you're also creating uh a lot of people in the crowd to be able to uh like

9:37

participate in this so it's opening up science to people who are outside of it uh generally speaking all right next yeah uh all right next uh next next so you connect your like AA ring to the site um you can and then you bet on people you can go to the next slide so it's like a it's almost like a game um and there's this interface where you see all the different participant participants who are here and then you bet on if you think that they're going to be the top sleeper that night and they list all the different interventions that they're currently working on the drugs that they're currently taking so we have people taking melatonin we have people taking peptides we have people using the sleep

10:19

eight we have people um using a sleep mask all sorts of things and we're generating this like really interesting uh model for which we can like judge how healthy people are and for which people other people can judge how healthy people are all right next slide uh next slide oh okay um okay so that's that I'll go back to the first slide with the a ring so if you want to be one of the competitive sleepers um you can Chine up for our beta test so who here has an aura ring Ora ring user Ora ring user Ora ring user who has a health kit or like an apple watch or something like that okay yeah so if you were have an Apple Watch or a whoop or a whatever um SC and

11:00

you want to be sort of in the Beta Trial for this and you want people you want to uh be an extrovert and allow people to see your sleep data and allow people to bet on it uh like scan the QR code cool cool cool yay and I I'll send you a message later maybe you just want to like send the email friends with me then you can also sign this um cool all right uh wait one more cool all right uh all right go for a bunch um oh go back oh we're also partnering with this uh meme coin token uh called water bear um if you guys want part of the be part of the airdrop for this you can get here it's basically

11:41

free tokens it's very cool uh it's a free airdrop um you will'll get your wallet address and you will get tokens and there I like that's all I can say but uh yeah and it's also partnered with scub as well they help to sponsor these events okay uh next slide oh wait did you go back which one uh go back uh forward or whatever this one the other one I'll fix this next uh forward forward okay um if you want to download the stadium science app you can download it here that's the ordering thing if you want to like if you want to like stalk my sleeping or the other people who are participating it we have a bunch of people here who

12:22

are using it as well so that's kind of exciting you can see how well everyone's sleeping and copy trade what's working well for them sleep is one the most important things actually for um being healthy so it's really important so you Android iOS cool and then lastly next slide uh if you want to work for with us on this um we're looking we're really looking for uh you know BD people marketing people and design people you can scan this QR code as well if you're looking for a job you hate your job I don't know whatever uh you can you should you should scan this cool all right I think we covered what is dii what is Stadium science um what is D NYC so without

13:03

further Ado I am going to welcome uh sage no but I want to set your laptop up too oh yeah um uh I am going to welcome SJ and SJ is um maybe you can uh give it to Ben and you guys can set up SJ is a Ben go to the first slide just so people can tag us again on Instagram while you're setting that and you you can help SJ set up the the the poster we have a quick question oh yeah yeah yeah yeah yeah go ahe that have you done like a peer review design of that sleep study or is it just kind of game a it's not a peer it's a great question so it's not a peer reviewed study because we are going to allow other we're just going to do things for like melatonin

13:44

and things that are relatively easy to test and then the goal is to uh not do like an explicit study but just to collect a lot of data from a lot of different people and hope that the uh we get enough data that will cancel out the and to allow it but we're we're not sure yet we're still right now we're still just like gamifying it and then we'll get to how the study will work in a little bit if you have experience in that sort of thing like we're very open to it but it's very hard to conduct a study on this type of platform because people are wearing different types of devices people have different efficacy rates of adhering to a protocol but this is true for also a traditional trial as well so like that's nothing different so

14:25

it's a different type of study that gives you different type of results right but that's part of the sort of you just have to account for that level of variance when you're doing this type of study like maybe it's only for people that you trust and that are people in your circle or people that you know directly and then you'll sort of copy that but we're still figuring out the logistics of how uh to make this like clinically useful data collection yeah for sure yeah do you plan to make a a web app for oh it's a web app there you can just go to Stadium do science also and U and do that as well um was there another question well that was just I'm not sure why it would need a formal study I mean already having this sort of data available for free uh and for people to do studies on it is already

15:07

very valuable I guess yeah exactly exactly I mean long term the goal here is to create like a clinical trial platform where people can deploy um different novel molecules onto it and then participants can uh opt to participate in that and then get rewarded with tokens for that novel molecule so say someone developed some sort of new drug or they want to try metformin or an existing drug they can like deploy that and people that are sort of interested in that in that by themselves without having to go through the FDA or something can say like oh this sounds really cool I've read a lot about this on Reddit I want to be one of the early participants in that and get rewarded with tokens uh they they can do that for that but yeah we're still we're still do you have do you have experience

15:48

with clinical trials or I no I'm not I'm not a researcher in that space got it got it excellent um any other question did we get do you have to I he's got the is it unlocked yeah it's good oh wait cool I just how do you full screen it uh oh that's that's for the end but just a slide we'll start off with the slides that's on a different okay is it a different URL yeah okay I that's right so we have Sage here and SJ is I I I knew you guys can come in come in I've known Sage uh for quite a long

16:28

time uh he hasn't known me though but I started watching sj's uh sj's videos on YouTube sort of during pandemic he has a really popular YouTube on what is bitcoin that he did very early uh he had a really PO he has like a lot of popular uh videos on AI and like the word Toc and all that sort of space and uh I he's like one of the he's a very prominent YouTube educator on uh on on AI and crypto subjects he has over uh 750,000 uh subscribers on YouTube a very big Twitter following and he's very clear in his explanations and also he

17:09

doesn't do stuff that's sort of normal right like he's always pushing the edge he has a company called uh Trader GPT and wager GPT to enable PE just the average person uh to uh get access to the financial markets uh at dinner we were talking about one of his goals in life is to enable of anybody to be to write a p a PhD level research topic uh not and and open that up to sort of anybody in the world so uh I think he's like a really good speaker for dite and um without further Ado let's give s your

17:50

hand thanks everybody it's it's an honor to be here it's a very crowded room again it's an honor I think everybody here is very early in this space it's still very very early raise your hand if you remember what AI was like in 2017 yeah we have gone really far you can see like you guys are pioneers for even being here so again it is an honor for me to be here this talk is about AI for science in 2025 and the key question that we want to answer is how do we accelerate the rate of scientific discovery 1,000x we go to the next slide if not we can just oh perfect so I'd like to start

18:31

this off with an educational post that I made last week on X and I'm going to read it to you um open aai for context released 01 Pro that's $200 a month and in the first hour of them releasing 01 Pro I opened up X and I saw a bunch of AI influencers and Educators that I follow posting tweet threads about all the 10 things you could do with 01 Pro and so you know you guys remember what it was like back then no one was talking about it and within an hour everybody talking about it so I decided to basically troll science by posting this and I said I spent $200 on1 Pro and casually asked it to solve physics UniFi general relativity and

19:12

quantum mechanics I said no String Theory no Loops it gave me wild new math and testal predictions like it was nothing full link and I put the link out there for anybody to look at it and this post went viral we go to the next slide yeah and what 01 Pro gave me was this and I'll read out some of this for you guys this Grand Theory of physics transplan hyper algebra is the first thing what is that you might ask well it defines it at a transan transplan hyper algebra or tph is an associative byg graded algebra of

19:54

course we all know what that is equipped with a higher category structure duh okay it's got axioms It's got scaling it's got a sheath of Happ for algebra right so what is this right so we go to the next slide so an actual physicist came in NYU grad student and he's like I mean no it did a good impression of how someone would summarize their work if they had invented new math and maybe it made new predictions but it didn't actually do those things and he wasn't the only one a bunch of full-time phys came in and they're like laughing ridiculing this Elon Musk comes in to weigh on the discussion he's laughing

20:36

too but he comes and defends me at some point it will be real though thank you Elon you can always count on Elon to be real so you know did I believe that 01 Pro could actually solve physics absolutely not but do I believe that it could possibly solve it maybe yes but this was like enlightening for a lot of people because yes I was kind of like trolling them and I was saying things like us sounds like someone missed their trans planking hyper algebra class but but really like I prompted it further I used

21:17

Claude to generate better prompts and I I was giving them I was giving Claude what these guys had said I was like here's their exact criticisms make it better make it better math and I as a prompt engineer believe that there is a valid prompt for everything right if if there's a will there's a way but it turns out that even after repr prompting it three times with domain knowledge and giving it their criticisms it generated something that was even worse than it generated at first it was even more of a hallucination that it did did at first now I deal with hallucinations every single day in my life we go to the next slide uh so like this is a real problem it's in sports betting I've got wager GPT I've got a bunch of DJ who paying

21:59

$200 a month to get the best hottest picks for for teams and it works but there's hallucinations you know and if it doesn't work I can blame the hallucinations right but you can't always blame the hallucinations especially when it comes to financial data and I have Trader GPT as well as as a as a product with paying customers you can't have an AI hallucinate returns for you right you either make the money or you don't uh so hallucinations are popping up all over the place but they cannot pop up in science we don't want it to pop up in science so who is doing this who is automating science with AI well it turns out that this company Sakana doai out of Tokyo Japan has built

22:40

this AI scientist and their goal is to automate the entire pipeline of science starting with the idea and moving it to check all of the existing citations on the web does this idea exist let's give it a score let's create a template for an experiment let's run the this code and tested it out let's update the plan let's make this a recursive feedback loop let's then create the paper let's generate the paper and now let's give it to an llm to peerreview Michael was talking earlier about how peer review is broken he is absolutely right so this looks at its at its face value like a great idea so Conor raised what 50 million they've got hard Maru from X one

23:22

of the ogs of AI Elon is lessed you would think that they're going to solve it it's called the AI scientist for God's sake it's an agent you go to their GitHub they have 300 plus Stars they have this thing has already published 12 papers but how can we be sure that it's not hallucinating well they're telling us that they're running on this llm and they're experimenting but that doesn't change the fact that I paid for 01 Pro and got mocked by every physicist on Earth for claiming to do science what makes you think Sakana AI that you can do science we go to the next slide look at this 2024 this year scientific

24:03

article with insane AI generated images somehow passes peer review this is disgusting this got into frontiers of medicine right this is one of the leading papers that everyone's like peerreview is the all be all end all your llm is never going to do it actually this is relevant for the reason that it shows an example of AI slop and how p is bad but it also shows how how are we going to verify anything like we need we need actual verification this is like if if we don't get this part right then we're going to have these actual phds who work their whole lives to attain this knowledge we're

24:44

going to waste their time reviewing all this slop that we're sending at them so I can't just say okay guys go out there and generate AI that's actually not very responsible right we want to actually accelerate science we don't just want to say that we did something for for the sake of it so we go to the next slide what turns out that this is not some kind of new idea in their Charter open AI Charter what are they even here to do our mission is to ensure that artificial general intelligence AI systems that are generally smarter than humans benefits all of humanity all right let's get past that PR talk what do you specifically mean if AGI is successfully created this technology

25:24

could help us Elevate Humanity again vague platitudes by increasing abundance a little bit more quantifiable turbocharging the global economy I guess you could quantify that and aiding in the discovery of new scientific knowledge that changes the limits of possibility open AI themselves marketed 01 Pro as a PhD in your pocket and yet when someone goes out there and uses it as a PhD in your pocket of course the actual phds are going to get pissed so this should not be a people can be very tribal this is not an US versus them dhds versus not we are all in this for science science doesn't care about your pronouns or anything right science just

26:06

exists it's like mathematics so if this is their mission we're all aligned we want to live in this fully automated luxury space Utopia where we don't have to work and pay rent right we just live We Exist we enjoy each other's companies we can do that with AGI but we have to actually get AGI to do the hard part which is science right we can't just have this thing generating slop and embarrassing all of our us all day so we can go to the next slide can this this is really the key question can this even be automated if Sakana can't do it if you know if Elon Musk is laughing at this if if uh you know anyone who tries it's just going to be uh llm hallucinations how can you be

26:48

sure without a human in you know in the loop well let's see this is a scientific method right we know this this has always existed observation is the first step can AI automate this of course we have self-driving cars we have cameras machines we can automate observations that's easy question can we automate this absolutely we can have an AI generate a bunch of random questions till the end of the till the end of time hypothesis absolutely we can have an AI generate a million of these hypothesis experiment now this is the Crux of AI in science right here experiment how do we automate that we don't have a lab we don't have a wet lab we don't have physics facility we don't have a large

27:30

hadrin collider in Geneva we can't be running particle physics experiments in our laptop we can't be doing that so how do we automate this we go to the next slide science considers there to be three mediums to run experiments on it's and it's called these three Latin terms in vitro it which is in a lab in Vivo in an organism or in silico in a computer that these are the mediums for Science and ideally we can just have it in silico we don't have to deal with these two and all AI scientists are aware of this they want this to happen and that

28:11

brings us to the next slide which is Alpha fold 3 this came out this year which is built on I think five years of work from Google and Deep Mind on trying to predict how a protein will fold and if we are able to predict how a protein will fold we can design drugs to help Target these proteins and essentially cure a bunch of diseases that's what it comes down to the reason protein folding is important is because it helps us design drugs to cure diseases potentially um not potentially it actually does that but the potential is for us to be able to move so much of what existed in vitro to in silicone and that's what Alpha fold did it moved so

28:52

much of the experimentation that scientists would have to do in a wet lab by trying out different combinations of these proteins and seeing how they interact it was able to simulate that and it didn't need a quantum computer by the way I thought Quantum Computing would be necessary findan himself said that in order to simulate quantum mechanics we're going to need a quantum computer actually the CEO of Deep Mind Demis hassabis said a few weeks ago that classical touring machines were probably going to be enough and he's talked to some of the smartest Quantum scientists and it's going to be enough so this is actually very very promising because it showed how much of the drug Discovery process which involves trying to test

29:32

out all sorts of combinations of how proteins are going to fold how they're going to interact with all of these uh targets and it does that in silico so this was a huge deal and using this Google was able to generate a database of 200 million proteins that they were able to generate the model for with Alpha fold and it's already being used to cure a to treat a bunch of cancers so this was a great example of a world changing Cutting Edge example of moving science to a laptop out of the lab onto the laptop but there was another what's the next one uh but before we get to the next one let me just briefly in two

30:13

sentences explain more about how alpha fold 3 works the way it works is a slide no no this this was the perfect slide oh okay yeah just just just high level there's three parts here the first step is input preparation every variation of alpha cold they added in more data because that is a part of the scaling law one of the two scaling laws that we're going to talk about which is the more data and more compute you give these neural networks which are also called Universal function approximators because they can approximate any given function if given the right parameters through back propagation in this input process they was given even more data than Alpha fold 1 and two in that it's not just able to predict protein for humans but for any living organism and all the interactions

30:56

of a variety of molecules not just proteins actually liens a bunch of different ones so it's just basically it's trying to be a general purpose llm for biology and it took Google scale resources to do this and what we know about deep learning is that it is The Cutting Edge tool for representation learning being able to find the features and important Connections in this data set but the part that made Alpha fold 3 unique from alpha fold 1 and two was this diffusion model and who gear has heard of stable diffusion raise your hand diffusion in the past few years great I love to see that in the past few years has really shown people the power of generative modeling for images uh but it's also useful for biology it's

31:38

actually it turned out to be very useful for generating a bunch of random noise from these set of proteins and then iterally denoising those proteins to try to reconstruct them and this allowed the machine to over time learn how to construct novel proteins that had never seen before and predict how they would interact we go to the next slide so Google's got its whole domain it's got its team focus on biology they're good uh but we want to as individuals without a lot of resources who just want to do science we want to do something that we can actually do that's accessible to us so here is this donut of knowledge I found from Reddit and it

32:20

shows very roughly all the different sciences and I guess art forms that we could be using AI for and unfortunately a lot of these require remember those two concepts in Vivo or in vitro whether it's chemistry human biology medicine uh all of these are going to require some physical equipment now the one that's obviously not going to require that is computer science but we're not going to focus on computer science but if if you notice here there's one branch of science that uh is very close to computer science and that's the one of the formal sciences and that's mathematics so that's that's really interesting mathematics is interesting because it's not quite computer science

33:02

it's in in a way it's it's a superet of computer science but in a way it's different from computer science so what if we built an AI that could make new discoveries in math then it wouldn't need a wet lab ideally it would just need a computer we go to the next slide well damn it Google already did that too and they did that this year and it's called Alpha geometry and it is an Olympiad level AI system for geometry and once again they took this idea of Science and they moved it completely in silicone and what Alpha geometry did is it out competed all but the gold medalists in the in the Olympiad and it

33:45

was able to solve proofs for a subset of math geometry and it didn't create new math but it was able to solve proofs better than almost all of the other participants and this is really amazing people didn't think that an AI could do this that it could solve proofs at a level of a human at this state at this stage of thought next slide uh and before we go on further I just want to take a second to just go over a our first proof with you here just so we get we're all on the same page of like what a proof actually is in this specific scenario so we're going to

34:25

have a theorem this is our theorem we're scientists here we're going to propose this today we're going to declare that for all integers n if n is even then n s is even so I'll say that again if n is even so if n is two then the square of n is also going to be even so the square of two four that's also going to be even if n is 10 that's an even number the square of n is going to be 100 that's also going to be even that's the theorem that we're making so in Python code we could just create a function that says you know if this take this input n and then Square it and if the square is even return true something very simple in three lines we could do that but that doesn't prove it it just shows for one

35:06

case in order to prove it we're going to need to write out a deduction and this is how we do it we're going to pick an arbitrary even integer n we're just whatever it is it's an even integer we're going to call it n and we need to show that N2 is even how are we going to do that well we can logically deduce that since n is even there's some other we're going to call it K such that n equal 2K so if n equal 2K that means that n² equal 2k2 well 2k2 is the same as 4K 2 well hey 4K 2 is the same as 2 * 2K 2 so from this we see that there is an

35:47

integer M namely 2k^ 2 where n^2 equals 2 m therefore N2 is even which is what we wanted to show now we did this through a series of algebraic relations these are operators that are just that are logically sound that don't care what you believe they just exist and there's a set of these and we can construct a database of these and we can call this a deductive database I just want to show you guys one more proof really quickly and that is this proof you don't have to understand everything about it but I want you to understand the general idea and that is we we propose a concept of

36:28

theorem in this case it's going to be cose x = 1 plus theang s of X that's what we want to prove well to start we're going to perform this operation on both sides we're going to divide both sides by sin squ and then we're going to factor it so it looks different and then we know these two facts already are existent already exist outside of this equation and because these two facts exist outside of this equation this is true and we can prove the initial theorem so it's essentially making a proof is making a set of logical statements that successively deduct that the original theorem uh is true I think I just went into a recursion there

37:10

but but hopefully I give you some idea of a proof all right let's move on so like I said there's a ton of databases out there we got a realtime database centralized database parallel databases distributed databases but what Alpha geometry used is a deductive database that we talked about here and the store in this database all of these relational algebraic operators that exist and these are just Pro they're mathematical uh facts and by combining them like Lego blocks we can make we can prove anything to be true and and that's actually really useful that idea that we can mathematically prove something to be true when it comes to hallucinations we

37:51

go to the next slide so the idea for Alpha geometry was to take a proof let ABC be any triangle with AB equals AC prove that the that the angle ABC equals the angle BCA and they would deduce this to a series of algebraic operators that the llm the neural network would generate and it would try out different combinations of these algebraic relations through trial and error and it was essentially a search it's essentially a search problem and mathematicians will tell you the same thing when they're trying to think of a new idea they are searching through the space potential ideas sometimes it comes from their subconscious you don't know where it comes from but it comes

38:32

from search and this was one search strategy that was somewhat effective but it didn't quite beat the gold medalist so it wasn't that effective but this idea of using a neural network to come up with logical rules that it then tests through reinforcement learning trial and error to come up with a theorem a that is provable that's actually very useful to this idea of AI for science we go to the next slide and uh one more thing about Alpha geometry the way he was able to learn all this is through selfplay which this Alpha zero was one of their earlier AIS that was able to learn through selfplay the game of Go and it was built off of alpha go and they learned that selfplay

39:14

synthetic data generation is actually much more useful than watching humans play selfplay is one of the best paper one of the best papers in AI uh George Hots who runs comma.ai he personally told me this was his faor paper I never forgot that I think this is a very beautiful idea because it shows when a lot of people talk about the data wall or like we've hit a wall in AI if you look at ilas eser at the nurbs conversation uh last week we haven't hit a wall it's all about synthetic data generation the top Minds in AI know this the results show this so at the end of the day selfplay is the key and selfplay can be thought of in a lot of different ways you could think of it as prompting

39:55

your Frontier Model to create a bunch of synthetic data sets that's one way but reinforcement learning is like the OG way of doing it and that is what Alpha geometry did it learned through a deductive database trying out a bunch of proofs and then coming up with the with the optimal solution next slide but uh you know deep mind's worked on Geometry but what hasn't been worked on uh I want to start with number Theory because number Theory it's completely in silico you don't need a lab you can actually make a novel DEC Discovery and you can prove it with uh this technology that I'm talking about that I'm I'm about to reveal in the next slide but

40:36

number theory is really cool because it describes all the numbers in the world and the relations between them next slide we see this all over the place you know it's in the golden re ratio the Fibonacci sequence uh this was a beautiful number Theory problem from the international math Olympiad find all integers MN such that m^2 + 1 232 = three to the end uh Ramen Newan was a famous Indian computer scientist he he was quoted as saying an equation means nothing to me unless it expresses the thought of God I thought that was a cool quote to add uh he was a number theorist number theory is very cool you know it's not just because it's the only thing we can do with AI right now that's fully

41:16

automated it is but it's also because it's very cool so next slide please so we want to make an AI that will generate novel math that works legit math something that any scientist will look at and say Yep this is legit and an AI did the entire thing it came up with the idea it came up with the hypothesis it experimented on the machine it verified those results with mathematics in a logically sound structured Rock Solid way and it used AI to peerreview itself through a a series of Agents of peerreview Agents um but

41:57

how how do we and we know about all those parts but really the hardest part here is the experimentation part and so we want an AI that's going to be able to generate proofs for everything that it generates and there's three programming languages currently for generating proofs Isabelle is one not enough libraries I would say uh COA is the other one and lean is the one that I want to use it was actually created by Microsoft and it's used by the ethereum community to secure smart cont tracks uh there's a huge history of automated theorum proving which is what the subfield is called in ethereum in blockchain if you guys have heard of ZK

42:37

snarks ZK proofs all that stuff uses mathematical verifiability it's using these algebraic operators that I'm talking about just not for AI all right next slide please so I want to choose lean for this and I'm just going to show one quick example of lean just so you guys get a little flavor of what it feels like so uh let's say we want to write out the statement x equals the absolute value of x I'll say it again x equals the absolute value of x and that's that's really simple to write out in Python definition absolute value input parameter X return the absolute value of x but it only works for some numbers we can test out a bunch of

43:18

numbers and but how do we know that it's going to work for all possible numbers well in lean we can write out the same thing but instead of saying depth we say theorem and we name our theorem what we want to call it we're going to call it ABS non NE for absolute value non- negative and then we're going to define a number X Out of all real numbers Z and so it's using these algebraic operators to say it's going to be equal to the absolute value of x and built into this language is a mathematical operator that will deduce the statement that I said using a series of algebraic operators if everyone wants to know if this is true it's going to Output that proof it's going to Output the proof and so this

43:59

idea of the system outputting a proof I want us to get really familiar with and really like because we're going to need that next slide please and why will we need that well Toyota had an unintended acceleration that has killed 89 people uh that was a headline night Capital says a trading glitch cost 444 million who here has seen a broken 404 page by raising hands every single person the reason this is happening in 2020 CIS estimated that software failures cost the economy $1.56 trillion the reason this is happening is because most software that we ship is not mathematically verified and why

44:40

would it be we don't have time to sit here and learn about proofs and trans algebraic geometry and we we just want to build stuff that works and and and profit and and help people uh but unfortunately it's causing a lot of unintended consequences that can result in death so this is actually really really important because hallucinations are killing people so to solve hallucinations we need to use math and we need to use provable math that is going to show that our AI isn't hallucinated we need to show it we can't just say it we have to show it with math so next slide please so this this this area of using

45:22

AI uh in theorem proving it's called automated theor improving it's been around for a couple decades but like all of AI not really much progress has been made until the past 5 years Alpha geometry was the best paper uh in this space and the idea is that it's just going to come up with a bunch of tactics it's going to create a tree of possibilities through Brute Force trial and error does it work and uh that's all we need to do is we need to get an AI to come up with the right proof for whatever it's trying to propose or generate next slide and why has this not been done well it's prohibitively difficult because a lot of

46:04

math is not in computers it's just still in papers it's literally still in like paper form tons of theorems from the 1700s and 1800s um it could take a lot of work now with Claude you know we can just uh run Claude computer usage and just uh Vision mode and just take that 11 person years of work and turn into an agent but it's virtually all all software that ships today is unverified and uh this is actually really exciting by the way because a lot of us including myself thought like the fun in AI came from the complexity of all the papers and like it was so fun to look at all the techniques and now it's just like you know Transformer some reinforcement

46:45

learning thre in there uh but this really allows a whole new possibility of like if you're a smart motivated person and you want to do something in AI that isn't uh very saturated uh this is a great area to focus on because Downstream of this is science and uh basically the end of hallucinations next slide so yeah this just kind of further reinforces the idea of you have a Transformer it generates a bunch of tactics which are algebraic relations and it just combines them like Lego blocks until it finds the right combination that's going to work and it returns that next slide and uh there's been a lot of talk

47:26

about this new scaling law uh which is inference time scaling we've had the the pre the pre-training scaling laws gbt 1 2 3 4 we all know about that more data more compute makes the mod model smarter across a whole bunch of evaluations but with 01 we saw inference time compute arise and that shows that the longer we let a model think when we prompt it the more like the higher the likelihood that the output is going to be aligned with the evaluation that we're running it for I actually think that there's a third scaling law that we're not talking about and that is a law involving these algebraic relations that I'm talking about automated theorum proving it

48:07

doesn't matter how long 01 thinks it's still going to Output if we say generate novel physics the only way it's not going to do that is if it's thinking and reasoning using these algebraic operators and mathematical theorem proving uh programming language that I'm talking about that's the only way that it doesn't matter how smart you are show me the math at the end of the day and so uh let's not just I just want to put this slide out there to say that let's not just think because Owen is reasoning that we've solved reasoning actually uh reasoning is just getting started and so much of reasoning uh doesn't have to do with text based autor regressive models

48:47

it has to do with uh math and and reasoning Through Math uh logically and deductively next slide and so um yeah I wanted to build this so I built a little proof of concept that's going to automate the entire pipeline for this and uh it's an open source project and it's free and everyone can use it and uh we're going to give it it's called science flow and it's live on the web too and uh it wasn't really that hard to build anyone can build it essentially I asked GPT for to generate lean proofs specifically so fortunately GPT before was trained on lean as a programming language so I don't just ask

49:29

you to generate anything I I say make uh a proof in lean and then verify it in lean and that way it can actually you can verify that the science it's saying is real so I want to give it this famous unproven number Theory uh conjecture the collap conjecture and it's a famous unsolved problem in mathematics but it's actually really simple to State all it states is that for every for even numbers divide by two for odd numbers multiply by three and add one and if you do that enough times all positive integers will converge to one that seems true people have tried out up to hundreds of thousands of numbers and it works for

50:11

all of them this conjecture but they haven't proved it yet because the proof is too complicated and nobody's been able to do it but let's ask AI to do it and see if it can do it live today uh this is the last slide you don't have to go to the next slide um I'm just going to go back there and enter in a prompt and then I'll come back here so before you guys clap let me just go back to while he's doing that um guys all smile and all right three two one Smile Smile one more all right actually smile this time smile there we go all right this is now we can do the

51:06

[Music]

51:24

yeah can SHO

51:35

yeah okay so I just asked get the prompt um what happens if we modify the colad sequence and okay so I just did it so I asked it to do new ma solve this problem and it just did it so here's here's the lean it generates a hypothesis it automates the discovery process it validates the proof in the lean compiler so we don't have to just word and then it generates the paper that you can then download so this is the full scientific pipeline automated for the only science that is completely in silico right now um and that is number Theory idea generation pattern analysis automated Discovery proof validation paper generation publication you can submit it to Archive uh but that's after

52:17

it's been verified if you could scroll down to the bottom that would be awesome keep going uh stop and then it verifies um uh no automated empirical verification would perform because the hypothesis could not be parsed that's actually a great thing that is the AI saying that it was some that was a hallucination I couldn't compile it but it's so and and look the peer review even it said your proposal to modify the colx conjecture by changing the function applied to odd numbers is intriguing this guy says it's intriguing even though it doesn't work so even the peer review is hallucinating right now it's also because it's GPT 3.5 turbo it's not the latest model and that was just foresting uh but at the end of the day all parts of this

52:59

could be hallucinating but the verification will never hallucinate because math never lies so uh I just want to leave you with that U math never lies automated theorem proving is awesome I hope you guys learn more about it the future of AI and science is really cool if you want to follow if you if you're interested in this uh follow me on X um I'm I'm going hard on X this year mostly and then YouTube secondary and then of course thei shout out to Michael love this Community thank you guys right let question here who has the question with doing the formal

53:41

verification as a test case does it like iterate through every real number on the simler level it's not a CL right I don't think it does no this was like a very Bare Bones like does it take a long time though yeah it does actually that's why I had to use gbt 3.5 turbo because it times out on Heroku which is this is uh you know it takes like 120 seconds for inference so I have to cut it down to 20 seconds uh that's why it's it's it's it's a lot better if you make it oh I see so most of the time is lean doing the formal verification right yeah okay yeah what did you mean like

54:23

software it's not verified how would you verify for any code base you have it's like writing unit tests like test cases for it it's like a reflective uh language that's on the side it's kind of like Cuda optimization it's almost like GPU optimization like you can have a for Loop in Python but in Cuda you can optimize the specifics of it like where it's running on what chip what parts of the chip similarly for like software if I have a loop running um I can write a lean script that's going to formally verify that it does what it says it's going to do for all cases python will only ever show like

55:05

some cases well any programming language will only show shown cases but formal theorem proving languages will show that something works for all cases and that's really the key difference all versus some it's it's it's hard to understand I'm still learning it myself but yeah Yes you heard about the test center development test based development yes it's like a lot of like one Frameworks that develop this for different uh levels of software yeah no I it's not mathematically based on mathematics based on how we structure code to be able to test it because if

55:47

you brute force it is too much is uh it's like a combinatorial explosion yeah unit tests aren't a full formal verification of a program though it's just like some arbitrary thing that you're testing right that's tdd like testing development is often for like unit test integration tests like these different things but like an automated theorem impr can prove with mathematical correctness like some assertion it's yeah well said an assertion yeah that's a good word for it yeah is this the same as symbolic verification that's a good question question uh I would think symbolic

56:29

verification there's this is included but there's more to symbolic verification like this is like yes it's verifying symbols which are like algebraic symbols but I think symbolic verification could also include like all sorts of other cryptography and like terms that are not like necessarily automated theem proving yes like a proof Library so like once you run this one like okay when I run it it runs in like a second it already has Pro yeah and that's a great comment and I forgot to mention that yes there is a proof library and once you use it once it's reusable forever it's like a library of Truth like an actual library of Truth and uh we can build

57:11

that that's actually really exciting yes verify code also similar to debugging and also testing UI ux so might run in one like browser and then crash in another one give you or different yeah uh it could if they're both using the dependency for like lean if they're both trying to verify a client side but ideally that's not the case because of the way the system set up I I have had that issue on Heroku where like the compiler wasn't working and I had to install it so yeah I still see there being some dependency issues yeah even if this is like you got to make sure the dependencies are there you yes oh I know

57:53

the main point of the program was just I guess verify scientific work but I guess how do you see yourself or do you see it's possible integrating into the own your own programs that you built just verifying things that I guess go through Ai and people have questions about but because even when you prompted to do things there's all these hallucinations that happen um you know regular I guess dat use of like chat of a program like chat gbt yeah like anything like Mission critical I want to use this for like if I was working on Medtech um you know device medical devices use this as well like you want to make sure that a device doesn't return a negative one and like you can write a python script but like

58:33

if it does someone's pacemaker could die if you're like uh secure tunneling into a pacemaker you got like an API for that it can never return negative one or else the thing turns off so like you want to mathematically verify for that for all cases it'll never return negative one but like if I'm just writing like a script for like um like uh you know some sales AI tool probably won't need it there yeah yes thanks thanks for coming great thought um just to clarify are you proposing that automated thear improving should be used across many domains where this is an example of one approaching

59:13

one I'm saying that this should be used in many domains to help solve the hallucination it's one part to help solve the hallucination problem and people are already thinking about this but just like putting more light onto try to shed more light onto it like this is a really useful area of AI research and and cryptography it's like an intersection right yes the verification is done bym or is it it's it's different instances of the same llm and they're all prompted to be different it's it's all GPT 4 or gpt3 for5 I think there should there's probably a theorem which says that llm

59:54

or any llm canot determine if another hasn't Hallin I think there is a famous theorem in computer science program cannot tell us whether it will ever finish in a finite time I think it's inherently incompatible that it doesn't know that it's so I feel like there is incomparability between mathematics and and yeah it's like if the compiler's output is true or false then it could say it but yeah it could still hallucinate if it's hallucinating the output of the compiler yeah so I guess that's still like a an issue you would need to think about

1:00:37

that yeah what was your inspiration for asking it to modify the col uh Conor what what did you find just ask it directly to prove the original col injector uh I asked chbt for the optimal prompt for it that would make it most likely I always ask if this like give me a scalar value for the highest likelihood that this is going to do the task that I want and then it gives me like top three like this prompt is you know 98% chance going to come up with a novel scientific discovery give it your system and I gave it the code base yes would you be concerned that uh you know all the sort of value would a creep the big players in the space if inherently you would reach uh some sort

1:01:18

of compute limit uh or would you consider some sort of you know if you're looking to solve these big problems like eventually the compute limit irrespective of whether or not you're indexing against mathematically proved theor to sort of substantiate your answer but uh would you consider that to be a good use case to decentralize compute I believe with Maran prime numbers they do something similar to that where they decentralize compute and you can actually offer a compute from your computer to help find the next Marian Prime I've always loved that idea uh bit tensor um there like any startup that's trying to centralized computer I'm always a fan of it's like it's very

1:01:59

hard obviously um I'm sure people are doing it well uh yes it's related to this yeah we're doing something I can talk about it oh perfect yeah yeah I I think yeah if if it if it works yeah in that case where there's a working world where decentralized compute works this would absolutely be amazing I think this could tie into incentives as well and yeah like ideally we just have people who are excited to do science and there's a path for them to excel outside of Academia and incentives are like a huge part of that so yeah it's really cool yes I didn't understand but what exactly is the hallucination problem oh

1:02:39

yeah yeah no good question hallucinations are something I deal with every day uh basically it's an AI that is going to Output text that is not true it's not grounded in physical based reality and it's a huge problem so it's just making up an answer it's making up an answer okay word salid the the AI for the inference the tone the syntax the logic is something is missing

1:03:26

in the translation in the transmission are you techn yeah I no I I think it's a really good question like um like there was this project deep dream where it showed how like an AI like Google had like many years ago where it was trained on a bunch of like dog images and then it was asked to like generate other images and it sh it still showed dogs and it was very trippy and it basically showed how neural networks and dreaming are very similar in like humans like this idea of like we see things but it's also like it shows patterns of things we've already seen and it concatenates everything together and so hallucinations I think are like like a waking dream of like a

1:04:08

of a language model where it's like it thinks that it's making the right connections between the weights of the network but but it's not in the right context it's it's essentially Dream It's ties of prompts though cuz like I think they they definit the studies where it's like you can ask like the same intended question but if you phrase it certain ways you'll get different responses yeah yeah yeah like this is like probably the biggest problem in AI today like the number one thing hallucinations and yeah I think this is probably the best way to solve them just

1:04:48

mathematically verify the outputs of your AI before it gives it to you so if ideally the AI does it before it even shows it to you it's like here's what I have to say it's like how can you be sure let me show you the proof and here's the proof and then here's all the citations and that's where we want to get to IDE yes I think this sort of um like you I believe articulated two or three levers in terms of uh you know improving the reliability of llm outputs one of which is uh you know pre-training the next would be test time computer comp but I'm sort of thinking now too like

1:05:29

the controls that we clearly don't have on academic research papers that you know are getting published with like Preposterous images and I've even heard of other you know situations where people sort of in a trolling type way submitted and were accepted to prestigious uh Publications like it really just does seem like if that is sort of the ground source of Truth for a lot of like the training Corpus Incorporated in these models like that's such a big problem that we're going to need to solve in order to just you know boost data quality sort of going in I feel like the decentralized approach might be the right way to do that absolutely that's a great Point uh like all these models are hallucinating and

1:06:09

they're hallucinating similar answers too like the word delve I'm not sure if you're you know Del all these models are saying delve and why is that why are all these models hallucinating the same way well there's two possible explanations for that the first one is that after GPT 3.5 every single language model was trained on synthetic data data generated by the GPT family so they're all just training off of GPT synthetic data that's one explanation but a more interesting explanation I think is called the pl is called the platonic ideal hypothesis and it was a paper that just came out of MIT two weeks ago I saw an X where it said that all llms converge on the same shared understanding of a statistical reality

1:06:50

because there exists an ideal statistical reality uh and play's suggested that back in the day there's this ideal reality all L are converging to it so that's one interesting one but uh it's it's unclear like is it the data or I thought like um on read it's like they people like label data for gbt and they sort like they come like to come from like one region so like tend to use the same like language just right like they um they use lot I remember somewhere right ah I could see that delve all over the place or certainly I remember CLA always starts with

1:07:30

certainly that yes do you think that like the fundamental issue of hallucination can be like resolved or is it something that's going to just kind of always be around an AI could only be like circumvented through Inc ification I think hallucinations are like going to be solved but they're always going to be at that base layer like the only way we're going to solve them is through math we're not going to solve it through more text for pre-training as far as understand hallucinations come from using neural networks and if that proof system that you just outlined here does not use neuron networks there will be no hust hallucinations in it because it just

1:08:10

uses a different method the sorry what if if if you're math proving automa proving system uses a different method but artificial intelligence that is not based on neural network then it will not hallucinate right it's clearly why more mod hallucinated when hallucinate when you understand how neuron networks actually works in practice the search some sort of optimal but they can't prove that the optimal is is global so if they're coming from a point where they're reaching a local optimal that doesn't work in a particular situation that is what people perceive as hallucinations but it is not a feature of artificial intelligence it's a feature of neural network which is the basic of current AUD inell that's right

1:08:51

the way I see it that's right this is not neural networks this is just its own functional programming so that's why it will not hallucinate that's right yeah yes mathematics is really infamously incomplete going back to's incompleteness theor do you ever worry about that like in the same sense the 100 years ago they tried to a aati all math they couldn't you wor that we won't be able to you know do that yeah I do worry about that but it's like um I I come back to like this idea of someone was mentioning like can you reuse these things and yes we can create this shared library of truth of just

1:09:32

these algebraic relations and then it's going to set off this self-improving feedback loop where like this AI is at the edge of human knowledge if you think of all knowledge as a bubble it's right at the edge it's got enough context of everything we've ever done and now with this search strategy that's optimize it's going to find new knowledge and we just buckle up and we're ready for the ride because it just gets faster and faster but you're right though and I should have mentioned Gold's Inc completeness theorem and I'm still learning about that that's a really great point that all math is still not verified and that is still a huge issue but yeah I always come back to it we'll just get the AGI to do it not only not it's verified cannot be

1:10:14

verified Cann be doesn't matter how much computer get so the math of all what however you specify there are some things you cannot prove put 100 million llms still not going do it yeah no that's a that's a huge issue I got to learn more about that yes do you see a potential to use this system to perhaps upload a a a journal you know a piew study and have it si through it and understand which parts are true and which aren't yeah and and there's existing tools that

1:10:54

do that I wanted to build that at first but I was like damnn it they already built it so but yeah it's just you upload it it's like there's a bunch of like SAS tools I've seen it's called research paper. I saw this one yeah that would be super useful as well just to do that yes why is it proper to like useof by uh deduction rather than like contradiction or like exhaustion not Sor no no you're right in the back no you're right it it could be done through exhaustion or I just chose one example here but you're right there's so much more to how like so many more ways we could prove something yes is is you have like a very

1:11:35

well- defined problem here where it's like you're trying to like solve an optimization problem sort of for like a mathematical function do you think all of science can be um like abstracted to that or is like wet lab stuff fundamentally different I do think fundamentally we can do it in silico it's just about like solving physics like that Bas you know Elon Musk always talks about like looking at the world through like a physics based first principle view uh physics like biophysics like that's a whole field of study like how do these molecules and drugs interact if we can solve physics like the base layer in silico then we can do all sorts of drug Discovery yeah

1:12:15

I think we can simulate all of reality on a computer uh we could be in one right now so so is there no need for like envio or stuff no need what nid experimentation in the future we have the compute right now we don't have the compute right now eventually I would that's the goal I I don't think eventually we will meet that yeah got it it's very archaic 12 years I want to see a world where a startup of three people can just make a drug Discovery a drug Discovery ST you never see that you need bu a lot of money but if you could do it all in silicone verify it then you can have people saying like here's our drugs and it's like a whole new industry are there any scientific questions that are

1:12:55

like so open-ended that like don't fall under this category like I mean I can't even think of one now I like life extension maybe that is actually a very easy thing to optimize for because you're just trying to make people live longer but are there any like when someone's like beginning a PhD they don't really know what they're doing and they explore a lot of things is there no is that is that just a well not a well defined scientific research question then I think that there there are sciences that are not able to be defined mathematically in this way a lot more like empirical science like social science um I haven't thought about it

1:13:37

but I do I do think that that's like is there an objective Beauty like I I think of it like that kind of question like it's it's I'm not sure okay but I I would think there is something outside the scope of this yes just building one like you're saying I mean how how optimistic are you that this generalizes Beyond mathematic I mean the defining characteristic of mathematics is purely abstract and even Alpha was not proven it was Benchmark MH so how optimistic are you that we can use a totally automated Discovery pipeline like this realistically in

1:14:17

other fields I'm pretty optimistic it's just a question of motivation I feel like the alpha geometry Alpha teams they collected the right data sets they put in a lot of time I feel like this can be done uh in an open way uh stable diffusion they have like stable bio as well um there's a bunch of open science projects out there obviously D but like I I do think it can be done it's going to be difficult but it can be done I don't see I don't I don't really see any like blocker

1:15:00

yeah yes when it will be done and everything will be you know really work what if we will start to use your model for you know the 100 century ago proven uh mathematical stuff or I don't know and we will see that people that time was lying or you know missing yes and it it look like with who have to trust with machine or with people and it looks like where will be new uh models for whom we have to you know really believe and maybe teachers will teach uh future children how to uh connect with machine

1:15:43

and believe trust in machine not with you know our our mindset how we are thinking as human when we solve equals you know when we use algebra or whatever it's it's so really interesting and it's like decentralized so what do you think about your future where are you going because it's changing it's like already 100 people will talk about it and it's like booming like you know one second what do you think yeah I I mostly feel excited for the future you're right it's moving so fast uh you mentioned connecting brains I I I do think that's

1:16:25

really exciting there's a great show on this that I just finished watching it's called Pantheon it's about uploading your mind to the internet um and uh yeah I just feel very excited I feel like uh in a lot of ways the Golden Era hasn't even started it's about to start in the next 5 to 10 years and we're going to look back and be like how could we even live like that before it's going to be so much better we're going to have so much more capability when we connect our brains to the machines and um there's like Concepts like music that we understand as human because our brains are smart enough but like an animal couldn't understand I feel eventually when we connect our brains to the internet there will be Concepts like music that will begin to understand that

1:17:06

right now we don't even understand what they are similarly in Math and Science and Arts and everything it's going to be a it's going to be wonderful they have BMI meet grp they have what kind of meet upch oh that's very cool in New York and BCI BR computer awesome yes I'm I'm not as enthusiastic potential for manipulation is so great you know you see what's happening in our last election you know yeah yeah the big the big guys are going to be having control over this technology and people who are going to connect their brains to the internet they're going to be controlled by these people also we got to decentralize the computer right

1:17:47

yeah could go badly though yeah any other questions thank you guys [Applause] for sure to follow SJ on Twitter what's your handle you want SJ s i r a j r a v a l so make sure to follow up on Twitter make sure to follow him on YouTube he has really great YouTube content he has really great Twitter content so make sure to give him a follow and feel free to ask him questions after so thank you again I appreciate it have a good night everybody [Applause]

1:18:42

[Music]

1:19:11

you know I [Music]

1:19:36

simar [Music]

1:22:09

he is a very

1:23:08

different

1:23:37

all [Applause] [Applause] [Music] as far as I can tell

1:24:30

sometimes

1:25:16

maut down

1:25:41

[Applause]

1:25:53

El you [Applause]

1:26:13

[Applause]

1:26:24

the world

1:26:36

[Music]

1:27:53

something

1:28:22

somethings s

1:28:57

[Music]