How AI is Eating Finance - with Mike Conover of Brightwave

Hey everyone welcome to the lateen space podcast this is alesio partner and CTO and residence and deible partners and I have no co-hosts today as you can see um this works is in Vienna at iclr having fun in in Europe and uh we're in the brand new studio um as you might see if you're on YouTube there's still no sound panels on the wall Mike tried really hard to put them up but the the glue the glue uh is a little too whole for that Um so if you hear any echo or anything like that

um sorry but we're doing we're doing the best that we can um and today we have our first repeat guest Mike Conover um W Mike who's now the founder of Bri wve that's right not not a data breaks anymore that's right yeah um pleased to be back yeah no no the our last episode was one of the fan favorites and I think this will be uh just as good so for for those that have not listened to the first episode which Might be many because the podcast has gr a lost since then thanks to people

like Mike who have interesting conversations on it um you spent uh a bunch of years doing ml as some of the best companies on the internet um things like workday you know Skip flag LinkedIn um most recently at data brecks where you were leading the the open source large language models um Team working on doly um and now you're doing briway which is in the financial services uh space but This is not something new you know I think when you and I first talked um about briway I was like why is this guy doing a

financial services company and then you look at your back ground and you were doing papers on nature about on the nature magazine about uh LinkedIn data predicting S&P 500 stock Movement Like many many years ago so what's what what's kind of like the uh some of the tying elements in your background that maybe people are overlooking that Brought you to do this yeah sure yeah so I would say my so my PhD research was funded by DARPA and it we had access to the Twitter data set ear in the early in the natural history of

the availability of that data set and it was focused on the large scale structure of uh propaganda and misinformation campaigns and Linkedin we had Planet scale descriptions of the structure of the global economy and so primarily my work Was was homepage Newsfeed relevance so when you go to linkedin.com you would see updates from one of our machine learning models um but additionally I was a research liaison as part of the economic graph Challenge and um had this nature Communications paper where we demonstrated that 500 million jobs transitions can be hierarchically clustered as a network of

Labor flows and our predictive next quarter S&P 500 market cap changes And at workday I was director of financials machine learning um and you start to see how organizations are organisms and I I think of the way that like an accountant or the market uh encodes information and databases similar to how social insects for example organize their work and make Collective decisions about where to allocate resources or time and attention and um that especially with the the work On Twitter we would see um Network structures relating to polarization emerge organically out of the interactions of

many individual components and so like much of my professional work has been focused on this idea that our lives are govern by systems that we're unable to see from our locally constrained perspective and that we when we when humans interact with technology they create digital Trace data that allows us to observe the Structure of those systems as though through a microscope or a telescope and particularly as regards Finance I think this is the ultimate the markets are the ultimate manifestation and record of that Collective decision- making process that humans engage in just to start going

off script right away uh how do you think about some these interactions creating the polarization and how that reflects in the language models today because They're trained on this data like do you think the models pick up on these things on their own absolutely yeah I think they are a compression of the world as it existed at the point in time when they were pre-trained and so I I think absolutely the and you see this in word devac too I mean just the the semantics of how we uh think about gender as Rel relates to

professions are um encoded in the structure of the these models and like language models I think are you Know much more sort of uh complete representation of of human sort of beliefs MH yeah that's awesome so we left you at data bricks last time you were building Dolly tell us a bit more about bright wave this is the first time you're really talking about it publicly yeah yeah it's a it's a pleasure I mean so we we've raised a $6 million seed round uh including participate led by deciel uh we love working with and including

participation from 72 one of The largest hedge funds in the world and and Moonfire Ventures and um we are uh focused on like if you think of the job of an active asset manager the the work to be done is to understand something about the market that nobody else has seen in order to identify a mispriced asset and it's our view that that is not a task that is well suited to human in elector attention span and so much as I was gesturing towards the ability of these models to perceive more than a Human is

able to um we think that there's a a unique historically unique opportunity to expand individual's ability to reason about the structure of the economy and the markets um it's not clear that you get superhuman reasoning capabilities from Human level demonstrations of skill and by that I mean the pre-training Corpus but then Additionally the fine-tuning corpuses I think you largely mimic the demonstrations that are Present uh at at model training time um but from a working memory standpoint these models out class humans and their ability to to reason about these systems Y and you started briway

with Brandon what's the what's the story you two work together workday uh but he also has a really relevant background yes so Brandon Katara is my co-founder uh the CTO and he's a a very special human so he's has a deep background in finance so he was the former CTO of a federally Regulated derivatives exchange um but his first deep learning patent was filed in 2018 and so he had he spans worlds he has experience building Mission critical infrastructure and highly regulated environments for finance use cases um but also uh was was very early to

the Deep learning uh party and and understand he led at workday um was the tech lead for semantic search over hundreds of millions of resumés and um job listings um and so just has been Working with uh information retrieval you know neural information retrieval methods um for a very long time and so uh was a an exceptional person and I'm I'm glad to count him among um the people that we're doing this with yeah and a great fisherman so yeah very important Talent very enthusiastic exactly um and then you have a bunch of amazing engineers

then you have folks like jpu used to work at Goldman Sachs yeah how should people Think about team building in this more vertical domain obviously you come from a deep ml background but you also need some of the industry SS what's what's the right balance yeah I mean this it's so I think one of the things that's interesting about Building verticalized Solutions in AI in 2024 is that historically um you need the AI capability you need to understand both how the models behave and then how to get them to interact With other kinds of uh

machine learning subsystems that together perform the work of a system that can reason on behalf of a human there are also material systems engineering problems in there so I saw I forget who this is attributed to but a tweet that uh sort of made reference to all the AI comp all of the traditional software companies are trying to hire AI talent and all the AI companies are trying to hire systems engineers and that is 100% the case Getting these systems to behave um in a predictable and repeatable and observable way is um equally challenging to

a lot of methodological challenges but then you bring in whether it's law or medicine or public policy or in our case Finance um I think a lot of the most valuable like grammarly is a good example of a company that has generative work product that is um a valuable by most humans whereas in finance the the character of the Insight The depth of insight and the non-consensus of the Insight um really requires uh fairly deep domain expertise and like even operating an exchange I mean when we went to to raise our R A lot of

people said why why why don't you start a hedge fund MH um and it's like that is a totally separate there are many many separate skills that are unrelated to um AI in in that problem and so like we've um we brought to get brought into the full domain Experts um in finance who who can help us um evaluate the character and sort of steer the system M yeah so that's the team what does the system actually do what's the the briadway product yeah I mean it's it's in it does many many things um but

it will it it acts as a partner in in a partner in thought to finance professionals so um you can ask bright wve um a question like how is nvidia's position in the GPU Market impacted by Rare Earth metal shortages And it will identify um as thematic contributors to an investment decision um or or developing your thesis that um in response to export controls on a100 cards uh China has put in place licensers on the transfer of geranium and gallium which are not rare earth metals but they're semiconductor production inputs and has expanded its control

of African and South American mining operations and so we See uh if you think about you know we have a a $20 billion crossover hedge fund uh and their their equities team uses this tool to go deep on a thesis so I was describing this like you know multi multiple steps into the value chain or supply chain for for companies um we see uh you know wealth management professionals using bright wave to get up to speed extremely quickly as they step into nine conversations tomorrow uh with clients who are assessing like do You know something

that I don't can I trust you to to be a steward of my um my financial well-being um we see investor relations teams using brightwave to um you just think about the universe of coverage that a person working in finance needs to be aware of the ability to rip through filings and transcripts and have a very comprehensive view of the market um it's it's extremely rate limited by how quickly a person is able to read and not just read but like um Solve the blank page problem knowing what to say about a factor of finding

what all can you share about customers that you're working with yeah so we we have seen uh traction that far exceeded our expectations uh from the market this the you sit somebody down with a system that can take any question and generate tight actionable financial analysis on that subject and the product kind of sells itself and so we see um many many different funds firms and strategies That are making use of bright wave so you've got um 10 person owner operated registered investment advisor the classical wealth manager you know $500 million in AUM um we

have crossover hedge funds that have tens and tens of billions of dollars in assets under management very different use case so that's more investment research um where the wealth manager is going to use this to step into client interactions just exceptionally well prepared we see Investor relation or investor relations teams um we see corporate strategy types that are um needing to to understand very quickly um new markets new themes um and just the ability to very quickly develop a view on any investment theme or sort of strategic um consideration is is broadly applicable to many

many different kinds of personas yep um yeah I I I can attest to the product selling itself given that I'm a user um let let's jump into some of the technical Challenges and work behind it because um there are a lot of things so as I mentioned you were on the podcast about a year ago yep you had released Dolly from data bricks which was one of the the first open source llms yep um Dolly had whooping 1,24 tokens of contact size and today you know I think a thousand tokens a model will be yeah

exactly um how how did you think about the evolution of contact sizes as you built the company um and where we Are today what are things that people give wrong any any commentary there sure so I we we take a very very much take a systems of systems approach so I think that this is when I started the company I think I had more faith in the ability of large context Windows to generally solve problems relating to synthesis and actually like if you think about the attention mechanism and um the way that it computes similarities

between tokens At a distance um I on some level believed that as you would scale that up you would have the ability to simultaneously perceive um and draw conclusions across vast disparate bodies of content and I think um that is does not empirically seem to be the case so when when for example you and this this is something anybody can try um take a very long document like needle and a Hast stack I think sure we can do information retrieval on like Specific fact finding activities pretty pretty easily but if you I kind of think

about it like summarizing if you write a book report on an entire book versus a synopsis of each individual chapter MH there there is a characteristic output length for these models let's say it's about 1,200 tokens like it is very difficult to get any of the commercial LMS or llama to write 5,000 tokens and you think about it as like what is the Conditional probability that I generate an n token it just gets higher the more tokens are in the context window prior to that um sort of next INF step and so um if I

have a thousand words in which to say something the level of specificity and the level of depth when I am assessing a very large body of content is going to necessarily be less than if I am saying something specific about a subpassage and so we I if you Think about um drawing a parallel to Consumer internet companies like LinkedIn or Facebook um there are many different subsystems within so let's take the Facebook example Facebook uh almost certainly has uh I mean you can see this in your profile your inferred interests what are the things that

it believes that you care about those assessments almost certainly feed into the feed relevance algorithms that would judge What you are you know am I am I going to see you snow am I going to show you snowboarding content I'm going to show you um Aviation content it's the outputs of one machine Learning System feeding into another machine Learning System and I think with modern Rag and sort of agent-based reasoning it is really about creating subsystems that do specific tasks well and I think the problem of um deciding how to decompose large documents into more

kind of a I Reasoning units is is still very important now there it's an open question whether you have um in the whether that is a model that is addressable by pre-training or um instruction tuning like can you uh have synthesis oriented uh demonstrations in the at training time and and now this problem is more robustly solved um because synthesis is quite different from uh Del the next word in The Great Gatsby y um but it I think empirically Is not the case that you can just throw all of the SEC filings in you know

a million token context window and get um deep Insight that is useful out the other end yeah and I think that's the main difference about what you're doing it's not about summarizing it's about coming up with different ideas and kind of like thought threads yeah I think this specifically like helping a person know um you know if I think that gp1s are going to blow Up the diet industry um identifying and putting in context a negative result from a human clinical trial that's or for example that adherence rates to OIC after a year are just

35% what are the implications of this so it there's an information retrieval component and then there's a not just presenting me with a summary of like here's here are the facts but like what does this entail Y and how Does this fit into my worldview my fun strategy um broadly I think that you know I mean this this idea I think is very um eloquently puts it which is and this is this is not my insight but that language models and help help me know who said this you may you may be familiar but language

models are not tools for creating new knowledge they're they're tools for helping me create new knowledge like they themselves do not do That work um I think that that's the presently the right way to think about it yeah I I read a tweet about Needle on the Hast stack actually being harmful to some of this work because now the model is like too focused on recalling everything versus saying oh that doesn't matter or like ignoring if you think about a S1 filing like 85% is like Bower plate it's like yes you know previous performance doesn't

guarantee future performance Like the company might not be able to turn in the future blah blah blah all these things they always come up again bit and currency fluctu y y y we have a large War force and all of that um have you had to do any work at the model level to kind of like make it okay to forget these things or like have you found that like kind of like making it a smaller problem then cutting putting them back together kind of solves for that Absol absolutely and I think this Is where

um having domain expertise around the structure of these documents so if if you look at the different chunking strategies that you can employ to um understand like what is the intent of this clause or phrase and then really um be selective at retrieval time in order to get the information that is most relevant to a user query based on the semantics of that unique document and I I think it is not um it's certainly not Just a sliding window over that that that Corpus yeah um and then the flip side of it is obviously factuality

don't want to forget than that they were there um how do you tackle that yeah I mean of course that's it's a very deep problem and I think you know I'll be a little circumspect about the specific kinds of methods we use but um this sort of multiple passes over the material and saying how convicted are you that what you're saying is in fact True and you I mean you can take Generations from multiple different models and compare contrast and say like do these both reach the same conclusion um we you can treat it like

a voting problem um you we train our own models to assess um you know you can think of this like entailment like is is this supported by the underlying primary sources and I think that you have um methodological approaches to this problem but then you also have product Affordances so like there's a a great blog post on bar from The Bard team describing um an abil it was sort of a designled um product Innovation that allows you to ask the model to double check the work so if you have a surprising finding we can let

the user uh discretionarily spend more compute to double check the work and I think that you want to build product experiences that are fault tolerant and it is Unclear that you ever that the difference between like hallucination and creativity is fuzzy and so do you do you ever get language models with next token prediction as the loss function that are um guaranteed to not contain factual misstatements that is not clear now maybe being able to invoke code interpreter M like code generation and then execution in a secure way um helps to solve some of these

problems especially for quantitative reasoning That that may be the case but um for right now I think you need to have product affordances that allow you to live with the reality that these things are um are fallible y yeah we did a rhf 2011 episode just talking about different methods and and whatnot how do you think about something like this where it's maybe unclear in the short term even if the product is right you know it might give it might give an Insight that might Be right but it might not prove until later so it's

kind of hard for the use to say that's wrong because actually it might be like you think it's wrong you know like in investment that's kind of what it comes down to you know some people are wrong some people are right um how how do you think about some of the product features that you need and something like this to bring user feedback into the mix and maybe how you approached it today and how you think About it long term yeah well I mean I think that to your point about the model may have the

model may make a statement which is not actually verifiable it's like this this may be the case I think that is where the reason we think of this as a partner in thought is that humans are always going to have access to information that is not not been digitized and so in finance you see that especially so with regards to expert Call networks um the sort of like un like your in the unstated investment thesis that a portfolio manager may have like we just don't do biotech um or we do not believe that um we

think that Eli Eli Lily is actually very exposed because of how unpleasant it is to take exek right those those are things that um are beliefs about the world but that may not be um like falsifiable right now and so I think you have to you can again take play pages from The consumer web Playbook and think about um personalization mhm so it is getting a person to articulate everything that they believe is not a realistic task um Netflix doesn't ask you to describe what kinds of movies you like and they give you the option

to vote but nobody does this um and so what I think you do is You observe people's revealed preferences like what so one of the capabilities that our system exposes is um given everything that brightwave Has read and assessed and like the the sort of synthesized financial analysis what are the natural next questions that this that a person investigating this subject should ask and you can think of this Chain of Thought and this deepening um kind of investigative process and the direction in which the user steers the attention of this system uh reveals information about

what do they care about what do they believe what kinds of things um are important and so uh at the Individual level but then also at the the fund and firm level you can develop like an implicit um representation of your beliefs about the world in a way that um you just you're never going to get every somebody to write everything down like that yeah yeah um how does that tie into one of our other favorite topics evals uh we had David Lan from adapt and he mentioned they don't care about benchmarks because their customers

don't Work on benchmarks they work on C you know business results uh how do you think about that for you and maybe um as you build a new company when is the time to like still focus on The Benchmark versus when it's time to like move on to your own evaluation using maybe yeah uh labelers or or whatnot so I mean we we use a fair bit of llm supervision uh to evaluate multiple different subsystems and I think that one of the reasons that I mean we we we pay human annotators to Evaluate the quality

of of the generative outputs and I think that that is always the the reference standard but we frequently first turn to LM supervision as a way to um have uh whether it's at fine-tuning time or even for subsystems that are not generative like what is the quality of the system and I think we will generate a small Corpus of highquality domain expert annotations and then always compare that against how well is either llm Supervision or even just a heuristic right like a simple thing you can do this is this is a technique that we do

not use but an example um do not generate any integers or any numbers that are not present in the underlying Source data right you know if you're doing rag you can just say you can't name numbers that are not you know it's very sort of heavy-handed but you can take the annotations of a human evaluator and then compare that I mean Snorkel kind of takes a similar perspective like multiple different weak um sort of supervision data sets can give you substantial more than any one of them does on their own and so I think you

want to compare the quality of any evaluation against human generated the the sort of like Benchmark uh but at the end of the day like eventually you especially for things that are nuanced like is this Transcendent poetry mhm there's just no way to multiple choice Your way out of that you know and so um really where I think a lot of the fly wheels for some of the large LM companies are it's methodological obviously but it's also just data generation and you think about like you know for anybody who's done um crowd Source work and

this I think applies to high-skilled human annotators as well like you look at the Google search quality evaluator guidelines it's like a a 90 or 120 page rubric Describing like what is a high quality search result and it's like very difficult to get on the human level people to reproducibly follow a rubric and so what is your process for orchestrating that motion like how do you um articulate what is high quality Insight I think that's where a lot of the um work actually happens uh and that it's sort of The Last Resort like I everything

like ideally you want to automate Everything But ultimately the Most interesting problems right now are those that are not especially automatable one thing you did a data brecks was the um well not that you did specifically but the thing did was like the dolly 15K data set um you mentioned um people Mis value the the the the value of this data why has no other company done anything similar with like creating this like employee Le uh data set you can imagine you know some of these like Goldman Sachs they got like Thousands and thousands of

people in there um obviously they have different privacy and whatnot requirements uh do you think more companies should do it like a do you think there's like a misunderstanding of how valuable that is or uh yeah so I I think data bricks is a very special company and led by people who are um very sort of cor courageous I guess is one word for it just like let's just ship it and I think it's um it's unusual And it's also because uh I think in like most companies will recognize like if they go to the

effort to produce something like that they recognize that it is competitive advantage to have it and to be the only company that has it and I think data bricks is an un is in an unusual position and that they um benefit from more people having access to these kinds of um sour but you also saw scale I guess they they haven't released it I'm sure they have it Because they charge people a lot of money but they they created that alternative to the uh gsmk 8K I believe was was the is how that said and

um I I guess they you know they two are not releasing that yeah um it's it's interesting because I talked to a lot of Enterprises and a lot of them are like man I spent so much money on scale and I'm like why don't you just do it and They're like what but this so I think this again again gets to the like the human process orchestration like how like it's one thing to do like a single monolithic push to create a training data set like that or an evaluation Corpus but I think it's to

have a repeatable process and a lot of that I think realistically is um pretty unsexy like people management work yeah so that's probably a big part of it we have this uh four Wars of AI framework uh the Data quality War we we kind of touched on a little bit now about rag that's like the other Battlefield Rag and context sizes and kind of like all these all these different things sure uh you work in a space that has um a couple different things one um temporality of data is important because every quarter there's new

data and like the new data usually overrides the previous one so you cannot just like do semantic search and hope that you get the latest one um And then you have obviously very structure numbers thing that are very important to the Token level like you know 50% gross margins and 30% gross margins are very different but you know the tokenization is not that different um any thoughts on like how to build a system to to endle all of that as much as you can share of course yeah absolutely so I think this again rather than

having open-ended retrieval open-ended reasoning you our Approach is to decompose the problem into multiple different subsystems that have specific goals and so you I mean temporality is a great example um the when you think about time I mean just look at all of the libraries for managing calendars time is kind of at the intersection of Lang anguage and math and this is one of the places where without um without taking specific technical measures to ensure that you Get high quality narrative overlays of Statistics that are changing over time and have a description of how a

PE multiple is increasing or decreasing and a rag like a retrieval system that is aware of the time sort of the time intent of the user quer right if I'm asking something about breaking news like that's going to be very different than if I'm looking for a thematic account of the past you know 18 months in fed interest rate policy um you have To have retrieval systems that are your point like if I just look for something that is a nearest neighbor without any of that uh temporal or other qualitative metadata overlay you're just going

to get a kind of a bag of facts and that that is like explicitly not helpful um because the the worst failure state for these systems is that they are um wrong in a convincing way and so I think at least presently you have to have um subsystems that are aware of the Semantics of the documents or aware of the semantics of the intent behind the question um and then have multiple evalu we have multiple evaluation steps once you have um the generated outputs we we assess it multiple different ways to know um is is

this a factual statement given the the sort of content that's been retrieved MH yep and what about I think people think of financial services they think of privacy confidentiality um yeah what's kind of Like customers interest in that as far as like sharing documents and like how much of a deal breaker is that if you don't have them um I don't know if you want to share any on uh about that and how you think about architecting the product yeah so one of the things that gives our customers a high degree of confidence um is

the fact that Brandon operated a a federally regulated derivatives exchange um that experience in these highly Regulated environments I mean I additionally at at workday I worked with the financial's product and you know without going into specifics like um it's exceptionally sensitive data um and you have multiple tenants and it's uh just important that you take the right approach to being a steward of that material and so from the start we've built in a way that anticipates um the need for controls on How that data is managed and and who has access to it and

how it is um treated throughout the life cycle and so that for um our customer base where frequently the most interesting and Alpha generating material is not publicly available um has given them a great degree of confidence in Sharing um some of this the most sensitive and interesting material um with systems that are able to combine it with content That is um either publicly or semi-public available um to create non-consensus insight into some of the most interesting and challenging problems in finance yeah we always sayra are recommendation systems for llms uh how do you think

about that when you have private versus public data where sometimes you have public data as one thing but then the private is like well actually you know we got this like inside model like with this inside scoop That we going to like figure it out um how do you think in the r system about a value of the different documents you know I know a lot of it is secr sauce but no no it's it's fine I mean I think that there is um so I I will gesture towards this by way of saying context

where prompting so you can have prompts that are composable and that have different um sort of command units that like may or may not be Present based on the semantics of the content that is being populated into the the rag context window and so that that's something we make great use of which is um where where is this being retrieved from what does it represent and what should be in the instruction set in order to treat and respect the underlying contents not just as like here's a bunch of text like you figure it out but

um this is this is important in the following way or you Know these this aspect of the SEC filings are just categorically uninteresting or this um this is cellsite analysis from a favored Source right and so it's that um creating it much like you have with the the quality the the problem of organizing the work of humans you have the problem of organizing the work of all of these different Ai subsystems and getting them to um propagate what they know through the rest of the stack so that you know If you have multiple you know

7 10 sequence uh inference calls that all of the relevant metadata is propagated through that system and that you are um aware of where did this come from how convicted am I that it is a source that should be trusted I mean you see this also just um in analysis right so different like Seeking Alpha right is a good example of just a lot of people with opinions and some of them are great some of them are really mid and how do You um how do you build a system that is aware of uh the

user's preferences for different sources mhm um I think it this is all uh related to how you we talked about systems engineering it's all related to how you actually build the systems mhm and then just to kind of wrap on the r side how should people think about knowledge graphs and um kind of like extraction from documents versus just like semantic search over the documents Knowledge Graph extraction is An area where we're making a pretty substantial investment and so this I think that it is underappreciated how powerful like there's the generative capabilities of language models

but there's also the ability to um program them to function as arbitrary machine Learning Systems basically for marginally zero cost and so the ability to extract structured information from huge like sort of unfathomably large bodies of content in a way that is Single pass so rather than having to um reanalyze a document every time that you perform inference or respond to a user query we uh believe quite firmly that you can also in an additive way perform single pass extraction over over this body of text and then bring that into the rag context window and

um this this really sort of levers off of my experience at LinkedIn where you had this structured graph representation of the global economy where you said Person a works at Company B we believe that there's an opportunity to create a a knowledge graph that has resolution that greatly exceeds what any you know whether it's Bloomberg or LinkedIn currently has access to where we're getting as granulars person X submitted Congressional testimony that was critical of organization Y and this is the language that is attached to that testimony and then you have a structured data artifact that

you can pivot through And reason over that is complimentary to the generative capabilities that language models Expose and so it's the same technology being applied to multiple different ends and this is Manifest in the product surface where it's a highly facetal pivotable product uh but it also enhances the reasoning capability of the system yeah uh you know when you mentioned you don't want to requery like the same thing over and over a lot of people may say well I'll Just fine-tune this information in the model you know um how do you think about that that

that was one thing when we started working together you were like we're not building Foundation models a lot of other startups were like oh we're building the finance financial model like the finance Foundation model or whatever um when when is the right time for people to do um fine tuning versus rag is like any tics that you can share that you use to to think about it so We in general I do not I'll just say like I don't have um a strong opinion about how much information you can imbue into a model that is

not present in pre-training through large scale fine-tuning um the benefit of rag is the ground the capability of ground grounded reasoning so the you know forcing it to attend to a collection of facts that are are known and available at inference time and sort of like materially like only using these facts um the at least In my view the the role of fine tuning is really more around I think of like language models kind of like a stem cell and then under fine tuning they differentiate into different kinds of um specific cells so kidney or

an eye cell and um we if you think about like specifically like I don't think that unbounded agentic behaviors are useful and that instead a a useful LM System is more like um a finite State machine where the behavior of the system is occupying one of many different behavioral regimes and making decisions about what State Should I occupy next in order to satisfy the goal um as you think about the graph of those states that the language mod that your system is moving through once you develop conviction that one behavior is useful and repeatable and

like worthwhile to um Differentiate down into a specific kind of subsystem that's where like fine-tuning and like specifically generating the training data like having having human annotators produce a corpus that is useful enough to get a specific class of behaviors that that's kind of how we use fine tuning rather than trying to imbue new net new information into these systems MH yeah um and how what you know people always try to turn llms into humans it's like oh this is my Reviewer this is my editor I know you're not in that camp so any any

thoughts you have on like how people should think about um yeah how to refer to models like and so I mean we we've talked a little bit about this and it's it is an it's notable that I think there's a lot of anthropomorphizing going on and that it reflects the difficulty of evaluating the systems is it like does does saying that you are uh that you're the a Journal editor for nature does that help like you're you know you've got the editor and then you've got the reviewer and you've got the you know you're the

private investigator you know it's like this is I think um literally we wave our hands and we say maybe if I tell you that I'm going to tip you that's going to help and it sort of seems to and like maybe it's just like the more Cycles the more compute that is Attached to the prompt and and then the sort of like Chain of Thought uh at inference time it's like maybe that's all that we're really doing and that it's kind of like hidden um hidden compute but I our experience has been that you can

get really really high quality reasoning from roughly an agentic system without um needing to be too cute about it you can describe the task and um you know within welldefined m Uh you you don't need to um treat the llm like a person in order to get it to generate high quality outputs yeah and the other thing is like all these agent Frameworks are uh assuming everything is an llm you know for sure and I think this is one of the places where um traditional machine learning has a real material role to play in producing

A system that hangs together and there are you know guarantee like statistical promises that Classical machine Learning Systems to include traditional deep learning can make about you know what is the set of outputs and like what is the characteristic distribution of those outputs that LMS cannot afford and so like one of the things that we do is we as a philosophy try to choose the right tool for the job and so sometimes that is um a a denovo model that has nothing to do with LMS that does one thing exceptionally well and whether that's uh

Retrieval or uh critique or multiclass classification it um I think having many many different Tools in your toolbox is is always valuable this is great um so there's kind of the the missing piece that maybe people are wondering about uh you do a financial services company and you don't do anything in Excel uh what's the what's the story behind what you're doing partner in thought versus hey this is like AI enable model that understands Any stock and and all that yeah and to be clear we we do brightwave does a fair amount of quantitative reasoning

I think what is an explicit non-goal for the company is to uh create Excel spreadsheets and I think when you look at the products that work in that way you can spend hours with an Excel spreadsheet and not notice a subtle bug and as a like that is a highly non-fa Tolerant product experience where you encounter a misstatement in a financial model in terms of how a formulas composed and all of your assumptions are suddenly violated and now it's effectively wasted effort so as opposed to the partner and thought modality which is yes and like

if if the model says something that you don't agree with you can say takeen under consideration I you know this is not interesting to me I'm going to Pivot to The next finding or claim and it's more like a dialogue um the other piece of this is that the financial modeling is often very when we when we talk to our users it's very personal so they have a specific view of how a company is structured they have the you know one key driver of asset performance that they think is really really important um it's kind

of like the difference between writing an essay and having an essay I Guess like the pro purpose of homework is to actually develop what do I think about this and so it's not clear to me that like push a button have a financial model is is solving the actual problem that the financial model affords um and so we that said we take great efforts to have exceptionally high quality quantitative reasoning so we um you know you think about um and I I won't get into too many specifics about this but uh we deal with with

a fair number of Documents that have tabular data that is uh really important to making informed decisions and so the way that uh our rag systems operate over and retrieve from tabular data sources is um it's something that we um place a great degree of emphasis on it's just I think the the the medium of like Excel spreadsheets is just I think not the right play for this class of Technologies as they exist in 2024 mhm yeah what about uh 20343 are people still going to be making Excel models or like yeah I think to

me the most interesting thing is like how are the models abstracting people away from some of these more syntax driven thing and making them focus on what matters to them yeah I wouldn't be able to tell you what the future you know 10 years from now looks like I think anybody who could convince you of that um is is not necessar to be trusted um I do think That so draw let's draw the parallel to um accountants in the the 70s so visaal I believe came out in 1979 and historically the core ID you know

you would have as an accountant as a finance professional in the 70s like I'm the one who runs the I'm I run the numbers I do the arithmetic that's like my main job and we think that I mean you just look that no that's not a job anybody wants and the sophistication of the analysis That a person is able to perform as a function of having access to powerful tools like computational spreadsheets um is is just much greater and so I think that with regards to language models it it is probably the case that there

is a a play in the workflow where it um is commenting on your analysis within that you know spreadsheet based context or it is taking information from those models and sucking this into a system that does qualitative reasoning on top of that um But I think the um it is an open question as to whether the actual production of those models maintain is is still a human task but I think the sophistication of the analysis that is available to us um and the completeness of that analysis um just necessarily in increases over time mhm yeah

uh what about AI hedge funds obviously I mean we have quants today right but those are more uh kind of like momentum driven kind of like signal Driven unless about long thesis driven uh do you think that's a possibility it's this is an interesting question I would I would put it back to you and say like how um how different is that from what hedge funds do now I think there is the more that I have learned about how teams at hedge funds actually behave and you look at like systematics desks or semi-static trading groups

man it's a lot like a big machine learning team and It's I sort of think it's interesting right so like if you look at video games and traditional like Bay Area Tech there's not a ton of like Talent Mobility between those two communities you have people that work in video games and people that work in like SAS software and it's not that like cognitively they would not be able to work together it's just like a different set of skill sets a different set of relationships and it's kind of like Network clusters that don't interact at

I think there's probably a similar phenomenon happening with regards to uh machine learning within the active asset allocation community and so like I'm it's actually not clear to me that we don't have ai hedge funds now the question of whether you have an AI that is operating a trading that that seems a little um maybe like I don't have line of sight to something like that I'm always Curious you know I think about uh Asset Management on a few different ways but uh venture capital is like extremely power law driven it's really hard to do

machine learning in power law businesses because you know the distribution of outcomes is like so small versus public equities most high frequency Trading it's like very you know bell curve normal distribution it's like even if you just get 50.5% at the right scale you're going to Make a lot of money yeah uh and I think AI starts there right and today most high frequency trading is already AI driven you know Renaissance started a long time ago using using these models uh but I'm curious how it's going to move closer and closer to like power L

businesses right I would say some Boutique catch funds their their pitch is like hey we differentiated because we only do kind of like these long only strategies that are like thesis driven Versus uh you know movement driven y um and most Venture capitalists will tell you well our fund is different because we have this unique thesis on on this Market um and I think like five years ago I've wrote this block post about why machine learning would never work in Venture because the things that you're investing in today there just like no precedent that should

tell you this will work most new companies a model will tell you this is not GNA work yes you Know versus the closer you get to the public companies the more any Innovation is like okay this is kind of like this thing that happened um and I feel like these models are quite good at generalizing and thinking again going back to the partner and thought like thinking about second order yeah that's that's maybe where so a concrete example I think it certainly is the case that we tell retrospective to your point about Venture we tell

retrospective stories Where it's like well here here was the set of observable facts this was knowable at the time and these people made the right call and were able to cross-correlate all of these different sources and said this is the bet we're going to make I think that process of idea generation is absolutely automatable and the question of like do you ever get somebody who just sets the system running and it's making all of its own Decisions like that and it is truly like um doing thematic investing or of the like what a human analyst

would be um kind of on the hook for as opposed to like hftd uh but the ability of models to say here is a fact pattern that is noteworthy and we should we should pay more attention here because if you think about the The Matrix of like all possible relationships in the economy um it grows With the square of the number of facts evue like polinomial with polom with the number of facts you're evaluating and so if I um if I want to make bets on AI I think it's like what what are you know

ways to profit from the the rise of AI it is very straightforward to take a model and say parse through all of these documents and find second order derivative bets and say oh it turns out that energy is like very very adjacent to investment an AI and may may not be priced in the same way that uh gpus are and a derivative of energy for example is um long duration energy storage and so you need a bridge between Renewables which have fluctuating demands and the compute requirements of these data centers and I think that and

I'm I'm telling this story as like having witnessed bright wave do this work you can take a premise and say like what are second and third order bets that we can make On this topic and it's going to come back with here's a set of reasonable thesis and then I think a role in that world is to assess like does this make sense given our fun strategy does this is this coherent with the calls that I've had with the management teams there's this broad body of knowledge that I think humans um sort of are the

ultimate like synthesizers and deciders and like maybe I'm wrong maybe the world of the future Looks like an the AI that truly does everything that is a I think it is kind of a singularity like fact where it's like really hard to reason about like what that world looks like and I you asked me to speculate but I'm actually kind of hesitant to do so because it's just the forecast the the hurricane path just diverges far too much to have it you know real conviction about what that looks like um awesome I know we we

already taken up a lot of your time um But maybe one thing thing to to touch on before wrapping is open source LMS obviously you were at the Forefront of it uh we recorded our episode the day that red pajama was open for us and we were like oh man this is this is mind-blowing this is going to be crazy um and now we're going to have a open source uh dense Transformer model that is 400 billion parameters that I don't know if one year ago you could have told me that that was going to

happen uh so What what do you think matters in open source what do you think uh people should work on uh what are like things that people should keep in mind to evaluate okay is this model actually going to be good or is it just like cheating some benchmarks to look good it's like is there anything there like yeah the this is the part of the podcast where people already dropped off if they wanted to so they want to hear the hot thingss right now I mean I do think that That's another reason to have

your own private evaluation corpuses is so that you can objectively and out of sample uh measure the performance of these models um and again like sometimes that just looks like giving everybody on the team 250 annotations and saying we're just going to grind through this and like you you know you have to tell does this meet the other thing about like doing the work yourself is that you get to articulate your loss function precisely What is the thing that I what do I actually want this system to behave like do I prefer this system or

you know this model or this other model um yeah and I think the the work around over you know overfitting on the the test I think is like that 100% is happening um one notable in contrast to a year ago say and you know the the incentives the economic incentives for companies to train Their Own Foundation models I think are Diminishing so the like window in which you are the dominant pre-train and let's say that you spend 5 to $40 million you know for like a call it kind of a commodity is pre-train not I

know 400 billion would be another sort of it cost more than 40 million another Le but the kind of thing that you know like a you know small uh you Know multi-billion dollar Mom and Pop Shop might might be able to pull off um the benefit that you get from that is like I think diminishing over time and so I think fewer companies are going to make that Capital outlay um and I think that you know there's probably some some material negatives to that but the other piece is that we're seeing that at least in

the past two and a half three months there's a convergence towards like well these Models all behave fairly similarly and you know it's probably that the like training data on which they are pre-trained is substantially overlapping and so it's generalizing a model that generalizes to that training data and so it's um unclear to me that you have this sort of balkanization where there are many different models Each of which is good in its own unique way um Versus something like llama becomes like listen this is a fine standard to build off of um we'll see

it's just like the The Upfront cost is so high and I think for the people that have the money the the benefit of doing the pre-train is now um less the where I think it gets really interesting is how do you differentiate these into all of these different behavioral regimes and I think the cost of producing um instruction tuning and fine-tuning data That creates specific kinds of behaviors I think that's probably where the next generation of of really interesting work starts to happen and that's if you see that the same model architecture trained on much

more training data can exhibit um substantially improved performance it's the the value of modeling Innovations it it's you know for fundamental machine learning and AI research like there is still so much to Be done but I think that the much more like much lower hanging fruit I guess is developing new kinds of uh training data corpuses that elicit new behaviors from these models in a specific way and so that's where when I when I think about the the availability to Mo like a year ago you had to have access to uh fairly high performance gpus

that were hard to get in order to get the experience of multiple reps fine-tuning these models and what you're doing when you you know Take a corpus and then fine-tune the model and then see across many inference passes what is the qualitative character the output you're developing your own inter model how does compition trus shape the behavior of the model in a Quita way a year ago it was very expensive to get that experience and now you can just recompose multiple different training corpuses and see like well what do I what do I do if

I insert this set of Demonstrations or I ablate that set of demonstrations and you can that that I think is a very very valuable skill and one of the ways that you can have models and products that um other people don't have access to and so I I think as more people as that those sensibilities proliferate because more people have that experience you're going to see teams that um release data corpuses that just imbue the models with new behaviors that are especially Interesting or useful and I think that that may be where um some of

the the next sets of kind of innovation differentiation come from yeah yeah when people ask me I always tell them the the halflife of a model it's much shorter than a half life of a data set know like I mean pile the pile is still around in like core to most of these training runs versus all the models people trained a year ago it's like they're at the bottom the lmc's litter board it's kind of Crazy like I don't just the parallels to other kinds of computing technology where like the work involved in producing the

artifact is so significant and the like shelf life is like right a week it's like I you know I'm sure there's a precedent but it is it is remarkable mhm yeah I remember when dolly was the best open source model we dolly was never the best open source model but it would it Demonstrated a new something that was not obvious to many people at the time yeah but we always were clear that it was never state-ofthe-art state-ofthe-art whatever that means right um this is great Mike anything that we forgot to cover that you want to

add any um call I know you're you know thinking about throwing the team iring across the board um AI uh engineering classical machine learning systems engineering and distributed Systems front-end engineering uh design uh we we have many open roles on the team uh we hire exceptional people um we fit the job to the person as a philosophy um and would love to work with uh more incredible humans awesome thank you so much for coming on Mike thanks celesio [Music]