Intel unveils next-gen solutions with Xeon 6 processors and Gaudi 3 to tackle enterprise needs

[Music] [Applause] [Music] thank you for spending some time with us today this is an exciting moment for Intel and our Enterprise AI products and solutions before we get into these sessions I want to set context on our strategy not just what but why I've spent a lot of time with our customers examining our portfolio and taking a closer look at our road map and our overall strategy we've been making changes for long-term sustainable growth and it starts with Zeon as our host CPU and accelerated Computing for AI with gouty AI accelerators I'm excited about our

Collective Mission and future we're still in the early stages of the AI deployment and the data continues to say that enterprises will drive broader adoption and growth however the compute required has put a spotlight on availability performance cost Energy Efficiency and security these are key considerations for Enterprises today to address these we're enabling an open ecosystem just like we have in x86 and it extends from the hardware to the networking and interconnect technology and all the way up the software stack including through open Frameworks and models we're focused on delivering AI systems that provide viable

economics for deployment by driving efficiency with performance watt we're optimized for open-source models and we're enabling the Enterprise ecosystem to deploy AI applications faster leveraging an open- Source approach we're excited to talk more about our announcements with xon 6p cor and gudy 3 Intel Xeon processors are the ideal CPU head node for AI workloads and they operate seamlessly in a system with Intel gudy AI accelerator ERS which are purposely designed for Gen AI workloads together these two platforms offer a powerful solution that seamlessly integrates into existing infrastructure we're addressing many considerations top of mine for

Enterprises with Intel xon 6 with performance cores they offer significant performance leaps for compute intensive workloads like AI high performance performance Computing and database applications Zeon 6 delivers this performance with greater power efficiency and continues to deliver the state-of-the-art confidential Computing features included with Intel's trusted domain extensions Zeon 6 with peor is also the best CPU for AI head nodes with 96 cores pcie Gen 5 support and the ability to integrate the most advanced D technology in the market you can build massive gen training clusters with confidence our accelerator portfolio and investments in optimizations at

a systems level are critical to Broad Enterprise AI adoption the gouty architecture gives customers the Gen AI performance they seek with a price performance advantage that provides choice and fast deployment at a lower TCO the Intel gud 3 accelerator was optimized for llms and designed to scale up and scale out efficiently from a single node to thousands of nodes so it can support Enterprise needs whether they are training proprietary models or fine-tuning and serving the latest open-source models in today's evolving AI Market the capex required to invest in scaling Services can be substantial that is

why our focus is on building AI systems that reduce the cost per token and give you the flexibility whether you're deploying in a green field data center or an existing data center environment let me share a few examples of where customers see value in this approach first we're combining forces with IBM to lower the TCO of AI compute in the IBM cloud IBM has a rich history of working with Enterprises around the world and they're choosing gouty another example is that Google is bringing Zeon 6 to their Cloud they chose zeeon to help improve TCO

and performance for general computing workloads including for AI and enabling confidential Computing this is a benefit of the optimized performance and efficiency that we deliver with both Zeon and gouty because we start by taking a systems approach We Believe demand will shift from being dominated today largely by Frontier Model training and serving to one where the market is more balanced and there's a much greater focus on task-based model training and serving as well as open-source model serving and we believe this will happen largely driven by Enterprise AI adoption and we hear that from Enterprises that

they want this capability by leveraging standard silicon products in the road map and our open- source-based software stack we're making it easier for Enterprises to adopt new technology a great example is with AI agents or rag workloads which are becoming increasingly important as Enterprises look to deploy and scale AI models and to address the challenges associated with Enterprise rag deployment we're introducing Intel AI for Enterprise rag this includes easy to deploy fully validated use cases built on our open platform for Enterprise AI or Opa Opa is an open- Source initiative which we launched in April

and it now has over 40 industry participants we're also working with Dell to co-engineered gen AI Solutions powered by gouty 3 and xon and running in the Dell AI Factory to accelerate time to value for our Collective customers so when you're ready to get started you don't have to wait in fact with the Intel Tyber developer Cloud we provide an option for quickly ramping commercial deployment and we're already seeing companies like Seeker help customers gain immediate access to a turnkey platform running on the Intel Tyber developer Cloud where they can train and deploy their own

proprietary AI Solutions now I will hand it over to Ryan tabra to walk you through Intel xon 6 thank you Justin earlier this year we launched Zeon 6 with ecores and we're excited by the markus's reaction to our first ever efficient core product not only did we beat the program schedule that we committed to our customers but we enabled them to get Pilots up and running honor processors faster our customers are adopting xon 6 with ecores for workloads like web services networking storage and more feedback from customers has been positive the efficiency that our ecores

bring has allowed them to consolidate a significant portion of their existing data centers with better performance and at the same time lower power envelopes today we are excited to launch the next family member of the xon 6 platform Zeon 6 with performance cores or pees are uniquely architected to deliver significant performance leaps in critical growth spaces like AI HBC and databases we are lowering power consumption so our customers can scale within existing power constraints we enable this while simultaneously maintaining the same Hardware based security features that our customers expect from Intel like confidential Computing and

Trust Services to help keep data secure and the results are in we are delivering leadership on Intel Xeon 6 with peores versus previous generations specifically almost up to two times the performance per watt Advantage at a 40% server utilization on HPC workloads Zeon 6 takes advantage of more cores higher memory bandwidth and AVX 512 and we're seeing a 2.5x better performance for AI inferencing Zeon 6 shows up to 5.5x Performance increase using resinet 50 workload compared to the competition we are not only seeing success with Zeon for AI inferencing but also as the most deployed

host CPU for AI accelerated systems this is because xon 6 uniquely meets the requirements that our customers are asking for in their host CPU like Superior IO and high single thread performance and speaking of customers we're looking forward to our partners and ecosystem delivering xon 6 to the market and now to hear more about our xon 6 processors and gud 3 and our AI systems approach I'll hand it over to sarra thank you Ryan I would now like to talk about the work we are doing on Enterprise AI with our data center accelerator product portfolio

Justin talked about our overall strategy earlier and following up on that I will also give an overview of our systems and solution strategy and plans with ci3 we look at the market through the lens of four key segments categorized by specific use cases and model types namely Frontier models task based or domain specific models open source models for Enterprise models and HBC and superc compute infrastructure stamps for AI today are dominated by training needs but over time we anticipate inference will dominate as companies start to derive value out of and monetize AI laying our sight

on this trend we are focusing on inference and fine-tuning initially Taking A System's first approach our systems for strategy is anchored on the tenets of working with our ecosystem partners and leveraging open industry standards to that effect we are publishing a reference design for our Gaudi 3 scalable system with 32 nodes or 256 Gaudi 3 accelerators making it easier for more customers to deploy Gaudi Bas systems this design is modular and can be scaled out to up to 8,000 Gaudi 3s with the same scalable building block and ethernet based scaleout fabric we are already Building

Systems with this configuration to support our qualification and workload benchmarking efforts as well as for customer access in the future on our developer Cloud we have a rich history of pioneering open platforms and reference architectures with Zeon we are bringing that same rigor and experiences to accelerator based systems starting with Gaudi 3 reference architectures now let's shift our attention to Enterprise AI Solutions conversations about hardware and infrastructure are great but when it comes to Enterprise AI our customers care about Solutions and endtoend experience to extract true insights from llms retrieval augmented generation or rag is

a technique that is getting rapid adoption in the Enterprise rag enables Enterprises to customize llms on their proprietary data without having the need to retrain or fine-tune the llm all the time rag also minimizes the adverse effects associated with llms such as hallucinations and enables organizations to truly customize and launch generative AI applications much more reliy and coste effectively Vector databases play a crucial role in a rack Pipeline and today the best destination to store these databases on and run queries on them is a Zeon subsystem gudi 3 on the other hand offers great performance

per TCO for the llm itself Intel is leveraging the goodness of Zeon and Gaudi 3 on the same platform to enable the entire rack pipeline focused on scalability ease of use improved TCO security and an open ecosystem based approach today I'm excited to announce our Intel AI for Enterprise rag offering with this offering we will deliver endtoend solutions for use cases such as chat Q&A code generation content summarization and many many more to come these endtoend Solutions will all be based on our open platform for Enterprise AI or Opa so Enterprises can explore experiment and

easily deploy rag based applications Intel AI for Enterprise rag will be fully qualified on Zeon and Gaudi 3 systems in collaboration with our OEM Partners finally I'd like to provide some details on the Gaudi 3 accelerator which we are launching today following the tradition of the Gaudi family of products from Intel Gaudi 3 is a purpose-built accelerator for AI it will offer competitive price performance and power efficiency with up to 20% better performance than nvidia's h100 GPU on models such as the Llama 270 billion parameter model for inferencing on price performance the benefits are even

greater compared to the h100 over time we expect performance to improve further with continued software enhancements gudi 3 will be available in both the oam form factor on a ubb with eight accelerators connected in an all to all topology and also the PCI cem form factor with the Gaudi 3 PCI form factor we are bringing more efficiency to gen with low power and cost and ideal for inferencing fine-tuning and small model training customers can start small by dropping in four cards with relative ease and then easily scale their compute based on their needs besides silicon

and systems a performant and easy to use software stack is critically important for customers to harness the true power of AI infrastructure the Gaudi software stack offers support for leading AI Frameworks such as pytorch and provides libraries and tools that helps developers to easily Port models over to Gaudi Hardware we are taking a full stack approach to software with a keen emphasis on components such as cluster management job orchestration Telemetry data management microservices based Solutions Etc besides the core software stack for AI with Gaudi we're addressing Choice with accessible platforms and affordable performance per dollar

with an open ecosystem software gudi 3 systems will be available in the market from our OEM Partners Dell HP and super micro additionally select customers will gain early access to Intel Gaudi 3 for validating AI model deployments in the developer cloud with Gaudi 3 clusters set to roll out next quarter for large scale production deployments speaking of customers IBM has decided to adopt Gaudi 3 in IBM Cloud underscoring its performance perw advantages and cost efficiency especially for Enterprise AI applications I will now hand it over to Anil nanduri who is hosting a round table where

you can hear more about that and our other customers which are leveraging Gaudi 3 for their AI needs so thanks good to have Rohit and VI back on um you know our customers as well as extremely wonderful partners for very many years um thank you for joining us um of course and um you know as we know AI is evolving very rapidly right and uh you know we can talk about kind of building the largest llm and you know all the CRA behind the models but you know I work closely with a lot of the

customers Enterprise customers who are actually trying to consume some of this AI um and there's a lot of domain knowledge and domain data uh sitting behind the Enterprise uh uh firewalls um and for them they're looking at primarily how to deploy you know AI um at scale U have flexibility um you know have the uh Choice uh that they want to build into their infrastructure uh but looking at value of the AI deployments and especially around how to apply AI into their uh work streams um and so some of the questions we would like to

kind of bring that with your experience and and you work with a lot of end customers and Enterprises um and wanted to have this uh panel focus on some of those topics uh on the challenges the customers are facing and and how uh we working so Rohit if I could start with you um you know Intel and and IBM have been co-engineering to deploy a AI Solutions um and um you know we recently announced um U some AI tools for Enterprise customers what do you see as the top challenges that your customers are looking for

uh in the market and how is Intel the right partner to help address this and work uh with IBM on this great so uh let me maybe back up and give a little bit of context on where we're focused as a provider and then I'll get into um you know the Intel partnership so as a cloud provider we've been very focused on being kind of the Enterprise scale cloud provider right we're not trying to go after compete with hyperscalers we're very very focused in the segment of clients that IBM calls hybrid cloud and AI um

and so we've really engineered over the last several years significant Enterprise capabilities around kind of the five aspects that I believe clients need resiliency performance uh security compliance and total cost of ownership those are the characteristics I hear over and over as I look at clients adopting hybrid Cloud Solutions but also AI so now when you look at the adoption of AI we believe these two I would say amazing Technologies fuel each other right we're seeing many use cases where clients want to adopt AI on the cloud but there also many in use cases on

Prem the the aspects that fuel that different that decision making is things like sovereignity data Protection security you know how you're dealing with their data and so we're really making sure that across that Spectrum IBM as a company is helping clients on their Journey on Enterprise AI use cases that's why we're focused our our gen platform is called whatson X we've got a bunch of assistants around that now where Intel comes in is of course we on our Cloud we have other vendors Tech vendors as well in our early testing with Intel uh we of

course do a lot with Intel on the Zeon side um in our Cloud a core of our cloud is Intel across our mzr Fleet across the world but specifically on gudi 3 what caught our eye was um in our early testing with our research team we found significant cost performance benefit right and so we've been you know co-engineering in the early phase and that led to kind of the announcement we'll be rolling out some of this in our mzr fleets and going after joint Enterprise clients that's uh wonderful and uh thank you and um maybe

let's move to V from Dell you know we've had what 30 years of collaboration a long history and Partnership of doing um you know many great things together um and you know can you share a little more about you know Dell's uh approach to AI um and uh how does your approach helped the customers to start on their AI Journey uh because you work with a lot of Enterprise and um you know end customers and the kind of innovations that Intel and D are working together with sure absolutely um we've been I think as I

mentioned earlier this morning we first and foremost think our about ourselves as customers zero so we are manically focused on how do we take advantage of AI for ourselves and then of course we've got this tremendous reach across the whole world so we have tens of thousands of direct sellers lots of Partners which gives us unique insights into what Enterprise want Enterprises want to do so with those two lenses um what we realized is that what Enterprise are really looking for is a way to easily deploy in scale Ai and to extract the benefits and

the advantages which are around TCO and Innovation and and other business priorities um as well as have a choice and that's really what went into our thinking our launch of what we call AI Factory which we announced at Deltech World in May and when you think about AI Factory it has multiple levels to it it has it clearly starts with the use cases because that has to be the Anchor Point it have data on the other side because you have to bring data to take benefit take advantage of the use cases but then what we

have in the AI Factory are three key components the first is the infrastructure layer the second are the solutions and the third of the services it's really services is around day 01 and two services to help our customers get started and deploy Solutions are our Dell validated designs and D reference designs we have well over 50 of these designs to help customers deploy them and infrastructure is really where we work really really closely with I with Intel around servers uh storage networking software on servers particularly a flagship 9680 which we'll be launching with got3 and

and Zeon uh which we are really excited about it gives customers a choice as well and then on software as I mentioned this morning with our Omni F stack being interoperable with your uh Opa that that provides for an immense ability for the customer to adopt and and scale it relatively easily so you're building TurnKey easy button for your Enterprise customers to be able to deploy really quick absolutely because it has to be for when you think about Enterprises they do need that kind of an easy button to be able to deploy it but have

confidence it's actually going to result in an Roi for them um with that I think one of the challenges we see in the ecosystem is uh privacy and security um and uh especially the rice of gen right and and this is becoming very Paramount right and so Rohit more from an IBM perspective right um how do you see uh what is IBM's you know work and the strategy uh to actually you know um ensure that data is protected uh for the clients um as we bring in these new technologies for scaling AI uh how are

you approaching it at IBM and then how are we working together on this yeah this great question so um you know I I talked about like hybrid Cloud fueling Ai and AI runs on hybrid Cloud I mean we've been on this Mission from an IBM Cloud perspective to really build out an Enterprise Cloud platform and underpinning that are those characteristics that I talked about but we've we've made sure that we've kind of co-created that with end clients in certain regulated Industries so we've got 100 plus Financial Services we've got several Healthcare government agencies that have

helped us build this Cloud platform for their needs that serve the industry but also what they have to serve from a regulator perspective and that's the kind of Innovations that's now also fueling AI so for instance we run on our Cloud the highest levels of encryption that you could find in the IND industry we've codified into the platform compliance controls that are needed and provide the auditability that are needed for any regul regulated client to prove to their Regulators right so when you now apply an AI workload on that it inherits those characteristics because AI

runs on data you inherit those characteristics and we've also worked with clients and Regulators to to come out with profiles around guard rails around AI that give you that level of visibility at the model layer so we've got capabilities like what's the next start governance as well as at that infrastructure layer that you can protect the data make sure it's you know secure compliant Etc uh that's great and I also understand you know you have the ability to kind of uh uh you know have the responsible AI part of the stack so you can give

some Assurance to customers abolutely on uh the uh the the quality of the responses and the AI outputs that you exactly absolutely Ely okay um so Weck um you've been at Dell for a long time and you've moved around and uh you know and come back to Dell um and you know you have lot of experiences uh um you know you know what have you learned um from your experiences at Dell uh and the other places where you worked at that actually would help customers kind of share a little more insights into your personal experiences

well as you mentioned I I am a boomeranger at Dell I have kind of traveled uh in different spheres and come back I took this role on to lead corporate strategy about 15 months ago um we had Dell again with the customer zero mindset was it was immensely clarifying for us because we very purposely started thinking about how do we want to apply Ai and how do we want to think about it so the first thing we did is we uh categorized we created a simple strategic framework to think about AI in on forign with

AI in is in our products AI a for is a for ourselves how are we applying AI for ourselves AI on is what the customers do on our products and AI with is what do we do with the other ecosystem Partners to bring it to our customers with that Clarity the next thing we did is we thought about how do we want to deploy Ai and how does it support our business strategy and that anchored ourselves into the four key parts of our unique operating model which we believe really allows us to win the marketplace

ranges from end to end product portfolio where we are number uh one in over 20 different categories stradle Services server storage software PCS the entire breadth second of the largest go to market engine in the industry with lots of tens of thousands of direct Sellers and lots of uh Channel Partners third are our Global Services presence which is across 170 plus countries 2200 service centers tens of thousand of direct team members and of course our industry leading supply chain all anchored by our culture so we said how do we use AI to accentuate and enhance

our business strategy which then led us to four um uh use cases we focused on software development for our products content management for our sales teams and more broadly sales tools and then our customer services as well as our supply chain so as we thought about it and then we thought about what architecture we need to bring in what data platform we need to have we set up a chief AI officer role which we were one of the first in the industry to designate that we set up data governance around and now we're bringing all

of those learnings to our customers and guess what the first two custom first two questions the customers typically ask us are where do I start how do I pick all these use cases and amongst them sift them out and number two what do I do with our data which is really where our Professional Services organization comes in again I think thinking of our sales of customer zero has been immensely clarifying and we're bringing those learnings to our customers now actually that's great and we use our Intel internal manufacturing and our it as our customers Z

so I think there's a lot of learnings there um you know as we look at our spectrum of our joint customers that we have you know there are customers who are deep Tech uh who understand the architectures you know can program and then we have customers who have data scientists but don't have the you know knowledge of the you know uh um the AI Frameworks and uh and deployment of different stacks and then we have other customers who can't afford some of these right and so um as we look into the you know different permutations

and and capabilities of AI Stacks what customers want is an outcome hey I want to do you know generate a chatbot or I want to do kind of a you know query or do I want to do code translations or be it you know uh some kind of uh agentic and functional you know trends that we have in AI is all about applic applying AI right and so rohab back to you you know at IBM uh you have whatson X and I know you spoke a little bit about it uh but uh you know uh

how do you see the competitive advantages and benefits that Enterprise customers can have um when you leverage the Watson X and something like a Gaudi yeah look we've been you know we've been at this wats Watson Mission and AI for a long time but Watson X is very focused on you know building out a strong geni platform it has the breadth and the portfolio and assistance and like V mentioned right in in IBM we really have this client zero culture so it's being applied to our own productivity to our own kind of internal digital labor

transformation our own code generation and uh you know the efficiency of our developers so you know it's a very strong platform and what and from a technology pers perspective it's being built jointly with our Cloud it is also offered as multic Cloud right but it's be a fine-tuning that uh from a security performance the characteristics I talked about through our Cloud we also are blessed with a super strong Consulting team right that go to market on AI right so we're not going to clients saying here's another AI platform we're going with very specific use cases

right digital labor code efficiency modernization Etc right and there are eight or nine categories of use cases that we've co-created with clients that are built and you could quickly see this is a full stack all the way from infrastructure and yes infrastructure is trendy to um to the platform to the assistant and then the go to market through kind of use cases and that's how we go to market now where gudi fits in and uh you know is the early signs for for me are super appealing is that as clients starting to are starting to

use these in production use cases specifically the area we've been focused on is inferencing cost um you know we see some advantages in our early testing and which is why we're keen on uh you know deploying this in some of our mcrs but more importantly we got to continue co-invention co-creating with clients of course we'll work together and we work very closely with Dell as well uh but really the next you know 12 months this is co-creating with clients to continue to push the envelope when it comes to cost performance specifically for in in closing

there are three things I want to take away from today number one we are creating high value AI systems with TCO advantages number two we're driving efficiency with performance per watt optimized for the latest open- Source models and Frameworks and number three we're empowering the Enterprise ecosystem to deploy AI applications faster using open- Source tools thank you again for being with us today [Music]