YouTube Deep Summary

even if you're a developer you might still be thinking about AI wrong and the reason is it's not just chat GPT it's not just agents and it's not just your cursor code editor that's helping you write code better in fact it's an entire platform with multiple levels of the stack that you can develop on like other let's say platforms like iOS web having a solid development environment to take advantage of this coming opportunity it's super important and that's the reason why I picked up an RTX laptop personally I got the Asus Rog zephrus 470 which is not just good for gaming it has a lot of advantages when you're running local llms and trying to stay ahead of the curve on this stuff and be well positioned for the coming months and years so we're going to talk about the AI stack why would you even want to run llms locally and also some kind of fun quality of life stuff that you can do with this laptop because I've personally been a Mac User for 10 years and this is finally like a valid reason for me to upgrade and even Switch to Windows I'll say now this video is sponsored by Nvidia who recently had some amazing jaw-dropping announcements in the CES conference the new gpus coming out super interesting small computers that can run insane models and I'm super happy they got in touch because I'm now working full-time on various AI apps and I really think it's still so early so let's dive into this so running models locally seems like a pain why would you want to do it well it's actually super easy you just go to ama.com you can download the Open Source model of your choice pre-trained and then you can run it locally on your computer but trying to run this on a Mac you're going to encounter one issue and you need to be able to fully load the model by its size so let's say it's 8 GB you need at least 8 gigs of vram in your GPU to run that model at least at a speed that is actually viable useful and comparable with apis Beyond just vram there's a reason Nvidia is leading the industry they're the standard for all large AI companies it's that their architectures are also highly optimized you don't have to take it for me you can go to AMA right now download one of the models for free it's going to be slow unless I can load it fully in my GPU the way you check that is you get your model you start running it and then you run the command o Lama pce you'll see a utilization breakdown CPU versus GPU if even 10 20% is being loaded into your CPU because you don't have the parallel capabilities of the GPU your model is going to run exponentially slower like 3 to five to even 10 times depending how big it is now of course you could also go to let's say a API provider so you can see this test Iran running on a pretty fast Network and the response time being around 2x higher so two times yeah it makes a huge difference if you're developing if you're running long complex AI flows but the more important consideration or bottleneck with comparing this to an AP is the cost aspect open ai's pricing is based on tokens and when you're running an agent or you're running a complex task you're usually feeding the entire history into each context window when you do this each call is going to eat up a pretty substantial amount of tokens especially if you're using the newer models like A1 or let's say you're building an agent each agent might have 10 50 200 llm calls to complete a task so let's come back to AI being a new platform and there's kind of three levels in my mind I see it broken down into the highest level is just llm calls and orchestrating them into complex workflows and tasks like agents which why combinator has said can actually be 10 times bigger than the whole SAS industry whether or not you believe this they are going to be part of the future and when it comes to building agents being able to do this locally even if just for development purposes is hugely advantageous coding your own orchestrator and agent flow on your local computer it's a great project even if you're just trying to get hired in the coming years and I personally coded an agent for this video to find anyone's LinkedIn profile with a broad query my agent will do web scraping it will analyze LinkedIn profiles and it will crawl the web for me completely for free so I think this is a really good AI project starting out but this is still the highest level of this stack you go one level down things get even more interesting with nvidia's a AI workbench optimized specifically for their chips you have an entire Suite of tools to play with the most interesting part for me at least is the ability to fine-tune models in other words you can take a smallish model like llama 3.2 and it can become almost as performant as a larger model because you've made it specialized fine tuning is critical because when you consider the AI space so many people are just using generic larger models that are not customized you probably know what fine tuning is but when you can do this locally it really makes those smaller models quite formidable and then you combine that with the other benefits on the lowest level you have actual Cuda programming you can run low-level code on your GPU hardware and the mind-blowing thing is the parallelization capabilities are insane when you think about new AI Graphics procedurally generated games this is where your Cuda programming is going to be really interesting going into the future it really good simple example if you're struggling to understand why it's useful is something like FFM Peg most experienced programmers know this is for video processing modifications and basically manipulating video files and there's a lot of operations required to modify the individual frames but using Cuda using the GPU you can parallelize this huge set of tasks and your performance can go up in terms of speed so this is low-level programming and if you can wrap your head around it for that reason there's going to be huge opportunities here and if you can position yourself now to learn Cuda well it's going to be insane so how about the fun stuff things that you know everyone can take advantage of whether you're a developer or not things I've enjoyed personally the first one is with games obviously games run really well on a 4070 but you also have with Nvidia frame generation your GPU is able to fill in gaps with AI in real time and you get a better frame rate than what your native Hardware is actually capable of as the frames are being rendered to you it seems more smooth and this is done dynamically and Nvidia also has upscaling even for YouTube so if you're watching a YouTube video at 480p 1080P and this works on every browser the GPU will actually be able to increase the quality of that video Beyond even its compressed size and improves the image in the same way that upscaling models do as well but this is done in real time and it's really mindblowing so I don't know if it's me but I think the sooner you can dive into this stuff and actually just Embrace okay we have this new platform these new tools these new types of software these laptops for like actually having a developer environment it's kind of what you need so that's the reason I switched let me know what you think of this video and shout out to Nvidia for sponsoring and hope to see you guys in the next one we'll do an agent workflow on the laptop very soon so I'll see you guys in the next one