{ "episodeId": "ep2", "episodeTitle": "The Great Hardware Winter: When AI Ate the World's RAM", "speakers": [ "Vikas", "Sachin & Kunal" ], "speakerMap": { "speaker-0": "Vikas", "speaker-1": "Sachin & Kunal" }, "duration": 2115, "durationFormatted": "35:15", "wordCount": 5993, "segmentCount": 49, "segments": [ { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 52, "timestampFormatted": "0:52", "text": "Hi everyone, welcome to the AI Valley podcast. I know it's been a while and we've missed you, but I hope you had a good set of holidays. Just in case, belated Christmas and a great New Year. I hope all of you guys have a great New Year going forward and we wish you all the best." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 69, "timestampFormatted": "1:09", "text": "Hi, and this is Vikas and wish you a great new year. We have had a amazing time traveling with family, friends, work and lot of things have been busy, not just work, but also the way things are shaping in both AI and in general. And today we have a very special episode because we have with us a very special guest between us, Kunal. Kunal is not just a dear friend, but he's One, I've seen lot of people who are passionate about hardware and accumulating vintage computers and continuously building capacity for their personal usage. And he's someone who's been not just passionate about computer hardware, post AI, but he's been all around. So given the fiasco that's happening in the world of the DRAMs and the AI build out. We thought there has been everywhere we go, we hear people talk about. So we thought to bring a very special episode about hardware winter and everything that's going crazy in the world of hardware. So welcome Kunal, how are you?" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 135, "timestampFormatted": "2:15", "text": "Well, Vikas and Sachin, thank you so much for inviting me here. Really appreciate it. Actually, know what, folks, I have a very interesting story to share. a couple of months back, think maybe slightly more than a couple, think three or four months back, I was trying to build my own PC and we'll come to this. just a PC, but a workstation. Yeah, a proper workstation. And I guess all thanks to the man here because A lot of the components that are in that workstation today were given by him, suggested by him. In fact, the nice little 4090 that sits in and does all the work for me, that's also gifted. I mean, not gifted, but he gave it to me. But I think, know, and here's why I thought, you know, we should have this discussion today, right? Because obviously with the workstation, I was looking at RAM, we went ahead, you know, did a purchase and It was such a funny story because we went ahead and purchased these four sticks and the user I mean the seller gave me a good rate which we thought is a good rate All of a sudden then we decided you know what we don't need such a beefy station and we wanted to return that back But the seller was not ready to do it and by the way, this was 64 GB DDR5 ECC memory. Yes. Yeah, four four sticks of it. It's you can imagine and We wanted to return it and I'm so thankful, so thankful that the person didn't take it back because now this is definitely the discussion of today. Look at what we did four months back and the world has changed in the next four months. Do you remember the cost at which you bought each stick? I mean, yeah, 90k for all the four. All the four, right? Right now, if you to" }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 232, "timestampFormatted": "3:52", "text": "100k, 90k is 1k." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 232, "timestampFormatted": "3:52", "text": "Yeah and right now if you look at it a single stick of RAM is almost $300 64 GB" }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 243, "timestampFormatted": "4:03", "text": "Yeah. So I think the" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 244, "timestampFormatted": "4:04", "text": "Sorry, not 300 dollars. 1500 dollars. Is a single stick of 64 GB RAM." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 250, "timestampFormatted": "4:10", "text": "So Kunalji, think the biggest AI event, I think last year was when Sam Altman walked into Seoul and walked out with 40 % of the world's DRAM supply. So in many words, he was playing a 4D chess to block everyone else from hyperscaling. the story goes is OpenAI signed agreements with Samsung and ASK INIX for up to 900,000 DRAM wafers per month, right? Because it seems like he, neither company both SK Hynix and Samsung knew that what the deal other way, you know, signing and Samsung assumed that they were giving OpenAI a meaningful allocation. So did SK Hynix and by the time both realized they collectively handed over nearly half of global DRAM output, right? And so to your point, the 32 GB DDR5 that sold in $95 in September is $400 in December, right? That's about four times the price and Obviously gamers and PC builders and general consumers are hating it. But I believe, I think we've had lot of discussions around this. The implications go way beyond this, right? Because this hardware winter is going to last for maybe a year, two years, four years. Who don't? Nobody knows, right? So what's your thought on how this will play out in everywhere there is memory, which is, you know, your PCs, your... cars, know, handhelds, consoles, right? How is the world of computer hardware is gonna look like?" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 336, "timestampFormatted": "5:36", "text": "It is going to be quite difficult because as you know, memory is everywhere, not just computers but also in smart TVs, watches, and even smaller things like say a microcontroller. So memory is quite a fundamental building block, which essentially the impact will be lot more universal. And there is not even what you call a deadline on when this will be over. So nobody is talking about standing up new fabs. The manufacturers of these companies, there only three companies, SK Hynix, Samsung and Micron. They are all projecting that things will be like this till 2028. And you wouldn't expect a lot from these manufacturers anyway. There have been many cases in which these companies have colluded to price fix. RAM itself and now they have such an amazing opportunity that if they would have wanted, they could stand up new fabs, but there is no intention on their part and they all are shutting down consumer businesses. So for example, you would have seen Crucial, which is the consumer brand of Micron, being shut down because they want to focus on enterprise market and as you know, enterprise market typically is higher margin market. And because of all of that, what they're expecting the trend is, hey, why do you need to own your hardware? You can just rent it. And this impacts a lot of hobbyist or anybody who wants to build a system. Because if you look at the prices of RAM, Vikas, you remember, right? You build out the entire system for, I think, like 50,000. Now, RAM alone will be like one like 30,000 in that price component." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 428, "timestampFormatted": "7:08", "text": "Absolutely, think for a price of a RAM stick now, know, entire PC would have been built in six months. That's very clear. See, what's most surprising to me, right? And when I went through the details, right, there is a sort of a second school of thought about artificial scarcity, because if you look into the details of the deal that Sama had done with SK Hynex and Samsung is these are to buy unfinished memory modules, right? To people who want to understand what unfinished memory model. So, which basically means that these unfinished memory chips are not allocated to a specific DRAM standard, right? So, it's unclear to me how they will finish them into RAM stick into HBM. You know, right now it seems like these wafers will just be stockpiled into warehouses, right? I don't understand." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 478, "timestampFormatted": "7:58", "text": "There could be right so typically the same DRAM can be made into an HBM can be made into DDR can be made into GDDR which is the graphics memory. So see there could be price collusion but I wouldn't say this is artificial scarcity because if you are a supplier and somebody already gives you a contract that this is the DRAM I want then you will block it right and if you don't have that then you can't allocate it and the thing is Sam has purchased or because these are enterprise deals right HBM or any enterprise the margins will be lot higher. So you would essentially do that instead of provisioning say RAM sticks particularly these consumer RAM sticks they will not look at that. So the fundamental issue is this all of these RAM types ultimately require the same manufacturer the bottleneck is essentially there the materials are same." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 528, "timestampFormatted": "8:48", "text": "Okay, so now tell me as a hobby, PC builder and how does this impact you personally, right? Because you've been buying hardware at a very sustained rate at different price points, right? Now, are you still buying? Are you not buying? What advice do you have for PC builders? Should they wait this out? Should we buy stabilizers?" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 549, "timestampFormatted": "9:09", "text": "No, wouldn't. Stabilizer, sorry. So I wouldn't recommend waiting this out at all. Let's break it, break this problem, right? If you have a need, right? You want this to actually finish your job or complete something meaningful, then you should go acquire that RAM no matter the cost, because it is only going to increase. It is as simple as that. The second thing is it is just nice to have. It is not critical, but you are probably using it for gaming or some other purpose. Then you have to look, you make do with the hardware that you have? If the answer is yes, then use it. And one of the side effects that you will see of this winter, because it's a long winter, software which we'll talk about later, right? That will also be optimized to run in smaller amount of RAM memory itself. So my recommendation is If you need it, you need to get it now." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 599, "timestampFormatted": "9:59", "text": "And I was looking at one another list of obviously RAM is burning but what is almost burning is also SSDs and small pre-built PC component and Radeon GPUs right because as you're aware AMD doesn't bundle RAM in their bomb kits to like the Nvidia does right so the RTX 9070 GRE 16 GB you know lot of these products have been canceled because of the lack of RAM, right? And similarly, Xbox, we're not hearing any news and" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 636, "timestampFormatted": "10:36", "text": "Yeah, next version of all these lower margin GPUs, I think 5060s and 5070s, they are cutting production, they are not bundling the VRAM with it. And they are looking to sell more and more of these higher cost professional workstation series of GPUs because they can make margin." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 652, "timestampFormatted": "10:52", "text": "think that if we have to plan out a PC build out, even though RAM is burning, we should go ahead and do so because things are going to..." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 660, "timestampFormatted": "11:00", "text": "Look at second hand market, if you can see some deal, I still see on ebay.com or on various classified sites, whether it is OLX or whether it is a bunch of other sites that you see, you will see older generation, the Xeon workstations or AMD workstation with DDR4 memory can still be have at like relatively lower cost. If you don't have that see the point is this it is not going to the price is not going to go down in like next year or so it is going to continue for a long." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 696, "timestampFormatted": "11:36", "text": "Okay, I have one question like say I have a bunch of RAMs, know, or years just like you we have collected lot of hardware. So should we book profit and sell out? Is it a good time to sell old laptops? You know with those DDR4s and files?" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 709, "timestampFormatted": "11:49", "text": "It depends on what profession you are in. So if there is no need and you had like lot of ramp sitting in attic and you want to maximize profit, then you can probably wait it out even further because like we said, Ramp prices are not coming down yet. But obviously like stock market is difficult to time these things. So but yes, they are pretty high now, but there is no way of saying that it will not increase further. So, Kunal, see, generally what we have noticed, this has been cyclical in a way, right, this has happened in the past as well. But do you think this cycle is any different? Do you feel this will play out a lot more longer or do you feel there is genuine reasons why companies are doing what they're doing? This cycle is quite different than what had happened previously. So, for example, if you remember when the hard drive prices went up, It was because of the flood in Thailand. And then these manufacturers essentially hard on that and the prices were up for a long time. The DRAM prices also went up but because mostly due to collusion." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 770, "timestampFormatted": "12:50", "text": "Also in covid I think when there was a supply chain" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 774, "timestampFormatted": "12:54", "text": "This is supply is like too much demand and also the supply chain is not moving as efficiently especially China being closed but this time the math is like fairly straightforward and simple and the way it is looked like is folks are building data centers and they need more and one way to look at it is If you we spoke about these classified websites like Craigslist or eBay. You do not see. Ampere generation of GPUs in those market marketplace as you will not find a lot of A100s. You know when A100 came like four or five, six years ago, those are still being utilized. On top of that, Jensen Wang keeps releasing new GPUs. you would have seen in the recent keynote at CES he has announced Vera Rubin and even though Rubin and before that you had Blackwell, Hopper they were improving massively so for example Blackwell was 10x of Hopper Rubin is 10x of Blackwell but do you see any of those GPUs Jensen would say in the keynote that people couldn't give it away even if they would have wanted. And this is essentially the proof that market will eat up any amount of GPU that you give them. And that's why it's good to be invested in Nvidia. This is not we telling it. Right. So my point essentially is that this demand is there. There is RAM, the GPU. these newer graphics cards, they started with 80 GB on the enterprise side, now then went to 150, 180 and now these newer chips are even more, right? So I don't see it slowing at all unless AI doesn't work out, which is not the case." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 878, "timestampFormatted": "14:38", "text": "point I think and Sachin and I we were actually discussing this earlier today and this is a question for Sachin you know like just let's let's play that scenario do you think this shortage is going to lead to you know a different paradigm shift in software where you know the memory hog browsers application who have been very liberal in their RAM usage are going to take a new look where there are new compression algorithms which offload their sort of RAM usage onto the disk or handle their memory usage maybe you know Chrome Lite or something like that where every browser or every application becomes extremely cautious and judicious about their memory usage right it can drive a lot more new creative do you think We are at a point that people can use these AI coding and use the next wave of these improvements to build these memory efficient softwares." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 935, "timestampFormatted": "15:35", "text": "I mean, yeah, absolutely. think that's an interesting point you've raised. think, look, every challenge is an opportunity, right? So I think where we see a challenge, there could be someone else who's looking at it as an opportunity. And they may try to build something that circumvents this problem. In fact, was actually quite coincidentally was the question that I wanted to ask Kunala as well, is that what happens to developing countries, right? Because while we look a lot towards the West and we see that countries over there will be able to afford some of the outcomes of high-priced memory, there are also countries on the other side where it may not be so affordable, right? Because memory is in every device that we use. So I think this is going to play out on the consumer durable segment as well. People may have to buy more expensive hardware for some time, more expensive durables for some time. But I think there will be innovation that will happen in the software side. what do you think about that? How will this play out for countries that are now just beginning to afford some of the most expensive durables?" }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 999, "timestampFormatted": "16:39", "text": "for another year before I update my iPhone or phones" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1003, "timestampFormatted": "16:43", "text": "So India in a way has leapfrogged US or other developed countries in some adoption factor. So for example, gaming is much bigger on phones here compared to other countries. We have We have prices impacting these computers and servers more. Phones hasn't happened yet. It will take time. I'll say general public may or may not see it quite as we do because we typically are lot more invested. The Western folks will also see because they generally participate with lot more devices like this. Software will definitely get updated to reflect this change. As you would have seen like an NES game in like 1988, you could put an entire game of Mario with like 11 levels in just think a few kb's and I'm not saying it was 3D and all but my point is you lay out a limitation and you can essentially live in it. There is some sort of laziness right now in today's software that you will see everywhere. These websites, have these powerful JavaScript compilers, right? Even though you are just looking for some interactivity away to know information when you want. Those things are just fancy things that may or may not require happen. You know, we may go back to native software a lot more because they are lot more efficient, energy efficient software. And even if you look at, even before this RAM thing happened, all the religious that happened, they always look for something called performance per watt. Right? Performance per watt was like essentially on the energy part, right? Now people can look at things like performance per memory. Like for this amount of memory, what do you do? There are a lot of algorithms. You remember, right? A lot of people hate answering. these lead code type of questions in the interview, right? Space complexity like, hey, do you store O, N or O, N square? People care about those things, right? So the all this negativity, the good thing that comes out of it is people will care about memory uses of their software again." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1123, "timestampFormatted": "18:43", "text": "And also I believe it will reflect in the choice of backend frameworks as well. I just, you know, reading there are languages like web assembly, which offload the compute from server to browser and stuff like that. So there'll be a lot more distributed paradigm where, know, since I can't maybe horizontally scale on expensive because it's not a commodity compute anymore. probably I'm going to serve your request and offload as much compute on these devices. In fact, you know, one of the thing that Kunal has been doing after this DRAM prices become DRAM and NANDs have become the new oil is he's started to sort of give put his focus on acquiring these small form factor devices. And I think one of the interesting takes that he has is, know, as model becomes smaller and smaller parameter model, even a 7B models are getting a lot more. It would be possible to run some of those things on an edge device, right? And some of these, and given the RAM crisis, they would like a nimble GPD box might be a server grade in a hardware winter, right? So I think these things are definitely going to show up in, you know, not just in the play. Kunal, you mentioned gaming. think what worries me most is the quietness that Valve have or because with a lot of fanfare, I think about two months back before all of these blew out, right? You know, they announced their steam machine, right? Obviously, it is unclear and unknown whether they bought pre-bought the RAM before announcing their much hyped steam machine. or if they already have a stockpile of ample supply of DDR5. If not, then I'm sure the steam machine launch should be affected as well because like every gamer, believe I'm very actively waiting for that device. But if the supply could dry up temporarily at some point, they'll wait for the prices to drop." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1243, "timestampFormatted": "20:43", "text": "But man, I still think it's a good thing because Steam is more than a box, right? So Steam's most of the thing that they have done is on the software side. the first time in the history, they have built out a platform that stands tall in front of windows and can be a viable gaming platform today. So there is a distribution of Steam called Bazite. and it runs Windows game because of all the work that Valve has done in terms of optimizing their stack, providing compatibility layer for DirectX games to run on Linux. Now what that gets you in this world of this hardware winter where RAM is not available because Windows is lot more resource heavy. There is lot of processes that Windows itself will consume. And time and time again, many reviewers have proved that the games written for Windows, not written for Linux, work better on Linux using Steam's layer that they have built, than it will be on a native platform. This was unheard of, like you would achieve this kind of performance. So my point being is, if you have a system which is resource constraint, soon, It is much better to install Linux and use something like Bazite or Steam to run games than this. Of course, Box, we do not know what has happened there, but you would have seen the pricing that they were attempting, right? So I think it's a good thing that Steam exists and it will be more popular in this hardware constrained world." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1329, "timestampFormatted": "22:09", "text": "Sure, absolutely. What else, Hashim? What's keeping you excited, I think? We've been working on lots of interesting things around. We will have another next episode coming soon, something on context graphs. I think there has been lot of chatter on X post. Anshu Garg and I think there's some post. So I think everybody sort of have understood the need for a universal context graph as a product for enterprise. So we'll be talking about it soon. In fact, we've been thinking deeply and I hope next episode will be on that. So Kunal, think, tell me as a hardware collector, do you, you know, are you enjoying that you're sitting on a bunch of, you know, like You have more RAM than the network of lot of people. How do you feel about it?" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1383, "timestampFormatted": "23:03", "text": "give it away man. So if I were a trader I would be super happy right because I would just sell the stuff and make money but because I have this what I would say a weird desire to just wander on just live and looking at my ramsticks those gave me so much joy keep in mind even if they are not powered on I really like it so I'll not sell it but I do find it very very stupid and that I have these two Threadripper boards, Threadripper Pro boards that I'm not able to populate because it requires eight DDR5 strings and no matter how I made it up. So that's the downside because I can't acquire more. That's why I'm unhappy, but I'm not very happy that I have so much. So I'm like human that way." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1425, "timestampFormatted": "23:45", "text": "But interestingly, think this is also a CES week and lots going on in CES. One of the things that caught my attention and I didn't know if it was timed like that, but Jensen released a brand new memory storage platform this week at CES conference. It's called as, which is equipped with four Bluefield, four DPUs, right? Each DPU supporting 150 TB of capacity, right? And this basically is tailor-made for NVIDIA's AI server act. what my point is, even though there seems to be a shortage on the back of the build out, right? The innovation on the next generation of AI hardware doesn't seem to be stopping, right? So which sort of gives me sort of hope that I think a lot of people will sort of solve this problem. you know, I think this maybe so what's what's your best bet? Would you Kunal, do you think the prices will ever go back to their original price or this is a new reality?" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1485, "timestampFormatted": "24:45", "text": "Man, I really wish that it could go. One of the person where I buy RAM from, first I spoke, he said 30,000 for 64 GB. I didn't buy. Then he said 60,000. Then he said 90,000. These days he says 1.5 lakh. So I don't know when that will go down. But I would like to comment on the thing that you have mentioned about what Jensen said. One of the things that Nvidia announced was that these large language memory are heavy on the KV context. The context that is stored in a KV cache. that's if your GPU, if you take 60 GB for model memory, then almost 20 GB is kept there. So while Janssen is doing for the enterprise world, lot of people will enjoy that. But there has been lot of effort in the open source world as well, where NVMe offload is being thought of. as one way to basically store these cache on a very fast storage system. So I'm excited about that. There is a work going on in llama.cpp to get that going. It doesn't support it yet. There is some frameworks here and there that do it. like it always happens, right? Whenever constraint comes, you will see a bunch of innovation happening. And this will be like one of those things." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1559, "timestampFormatted": "25:59", "text": "Yeah, one thing which I was super upset was I have been following up the, you know, the sort of release of GE Force 6 series, is so looks like the GE Force RTX 60 series graphic art would only come by the second half of 2027, which is" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1578, "timestampFormatted": "26:18", "text": "Which is not bad man, I think the press is essentially making this too much even if you look at the time difference between 3000 and 4000 series or 4000 and 5000 series there is huge difference because NVIDIA is not that much interested about it." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1594, "timestampFormatted": "26:34", "text": "You're right but I NVIDIA themselves announced RTX 50 Super Series card that is also" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1601, "timestampFormatted": "26:41", "text": "Yeah, you are right. Yes, yes, yes. So the Super Series got cancelled purely because of the memory. Correct." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1609, "timestampFormatted": "26:49", "text": "So a lot of it's not so I am definitely but I think one silver lining is you know the Rubin GR20X series which is essentially designed for AI. I've also been reading that they also have a version that released this year that can be used for gaming. So you know I believe you know there are definitely the timeline changes and lots happening in this. And, know, from an investment point of view, think let's, if we, we just from a pure stock search point of view, know, some of the companies who are into building these like Micron, SK Hynix, Samsung, do you think they are going to be, they're going to have the run similar to what? Nvidia or some other AI stocks had in say last couple of years." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1662, "timestampFormatted": "27:42", "text": "Man as you say that right, knowing very well in advance I didn't put any money in micron but yes it looks like they are only going up because there is nobody else right. Again we don't give stock advice but what I have seen is the market in general is still expanding. The day NVIDIA reports you know, something that is not huge growth. Only then we'll see something turning the other way around. If Nvidia keeps selling GPUs, why would you not need everything else? Why would you not need memory? Why would you not need SSD? Why would you not need power? Even if you look at something like Delta Power, which manufactures words like 80 % of power supply, they are going up. They're like significantly up. Same goes for every other component that exists. So the whole supply chain is being... taken together to new heights." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1711, "timestampFormatted": "28:31", "text": "Yeah, but basic question and I don't know, I think I probably have to read up more about this, but is making memory such a hard thing? is China not? Why isn't this, you know, like I understood this being a hard problem about, you know, semiconductors and certain nanometers in chip fabrication, but I don't think the same amount of complexity exists, should exist for memory as well right so why aren't some of these mass market products why do we have dependency on just few companies" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1745, "timestampFormatted": "29:05", "text": "Maybe just as an addition to that question as well, look, when GPUs started becoming such a big deal, that gave rise to other companies or other architectures of sorts. And there have been many examples in the past where Sysc was outdone in a way by RISC. New architectures came. GPUs were great at what they were doing. But then obviously, Grok. and Google showed that there are other alternatives as well and NVIDIA has shown its interest in that space as well. do you think there's, we also see a moment like that in memory where suddenly something will change and something new will come up. But Sachin if you dig deep on what you said and if you look at those alternative architectures, they are not there man. X86 is essentially one and the only other alternative is ARM and nobody else and ARM only became alternative and Apple essentially put their might behind it. If you look at GPUs, NVIDIA is making all the money, who else is there? Only Google's TPU is there but TPU, no you cannot own it, you can only rent it. Everything else is so distant. In the PC market, people always standardize too much. There has been work going on to create new types of RAM. But think about it man, the amount of money that it takes to move to a new architecture is so high. The resistance is so much that people, the catch will be this right? Either you optimize this versus move to a new architecture. It will feel like the same amount of work, but no buying new hardware. On to Vikas's question right? DRAM making is not easy. If this were easy, you would have more people making it. It needs like extreme precision. to shrink these capacitor transistor cells which struggle to hold charges as they get smaller and they require constant refreshing. know these days the RAM runs at like 6000 and even 7000 empties. And that's why the news that you would have seen, right? That China getting some information on how Samsung creates RAM is such a big deal and they protect it with their, you know, everything that they have. Did you?" }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1872, "timestampFormatted": "31:12", "text": "Did you know Sandisk stock jumped nearly 30 % in one day. So I'm just looking at some, some look at this data, right? Sandisk stocks in one year, has given 882 % return. Now read that and they don't even build RAM. This is just nine days." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1894, "timestampFormatted": "31:34", "text": "Because NVMe prices have gone up. So the demand for NVMe is following that all these models, they load data at very, very high rate. All the models even at my home, I'm not storing those on regular HTT. They're on SSDs because it is a must have. They go hand in hand and in hindsight, man, it looks so natural that this will happen." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1913, "timestampFormatted": "31:53", "text": "So." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1918, "timestampFormatted": "31:58", "text": "and I am now cursing myself to not pick up on this. Man, I should have picked up on everything like GPU and memory. I think, you know what, there is a component of networking also, which is going up, but you won't see that separately. You will not see that separately because NVIDIA is the largest networking equipment manufacturing after their acquisition of Mellanox." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 1930, "timestampFormatted": "32:10", "text": "JL center networking. Nice. All right. So I think, you know, we have, you know, we have had lot of discussions and we, we, I probably we left you with lot of questions than you had because we don't have all the answers, but definitely it's an interesting time. Any closing comments Sachin Kunal?" }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 1957, "timestampFormatted": "32:37", "text": "Yeah, think, look, we are entering a very interesting phase, right? I wouldn't call it all dull and gloomy like the media makes it out to be. there's a little, there is some sunshine on the other side. So while there could be some unrest in some near time future, but I think it will settle down and things will get better. So it may look a little difficult for now. And given that memory touches a lot of aspects of what we do, it touches a lot of aspects of the hardware we consume. But I think like everything that has been cyclical, whenever there's, and I think Kanal said it the best, right? Whenever there's constraint that has given rise to innovation. So I think we will see, we have to embrace this cycle because I think this cycle is what is going to give us a very novel set of innovations that I think will really help us. selectively as a species, right? So I think we should just look forward to this. Kunal, to you. Man, we were discussing Micron and in one month they have gone from 250 to 340. It's just the face of regret of not having bought the stock. True, true. And I have been knowing this since December. December it was 228, now it's 339." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 2030, "timestampFormatted": "33:50", "text": "You know interestingly Kunal was the first person who told me about these crazy RAM prizes And and this was long before it was in media true, right? This was in early November that he's been telling me and first when he thought I thought okay, come on That can't be that bad, maybe I thought it's just a little jump in the price of pre-loved hardware, but I never knew And every meeting now we go work where people especially who are in the data centers and enterprise. This is has been the discussion point people are, know, everybody knows that this is a new reality. it's." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 2071, "timestampFormatted": "34:31", "text": "So what I will recommend is essentially what I have said before if you need it buy it now because it is only going to become expensive look at secondhand market as well if you can and that's about it like Sachin said there is enough and more interesting stuff that you will have anything that you do in this domain will be lot more appreciated by others as well. So if you have projects in mind or things in mind that will reduce memory consumption or use memory in interesting way, people will really appreciate that." }, { "speaker": "Vikas", "speakerRaw": "speaker-0", "timestamp": 2097, "timestampFormatted": "34:57", "text": "Okay, thank you. Thank you, Kunal. And we'll do a lot many episodes with Kunal, Sachin and Vikas. And thank you for listening to us. And as always, everything that's spoken about here represents our own opinions and doesn't represent that of our employers. With that, have a great day. Thank you." }, { "speaker": "Sachin & Kunal", "speakerRaw": "speaker-1", "timestamp": 2115, "timestampFormatted": "35:15", "text": "Thank you." } ], "chapters": [ { "title": "Hi everyone, welcome to the AI Valley podcast", "timestamp": 52, "timestampFormatted": "0:52" }, { "title": "100k, 90k is 1k", "timestamp": 232, "timestampFormatted": "3:52" }, { "title": "Absolutely, think for a price of a RAM stick now, ...", "timestamp": 428, "timestampFormatted": "7:08" }, { "title": "And I was looking at one another list of obviously...", "timestamp": 599, "timestampFormatted": "9:59" }, { "title": "Also in covid I think when there was a supply chai...", "timestamp": 770, "timestampFormatted": "12:50" }, { "title": "I mean, yeah, absolutely", "timestamp": 935, "timestampFormatted": "15:35" }, { "title": "And also I believe it will reflect in the choice o...", "timestamp": 1123, "timestampFormatted": "18:43" }, { "title": "Sure, absolutely", "timestamp": 1329, "timestampFormatted": "22:09" }, { "title": "Man, I really wish that it could go", "timestamp": 1485, "timestampFormatted": "24:45" }, { "title": "Man as you say that right, knowing very well in ad...", "timestamp": 1662, "timestampFormatted": "27:42" }, { "title": "Did you know Sandisk stock jumped nearly 30 % in o...", "timestamp": 1872, "timestampFormatted": "31:12" }, { "title": "You know interestingly Kunal was the first person ...", "timestamp": 2030, "timestampFormatted": "33:50" } ], "quotes": [ { "text": "So Kunalji, think the biggest AI event, I think last year was when Sam Altman walked into Seoul and walked out with 40 % of the world's DRAM supply.", "speaker": "Vikas", "timestamp": 250, "timestampFormatted": "4:10" }, { "text": "So for example, you would have seen Crucial, which is the consumer brand of Micron, being shut down because they want to focus on enterprise market and as you know, enterprise market typically is higher margin market.", "speaker": "Sachin & Kunal", "timestamp": 336, "timestampFormatted": "5:36" }, { "text": "You want this to actually finish your job or complete something meaningful, then you should go acquire that RAM no matter the cost, because it is only going to increase.", "speaker": "Sachin & Kunal", "timestamp": 549, "timestampFormatted": "9:09" }, { "text": "Yeah, next version of all these lower margin GPUs, I think 5060s and 5070s, they are cutting production, they are not bundling the VRAM with it.", "speaker": "Sachin & Kunal", "timestamp": 636, "timestampFormatted": "10:36" }, { "text": "think that if we have to plan out a PC build out, even though RAM is burning, we should go ahead and do so because things are going to...", "speaker": "Vikas", "timestamp": 652, "timestampFormatted": "10:52" } ], "fullText": "[0:52] Sachin & Kunal: Hi everyone, welcome to the AI Valley podcast. I know it's been a while and we've missed you, but I hope you had a good set of holidays. Just in case, belated Christmas and a great New Year. I hope all of you guys have a great New Year going forward and we wish you all the best.\n\n[1:09] Vikas: Hi, and this is Vikas and wish you a great new year. We have had a amazing time traveling with family, friends, work and lot of things have been busy, not just work, but also the way things are shaping in both AI and in general. And today we have a very special episode because we have with us a very special guest between us, Kunal. Kunal is not just a dear friend, but he's One, I've seen lot of people who are passionate about hardware and accumulating vintage computers and continuously building capacity for their personal usage. And he's someone who's been not just passionate about computer hardware, post AI, but he's been all around. So given the fiasco that's happening in the world of the DRAMs and the AI build out. We thought there has been everywhere we go, we hear people talk about. So we thought to bring a very special episode about hardware winter and everything that's going crazy in the world of hardware. So welcome Kunal, how are you?\n\n[2:15] Sachin & Kunal: Well, Vikas and Sachin, thank you so much for inviting me here. Really appreciate it. Actually, know what, folks, I have a very interesting story to share. a couple of months back, think maybe slightly more than a couple, think three or four months back, I was trying to build my own PC and we'll come to this. just a PC, but a workstation. Yeah, a proper workstation. And I guess all thanks to the man here because A lot of the components that are in that workstation today were given by him, suggested by him. In fact, the nice little 4090 that sits in and does all the work for me, that's also gifted. I mean, not gifted, but he gave it to me. But I think, know, and here's why I thought, you know, we should have this discussion today, right? Because obviously with the workstation, I was looking at RAM, we went ahead, you know, did a purchase and It was such a funny story because we went ahead and purchased these four sticks and the user I mean the seller gave me a good rate which we thought is a good rate All of a sudden then we decided you know what we don't need such a beefy station and we wanted to return that back But the seller was not ready to do it and by the way, this was 64 GB DDR5 ECC memory. Yes. Yeah, four four sticks of it. It's you can imagine and We wanted to return it and I'm so thankful, so thankful that the person didn't take it back because now this is definitely the discussion of today. Look at what we did four months back and the world has changed in the next four months. Do you remember the cost at which you bought each stick? I mean, yeah, 90k for all the four. All the four, right? Right now, if you to\n\n[3:52] Vikas: 100k, 90k is 1k.\n\n[3:52] Sachin & Kunal: Yeah and right now if you look at it a single stick of RAM is almost $300 64 GB\n\n[4:03] Vikas: Yeah. So I think the\n\n[4:04] Sachin & Kunal: Sorry, not 300 dollars. 1500 dollars. Is a single stick of 64 GB RAM.\n\n[4:10] Vikas: So Kunalji, think the biggest AI event, I think last year was when Sam Altman walked into Seoul and walked out with 40 % of the world's DRAM supply. So in many words, he was playing a 4D chess to block everyone else from hyperscaling. the story goes is OpenAI signed agreements with Samsung and ASK INIX for up to 900,000 DRAM wafers per month, right? Because it seems like he, neither company both SK Hynix and Samsung knew that what the deal other way, you know, signing and Samsung assumed that they were giving OpenAI a meaningful allocation. So did SK Hynix and by the time both realized they collectively handed over nearly half of global DRAM output, right? And so to your point, the 32 GB DDR5 that sold in $95 in September is $400 in December, right? That's about four times the price and Obviously gamers and PC builders and general consumers are hating it. But I believe, I think we've had lot of discussions around this. The implications go way beyond this, right? Because this hardware winter is going to last for maybe a year, two years, four years. Who don't? Nobody knows, right? So what's your thought on how this will play out in everywhere there is memory, which is, you know, your PCs, your... cars, know, handhelds, consoles, right? How is the world of computer hardware is gonna look like?\n\n[5:36] Sachin & Kunal: It is going to be quite difficult because as you know, memory is everywhere, not just computers but also in smart TVs, watches, and even smaller things like say a microcontroller. So memory is quite a fundamental building block, which essentially the impact will be lot more universal. And there is not even what you call a deadline on when this will be over. So nobody is talking about standing up new fabs. The manufacturers of these companies, there only three companies, SK Hynix, Samsung and Micron. They are all projecting that things will be like this till 2028. And you wouldn't expect a lot from these manufacturers anyway. There have been many cases in which these companies have colluded to price fix. RAM itself and now they have such an amazing opportunity that if they would have wanted, they could stand up new fabs, but there is no intention on their part and they all are shutting down consumer businesses. So for example, you would have seen Crucial, which is the consumer brand of Micron, being shut down because they want to focus on enterprise market and as you know, enterprise market typically is higher margin market. And because of all of that, what they're expecting the trend is, hey, why do you need to own your hardware? You can just rent it. And this impacts a lot of hobbyist or anybody who wants to build a system. Because if you look at the prices of RAM, Vikas, you remember, right? You build out the entire system for, I think, like 50,000. Now, RAM alone will be like one like 30,000 in that price component.\n\n[7:08] Vikas: Absolutely, think for a price of a RAM stick now, know, entire PC would have been built in six months. That's very clear. See, what's most surprising to me, right? And when I went through the details, right, there is a sort of a second school of thought about artificial scarcity, because if you look into the details of the deal that Sama had done with SK Hynex and Samsung is these are to buy unfinished memory modules, right? To people who want to understand what unfinished memory model. So, which basically means that these unfinished memory chips are not allocated to a specific DRAM standard, right? So, it's unclear to me how they will finish them into RAM stick into HBM. You know, right now it seems like these wafers will just be stockpiled into warehouses, right? I don't understand.\n\n[7:58] Sachin & Kunal: There could be right so typically the same DRAM can be made into an HBM can be made into DDR can be made into GDDR which is the graphics memory. So see there could be price collusion but I wouldn't say this is artificial scarcity because if you are a supplier and somebody already gives you a contract that this is the DRAM I want then you will block it right and if you don't have that then you can't allocate it and the thing is Sam has purchased or because these are enterprise deals right HBM or any enterprise the margins will be lot higher. So you would essentially do that instead of provisioning say RAM sticks particularly these consumer RAM sticks they will not look at that. So the fundamental issue is this all of these RAM types ultimately require the same manufacturer the bottleneck is essentially there the materials are same.\n\n[8:48] Vikas: Okay, so now tell me as a hobby, PC builder and how does this impact you personally, right? Because you've been buying hardware at a very sustained rate at different price points, right? Now, are you still buying? Are you not buying? What advice do you have for PC builders? Should they wait this out? Should we buy stabilizers?\n\n[9:09] Sachin & Kunal: No, wouldn't. Stabilizer, sorry. So I wouldn't recommend waiting this out at all. Let's break it, break this problem, right? If you have a need, right? You want this to actually finish your job or complete something meaningful, then you should go acquire that RAM no matter the cost, because it is only going to increase. It is as simple as that. The second thing is it is just nice to have. It is not critical, but you are probably using it for gaming or some other purpose. Then you have to look, you make do with the hardware that you have? If the answer is yes, then use it. And one of the side effects that you will see of this winter, because it's a long winter, software which we'll talk about later, right? That will also be optimized to run in smaller amount of RAM memory itself. So my recommendation is If you need it, you need to get it now.\n\n[9:59] Vikas: And I was looking at one another list of obviously RAM is burning but what is almost burning is also SSDs and small pre-built PC component and Radeon GPUs right because as you're aware AMD doesn't bundle RAM in their bomb kits to like the Nvidia does right so the RTX 9070 GRE 16 GB you know lot of these products have been canceled because of the lack of RAM, right? And similarly, Xbox, we're not hearing any news and\n\n[10:36] Sachin & Kunal: Yeah, next version of all these lower margin GPUs, I think 5060s and 5070s, they are cutting production, they are not bundling the VRAM with it. And they are looking to sell more and more of these higher cost professional workstation series of GPUs because they can make margin.\n\n[10:52] Vikas: think that if we have to plan out a PC build out, even though RAM is burning, we should go ahead and do so because things are going to...\n\n[11:00] Sachin & Kunal: Look at second hand market, if you can see some deal, I still see on ebay.com or on various classified sites, whether it is OLX or whether it is a bunch of other sites that you see, you will see older generation, the Xeon workstations or AMD workstation with DDR4 memory can still be have at like relatively lower cost. If you don't have that see the point is this it is not going to the price is not going to go down in like next year or so it is going to continue for a long.\n\n[11:36] Vikas: Okay, I have one question like say I have a bunch of RAMs, know, or years just like you we have collected lot of hardware. So should we book profit and sell out? Is it a good time to sell old laptops? You know with those DDR4s and files?\n\n[11:49] Sachin & Kunal: It depends on what profession you are in. So if there is no need and you had like lot of ramp sitting in attic and you want to maximize profit, then you can probably wait it out even further because like we said, Ramp prices are not coming down yet. But obviously like stock market is difficult to time these things. So but yes, they are pretty high now, but there is no way of saying that it will not increase further. So, Kunal, see, generally what we have noticed, this has been cyclical in a way, right, this has happened in the past as well. But do you think this cycle is any different? Do you feel this will play out a lot more longer or do you feel there is genuine reasons why companies are doing what they're doing? This cycle is quite different than what had happened previously. So, for example, if you remember when the hard drive prices went up, It was because of the flood in Thailand. And then these manufacturers essentially hard on that and the prices were up for a long time. The DRAM prices also went up but because mostly due to collusion.\n\n[12:50] Vikas: Also in covid I think when there was a supply chain\n\n[12:54] Sachin & Kunal: This is supply is like too much demand and also the supply chain is not moving as efficiently especially China being closed but this time the math is like fairly straightforward and simple and the way it is looked like is folks are building data centers and they need more and one way to look at it is If you we spoke about these classified websites like Craigslist or eBay. You do not see. Ampere generation of GPUs in those market marketplace as you will not find a lot of A100s. You know when A100 came like four or five, six years ago, those are still being utilized. On top of that, Jensen Wang keeps releasing new GPUs. you would have seen in the recent keynote at CES he has announced Vera Rubin and even though Rubin and before that you had Blackwell, Hopper they were improving massively so for example Blackwell was 10x of Hopper Rubin is 10x of Blackwell but do you see any of those GPUs Jensen would say in the keynote that people couldn't give it away even if they would have wanted. And this is essentially the proof that market will eat up any amount of GPU that you give them. And that's why it's good to be invested in Nvidia. This is not we telling it. Right. So my point essentially is that this demand is there. There is RAM, the GPU. these newer graphics cards, they started with 80 GB on the enterprise side, now then went to 150, 180 and now these newer chips are even more, right? So I don't see it slowing at all unless AI doesn't work out, which is not the case.\n\n[14:38] Vikas: point I think and Sachin and I we were actually discussing this earlier today and this is a question for Sachin you know like just let's let's play that scenario do you think this shortage is going to lead to you know a different paradigm shift in software where you know the memory hog browsers application who have been very liberal in their RAM usage are going to take a new look where there are new compression algorithms which offload their sort of RAM usage onto the disk or handle their memory usage maybe you know Chrome Lite or something like that where every browser or every application becomes extremely cautious and judicious about their memory usage right it can drive a lot more new creative do you think We are at a point that people can use these AI coding and use the next wave of these improvements to build these memory efficient softwares.\n\n[15:35] Sachin & Kunal: I mean, yeah, absolutely. think that's an interesting point you've raised. think, look, every challenge is an opportunity, right? So I think where we see a challenge, there could be someone else who's looking at it as an opportunity. And they may try to build something that circumvents this problem. In fact, was actually quite coincidentally was the question that I wanted to ask Kunala as well, is that what happens to developing countries, right? Because while we look a lot towards the West and we see that countries over there will be able to afford some of the outcomes of high-priced memory, there are also countries on the other side where it may not be so affordable, right? Because memory is in every device that we use. So I think this is going to play out on the consumer durable segment as well. People may have to buy more expensive hardware for some time, more expensive durables for some time. But I think there will be innovation that will happen in the software side. what do you think about that? How will this play out for countries that are now just beginning to afford some of the most expensive durables?\n\n[16:39] Vikas: for another year before I update my iPhone or phones\n\n[16:43] Sachin & Kunal: So India in a way has leapfrogged US or other developed countries in some adoption factor. So for example, gaming is much bigger on phones here compared to other countries. We have We have prices impacting these computers and servers more. Phones hasn't happened yet. It will take time. I'll say general public may or may not see it quite as we do because we typically are lot more invested. The Western folks will also see because they generally participate with lot more devices like this. Software will definitely get updated to reflect this change. As you would have seen like an NES game in like 1988, you could put an entire game of Mario with like 11 levels in just think a few kb's and I'm not saying it was 3D and all but my point is you lay out a limitation and you can essentially live in it. There is some sort of laziness right now in today's software that you will see everywhere. These websites, have these powerful JavaScript compilers, right? Even though you are just looking for some interactivity away to know information when you want. Those things are just fancy things that may or may not require happen. You know, we may go back to native software a lot more because they are lot more efficient, energy efficient software. And even if you look at, even before this RAM thing happened, all the religious that happened, they always look for something called performance per watt. Right? Performance per watt was like essentially on the energy part, right? Now people can look at things like performance per memory. Like for this amount of memory, what do you do? There are a lot of algorithms. You remember, right? A lot of people hate answering. these lead code type of questions in the interview, right? Space complexity like, hey, do you store O, N or O, N square? People care about those things, right? So the all this negativity, the good thing that comes out of it is people will care about memory uses of their software again.\n\n[18:43] Vikas: And also I believe it will reflect in the choice of backend frameworks as well. I just, you know, reading there are languages like web assembly, which offload the compute from server to browser and stuff like that. So there'll be a lot more distributed paradigm where, know, since I can't maybe horizontally scale on expensive because it's not a commodity compute anymore. probably I'm going to serve your request and offload as much compute on these devices. In fact, you know, one of the thing that Kunal has been doing after this DRAM prices become DRAM and NANDs have become the new oil is he's started to sort of give put his focus on acquiring these small form factor devices. And I think one of the interesting takes that he has is, know, as model becomes smaller and smaller parameter model, even a 7B models are getting a lot more. It would be possible to run some of those things on an edge device, right? And some of these, and given the RAM crisis, they would like a nimble GPD box might be a server grade in a hardware winter, right? So I think these things are definitely going to show up in, you know, not just in the play. Kunal, you mentioned gaming. think what worries me most is the quietness that Valve have or because with a lot of fanfare, I think about two months back before all of these blew out, right? You know, they announced their steam machine, right? Obviously, it is unclear and unknown whether they bought pre-bought the RAM before announcing their much hyped steam machine. or if they already have a stockpile of ample supply of DDR5. If not, then I'm sure the steam machine launch should be affected as well because like every gamer, believe I'm very actively waiting for that device. But if the supply could dry up temporarily at some point, they'll wait for the prices to drop.\n\n[20:43] Sachin & Kunal: But man, I still think it's a good thing because Steam is more than a box, right? So Steam's most of the thing that they have done is on the software side. the first time in the history, they have built out a platform that stands tall in front of windows and can be a viable gaming platform today. So there is a distribution of Steam called Bazite. and it runs Windows game because of all the work that Valve has done in terms of optimizing their stack, providing compatibility layer for DirectX games to run on Linux. Now what that gets you in this world of this hardware winter where RAM is not available because Windows is lot more resource heavy. There is lot of processes that Windows itself will consume. And time and time again, many reviewers have proved that the games written for Windows, not written for Linux, work better on Linux using Steam's layer that they have built, than it will be on a native platform. This was unheard of, like you would achieve this kind of performance. So my point being is, if you have a system which is resource constraint, soon, It is much better to install Linux and use something like Bazite or Steam to run games than this. Of course, Box, we do not know what has happened there, but you would have seen the pricing that they were attempting, right? So I think it's a good thing that Steam exists and it will be more popular in this hardware constrained world.\n\n[22:09] Vikas: Sure, absolutely. What else, Hashim? What's keeping you excited, I think? We've been working on lots of interesting things around. We will have another next episode coming soon, something on context graphs. I think there has been lot of chatter on X post. Anshu Garg and I think there's some post. So I think everybody sort of have understood the need for a universal context graph as a product for enterprise. So we'll be talking about it soon. In fact, we've been thinking deeply and I hope next episode will be on that. So Kunal, think, tell me as a hardware collector, do you, you know, are you enjoying that you're sitting on a bunch of, you know, like You have more RAM than the network of lot of people. How do you feel about it?\n\n[23:03] Sachin & Kunal: give it away man. So if I were a trader I would be super happy right because I would just sell the stuff and make money but because I have this what I would say a weird desire to just wander on just live and looking at my ramsticks those gave me so much joy keep in mind even if they are not powered on I really like it so I'll not sell it but I do find it very very stupid and that I have these two Threadripper boards, Threadripper Pro boards that I'm not able to populate because it requires eight DDR5 strings and no matter how I made it up. So that's the downside because I can't acquire more. That's why I'm unhappy, but I'm not very happy that I have so much. So I'm like human that way.\n\n[23:45] Vikas: But interestingly, think this is also a CES week and lots going on in CES. One of the things that caught my attention and I didn't know if it was timed like that, but Jensen released a brand new memory storage platform this week at CES conference. It's called as, which is equipped with four Bluefield, four DPUs, right? Each DPU supporting 150 TB of capacity, right? And this basically is tailor-made for NVIDIA's AI server act. what my point is, even though there seems to be a shortage on the back of the build out, right? The innovation on the next generation of AI hardware doesn't seem to be stopping, right? So which sort of gives me sort of hope that I think a lot of people will sort of solve this problem. you know, I think this maybe so what's what's your best bet? Would you Kunal, do you think the prices will ever go back to their original price or this is a new reality?\n\n[24:45] Sachin & Kunal: Man, I really wish that it could go. One of the person where I buy RAM from, first I spoke, he said 30,000 for 64 GB. I didn't buy. Then he said 60,000. Then he said 90,000. These days he says 1.5 lakh. So I don't know when that will go down. But I would like to comment on the thing that you have mentioned about what Jensen said. One of the things that Nvidia announced was that these large language memory are heavy on the KV context. The context that is stored in a KV cache. that's if your GPU, if you take 60 GB for model memory, then almost 20 GB is kept there. So while Janssen is doing for the enterprise world, lot of people will enjoy that. But there has been lot of effort in the open source world as well, where NVMe offload is being thought of. as one way to basically store these cache on a very fast storage system. So I'm excited about that. There is a work going on in llama.cpp to get that going. It doesn't support it yet. There is some frameworks here and there that do it. like it always happens, right? Whenever constraint comes, you will see a bunch of innovation happening. And this will be like one of those things.\n\n[25:59] Vikas: Yeah, one thing which I was super upset was I have been following up the, you know, the sort of release of GE Force 6 series, is so looks like the GE Force RTX 60 series graphic art would only come by the second half of 2027, which is\n\n[26:18] Sachin & Kunal: Which is not bad man, I think the press is essentially making this too much even if you look at the time difference between 3000 and 4000 series or 4000 and 5000 series there is huge difference because NVIDIA is not that much interested about it.\n\n[26:34] Vikas: You're right but I NVIDIA themselves announced RTX 50 Super Series card that is also\n\n[26:41] Sachin & Kunal: Yeah, you are right. Yes, yes, yes. So the Super Series got cancelled purely because of the memory. Correct.\n\n[26:49] Vikas: So a lot of it's not so I am definitely but I think one silver lining is you know the Rubin GR20X series which is essentially designed for AI. I've also been reading that they also have a version that released this year that can be used for gaming. So you know I believe you know there are definitely the timeline changes and lots happening in this. And, know, from an investment point of view, think let's, if we, we just from a pure stock search point of view, know, some of the companies who are into building these like Micron, SK Hynix, Samsung, do you think they are going to be, they're going to have the run similar to what? Nvidia or some other AI stocks had in say last couple of years.\n\n[27:42] Sachin & Kunal: Man as you say that right, knowing very well in advance I didn't put any money in micron but yes it looks like they are only going up because there is nobody else right. Again we don't give stock advice but what I have seen is the market in general is still expanding. The day NVIDIA reports you know, something that is not huge growth. Only then we'll see something turning the other way around. If Nvidia keeps selling GPUs, why would you not need everything else? Why would you not need memory? Why would you not need SSD? Why would you not need power? Even if you look at something like Delta Power, which manufactures words like 80 % of power supply, they are going up. They're like significantly up. Same goes for every other component that exists. So the whole supply chain is being... taken together to new heights.\n\n[28:31] Vikas: Yeah, but basic question and I don't know, I think I probably have to read up more about this, but is making memory such a hard thing? is China not? Why isn't this, you know, like I understood this being a hard problem about, you know, semiconductors and certain nanometers in chip fabrication, but I don't think the same amount of complexity exists, should exist for memory as well right so why aren't some of these mass market products why do we have dependency on just few companies\n\n[29:05] Sachin & Kunal: Maybe just as an addition to that question as well, look, when GPUs started becoming such a big deal, that gave rise to other companies or other architectures of sorts. And there have been many examples in the past where Sysc was outdone in a way by RISC. New architectures came. GPUs were great at what they were doing. But then obviously, Grok. and Google showed that there are other alternatives as well and NVIDIA has shown its interest in that space as well. do you think there's, we also see a moment like that in memory where suddenly something will change and something new will come up. But Sachin if you dig deep on what you said and if you look at those alternative architectures, they are not there man. X86 is essentially one and the only other alternative is ARM and nobody else and ARM only became alternative and Apple essentially put their might behind it. If you look at GPUs, NVIDIA is making all the money, who else is there? Only Google's TPU is there but TPU, no you cannot own it, you can only rent it. Everything else is so distant. In the PC market, people always standardize too much. There has been work going on to create new types of RAM. But think about it man, the amount of money that it takes to move to a new architecture is so high. The resistance is so much that people, the catch will be this right? Either you optimize this versus move to a new architecture. It will feel like the same amount of work, but no buying new hardware. On to Vikas's question right? DRAM making is not easy. If this were easy, you would have more people making it. It needs like extreme precision. to shrink these capacitor transistor cells which struggle to hold charges as they get smaller and they require constant refreshing. know these days the RAM runs at like 6000 and even 7000 empties. And that's why the news that you would have seen, right? That China getting some information on how Samsung creates RAM is such a big deal and they protect it with their, you know, everything that they have. Did you?\n\n[31:12] Vikas: Did you know Sandisk stock jumped nearly 30 % in one day. So I'm just looking at some, some look at this data, right? Sandisk stocks in one year, has given 882 % return. Now read that and they don't even build RAM. This is just nine days.\n\n[31:34] Sachin & Kunal: Because NVMe prices have gone up. So the demand for NVMe is following that all these models, they load data at very, very high rate. All the models even at my home, I'm not storing those on regular HTT. They're on SSDs because it is a must have. They go hand in hand and in hindsight, man, it looks so natural that this will happen.\n\n[31:53] Vikas: So.\n\n[31:58] Sachin & Kunal: and I am now cursing myself to not pick up on this. Man, I should have picked up on everything like GPU and memory. I think, you know what, there is a component of networking also, which is going up, but you won't see that separately. You will not see that separately because NVIDIA is the largest networking equipment manufacturing after their acquisition of Mellanox.\n\n[32:10] Vikas: JL center networking. Nice. All right. So I think, you know, we have, you know, we have had lot of discussions and we, we, I probably we left you with lot of questions than you had because we don't have all the answers, but definitely it's an interesting time. Any closing comments Sachin Kunal?\n\n[32:37] Sachin & Kunal: Yeah, think, look, we are entering a very interesting phase, right? I wouldn't call it all dull and gloomy like the media makes it out to be. there's a little, there is some sunshine on the other side. So while there could be some unrest in some near time future, but I think it will settle down and things will get better. So it may look a little difficult for now. And given that memory touches a lot of aspects of what we do, it touches a lot of aspects of the hardware we consume. But I think like everything that has been cyclical, whenever there's, and I think Kanal said it the best, right? Whenever there's constraint that has given rise to innovation. So I think we will see, we have to embrace this cycle because I think this cycle is what is going to give us a very novel set of innovations that I think will really help us. selectively as a species, right? So I think we should just look forward to this. Kunal, to you. Man, we were discussing Micron and in one month they have gone from 250 to 340. It's just the face of regret of not having bought the stock. True, true. And I have been knowing this since December. December it was 228, now it's 339.\n\n[33:50] Vikas: You know interestingly Kunal was the first person who told me about these crazy RAM prizes And and this was long before it was in media true, right? This was in early November that he's been telling me and first when he thought I thought okay, come on That can't be that bad, maybe I thought it's just a little jump in the price of pre-loved hardware, but I never knew And every meeting now we go work where people especially who are in the data centers and enterprise. This is has been the discussion point people are, know, everybody knows that this is a new reality. it's.\n\n[34:31] Sachin & Kunal: So what I will recommend is essentially what I have said before if you need it buy it now because it is only going to become expensive look at secondhand market as well if you can and that's about it like Sachin said there is enough and more interesting stuff that you will have anything that you do in this domain will be lot more appreciated by others as well. So if you have projects in mind or things in mind that will reduce memory consumption or use memory in interesting way, people will really appreciate that.\n\n[34:57] Vikas: Okay, thank you. Thank you, Kunal. And we'll do a lot many episodes with Kunal, Sachin and Vikas. And thank you for listening to us. And as always, everything that's spoken about here represents our own opinions and doesn't represent that of our employers. With that, have a great day. Thank you.\n\n[35:15] Sachin & Kunal: Thank you.", "generatedAt": "2026-01-18T19:16:00Z" }