Integrating Hardware and Software Teams
Awesome. John, good to good to hear from you. How are you?
Speaker 2:Yeah. Hey. Thanks for inviting me. Yeah. I'm doing I'm doing well.
Speaker 1:I, you know, I I feel like this is like a it's a crossover episode between On the Metal and, and Oxide Friends. It's it's like a, you know, it's like 2 cinematic universes coming together. Actually, something like that. But, you know.
Speaker 2:Let let's milk it for a few years, and then we'll we'll have some very interesting crossroads. Exactly.
Speaker 1:Well, the and and I feel like Tom Line's been here as well. Tom regular, which has
Speaker 2:been great.
Speaker 1:So another on the metal guest, esteemed on the metal guest.
Speaker 3:Howdy, howdy.
Speaker 1:And as someone who's who's lived, obviously, his life at the hardware software interface. So, John, I mean, obviously, this got started with your tweet, which I've got so that just to give people context, and, Adam, I don't know if you wanna drop it the his tweet in. But this is, John's tweet decrying the, the poor relationship that can develop between hardware and software teams. And, John, I gotta believe maybe this is wrong. I gotta believe that's gotta be a subtweet.
Speaker 1:There's gotta be an incident that you maybe don't need to go into detail on. But, I I could just feel your frustration when you said we have an even bigger crisis, which is that we still militantly, your emphasis, separate hardware and software people. We put them in different orgs, discourage them from talking to one another, foster mutual hatred, all of which is disgusting and all caps, wrong. Fix the broader issue, which is stupidly siloing. We obviously strongly agree.
Speaker 1:Was there an incident that pushed you over the edge that you can at least speak with speak to abstractly?
Speaker 2:Yeah. You know, I I love to get myself trouble. It's it's it's what I do. Right? Now it so firstly, it's nothing to do with what I currently do for a living.
Speaker 2:I'll put that out there.
Speaker 1:That's right. Soliciting. I'm not subtweeting any of my coworkers as far as we know. Yeah. Yeah.
Speaker 1:Yeah. No. No.
Speaker 2:I wasn't. It was it was I think it was I was retweeting, actually, a a tweet somebody else had about, the the state of hiring and the the the shocking state of, you know, we have all these folks going into software, but not into hardware. And and I'm like, yeah. Because, nobody knows any better. I mean, we we we just we we do a lousy job as an industry at integrating everybody into one cohesive whole.
Speaker 2:We don't do that. And, if you're coming into the industry today, you've probably, from an early age, had it beaten into you that, you know, learning Python or, you know, whatever else is is the thing to do and go into software and, you know, practice leak code and get hired at some tech company and all this kind of thing. Right? And and, there's a there's a broader meta issue, which is sub subtweeting our, on the metal episode. Right?
Speaker 2:Moore's Law is slowing down. The free lunch is over.
Speaker 1:Yeah.
Speaker 2:And if we and and and people are people want more interesting stuff. Like, we hear a lot of people talking about buying you know, I'm a I'm a Mac fan. Okay? But people have thought fond of different things. But there's a trend towards very integrated optimized devices and experiences.
Speaker 2:And you don't get those by having your, you know, 19 eighties PC with your plug in adapter cards. And, you know, I I'm I'm I'm definitely a PC enthusiast. Right? But the old school world of I, you know, buy a box, and I put all these pieces in there, and I don't think about the whole thing from the beginning. That's going away, and it ain't coming back.
Speaker 2:And it's because the free lunch is over. Right? And so you have to design these things more deliberately. And if you're doing that, let's, for goodness sake, fix the hiring mess and the the industry mess of people not not working together. And and I tweeted it in that particular way because I've spent not as long as you in the industry, but almost.
Speaker 2:Right? We've we've had similar journeys over the past couple of decades of just seeing done so painfully in so many places. And it's time had changed.
Speaker 1:Amen. So yeah. Well, okay. A bunch of threads that that I wanna pick up. One of them is the the I mean, you you talk about the this kind of fostering of of mutual hatred, which I definitely feel I have seen, where people will criticize the elements they don't understand, or they'll trivialize those aspects the problems they don't understand.
Speaker 1:And I I I feel that we I mean, maybe maybe I'm biased because I think that the problems hardware and software problems are different. I mean, we're trying to build a system with them, but they are and in particular, like, the hardware actually has to work, and the Does that Right. Exactly. But I I feel that, like, the constraints on the problem are so different because it's gotta be it's gotta stick the landing. And software can iterate to the point where I mean, software is to the contrary, like, often never done.
Speaker 1:And I I do feel that there's often some misunderstandings and some some I some of the these perceptions that arise are really inaccurate. And I feel that, you know, you get you definitely get folks that are disparaging hardware as and trivializing it because all they have known is functioning hardware. And they are
Speaker 2:All they've known is boring. Right? They've had a PC for decades that just they turn on and and they grumble about, oh, this BIOS update sucked or, you know, I didn't like that update or something like that. And and I'm like, you have no idea. Like, it worked so much at the time that Katie was about to point out.
Speaker 2:You know, I I think you pointed out just now the the the relative cost of failure or mistakes, right, in the 2
Speaker 1:different domains, if you wanna separate
Speaker 2:them out. Right? Like like, the cost of making a mistake, domains, if you wanna separate them out. Right? Like like, the cost of making a mistake with a piece of silicon is, you know, tens to 100 of 1,000,000, if not more dollars and months.
Speaker 2:Right? It's these days, it's more like and you've delayed your product by 6 months, and you're screwed. Right?
Speaker 1:Right. Because you
Speaker 2:needed to make the holiday or whatever it was you were targeting. And software, software, it doesn't have to be done because you can just ship an update. But that's an unhealthy mentality too. But but I but I think there is sort of, you know, in the software community, there's this idea that hardware sucks, hardware, people are always making mistakes and Oh, if they would just fix it and, and, you know, that someone turns up in a in a, say, the Linux kernel community with some piece of hardware and they get I will filter my response. They get, they get pooped on by by by by people.
Speaker 2:Right? Your hardware sucks. You need to do this and this and this. And meanwhile, they're like, yeah. We totally get it.
Speaker 2:But also, we're shipping this right now in millions of units, and we need it to, you know so expectations are different as well. You're absolutely
Speaker 1:your mouth, but that may have been, what played an instigating role here.
Speaker 2:Over the years. Right? It it's it's constant. It's continual, the the sort of hating on hardware people. And I'm like, look.
Speaker 2:There's a lot of hardware that's really, really bad. Don't get me wrong. But but you you can't just say, oh, those hardware people. I mean, they have a lot of challenges. Like, you're shipping hardware, Brian.
Speaker 2:Right? If you make a an operator is just standing by. Right? They can buy hardware from you very soon. Right now.
Speaker 2:Right? So but if if if you make a mistake in your wonderful, rusty, magic firmware that's great, you can ship an update. If you make a mistake on a board, like, you're dealing with supply chain, what's what how long does it take you now? Ignore silicon. Right?
Speaker 2:Just just making a a a
Speaker 1:PCB change. How long does it take you? It is very cool. I mean, this is where maybe a good a good juncture to get Nathaniel in here, who's definitely on the on the cold face of that. Yeah.
Speaker 1:Nathaniel, how long would the port then take? It would be a while.
Speaker 4:If if we had the material on hand and it was a very targeted change, you're probably looking at, 8 weeks. And if you don't have material on hand or parts or your, sub tier contractor wants to, like, change the, you know, your SMT line time or whatever, then it could be months or quarters depending on what you do, you know, the scope of the change.
Speaker 1:And it would be regardless, it would be an excellent opportunity to learn how much more complicated the that whole process is than any I mean, I it is amazing to me how comp and I've I've said this before, but please can someone write the definitive history of the PCB? Because it is so important to every single thing we do, and I feel like I am still I mean, I'm still learning so much about, like, the the curing of the pre preg. Nathaniel was one of these things you're like, okay. Wait. Wait.
Speaker 1:The the pre preg is like milk. Like, we have to, like, drink it spires.
Speaker 4:Expires. Yeah.
Speaker 1:Dude, actually, just, like, talk about that particular issue. Not to not to dwell on that one. But yeah. So
Speaker 4:we're we're using, you know, we're using a tachyon a 100 g material on 1 or more of the boards we're doing. And, you know, we aspirationally ordered some ahead to be, you know, kind of, on top of things. And then, you know, of course, the schedule slides a little bit and we're, you know, a little bit behind on some of the design. And, you know, we're realizing that, all this prepreg that we, which prepreg, it's basically resin in a a fiber weave, and so it's our the resin has already been injected in there. So it has a shelf life.
Speaker 4:And, and and, you know, our vendor basically said, hey. You know, you have to use this by such and such a date. And, you know, we were gonna be a couple of weeks out of bed there. And so, you know, we ended up, like, having to work with them to put it in a freezer and, you know, like, keep it from expiring for a couple more weeks in order to, you know, get the thing done.
Speaker 5:Hey. Nathaniel, this is a very dad question, it goes off? Like, what what what's the what's the phenomenon?
Speaker 4:So your your the resin won't cure correctly as you, you know, as you, like, build the circuit board. And so your circuit boards can delaminate or they'll have various reliability problems. And they'll, you know, it it's a bad thing if, you know, your 22 layer circuit board, like, rips in half and you only have 10 layers left or something. So, like, that's kind of a problem.
Speaker 5:K. Yeah. That that sounds like a problem.
Speaker 2:Is this what's sticking the the sort of, like, all 4 layers together? I mean, how does that how does
Speaker 4:that fit in with Yeah. So the they have cores and and then fiber. And so they have a, like, rigid core and those those just have copper plated on both sides. Yep. And, those don't expire.
Speaker 4:They're just, you know, they're already cured. Right? Somebody has already done the work. And then, between that layer, then you'll put a prepreg layer down, after you etch etch the copper, and then you'll put another core on top of that, and and, you know, you just build up however many you need. But that stuff is you know, it's like wet chemistry.
Speaker 4:So you like, it has
Speaker 1:a Okay.
Speaker 6:It has
Speaker 1:it has a time. Well and this is what you something. Oh, no. Totally. And then, John, I feel like this is, like, every day in oxide.
Speaker 1:I learned that I don't actually understand how computers work and that actually something that has been load bearing for all of humanity. I have never heard of. And by the way, we can't get any of it or in the case of the prepreg, we've it's gonna expire. It's gonna go although not Adam's. Adam is still gonna Adam's gonna drink his, but the rest of us are not gonna use the Yeah.
Speaker 1:But let's try let's try a little exact Yeah. That's right. Just just just just snow around the edge. But I and I feel that, like, this John, I do feel that, like, one thing that I'd I certainly been very helpful on Oxide, and I would like to think we can do more generally, is just a little more cross pollination and education
Speaker 7:is that part of
Speaker 1:a double e undergrad curriculum? Because I don't feel that it is. I is is that part of a double e undergrad curriculum? Because I don't feel that it is. I feel like that that's something that people learn in the industry.
Speaker 4:It definitely wasn't in part of mine. So and, you know, I am a double e. So, you know, I learned that on the job. And, you know, I think the supply chain crisis has really brought a lot of that stuff, like, under the microscope because I think, you know, previously, you know, if you work at a big company, like, you you have people
Speaker 7:for that. Right? Like,
Speaker 1:somebody goes and does it, and
Speaker 4:your board manufacturer's like, oh, it's no problem. It's, like, 2 week lead time.
Speaker 8:You you actually have
Speaker 9:to source it. Right?
Speaker 1:Well, yeah. And I mean want
Speaker 2:I want a board. Oh, okay. Go to the board company I know locally or I want a small number or, you know, somewhere in, you know, Taiwan or wherever it is. Yeah.
Speaker 4:Yeah. And and you still do that, but they're like, you know, they're gonna tell you, okay. Well, you know, to get that fancy material, the lead time, instead of being the normal 4 weeks that it would be, which is usually, you know, hopefully, you're you're building circuit boards more than 4 weeks ahead of time if you'd kinda know what you're doing. And so, you know, it's not usually, like, the the long lead on anything. And then, you know, you get to hear it.
Speaker 4:It's like, well, you know, that's gonna be 22 weeks, for that.
Speaker 1:And it's like, well,
Speaker 4:22 weeks is kind of, like, out of our product project schedule for this. So, like, we have a problem. And, you know, so then you kinda get into all of the, you know so then we're buying stuff stuff a lot earlier than you normally would because you can't just in time it. And then when you buy it too early, it expires.
Speaker 1:It expires. So actually, this is a really good point though, Daniel, because, you know, I've often said that part of what I like about a system that is failing is that it reveals itself. And that when you have a system that has failed, when you've got a crash up or a cord up or whatever, you know, panic, whatever you have, you've gotta understand that full layer of that full stack of software. And I it's kinda like a just your point about this, the supply chain crisis is actually forcing us to be aware of some of this stuff that was load bearing that that even those who are who are practicing the art don't were necessarily aware of.
Speaker 3:Right? Yeah.
Speaker 1:I mean, I'm true. Yeah.
Speaker 7:I I
Speaker 2:would I would add, you know, I by the way, I I'll I'll admit I've never heard pre pre reconciled just now. So that that's great. I've learned something today. Right? I like to think you know, I know a little bit about PCBs.
Speaker 2:Normally, you my law my knowledge stops at, oh, I have this many layers, layers and you understand you have this many ground planes and all this kind of thing, but and power planes. But but the actual detail of it has always been this kinda nebulous black box that, I guess, I never had to care about. I mean, I care. I'm very interested. In fact, I'm gonna go and go down a what a Wikipedia rabbit hole on that later, But but, you know, it's not the sort of stuff that that you get exposed to.
Speaker 2:Right? Even if you are interested in this. That's like that's an that's an industry revelation right there.
Speaker 1:It it totally is. And I they totally is. And I think we need to do a better job as an industry of of communicating how complicated this stuff is and how we should be grateful when these things work at all. I mean, I feel like I mean, the thing I'll take another one. Is just I personally feel I don't understand well and I'm learning something new about every day is DDR and all of the tricks and stunts that are being pulled to increase bandwidth.
Speaker 4:Oh, yeah. Yeah. That's like a miracle that it works. So true miracle that it works.
Speaker 2:DDR 5. Right? That one is a is a true, you know, black magic. Right?
Speaker 1:The the well, I mean, d d d works
Speaker 4:is still in question.
Speaker 1:Because it's like, DDR5 will be a miracle when it works. Yeah. When it works.
Speaker 2:Right? But but, like, the the the latest one I was thinking about is it does easy it does built in ECC. Right? So so it it it it it's great, except you don't know how many things were corrected.
Speaker 4:Well yeah.
Speaker 2:And and
Speaker 4:and you know you know that they built in the ECC because there are errors intrinsically.
Speaker 2:Yes. The signs have
Speaker 8:error. Yes. Yes.
Speaker 1:Yes. Well, in just the whole idea of DIM training. I mean, I feel that, like, certainly for me, like, I did not learn about DIM training until Oxide even though I I realized in hindsight that dim training in it had adversely affected my life, namely the the poor dim training. And that I I mean, I feel and Robert's not here, but we had some some issues at at in previous life that I think were absolutely due to the fact that we were we're seeing dim training issues that were causing a ECC errors. But I feel like even dim training, I is this total like, this essential part of the machine Well that we don't talk
Speaker 2:about. That one's weird though. Right? So, like, if anyone doesn't know, you know, memory doesn't just work. You have to train the the timings on various signals, and it's all very device specific.
Speaker 2:And this is why, like, when you boot your server, you sit there watching the scrolling little dots go across. You know, DDR training is happening.
Speaker 1:DDR training is happening. It's very it takes wait. Because it that's the one, John. That is the one, like, blob that we don't deliver in the Yes. So we basically That's
Speaker 2:like that's like a secret little tree. That's the other thing that's really weird is that, like, they'll give you everything except, oh, the DVR training is special. Like, why? What's the value there? Right?
Speaker 2:It's it's weird. That one's very weird.
Speaker 1:I guess I'm glad that we on the one hand, we're not our code is not during the duet. On the other hand, like, you are just like looking at your stopwatch, you know, because it takes a lot it actually takes a long time to train the temps. Also, Nathaniel, is is training the right should it be like dim discovery?
Speaker 4:Well, it's it's doing more than discovery. Right? It's doing channel characterization and trying to find the middle of an eye. But, like, the trick is, you know, we do this on on SerDes and that kind of thing, and it's all kind of a fairly well known thing except that DDR is like a garbage channel. And so it's it's built that way because we need, you know, we need all that, you know, massively parallel high bandwidth stuff, but we also want it really cheap.
Speaker 7:Right.
Speaker 4:And so, like, you don't wanna use a, and and, you know, we're I mean, like, we're headed there. Right? Like, DDR 5 looks a lot more like a SerDes than, DDR 4 even does. But, like, the the cost of putting in the fancy, fancy stuff is is not really sustainable at that kind of parallelism, and you can't gain you know, like, you like we like to you know, if you wanna do 2 dibs per channel, well, you can't do that in a, like, SerDes based system. We get you can't just stick 2 people on the same transmit, you know, wire.
Speaker 4:But it but we do that for cost. And so it's doing it's doing electrical characterization to try and make it the best it can, given
Speaker 6:the channel that it has.
Speaker 2:And it cheats as well because it has there's an there's an I squared c channel that that you read this SPD info. So there's, like, the the VIM itself will say, hi. I'm this. Yeah.
Speaker 4:Brian knows that one really well.
Speaker 1:Oh, the the the this is, like, my favorite piece of hardware that so, Nathaniel, early on so we the we basically interpose on that. So are the service processor in the sled actually controls the DIMMs, but it means that it needs to at some point, the CPU is like, hey. I I actually need to know, like, what the SPD data. So the SP actually proxies that SPD data. And so which was very nerve wracking because it felt like, god.
Speaker 1:I this is one of these things that should theoretically work, but no one is doing it this way. And if it doesn't work, like, the machine's definitely not gonna boot. But to test this, I was like, man, I'm looking for and, John, you think this would be easy to find something that can take a dim where, actually, I've got no interest in the, like, the important stuff, the high speed channel. All I want is the I squared c interface. And I, like, kept, like, going back to Mouser over and over again being, like, I've got to be missing something on Mouser and Digikey.
Speaker 1:And I I for a while, I was convinced that there was, like, I there there was a a hidden art to searching Mauser and Digikey that I was unaware of, but I think it's just I maybe there is, Nathaniel. I don't know. But the Brut force. Yeah. I think that's yeah.
Speaker 1:I think it is brute force and just, like, absolute resilience. And I was I was kind of, like, asking, like, you know, this this this thing happen. Just and I remember you're just like, this thing should exist. Like, I think I can just, like, bang this thing out pretty quickly. And we have dimlets.
Speaker 1:Have you seen the dimlet?
Speaker 5:No. I haven't seen this. No.
Speaker 1:Oh, god. These dimlets are great. So the the the dimlets are take dims and then plug into a dimmlet. So we did which is our a dimmlet which is our smaller dimmlet that we use for SP development. It's our bring up vehicle effectively.
Speaker 1:And we're able to do all of our validation on that. And it's a great piece. I mean, Dana, we should throw that design out there because that's it's gotta be useful to someone else. Right?
Speaker 2:You should. Because because it's like I I've seen who knows where, but I've seen that done. It's like everyone's built one of these. Piece. Goodness me.
Speaker 2:I I mean, to
Speaker 10:to to be completely honest, the reason this thing is not on Digi Key is the fact that, like, you can just spin the board in, like, 4 days and you'll have it. Right?
Speaker 2:Yeah. I guess that's right.
Speaker 10:Everyone who needs to break out low speed signals from DDR, one, will never agree on the set of signals to be broken out.
Speaker 1:Right.
Speaker 10:Secondly, like, once you're at the point where you're talking about breaking out a DDR 4 DM, you better as hell know what you're doing anyway.
Speaker 1:Well and so so, Matt, you're hitting on an interesting point. And this is John. I wanna get this is one
Speaker 3:of the threads I want to
Speaker 1:pull on because one of the things that you said at the top, and I totally agree with, I think we all agree with, is that that when we design the entire system, hardware and software, we can deliver greater utility to the end user. There's there are things that we can do, and with Moore's Law slowing down, there are things we're gonna need to do. One of the interesting revolutions that, obviously, that we see is around open hardware, open source firmware, making it much more possible than it has been. I mean, because, historically, the and this is where, like, the double e's have been dealing with the absolute worst of software. I mean, can we just say can we just apologize to all double e's on behalf of all software that we're very, very sorry about the atrocious proprietary software that you needed to deal with.
Speaker 1:And it feels like open source software here I mean, because and they know that was a key I think you did that with KeyCad.
Speaker 4:Yeah. That's that's a KeyCad, isn't it?
Speaker 1:And, you know, and and KeyCad is not there yet, but feels like it's getting there for the bigger boards. We're definitely using it for all of our smaller boards. And it feels like, you know, increasingly, we are moving to an open world. And, Nathaniel, you've you've seen that happen from your perspective. Do you I mean, it feels like that would change a lot.
Speaker 4:Oh, yeah. I mean, I think it does. You know, I mean, it's interesting. You know, starting at Oxide was the first time I had even used, you know, open source FPGA tooling, is, you know, that's my big background is FPGAs. And so, you know, just, like, looking at the fact the fact that you can get out there and, like, go look at the code is really useful in a lot of ways because, you know, if there's a bug or you're using it wrong, like, those are things that you can get out there and do.
Speaker 4:And so I'm I'm totally looking forward to the day when we can have our whole sled, you know, done in some kind of open source CAD because, you know, we're we're living the dream without it right now.
Speaker 1:Well and it's I mean, I do feel but, like, because, Jon, I mean, you and I in our careers have have Arc and Atom, and we've we've kind of arced it, seen open source arcing in the system software and seen all the differences made at first with the operating system and then the database and all these other tiers system software. And yet that revolution is still coming on and we're still dealing with these proprietary tools that are just as bad as proprietary tools have always been, and it's so going. It's very, very frustrating. And just feels like there's a lot to unlock there.
Speaker 2:Well, I think I think, if I may, I think the other thing is, there's my wife has dropped. I guess I got I guess I got boring. So hi, Sarah. So, I I think I think, we we we are heading towards more and more open source hardware. It it's certainly possible to you know, if you're if you're in school, you know, 10 years ago, it was maybe open risk.
Speaker 2:Right? And and playing around with, you know, FPGAs and and starting to look at that kind of thing. Now, obviously, risk 5 is everywhere. You can download a design. You can you can play with things.
Speaker 2:There's there's, you know, there's there's, Matt Venn's 0 to ASIC course. There's there's stuff with SkyWater. You can go and take out chips. You know, this is all great. But still, it's pretty it's pretty hard for someone who doesn't know, the the whole stack, the hardware software stack to kind of do this from scratch.
Speaker 2:You know? Like like, remember Linux from scratch? Right? Now now I admit I never actually did the thing the whole way. Right?
Speaker 2:I built embedded Linux systems from scratch. But, you know, there was a Linux from scratch project. And if Gen 2 wasn't exciting enough for you, then you could you could do Linux from scratch and build everything from from the beginning, and those projects are great. I think you kinda need a system from scratch at some point that that that says, okay. You're gonna take a, you know, a RISC 5 core.
Speaker 2:You're gonna, build a little SOC, on an FPGA. You're gonna go and do the firmware. You're gonna go and do the software. You're gonna figure out everything from soup to nuts. I don't actually think that exists today.
Speaker 1:I don't think it does either, Nathaniel. I mean, you I I it's certainly like, that that would be, like, the dream undergraduate curriculum that would take you across that hardware software bound ring, which would be
Speaker 4:amazing. Yeah. I'm not aware of that. I'm also not you know, it's it's interesting because I feel like when you look at undergraduate programs, I'm not sure you'd actually get it done in your undergraduate program.
Speaker 1:That's that's the thing I was gonna ask.
Speaker 4:Yeah. Like, these systems are extremely complicated. And so, like, it depends on where you wanna draw the lines and what parts, you know, I mean, if you'd maybe if you have the hardware kinda built and just connect it, maybe, or if you have some of the software stack kinda ready to go. But it, like, I think it would be tough, you know, you I mean, I figure, like, I don't know that until sophomore or junior year, I even had enough skill, you know, in kind of like the electronics field up and down to be able to go that far into any one of those things.
Speaker 10:So But I I mean, there there are some things that would really help. Right? I mean, a lot of undergraduate curriculum still start
Speaker 1:off with breadboards and 74 series logic and spend
Speaker 10:a a rather sizable amount of the time in education having people plugging wires into breadboards.
Speaker 4:Oh, yeah. We gotta make an ALU.
Speaker 10:Right. Right. And and and clearly, this is a good use of everybody's time because as soon as we make it to industry, we swear off 74 series logic and never touch it again except for, like, 1 AM gate somewhere on a board.
Speaker 1:But also I I you I feel those exercises, though, are still valuable. Don't you? I mean, because I
Speaker 10:Brian. Here here's the problem. The theory is they teach debugging, but they mostly teach you figuring out where there's a broken piece of solid core wire because your lab techs were cheap.
Speaker 1:With That's an important like, listen. That that that's an important life skill, though. I mean, I remember I'm like DuPonts. I'm DuPonts. Oh my god.
Speaker 1:Alright.
Speaker 2:Alright. I I'm I'm gonna I'm gonna I'm gonna disagree with you, Brian, actually. I I I I think it's good in in school. I do think it's good to have these classes, but I'm I'm gonna sort of take a middle ground here, I guess, not disagree exactly. But, look, I'd I use an industry very little of what I learned in school.
Speaker 2:I didn't actually find most of the programs that I was in particularly beneficial. No offense to any schools I went to. But but, like, everything I know about Linux, I taught myself. Everything I know about electronics, I taught myself. Everything I know about computer microarchitecture and CPU design, I taught myself.
Speaker 2:And so I think there is space for this kind of material, but it doesn't need to target, schools. Like, because we run the risk if this is, like, oh, you know, top tier, Oh, you went to Stanford, and you learned everything, and you're great. Or you went to CMU or insert school here. But
Speaker 7:I think there's a lot
Speaker 2:of people in industry who've been in the industry, say, 10 years. Right? They they they're frustrated because they're really smart. They understand all these different the parts of the world that they're in, but they wanna understand more. And it ought to be more accessible.
Speaker 2:Totally. Like yeah? Yeah.
Speaker 7:Like the
Speaker 1:No. No. No. No. I I totally agree.
Speaker 1:And I think, like, honestly, like, Ben Eater's stuff, I think has done wonders for I I think I mean, I don't think the
Speaker 7:Yeah.
Speaker 1:The initial note that kind of what the target demographic of that is, but I feel I well, I don't feel like I definitely did that as undergrad. So I don't think I I'm the target demographic for that, but I do feel that there are, I I think he is open. I mean, that that's kind of the first part of what we're talking about where it's kind of discrete logic to get to to a CPU. But I do feel that, like I mean, the thing that I would like to get people doing it, like, I I like to do myself is getting people over the hump of making their own PCBs because I think what right right now we've got this myth that hardware is hard. And, of course, it is hard, but it's actually easier than it's ever been.
Speaker 1:I mean, if you don't correct me if I'm wrong, Done this for a career, but it feels like it's easier now than it has been.
Speaker 4:Oh, for sure. And, I mean, I would say a lot of schools have options for doing that. I mean, I definitely was on a team that made a circuit board my senior year for senior design project. So we we did FAB Aboard in the course of a semester.
Speaker 1:And and
Speaker 2:Well, like, one one of
Speaker 10:the big problems you get with that, though, is is that a lot of the open tooling really does not measure up whatsoever against the commercial tooling just in terms of usability.
Speaker 5:So so so let's let's turn this apart, Matt.
Speaker 10:PCB design is KiCad. Like, good grief. Like, it it oh, it's a lot better than it used to be.
Speaker 5:Now we've talked about this a bunch. So what are the what are the pieces that that are missing here? Because, you know, I'm I'm I'm passionately familiar with, like, OrCAD and that tool suite. Is that is that the big one? Or and I I've also seen that KiCAD is made big strides.
Speaker 5:Are there other big pieces, or what's missing?
Speaker 10:So, I mean, the things, like, I I'm more of an Altium guy. Altium makes Altium's got a lot of very nice wizards and, like, the ability to to, like, write up a whole table of every pin on your IC in Excel and very easily graphically import that and arrange the symbols. And so, like, you know, you can I mean
Speaker 7:Yeah?
Speaker 1:My reading said that it it is, like, there's a lot of polish. Mean, on the, you live in in I get a little more CAD every day. The Sure. You can.
Speaker 4:Yeah. I mean and I've used Altium before too. I I mean, I find, I mean, you know, there there is a a definite gap between keypad and and the Pro Tools for sure. And but, like, that it's not an insurmountable gap, especially on the schematic side. And so I think, you know, that that's an area where we, you know, as a a team of people who do this and and are open source software fans, you know, we can help do things in that space and try to help catch it up.
Speaker 2:This is
Speaker 1:where Matt is gonna remind me that I Matt Keter is gonna remind me that we need to actually be, pay we we need to be helping out OrCAD publicly or the KiCad publicly and and, as a company. I think you're right, Daniel. The because it I mean, not that the arguments you're making are the arguments that have always been made against open source, and they've been made against open source databases and open source operating systems. And the thing about open source is that it's a ratchet. It, like, it will it the and it can actually catch up proprietary software in the limit, which I think is John, I think that's gonna help address the gap that you were highlighting of, you know, the number of double e's.
Speaker 1:I'm not sure, like, what by the way, that thing that you were quoting, like, what even was that? Was that like, I don't think the acties were labeled. Was
Speaker 2:that There was some it was some research that I think, you know, Roger Cadore, over at Intel, you know, picked up on and posted or something. Right? But, I mean yeah. It the the chart was a little, funky. Right.
Speaker 2:Use a technical term. Right? But but it still made the point. I it was
Speaker 5:the It it made a point, but I don't I I I was wondering how valid that was. I mean, the the point they were saying was that the proportion that that there were more computer science grads than EE's, proportionally. But I also wonder computer science has often tracked, much more closely, like with the stock market in terms of enrollment. No.
Speaker 7:No. It's
Speaker 1:sad. It's true. I know. I know. It's true.
Speaker 1:It's true. You're right. I mean, it's it's sad, but it's true. It's sad because it's true. That's right.
Speaker 5:But, so I also wonder if we're if we're gonna see a reversal of that ratio. But I wonder also if, like, the EE numbers have been pretty stable over the years or or even, you know, up, and and putting it next to some other discipline, which is related, but also has these externalities.
Speaker 1:Matt, I saw you jumping in here. I mean, you're obviously, your background is as a double e, but you're very much I mean, you're right at the software hardware interface. What's your perspective on this?
Speaker 11:Yeah. I mean, it's been interesting. Can you hear me? Am I coming through?
Speaker 1:Yep. We can hear you. Yeah.
Speaker 11:So thinking about the evolution of tools, like, when I was designing boards probably 10 or 12 years ago, I was using Eagle before that got bought by AutoCAD. And thinking back to, like, that version of Eagle versus the boards I'm designing in KiCAD, you know, within the past 6 months, KiCAD has definitely caught up and surpassed Eagle from back then. So, like, the tools have caught up a great deal, and the the open source tools these days are certainly usable. Like, I've designed 3 or 4 boards at oxide on them, and other people have designed a solid dozen small breakout boards. Altium I I haven't used OrCAD as much, but Altium is definitely more polished from past experience.
Speaker 1:But this ability to spin quickly on these boards and and, Matt, can you speak with some of the little boards that you've done? Because we've done a we I this has been, I think, a very important tool in our tool belt.
Speaker 11:Yeah. I mean, I've been doing a lot of adapters and breakout boards. So, like, one of the original boards that was designed by, I think, Cliff was called the gimletlet, which Brian mentioned, which is a breakout board for the main service processor. So an STM 3287, like, big, big microcontroller. And this board has a ton of different plugs and ports around the edges.
Speaker 11:And so I think now if you stick all of the possible adapter boards onto a gimlet, then you'll have, like, 6 different boards sticking out of it like a Voltron
Speaker 1:interaction here. Oh, awesome.
Speaker 11:So so I made, like, I made a network card, which, breaks out an external PHY so you plug Ethernet into it. Nathaniel already mentioned the DIMMletlet, which lets you plug DIMMs, into it. There is an adapter which lets you plug a different adapter for our root of trust into the gimlet. So that lets you actually simulate the full root of trust system that we're gonna have on the servers, like the LPC 55 plus the STM 3287. What adapters am I forgetting?
Speaker 11:I think there's probably 1
Speaker 4:or 2 more. We did the spy mux, so which has the s s p's mux, for the host processor boot flash.
Speaker 1:And we can able to spin super quickly on these. So you go to kind of from, like, ideation to getting it out the door. It it happens quickly. Yeah. Would in a day for 1 of the one of the boards that Cliff did.
Speaker 1:It was, like, 8 hours or something before, you know, he, like, started it.
Speaker 7:And he
Speaker 1:that afternoon, he's like, okay. I'll be here from the fab in a week.
Speaker 11:Yeah. And this is also kind of the other side of the the tachyon discussion earlier where, like, the tachyon materials are extremely exotic, and you have to think a lot about them. But for low end boards, and by low end, I even mean, like, 6 or 8 layer boards, like, that's kind of an off the shelf product now where you can design it. You don't have to think too much about it. There are dozens of vendors that'll give them to you.
Speaker 11:So that bar has definitely come down a long ways. Like, even in the last 10 years, I've seen it.
Speaker 4:And and I would say too, like, I I I did the, assembly of both the dimlet and the, spy mugs here at the house. And, you know, with the toaster oven reflow, I mean, the spymocks has a, point 8 millimeter BGA on it. And, like, that's totally approachable with, like, an investment of a couple $100 really to have a toaster.
Speaker 1:Which is amazing. Tim, you Tim, you got you got your hand up, and we just we have hit our speaker capacity because Twitter Spaces is disappointing. So if you've requested it, we haven't we'll we'll get you on as soon as we can have, as soon as we can. So it's not it's not you. It's Twitter Spaces.
Speaker 1:Timon, you're up. How do you pronounce
Speaker 4:your name?
Speaker 3:Timon. Like, t like, tea the drink and then Pokemon. That's right. So, two points. One on the education side.
Speaker 3:So, my I went to university. I did not finish my degree, but it was I mean, let's see. Compared to most of the year, maybe a bit more recent. I'm 29. I did it when was that?
Speaker 3:I don't know. I think 7 years ago. And I did e, and I was working before I started. So, like, I was working my job, in, like, media system design, started doing some PCB stuff. I was with software, and I was like, okay.
Speaker 3:Maybe that's I should get a degree and start doing that, during the job. And it was so frustrating because the whole experience was very centered around mathematics. And you were basically solving formulas for 4 years before you got to do anything interesting. And that is so far removed from my opinion, at least. And I think probably many these would agree.
Speaker 3:And the job is not so heavy on mathematics most of the time. It depends on the area you're working in. But a lot of the jobs, you're not gonna spend all your day solving formulas. That's really not how this looks like. And I think it's really hard to get the right people interested in that.
Speaker 3:If you approach education from, you know, a purely mathematic standpoint, there's so much more to electronics and everything, you know but it I mean, when I say only mathematics, really every course, physics, electronics. First 2 years of electronics was really just solving math formulas. I mean, you know, you never you never build a PCB in the whole bachelor degree.
Speaker 7:Right.
Speaker 2:Like, you
Speaker 3:do that, then there is, it would kill us. And we were
Speaker 2:we were doing
Speaker 3:that whole, like, sorry, the whole breadboard thing that you mentioned, did exactly that. It was a seventies breadboard. I used chips from the seventies, connect them together to make, you know, logical circuits, you know,
Speaker 1:That's bad. Yeah. And well, I I mean, I think to ideally, the you know, with more open source tooling, 1, if you can get that deeper into the education system. And then 2 no. Actually, I'm just gonna mute either 2 minutes in your mind that the oh, background noise there.
Speaker 1:The and then but also, John, to your point, like, maybe this is content that really should be aimed at interested professionals. And, hey, your education does not end at your undergraduate. And if you and if you start
Speaker 2:if you start with, like, you know, Maxwell's equations and and, you know, Horowitz and Hill as your starting point, you you know, you you people people will run away scream. It's like it's like if you were to start computer science with, you know, Knuth straight away. Right? Like, the other computer programming. It's a great set of books, but it may not be the thing I'd recommend you start reading if you're interested.
Speaker 2:Right? Like, there's something
Speaker 10:Oh, come on. Come on, John.
Speaker 1:Yep. Don't we all do
Speaker 10:a triple integral in the morning before breakfast?
Speaker 2:Right. Yeah. Yeah. Our brain uses it. You know?
Speaker 2:Yeah. He doesn't. Right? Yeah. Exactly.
Speaker 2:Yeah. But, you know, that that's what scares people away. Like, you know
Speaker 6:Well well, that's that's that's why there are these walls. Right? If if you don't have good walls, people just go crazy. There's no no focus. And there a lot of people, not not including any present company, aren't interested in leaving the room and learning what else is out there.
Speaker 1:Yeah. No. And I think getting some curiosity and some understanding about how these different layers of the stack work is very important to kind of building that that empathy and that that mutual trust that, John, you were complaining about the lack of, certainly. Yeah.
Speaker 12:I I I I
Speaker 6:see a similar thing with networking. Right? People will work all day on their computer writing code and then absolutely no clue how a network works.
Speaker 2:I think that's the thing. I think a little bit of knowledge what what I guess what I'm getting at is Yeah. A little bit of knowledge about the other parts of the stack. Right? You don't have to be an expert in every single other piece, but an understanding of what's above and below you.
Speaker 1:But and what and I think there's also
Speaker 10:a very important difference between not knowing and not wanting to know because I I feel like there's a bigger problem on the second set. Right? I mean, the
Speaker 2:answer Exactly
Speaker 10:don't know anything, but, like, you stand up in front of a whiteboard for half an hour and suddenly they understand phase locked loops. Right? And and, like, they all wanted to know how that works, just they never had found out, versus, you know, versus thinking that, oh, networks are scary and, you know, we just hurl packets into the abyss and, you know We teach the way in Korea. We just take
Speaker 1:a couple of them on
Speaker 7:the way. And
Speaker 6:And what what one of the reasons that people are are at least aware of the network is because it doesn't work all the time. Whereas with a lot of hardware, it just kinda works all the time and yeah. We we now interesting.
Speaker 10:We we now introduce our new version of TCP Reno, known as TCP YOLO, in which we just throw data across the network and increment the counter and just sort of pray
Speaker 2:it works. Well, so so you guys mentioned mentioned networking and and you mentioned building breakout boards. And I was just that was my my brain is weird. It was thinking about, you know, what is it? Type 49 or 43 or something.
Speaker 2:You know, the the the PHY information that goes over this RMII interface. Right? You have the separate bus that you enumerate the kind of PHY that's connected. And
Speaker 10:And and and then there's then there's the fun little game of, do we cut the orange wire or the blue wire to not go to that 100 meg? You know? Like, you
Speaker 1:Yeah. And until you and until you
Speaker 2:know this stuff, you have no idea. Right? You're like, oh, wait. There's a negotiation that goes that your your your your Mac is talking to your Fi and figuring out of course it is. Right?
Speaker 2:They must be doing that somewhere. How does that work?
Speaker 1:I it's And and and we thought it could
Speaker 11:be started on Fis.
Speaker 1:Yeah. Exactly. I was gonna Matt, I think Matt may have passed out because Matt is definitely in the in the absolute trenches negotiating with Fi. I feel like, Matt, you are negotiating with Fi's right now.
Speaker 11:Yes. I'm personally negotiating with Fi's. Right. All the all of these Fi's have a datasheet, and one of the lines in the bring up procedure is apply the patch. And that's the entire ex explanation.
Speaker 11:So you have to dig through the SDK and find the patch, which is an undocumented blob, and then throw that into the PHY and cross your fingers and hope it comes up.
Speaker 4:Which is probably fixing a hardware screw up to get back onto the hardware software thing. Right?
Speaker 11:It's totally fault.
Speaker 6:Well and there's so much of the world that's that's based on these horrible clues that that become fixed over time in into the world architecture.
Speaker 10:Now now are are we talking 100 meg, or are we talking 100 gig? Just curious.
Speaker 13:Funny you should mention that because, you know, we talk about these breakout boards, and one that I currently have in design is actually QSFP 28, DD cages to gimlet led, specifically just to bring out the low speed signals because we need to look at those and figure out how you program these transceivers for 100 gig and how do you interact with them. And, you know, I think this we keep talking about these as as a a bring up activity we do, and then a lot of companies do it. But it's sort of this interesting piece of bringing the software people to the hardware at some of these devices that we're talking about, the cost of the hardware is really high, and the complexity of the hardware is really high. And a lot of what we're doing with these bring up boards is actually bringing down a lot of the complexity, making it inexpensive so that everybody can have a one. They don't need the full functionality.
Speaker 13:They only need to deal with the part of the problem that they're trying to solve, and and letting people work in parallel. But it it's sort of if you try to take a a broad systems approach of, say, a network switch, you're never gonna make a whole lot of progress because there there are so many different layers that you need to deal with. There the the total sum of knowledge required is pretty darn large. But the way these systems get designed and deployed in the field, they sort of assume that it all works together. And it's it's hard to pair that down into smaller chunks that you can learn on and experiment on.
Speaker 13:And that's where we've been, you know, doing a lot of that, of developing smaller boards that we can turn more quickly, that we can do more development experimental work on instead of having single big test fixtures that, you know, only 1 person can use at a time.
Speaker 1:And I feel it, Rick, that that has not only that we've moved faster as a result, but it's also fostered a bunch more understanding. Certainly, I feel like I understand many more components when I can kinda get them in front of me and understand, you know, this aspect of it.
Speaker 5:And Brian, on the topic of, you know, tighter integration of software and hardware teams, which is where we started, it seems like a bunch of these fixtures are easy to build, but sort of hard to
Speaker 1:conceive of the question for them. That is to say, if you're on some isolated software team, you might not know
Speaker 5:and it's only the the integration of these teams that let us even, you know, get these questions on the table. Yes.
Speaker 3:Yep.
Speaker 2:Adam, Adam, that's a beautiful point. I'm I'm I'm gonna throw something out there there. You know, in various lifetimes, I've worked on, you know, CPUs. Right? And, one one of the interesting challenges is, particularly in that world, you have very historically, very rigorously separated hardware and software teams.
Speaker 2:Right? Like, if you look at a well, I won't pick on examples. Brian probably hopes I would.
Speaker 1:But Come on.
Speaker 2:How about traditional CPU vendor comes out.
Speaker 1:It's too slow. First hand. I don't know I don't know who you're talking about.
Speaker 4:Rounded in an example. Dun. Dun. Dun. Dun.
Speaker 4:Dun. Dun.
Speaker 2:Anyway, no. You know, a small example. Right? They're they're
Speaker 1:Different side.
Speaker 2:Yes. They tend to be very, you know, separated, hardware, software organizations. And you hear from people who, you know, are in are in software organizations who are really, really trying to make friends and and and break through and talk to the other, and it's very, very difficult. And I think the world's changing, but, you know, the point is people don't know how hard or easy something is for the other side. Because there's some stuff that's really, really easy to do in hardware, right, in in logic.
Speaker 2:And there's some stuff it's like, what? That's not possible. You know, that that's like, even with the world's most amount of microcode, I can't make that work, you know? And and people don't and whereas, if you if if this knowledge was there, then a lot of these conversations, a, wouldn't happen in the 1st place, because because the design decisions would go to a different way. It's like, oh, okay.
Speaker 2:Well, I'll do this in the software, because I know that that's where that has to happen. But the hardware could help me here and make it 10 times faster. You know? People just don't know what can go where. That that that knowledge is missing.
Speaker 1:I totally agree with you, John. And I I think that the and we've seen this too. I feel it, Oxide, where there have been things where on both sides, where, like, you know, Nathaniel's able to dimwit, which is delightful and quick. But, Nathaniel, I feel like there's also been some things where, you know, harbor folks are like, hey. Could we possibly do this in software?
Speaker 1:Like, yeah. This is like Yes.
Speaker 7:Well And
Speaker 4:yeah. And I I think some of it you gotta remember too is it's not just the knowledge, I think, that isn't exposed. But, like, when the teams are separated, I think a lot of times they lack empathy for each other and and the needs. And so and then when you then you get into a rut of blaming each other for problems. And then who's likely to come to the table and, you know, come up with a cool solution where, you know, it's like, well, that team never gave me any debug tools, so I'm not gonna, you know and and, like, people do this, like, even unconsciously.
Speaker 4:Right? It's not even, like, you're actively making that choice. You just continue to reinforce this barrier between the teams, and it becomes very unhealthy. And and then you don't even have the conversation that would allow that knowledge to surface.
Speaker 1:Totally. And I think it's very important to kinda walk into one another's shoes to learn, John, what you're talking about. Like, what is some things are really easy and some things are really hard. And if we don't know what's what, we actually don't know what to even conceive of asking for. Peter, I know you had your hand up a while ago.
Speaker 1:I wanna make sure we got to you. So sorry about that.
Speaker 12:No. No. That's fine. I've been enjoying the conversation. This is a great space.
Speaker 12:What I was gonna say is, I mean, you can see in my profile pic and a bunch of others, some of us have gray hairs. Right? And and what that represents is the fact that when we were in undergrad, you could pretty much identify every single chip on your motherboard. You know, you you could pretty much follow everything that was going on. How many of us remember the Hays, you know, modem command set?
Speaker 12:Like, how many of us had a crimping tool, like a punch down tool, wire strippers. Right? Like like everybody at that generation was a hardware and a software geek. We had to be. Right?
Speaker 12:And what's happened since then, like, if you take a look at somebody learning to develop these days, like, let's say a mobile developer. Right? You don't even develop on the phone. You develop in an emulator, a simulator off the box. And eventually, after playing a whole bunch of rounds of please mother may I, you may or may not get your application deployed to a phone.
Speaker 12:And if you take a look at a typical, cloud developer these days, they're gonna be developing in Docker on their laptop. And after a bunch of, tortures of the dam, they're gonna put it up into some sort of development environment for testing. Then somebody else is gonna take it to staging, and eventually somebody else is gonna take it to production. And then who operates it and maintains it is a totally separate team than the people that originally developed the app. And so now what's happening is you have this highly specialized situation where you are being insulated from the environment you're running on.
Speaker 12:God help you if you're running in a JVM, you're literally not allowed to understand what the hardware is doing underneath that. Like you're you're insulated. You're coddled. You're in this comfy JVM that's not supposed to understand the hardware. That's not your job.
Speaker 12:Right? And so how can you, you know, do very low level co coding, integration? You can't. Like, that's not your job. Now maybe people doing Rust or c plus plus, they have more of an opportunity to take advantage of that hardware.
Speaker 12:They can actually see the bare metal. But a lot of people these days are running in interpreted or, you know, byte code kind of stuff. And and and so those people are very far removed from the hardware. And I just think that that's one of the things we have to pay attention to is that in order to break down these barriers and silos, you know, somebody may have to have root access on a system somewhere.
Speaker 4:Agreed. But but but I think
Speaker 6:but I think, it's important for making things work that your software doesn't try to pierce the abstractions. On the other hand, it's important that people try to pierce the abstractions to learn more.
Speaker 2:I I think I think you wanna make it so that people who want to learn I think I think if I could if I could have a PSA moment, it would be there's a whole world of awesomeness out there, in both hardware and software and in the middle. And here's here's where you can go to find it out. I think that's not defined well enough. Some good examples, some ways that you can play with projects that are inexpensive. I'm talking, you know, at most tens of dollars or or, you know, inexpensive, we can, hobby money for for almost anyone on on, you know, a professional salary, let's say, to to go and fill in gaps in their knowledge.
Speaker 2:But I don't think we're saying, and in let's tear down the walls in production. Right? I think it I think it's fine to have walls in production, but you wanna understand what's above and below you, to be a better engineer.
Speaker 1:Totally. And I, John, I love something you're saying, and that's something that I really believe too about, like, when you say there's a lot of awesomeness out there. I do feel that, like, something that we've kind of lost in some of the quotidian details of software engineering is the majesty. Like, the fact that that we should be so blown away that these systems boot so reliably. It's the miracle of boot, you know.
Speaker 1:And we we should I mean, it's it's amazing that this stuff works. And we are we are truly standing on the shoulders of all of these generations that have come before us. And I feel like we've lost some of that majesty. I feel like the you know, when you are and just as you say, getting, you know, getting one of these dev boards and being able to put your own software on it should feel like watching, you know, a sunrise come up over a mountain top. There should be some sense of, like, splendor and majesty there.
Speaker 1:And, I mean, it's awesome in in the most literal way. Right? And I feel like we do kinda lose that.
Speaker 7:But but we we
Speaker 2:to to get there, sorry. Just to throw in there, I think, to to to get to that point, one other thing we have to do is we have to stop beating each other up. Right? Because because Yeah. That that sort of unhealthy mentality, that's the thing that really motivated me to, you know, post that in particular.
Speaker 2:It it's when you go on I'll pick on LKML. Right? You know There we go. Oh, the hardware people did this and the hardware people did that and they suck and, and, you know, hardware is awful and, you know, blah, blah, blah, blah, blah. Well, yeah, but they were operating under a very different set of constraints.
Speaker 2:They had no budget. They had to do it in 10 minutes, and you're sitting there, you know, in a in a probably a high paid tech job at some tech company, right, with your opinion. And, it's unfair. And, I think I you know?
Speaker 6:But but this is part of the dark dark side of software that too many hardware vendors push off the upfront cost into later term software, which which they then push off to the customers, then it's just a mess.
Speaker 1:And the next thing you know, Matt is loading 80 51 firmware blobs that we can't that we have got no visibility to make the hardware actually work.
Speaker 6:So so there's a there's definitely a dark side to all this.
Speaker 1:Sudarth, you've had you had your hand up for a while. I wonder if you got in here.
Speaker 7:Yeah. Thanks. So I actually have a lot of visibility to a lot of the points that have been made maybe a while back, but, especially with regards to, you're saying are the number of double Ds decreasing? That's actually true. So, we've had this at my university, like surveys, and this is like a major National Science Foundation and DARPA and basically every funding agency out there and every company out there, right, they're all worried, that there are just not that many people getting into Double E.
Speaker 7:It's actually gone down over the years, right? And not just Double E. If you start looking at, you know, microelectronics or VLSR, anyone who can actually anyone who knows what a standard cell looks like by the end of their undergrad, right, that number is decreasing dramatically. And it's decreasing to the extent that let's pick a top 3 CS university. Up until a couple of years ago, they actually just had 12 people out of 500, graduate having taken any VLSI course from their undergrad.
Speaker 7:Yeah. Wow. So that's Yeah.
Speaker 1:That's sad.
Speaker 2:Yeah.
Speaker 1:That's a problem.
Speaker 7:So so so that's the kinda thing that's going on. And it is actually, in part because despite how good, you know, the tooling is now, you still have to compare it to what the software tooling is. Right? So I can file up a Jupyter notebook and do everything there versus if I want to actually get something running on an FPGA. Yes.
Speaker 7:You've got and everything, which is really difficult to get undergrads to actually go through without alternatively, you're using Quartus or Xilinx, and, I mean, the compiled times and
Speaker 1:those things.
Speaker 7:Just yeah. Exactly. The synthesis times. Just the plus, I mean, having to work in HDL, compare that to having to work with Python, it's just it just drives people away, honestly.
Speaker 2:But but, but I'll push back on that and say, you you know, in in school, I I didn't learn any VLSI. And, you know, sort of, you know, 6 6 transistors, 6 transistor SRAM, memory cells. Right? For example. Right?
Speaker 7:Mhmm.
Speaker 2:I didn't learn any of that in school. I I I've I've learned very standard cells because I'm interested, and I read about it. And the information is there for someone who's interested to go and, you know, arbitrarily study anything. And so, I think a lot of this can also be fixed by just making it, right now, it's almost socially acceptable to hate the other guy rather than to say, Oh, I don't understand your space. I sh I should feel bad for not understanding your space, not hating on you just because you're in a different you know, you're in hardware, and I'm a software guy or whatever or girl.
Speaker 7:So, I understand what you're saying. I I'm not quite sure it's exactly just hate. Although, I will agree that a lot of, you need to understand how to do security right kind of threads were written by people who've never written even a simple, you know, HDL thing in their life. But, I think a lot of it is just the friction of getting something working and the ability to iterate. I mean, you can do so much with a 100 lines of Python, which it's really hard to do, and this is coming from someone who does ASIC design.
Speaker 7:Right? So so this is near and dear to me, and I want life to be a lot easier on that end.
Speaker 12:Yes.
Speaker 7:But at the moment, for a lot of let's say let's say for a lot of universities, it isn't. Right? So purely from that limited perspective of how do we get students who are more, interested on this side of things. It is getting more and more difficult. And on the other hand, honestly, ML is cannibalizing everything.
Speaker 7:Like, the I mean, even if you look at a lot of the computer architecture kind of groups, the Yes. Number of students coming in saying, yes. We wanna do computer architect. We wanna do hardware accelerators for ML, but mostly from, like, the ML side. Right?
Speaker 7:So I feel that's sort of cannibalizing a lot of that talent as well. And it should, like I mean, not it should. Not it's not a it's not a prescription, but, I mean, just the amount of money in that is so much.
Speaker 2:Yeah. It's a market.
Speaker 7:Yeah. Yeah. Yeah. It's market driving it that way.
Speaker 2:So so to ask Mhmm. Question related to that. So because I I've often thought, you know, that you put a notebook experience with Python Python is fabulous, particularly in the machine learning space. Right? You want to figure something out.
Speaker 2:Oh, you worked through an example, click this line, compile it. There is no reason why we couldn't have that exact experience for open source EDA tooling.
Speaker 7:There there is similar experience. I know Metro's got, like, some of the basic stuff for that funded through some of the, you know, Google money. But I am not sure it is, as of now, stable enough for me to be able to use it in a classroom of a 100 people. Right? So it is still something which requires us to work really hard just for us to use as a hobbyist tool.
Speaker 7:Right? And that's after understanding how different components fit together. I mean, one thing, and, I think someone had sort of mentioned this earlier on, but this is so I don't know if you've all seen this, but a lot of students nowadays, because their experience of computers is so different from the experience you or I might have had, their understanding of file system is very different from your understanding of file system. Yes. I saw that.
Speaker 1:Is this gonna track people who call it folders? Can we just, like I don't I'm not trying to add anybody, but, like
Speaker 2:They find it more.
Speaker 1:No. There's, like, a divide between, like, directory people and folder people. I mean, I here I am, like, fostering the mutual content.
Speaker 2:They don't use folders anymore. It's just the thing. What they they they only used, Google Drive. Right? They just search.
Speaker 2:They've never had the idea of any any hierarchy.
Speaker 1:Wait. Exactly.
Speaker 2:Are you still very common? Yes. Oh, yeah. Yeah.
Speaker 7:People are
Speaker 2:people are starting undergrad now who have only ever used, like, a
Speaker 7:a a a
Speaker 2:drive like oh, yes.
Speaker 6:That's the way my email is now.
Speaker 7:Right. So so why are
Speaker 1:we using
Speaker 12:it soon? Because, you know
Speaker 1:Sorry. It's not funny. It's deadly serious.
Speaker 7:I mean, we we've made, computers so easy to use. Right? And it's a good thing. Yeah.
Speaker 8:It's, like, yes.
Speaker 1:Right. Right. Right.
Speaker 7:It's reduced the barrier of entry. But
Speaker 6:Yeah.
Speaker 7:At the same time, so many people who've then used computers haven't had that same sort of Yeah. Well, the and then experience. Right?
Speaker 1:This is the challenge that and where we where things are so mature that that everything is kinda taken for granted. And then by the time someone shows up and they're 18, they actually don't know that, like, oh, by the way, now we need to take down all of this stuff. And you That's right. I mean, I was, you know, having a conversation with my 15 year old about, you know, he's, actually, in this case, I was kicking his ass at MLB the Show. Have you played that MLB the Show, by the way.
Speaker 1:Oh, yeah.
Speaker 5:I'm gonna come over and play with
Speaker 6:you guys later.
Speaker 1:Oh, that's awesome. We it's and just like it is the these console games. I mean, it's not at all a surprise that this thing is, like, throwing off as much heat as it is into my living room considering I mean, it is amazing how much is happening from a graphics perspective, logic perspective. And he's like I mean, honestly, he's like the hardest thing about this is loading everyone's stats into the game. I'm like, that is, like, so far and away the easiest problem that is being solved by this thing.
Speaker 1:Like, all and I certainly you're kinda walking him through just, like, all of the problems that need to be solved and how quickly they need to these all have, like, software time constraints. And I do feel like I've done my own children a disservice by presenting them a world that is broadly functional. And the world I grew up in was broadly not functional. We were building this stuff at, you know, when you had a personal computer in the eighties. Or sorry, John.
Speaker 1:It's not now time to to to sing God Save the Queen as we talk about the BBC micro versus the I I I I,
Speaker 2:My country too.
Speaker 1:There you go. Right. There you go. Right. Right.
Speaker 2:Always sing. They do they do God Save the Queen. I do My Country.
Speaker 1:Very good.
Speaker 2:Bad immigrant.
Speaker 1:But the I mean, you were for the things were so much more primitive that you were you were for and I think that it's like we need to encourage people to get curious across those those boundaries. I'm much I'm optimistic that it's possible. You know what we need, John? This is what we need. Maybe need.
Speaker 1:Content for the youngs that breaks down these abstractions.
Speaker 7:This is
Speaker 1:what we need.
Speaker 2:I I think I think I I've threatened to to do blogs on on this sort of stuff. And I I I I no. I think I should actually do it. Right? And it should be YouTube.
Speaker 2:And, I've talked to a bunch of people about this. So we we yes. It needs
Speaker 1:to be TikTok though, doesn't it? It's actually a it needs to be TikTok.
Speaker 2:It should be a dance. Yes. Yes. And and, you know, absolutely. Right?
Speaker 2:But but there's also, like I I look at the next generation. I look at, like, you know, Alisa, who's been doing all the work on, you know, reverse engineering the GPU and the m one and, you know, the a lot of the work on the the Linux port there. And I I I believe she might have graduated undergrad. I'm not no. I think she's still in on in undergrad.
Speaker 2:Right? Just just phenomenally talented person who is coming up now and has just surpassed all of these issues of of the modern world is so different and figured it all out and knows these systems way more than I'll ever know them. Right? Like, so it's totally possible to be coming out
Speaker 7:of town.
Speaker 3:Totally possible. Yeah.
Speaker 8:Right. Figuring it out. Yep.
Speaker 2:It's just it's just increasingly harder with time because you have to be an absolute genius to, you know, figure that out.
Speaker 7:Right. So the clear. I'm I'm not saying that, you know, it's all doom and gloom. I mean, Sam Zeloff. I'm not I'm not quite sure how to pronounce his name, but dude did a whole fad in his
Speaker 1:Goddamn. Right? So Have you seen this, Adam?
Speaker 2:I've been banned. I've been banned from doing that in our history.
Speaker 1:My god. This guy is off the hook.
Speaker 7:Right? So he's just graduated undergrad. So Yeah.
Speaker 1:So so, Sarath, you can describe what he has done because it is bonkers.
Speaker 7:Yeah. I mean, he he's literally got self aligned lithography working in his garage. Like, he he can do, like, a full I'm not sure whether he can do CMOS or just NMOS, but he can do, like, a 100 transistor chip. Or maybe now even a 1,000. I'm not quite sure.
Speaker 7:But, yeah, he can do it in his garage. It's it's honestly amazing. Right?
Speaker 1:It is crazy. And I mean, it's amazing.
Speaker 7:A few microns, which is where we were as a civilization in the early nineties, just even that far back.
Speaker 14:Right.
Speaker 7:Like, it's
Speaker 1:I mean Okay. Is it so but this yeah. This gets to another interesting issue because I do feel like you it is now I know, definitely, at some point, Oxide will buy our own pick and place machines because there is too much emotional energy pointing us in that direction to not if any VCs are listening to us.
Speaker 5:I'm just I'm relieved we're not building our own.
Speaker 7:Well, have you
Speaker 1:Tell me what you're saying. Worries.
Speaker 12:Have you seen the open
Speaker 4:have you seen the open PNP by Lumen or Opulo, I think? Yeah. Opulo. It's a guy a guy yeah. A guy who left, the guy left, the 3 d printing company, Formlabs, and is building an open source pick and place, like, for your house, and it's, like, $1500.
Speaker 4:So, Adam, there's
Speaker 1:just a chance to answer your question. So we are we are building our own pick and place machine,
Speaker 12:apparently. Right?
Speaker 4:I have one in my house. I have one I'm building.
Speaker 1:So I
Speaker 11:think Dan has one as well.
Speaker 1:So in that, I I because I I actually wanted to get because, I mean, you I was coming out of Formlabs and seeing a bunch of what people were doing at 3rd 3rd printing. It does feel like the, you know, Moore's law I mean, to what degree did Moore's law require Moore's law? Right? To what degree did certainly, we could not build a the the the the parts that we build today, we could not build on a computer from 40 years ago. We actually needed to have much more advanced computation.
Speaker 1:And to what degree is, like, getting 3 d printing in more hands, getting these technology in more hands, going to allow I mean, I don't know to what degree, sir. I think you know this to what degree he relied on being able to 3 d print the the the for the the NMOS experiments, the same experiments?
Speaker 7:Yeah. I don't know that much. I think he mostly got stuff off of eBay and refurbished and sorta fixed things up. Like, it's mostly photolithography. Right?
Speaker 7:It's not, through
Speaker 2:Using using projectors. It's it's fab he's got a bunch of YouTube videos that are fabulous, like how he built the he just, like, just repurposes an old projector. Thank you, God. What? And that works?
Speaker 2:Okay.
Speaker 7:Yeah. And I I I do wanna point out one thing that I I don't remember her name, but there was this maker. She's in somewhere in Orange County, and she did one of the first iterations of this, and he credits her for, you know, doing some of the first few iterations of homebrew PCBs and, ICs. So I I do wanna give credit by credit dispute.
Speaker 2:You mean, Jerry Ellsworth doing the,
Speaker 7:I I I think so. I don't remember her name. Sorry.
Speaker 12:She's not was it?
Speaker 2:5. Okay. Yeah.
Speaker 3:Yeah. That was true.
Speaker 2:She she did transistor in her oven. She she's in Oregon. Right? She yeah. She did she did transistors in her oven at home because why wouldn't you?
Speaker 1:And, Matt, I know. Sorry. You that you were trying to get in here as well.
Speaker 11:Oh, yeah. I was just gonna say that certainly the the analogy of computers that let us build more computers definitely applies to the 3 d printing. Like, the amount of 3 d printing that was done for in house prototyping at my previous gig using the 3 d printers that existed in house did a huge amount to speed up engineering development. Like, there's definitely a positive feedback loop
Speaker 1:there. And I feel like we've used 3 d printing quite a bit. I mean, Nathaniel, certainly, a bunch of the stuff that you've done has been 3 d printed. I mean, I know we've we've been using it quite a bit at oxide.
Speaker 4:Yep. Absolutely. I mean, a lot of the mechanical stuff. I mean, we have our, our m.2 holders on all our rev b gimlets are all 3 d printed because we don't have our injection molded pieces yet.
Speaker 1:And our air shrouds were all in the rev a. We're all Yep. 3 printed. Right? Yep.
Speaker 1:Yep. Pretty fancy 3 d printing.
Speaker 4:Yeah. Those are PolyJet.
Speaker 1:Yeah. I don't wanna know how much they cost. I know they're very expensive. They're they're gorgeous. They're they they look I mean, they look better than the injected molding ones, I think.
Speaker 1:The I mean, they're they're amazing. So, Bob, I know you wanted to get in here, and then I I I know you got other hands dropped. But, Bob, do you wanna hop in here?
Speaker 9:Yeah. Hi, Brian. How are you? Can you guys hear me okay?
Speaker 1:Yep. We can hear you.
Speaker 9:Awesome. Because, you know, space is kind of flaky sometimes. First of all, I just I just wanted to give a shout out to John Masters. When when I was, running Linux engineering at a very large Wall Street Bank, about 10 years ago. He and he was at Red Hat.
Speaker 9:He was tremendously helpful with us, working with, like, software vendors. Well, hardware software vendors that had third party kernel modules that just wouldn't play nice and wouldn't, you know, deal with with, like, rel kernel upgrades and stuff. And and John just did some amazing work to to help us basically beat up these vendors and and get through those problems.
Speaker 5:So Well,
Speaker 1:I I was just gonna ask, is the statute of limitations, whatever crimes that John committed on your behalf against your software vendors? Has that I I mean, there's no statute of limitations for Burberry. You know? It's like, you gotta be careful about what we implicate, but, that I'm sure that was
Speaker 2:a it always great to
Speaker 1:have someone who's kinda helping you out, Bob, right from the inside. It was like a it it was certainly Adam and I liked to do that when we were at Sun, but I'm sure it was very frustrating to deal with vendors that were giving you bad kernel modules.
Speaker 9:Yeah. And I mean, all these these vendors were Red Hat partners. So, you know, I'm I'm assuming any crimes that he committed were were dealt with by some sort of, what do you call that, internal arbitration?
Speaker 1:Yeah. The the well, Red Hat Legal has a slush fund to deal with, Brian. The job he committed in the in the act of doing his job.
Speaker 2:I I was always a good boy. What do you mean?
Speaker 1:That's right. Right. But don't make me call the bad boy. I will call him if I have to. Like, don't make me do that though.
Speaker 1:I don't wanna do that. I wanna keep this between you and me.
Speaker 2:But Bob is clearly a happy customer. Right? That's what I used to that's what I used to enjoy is is is is there you go. Hardware software codesign. Right?
Speaker 2:I don't know Well,
Speaker 9:well, John John, the funny thing is I'm not a Red Hat customer anymore. I'm actually working at Red Hat for about a month now.
Speaker 1:Beautiful. Beautiful.
Speaker 9:Yeah. And and I'm loving it. Now the whole, premise of the tweet, poor relationship between hardware and software teams. You know, and, I mean, that has it's it's the silos. Right?
Speaker 9:And it's it's not just, I mean, hardware and software. It's it's also within all the software engineering disciplines that that organizations have all these silos, and and it's just a tremendous barrier to getting, you know, to to the cloud native panacea because if you wanna go cloud native, if you want to do cloud, right, all your engineering disciplines gotta be working together. They can't work in their own little islands. But but the the hardware and software part of it is is I mean, Brian, that is what oxide is all about. Right?
Speaker 9:Is is, you know, I mean, right now, we've got, oh, well, we get our we get our Linux or our our Kubernetes distribution from Red Hat or whoever, but we get our hardware from, you know, HP and Dell, and and there's, like, this this dividing line, you know, where where the kernel gets loaded in the memory and and before that, it's the hardware vendor and after that is Red Hat or whoever. And and, you know, the the the 2 shall never meet.
Speaker 1:Yeah. And I gotta say that my conviction about the importance of hardware software integration has only grown, through Oxide. I I I can't imagine that I'm not speaking on behalf of everyone in Oxide when I say this. But, I mean, because there are things that, like, I knew were problematic going into oxide, like SMM. Right?
Speaker 1:System management mode. Which, John, don't you think SMM is kind of, like, an acknowledgment about how overly divided the worlds have become. Yeah.
Speaker 2:Well, it it it it needs to die. It's it's a it's a if if anyone doesn't know what SMM is, right, it's it's a system management mode. It's like Intel, processors have, you know, 4 rings. Just kidding. They have negative rings too.
Speaker 2:Right? They have these hidden they have these hidden layers where they they had a a debug mode on the 38 6 that you could put the processor in, and they figured out, oh, that'd be cool. So why don't we use that to put, like, fan control and system management stuff? And, you know, I've been in the Arm space for a long time. And in in the Arm space, you know, I said I kicked off a lot of the standards that underpin Arm servers, about, what, 11, 12 years ago.
Speaker 2:And, in the very first meeting we had, I said, people are going to repurpose the equivalent on ARM for bad SMM done again. And what we should be doing is we should be moving that into little microcontrollers Yes. On on the on the SoC, on the on the chip, so we don't cycle steel. Now I'm I'm not saying that ever in my life did I work anywhere where I may or may not have built anything that didn't have any cycle stealing because it didn't need to do it because it put it in the right place.
Speaker 1:Oh, I get it. So it's cycle stealing for me, but not for thee. Okay. Okay. I get it.
Speaker 1:But if I had if I had, I you know, I mean, my my whole
Speaker 7:thing is
Speaker 1:If I didn't, if I stole your cycle, the John Masters tell
Speaker 2:No. No. No. My my whole thing is if you if you design that chip yourself, you'd be crazy to be stealing the cycles from the customer or from the user to be to be doing system management when a microcontroller is, you know, incredibly cheap in terms of cost, in terms of area. Right?
Speaker 2:Area is the important thing on that chip. Right? How much silicon are you using and so on. Well, on on on on some of these chips, you you have, you know, you hear about, you buy a consumer chip today. You have a buy an AMD chip, and you have this PSP, the security processor.
Speaker 2:You buy an Intel chip. You have this, you know, ignition and stuff running in there. They have a little, a little processor in there. People have heard about, well, you know, just having one is kinda quaint, actually. You know, what you really want on these SoCs, on these chips is is is is several or or even dozens.
Speaker 2:And because yeah.
Speaker 1:It's and we've got to add to that because I think this has been a very sore point for oxide. Those have to be documented. They have to be open source. They've got to be considered architectural state. Because, Bob, when you're describing it, like, you know, the these things boot the Linux kernel.
Speaker 1:The Linux kernel is or any kernel, any guest operating system, any either what's putatively host operating system, but is completely divorced from these hidden cores.
Speaker 2:It is 1,000,000,000 of years. It
Speaker 9:Well, I mean, honestly, I I assumed that the SSM was just there for the NSA.
Speaker 1:Right.
Speaker 2:Well, and and and if I may, I'll I'll have one more thing. You know, it's sort of billions of cycles, right, from the point of view of power on until you run the OS and all these things are running behind the scenes. What I'm trying to argue these days is it's part of your TCB. If you want to have, you know, confidential compute or something like this. Right?
Speaker 2:You can't build a truly secure, solution if you can't explain what's running in that chip. And so that then necessitates Open Source. Right? I may not be at Red Hat anymore, but that's my Open Source bleeding out. But it's true.
Speaker 2:Right? You you you have to think of all these things. They help you. Right? The the sort of offloading things is great, But you you you need to describe them.
Speaker 2:You need to document. You need to make that available to people.
Speaker 9:Well, I absolutely agree. And and and, honestly, I mean, I've been following Brian and Oxide for a while now, and I I almost feel like this company is the the the you know, it it like, Red Hat, it is they have follows the open source way as far as software.
Speaker 2:They're awesome. I can I say one oxide story, Brian, that
Speaker 1:I don't think we heard
Speaker 2:from the last the last discussion we had? Right? Oh. Is it, is it what's is his name, Matthew? I forgot.
Speaker 2:The the there's
Speaker 1:You did you you are now kinfolk with Andrew Stote.
Speaker 2:Yes. Yes. Andrew. Sorry. Yes.
Speaker 2:Yes. I think it's yes. Andrew. That's right. I'm, like, I'm, like, at a family gathering a few months ago.
Speaker 2:Right? And and he's sitting there listening to the podcast that we that we interview that we did. And he says, are you John? Like, oh my god. This is great.
Speaker 2:Right? In front of my in laws and family. Right?
Speaker 1:Oh. I did. You're a celebrity, Joel. You're a celebrity now. We outside made you a celebrity.
Speaker 1:We gave you your big break. We launched you onto the big stage. That's right.
Speaker 2:That's exactly right. Especially at this family, like, random garden party I'm at. I'm like, dear goodness. How how likely is this? It's great.
Speaker 1:That is that is great. Yeah. I did I believe that, he sent me a photo of the 2 of you. I'm like, oh my god. That it Andrew is it's with John.
Speaker 1:But I thought that was that was terrific, and I'm glad that we we made you a celebrity. Actually, you know, we when we were doing the the the because I get, you know, every once in a while, someone is, like, you know, a fan who, you know, pops up and says hello, which is great. I love it. Mike, it kinda drives my kids nuts because, like, dad, you're actually not famous. You're just nerd famous.
Speaker 1:Like, this is not
Speaker 7:actual fame.
Speaker 1:This, like, don't confuse us with But that actually the we got a got a discussion. Like, you know, what we should do is, like, there should be a service that allows you to be spotted as a celebrity. So you could be, like, look, I'm going to the stand where you're union. It's a big deal. I'm gonna go book a spotting, and then someone could come up to you at the time of the gallery.
Speaker 1:Like, are you wait a minute. Are you I'm sorry. I'm sorry to interrupt the conversation. Are you John Masters? Because and just gonna be, like, right in front of your family.
Speaker 1:Oh, you pay for that. Right?
Speaker 2:I think it's called Cameo.
Speaker 1:No. This not from no. Not from a celebrity. From a deliberate, like, from, like So Rudy Giuliani. Right?
Speaker 1:He's gonna tell No. No. No. No. No.
Speaker 2:You, John. I love you.
Speaker 1:No. Well, look. That that costs a lot more. Rudy Giuliani, why'd you go there?
Speaker 2:He's $300 on cameo. That's all that's all he costs.
Speaker 1:Yeah. Yeah. I don't I'm not sure where does that tense at. But but you can actually, you know what you can also get on Cameo is you can get the, most of the I'm a big fan of HBO's Silicon Valley. And I wanna have the actor that plays Juan La Flum, who's available on Cameo, call in as our new oxide general counsel.
Speaker 1:You you should. I I mean, how else would you spend venture VC dollars if you're not gonna spend if you're not gonna spend it on that, just acknowledge that you're
Speaker 2:I hope they're listening. I I think you can do that. And and actually, if you guys if you guys go public, you need to do that on an earnings call. Right? You need to do that.
Speaker 1:Absolutely. Ricky, baby. Ricky, baby. Yeah. Absolutely.
Speaker 1:We we we need to we need to go full at Silicon Valley. Aaron, you've, you've had your hand up for a while there.
Speaker 8:Yeah. I was going to chime in on the relationship with universities and hackerspaces. In the nineties, the computer engineering system at Georgia Tech, everyone actually made a chip. They used lithography and went into the dark room and made a 8 bit processor that had 4 registers.
Speaker 1:That's pretty cool.
Speaker 8:Not an impressive chip. But when the photography school decided to get rid of the dark room and replace it with a computer lab
Speaker 1:They also got rid of the photo list.
Speaker 8:Like, we couldn't do it without the dark room, and we didn't have enough clout to have a lithography lab if it wasn't being used by the photography students who actually used it all the time. And because of that, I went through an undergraduate program in computer engineering where having completed the degree was in this position of, oh, how do you make a microchip? Oh, you can't do that.
Speaker 1:Right. And you got I and that we gotta find ways to to alleviate that. And hopefully, I I don't no. No. The thing, Adam, we we I guess, apparently, Oxide, we are gonna be doing our own pick and place machine.
Speaker 1:I don't think we anyone's talked about doing our own photo lift. That I feel I feel I can say that with confidence. Although, I'm waiting for Nathaniel to unmute himself and be like, actually, I was thinking about next year.
Speaker 12:You know? Like, we gotta
Speaker 4:have goals for next
Speaker 8:year. There really is something to be said for doing things that are not state of the art.
Speaker 1:So Yes. Yeah.
Speaker 8:Yes. It's true that if you wanted to build a state of the art lithography lab at a university, it would be a $1,000,000,000. Okay. So actually, run that run that note. Clean room and say, I don't need to do 10 nanometer technology.
Speaker 8:I'm gonna do 10 micrometer. Like, if you're measuring in micrometers, you can actually do that with things that we can build now. Like, it's weird that we've gotten to a point where there are hackerspaces that are better places to go make a computer than Harvard.
Speaker 1:So on that note, I do feel one of the things that is actually kind of fostering interest and excitement up and down the stack in terms of hardware and software is vintage gaming. And we now have a couple of folks at Oxide who have come in they developed their interest in embedded systems or at the hardware software interface because they were doing absolutely wild things with the Game Boy in particular. It's gonna be, John, I don't know if you've seen some of the stuff that that people do with the Game Boy. But it is nuts.
Speaker 2:I played Game Boy, recently because I I my my old one from, from, you know, when I was a kid, it, disappeared. But yeah, I mean, and and it's crazy what people are doing with, you know, custom cartridges and and, who's the guy, oh, the guy who did the hackRF, you know, the the the the great Scott. Right? He does, YouTube videos about very interesting things he's built. He built a cartridge that added, Wi Fi and Ethernet networking.
Speaker 2:Yes. Yeah.
Speaker 1:Absolutely. No. Well, we have an engineer no. Dial into an oxide meeting through a Game Boy. It's like
Speaker 2:Yeah. Wow. Can.
Speaker 1:Because it and and but then it's it's also a good like, fun. It's playful, but it's also super educational. Right? Because you actually are you're in this constrained environment where performance really matters, where I mean, that you can learn a lot about the hardware software interface by getting that kind of stuff working. And I'm not sure that, you know I I don't know that it quite gets to Aaron to your get you doing photo with, for your undergrad, but I think it actually does, get people understanding, like, up and down the stack.
Speaker 1:And, so
Speaker 2:I It also helps them understand the the the the, you know, timing and and how and how, like, you know, the Game Boy had these constraints that it was operating under. Right? This is why you could only have sprites that had, you know, this many sprites on the screen and and and this this kind of movement and so on. Right. Like, that's the other thing that's interesting about that.
Speaker 1:Well and I do feel that you know, one thing one of the themes that came out in our on the meta conversations that I also feel that we've seen at Oxide is that constraints breed creativity. And I do think, you know, Johnny, you said a couple of times in terms of book, you know, in this this post Moore's Law or Moore's Law deceleration period, we are gonna need to get more creative about how we get more or less. I also feel and, Adam, I know you and I have talked about this a bunch. I feel that we will live long enough to see people care about how they they translate power into software execution, power for, like like, power generation. And because, you know, every time you're using power, you're using, you know, some fraction of nonrenewable resources, and people are just are caring about that a lot more, and they should.
Speaker 1:And we should be start thinking about what is, you know, what are the the the kilowatt hour consequences of this software. And I feel like that requires a lot. I mean, oh my god. It requires cutting across the hardware software boundary in so many different ways.
Speaker 8:So if you're programming for something like a Game Boy, it feels like you can see the hardware much better because if you're gonna use up your 2 k of memory, you'll notice that.
Speaker 1:You'll notice that. Exactly. Yes.
Speaker 8:So as the person who decided that JavaScript was a good systems programming
Speaker 1:Hey, wait a minute. Wait a minute. I okay. I don't I did the the the the this took a dark turn, put you up to this. Look.
Speaker 1:Okay. Like, we don't want to do this now? Like, it it won't
Speaker 8:be able people to actually see that. I've written JavaScript and been like, okay. I'm going to try and write this code in a way that doesn't torture the branch predictor. I have no idea whether my JavaScript is or is not torturing the branch predictor.
Speaker 1:That is the old country. I have left it behind. I am in I'm a Rust station now, now, so I do not get what you're talking about.
Speaker 2:But you have to give him credit for for for poking at that one. That was really good.
Speaker 1:You know, I was letting you walk right by I was not gonna do a UEFI tribunal on the war crimes you committed against ARM introducing UEFI. I was gonna walk right past that, and now I'm thinking we need to go back there. I think No.
Speaker 2:UEFI and ACPI are great. I think they're
Speaker 1:That's right. Perfect. Just stand by. Oh, you're that guy. You're the guy who's gonna stand by.
Speaker 1:Alright. That's fine. That Steven, you had your you had your hand up.
Speaker 14:Yeah. I was just reminded of the of the conversation about the fabs at the universities. One of the the great things that you can do to make sure that your, that your university has a silicon fab is to be like a like a a rogue nation, you know, subject to, I don't know. Let's say, sanctions. Yes.
Speaker 7:Yeah. Yeah.
Speaker 1:I was wondering
Speaker 7:if you
Speaker 1:were to mention this.
Speaker 7:I was
Speaker 1:gonna mention this if you didn't mention this, but yes. Growing up in South Africa. Yeah.
Speaker 14:Yes. Yes. Like, actually, the university where I where I studied, had a fab specifically to make, to make infrared CCDs for missiles. And the funny thing about that is that the same nations that were forcing that economically through sanctions were the ones that were fighting the proxy war through that country, you know, against the Soviet Union. So it was kind of interesting.
Speaker 14:What I actually wanted to say was, it seems to me like hardware software codesign is a term which is confused for various types of project risk management. So, like, you know, somebody mentioned the risks when you're trying to make silicon is you've got this very expensive, tape out. Like, once you make masks, like, you know, you've spent a couple of $1,000,000 type of thing. So you better you better run your project in such a way that you eliminate risks, you eliminate errors. I I find it very interesting you folks talking about how you make these little breakout boards.
Speaker 14:You kind of, like, incrementally develop and test your, your system. And I would say that it extends to software too. I mean, like, agile to me is a way of, of managing project risks when the requirements are unknown. And it seems to me like, it's it's kinda like you need to look at the risks in your in the thing that you're trying to build. You probably cannot agile a computer.
Speaker 14:Right? You you have to ship that thing, and you cannot incrementally deliver an OkCite server. So it seems like that's sort of like the overarching, concept here is that you're actually what you're doing is is you're managing project risk.
Speaker 1:Yeah. Absolutely. In terms of absolutely managing project risk. You're trying to derisk, and I think that there is a and I think, you know, kinda had said this or I think Rick made this point earlier where you the the you if you kind of are flipping the big switch and then just hoping everything works, like, that's not a very good strategy. And we have used the fact that it is much faster for us to develop these little boards to derisk various slices of it, like the dim lit lid, which derisks the, you know, one super narrow slice, but one that I was super sweaty about, because it was definitely like, if the machine didn't boot because of that, it'd bitten on me.
Speaker 1:So it it does allow us. It's absolutely project risk. And I do feel like that assuming I mean, it also is really important to we hardware has, I think, actually too much mystique about it. Venture capitalists, I'm now speaking to you, who where it's like, oh, like, hardware is hard. It's capital intensive.
Speaker 1:No one should be doing it. It's like, no. No. Wait a minute. Hardware is hard.
Speaker 1:There is there's truth to that. But it's also there are techniques that you can use to actually derisk a lot of this stuff. And, you know, if it's breakout boards, if it's simulation, which we do a lot of, John, you've I don't know if you dug at all into the Intel Tufino, which we're using for our switch.
Speaker 2:Yeah. I noticed you were using that. No. I haven't, but I well, I mean, aside from the the documentation. Yes.
Speaker 2:I I think about
Speaker 1:it. Best software product that Intel ships sorry, Intel. Best software product that it it's best software that Intel ships is the cycle accurate simulator that we have for Tavino. So we've been able to use a cycle accurate simulator to do a ton of our brain work, which has been I mean, it's and the thing actually does, like, 200 packets a second.
Speaker 7:It it
Speaker 5:to be clear. But, you know, is our is the, switching silicon. Silicon. And Right. The cycle accurate simulator
Speaker 1:I mean,
Speaker 5:this thing is ludicrously complicated, has its own you know, the p four language that it's executing programs in. So this is this is a wild and hugely productive simulator.
Speaker 2:By the way, though, take take take, you know, take take CPUs. Right, as an example. If if if if you if you wanna if you're coming up in this industry now and you wanna look at how a system is built from from soup to nuts, you can download, you know, gem 5. You can build a a model, a cycle accurate model of a hypothetical CPU or even less hypothetical. There's some real CPU examples in there.
Speaker 2:Yeah. You can play with these things, and you can learn how exactly how programs will execute.
Speaker 4:Which is pretty amazing.
Speaker 1:I mean, the fact that that this stuff is knowable, that it's easier than ever. So we should be able to foster some of this, this understanding across this boundary. Remi, you just got got in here. Do you have a what were your thoughts?
Speaker 15:Well, on on Tofino, nothing in particular, but, I was thinking about, mentioning that I I work on a high open source high level synthesis tool, at Google that we're using to kind of create these accelerator have software engineers create accelerator blocks.
Speaker 7:Oh.
Speaker 1:Yeah. What is it?
Speaker 15:So it's called XLS. It's GitHub Google XLS.
Speaker 1:XLS. Okay.
Speaker 15:Cool. And, yeah. We can take we we have kind of making it better. I wouldn't say it's, you know, really, come a a 100% there yet. But, but, yeah, it's it's pretty promising, I'd say, at this point.
Speaker 1:Okay. Yeah. This looks really interesting because I think that, like, HDLs are a a kind of a fruitful area to develop. We've been using bluespec, but there's definitely, I think, more blade than handle, Nathaniel. Is that a fair way of phrasing it?
Speaker 1:Yeah. I I mean, I think
Speaker 4:it has its place. I think it's it's rough around the edges for sure. And and we were we're trying to use it, like, to the pins, which I think is a a use case that maybe hasn't been exercised quite as much. And so I think a lot of BlueSpike modules get dropped in inside wrappers, you know, their cores of some kind.
Speaker 1:And we I mean, I I think I I there are things I really like about, but I think we're not gonna be a single HDL to rule them all. So I think we're definitely curious about other HDLs that are out there. I know, Nathaniel, you've been, like, looking at some of the open VHDL
Speaker 7:stuff that's out there.
Speaker 1:So this looks interesting. And, Nathaniel, have you seen this? The Accelaf? I'm not sure. Are you on there?
Speaker 1:I have not. This looks pretty cool.
Speaker 2:It's it's pretty cool.
Speaker 15:So I I think, one thing that's important is to distinguish between an HDL and a high level synthesis synthesis tool because I I think this is a common, conflation. You know, something like chisel, is essentially, at the end of the day, it's, you know, a a fancy front end over Verilog. Right? Like, you're you're just, you know Yeah. You you've got, like, a better type system, better better syntax, that kind of thing.
Speaker 15:But, you know, ultimately, you're thinking in signals. In an HLS solution, you're really writing code, and then there's, you know, pretty complicated compiler passes that then transform that code into hardware.
Speaker 1:So And you were calling it HOS or high level synthesis? Is that
Speaker 15:what you're saying? Yeah. So there
Speaker 2:It's the level.
Speaker 1:So Yeah.
Speaker 15:I'd say yeah. There there's other you know, there's a bunch of industry options like Catapult HLS, the Vado HLS, that, are comparable to this. I don't know I actually don't know to what extent BlueSpec exists on the I
Speaker 2:was gonna HDL,
Speaker 15:HLS spectrum. I think so I know that it has these, like, guarded atomic actions. I know that's kind of like there is maybe some kind of scheduling going on.
Speaker 4:There is. Yeah. There
Speaker 1:is. Definitely is.
Speaker 15:But I what I don't know is whether it does, the this thing called resource binding, which is basically like, you know, if you have, let's say, a 5 stage pipeline, that has a multiplier in one part and multiply in in another part, and you know that you're you only need you only need, this thing to have a throughput of 1 third, then you can, you know, take you could take a multiplier in one stage and actually, deduplicate it with the other multiplier by muxing its inputs.
Speaker 4:I I think there's I mean, I this is not an area I've explored a whole lot because mostly we're not doing, very complicated algorithms in our HDL on the server right now. But there is I I know BlueSpec kinda does sit, I think, in between the, like, HDL areas and HLS, and it it kinda has you know, it's a it's a little bit into both sides of things. So I think there is some, capability of doing that kind of optimization. How fleshed out and easy it is to use, I don't know.
Speaker 2:The thing that's really cool with it is that is, like, before I mean, I've heard of this Google company. I'm not here representing Google, but I heard of that company. You know, the thing the thing with XLS is that is that, it is something that's put out there that that people can can use because, you know, years ago prior to where I work now, you know, I I paid $6,000 out of pocket for access to Xilinx's Movado toolset including their HLS. Yeah. And and it's like the number of times I'm going to pay that, a, most people won't pay that one time, but the number it's a one time thing.
Speaker 2:Right? Oh, I'm really interested. Let me play. And, you know, that's the other issue with a lot of this stuff is that you get really interested and and there's no way you get access. Like, if you're in school, you might get access through education, if if a AMD, if you're Lisa Sue, if you're listening Oh, god.
Speaker 2:Hang on.
Speaker 1:Hang on. They're all waiting. Lisa do listening?
Speaker 7:I got
Speaker 1:I got something to say, Lisa Sue. You go sing because I got something to say. Please open source everything, AMD.
Speaker 2:Well, don't even Please. I mean, open source it.
Speaker 1:That'd be great. But even if you're not
Speaker 2:gonna open source it, at least make it, inexpensive for somebody who's on the weekend just trying to play and learn and actually interested in, you know, playing with something like Viberto or whatever it is. Right? Like, why should you, as an individual, pay $6,000 for access to Xilinx stuff?
Speaker 1:Right.
Speaker 2:You're not gonna do that. You're not gonna you're not gonna be able to learn. And and and if you can get access and play, maybe you'll use it at work. Right? I mean, come on.
Speaker 1:Well and we've tried to so we're using Lattice FPGAs for exactly this reason because Right.
Speaker 7:Because they're
Speaker 1:open because they're open source tool chains. Right? Well, they're it's such a curious kinda case because it's kind of, like, open source against their will. They
Speaker 2:were Yeah. Yeah. Yeah.
Speaker 1:Forced. They they they are like, they are choosing not to legally enforce that someone has reverse engineered that the, and, I mean, god bless her, has reverse engineered all this stuff to and we've been able to use it. But trying to explain the Lattice and the good news is like Lattice is not opposed to open source toolchains, but they are very confused by it. And they're like and we tried to explain them, you make money from the FPGA. That's what we buy from you.
Speaker 1:So if we the more we can make, like, the more we will buy of your FPGAs. And they're like, no. So it makes sense. You're like, okay. We're just not communicating.
Speaker 1:But I I do think that, like, yes, please, please, please, these tool chains need to get open. And and honestly, Remy, kudos to those folks inside of Google who are making sure that things like Excels get out there because I'm sure that that's that's a discussion that has to be had. And and it's it's a huge service to the broader industry that things like this are out there. So, I mean, this looks pretty cool. I wanna play around with it.
Speaker 1:So, it's interesting to me that some
Speaker 8:of the open source tools have such a reputation for not being high quality as the proprietary tools. Because at least when I was in university, one of the tools we had to use was this layout for schematic checker. It's like, okay, here's your chip layout. Those those schematic actually mean the same thing as your chip layout. And one day, I gave it the commands in the wrong order.
Speaker 8:And I told it my layout was my schematic, and my schematic was my layout. And the graphics card drew an error message to the screen, and the computer rebooted. Right.
Speaker 1:Well, so And,
Speaker 8:like, if I had the opportunity, I would have taken an hour and fixed that problem because I lost more than an hour's worth of work and wanted it to never happen to me again. And if everyone had the opportunity, someone would have fixed it before I got to it.
Speaker 1:And the issue is not quality. I don't think. The issue is actual features and polish, not quality. I think from a quality perspective I mean, Nathaniel, correct me if I'm wrong, but from a quality perspective, it's been, I think it's been surprisingly good.
Speaker 2:There's liability though too. Right? There's liability. But a lot of these folks, like, there's the fear, you know, like, if I use this commercial EDA flow, then nobody ever got fired for buying IBM. Oh, wait.
Speaker 2:They bought Red Hat, and I left. Oh, anyway. So but, you know, nobody ever goes by
Speaker 7:for buying
Speaker 2:a proprietary EDA tool. Right? Like, it's it's it's it's unfortunately the mindset. It's not necessarily true.
Speaker 5:This is sort of the natural evolution of all open source tools. Right? Like, initially, they are not as robust, not as feature full, and they they all sorta I mean, they all get there based on that need and and all the waves that just upgrade. Right?
Speaker 7:Sorry. I I don't mean to interrupt. But, I mean, one of the problems is silicon verification is huge. Right? And if you haven't silicon verified your entire chain, then anything could break.
Speaker 7:Right? And then, like, you don't wanna debug the actual hardware. You don't wanna debug Silicon. Right? That's, like, the worst situation you'll ever be in.
Speaker 7:So, I mean, yeah, the foundries work with the EDA tool providers, and they make sure that sign off on one end actually means what you think it means. And one of the, well, I wouldn't say issues, but one of the limitations of a lot of the open source tooling, at least on the silicon side, is that it's not people working with the foundries that have sort of, got this whole sign off verification flow and all worked out. You don't have, you know, a 100 chips taped out where you can say, okay. You know, all 100 of them worked. They might be an area.
Speaker 7:Yeah.
Speaker 2:That's my point. Right? It's it's it's it's the liability. It's like it's like, I mean, we'll we'll get there, but I can I can understand somebody who's writing a check for tens of 1,000,000 of dollars saying, I'm gonna use the EDA vendor, you know? Yeah.
Speaker 2:Exactly. It's a bit it's a bit fuddish, but it's also, like, the cost of failure is so much higher than I'm just gonna update the software.
Speaker 1:Yeah. Let's understand. When I think as an industry, we need to we definitely need to walk before we run. And if we can get to open PCBs and if we can get to to open FPGAs I mean, open a 6 would be great, but that is definitely to me, that's the next that that that's a much tougher mountain range to climb for all these reasons, because you are dealing directly with the foundry and the the consequences I mean, Sidharth, as you're mentioning, like, the consequences of bad silicon are so high. So I you know, Adam, normally, it is you who has the toddler, beaming on the door.
Speaker 1:I am I I've got the kids solo this week. The 17 year old is fine, but I've got a 10 year old who's definitely is getting, as she's explaining to me the background,
Speaker 7:hangry.
Speaker 1:So yeah. Exactly. I'm gonna have to go, go go deal with the the hanger here. But, John, this was great. Thank you so much for, I'm glad that you whoever set you off on LKML, I'm I'm just gonna be glad they did it because it's been great to have you here.
Speaker 1:This has been a really fun discussion.
Speaker 2:Yeah. It was really great to hang out, Brian. And, I guess I should join this, again sometime, Abibra.
Speaker 1:Absolutely. And, John, if you ever need to be cited as a celebrity, I want you to let we will we'll make that happen for you. I want you to know that.
Speaker 3:They're they're affordable cost. Right?
Speaker 1:And very affordable. Like, much cheaper than Giuliani. Like, I that guy's ripping you off. So we can, we'll make sure that you get cited as a celebrity at all of your family gatherings.
Speaker 2:I I will 100% take you up on that.
Speaker 1:Awesome. Alright. John, thank you so much. Adam, thank you as always. Thank you, Nathaniel, to everyone.
Speaker 1:Rick, Aaron, Simeon, Remi, Sudar, thank you very much. Tom, thank
Speaker 14:you for your help. Thank you.
Speaker 1:Oh, a lot of fun. Alright. Take care, everybody. Thank you.
Speaker 2:Thanks for
Speaker 10:letting us know.
Creators and Guests
