Rebuilding 'Castles in the Air'

Meredydd Luff 0:00
Okay, so So here's my controversial opinion. Okay? There's an awful lot of Stockholm syndrome in this industry. It's Stockholm syndrome. It's the psychology behind hazing.

Lee Ngo 0:16
Hello, everyone, welcome to Educative Sessions, a podcast series with people in the developer world about their coding experiences. This is brought to you by Educative which makes it easy for authors to provide interactive and adaptive courses for software developers. My name is Lee Ngo and I'm the host of Educative Sessions. Today my guest is Meredydd Luff of Anvil Works, who will talk about rebuilding castles in the air and we'll go a little bit into more detail about what we mean by that. Meredydd, welcome to the show.

Meredydd Luff 0:47
Thank you, Lee. Great to be here.

Lee Ngo 0:48
Yes, long time coming, really glad we finally made this happen. So, I want to begin with a quote that you shared with me as we were discussing the topic for today. From author Frederick Books- Brooks, excuse me. And to quote: "The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures." Meredydd, is there anything you'd like to add to this or interpret from here?

Meredydd Luff 1:31
So, that quote, which I- I mentioned to you, when we were talking about coming on this podcast is actually stenciled onto the wall of the Anvil office, because it's- it's possibly my favorite thing that's ever been written about programming. Possibly even- possibly even more favorite, is how it goes on because it goes, "get the program construct, unlike the poet's word is real, it moves, it works. It prints results, draws pictures, produces sounds, moves arms, the magic of myth and legend has come true in our time. One types the correct incantation, and a display screen comes to life, showing things that never word nor could be." And I think it captures the spirit of programming, that point where you can pull something out of your brain and cause a computer to do something useful. It is the closest thing to magic that humankind has yet invented. And that rush and that power is behind basically everything that we do.

Lee Ngo 2:42
That was wonderful. And I've often joked similarly when I've hung around a lot of programming friends, and they tend to operate very similarly to me to that of like how I imagined wizards and like wizarding guilds do. Like they kind of hang out in collectives. And now if you're a, maybe a D&D fan like I am, you would know that like, you know, wizards are distinguished from sorcerers, and form Warlocks, or what have you, by how they cultivate their power. And unlike the others who have to either summon or channel, or what have you, wizards have to study, right? They have to learn how to master the words from the textbooks themselves. And, it is from those books only, alone, that they become empowered through their magic. And so, you know, I'd love to, you know, push a little bit further about, you know, the topic of today. And you wanted to talk a little bit more about where programmers are now, and how they are training themselves and are being forced to think of their own crafts, either in a generalized or specialized way. Right? So, if you can elaborate more on that.

Meredydd Luff 3:49
Yeah. Well, so- I- I really identify obviously, with- with that description of the wizard. The the sense that these creations are- pours forth from your own brain. But I do think you can go too far with the requiring people to spend years studying. You know, that- that makes for a great fantasy setting, it does not actually make for a great way to participate in computing the internet. Back when I first got involved in programming, I could pick up something like Visual Basic. You could- you could, with a very small amount of programming understanding, you could bring into existence, an application that works like everything else on your system, which in those days was Windows Desktop. But the gap between first learning how to program just a little bit and having something that worked and was useful was really small. And I think that's- that's important- important inducement to follow the rest of that path into mastery. Or, indeed, for people for whom that's not going to be their whole career to be able to achieve something useful and kind of stop there, and you know, go spend most of their time being, I don't know, an engineer, an administrator. And if you make it easy to do- to do something real, then you throw the doors open, rather than saying, "well, you need a 4- 4 year degree to even start with this stuff. And- and what actually happened since the 90s and the days of things like Visual Basic and Delphi is that we invented the web. And the web is in many ways, titanic achievement. Anyone can build something that can be used by anyone else in the world. But the bar you have to clear to build something is, what? You need to know, HTML, and JavaScript, and CSS, and maybe Python for the back end, and SQL to talk to a database, and then React, and Redux, and Flask, and SQLAlchemy, and Webpack, and Bootstrap. And- I haven't even got started on the DevOps stuff- and the- again, it makes for a great fantasy setting to have the idea that these incredibly clever people had to go and spend years in a library to be able to- to do the most simple spells. It's not really a good foundation for our industry for our craft. And we've lost that ability to go from naught to something useful in an appreciably small amount of time, that makes it easy for someone to get started. And that is, I mean, if you ask me- so, the tragedy and the beauty of that, Fred Brooks quote, is that it is- it shows you what you can accomplish, and what these days someone has to do a lot more to do. If you want the display screen to come to life in a way that your colleagues, your friends, your customers are going to recognize, you're going to have to do an awful lot more work than you had to do when we were children. That feels like an awfully backward step for progress.

Lee Ngo 3:58
What do you think could account for this presumption? Right? Especially- what, is it- is it a question of optimizing, you know, coding employees to make them as skilled as possible? Like, can one person do the job of four people? And, what- because what I'm sensing is that there's something at least to say, improper, if not, I mean, do I want to even say exploitative? But at the very least, like this is might not be the a the most elegant way to approach what coding can be.

Meredydd Luff 7:46
Yeah, I mean, I don't think that it's- I don't think it's some dastardly exploitative plan. I think it's- it's path dependent. It's how how we built the world. When we built the internet, it was built out of this distributed system. And, you know, browsers only slowly evolved from bare- quite primitive document viewers into application platforms. And- I mean, frankly, that- the browser platform itself is kind of the rubble of the platform war of the late 90s, early 2000s, between Microsoft and Netscape, and- more late 90s, I suppose. And what that means is that we've always been, in the attempt to turn the web into something that you can build any significant application on, we've always been spackling over holes, and weak spots, and crumbly bits. And I think we spent so much time focused on the effort of making this grand distributed system work at all, that we forgot that we'd lost this- we'd lost the power and accessibility that we used to have. And people- like people are doing their best. It's what people do. People are built- when people build a framework like React, it's not because some engineer at Facebook was sitting there twiddling their moustache going, "I know what. I'm going to have to- I'm going to make people understand how the HTML DOM model works, and how JavaScript works in some quite deep and weird functional ways. And, actually, I'm going to make people understand how this library I'm building thing as well works before they can get anything done. I'm going to make people understand hooks." No, that wasn't-wasn't a dastardly plan. It was, "given the, the materials I have available to me, which is to say that every application needs a server side, then represented HTTP requests would be full of Jason, then objects in JavaScript, then downloads in HTML, then pixels on the screen. And we have to do all that transformation. That's the day to day work of a web developer is doing those transformations. Given that, I'm going to try to build Build a power tool to help me with the transformation between, in this case JavaScript objects and HTML DOM." And it- none of this is is badly intentioned, it just has the result of piling complexity upon complexity, such that something that used to be simple has become incredibly complicated. And, so, the- the difficulty of being a full stack developer- it- it arose by accident.

Lee Ngo 10:31
Right? Um, but I'm trying to wrap my mind around how to respond, because it's, it's like, I mean, there's aspects of- this is- what if this is part of- just kind of the evolution of technological development and particular solutions, like, you know, React and other things came about as the- you know- in a response to the needs of the technology, right? And, in one sense, it's elegant, but in another sense, now, like, has overall, it'd been a much more difficult process for a common programmer to, like, become comfortable in their own craft, like- and I guess, you know, let's- let's go to the end and think about like, what would you suggest should be done instead of this, you know, laundry list of prerequisites in order for you to become a viable developer? I mean, if that's not- even if- it's because at some point that's- can't be attainable anymore? Like, what is the alternative to that?

Meredydd Luff 11:30
So, I'd say there, I mean- there are a couple of projects that I've- that I think have captured some of that, "just get going; build something" spirit, I would say that one sort of big strain of it is that the Jupyter phenomenon. So if you're- if you're anything to do with data engineering, or data science, you've probably met a Jupyter Notebook. It is this interactive programming environment where you can execute sort of slabs of Python code, and you see output, you could- it's really very easy to at least draw graphs. And you can even get some like little bit of interactive widgets. Streamlit is another example of- kind of in that same- in that same vein of trying to go from zero, some kind of interactive widget in a data application- it's kind of limited to data. So, it's not the general solution. But it really does get you from, "hey, I can write a little bit of- a little bit of code," through to, "I can build something that's interactive enough, my display screen does come to life and show me the things I needed to without forcing everybody who wants to- you know, all these statisticians, which is what a lot of these scientists are, without forcing them to also become full stack developers, learn a whole nother careers worth of stuff in order to get anything done." Like the Jupyter ecosystem, and that I include, like Google's Colab notebooks, there's a great hosted product called Deepnote. There's- there's a bunch of work going on with this idea of an interactive notebook. And I think that's been really powerful, as it- as one attempt to push past this for a certain set of users. Another one that I've seen doing great work in education is physical computing. So the Raspberry Pi folks are really big on this. I'm speaking here from Cambridge, home of the Raspberry Pi biobrick folks, and they are really big on, "well, we want kids in particular, but anybody, especially kids, to be excited about being able to make this little computer do something. So we're going to make a computer that is little enough to hold in your hand and understand that this too, is a computer. And we are going to optimize for making it easy to, you know, make a light flash, make a robot move." And again, that's that sense of magic where something comes to life. But it is also you know, limited, it's mostly focused on educational environments. To do the more serious stuff, you typically end up having to bring some other set of skills into play, which will often often be web development. And so I suppose the- the final one I am going to bring up is of course Anvil, which is the answer we tried to come up with to this problem. Anvil is an environment for building full stack web applications without having to know all that stuff. So you drop into Anvil that works. You get a canvas. You can use drag and drop design to build your user interface, not unlike something like Visual Basic. But if you put a button on your page, and then you double click that button, well now you're editing the Python code that runs when that button gets clicked. And you can drive your user interface. You can interact with a database you can send things to and from the server, which is also hosted in Python. It's hosted by us, the default case, and you don't have to do any of the DevOps stuff, or setting up database, or setting up Python environments. You can just go- if- we're reaching for that holy grail of, I can go from, "I can write a little bit of code" to, "I have built something that is useful, it is online, it has a URL. I can, you know, give to a colleague and have them actually interact with it sensibly," without forcing you to learn anything more than a little bit of Python.

Lee Ngo 15:21
What a flurry of great tools that you've just discussed here. And it's nostalgic for me, because if it weren't for working in Jupyter, notebooks, I personally wouldn't have been able to access Python, especially Python and data science the way that I did. And I think it's a totally fine, if not a noble cause in itself for, like, if these things exist strictly for enabling educational purposes, like- I mean, that gets to the crux of the grand problem that you've talked about since the beginning, which is that like, right now, it is a insurmountable educational problem to presume that people can kind of just jump into-

Meredydd Luff 15:53
-Oh yeah-

Lee Ngo 15:53
-things in the raw and just presume that "oh, like, yeah-"

Meredydd Luff 15:57
It's that they're willing- they must be willing to climb all of these barriers before they can get anything done like that- that- that is- it's absolutely a huge educational barrier.

Lee Ngo 16:07
And I find that, you know, there's always that conceit that like if you can understand logic, or what have you, than, like, these things should, or if you're- if you have the mindset of an engineer, blah, blah, blah. I think that's a little bit conceited, because I like what you've just shared, which is, you know, this- there's been across the tools from Jupyter, to Raspberry Pi, to even the work of your own company. There's this insistence upon- but there's a deliverable delight to it, right?

Meredydd Luff 16:33
Yes.

Lee Ngo 16:33
It's this, like, part of- it's ironic, because, you know, engineers want to get under the hood and kind of know how everything works. But the magic is part of it too. Watching something kind of happen. And even if you know why it happens, it still looks very beautiful that it does. I think we shouldn't walk away from that spectacle perhaps, right?

It i- I mean, have- I think of it less in terms of spectacle and magic, and more in terms of, "are you familiar with the concept of inherent versus incidental complexity?"

Please.

Meredydd Luff 17:04
So, some- when you're solving a problem, there is some complexity that is inherent to that problem. And some complexity that just comes along the way and it's just the- the stuff you have to sweep out of your way to get something done. So, I want to- let's- let's imagine, I'm a data scientist. And I want to predict the- some time series. In that case, my prediction, the statistics, whatever tools, whether it be machine learning or something else, that I'm using, that's all inherent complexity to the problem I'm solving. I'm not going to get rid of that complexity, because that's my task. That is what I'm setting out to do. There is a whole bunch of incidental complexity to do with, like, setting up my development environment, making sure someone else can open it, you know, can open this thing and use the tool I've written. You know-

Lee Ngo 18:08
-data cleaning...

Meredydd Luff 18:09
-every time. Yes-

Lee Ngo 18:10
-Yes. Yeah.

Meredydd Luff 18:10
Yeah. And of course, I get, like this is- it's sort of task specific. Because, if you are working in and working different levels of the stack, then what you consider inherent to your problem, what you consider incidental, is different. But the- the delight comes from solving the problem you're setting out to. You know, a lot of the delight comes from puzzle solving. And so that comes from solving the inherent complexity of the thing you're trying to do. If you are trying to- if you are trying to build a- no- a collaboration tool, the delight there comes from implementing the collaboration and working out how these people should interact. What are they, you know, what's the rules? What's the logic of this of this system? The incidental complexity that slows you down is, every time- every- all- every second of time you have to spend installing software, or working out why, you know, why this version of React is not working with that version of some other piece of software, you know, tweaking your Webpack config file. Nobody- essentially, nobody is getting delight from tweaking their web- Webpack config file, they are getting delight from making a thing work. And so, what you've identified as the spectacle and the magic, is sort of purifying that I am getting something working sense and those results. Do you see what I'm getting at?

Lee Ngo 19:47
Yes. Yeah. I mean, I would love it if people were much more considerate of these things. I mean- like- and- and, you know, these are the kinds of things that instead of gatekeeping, or instead of, you know- or just presuming that like the messiness, or, you know, lop- leaving a lot of these difficult complexities as, "ah, that's just part of the gig," versus "no, let's optimize that. Let's make those a lot better." And-

Meredydd Luff 20:16
Okay, so- so here's my controversial opinion,

Lee Ngo 20:18
Okay.

Meredydd Luff 20:19
There's an awful lot of Stockholm syndrome in this industry. It's Stockholm syndrome, it's the psychology behind hazing. "I have had to drag myself over this broken glass to be able to produce anything out of this system we call the web; you should have to too." Or, "my effort should not be in vain." And so, I think, for a lot of people who've had to master a bunch of incidental complexity, they end up feeling, you know, feeling like it's more necessary than it was, because you know, there's a bunch of psychological literature demonstrating, if you- if you've built something, if you made something yourself, you feel- if you've had to work for it, you feel more warmly towards it. If you've had to work towards getting this thing to like to turn over at all, you feel like really committed to it. And then there's this separate phenomenon, which is that if- if that is the biggest thing you have going for you in career terms, then you know, if a bunch of my career capital is tied up with actually "I know react" as a substantial fraction of my selling point, I'm going to want that to be worthwhile, I'm going to get partisan about my web frameworks. Whereas, and you often see this sort of phenomenon, as the more- in the more senior people, the people who have less to worry about in terms of proving themselves, are actually much more likely to pick up whatever tool suits the job, whatever is at hand, whatever is going to be fastest, because they don't feel like they have to prove that the pain they went through was worth it.

Lee Ngo 22:08
I don't even know if we have time to get into all that. But- but I would be curious about a follow up conversation about that. Because, you know, and that goes into a lot of different kinds of just like- like rites of passage, and deep, like sense of culture. And- and then you know, I mean, I will leave this but this is a damn question. Please don't respond to it. It's just like the question of, you know, it's like- it's like the Wizard of Oz statement of like, "if I could just tell you how to do it. If I told you this is how you should get home, you wouldn't believe me," or something like that. And that's, and I will- and now I'm wondering if like that entire thing that the good witch told Dorothy is just simply a ruse just to make us have go on this crazy journey. But who knows? I mean, I think that's a- that's a long and more- we have to- we have to think that little bit through, wouldn't you say? But I appreciate you at least sharing that. And I think, at the very least, I do want to include with giving you another opportunity to really share more, because I- I firmly believe despite your controversial opinions, perhaps, that you- your heart is in the right place, and that you are coming to this cause with this fundamentally noble intentions. And so therefore, regardless of what other people might think, they should consider what you're thinking. So, is there anything else you'd love to share to the public? The floor is yours.

Meredydd Luff 23:33
So, I mean, I am going to shamelessly plug Anvil because it is my- it is my best attempt to work out, "okay, what would actually fix this- this idea that you can, in fact, build a web application without all that incidental complexity. And you can go from a little bit of programming, to doing a lot." And that just because you do that, and I'm aware that this is somewhat education themed as a conversation, but nevertheless, I do think it is important to say it's not just for beginners. That it is possible to be accessible to somebody who is just starting to code and also to be powerful enough that somebody senior isn't afraid to use it because they know they'll be using a real programming language with a real deep ecosystem and escape hatches for anything they want to do. So, yeah, check it out Anvil.works. It is free to use, free to sign up. There's a forum where you can come and say what you think of this design and of this approach of trying to remove some of the incidental complexity and connect you more directly to the joy of solving the problem you came here to solve.

Lee Ngo 24:59
Great and thank you so much for that. And, really, I mean, this has been- gosh- one of the most- I don't know how to- what words that are coming to my head right now- but I will say, I feel like I've been sucked into like a fantasy novel or something like that. And that's the equivalent of a podcast that has been this experience. And I really appreciate you sharing, not just, like, all the brilliant work that you've been doing, but also your candid thoughts about how we can be better, not just as a coding community or as learning code. And of course, you know, here with us at Educative, we're always open to listening to ways in which people can continue to optimize themselves. So I want to thank you, Meredydd Luff so much. And I want to thank Anvil Works, as well, for participating in this podcast. I also want to thank you, as well, for listening or watching this podcast. You can also check us out on YouTube or on any major podcasting platform. Lastly, final plug for Educative. You can check us out @educative.io to see what we do. So, for all of us here at Educative and beyond, thank you so much and happy learning. Bye bye now.

Hi there. Hope you enjoyed this Educative Session. Be sure to check out more on YouTube or on any podcasting app and be sure to like this episode, and don't forget to subscribe to our channel. Happy Learning.