062: UI for U and I

Show Notes:

• 00:56 - What does UI mean? UI = User Interface
• 02:26 - Software Interfacing with Software vs Human Beings
• 03:57 - At what point does UI stop?
• 05:55 - Responsive and Stateful UI
• 10:10 - Tooling: Past, Present, and Future
• 16:00 - Planting Business in UI Engineering
• 19:26 - How The Frontside Brands Itself; JavaScript Portability

Resources:

Linguistic Intelligence

Transcript:

CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 62. My name is Charles Lowell. I'm a developer here at The Frontside and I'm here today with Jeffrey Cherewaty and Robert De Luca, also developers here at The Frontside. We're going to do a little bit of navel gazing today. We're just going to talk about how we roll and who we are and because we've there's been a lot of conversations about that, as we had a couple of different projects come in that use different tool sets so what is it about us that is constant and what is it about us that changes? I just wanted to talk a little bit about that. The core idea is about UI. The shop has always been about UI and I'm curious for you guys, what does UI mean to you all?

ROBERT: The frontend to the frontend?

JEFFREY: That's a loaded question.

CHARLES: Okay, let's step back a little bit from that because we've been in business for a long time and right now, we're doing mostly Ember, a little React. Before that, we were doing all Ember. Before that we were doing mostly Rails. Before that we were actually doing a little bit of wxWindows stuff and before that we were actually doing stuff in Java. But all of it fell under the rubric of UI -- user interface. What is constant? I mean, there's some radically different tools. When you look back and you think, "My goodness." What could possibly connect those things that I just listed? But I think there is a common thread so I wanted to explore that a little bit.

JEFFREY: The common thread, I guess at the simplest level is always that in a lot of programming, you're dealing with the interface to the end-user of your code is simply text. It's a call-in response or they're consuming a library of yours that's where the code is the end product. What we focus on instead is the the end product of what we build is the interface as the graphic representation instead of a code or a text-based interface.

CHARLES: I'm curious. In terms of all the software that gets written, how much do you think- because I think that's a key distinction. There are certain software that interfaces with other software. The connections to other software, there's a very unique set of problems when dealing with that. But there is also a different class of software where it's interfacing on one side with other software but on the other, it's interfacing with human beings. What proportion of software they get? How much of it is actually interfacing with human beings? How much is actually interfacing with other software?

JEFFREY: I think the most software developers underestimate how much other human beings are interfacing with other software. Even if it is a simple HTTP protocol Rust type interface, it's still being used by human beings even if it's human beings writing code.

CHARLES: That's true. There's definitely like a whole metalevel and that's like --

ROBERT: Like the developer experience versus user experience?

CHARLES: Yeah but it's true. There's probably very few pieces of software where the ergonomics aren't important to actual people. That's true. There's definitely like a [inaudible].

JEFFREY: What you're getting at is that there's typically much more code underneath the surface than there is that actually is visible to a human.

ROBERT: That makes it all work.

JEFFREY: Exactly.

CHARLES: Yeah. The part that's lying just beneath the surface is an ocean in of itself, I would say. It certainly seems like because that's the ocean we swim in it, seems vast at times.

JEFFREY: Where does that UI stop? At what point does it stop becoming UI engineering and becomes more like API side of your server side? Obviously, you can say like, "Right when you start talking to an API. That's the end of it." But all of the code that goes on the frontend, the people refer to this as the frontend of the backend. All that code that is just there to get data and then massage that data so then somebody can go and present it to the user is at all UI at development. It's gotten a lot murkier in the last three years.

CHARLES: Yeah it's a lot murkier because the thing is the way we structure our server technologies are changing so that they can support various interactions. For example, one thing that springs to mind is treating the data that we have inside of our client and I'm thinking of a browser tab, for example. Just one browser tab, one browsing session, you load data from the server and you write data back to the server but what we're seeing is that degrades the UI experience if you treat it just like that.

I feel like the general trend is to treat the data that's housed in that browser tab is merely one partial replica of some distributed data set that's being replicated across a whole bunch of nodes where your storage nodes are one of those things, the node that other people are might be using in participating as application or other nodes and then that one browser tab in itself is a node in this distributed database. But that's crazy because that's what you would consider a classical server work. But to get around the fact that application state is updating and changing rapidly and trying to keep up with it in a non-buggy fashion can be really, really difficult.

I know the waters are getting a lot murkier but I think that example does touch on something that I would say is kind of classical UI problem. You have some sort of stateful something. You have some sort of long running process that's just sitting there that's kind of moving with the user because the users holding a lot of state in their head as they're interacting with your application. A lot of the context is stored in their head so the UI needs to be able to dance with them and be in lock step with them and kind of mirror the context and their understanding of where they are in the system so it's usually very stateful. At least the ones that we've been working on over the past years.

JEFFREY: I think another core principle of UI engineering is responding to events in a way that just doesn't happen as much in classical server engineering. You have to respond to a message or some kind of request every once in a while but the level of responding to changes in state and changes in how the human is interacting with the interface is a whole another level in UI engineering.

CHARLES: It is really is a dance, in the sense that as the human moves their virtual hand throughout your application, you have to be tracking that state at all times.

JEFFREY: It's like it really shines itself there. If you compare a state machine for the server side and then a state machine for the client side because we've built both and the client side is way more complex because there's just way more that's going on, impossible things that can happen. With an API, it's like almost crud in a way with an API and then on the UI side, we have to take a lot of user interaction and boil it down in order to persist that on the API side. There's more for us to take care of, on the UI side in track and it just more complex.

CHARLES: That's true. I think there might be some people who work primarily on the server. They're being like, "Wait a minute."

[Laughter]

JEFFREY: At least on the server side, you have one runtime that you can deal with and for frontend developer, there's just so much to contain.

CHARLES: Yeah so in terms of tracking that state, certainly one of the things that I think is also unique problem for user interface is that you have to constantly be taking in those events, that you're talking about, like you're constantly having to update your state so that you keep in lockstep with where the users at. That can also be part of that information that you'd be reacting to where other users are at, besides just that user so that they can absorb that context. Yes, so reacting to where they are but then also radiating information constantly to the user so that they're both inputting at extremely high rate but they're also receiving information. Like you said, it got to be extraordinarily high bandwidth so that state that you're tracking has to be represented accurately at any given time.

I think, basically the core driving principle of NBC is you have this model and that represents your state and you have these events which then update that state and then you somehow managed to instantaneously reflect that state change to the user in that tight loop. I feel like that pattern, despite people tried to bastardize it, or maybe not bastardize it but adapt it for their particular use case but it has been persistent. It's like a cockroach, man. Always live and never die. I feel people are trying to assign new acronyms and new names for it but the fundamental pattern has proven to be effective over and over again.

I would say that if you're using NBC, if you're building like a system with React or you're building a system with Java Swing, like we did back in the day, you're using the same basic pattern even though the mechanics and the tools are completely different but you have this abstract representation, this model of either your state or your interactions. You can model a swipe or you can model a mouse move then you're basically reflecting that to the user and then taking updates to the model.

JEFFREY: What's changed about the tooling, say back when Frontside was primarily on Java? We're like, "That was the tool." It kind of a blunt instrument for some of the UI challenges we have now. What was working well in that arena and where we come [inaudible]?

CHARLES: I think there's just so many exciting things that have happened since then. Especially in the past, like three years because Swing which was the Java UI framework which was fabulous, just absolutely magnificent, was at its core an observation-based system. You had a lot of observers. You can basically have these models and your view was a separate object but then you had your model. This was definitely a watershed moment. At least for me, from inside your code, you never actually updated the view. You never said I want to paint this thing. You never said I want to change this componentry.

Only thing you did in reaction to events was set properties on your model. It was a mutable model so you basically had your state which is like a chalkboard but your code, the key thing there was you could paint things in Swing, you could render things but you never actually did. You had your components that always render off of a model and you never call those imperative like draw this line here, draw this line there. You always update your model and the view reacted to it. That's fundamental and that's something that we still see today but we were using mutable models. The models themselves just got overwritten every single time.

The other thing is they were observer-based so you would observe this model, you observe properties on this model and when those properties changed, you're view would rerender essentially but I guess the paradigm was the view pulled data off the models and I feel like in the last three years, really with the advent of React, I think really popularized this pattern now and everybody else is adopting this.

ROBERT: The flow architecture?

CHARLES: Well, yeah. It's like a push model. Instead of having the view pull from the model but via observation, you have your controller essentially push a new model onto the view. But I still think the basic components were there, where there's two things that I think have change. One is this push model -- the pull view observation versus push from an event like event generates a new model and pushes on it and then the second thing which is probably coupled to it which is the idea of having your model be immutable.

The magic of that is that you essentially unbraiding time from your model so that you the programmer are managing time instead of the CPU clock, which sounds weird to say. See, you guys are like, "What the hell are you talking about?" But that's fundamentally when you have something that's mutable, you're saying this is one object. It's the same identity.

ROBERT: That you've changed over time.

CHARLES: That you change over time but you're writing it to it where is when you use a different model each time, you the programmer is essentially saying, "This is the same object. You are managing identity of the objects not the computer. Does that make sense? You’re saying here are a bunch of states but the identity of this thing is like an abstract concept and I'm saying all the states represent the same object or same entity just over time. What you're doing is you're essentially decoupling time from your model and that's I think a key innovation because then it gives you control over time so you can actually represent time, however you want to and that allows you to do things like history and what's the time travel debugging, which I think is very different than undo/redo. I think people conflate those unnecessarily but it allows you to do undo/redo history type stuff because the timeline is totally orthogonal now.

Whereas, if you have immutable state, it's like, "What this looks like if that time is absolutely coupled to right now?" I think those are key innovations but I think that the fundamental pattern is still there. You have models, you have views and you have control code that manipulates. I think the key thing that's happened over the last three years that have switch between having the views pull the model versus having the controller push on there.

For example, using essentially what amounts to an observable interface. I think observables as in ES7 observables are one of the coolest innovations to come around because it gives you that convenience of having a single reference point but access to all of the states that then you can react to it.

Anyway, that's a long winded thing of saying that I feel those core entities are still there, where you have this and then everybody who listens to me long enough will probably get tired of me saying this but I think that the primary one is the model, like understanding that.

ROBERT: Because your UI should be eventually derived from the data that you get?

CHARLES: Right.

ROBERT: And it should be because it is a source of truth.

CHARLES: Right and everything else can flow from that. Understanding how you're going to represent it, whether we are representing it as speech, whether you're representing it as touch, whether you're representing visually, whether you're representing it with your ears, to understanding what is it. There's still one thing that can be projected into a bunch of different media so getting at that thing is the thing.

ROBERT: That was a long, winded way of saying of what is UI engineering and what is unique about it but why would you want to plant your business in UI engineering?

CHARLES: Right. That's a great question. Why is it a good business to be in and why is it a good feel to study? I think I'll quote one of my favorite video games from my youth, which was Mortal Kombat 2. At the very beginning, they would say, "There is no knowledge that is not power." That was the big quote in Mortal Kombat 2 and I'm pretty sure that the knowledge of Kung Lao, I think, being able to throw his hat by hitting some key sequence of joystick and buttons, I don't think that's knowledge that's going to give me power. Sure enough that I don't think that knowing those key sequences really had much power in them but I still think it's a good saying.

[Laughter]

CHARLES: I was always wondering what that particular quote at the Mortal Kombat programmers really like. But anyway I think that it's good to understand it as a discipline, either here or in your organization because it gives you staying power, because you understand that deep structure. For example, if the winds change and the tools get updated --

JEFFREY: As they inevitably will.

CHARLES: As they inevitably will, right. We mentioned Java, we mentioned C, we mentioned Rails, Ember, React like back down knock out, all those things, those are transitory and they're ephemeral. But that knowledge isn't and it's a rock that you can cling onto as the change of storm over the landscape of the development community. If you understand that, your transition to the next thing will be a lot easier. I think that's the business value, at least from my perspective, the ability to [inaudible] those storms and be like, "Oh, this is the same thing. It's got different clothes. It's got some nice updated patterns but I understand fundamentally what's going on. I can work with it. I can find out what the things are."

They talk about their linguistic intelligence so your second language that you learn is very hard, your third one is a little bit easier but then you have these people who speak nine languages. Once they learn and go through language four or five, it's actually a lot easier for them to pick up language and you're like, "How can they do that?" And the answer is that they understand the deep structure of language that's basically baked into our DNA. Every human being is born with it --

ROBERT: There are patterns.

CHARLES: Yeah, there are patterns and they know them and they understand them and they just absolutely have intimate knowledge of them so the stuff that gets put on the window dressing, they can see beneath that like X-ray vision and they can pick that up and work with it and run with it and it becomes mostly just an exercise in learning vocabulary.

ROBERT: And for as long as users will be interacting with the things that we build, there will always be UI engineering. There will always be somebody out there needing work.

CHARLES: Exactly. Do you want people who are fatigued by the JavaScript churn, so to speak? There are a lot of different strategies to dealing with JavaScript fatigue but a great one that I know of is to understand the core principles that are going on beneath the tools and realize that the tools really are the icing on the cake, the tip of iceberg, so to speak.

JEFFREY: This kind of brings us back around full circle to what originally started this conversation on our office is how The Frontside brands itself. We have, for the past few years, been doing almost exclusively Ember work. We love Ember as a tool and a framework that's done really good things for us and we've enjoyed working with. But we want to not marry ourselves so tightly to a tool like that because we believe in the problems and the concepts more than the tools themselves.

CHARLES: Exactly. We want to be married to the problems not the tools because what we're really looking for is for when someone has a special interaction that they want to see made real when they have a special product that they've maybe spent a lot of time and energy and thought and maybe money on crafting the user experience. We want those people to think of us because they're saying, "Now it's time to build the UI."

Before there is even an inkling of what the toolset will be, what the implementation strategy will be, we want to be at the first and foremost at people's minds. Regardless of how it's going to pan out, we know that these folks can get it done. They are going to implement this thing that is going to be as good or not even as better than the way that we envisioned it. I think that's important so we've been trying to transition towards that type of message.

ROBERT: We talked about how UI has these things that are long-lived. One of the really long-lived things and I think is going to be here for a long time is JavaScript. We do JavaScript very well. Everything we've been doing since, I would say what? Mid-2015?

CHARLES: Yeah.

ROBERT: -- Has been to abstract away from the UI framework and just base JavaScript libraries because that can be portable to anywhere. That can go from the server, on Node, all the way to any UI framework that you want and even native now with native script in Angular or React Native.

CHARLES: It's true. It's amazing how portable JavaScript is.

ROBERT: We are betting on JavaScript UI and I have gotten a couple of tweets from people because they know that we're an Ember shop really and when we talk about React, they're like, "Are you just a React shop?" I was like, "No. That's a big distinction when [inaudible]."

JEFFREY: -- UI shop?

ROBERT: Yeah. We want to be able to pick the framework or library that we see fit for your project and it doesn't matter if it's an Ember or Angular or Vue or insert next year's framework because it's going to be written in JavaScript or a superset of JavaScript like TypeScript.

CHARLES: Exactly. We love all of those frameworks and we think that they have their individual tradeoffs but we have to think about what those tradeoffs are. Also, think about how can we share as for example, this Ember app. What can be shared between an Ember app and React Native app and you might think, it's not much. But my guess is probably a lot, like most of it could be. That's kind of a radical idea but at the same time, it's a very simple one and seeing that pan out, I feel we've actually been accomplishing that and this isn't just something that's like smoke and mirrors, this is what we've been doing and the strategy has been working and it's been paying off.

ROBERT: We do UI well and it doesn't matter what way we write the code, at the end of the day we're still producing this interface that users interact with and that's our bread and butter. That's what we do really well. The framework is just the thing that we're using as the flavor of the month, year or whatever it might be, to get the job done but we are betting on UI and we do it well.

CHARLES: I think, it's actually a testament to a framework as how well it can adapt to plain JavaScript. The way that we model these interactions really is with simple immutable POJOs with well-known transitions to new states. That's it. It's like secret magic sauce that is so, so simple.

ROBERT: Remember earlier when I mentioned the state machines between the server side and the client side?

CHARLES: Yeah.

ROBERT: We modeled these as immutable state machines,

CHARLES: Yeah and they're so incredibly portable. I think, Ember back in mid-2015 wasn't so great at hosting these simple objects. Now in the early part of 2017, it actually is. It's very easy to write those things. If you're using Redux or React, we're kind of more friendly to POJOs from the get-go but I would say this. I think there's clear value in not putting logic inside your reducers. Like what is a reducer for?

I actually think that you are better decoupling your JavaScript from a framework like Redux, in the same way that we reap huge benefits from decoupling all of our state transition logic from any framework artifact inside of Ember. It meant that we were ready to drop in those interaction models into React. It was like boom! There was very low friction. It was extremely low friction.

ROBERT: Because these interactions are in a base JavaScript library and the UI library is what's driving those interactions. You're actually talking to this underlying JavaScript library that we wrote and abstracted away from the UI framework. But we still need that UI framework that whatever you inserted into to play those actions.

CHARLES: Yeah, exactly. You need Redux to manage the current state to basically manage your concept of time but maybe think twice about coupling your actual interaction model to a framework because it's going to be portable. This is a lot of pain that people had surrounding like active record. I remember for people in the Ruby, there was kind of 'aha' moment in the Rails communities like, "Wait, we literally don't have to dump everything into active records.

There was this, I think mass awakening, followed by this mass sign of relief. Once kind of people realized that everything you do doesn't have actually have to fit into a one of these kind of framework archetype objects, whether it'd be a reducer or a component or a route or whatever. If you have something that's truly separate, that's portable like make it portable. Once people kind of started treating active record as just, "I want to persist on something. This is what use," and things became a lot easier.

ROBERT: Because you're separating yourself from Rails and using Ruby, the language.

CHARLES: Right. I think, you said it perfect, Rob. It's not to disparage any tools. The tools are absolutely critical. In fact, the tools are most of the code. The amount of code, for example an Ember application provides to you is just massive and it's all very high value. Some people would debate that but the point is either all things that you're going to need to do but they're not things that are unique to the interactions that you're trying to provide.

I would actually like to see a version of Ember Data that doesn't depend on the Ember object model. I think being able to really separate that. It's pretty amazing library and if it saves an enormous amount of time, it would be fantastic.

ROBERT: But only Ember can ever use it.

CHARLES: But only Ember can ever use it but it is definitely a common problem and something that we miss like the 90% use cases that it covers. It's something that we miss when we're working in other frameworks. I actually think there is one more thing to touch on and this is really what separates, I think what we do from a lot of maybe agencies that are heavy on the design side but not super heavy on the UI side. I think is a major differentiator between the UX and the UI. Obviously, UI is very implementation-centric as opposed to vision-centric. I think, a good comparison is architect making blueprints versus the builder that actually has to construct the skyscraper.

CHARLES: Make it real?

ROBERT: Yeah, it has to make it real and that's what we do is make it real and There is an engineering component to it. While we have a very heavy focus on design and beauty and elegance and smooth interaction with the user and we study a lot of key user experience principles, there is a component to UI engineering which supersedes framework, supersedes design -- sorry, not supersedes but it's present, it's ubiquitous and that is bringing to bear all of the industry best practices around making quality, robust software. I think that's an important thing to point out.

It kind of goes without saying but I think it's important to mention is if you want to have all those good things and if you want to have good models, in views, in your controller, you want to work with a bunch of different frameworks. It's all for not. If you're not using all of those things that keep quality software quality, if that makes sense -- having continuous integration, having test suites, having continuous deployment and thinking about all the operations surrounding the software.

Ultimately, the UI is software just like any other system and something that we could go off and talk about for ages is how to test UI software because they are. When you've got these big stateful applications, they have their own unique testing.

ROBERT: But there's a lot of engineering problems here that are to be solved.

CHARLES: Right and I think it's important to be every bit as that. Anyway, I think that's about it on these subjects. It's just been kind of bouncing around the walls here and we're kind of think like, "Who are we? What is it that we do? What would you say that you do here?"

JEFFREY: We have access [inaudible].

[Laughter]

JEFFREY: That's what we do here.

CHARLES: Right. UI engineering, man. It's taking those UX dreams, right? And it's bringing them to life. It's making sure that they're scalable, that they're maintainable, that they're testable, that all of those things can become real, regardless of the tool.

ROBERT: And live long.

CHARLES: And live long.

ROBERT: That's the hard part. We can throw something together and match the conf.

CHARLES: Right.

ROBERT: But, "Will it continue to match the conf in three months?" is the question.

CHARLES: Hopefully so, we think so. I think we hit that mark.

ROBERT: Absolutely. It's the engineering, that's the tough part. That's the hardest part. Making it happen and making it sustainable.

CHARLES: All right. Thanks everybody for listening. We are going to be back next week. We've got some pretty neat folks stopping by so be sure to check it out.