In this episode of Ruby Rogues, we talk with Ufuk about how Shopify made the transition to using Sorbet and about the benefits they felt they received from implementing it. Ufuk also reveals a little bit about how Shopify transitioned to fully remote and about how that will be the default moving forward.Picks Luke - https://github.com/asdf-vm/asdfJohn - Walmart Grocery PickupDave - https://www.amazon.com/s?k=Thin+ClientsDave - Apple ARM MacMini Ufuk - TCP/IP Illustrated, Vol. 1: The Protocols (Addison-Wesley Professional Computing Series)Become a supporter of this podcast: https://www.spreaker.com/podcast/ruby-rogues--6102073/support.
Hey, Ruby Rogues fans. I'm Jamin Holmgren, co-founder of Infinite Red. I used to do a lot of Ruby back in the day, including building iOS apps with RubyMotion. Anyone remember that? These days, we're a small but experienced team of 30 React Native consultants.
We've been around since 2015, and we have a lot of experience in building React Native mobile apps and more, going all the way back to the beginning when it was first released. So if you're looking for expert help building, optimizing, deploying, and supporting a React Native app, I'd love to chat. Just go to infinite.red slash Ruby Rogues.
And don't forget to mention that you heard about us through the Ruby Rogues podcast so that we can keep sponsoring the show. Now, back to the episode.
welcome to ruby rogues today on our panel this week we have dave kimura hi everyone and we have luke stutters hi and i'm john epperson and for a guest this week we have caseid leo lu yes hi welcome would you tell us what you're famous for what you do just a little bit about you so we can get kind of going here sure i'm currently working at shopify on the ruby and rails infrastructure team
I've been at Shopify for about a year and a half. And since I've joined, I've been first on the Rails upgrade project. And then I quickly switched over to the Sorbet adoption team. And so we spearheaded the adoption of Sorbet across our monolith and across our other code bases as well. And now I'm slowly transitioning to being the team lead for the Ruby part of the team.
So the team will be responsible for contributing to MRI and contributing to survey, basically solving and maintaining and building on the open source Ruby foundations. I've been in the software development industry for over 20 years. I've worked with various statically typed, dynamically typed languages across that time. Initially, I was working in a startup.
I was one of the founding employees where we were developing voice XML based applications and we built a voice XML platform. And then we did like various networking things, voice over IP, text to speech, speech recognition, et cetera, et cetera. I had a short stint in between for a couple of years doing startup acceleration and then went back to developing software.
And so, yeah, that's me in a nutshell.
Wow. So there've been a couple of talks that you've given at RubyConf that I know about. I don't know if you've given more than that, but... Yeah.
I had one talk at RubyConf 2019 last year, and I also submitted a RailsConf 2020, the couch edition, the remote one. I also had a talk there.
Yep. Sorry about the confusion. One was on Sorbet and one was on the network. I know that I particularly, for one, am interested in hearing about your adoption at Sorbet. I think that's super interesting to me. Anything that you particularly want to note about either of those things that you're super interested in yourself?
I think we can maybe talk about the network stack video and the talk from RailsConf near the end of the show if we have time, because I think most people would be interested in Sorbet adoption and how a company with a code base as large as Shopify has adapted been successfully adopting it. I'm presuming that's what most of the listeners would be interested in.
So we can get started with that if you want.
How large is the Shopify code base? Because Shopify moved $41 billion of merchandise in 2018. $41 billion of B. How many dollars per line of code does that work out to?
I haven't calculated that. I don't know. But in case some of the listeners aren't familiar with the company, maybe I should just give a quick intro to what Shopify does. So Shopify is a leading global commerce company that provides tools to start, grow, market, and manage a retail business of any size. So it started off in 2006. code base that was built on rails pre 1.0.
So it was the time of rails when it was still being shared as zip files. And it's been growing ever since it started as an e-commerce platform. And it's grown into being a multi-touch general commerce platform. So it provides solutions both for e-commerce and also in-store commerce, in-person commerce. So it has many solutions now. It supports more than 1 million merchants now.
And throughout its lifetime, it supported 172 billion U.S.D., in total sales for all the merchants. Of course, that's lopsided to the recent here. So the figure that you quoted is probably correct, Luke. So I don't know at the top of my head.
I got it from Wikipedia, so it's almost certainly wrong.
It sounds believable, though. As for the code base, it's most probably the largest Ruby on Rails code base in the world right now. It has about 30,000 Ruby files in our core monolith. So I'll keep referring to core. So when I say core, that's our main monolith that provides almost all of the Shopify solutions.
Even though we now have some other services that operate outside of core, most of our code base and most of our operation is actually still being handled by core, which is our monolith. And that's about 30,000 Ruby files. And on each push, on each deploy, there are about 150,000 tests that run to ensure that everything is still the same. So that's the size of the code base.
That's the size of the platform.
And that's crazy. I think a lot of us would dream to get to that kind of scale. Honestly, I think it would be a nightmare. Not only do you have this super huge code base to maintain, but then just the infrastructure alone and what that would look like would just probably be really crazy. Can you speak into the infrastructure?
Are you guys using a very simple infrastructure or has it kind of gone from simple to very elaborate?
So I've only been at the company for about a year and a half, and most of the infrastructure has been in place before I joined. And I'm also still learning about some of the infrastructure that we use, but like I can tell you it's it's a Kubernetes deploy. So everything's virtualized in Docker containers and deployed to Kubernetes pods. And there are like some load balancing solutions.
And we also have this concept of pods. Pods are these logical units that host, um, a group of certain merchants in one location with their own basic infrastructure. So that's like one way of sharding the customer database and the customer load across different machines.
I'm pretty sure that like others from the company have presented at various conferences talking about that infrastructure part, but that's not necessarily my strongest point. So I don't want to really go much more in depth about that. even though the team that I'm working on, the Ruby and Rails infrastructure team, is a part of the product line in the company called
Kernel is basically responsible for ensuring that the platform is up and running. So it's responsible for all the infrastructure problems, but also it includes developer acceleration, which my team is a part of as well, which provides solutions for the developers within the company so that they can do their work better. They can produce solutions faster.
And also we provide solutions for the infrastructure part of the team as well.
Yeah, I definitely remember more than one talk at RailsConf throughout the years on how you guys' infrastructure changed. And I remember sitting at the table with some Shopify people one year who were talking me through how they were moving from Vagrant to Docker. So yeah, I do remember the story changing.
I'm not expert enough to talk about it by any means, but that's been crazy to watch from the outside. Yes. So, all right. To get us down into Sorbet, maybe from a high level, talk about what you guys did to make Sorbet happen at Shopify.
Sure. Again, maybe I should give a little bit more context as to what Sorbet is, who developed it, what it's used for. Sorbet is a gradual type checker for Ruby that was developed by the fine folks at Stripe. So Sorbet provides you the tools needed
to add type checking information to your Ruby code base, and it gives you both static type checker and a runtime type checker to ensure that your type assumptions make sense and are valid. The static type checker is the most important part of Sorbet, The static type checker is a tool. It's an executable built using C++. It has its own Ruby parser.
So it's able to parse and understand Ruby and in some ways simplify the Ruby language. So it can do those kinds of static analysis without necessarily running your code. Well, I shouldn't say necessarily. It doesn't run your code at all. So in that way, it's a static type checker. And it's built to be blazingly fast.
So it can parse, understand, and type check our whole code base, the code base, the core code base that I've been talking about of about 30,000 files in a matter of like 10, 15 seconds. And it was built with intentionally to be fast and to be as safe as possible when you're developing Ruby. Sorbet was amazing. open sourced about a year ago. So it was June, 2019 when it was open sourced.
But it was being developed by Stripe for about, I think, two years before that. And by the time I joined Shopify, we had already gotten on the early release beta program where they were doing like code sharing with Shopify. So we were able to start our adoption earlier than when it was initially open source.
Then afterwards, Stripe also got like a few other companies that were interested into the public beta program as well. But so that's why our story is a little bit different because we ended up solving some of the problems that people who might want to adopt Sorbet don't have to solve right now because it's a much more mature product.
But back when we were trying to adopt it on our code base, it was mostly a tool that was built with the Stripe code base in mind. And I can't talk too much about what Stripe code base is like, because I don't know, but from our understanding is it doesn't use Rails. It tries to use as little meta programming as possible.
So they're like really big on intention revealing code and not so much meta programming. So Sorbet was the right tool for them and they were developing to solve their own solutions, obviously initially, but because our code base is Rails, which already has a lot of meta programming. So meta programming brings in a lot of dynamic methods that can only be seen when you're running the code, right?
So, so there are lots of methods that you don't see, for example, on active record models, you don't see all the attributes, right? Because the attributes are created at runtime. All the methods that relate that are related to attributes are created at runtime. after Rails has had a chance to introspect your database tables.
Obviously, that's a big problem for Sorbet because it looks at the code, tries to understand it statically, and it says, oh, this model has no methods. And so in another part of the code base, when you're trying to do a shop.something, Sorbet says, oh, like shop doesn't have this method. So that ends up being a problem.
So one of the first things we had contributed to the Sorbet codebase was a way to have the static type checker understand some of the meta programming concepts. So our initial idea was that we could write scripts in Ruby and Sorbet could basically execute them so that the scripts could generate what methods would exist if the code had ran, but it would still be a static type checker.
It kind of tided us over that solution for a while, but it ended up slowing down the type checking process a lot, but at least it allowed us to do our own like initial adoption at the time. Another problem as we're adopting was that all the types that are coming from our gem dependencies. So our core monolith has about 400 gem dependencies, which is already like a huge maintenance problem, right?
Like we need to maintain them to be on the latest versions and everything. But it also means there are lots of classes, modules, constant definitions coming from those gems, but Sorbet doesn't go and look for class definitions inside of gem source code. It just looks at your code base. So it says, oh, you're using this foo class here or like this foo constant here, but I have no idea what foo is.
So it needs to be told what foo is, what kind of methods it has, what kind of subconstants it has, what type of a constant it is. Is it a class? Is it a module? What's its superclass? What are the mixins, et cetera, et cetera. So we needed a tool that would just take a gem, somehow introspect it, and then generate an RBI file. So it's called an RBI file. That's a Ruby interface file.
It's almost the same thing as a C header file. So it's just the definitions of all these things that I just said. Is it a class? Is it a module? What's the superclass, mixins, the methods and its arguments? So it just generates those without any of the implementation. So we had to build a tool and we open sourced it afterwards as Tapioca that does that.
It takes your gem file, it discovers all the gems, it loads them into memory. Then it does runtime reflection on all the types it can find from all those gems. And then it tries to understand what the shape of all those types are and it generates into a set of RBI files, which then Sorbet can read, and then it knows about foo coming from the gem foo or something.
So just a quick follow-up there. So you guys have this problem in Shopify because you have a bunch of gems. Did the people that created this gem... Stripe, yeah. Stripe, yes. I don't know why I couldn't think of it for a second. Did Stripe not have any gems dependencies then?
No, they did. So basically, they had a script that would kind of do the same thing that they were using internally, but it wasn't part of the Sorbet tooling at the time. So they shared the initial skeleton of that script with us. And then we ended up turning that into the tool that's now Tapioca.
And then the Sorbet team at Stripe, they actually also took that script and then built it into the Sorbet init script. which does something very similar. It also can look into gems and load types and generate Ruby interface files for you. So that ended up being a part of the tool, the Sorbet tool. But we took a slightly different route than that tool.
And so that's why we're actually using Tapioca internally rather than the Sorbet in its tooling.
Gotcha. Okay. What else did you encounter?
Yeah, so the other thing, so going back to the DSL problem, it turned out that making Sorbet run Ruby scripts as it's trying to make sense of your code base statically was a huge slowdown.
So we basically stopped doing it and then also realized that the same approach, even though it has some disadvantages, but the same approach of generating Ruby interface files for all the meta programming would be a much better fit for our problem. So recently we've been working on that problem. There are other
solutions in the same area as well one of them is for example the gem called sorbet rails that's built by people at the chan zuckerberg initiative so they had the same problems of sorbet not understanding some of the meta programming that's coming from rails again like all the column related methods on models all the association related methods on models all the the
Class methods, sorry, all the instance methods in mailers that are turned into class methods, all the helpers on your controllers, they're all happening at runtime in Rails. So Sorbet Rails was the first attempt to actually build a gem. that could generate Ruby interface files for all those things so that Sorbet can look at the Ruby interface file.
So you have your shop model or like shop.rb file, which hosts your shop model, but you also have this shop.rbi file that's generated for you, which contains all the method definitions that would be on the shop model dynamically. So having that Sorbet can say, oh yeah, okay, I know shop has this name method. So if you call shop.name, then yes, I know what that is.
And even better, if you can also create signatures for it. So if you can tell it what parameters it takes and what the return type is, then you can also keep typing those things as well. So you can build on those types as well. So Sorbet Rails was the initial solution to this.
But we also realized that we had other metaprogramming DSL concerns within our code base that obviously wasn't addressed by Sorbet Rails. So we wanted a broader framework. So we built one in-house and we actually are in the process of folding that also into Tapioca so that Tapioca becomes this one Swiss army tool of generating RBIs, either from gems or
or from all the DSL metaprogramming that's going on across your codebase. And initially we've again targeted the same Rails metaprogramming concepts. So we borrowed and built on some of the solutions from SurveyRails, but we've also built DSL generators for
other common gems across our code base and we intend to to grow that set based on contributions from the community so we have a generator for active record type store frozen records identity cache that's also a shopify gem so like rails plus a few gems will already be addressed with Tapioca.
And that was the biggest pain point, stopping us from increasing our typing even further across our cloud base.
Okay, so you guys went through all this work and effort to get Sorbet working at Shopify. So what did you get from that? What was your value? What was you guys' value proposition as best you understand it?
Yes, great question. So the obvious headline thing for Sorbet is to obviously to ensure that your code is typeset, right? So that you're not making silly mistakes, that you don't have references to classes or modules across your code base that are no longer there.
We've found many instances of that across our code base through the use of Sorbet, for example, or that you're not calling methods on constants that don't actually have those methods, or that you're not providing arguments to those methods that they don't actually accept. So all of those things, if you're working on a normal Ruby code base,
all of those concerns, you can only discover them at runtime. So you either need to have extensive testing to ensure that all those edge cases are executed on your CI, or you discover them in production. So the initial goal is obviously was to reduce those types of trivial errors as much as possible. Unfortunately, in our initial adoption phase, we couldn't validate or invalidate that hypothesis.
But we did other things to compensate for that. So there's a very understandable reason for why we couldn't conclude on that hypothesis of if we were able to reduce trivial errors is because our core monolith does more dynamic things than the stuff that we have in our code base. So it
kind of like serializes objects to the database and then reads serialized objects from the database, et cetera, et cetera. And there's no way a static type checker can like prevent you from, you know, deserializing an object that definition for which has mistakenly been remote from the cloud base, right?
So suppose you had like a user class and there was a user object serialized somewhere and then someone removes the user class and now someone tries to deserialize the user object. and then boom, your application fails, right? Because the user class doesn't exist anymore. So you can't deserialize it. So we obviously can't prevent those kinds of errors.
So there are like other dynamic things going on. And like I said, core is such a large code base and there's like so much variance across like it's failure modes that it's hard to get the signal that we were looking for among all that background.
So then we said, we actually need a smaller code base to test our hypotheses on, to see if we're able to move the needle in that type safety in any direction. So after we did the initial adoption on core, we turned our attention to a much smaller and more opinionated code base within the company. And that's a tool that we call dev. So dev is a tool that
every developer at Shopify has on their computer. And it's the tool that every developer uses during their daily work workflows. So there is a command line tool, you use it to clone Shopify repositories, then you only invoke dev up and it knows Based on a configuration file within the repo, it knows which dependencies to install, which commands to execute to bring the application up.
And it also gives you different shortcuts like dev test or dev type check, which was something we added to run various things. So dev is a very opinionated code base because it's a developer tool. It's a command line tool. Everyone uses it throughout the day. So it needs to be fast. It has, for that reason, it has no external dependencies. So it doesn't use any gems or anything.
If it needs some other code, it vendors them in. So it was like the ideal code base. And the team that's working on that code base is a small team that's been working on the same code base for a long time. So we talked to that team and we told them that we were interested in adding type annotations to the majority of the dev code base and to see what kind of an impact we could get on type safety.
That was a project that took a couple of months because it's especially hard to add typing to code that you already have because it means you need to understand how data is flowing throughout the code because you have some methods that ends up calling some other method and then combining the results from that other method by the result of another method call or something.
So you have these chains of method calls that you need to understand. what types they return to understand what types your method in question is returning. So it becomes like a arduous process to fully type an existing code base. So that took a couple of months, but we approached that very methodically.
So we took a look at the ways dev fail, and we took a look at all the ways that dev fails that's related to type related problems. Any exceptions that Dev was raising, they were always like all being captured anyway. So any exceptions that it was raising that were type errors, no method errors, or argument errors, we collated that. So we drew graphs and everything.
So that was before we started this experiment and we drew the same graphs for after we finished this experiment. And our findings are interesting. So we were actually able to completely eradicate name errors because that's the lowest hanging fruit. So the basic premise that Sorbet gives you is
to ensure that all the constants are resolved somehow so that you don't have any constants that Sorbet doesn't understand in your code base. So, and that's what a name error is, right? Like you refer to foo, but foo doesn't exist. So you get a name error. And if you're not doing like const get or something, something dynamic in your code base,
Just static analysis is enough to eradicate all the name error problems. And then we also looked at, like I said, type errors. There were a few type errors left still in the dev code base, but we did trace them back to files that weren't marked type true. And maybe I should just explain what type true, type false, all those levels are. Okay. So Sorbet is a gradual type checker.
So you don't have to convert your hold code base to be typed overnight, but you can opt into as much of Sorbet as you want. So when I said, you know, we adopted Sorbet in our core code base, of course, I'm not saying like we, you know, added types across the whole code base, but we
enabled all the developers to add those type signatures and to have their codes checked against those type signatures. So the same thing in dev. So we did add types to the majority of the dev code base, but not to all of it. And we were able to track the type errors to the parts of the code base that were where the files were marked type false. And type false says
Do not type check any of the method calls in this file. Only check that like the constants resolve to constants that you know. It does the basic type checking. Whereas in type true, the file is type checked for all the method calls and for all the parameters that are being passed correctly. And if the methods have signatures, that all the methods have the correct types.
Yeah, I was just going to say, it sounds like you were able to test more or less your premise, right? Which is that you were going to be able to eliminate all of these name errors.
Yes. So the name errors is the lowest hanging fruit because you take a typed false on top of your file and run sorbet over it. And it will tell you, oh, you have this constant that you're referring to here in this file, but I have no idea what it is. So it's either coming from somewhere else in your code base that Sorbet isn't processing, or it really doesn't exist.
So you just, you know, remove your usage of it. So name error was a simple one, but we were able to reduce type errors as well. And we also realized that we were able to reduce argument errors and no method errors as well, though not to the same extent. Because even though you're opting into types, there are still situations where when you call a method, the method doesn't have a signature.
Sorbet's best guess is it could return anything. that's represented as a T untyped in Sorbet. That's the equivalent of any in TypeScript if you've worked with TypeScript before. So it's a type that can be anything. So you can call any methods on it with any parameters. So basically Sorbet stops doing type checking past that point.
So a T untyped always returns a T untyped when you call a method on it and everything. So that kind of lessens the amount of strict typing that you're doing. You can opt into higher types, strictness levels in your files to prevent this. So you can opt into type strict, which will tell you if you're doing this unsafe calls and it will say, oh, you're actually calling into a method.
that doesn't have a type signature. So, you know, there's something wrong here. So go and add a type signature to this other method that you're calling to make sure that this file has a stricter type checking. So we didn't go that far in the dev code base. We wanted to see how much we could get with simple typing.
So there were still a few argument errors or no method errors that leaked in, but we were able to reduce those kinds of errors as well. But that's not the only thing we did. We also looked at if running type checks on CI had any effect on the CI success rate. So were there like any false positives that Sorbet was failing on that were breaking CI builds or vice versa?
Were there any test failures that Sorbet wasn't catching? And we realized that Sorbet didn't have an effect either way. So it wasn't causing superior failures on CI, nor was it really missing anything. So there weren't really that many test failures that survey also didn't catch. But we also talked to the team. So I think that's one of the most interesting things we've done.
We talked to the team both before we started this experiment individually and talked to them after we finished this experiment. to see. So before we wanted to see how much they knew about Sorbet, how they felt about Sorbet, were they enthusiastic or did they have questions about its utility or not? So we compiled those.
And then after we finished this experiment, we also asked them if they were using Sorbet type checking on their machines, if they were happy adding signatures to the codebase, or if they had problems having to deal with fixing signatures when they were factoring. So we were able to also get a qualitative feedback on how the team and how the developer happiness was impacted or not.
And we actually heard good things about that. So there was one person on the team who really didn't like signatures. They found it really distracting. One of the things about Sorbet signatures is because Sorbet is not integrated with the language. Obviously Ruby as a language doesn't support types. So you need to add a signature annotation on top of your method definitions.
And that signature annotation is also Ruby code. So you do like a sig block. So you say sig and then you start a block and then you say params and you declare your parameters and their types. And then you do a dot returns and then you declare the type of your return. And then you can do more things in there as well.
But obviously sometimes for a two or three line method, your signature can end up being five or six lines if it's doing something like non-trivial or if your types have like some generics or something. So some people actually find that. to their posts and sometimes distracting. So they say that it makes it hard to actually see the code.
They end up seeing like all these signatures, but that person on the team actually developed like a few Vim configs to de-emphasize the signatures. So they basically turned the opacity on the signatures so that they look like comments, which they kind of are, right? Like, because that's the same thing with Yard definitions. So when you add type annotations in Yard comments, that's the same thing.
And then someone at the company also built a Visual Studio Code extension to do the same thing. I think it's called Vysig. So it parses the signature blocks and then turns the opacity on. And I've actually been using it, and I'm also enjoying it because I don't necessarily need the signatures to be in front of me all the time.
But apart from that, we heard that developers actually like the safety guarantees that static typing gives them as they're doing refactoring, for example. So for example, if they're removing a class or a module from the code base, they're almost guaranteed to know that when they run the static type checker, that it will complain if there's like any dangling usages of that class or module.
So they really liked those guarantees and they didn't really mind writing the signatures in the first place. And it's a really easy ramp on as well on the signature syntax. So we actually heard good things from developers on working on the code base too.
So has using Sorbet made you guys write better code?
Yes. That's also another one of our findings. We also talked to three different teams who were working in core, and they were one of the early adopters of Sorbet in their part of the code base. So we have this componentization thing going on in our core, in our monolith, because it's a huge code base. Otherwise, it becomes mayhem.
So there's these components, and some components were the early adopters of typing. So we did interviews with team members from those teams as well to understand how they felt about those early initiatives. And one of the things that we ended up hearing over and over was that adopting types led to better code design, which initially was interesting for me.
But that seems to be the biggest benefit from adopting static or gradual typing rather than runtime type safety. And so my opinion is a little bit swayed now. Because too often when we're working on simple solutions, we reach out for like these bags of values like arrays or like most of the time hashes, right? So we just like reach for a hash and then we keep passing those hashes around.
But then it's never documented what the keys are. Like if let alone if the keys are symbols or string. How many times did we assume that we could access some hash using symbols? It turns out to be that it was indexed by strings. When your code base is small, that's fine. I'm not judging anyone who's doing that. But as your code base is growing,
you need people to more easily onboard onto your team. And if they're fixing something and if their focus is on, let's say one method or one file, you don't want them to go hunting around seeing, oh, where's this hash actually being coming from? Where was it created initially? What are all the parameters that are in this hash? So that's not like a very efficient way of working at that scale.
So at that scale, it's better to give a name to that, to turn that into something like a structure or create class for it or something to basically start using types, right? Once you turn something that was a hash into a struct or a class or a module, you give a name to it, you give a type to it. So you start using types.
And those types actually convey more information than passing around hashes would. So it leads to actually better code design. And we've heard that over and over again from various people within the company. But it also leads to evergreen documentation as well.
So if you're looking at a method and it has a signature on top of it, and it says these are the parameters that it takes of these types and it returns this type, then you don't need to go and look anywhere else, basically, if you know what those other types are as well. You don't need to go look anywhere else, like to all the colors of this method or whatever. It's just evergreen documentation.
And the reason why it's evergreen is because if somebody changes the method signature, they have to change the signature on top of the method as well. Otherwise you will have like the static type checker breaking in CI, right? So it has to be evergreen. People need to keep it up to date. And that makes it very easy to onboard new people to your code base or to your team or to your company as well.
So we heard from developers that those were all great benefits of adopting types. And like I said, we didn't adopt types across the whole code base, but we're allowing people to adopt those types gradually as much as they want. And our team is also helping people within the company in their adoption process. So sometimes we help them We review their PRs.
Sometimes we actually pair with them to make that adoption easier. Or we just answer questions in a Slack channel when they're stuck.
I hate types. Sure. I hate them. Some people. I spent years trying to get away from them. The only reason I still program in Ruby is because it just doesn't happen. That's the kind of, it's why I like it. It's this freedom from types. So my question is, what's wrong with Ruby? Absolutely. What's wrong with Ruby? Because this is obviously not just a Shopify project.
This is something Stripe also looking at. So there must be some kind of common problem that led you both to say, we can improve our product. We can improve our developer experience. We can find problems sooner. We can have this kind of really kind of pickup and productivity by introducing this system on top of Stripe. Ruby, and it's a pretty clean system.
There's a link to a little, what do you call it, a sandbox? Yes. A playground where you can try it out. What I really like about it, as you said, is that you don't have to go through your whole code base and start typing in types for everything. If you've got something, a mistake that you keep making in a project, then you can just drop this in.
on that problem class, that problem method, and then no one will ever have an excuse for breaking it again. So it's a real hammer to kind of drop on that. But what's your opinion? I know you talked a lot about the developers and how they feel about introducing typing and what they got from it. This is something you must feel very passionately about, right? Where do you stand on Ruby and types?
Okay, so personally, just before I joined Shopify, I was working on code bases that were built by TypeScript. And actually, TypeScript was, I think there is something wrong with me, and I'll come to that.
So TypeScript was, I was at the time working in a web agency, but I was on the team that were doing like cutting edge two week sprints that were like trying to see if we could make certain things work. And we were mostly working on JavaScript and I consciously chose to try TypeScript to see how it would work. And TypeScript coupled with Visual Studio Code was a joy to use.
And I started loving it for all the code completion and all the type things. And you don't look at the documentation a lot.
Anyway, like I was really enjoying TypeScript before I joined Shopify and I had already heard about Sorbet and like Sorbet got me similarly excited because it has, it still has the promise of doing the same thing that TypeScript did for JavaScript, which was not to destroy the language as a language.
But to add features on top of it that people could opt into if they wanted or needed more strict typing. And it's again, this whole thing, because TypeScript is a hundred percent JavaScript plus some other things. The difference with TypeScript is it compiles into JavaScript. So it does not type checking at runtime. Whereas with Sorbet. It's built inside of Ruby.
So all the sorbet annotations and everything are also Ruby constructs. So those Ruby constructs can also do type checking or type assertions at runtime. So that's the kind of difference, but looking back, I kind of see myself having almost exclusively worked in code bases that had some types somewhere. So I'd worked with Delphi for a while and then C++, C Sharp. And then, like I said, TypeScript.
But before TypeScript, we were working on a PHP code base that was using type hints. So that was a PHP 5.3 code base, if I remember correctly, which had type hints. And we were really relying on type hints. So the project that started before I joined and it was built like that. And I ended up feeling like that was a good way to work because it was preventing us from shooting ourselves in the foot.
Even though that was still doing type checking at runtime, it was still failing fast, right? So that's like one of the principles that we software developers use a lot, this ability to fail fast. So static type checking really gives you that. So it allows you to fail fast.
Either in our base case, you just run the static type checker and within a couple of seconds, it shows you if there's like some assumptions in your code base that don't check out. And that's way better than waiting on the CI for a couple of minutes. you can just run this tool constantly on your machine, right? Right.
So this is where it kind of seems to differ in terms of developer tool to running a test suite. This is not really the same as running a series of tests. This is something which you can use kind of almost kind of running continuing to black in the background in Visual Studio Code. Is that how you use it? Yes.
So Sorbet actually has an LSP mode. So the Sorbet binary, that's the C++ static checker binary. If you download the Sorbet static gem or the Sorbet gem, which pulls it in, you can actually do srbtc dash dash LSP, which turns on the LSP mode. So LSP stands for the language server protocol, I think.
I'm not so sure about the acronym, but it's developed by people at Microsoft as they were developing Visual Studio code. They developed that specification, so it allows different languages to integrate with different editors, and LSP is the language that sits in between.
So Sorbet has this LSP server mode, and they're also working on an official Visual Studio Code extension that you can install that would bring all that richness into Visual Studio Code. And I'm one of the early testers of that inside of Shopify. And I've been using it for a while now and it's actually integrated into your editor.
So it gives you the best auto-complete that you can ever find, way better than like all the other editors can do. But it also shows you all the errors within your editor. That's a great way to work because you get instant feedback on your code changes. I don't want to give too long an answer, but just to address the second part of your question is like, when would you reach for a tool like this?
So scale is very important. So when you have like in our code base, tens of different components and 30,000 different files, then it becomes really important to have better contracts between all those different moving parts. And what better contract than types? And we've actually realized that types really work well for that.
And I think that was the initial impetus for why Stripe started working on this as well. And I'm not saying that all Ruby codes should be typed or that everyone should reach for typing as soon as possible. Obviously it's a matter of needs and scale. But I also feel like that it's the right tools for the scale that Shopify is in right now.
And the teams that are really relying on types are getting a really good return for their investment.
So obviously we're just coming out of the COVID-19 time. And I understand that Shopify has changed their working practices. What's it been like at Shopify over the last few months?
Actually, Shopify's response to this whole pandemic has been really impressive because senior executives at Shopify were aware of where the whole pandemic situation was going really early on. And they were really quick in taking measures to ensure that people working for Shopify are safe. So that was very early on. So they basically closed all the offices.
before like most countries were like aware of what's even what's going on. So for me personally, I can't quite talk for those office closures because I've always been working remotely for Shopify. So I live in Cyprus. I work for Shopify remotely from here and our whole Ruby and Rails infrastructure team has been distributed, has been a distributed team as well.
So we had a couple of people working in the offices in Ottawa. Someone in Montreal, one person in Poland, one person in France, UK, Cyprus, all over the world. So our whole team dynamics and everything didn't obviously change that much.
But obviously, as everyone's feeling it, starting to work remotely during the pandemic was a huge burden to a lot of people because they weren't ready to work like that. Also, it wasn't a normal working remotely setup either, right? Because Schools are also closed. Everything else is closed. You're trapped in this house and you can't go out. And you're also trying to do work.
So it's not necessarily the normal working from home situation. But like you said, our senior executives looked at the situation going forward. So first of all, Shopify isn't reopening any of their offices. before the end of this year, whatever happens, because the outlook isn't that great. It doesn't look like this pandemic is over anytime soon.
So there's no point in like opening offices and making people feel safe in going there just to then like close them again. But also, the company has, based on this experience that we've had for the last couple of months, the company has also decided to be digital by default, which doesn't necessarily... It's digital already. It is digital already, but it wasn't digital by default, right?
So that's the extension. So what used to be the case is, for example, if there were people in the office and if there was a meeting, so if there were like 10 people in the office and three people joining remotely... then the 10 people in the office would be huddled up in a meeting room, and then the three people would join in on a call remotely. But that isn't necessarily the best way of working.
because the people who are joining that call remotely don't get the same experience that those 10 people in the room get. So the digital by default kind of changes that. And they say, we're not closing our offices, but they're saying the offices won't be the same offices. So they'll be revamped to accommodate this new digital by default future.
But basically then in that future, we'll get to experience it obviously, but in that future, those 10 people will also individually join that same meeting as well. So everyone will be digital in the meeting by default rather than having 10 people co-located somewhere and then three people remote.
So that's the biggest difference because the decision there was very simple because you either do fully co-located or you do fully remote. In between is this hybrid thing that doesn't work that well for either party, right?
You have a few people that are co-located in an office, a few people that are remote, but then you need to have all your communication digital to ensure that the remote people have the same context as the people who are co-located. But people who are co-located, if they have chats that they have in between and they don't report them back, then you have information asymmetry, et cetera, et cetera.
That's the worst way to work. I've also worked remotely, being the only remote person on a team that was co-located before, not at Shopify. And that wasn't a horrible experience because my team was very understanding of my situation. But it's still not the best way to work when all your teammates are all in the same office and you're the only one remote.
So the future for Shopify is digital by default for everyone. So even if you're in the office, you're still communicating with everyone digitally. That will be the primary mode of communication. And most of our work will not be done from offices anyway.
Sweet. Then let's roll on into PIX then. Dave, would you start us off with PIX this week?
Yeah, real quick. If people want to follow some of the things that you're doing online, where should they go and look?
All the engineering work that we're doing, we have a Shopify engineering blog. That mostly gets a new blog post once a week, I think. Or the Shopify engineering Twitter account. They can follow. And I should also say that Shopify is still hiring and we've been hiring. Actually, we ramped up our hiring across the pandemic as well.
So if anyone's interested in a position at Shopify, also now that we're digital by default, we're more open to remote people. They can go to Shopify.com slash careers or our careers page on our website. And they can be a part of the team so that they can follow the developments from the inside.
Awesome. Yeah, and I'll go ahead and kick us off with some picks. So my first pick is the wise, thin clients. So I've been getting my kids ready for the digital learning in the upcoming school year. And instead of giving them each a... $600 computer or something, I have deployed a thin client set up at home, which is probably way overkill for most home networks and stuff.
But I would much rather replace a $100 all-in-one thin client than a $600 computer and monitor. So that's my first pick. And second pick is I, on this call actually, I got the announcement that I got the Apple TDK. So the developer transition kit with the ARM-based processor. So I will be releasing some Drifting Ruby videos on developing on ARM-based processors soon.
That's pretty cool. Nice. I'll be pointing some people that I know are super interested in that to you, actually.
Yeah, we're really interested in that on my team as well, because the people are doing Drifting Ruby and MRI work. want to make sure that both of those platforms run really well on the new ARM chip.
Luke, what do you have for us this week?
Well, to tie in with Ufuk's talk at Rails Conf 2020 Couch Edition, which talk is called Peeling Away the Layers of a Network Stack. It's a good talk. My pick is the evergreen TCP IP Illustrated, a big book of how networking works. If you want to learn how to perform hilarious office pranks like ARP poisoning,
If you want to get to grips with networking, this is a great book to really drill into detail. So there we go. TCPIP Illustrated, recommended to me many years ago by a man from Florida. And boy, was he right.
It's a great read. Awesome. So I have a couple of picks for this week. One is actually something that's been out for quite some time now, but I really wasn't introduced to it until very recently. It's called ASDF.
If you're familiar with RBM or RVM or any other version manager or any language, ASDF is more or less a similar thing, except that it's really just cross-language, cross-tool kind of thing. They have plugins for things like Postgres and MySQL.
I don't really see a lot of value with Postgres personally, but I did see, I was trying this out on a project that had MySQL and switching between versions is legit.
Basically, the thing about this tool is it's really for making it so that you can set up your development machine to switch between versions of Python and Ruby and Node and your database that you have and a whole bunch of different tools. I mostly use Docker these days. So this really isn't like a thing that I'm going to use all the time.
But I do have a couple like outlying projects that are just fun. And so I explicitly, I was introduced to this. I was like, all right, I'm gonna try this out. It's legit. So I don't know that I'm ready to completely give up RVM for this yet. Mostly because I do almost all Ruby work. But it's definitely taking some mindshare up in my brain at this point. So it's pretty cool.
Definitely recommend checking that out.