Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Pricing

Paul Zaich

👤 Person
168 total appearances

Appearances Over Time

Podcast Appearances

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So composite monitor is basically a combination of several different metrics that you're measuring using, tuning together those with and or or statements. So maybe referencing what I was talking about before, you might want to have a custom metric that you're looking at and you want to look at how many of those are coming through, how many events are coming through.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So composite monitor is basically a combination of several different metrics that you're measuring using, tuning together those with and or or statements. So maybe referencing what I was talking about before, you might want to have a custom metric that you're looking at and you want to look at how many of those are coming through, how many events are coming through.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

And then you might also want to look at, in this case, the error rate for HTTP status, maybe how many 400 errors you're getting. relative to 200s. You could basically do something where you have an and statement between those two different measures and those Boolean evaluations. Or you could do something where you have an or.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

And then you might also want to look at, in this case, the error rate for HTTP status, maybe how many 400 errors you're getting. relative to 200s. You could basically do something where you have an and statement between those two different measures and those Boolean evaluations. Or you could do something where you have an or.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So you could say, these are basically signaling for the same type of issue that I want to alert on, but I'm going to look for these different conditions. all in the same monitor.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So you could say, these are basically signaling for the same type of issue that I want to alert on, but I'm going to look for these different conditions. all in the same monitor.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

Right. So I think it gives you the ability to tune things to potentially make something have a higher fidelity when it alerts.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

Right. So I think it gives you the ability to tune things to potentially make something have a higher fidelity when it alerts.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So you're not getting... One, you can set the thresholds actually higher and keep things... It depends how you want to use it, but you can... In this case, you could set the thresholds higher, but you could have something where it's like, well, if there aren't any errors coming through, then... Maybe we're okay with that, even though the number's a little bit lower.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So you're not getting... One, you can set the thresholds actually higher and keep things... It depends how you want to use it, but you can... In this case, you could set the thresholds higher, but you could have something where it's like, well, if there aren't any errors coming through, then... Maybe we're okay with that, even though the number's a little bit lower.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

Or you can do things where you can be more, and again, you can also tune this to be more sensitive. In this particular incident, if we had had some air monitoring around 400s, in addition to the threshold that we had that was pretty low, I think we would have been alerted on that within maybe an hour.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

Or you can do things where you can be more, and again, you can also tune this to be more sensitive. In this particular incident, if we had had some air monitoring around 400s, in addition to the threshold that we had that was pretty low, I think we would have been alerted on that within maybe an hour.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So you can do things there that give you more sensitivity without necessarily causing a lot more false alarms. And that's something that you have to just be really careful with any kind of monitor on a team. You really need to make sure that you're not creating false alarms. I'd say it's almost as important or equally important to the sensitivity of the alarm as well.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

So you can do things there that give you more sensitivity without necessarily causing a lot more false alarms. And that's something that you have to just be really careful with any kind of monitor on a team. You really need to make sure that you're not creating false alarms. I'd say it's almost as important or equally important to the sensitivity of the alarm as well.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

Because if you're creating false alarms all the time, it's just... not really give them the review that they need. So if you're doing that all the time, you're probably going to miss something inevitably when there's actually a real issue.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

Because if you're creating false alarms all the time, it's just... not really give them the review that they need. So if you're doing that all the time, you're probably going to miss something inevitably when there's actually a real issue.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

I just want to emphasize that this is a growing process that I think every team should go through. It's something that is going to evolve over time. And as your product becomes more important to customers and can use to grow, you need to just be constantly revealing what your approach is to this.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

I just want to emphasize that this is a growing process that I think every team should go through. It's something that is going to evolve over time. And as your product becomes more important to customers and can use to grow, you need to just be constantly revealing what your approach is to this.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

What's going to work for brand new product, brand new startup, brand new company isn't necessarily going to be the And it's something that you need to evaluate.

Ruby Rogues
The Sounds of Silence: Lessons From an API Outage with Paul Zaich - RUBY 652

What's going to work for brand new product, brand new startup, brand new company isn't necessarily going to be the And it's something that you need to evaluate.