What is a Multi-armed Bandit Experiment?
Problem Intuition
Let’s imagine you’ve decided to hit up the casino and try your luck at the slots. You don’t know which slots are best, but you have reason to believe that some of the slot machines are better than others. You only have a limited amount of money to play with, so you want to play each of the machines enough to learn which is going to be best (explore), and then as you learn, play the best machines most frequently to maximize your return (exploit). Choosing which machine to play is referred to as the exploration / exploitation dilemma.
Read More
Tags:
experiments,
ruby,
engineering,
kafka,
dynamodb
Oftentimes database tables begin with the best of intentions. They can start out narrow and clearly named with well-defined content, but as our applications and teams grow, we may start to notice that those same tables begin to take on a different shape. It’s possible that they grow wider and are harder to work with than you remember. They could be holding data around multiple concepts, which might mean your models are breaking the single responsibility principle. Perhaps that table didn’t start out concise at all. There are many common scenarios that cause tables to exist in a cumbersome form, but it doesn’t mean they have to stay that way.
Read More
Tags:
ruby,
development,
engineering,
rails,
activerecord,
migrations,
database
A little while ago, we decided to define and document ezCater's pull request culture in an effort to streamline onboarding and facilitate collaboration as our team doubles, quadruples, and beyond in size. Clearly communicating both how we operate and also why we do things the way we do, sets people up for success from day one. We're proud that new hires are able to contribute to the codebase during their first week and are always looking for ways to simplify onboarding.
Read More
Tags:
ezcater,
culture,
values,
github,
development,
engineering,
coding,
creating
Charting a Path from Redshift to Snowflake
ezCater is a data-driven company. For a long time, the principal source for our metrics was our Redshift data warehouse. As a result, we invested heavily in building out the data systems that fed Redshift. We also staffed up a team of engineers dedicated to building, enhancing, and growing our data systems. (And yes, we're hiring!)
Read More
Tags:
data warehousing,
snowflake,
ruby
...and what you can do instead!
We've been working with React for a few years now here at ezCater, during which time our unit testing story has been steadily evolving. One of the recent strategies we've tried out is Jest snapshot testing.
Read More
Tags:
unit testing,
jest,
snapshot testing,
javascript,
react
Working at ezCater is an unbelievable opportunity to witness scaling close up. Requests, transactions, revenue, it’s hard to find something not going up and to the right. That’s great, but in each area of growth requires figuring out how to scale it.
The Hardest Thing to Scale
The particular metric that was keeping me up at night was the “number of engineers trying to work together in harmony”. Ten, twenty, forty engineers are simply different scales and they require different systems to ensure things move smoothly. But how would we even describe a “happily scaling” organization from a developer productivity standpoint?
Tom Petr addresses this in his slides on Maximizing Developer Productivity in a Microservices Environment. This presentation shows the KPIs & shape of the curves I want for our organization.
Read More
Tags:
devops,
kubernetes
At ezCater, we believe that designers should care deeply about using data at all stages of the design process. Whether qualitative or quantitative, it helps to inform and empower our design decisions.
Data is a powerful tool that allows us to:
- Validate that we are solving the right problem
- Test small assumptions to get to the right solution with quantifiable impact
- Understand what the business cares - and should care - about, which influences design strategy
Product manager Lindsay Matthews and I gave a talk at the recent UXPA Boston conference about all the ways that designers can use data to be more effective. Learn more in the slides below:
Read More
Tags:
experiments,
women_in_tech,
ezcater,
design
What did we learn about machine learning?
This is part 2 in a series about Estimating the LTV of new SaaS customers. To do this we're taking all the datapoints we have about a user after X days and seeing whether we can predict how much revenue they'll spend in their first year.
In Early Prediction of Lifetime Value (LTV) we left off when we had a CSV full of features and the Year1Revenue(Y1R) that we were trying to predict. It was time to put the pedal down and see whether machine learning is all it's cracked up to be.
Read More
Tags:
machine learning,
ltv
It begins: The promotion
You’re a SaaS business. You've been acquiring customers and on average they're worth about X. Somebody clever suggests a promotion, "$5off your first order".
Read More
Tags:
machine learning,
ltv,
growth
