An Elegant Puzzle

Good goals are a composition of four specific kinds of numbers: 1. A target states where you want to reach. 2. A baseline identifies where you are today. 3. A trend describes the current velocity. 4. A time frame sets bounds for the change. Put these all together, and a well-structured goal takes the form of: "In Q3, we will reduce time to render our frontpage from 600ms (p95) to 300ms (p95). In Q2, render time increased from 500ms to 600ms."

There is a moment in every company's growth when top-level planning shifts from discussing specific projects to talking about goals. This happens recursively across each scope of leadership, as areas of accountability become too broad or complex for their leaders to consistently understand every project's details. This can be a very empowering moment because goals decouple the "what" from the "how," but it can also be a confusing transition for everyone involved: writing clear goals takes a bit of practice.

What I needed was a way to coordinate my approach across teams, both in terms of very specific challenges and in terms of our long-term direction. After experimenting with a handful of different approaches, agreeing on strategy and vision has been the most effective approach that I've found to alignment at scale.

Solution validation: Justify switching costs. What will the costs of switching be for users who move to your solution? Even if folks want to use it, high switching costs may mean that they simply won't be able to. Test with your potential users if they'd be willing to pay the full cost of migrating to your solution instead of their existing planned work.

Solution validation: Write a customer letter. Write the launch announcement that you would send after finishing the solution. Are you able to write something exciting, useful, and real? It's much more useful to test it against your actual users than to rely on your intuition. Find reference users. Can you find users who are willing to be the first users for the solution? If you can't, you should be skeptical whether what you're building is worthwhile. Find the path more quickly traveled. The most expensive way to validate a solution is to build it in its entirety. The upside of that approach is that you've lost no time if you picked a good solution. The downside is that you've sacrificed a huge amount of time if it's not. Try to find the cheapest way to validate.

Problem discovery: Compounding leverage. What are the composable blocks you could start building today that would compound into major product or technical leverage" over time? I think of this category of work as finding ways to get the benefit at least twice. These are potentially tasks that initially don't seem important enough to prioritize, but whose compounding value makes the work possible to prioritize.

Problem discovery: Cohorts. What is hiding behind your clean distributions? Exploring your data for the cohorts hidden behind top-level analysis is an effective way to discover new kinds of users with surprising needs. Competitive advantages. By understanding the areas you're exceptionally strong in, you can identify opportunities that you're better positioned to fill than other companies. Competitive moats. Moats are a more extreme version of a competitive advantage. Moats represent a sustaining competitive advantage, which makes it possible for you to pursue offerings that others simply cannot. • What do your existing moats enable you to do today? • What are the potential moats you could build for the future? • What moats are your competitors luxuriating behind?

Problem discovery: The themes that I've found useful for populating the problem space are: Users' pain. What are the problems that your users experience? It's useful to go broad via survey mechanisms, as well as to go deep by interviewing a smaller set of interesting individuals across different user segments. Users' purpose. What motivates your users to engage with your sys-tems? How can you better enable users to accomplish their goals? Benchmark. Look at how your company compares to competitors in the same and similar industries. Are there areas in which you are quite weak? Those are areas to consider investing in.

If your model is a good one, opportunities for improvement should be immediately obvious. However, to truly identify where to invest, you need to identify the true values of these stocks (pull requests, ready commits, deployed commits, incidents and reverted commits) and flows (coding rate, code review rate, deploy rate, defect rate, recovery rate, debug rate)! For example, if you don't have a backlog of ready commits, then speeding up your deploy rate may not be valuable. Likewise, if your defect rate is very low, then reducing your recovery time will have little impact on the system.

we can model these 4 velocity measures into a system that we can use to reason about developer productivity: • Pull requests are converted into ready commits based on our code review rate. • Ready commits convert into deployed commits at deploy rate. • Deployed commits convert into incidents at defect rate. • Incidents are remediated into reverted commits at recovery rate. • Reverted commits are debugged into new pull requests at debug rate.

Developer velocity: 4 measures of developer velocity: 1. Delivery Lead Time is the time from the creation of code to its use in production. 2. Deployment Frequency is how often you deploy code. 3. Change Fail Rate is how frequently changes fail. 4. Time to Restore Service is the time spent recovering from defects.

Finally, a related antipattern is the gatekeeper pattern. Having humans who perform gatekeeping activities creates very odd social dynamics, and is rarely a great use of a human's time. When at all possible, build systems with sufficient isolation that you can allow most actions to go forward. And when they do occasionally fail, make sure that they fail with a limited blast radius.