Daniel's notes on An Elegant Puzzle

Teams are slotted into 4 states: A team is falling behind if each week their backlog is longer than it was the week before. Typically, people are working extremely hard but not making much progress, morale is low, and your users are vocally dissatisfied. A team is treading water if they're able to get their critical work done but are not able to start paying down technical debt or begin major new projects. Morale is a bit higher, but people are still working hard, and your users may seem happier because they've learned that asking for help won't go anywhere. A team is repaying debt when they're able to start paying down technical debt and are beginning to benefit from the debt-repayment snowball: each piece of debt you repay leads to more time to repay more debt.

Fundamentally, I believe that sustained productivity comes from high-performing teams, and that disassembling a high-performing team leads to a significant loss of productivity, even if the members are fully retained. In this worldview, high-performing teams are sacred, and I'm quite hesitant to disassemble them.

Teams take a long time to jell. When a group has been working together for a few years, they understand each other and know how to set each other up for success in a truly remarkable way. Shifting individuals across teams can reset the clock on jelling, especially for teams in the early stages of jelling, and when there are significant differences in team culture. That's not to say that you want teams to never change-that leads to stagnation-but perhaps preserving a team's jelled state requires restrained growth.

Okay, so what does work? I've found it most fruitful to move scope between teams, preserving the teams themselves. If a team has significant slack, then incrementally move responsibility to them, at which point they'll start locally optimizing their expanded workload. It's best to do this slowly to maintain slack in the team, but if it's a choice of moving people rapidly or shifting scope rapidly, l've found that the latter is more effective and less disruptive.

Shifting scope works better than moving people because it avoids re-jelling costs, and it preserves system behavior. Preserving behavior keeps your existing mental model intact, and if it doesn't work out, you can always revert a workload change with less disruption than would be caused by a staffing change.

In my opinion, probably the most important opportunity is designing your software to be flexible because the best system rewrite is the one that didn't happen, and if you can avoid baking in arbitrary policy decisions that will change frequently over time, then you are much more likely to be able to keep using a system for the long term.

Finally, a related antipattern is the gatekeeper pattern. Having humans who perform gatekeeping activities creates very odd social dynamics, and is rarely a great use of a human's time. When at all possible, build systems with sufficient isolation that you can allow most actions to go forward. And when they do occasionally fail, make sure that they fail with a limited blast radius.

Developer velocity: 4 measures of developer velocity: 1. Delivery Lead Time is the time from the creation of code to its use in production. 2. Deployment Frequency is how often you deploy code. 3. Change Fail Rate is how frequently changes fail. 4. Time to Restore Service is the time spent recovering from defects.

we can model these 4 velocity measures into a system that we can use to reason about developer productivity: • Pull requests are converted into ready commits based on our code review rate. • Ready commits convert into deployed commits at deploy rate. • Deployed commits convert into incidents at defect rate. • Incidents are remediated into reverted commits at recovery rate. • Reverted commits are debugged into new pull requests at debug rate.

If your model is a good one, opportunities for improvement should be immediately obvious. However, to truly identify where to invest, you need to identify the true values of these stocks (pull requests, ready commits, deployed commits, incidents and reverted commits) and flows (coding rate, code review rate, deploy rate, defect rate, recovery rate, debug rate)! For example, if you don't have a backlog of ready commits, then speeding up your deploy rate may not be valuable. Likewise, if your defect rate is very low, then reducing your recovery time will have little impact on the system.

Problem discovery: The themes that I've found useful for populating the problem space are: Users' pain. What are the problems that your users experience? It's useful to go broad via survey mechanisms, as well as to go deep by interviewing a smaller set of interesting individuals across different user segments. Users' purpose. What motivates your users to engage with your sys-tems? How can you better enable users to accomplish their goals? Benchmark. Look at how your company compares to competitors in the same and similar industries. Are there areas in which you are quite weak? Those are areas to consider investing in.

Problem discovery: Cohorts. What is hiding behind your clean distributions? Exploring your data for the cohorts hidden behind top-level analysis is an effective way to discover new kinds of users with surprising needs. Competitive advantages. By understanding the areas you're exceptionally strong in, you can identify opportunities that you're better positioned to fill than other companies. Competitive moats. Moats are a more extreme version of a competitive advantage. Moats represent a sustaining competitive advantage, which makes it possible for you to pursue offerings that others simply cannot. • What do your existing moats enable you to do today? • What are the potential moats you could build for the future? • What moats are your competitors luxuriating behind?

Problem discovery: Compounding leverage. What are the composable blocks you could start building today that would compound into major product or technical leverage" over time? I think of this category of work as finding ways to get the benefit at least twice. These are potentially tasks that initially don't seem important enough to prioritize, but whose compounding value makes the work possible to prioritize.

Solution validation: Write a customer letter. Write the launch announcement that you would send after finishing the solution. Are you able to write something exciting, useful, and real? It's much more useful to test it against your actual users than to rely on your intuition. Find reference users. Can you find users who are willing to be the first users for the solution? If you can't, you should be skeptical whether what you're building is worthwhile. Find the path more quickly traveled. The most expensive way to validate a solution is to build it in its entirety. The upside of that approach is that you've lost no time if you picked a good solution. The downside is that you've sacrificed a huge amount of time if it's not. Try to find the cheapest way to validate.

Solution validation: Justify switching costs. What will the costs of switching be for users who move to your solution? Even if folks want to use it, high switching costs may mean that they simply won't be able to. Test with your potential users if they'd be willing to pay the full cost of migrating to your solution instead of their existing planned work.

What I needed was a way to coordinate my approach across teams, both in terms of very specific challenges and in terms of our long-term direction. After experimenting with a handful of different approaches, agreeing on strategy and vision has been the most effective approach that I've found to alignment at scale.

There is a moment in every company's growth when top-level planning shifts from discussing specific projects to talking about goals. This happens recursively across each scope of leadership, as areas of accountability become too broad or complex for their leaders to consistently understand every project's details. This can be a very empowering moment because goals decouple the "what" from the "how," but it can also be a confusing transition for everyone involved: writing clear goals takes a bit of practice.

Good goals are a composition of four specific kinds of numbers: 1. A target states where you want to reach. 2. A baseline identifies where you are today. 3. A trend describes the current velocity. 4. A time frame sets bounds for the change. Put these all together, and a well-structured goal takes the form of: "In Q3, we will reduce time to render our frontpage from 600ms (p95) to 300ms (p95). In Q2, render time increased from 500ms to 600ms."