Apologies for the long delay in posts. The run-up to and through the summer has been eventful, especially on the personal front where I have been spending about two weeks living and working (remotely) from Israel. Quite a change of pace from being in either Chicago or Boston. There’s been plenty of fodder for posts, but not so much time to write.
However, I decided it was worthwhile to take some time out to write to help me collect my thoughts on planning. For the past 5 or 6 months, I made a push to try to organize how we complete “infrastructure” projects at work. I define infrastructure projects as those that don’t deliver a specific tradable strategy, but instead provide tools or reports that will be used to help accelerate the creation of or improve the quality of these more tangible deliverables.
There is broad agreement that infrastructure work is critical to the success of our organization. Yet, somehow, we find it difficult to consistently make substantive progress against our infrastructure goals. I don’t think that this is for lack of definition on the goals — I spent many hours working with our CTO to hammer out and prioritize potential projects. I suspect that there are a number of complex issues that are limiting our progress, at least one of which is a lack of committed focus from developers to infrastructure projects. While I have been primarily focused on building and extending our infrastructure, most other developers have split their time with the balance typically tilted more towards working on delivering and maintaining trading strategies. The theoretical 25 to 50 percent that is left for infrastructure work seems to often get lost to fighting fires or just one more analysis. With this in mind, my CTO and I determined that we would start making plans that did not count on the contributions from developers who didn’t have the majority of their time committed to the infrastructure development projects. I am sure we will still get some interesting contributions from the other developers, but it will be more opportunistic and less planned.
The question that remains is how my infrastructure-focused team will organize itself. Previous to my current role, I spent a few years leading development teams that followed the Scrum project management methodology. As a Certified ScrumMaster, it seems appropriate that I might return to running sprints with my team. To that end, I suggested that our CTO check out Agile Project Management with Scrum, which I thought was the most accessible of the books that I have read/skimmed on the subject. I am interested to hear his reaction after spending some time reading and thinking about how to Scrum.
That’s all well and good, but I’m not convinced that we should go to Scrum without some more thought. While my previous teams have been successful with Scrum, it always seemed that there was room for improvement – even beyond what could/would be achieved through the team’s monthly retrospective efforts. For example, it often felt that the team’s planning process was relatively forced. As the ScrumMaster and the manager of the team, I often felt like I had to push hard to get real, consistent involvement in planning from team members. We experimented with other approaches to planning that were suggested by some of my great team members, for example, assigning out “pre-planning” work to individuals or pairs from the team, so there would be a de facto technical owner other than me who would be engaged and well-informed and could therefore help to push ahead the planning meeting for each story. I think that helped, though I wonder whether we lost some of the benefits of Scrum by doing this in that others on the team may have felt like they were not as accountable for those stories. The right answer is probably to keep tweaking small things to make the team more effective, but perhaps the retrospective questions shouldn’t be “how to scrum better”, but simply “how to work better”.
The main contender to the Agile project management throne, as far as I know, is Kanban. I haven’t used it, but plan to read up on it over the next few days to figure out if it might be a better fit for our organization. Or, more likely, whether it would be worthwhile to run some experiments with some teams running Scrum and others following Kanban. I am excited at the prospect of rethinking how to improve the planning and execution of software projects and am also open to other suggestions or feedback if anyone has any.