From 2015 to 2017 I was fortunate to be involved in the formation of a Product Growth team at a successful Australian startup. While working on the Growth team I was given the opportunity to work as the team’s technical lead. In August 2017 I resigned from this position to pursue other goals, which you can read about here. This is an article about my role as a Growth Engineer and a Tech Lead.
One of the unique traits of working in a Growth Team is the experiment driven workflow. At a high level, the experiment workflow looks like so:
- Ideate: generate ideas that can result in growth.
- Validate: validate the ideas by research; most typically this meant validating through quantitative analysis of analytics data, but could also involve qualitative analysis through user research.
- Design: design the UI of the experiment for inclusion in the product.
- Implement: build the software solution.
- Analyse: build analytics dashboards and graphs to verify if the experiment was a success.
- Report: report on the outcome of the experiment and document learnings to feed in to future experiments.
A big part of the reason why I received my promotion was due to my ability to manage projects. In the early days of our Growth team we had no dedicated project manager for the larger experiments; having a desire to expand my skill set I took the opportunity be the project manager on a few occasions. Two such projects involved several engineers, a designer, and an analyst; and both of these projects were for my own unique ideas and have since become fixtures of the product. In the final few months of my term I was managing the workload for three iOS engineers, including myself. On top of this I ran dozens of end-to-end experiments with a wide range of complexities.
As the Growth team was expanding and the company was maturing it was becoming more and more important that the team didn’t introduce any critical regressions; this was one of the big reasons for the creation of the technical lead role.
Firstly this task required that additional processes be put in place to ensure that code was reliable when shipped. For us this primarily meant:
- Running blitz tests on the major projects,
- Ensuring that code had a high level of unit test coverage,
- Discussing and documenting code changes that had risk. Additional risk could be attributed to the technical debt inherent in some of the older systems as well as systems that had a large number of dependencies / dependent systems.
This helped to mitigate potential problems, but the other aspect of this responsibility was to be accountable for when problematic code was shipped. For this the were several responsibilities:
- Be available: when things inevitably went wrong you would need to be available to see the issue through to its resolution.
- Have contingencies: this one was quite easy for us as every feature we released was controllable via a server variable. Therefore if anything ever went wrong we could turn off the experiment either for the individual who reported the issue or for everyone enrolled in the experiment. Fortunately the latter never happened.
- Document learnings: once an issue had been identified a document was created so that we could record critical information such as the cause of the bug and how we would prevent this situation from arising again.
A big part of being a Growth engineer was contributing ideas to the team’s backlog. There were several techniques employed to generate ideas:
- Teardown apps and services within different industry verticals and within the same industry,
- Read customer feedback from support channels,
- Run user testing sessions on the various systems within the app, in particular on the onboarding flow,
- Do self-directed exploratory work of the analytics data,
- Build feedback collection mechanisms in to large-scale experiments. Review feedback as it was arriving,
- Quiz fellow engineers on what enhancements they think can be made in to the app,
- Participate in creative off-sites,
- Perform self directed brainstorming and consideration of the apps features and design.
One achievement I had regarding this was receiving the company’s quarterly “Bring Ideas” award. In addition I led two teams to victory in company wide hackathons. At the time of resigning there had been only 5 such events.
As a Growth engineer I was responsible for building experiments for the iOS platform. Engineering tasks would include:
- Implementing experimental features,
- Architecting frameworks to support experiments,
- Maintaining the analytics infrastructure,
- Designing and implementing event tracking for accurate analysis of experiments.
A big component a Growth engineering is monitoring experiments. This involves:
- Building graphs and dashboards in analytics platforms,
- Building in mechanisms to collect qualitative user feedback,
- Dissecting analytics events,
- Reporting on findings.
In addition to this being so close to the analytics data, I would periodically check other app events such as:
- Crash rates by platform,
- Core metrics related to general application usage and health,
- App rating trends.
On the Growth team our work often spanned the full breadth of the product. This meant we had to do a lot of communication over what we were changing as we weren’t necessarily the custodians of the systems we were experimenting on.
Stakeholder communication involved:
- Writing consistent documentation for experiments. Documents included: experiment goals, targeted user cohorts, success and failure metrics, screenshots of changes, experiment runtime, and analytics results.
- Attending technical leadership meetings to discuss changes being made.
- Announcing when experiments were going live, when split sizes were changing, and notifying support of change implications.
- Reporting on experiment results in team meetings and product leadership meetings for larger projects.
Another important aspect of my position was to provide technical leadership and mentorship to other engineers. This involved:
- Scoping major projects,
- Providing estimates for the amount time required to complete pieces of work,
- Advising on engineering best practices,
- Educating and assisting non-growth team members to run their own experiments.
In addition to being the Growth Tech Lead I was also a Senior iOS Engineer. As a part of this I would provide technical leadership to other iOS engineers. This included:
- Participating in discussions on platform best practices,
- Performing code demonstrations to the iOS guild,
- Researching and advocating alternative architectures for iOS apps,
- Releasing new iOS versions to the App Store,
- Being primary interviewer when expanding the iOS engineering team,
- Onboarding newly hired iOS developers.
You can see that it was quite an involved role that required a lot of duties beyond the scope of a “normal” engineering role. I needed to:
- Be able to think laterally to come up with new ideas,
- Be a good communicator in order to be able to pitch your ideas and to document your learnings effectively,
- Be able to use analytics platforms in order to analyse experiment results and to do exploratory research,
- Have some design skill in order to create visualisations of your ideas,
- Be an efficient engineer as experimentation should have a high cadence in order to learn as much as quickly as possible.
I don’t claim to be an expert in all of these areas, but over the course of two years I went from having a T-shaped skill set of an engineer to having the V-shaped skill set necessary for self-sufficiently running end-to-end experiments.
Thanks for reading this article. A lot of extra detail could be added to all points in here. If you have any questions then please feel free to comment below or email me directly.
Please see the rest of my Growth Articles for more.