Menu
.png)
Data is a double-edged sword – when used well, it can help us challenge assumptions, move beyond anecdotes to see broader trends, and be more fair in our comparisons. But when it’s used poorly, it can become an excuse for bad decisions (in the worst case, when a leader stack-ranks engineers by lines of code written and fires the bottom quartile; there are SO many things wrong with that approach).
It’s a challenge to know how to use data well – and it’s even harder to figure out how to bring your team and organization on a journey to being more data-driven together. Especially for leaders who are human-centric, there can be worries about the risk of mis-use – even if you know you’ll use the data well, could someone else use it to make poor decisions?
To help us learn more about this, we’re bringing in someone with extensive experience in using data to support people. Huanhuan Huang is an engineering leader with experience at Atlassian, SafetyCulture, SuperObvious, and now Multitudes. Across organizations, she’s known as the person to bring in when you want to build a high-performing team, and when you want to do it in a way that supports people. She’s used different types of data to help her do that – from product metrics to engineering ones – and she’s had lots of conversations with executives and teams about how to use data well.
Huanhuang will speak to topics including:
We’ll have plenty of time for Q&A with Huanhuang too – you’re welcome to send us your questions in advance or bring them on the day of the event.
About the speaker
Huanhuan Huang, currently serving as the CTO at Multitudes, is an accomplished woman tech leader with extensive experience in both startups and scale-ups. Her leadership journey includes pivotal roles at Thoughtworks, Atlassian, and notably SafetyCulture, where she spearheaded large teams through intricate challenges. Huanhuan was instrumental in SafetyCulture's evolution from a nascent startup to a celebrated scale-up, marking its status as one of Australia's premier unicorns. Honored in BFSI100 magazine as one of the top 100 IT leaders in Australia, she masterfully blends strategic insight with technical expertise, propelling innovation and growth across the tech landscape.
See below for:
See below for a recording of the talk
Vivek: These are the recording. They have been resolved this time. great. So I'll just give a quick introduction to Huan Huan and then I'll pass it over to you. But, Huan Huan is now the CTO at multitude, so we're very lucky. I met Huan Huan I think, three years ago, when she gave really early feedback, on our beta product, and then every time I'd visited Sydney. Han Wan has always been someone I've caught up with for a coffee over the years, and so I've got to know her quite well and now I've been working really closely with her. But, yeah, she's had a very amazing career working at several australian success stories. Atlassian safety culture, has always been the go to person to build high performing teams. Super analytical thinker as well, and just an all round amazing unicorn that we've been really lucky to work with at multitudes. yeah, so I'm going to hand it over to you, Huan Huan and yeah, you can introduce yourself as well. but yeah, super lucky to have you at multitudes and best of luck with the presentation.
Huanhuan: Yeah, thanks a lot, Vivek.
So, let me quickly share my screen to begin with. Give me a sec. Cool. Can you see my screen right now? Can you see my presentation? Cool. Yay. Hey, everyone. So how do you know if your engineering team is truly high performing? Is it just a gut feeling or do you actually have data to back it up? Think about your own teams. Are they hitting their goals? Do they feel engaged and motivated? What metrics are you using to measure the performance? A few years ago, when I was a director of engineering at safety, culture, those were the questions I started asking myself. My team often referred to me as a manager who led high performance team, and candidates were encouraged to talk to me about building such teams. But what does a high performance team actually look like? I spoke to many managers and surprisingly, many of them believed that they were leading high performance team. Is it just like parents believing their baby is the cutest in the world? We need a, ah, way to objectively demonstrate that our team was truly performing well. And that's when I started to explore the metrics that could tell that story. So my name is. You can also call me triple h. Over the past 15 years, I work in product engineering teams and industry leading companies like Atlassian thoughtworks and safety culture. In my previous leadership role in my current CTO role at Multitool, I have always strived to build a data driven team that truly engage with their work at multitude. Our mission is even more exciting because we try to use data to help create millions of happier and high performance engineering teams. So today, I'm, here to share some strategies for building a data driven organization that your team will love. By the end of this talk, I, hope to build your confidence in applying those tips to start building your data driven strategy and make a meaningful difference in your team.
So, what's data, driven organization? A data driven organization is one that leverage data, as a central component of its decision making processes. Such organizations are, usually characterized by their commitment to continuously improvement. They constantly measure and analyze performance and using data to identify area for enhancement and to track success of initiative over time. This approach may sound familiar to many of you already as, many successful company already integrates data driven strategy into their day to day operations. Typically those companies, they will set high level goals as company okrs and then create team specific objectives that contribute to these overarching goals. And then team will try to run all sorts of various experiments to try to move the needles and achieve those targets. If this doesn't sound familiar to you, now is really now the time to, adopt data driven strategy. Because according to a study by McKinsey, data driven organization is 23 times more likely to acquire customers and six times more likely to retain customers, and 90 times more likely to be profitable as a result. Using data effectively, many companies have successfully applied data driven strategy
00:05:00
in their product development. But what about team management, particularly engineering teams? Can we also share a bit of love from data driven approach? Of course we can. The benefits of applying data driven approach are anonymous compared to the decision that in the past, that based on gut feeling and flick your finger on the air, decision based on factual data reduce the risk of errors and biases. And also by applying some standard and framework, we can further minimize those biases. Building on top of that, it can also enhance alignment and collaboration as well. When, everyone references the same set of data, it fostered better discussion, collaboration and communication. Data become a common language for the team and breaking down solos, aligning everyone towards the same goal. Imagine the effectiveness and synergy when all team members are on the same page and driven by the same data insights. Furthermore, continuously improvement regular data analyze creates a feedback loop that is crucial for continuously improvement. This help organization remain agile and responsive to changes. It promotes a culture of innovation, learning and development. In this kind of environment, teams are constantly evolving and improving based on insight together leading to sustainable growth and finally risk management. This is something that not many people actually think about data actually helps proactively identify potential risk and issue early, allowing for timely measures for mitigating them. Often engineers are, not very outspoken and they may wait until they are, very unhappy or even in the exit mode before they are raising issues with data. We can identify those signs of disengagement early. R1 example is that during my stay at safety culture, one of the one one conversation raised my attention. My team member told me that recently our team has been struggled with PR review during collaboration project with multiple teams, and this has been an issue for quite a while, but it was only brought to my attention when a team member finally spoke up. I talked to many team members and they all have very similar feedback. And one of team members even felt hurt by the harsh PR review comments in the PR. It definitely had great tension across the teams. Imagine what happened if no one spoke up at that time. Looking back, have, we been tracking change lead time for PRSIN and sentiment in reviews? We could have identified and addressed the problem much earlier. Why? Adopting data driven approach in engineering teams has many benefits. It is not without challenges. Unlike, product development, data driven strategy for team management is actually still relatively new and immature. Let's explore some of those common challenges and then discuss some tips how to mitigate them.
So the top one is definitely misuse of data. So engineers often worry about the data we use against them, such as stat ranking the team members based on the number of prs or lines of codes. It is not uncommon at all. Even now, I know some well known company that has been using the number of pr people raise as part of their performance reviews. And the bottom 4% of performers will be put on a performance management plan. That doesn't really make sense because the benchmark becomes like, you don't have to run faster than the bear to get away, you just have to run faster than the guy next to you. This kind of ranking trigger unhealthy, competition within the team. The concern is totally valid. Data itself is neutral, but, its impact depends on how it is being used. Like a knife in the hands of chef, it can create delicious meals in the hands of criminal. You can use it to kill people. As a manager, there are several things we can do to mitigate those concerns. First, build trust and transparency. Start by clearly communicating how data will be used. Make sure the team member understands, the purpose and goals behind the data collection and analysis. When everyone knows the why and the health of data usage,
00:10:00
it builds a foundation of trust. Additionally, ensure the team member has access to their own data. Multijou has done really well on data transparency as part of our data ethic principles. This foster trust and allow the team to see how their contributions are measured. Next, training. Training is very important. Provide comprehensive training to managers on how to interpret and use data effectively. Use of data require understanding its context and limitation. Equip manager with the skill to communicate data inside effectively with empathy as well. They should be able to explain the data in a way that it is constructive and also supportive. So next, let's talk about objective performance measurement. Use data to measure performance against clear, fair standards and goals rather than pitting the team member against each other. This approach focuses on individual growth and team improvement rather than unhealthy competition. Similarly, measure performance against industry like benchmarks or team targets. Seeking improvement in this manner is much healthier than enforcing arbitrary ranking which can demotivate or create friction within the team.
Moving on to our final point, the change management introducing data driven approach is no different than introducing any other process into the team. We need to invoke the team into the data implementation process, lack feedback and make adjustments based on their input. Ensure the binding and alignment and also share successful stories and examples that highlight the positive impact of data. AH can also help. We've talked to many customers. They found that learning from other teams who have done the same thing can significantly ease the transaction for the engineering team. Next. Lastly, an iterative approach implement changes iteratively and also allow the team to adopt gradually and provide continuous feedback. The not a top challenge is the team member might gain the data once they know what's being used. This can distort the data, lead to misguided decision. Here's another interesting real world example. In one of the prototypes I used to work in the past, the number of reports viewed by user was used as a key metrics in that team. So a product manager decided to put report link everywhere in the app and even embed it in the core flow. As a result, the metrics of report view skyrocket. But it didn't, generally reflect the user engagement or satisfaction and in fact it actually negatively impacted other teams goals such as NP's or retention as user rate quick the app how do we mitigate situation like that? Here are some effective strategies. One approach is to, avoid using one single metric to measure the success. Instead, we should use different and complementary metrics to gain a holistic view. For example, if you want to see that the team deliver more frequently, we should also measure the change failure rate. Are they delivering quickly at expense of product warranty. What about working out of hours? Are they delivering more at the expense of their own health? It is also important to remember that data is only your conversation starter. Data and metrics shouldn't be the solo expert used to gain insights about the team. It should complement quality feedback and also quality one more conversations to get a full picture. Data works the best when combined with qualitative insights. At the end of the day, rules are, defined for people who follow them. We don't actually need to worry too much about people making the data. It is like the self checkout machine in a supermarket. If people want to steal, they will always find a way to do so. However, most people will choose to do the right thing.
The next challenge is that people often think data is cold and non human. They hated it. However, the perception largely depends on the context and why we are measuring
00:15:00
data. A couple of weeks ago, one of my female engineering friends told me that he was considering leaving the current company. When I asked her why and she said, I'm burning out. There are politics in the company and I have to work twice as hard to be seen. I feel really sorry to hear that. It made me think, what if the company had data to tell an objective story showing that she was working twice as hard as others? Would data still seem cold if it could help her to voice her efforts? Probably not. To make data more meaningful and human centric, consider the foreign strategies. It is essential to clearly explain why specific metrics are important and why they are being tracked. For example, instead of just telling people, hey, we are tracking the number of pr being reviewed. Explain how this metric can demonstrate team support and collaboration. Explain how in this way, this understanding can help the team to see value and relevance of the metric. Another strategy is to connect the metrics to career development. For instance, to become a senior engineer. For example, the conversation can be like that for you to become a senior engineer. We want to see you support others. And here are, the metrics we use to measure them. By linking metrics to personal and professional growth, team members can see how the contribution aligns with the career goals. It is also important to align metrics with the team OKR as well. For example, we can tell the team at the moment, the team OKR for the next period of time will be to improve our, change failure rate from 16% to 5%. How do you want to contribute to this goal? When metrics are, tied to team objectives, it creates a sense of shared purpose and directions. Finally, actions on data insights. This is a very important one, actually probably one of most important of all those tips. Data itself is worthless if we don't take follow up actions after understanding it. Showing that management team cares and take actions based on data, increase the team morale and we should celebrate wins when we achieve them. For example, if data shows significant improvement in a key area, make sure recognize and celebrate this success with the team at multitude. We do the same. We use data to find that we were not spending enough time on technical debt. This lead to meaningful conversation with key stakeholders and we agree on a healthy ratio. 25, percent, 35 percentage of engineering time should be spending on technical debt and 75% of future work. The team was really happy because they finally get to work on tasks that improve their developer experience and efficiency. Their life become better because the data driven approach building a data driven organization that your team will love involves embracing data as a powerful tool for insights and improvement, not as a weapon for scrutiny or competition. It requires openness, honesty, trust building, goal alignments, proactive actions and continuously improvement. Data should spark conversation, not replace human interactions. By capturing metrics, you gain the ongoing insight without disrupting the team workflow, boosting productivity, satisfaction, engagement and retention. Ultimately, the goal is to foster a culture where the data empowers and supports your team, helping teams to perform at their best and feel valued for their contributions.
As you leave here today, I encourage you to think about how you can start or enhance your data driven journey. ask yourself, how can I make the data my ally? What issue can I solve with data in my organization and what metrics will I use to measure the success of my process changes? Let the data be your guide in building a supportive, engaging and high performing team. Thank you.
Vivek: Thank you so much.
Huanhuan: Thank you, team.
00:20:00
Vivek: There were a bunch of questions that came through before the event, but as people have questions, feel free to put them in the zoom chat. Or, you can even just turn your mic off and ask them, over voice as well. that's probably preferable to make it a bit more human.
just one of the first questions that came through that, I thought I would share. That came on the chat washing. what tools, do you use to help visualize and share and communicate data with key stakeholders?
Huanhuan: Yeah, definitely, because, I've been doing data driven for quite a while. I definitely use a lot of true in the past, so definitely the product one, as I mentioned, probably more mature. A lot of time they're using Mixpanel or looker to understand how the customer have been using the data itself. Some other tool can using in product, for example, even like amputations and those kind of things has been really commonly used for the team itself. That definitely less mature and then less true available in the past. So I started to explore the tuning probably, like I mentioned probably when I was a director of engineering at safety culture at that stage, because I was building high performance team. And also the same time I want to. My question in my mind is also, how do I propagate those impact, right? How do I tell how people telling the true story about the team is really high perform? So that's in that time, there was a time that starting to explore some metrics how to helping that and then also exploring some Turing as ah, well, so at the time I explored, I believe I explore like Lina B and also multitude. That's actually how I started to know multitudes at that time. So Lena B. At the time, like, my feeling was that time just jab on my feeling, right? So that times that, when I use lean up B realized that, oh, my God, it's like information everywhere. I have no idea what I'm looking at. There's like two metagraphs in there. And then when I look at multitude, I started to really fall in love with it at that time. Because the reason being is that I realized that multitude focused a lot on the human side. It focused a lot of, for example, the one that really fall in love at the time immediately was the collaboration flow of people talking to each other. So I know that, oh, who is like working very closely with who and then the collaboration, how you work. Another thing that I really love focus on human side is also understanding people have been working out of hours or not. So that's why I feel like that magic like this is not just focused on, oh, have you delivered? Are you working fast enough? But we also care about the team itself. So, yeah, there is a lot of tool available. There's things that really needed to. And also very interesting that I really realized that a lot of people starting with manually, like general report first, and then they starting to feel like, oh, my God, it's so much troublesome for me to generate report all the time, get the information right here and there and there. And also quite disruptive to the team as well to generate those reports. Because those report all the time. It is like from bottom up, technical lead generates some report, and then the CTO calculating those report and then presenting to someone else, like key stakeholders. So require lots of human time. So that's why that in later stage also explore the tutoring as well. So.
Vivek: Yeah, great, yeah, thank you.
Huanhuan: I'll ask a question, whoever answered question. Yeah, yeah.
Vivek: Any other questions? I haven't seen any come through on the chat. anyone have any questions that they want to ask?
Speaker C: Ah, one, I've been using multitudes for a fair while now and I've been useful for my team, but I was curious if you had any thoughts on, lead versus lag indicators for team performance. it's something that I've been, my team's fairly, data literate in terms of that, and we've been talking a lot about. Okay, are we looking at too many lag indicators? So I was interested in your perspective on lead and lag indicators for team performance.
Huanhuan: Yeah. So at the moment, a lot of times that you learn the data, ah, based on what you already have. So that probably is a bit lagging at the moment. So by the time going, we're definitely thinking about how can we actually help to predict something could happen as well, even though we say lagging is already much faster than human understanding how it works. Exactly. Like, the example I was talking about, if nobody mentioned about that PR review like problem as a human way, probably I would never know that. Right? So that's why even though that using data could be a bit lagging still at the moment, like using the tooling we have, but still quite leading in terms of understanding how the team has been doing. Another thing that I try to raise as well, that for example, in the past that we. A lot of times that even in multitude right now, one of the problem trying to solve is that lots of human work are hidden. Ah, for example, at multitude, we try to high touch, we try to help
00:25:00
Huanhuan: customer to fix the problem as much as soon as we can. So that also end up, we have lots of support tickets in our team. So in the past that we didn't track those data. So end of that, people like keep working in the dark. It's like, oh my God, I'm so busy. But also at the same time, the kids hold their keep telling why we never be on time, right? So that becomes very difficult conversation where we don't have visibility about what's been working on. Maybe I can quickly share, like it's a real example. I think it's good. See m that. So for example, right, the change fails. You're right. Previously our team was about 16, percent up and down. And then like, after a while we become like roughly about less than 5% but that is not really real because as I mentioned, a lot of things that has been working under the radar itself. So I do something to introduce the incident management system and also support ticket system and actually since then, and then you can see the change failure rate rocket. But I'm not too worried about that because in this case actually resurfacing all the data then let people know that hey, we actually, the team has been really busy right now working on those support tickets. In this case, when we submit those data, we can also starting to learn the pattern of what has been going on under the hook itself that actually can help us to become more, leading to really understand then what could happen in the future and then we can actually change in the future to make it better as well. So does that answer your question?
Speaker C: Yeah, that's a great insight. Yeah, thank you.
Huanhuan: Cool. Let me stop sharing. Am I still sharing? Yes.
Vivek: great. So we're going to move to the small, group discussions now. so, thanks everyone for coming. I'm going to stop the recording. amazing talk, Huan Huan. And yeah, learn so much from you every single day. But I will be stopping the recording.
Speaker C: So.
00:27:32
Transcribed by https://headliner.app
