Your data projects might fail, and that's okay.
The first, second, the third, and so on…
I remember learning programming for the first time. It was the time when I had just been accepted into the Graduate Program, and they sent me a prerequisite learning material that I needed to learn before the class began. That mandatory material is the R programming course. I need to understand them within a month before my class starts.
If I see that course now, I would feel that it is a beginner course that I would not have any problem understanding. But I remember the first time I saw that R programming course, and it’s mind-boggling. I had difficulty catching up to the concept until the class started, as I had never touched any programming language.
In 2 years, I learned the R programming language on and off as I was also busy with my class, and not every course required R knowledge. However, my thesis needs me to analyze my data with R, and that’s where I feel the challenge. With limited time, I tried to finish the analysis with my basic R understanding. Luckily, I was able to push out until I could defend them.
Until now, I still feel my data analysis project is a failure, not because of the result but because the process could be more efficient and the analysis could be better. Nevertheless, that was my first data project, and I learned something. You can see my old code, as it’s always become my reminder.
Flashback to today, I have performed countless data projects within and outside my workplace. I have encountered many failures, from simple data analysis to end-to-end machine learning projects, but it’s okay as expected.
If we define failure, it means the conditions where we don’t achieve what we desire or the goal we set. But failure doesn’t mean we are stupid or incapable; it’s only a phase we must go through. The wise people would always try to learn from this failure, and that’s what we need to do.
Here are several points why your data project’s failure could help you.
Ability to Understand Why You Fail
Data projects contain many moving components that we might not understand if we haven’t tried them out. I remember my first project in my current company: to analyze a specific sales business funnel and develop a propensity model. The data project is, at a glance, a success as the team could provide insight into the business and develop the model. But over time, it slowly decreases as the impact is not meeting the business needs.
From this project, I learned a few things about why we as a team failed to deliver data projects that could have an impact on the business, including:
Bad data quality. The project utilizes data from various sources that haven’t been curated effectively, and that’s bringing down the project quality.
Unclear project objectives. Too many stakeholders are involved, and everyone has their own KPI to prioritize. This clouds the objective as no compromise is met.
Ineffective communication. No coherent communication can effectively bridge the understanding between the stakeholders and the data team. Which means there would always be something missing in the project.
Lack of domain expertise. I am new to the industry and haven’t had much time to learn business terms. That’s why some data project results were fragile and not meeting the expected target.
That’s some of the points I learn from my project, and it honestly makes me feel better after I evaluate why my project is failing. Because I could learn from my failure, I was able to make a better movement in the following projects. There is still a lot of failure, but understanding why we fail is essential for success.
Iterative Approach Mindset
Failure might make you paralyzed. Having things not going your way could stop you from doing anything, and you want to let go of whatever makes you feel bad. I understand that, as there are times when failure stops me.
It’s not necessarily only applicable to the data people but to any work we do. However, data projects certainly have higher learning curves, which deter people when they are failing. It’s something that I experienced as well.
That’s why I try to change my mindset when I get serious about entering the data field. Rather than avoiding failure, I expect to fail and use that as a new starting point. Iteratively, it makes me better as I learn from my previous failures.
Yes, my data project is a failure, but it doesn’t mean it would stop me from learning. The failure gave me a better understanding that the iterative approach is the best way to learn.
Skill Diversity
Understanding why we fail and performing an iterative approach brings confidence to trying out everything. Without fear of failure, we can learn new stuff and keep our focus. That’s why, every day, I can try out new stuff and see what sticks.
I didn’t understand much about Generative AI a year ago, and now I use it in my many data projects. If you read my previous post, I think there are also some critical learning that could be improved in my Generative AI skill pursuit. How I understand this new concept is a fruit of trying out stuff, even if it leads to failure.
That’s why failure in a data project would improve our skillset if we do it right. Don’t be afraid of failure, as it’s expected in any stage of our learning process.
Conclusion
Failure is hard to cope with, and expect failure in your data project. However, your data project failure could bring something else that would help you grow in the future:
Ability to understand why you fail,
Iterative approach mindset,
Skill diversity.
That’s all for now. I hope this is helpful for you.
Thank you, everyone, for subscribing to my newsletter. If you have something you want me to write or discuss, please comment or directly message me through my social media!