Why are my reports unreliable and how do I fix the data sources?
There is a specific moment in the life of any people manager (or people working with a lot of data) when they realize something is off with the quality of the data they are working with. It is at this point that a question starts to take shape: how reliable is the report I just created? It is not a simple question to answer because unreliable reports do not have just a single root cause; it is more a culmination of multiple factors.
When our reports are no longer adding any value or have become unreliable altogether, a bigger issue starts to appear: decision-making based on data becomes faulty. Across history, we have plenty of examples where unreliable data created catastrophic results, and for this exact reason, the topic of data hygiene should be a must-have on any leader’s mind, especially for first-time managers. We work with so much data today that the worry of not having access to it is a thing of the past, while a new fear is ever-so-present: how much can I trust the data I work with? With the variety of data types, multiple sources, hundreds of people handling inputs, and so many additional layers, even the most basic of reports should be under the microscope of a quality check. The less we focus on quality improvement, the worse the insights and incorrect statements we make with the data we are using. And once we start down the path of faulty insights, made-up statements, and estimated guesses… there is no turning back.
So, when my data is no longer reliable, what can I actually do?
Unreliable reports start at the source. This piece breaks down how messy manual inputs and human error sabotage decision-making. Learn how to clean your data, fix collection methods, and use simple frameworks or AI to keep insights sharp, practical, and actually useful.
In this article:
- Where the data goes wrong: Explores how faulty inputs ruin reports, why people waste time making bad data look pretty instead of fixing it, and how manual processes and human error lead to unreliable insights.
- Back to basics – the source: Focuses on auditing collection points, improving form structures, minimizing free text with smart drop-down lists, and applying the PDCA framework to redesign messy databases.
- Check, check, and recheck: Highlights the importance of regular audits, maintaining new habits with your team, managing pushback constructively, investing in practical automation tools, and using the DIG (Describe, Introspect, Goal) AI method to scan and clean data safely.
Where the data goes wrong
Ever had the feeling the report you just created is really not that good? It is a sort of epiphany when you realize that all the effort you are putting into this presentation is wasted and no relevant actions can come from it. When this happens, we need to look at the main causes, which more or less gravitate around incorrect inputs and/or outputs. If the inputs that feed your report or analysis are wrong, the entire process (from the moment you started collecting the data until the actual presentation) is a big waste of time, and when the output is wrong, it is all about the one doing the analysis.
Sadly, a lot of reports based on incorrectly gathered or analyzed data tend to be very artistic in nature; people just spend that extra time making them look better instead of trying to correct whatever errors the data has. Sounds like a strange thing to do, but it is actually very common. My theory here is that people just want to commit to the action they already started for fear of wasting time, missing deadlines, or feeling embarrassed about not managing the task correctly from the start. Of course, at times it can also be an oversight of really not checking the accuracy of the data.
The main problem with this is that it defeats the main purpose of a report: what do you think this will achieve if the data is wrong? What would happen if, based on your insights, an entire team project is started to correct, say, the AHT, when the actual problem is a new procedure change? Imagine the amount of time, effort, stress, pressure, and everything in between that comes with a significant project plan… and then ends up not working. While it is true that not all improvement initiatives are guaranteed to work or be 100% what we expect, it is one thing to improve something we have observed and know about from accurate data, and quite another to work on action plans based on incorrect information.
I always have this thought with me each time something new is presented to me: the fancier they try to make it sound, the less accurate it is. I believe in the simplicity approach, which goes like this: whatever information you need to present must always be deconstructed and reassembled into the simplest possible package that is most likely to resonate with your entire audience. Think about it this way: if you try to explain concept X to a room of concept X experts, whatever you are talking about is not strange or hard to understand because you all speak concept X. But what if your audience is part concept X, part concept Y, and some have no idea of either X or Y? How do you make sure the information provided will convey the same message to all those people? This is why when some try to make really fancy and complex data points, my inner trigger is pushed and I always end up asking: how reliable is this information if they make it so fancy and hard to follow?
We spoke a lot here about the irony of the new age, with tons and tons of information and technology all over the place, but a very difficult path to harvesting said information. In my experience, a lot of data collection methods still (sadly) rely on manual intervention, manual input, or in some cases, fully manual processes: trackers for team tickets, messy surveys with a lot of text, multiple sources of data that are not correctly linked to one another, reports that need a lot of human intervention to be accurate, and many more. Because of this, a lot of the data we work with comes with errors right from the start, most of them human-made. In fact, most data entry, spreadsheet mistakes, or general data quality issues are human-made.
For these reasons, your job as a leader in charge of data management and making data-driven decisions must always look at human-made errors from a data perspective before even attempting to make any sort of analysis out of it. Focus must always be on the quality check of your data, making sure the information is accurate, up-to-date, and correctly captured.
Back to basics: the source
A good practice, and not just for this specific topic but in general, is to always go back to basics when things are not as expected. In our case, we need to look at the source of the data; this is where most of your issues come from.
With so much diversity in gathering data, a clear and detailed overview of the data source is key. Here is where you need to look at all the aspects that create the collection of said data:
- Inputs: how are they structured? Is there a template/drop-down list, or do people just fill in whatever information they want?
- Formatting: is the information captured in the correct format?
- Mistake-Proofing: when incorrect data is added, can the source detect it and signal that something is wrong? For example, leaving a field blank.
- Frequency of collection: how often does the source update and/or collect data? Can you access it in real time or only after a specific time?
Once you have a bit more clarity on the source, it is time to look at the basic elements of any collection effort: the input. If there is one thing I have learned across the years about data, it is that less text is always better. Imagine a huge database that comes from free text and how hard it is to actually extract anything meaningful from it. The best way to go about this is by creating proper drop-down/category lists where people can just select whatever category they need instead of typing it in manually. Of course, we also must be very cautious about the number of drop-downs; if they seem more like a difficult chore, it defeats the purpose, meaning less is also more in this case. Who wants to spend 10 minutes filling in a database with 258 fields and multiple categories for each field? If you are in the middle of an already existing mess of a database, here is what I normally do:
- Get the biggest possible portion of said data, preferably in an Excel format.
- Go over each column and understand what it is asking the user to fill in.
- Look at each input and do a deep-dive analysis of the content from each column: what are we filling in, what are the most used keywords, etc.
- Match the identified words with proper business-tied categories. Be mindful here to create them in such a way that will help your business, not add more complicated and unnecessary fluff.
- Test it out: use a trial period and see what happens when you use it for a limited amount of time. You will learn a lot from this, and not just about the drop-downs, but also about how people feel about it, understanding, completion time, etc.
- Make the corrections needed and launch it.
Always remember the PDCA framework: Plan, Do, Check if it is working, and Act on the checks you have made to correct any mistakes.
Check, check, and recheck
Nothing lasts unless you invest time in maintaining it. This is the best advice I can offer first-time managers. The more time you spend on making sure whatever you did stays that way, the longer and better it will be. A lot of people assume that once a specific improvement is made and started, everyone is aligned and will follow whatever they need to do to the letter. This is, of course, completely wrong, mostly because people need a lot of time to adapt and make something new into a habit. For this reason, a lot of improvement initiatives fail: managers do not check and react to them.
When faced with an overhaul for our topic, the only way to maintain the new way of working is by checking on it regularly. When something is off, you need to step in and correct it without pulling out the micromanager. The first step here is to have a proper audit of your data to make sure that the information is captured as it should be. Just go over inputs captured during the day (or from the previous day), check to see how people are capturing the information, do you see anything wrong with it, are there any specific areas that might become problematic, etc. It also helps if you have a quick connect with your team to go over the new way of working and make sure everyone is aligned. Creating simple yet visually effective procedures and sending them to the team or to whoever is adding the inputs will help a lot. Just remember that sometimes just because you sent an email does not mean people are actually reading it, so the best way to be 100% sure is by setting up a quick connect and answering any questions they might have.
Your reaction to the audit findings is also important because when it is not properly calibrated, it will do more harm than good. Make sure you keep your composure, be there, listen, and do not judge or treat people like they are complete idiots. Remember, it takes a lot of time for people to get used to a new way of working, and even the best ideas can be improved. Be open to feedback and listen to it; do not brush people off just because you think they are wrong.
Transforming a new way of working into a habit is the last and most difficult step, mostly because you need to spend time selling the idea and also promoting the benefits. Again I say this: people do not like change and they will react. Pushback is very common, and you must expect it and not fear it. Keep on selling the idea, present the benefits, and try to talk the same language; when someone is telling you they feel the new way of working is taking up more time, do not talk about how accurate the data will be.
If your project can invest in tools that can ease human effort, go for it; do not brush them off thinking about costs. Wrongly processed data will cost you way more than a simple tool, and if the business case is correctly presented and justified, the company has no reason to reject it. After all, why use a manual tracker when you can implement a better, faster, and easier-to-work-with system? And in some cases, they do not even have to be new software; you can ask for Excel macros or a simple database system that someone can create (it can even be a special project with people from the company). The idea is to not rule anything out just because it might seem crazy or too expensive.
And since it is the age of AI, you can use it (provided you have the needed clearance and security checks done) to do a data quality scan. There are so many ways of doing this; here is one I found and use all the time: DIG – Describe, Introspect, and Goal.
Describe is about feeding the database to your AI model and NOT asking it to fix it (a very common mistake). Instead, make sure both you and the AI have the same understanding by asking it to describe the database. An example of a prompt would be: List all column names, determine their data types (text, numerical, date), and show me a sample of the first 3 rows. Highlight any immediate structural anomalies like missing headers or misplaced data.
Introspect focuses on using the AI model to scan for potential errors, mismatches, etc. Basically, look at any anomalies across the standard data quality dimensions (consistency, uniqueness, validity, and completeness). An example of a prompt here would be: Go over the entire database and look at the following quality dimensions: consistency (any formatting discrepancies), uniqueness (do we have duplicate entries, suspicious entries, or any errors from this point of view), validity (any strange values, numerical outliers, or inputs where we should have text but have data or vice versa), and completeness (do we have columns with incomplete, missing, or partially added data).
Finally, the Goal is where the magic happens. Based on the inputs from the previous two steps, you can now decide what to do next: either use the model to create better standards for your data, better input templates, categories, etc., or ask it to help you create a script to help you clean the data.
In the end, whatever method you use, just make sure it is working, and always check the output: if you no longer have garbage data in and you actually start generating meaningful and actually useful insights, the job is done. All you need to do is keep it on track. If you fail, do not stress too much; it is normal and part of the improvement process. Just go back to basics and start over.
Until the next article, stay safe, healthy, and happy!