Data Foundations for Enterprise: 10 Traps to Avoid

Countless organisations want to turn enterprise data into actionable insights.

But how do you do this if your organisation is early in the journey to becoming data-driven? What are some of the common pitfalls or traps to avoid? What are the key investments?

To learn more, we had the pleasure of sitting down with Calvin Ng, learning the principles he used in leading major data initiatives - the commercialisation of Woolworths loyalty and NAB transaction data, building data products at Quantium, Lonsec, Macquarie Bank, and in his current role as a Head of Data Quality and Linkage at Equifax.

As guest editor, Calvin shares 10 key tips in dealing with imperfect data, building your organisation’s data foundations and unlocking the essential investments to become data-driven.

‍

TIP #1 - Don't start by hiring a full team of data scientists

That’s a terrible idea, as they’ll just ask “where’s my data?!”

Instead, start by listing the questions the business needs answers to. A good place to start is a customer use-case: (1) who are my customers? (2) how many customers do we have? (3) can we attach customers to transactions over time? Be specific. Don’t get carried away with a long list of questions to answer. Stay focused. Then bring in people with the skill sets that can answer those questions.

‍

TIP #2 - Don’t invest in something that’s really fancy (unless you’re prepared for a 10 year, multi-million dollar data migration project)

A lot of technology organisations start off with a grand vision - “I’m going to build you the best data lake in the cloud and it will just cost you $5m”. If you start with this approach, you won’t reap the rewards early. You’ll end up in a data transformation migration project that’s going to take years.

Instead, start with the tech that’s just necessary to do your analysis. Start off with cloud services compliant with your data set. Start off with managed services like Snowflake, where you can quickly spin up your warehouse without having to pay for DBAs to optimise your queries.

Bottomline - If you put all of your money and time into one major initiative or technology, you’ll find it difficult to excel at showing value sooner such as guiding the organisation to better decision-making, marketing, customer service, or monetization.

‍

TIP #3 - Pay for tools. Don’t be stingy

Most companies fall into the trap of being too stingy with tools - buying one user licence only, avoiding buying the necessary licences to socialise data and insights out to the business. But what good is a report or insights, if you’re the only one who can see it?!

Having appropriate tooling makes it so much easier to socialise work downstream for building business cases (see TIP #7).

Bottomline - Pay for tools, but don’t pay for the full $5M shebang up front (TIP #2). Chances are, technology will simplify.

‍

TIP #4 - Avoid data cleaning rabbit holes

All data sparks joy. It’s just how much grief it gives you before it sparks joy! - Calvin

There are going to be things that are spurious and noisy. Real time data, for example, is always a rabbit hole. And unless you have real time channels to action real time insights, avoid it. Instead, the way to start is to ask “what’s important?” and focus on cleaning only that data.

Bottomline - Stay focused on your key questions (TIP #1) and on the data that will help address these questions. Ignore everything else.

‍

TIP #5 - Stop blaming the data. Ask these 5 questions instead

When faced with data, people are quick to point out imperfections - “Your data is wrong! That doesn’t look right!”

Given the criticism, it’s natural for data teams to get defensive when others question data quality. But instead of arguing, use the meeting as an opportunity to show others how to go beyond stating the obvious, to asking better questions - designed to unlock the ‘why’ - when faced with data that ‘looks wrong’.

1. Why is the data behaving like that? Is it the result of a bad process?
2. Poor data entry at point of collection?
3. Poor structures about how data can be entered into the ecosystem?
4. Is it an engineering thing?
5. Or is it just the way the data is?

Bottomline - Questioning data quality is a good thing and should be encouraged. But pointing out flaws without seeking to understand why is just noise. Always seek to understand why.

‍

TIP #6 - Pick up the phone

“Treat the business like a customer… because they are!” - Calvin

For many data and analytics teams, working with ‘the business’ is a daunting task. Picking up the phone isn’t second nature. But in order to succeed, you must engage the business as part of the design, question and build process.

Talk to them about their data quality issues or quirks. By asking questions proactively, they’ll be far more supportive and less surprised and standoffish by what you have to show. Ultimately the numbers are there to tell them how the business is doing.

Bottomline - Build insights and analytics for real people and teams. Work with stakeholders up front. You should never surprise them and say “voila, here are insights that I prepared earlier!”. Less surprises, more collaboration.

TIP #7 - Make insights widely available

Many data scientists contain their work, don’t socialize it and don’t provide it freely to the business. “But what’s the point of having an insight if you don’t share it.” - Calvin

Data science teams get a little defensive when non-data science teams question the outcomes. But don’t. Ultimately the data and insights that you produce, the people actioning the insights, will be used by the business. Be open to constructive feedback.

Bottomline - Having the appropriate tooling, like Hyper Anna, makes it so much easier to socialise work downstream for building business cases (see TIP #3).

‍

TIP #8 - Keep it simple

A data driven culture doesn’t require training all downstream teams in data and analytics.

Instead, it’s about designing insights, and delivering them in a way that’s easily understood by front line teams. A lot of the time data and analytics teams whip out graphs, really complex charts that look really cool, but impossible for front line teams to quickly understand what it means. So it’s about designing insights, tools and interfaces that are easily accessible, relevant and give people an opportunity to take action on.  

Bottomline - Endless charts, dashboards, and visualizations will always flat, drowning business teams in information overload. Whilst most data teams produce dashboard after dashboard, most frontline teams don’t require custom visualisations. Instead, they need insights that convey data not just in numbers or charts, but as a narrative that they can actually comprehend.

‍

TIP #9 - When selling data, sell it for what it is. Be absolutely clear.

Be really clear on helping people know exactly what the data does. There’s no point selling it based on the characteristics of the data - e.g. billion rows, 200 distinct fields - if you can’t explain what’s actually in it and the quality of insights that one can get out of it, then it’s pointless. Sell it on the merits of what it can do and make that super clear.

Bottomline - Be upfront. If the data is akin to crude oil - a raw unrefined ingredient that requires preprocessing - then sell it as this, a raw unrefined product. If the data has a layer built on top of it meeting a number of use cases, then again, be super clear and upfront.

‍

TIP #10 - What most data leaders forget to do (or procrastinate!)

As you finish your first piece of insight and you’re presenting it, people are going to ask “So what? Can I do anything with this insight?”.

You want to be ready to say “These are all the things that we can do”, instead of every answer being “We have to check with legal and compliance.”

This is probably the most critical thing that everyone forgets - no matter where you get your data from - even if you own it and generate it, your L&C team needs to be the one that says “that’s compliant”.

Bottomline - Have an upfront conversation with your legal and compliance team. Get data governance embedded upfront.

‍

__________________________________________________________________________________________________

Guest Editor: Calvin Ng
Head of Data Quality and Linkage - AU & NZ at Equifax

Calvin brings more than 15 years of experience in data leadership and commercialisation across the banking, finance, retail, FCMG, telco and credit sectors. He has held previous positions as Lead Data Scientist at Quantium, Lonsec and Macquarie Group, advising both multinationals and startups on data transformation initiatives.

Joining Equifax in 2017 to transform the data practice within the chief data office, Calvin currently leads the transformation programme in Australia and New Zealand. His areas of interest include bringing data and analytics to new emerging problems, data linkage and privacy.

___________________________________________________________________________________________________