All posts by Nate Watson

We’ll be releasing the Data Hierarchy Series as a Free E-book next month. Contact us if you’d like us to email you a copy.

From the beginning the Data Hierarchy series was designed to help people understand data science concepts. We hope you enjoyed it and would love to hear what you think.

Below is a quick survey about Data Hierarchy and how we did.

Thank you in advance for your feedback!

If you’ve gone through all 6 of the levels of Data Hierarchy hopefully you see exactly why it’s important to do these steps. So much of each step is contingent on the previous steps. Poor execution of one step can compound issues down the line.

If you remember in the first part of this series the goal of Data Hierarchy is to see the distant goal. The goal that might even be over the horizon from where you stand today. The codependency of the levels will always be there, but their functions are all dependent on the base data set.

Now for the mind- blowing part of this whole series. …take a deep breath…you may even want to sit down for this. Ready? Yes, the steps are codependent…No, you don’t always have to do them in order.

Think of it like building a house. You have to pour the foundation before you put the shingles on the roof. There’s just no getting around that. But, when you’ve got the framing in the house you can work in one room for a bit and then shift focus if needed.

So when the company who cut your marble countertops, because you’re worth it, was loading them onto the truck and slipped and broke into 1000 pieces. You don’t stop building the house while a new slab is quarried in Italy, you work somewhere else. You can hang cabinets without a countertop, you can tile a bathroom, you can start the landscaping early. All of which move you toward a fully built house without cutting serious corners.

Picking and choosing which sub-project to work on without skipping an essential part of the process is part of the long term plan. The Data Hierarchy is a 6 tier concept. However, in execution this could be 60 smaller projects, it all depends on what you’re trying to accomplish.

This is why it’s important to find someone to help develop your data vision. Someone who has been there before. Executed hundreds of projects, stood up multiple data science teams. Someone who’s website is www.canworksmart.com for example. (Shameless plug!

All joking aside, we are here to help and be with you every step of the way. Contact us today to learn what we can accomplish together.

About Contemporary Analysis (CAN):

Having a third party partner who can hold your hand through this trying process is unbelievably helpful. With most of these levels, it takes 10 people to build it, but only 2 people to run it long term.

Additionally, one of our key roles is to fill in a company as they move through the process is the “Data Visionary” role. This role helps you see over the horizon and provides strategy of that vision. It needs to be an employee at some point, but not at first. These individuals are hard to find and even harder to keep. CAN has the ability to fill this role while the initiative is still new and vulnerable. We have no desire to retain it, however. Our goal is to, at the appropriate time, train an individual and transition all of the IP, the leadership, and the know-how to an internal champion–someone who fits the culture and the design of the particular company. Because, in the end, the hardest part of this whole process is getting the decision makers in an organization to embrace the culture of change.

“We’ve done it this way for X years.” Is becoming the leading indicator of a dying business. If you’re 40 years old the technology available today wasn’t even conceptualized when you were in grade school. “We’ve done it this way for 40 years…” means you’re already behind the curve.

So, how do you start, and when? Now–Yesterday actually. You can’t wait any longer. And with a phone call to us. We want to help. We were built for it.

Quickly jump back to the previous segments using the buttons below.

*ominous music begins to play*

The year is 2056 and all of humanity has been enslaved by robots…ok that’s enough of that.

Hollywood has done a lot to help educate the public about technology. Artificial Intelligence (AI) seems to be the one that is “scariest” to humanity because of all the ways it can go wrong

So lets start there with de-mystifying AI and why it’s not going to try and conquer the world without you. Artificial Intelligence is a program that gets smarter every time you run it. If you’ve ever seen a Roomba robotic vacuum…everyone would assume that is AI. It is not. The robot is simply running the program: if you hit a wall, change direction. If you hit another wall, change direction again. When full or when low on power, find docking station with sensors. Repeat until obsolete with newer model found in Skymall.

If you’ve ever owned one of these, (non-weaponized Roomba) you know they randomly attempt to clean your floor. They don’t run 1000 cycles and are significantly better on the 1001 cycle versus the first. If a Roomba were Artificial Intelligence, it would “learn” every time it ran a cycle. If the robot was running that kind of programming it would learn where your furniture is and never run into objects. It would also find the optimal pattern to cover 100% of the floor over time

Until now we’ve not used the term Machine Learning (ML). We did that on purpose to help make a clear delineation of what ML consists of. Oddly enough, ML and AI get confused all the time, even among technologists. Most examples of AI is just machine learning done really well.

Everyone’s friends, and unofficial wiretapping devices, Siri and Alexa, are not AI. They are glorified Roombas running a scripted program. So how where do you draw the “line in the sand” between machine learning and artificial intelligence?

Machine Learning: Machine learning is a program that looks for patterns and provides recommendations for those patterns. It’s up to the human being to test out the recommendations to see which one is the best option.

Artificial Intelligence: AI is a much more complicated decision making program than machine learning. It has multiple decisions it could/can make when presented with a scenario. Not only does it have the ability to make multiple decisions, in most cases, it remembers what it did previous and looks for ways to improve using those different decisions.

So let’s put this into an example people can understand: a Roomba robot vacuum. Your Roomba is using machine learning to figure out how to cover a 100 sq foot room with furniture. It bounces around and eventually will clean the entire floor. In the process its hit some items multiple times and essentially uses “dumb luck” to complete the task.

Now, you rip out the computer chip and install AI because that’s how you roll. (you rebel you) Your Roomba now starts running random patterns and hits a table leg. It then circles the table leg, mapping out its location, and then moves on. After dozens or even hundreds of cycles like that it starts running parallel lines like its mowing grass, missing every table leg in the room. It then changes to diagonal lines, then circles, then a spiral square, then herringbone pattern. Eventually, the AI learns the most efficient pattern to clean your floor, all while avoiding furniture legs, and still gets close enough to clean the corners and avoid cords.

Your Roomba has taken basic programming concept of “run until hit wall, turn left” to “find the most efficient way to become full without running out of power or hitting furniture”.

The interesting thing is the line between a high function machine learning solution and a low level AI is tough to discern. As consultants we’ve seen a lot of AI solutions that are just machine learning.

There’s a bartender joke that applies here. “The difference between a rum and coke and a Cuba Libra? …$1.50 and a lime.” There is a significant cost difference between machine learning and AI. There’s also more marketability of AI over machine learning.

So, looking back at the Data Hierarchy we’ve already covered and how you get to AI. Your incoming data Reporting is comprehensive and all inclusive, it’s not being filtered or scrubbed. Your Business Intelligence is able to report the real time analytics from your data. Descriptive Data is diving deeper into the why behind your BI. Predictive Data is narrowing the “what if” scenarios down to a manageable number of options. Prescriptive Data is running up and down the data looking for how to optimize the outcomes to your desired results. And now *drum roll* you flip the switch on Artificial Intelligence and it does everything else for you…right? Wrong

AI takes all the data results you’ve had previously and learns your business. It learns what happens when you move any given variable. It learns what kind of outcomes you are looking for. It is the eccentric and task obsessed employee everyone needs but never finds. It learns how to make your business better 24/7 and it isn’t afraid to fail.

The “scary” AI from Hollywood is autonomous and/or sentient–meaning it comes up with a solution and executes it on its own. It inevitably determines humans are virus, they need bio electricity to themselves, humans become batteries, Neo is the one, and he can’t die because Trinity loves him. Hollywood AI is just that–Hollywood.

As it turns out, AI isn’t scary. AI is the super helpful employee nobody knows about. AI, when done right, will recommend well thought out data supported options for human evaluation and action.

Instead of HAL, from 2001 a Space Odyssey, asking (ominously) “What are you doing Dave?”. HAL says “Dave, you should buy blue teddy bears in bulk in October. Red teddy bears should never be bought in bulk and only in January. If you do those two things you’ll earn an extra $127,438.43 this year.”

The real kicker with AI is how important the preceding Data Hierarchy elements are to have a smart, successful recommender. If you skip steps, If your data isn’t accurate, if you haven’t had multiple iterations of your team correcting interpretations by your Machine Learning, your AI will be worthless. It will simply learn “bad habits” and suggest things that don’t make sense or work for that matter. The morale of the story is, you probably could skip steps–but don’t. There is no shortcut to AI. All the more important to start now.

If you wait until your competitor has AI, and your not already at ML, your doomed. You are just the next bit of dirt the Roomba is coming to pick up. Make your decision to start up the hierarchy now. The Roomba is on its way and it won’t slow down just because there is a table leg in its way…

Quickly jump back to the previous segments using the buttons below.

I didn’t want to say it…but using the last levels Predictive Data analysis, we were able to predict you’d want to read this one too.  Obviously that’s a joke, but prediction is pretty powerful when its correct.

So you can now predict the future with your data because you’ve followed the Data Hierarchy steps.  Congratulations, you’re still not the coolest kid on the block.

Predictions are more accurate “what if” type of scenarios.  You as a person have to make the decision on how to get there.  This is actually the difference between levels 6 and 7 which we will get to in a second.

Prescriptive Data is the “what should we do about it” phase.  (for example) Your Predictive model spit out 5 highly likely outcomes.  Your team all agree they would be great if you could make them a reality.  Prescriptive Data takes the same data set and looks for adjustments and/or variables to help achieve those outcomes.  Prescriptive Data gives you the actionable items that increase the likelihood of your predictions even more.

Anybody who’s ever hit a golf ball knows there are dozens of split seconds in a golf swing that equate to hitting the ball correctly.  Get them all right, and you’ll beat Tiger Woods. He’s good, but he’s not perfect. Get 2 or 3 of those moments wrong, and you’ll probably end up in the woods at the worst possible time.

In business your “golf swing” is how you execute decisions.  Lets say your biggest competitor is the Tiger Woods of your industry.  They get 15 out of 20 of the little things right almost all the time. Pretty good, can’t compete with them right?  Wrong! They’re still missing 20% of the things that make them good.

Prescriptive Data looks at your swing.  Because you’ve used the previous steps you know you are doing 13 out of the 20 things right.  Prescriptive Data runs hypothetical “tweaks” to your swing at every point with every variable until it can get 17 out of 20.  (for example)

Your data will tell you at any given point in your “swing” if you do X then Y will happen more often.  And its proven by the historic data you have on 1000’s of swings before. Tiger Woods won the 2019 Masters by 1 stroke…1 stroke.  Xander Schauffele, the guy who LOST by 1 stroke, wasn’t beaten by a vastly superior golfer…he was beaten by a detail (or more) being wrong in 2 swings.  A foot that was ¼ inch out of place, a shoulder that dipped ⅛ of an inch too low, and worst of all…he probably hit those shots and could feel it wasn’t perfect.

Hundreds of people leave cable companies every year. For what? A myriad of reasons. Better price, free HBO, wireless boxes, faster internet, Game of Thrones is finally over, and more. The ironic thing is that most people leave a cable company to get a feature the current company could have sold them if they

(1) Knew they were going to leave (predictive analytics)

(2) knew what they were leaving for most likely (descriptive analytics)

(3) knew what to sell them to keep them–free HBO on a phone call to see how they liked their cable (prescriptive data).

It’s only with Prescriptive Analytics that the whole hierarchy starts to make sense (and pay off).

**Next weeks post will be on July 3rd to accommodate the holiday**

Quickly jump back to the previous segments using the buttons below.

If you’ve enjoyed learning about what your data can do for you looking backward, you might want to sit down for this one.  You probably are already, so lets just say you were standing then.

Level 4 is Predictive Data.  Predictive Data takes the information learned in the previous steps and applies a “what if?” thought process.  If Levels 1-3 were reactive, Levels 4-7 are Proactive. Proactive is knowing what’s going to happen before it happens and it takes a completely new way of thinking to apply it to business. If you’ve ever played with a magic 8 ball as a child, this isn’t that kind of prediction

Prediction is looking at the variables that produce an outcome.  If X then Y then Z kind of relationships. The difference is prediction runs through the entire alphabet and double letters for every variable If A then Y then Z?  If B then Y then R? Then runs through it again in near infinite combinations of if AB then Y then Z? What about if both A and T both happen, then Z? The human equivalent of “throwing it at the wall and seeing what sticks” but thousands of times over and over.

Lets Review:

Your Reporting is giving the machine the raw data.  

Business Intelligence is setting a baseline of “known outcomes”.  

Descriptive Data is helping set the framework of what kinds of trends you should expect, and  Predictive Data is taking what is already known and testing “what if” beyond what you could imagine

Back to the teddy bears for a second…because who doesn’t like this analogy?  So you know there is a spike around Christmas and February for teddy bears. Great, now you also know there is an initial rush followed by lesser activity until the panic buyers kick in.  You’ve proven this with your data and have 4 years of historic data to support it

Using Predictive Data you begin to look at other variables connected to the data.  Is there a higher number of men vs women buying the bears, and is it tied to one or the other holidays?  Is there an age group that is more likely to buy a bear? Why did we pick teddy bears for this example? Does the color of the bear make any difference? Does it even have to be a bear?

The machine looks at the data, takes into account all of the Descriptive Data, and begins to tie that data to trends and patterns that are happening currently. This gives you the ability to predict the sales volume and inventory requirements for next week (for staffing), next quarter (for sales quota setting), and next year (for inventory forecasting). So your Descriptive Data provides your Predictive Data meaningful information for the example below.   (totally fictitious numbers

Men between the ages of 20-35 are 75% more likely to buy blue bears in December.  

Having 10% more blue bears in November than the rest of the year will cover demand.

Women between the ages of 35-45 are 45% more likely to buy a red bear in February.

Having 5% more blue bears in January than the rest of the year will cover this demand.

Now we know how many to order, when, and how many people are needed to staff selling them.

With Descriptive Data you were able to make market adjustments for when to market and to whom. With Predictive Data you can laser focus your marketing to exactly who you want to sell your goods to and know how many you are going to sell. When you’re that accurate you have a higher return on investment (ROI) than you would if you just “guessed” with your marketing efforts

Quickly jump back to the previous segments using the buttons below.

How would you describe your data?  That is a phrase you’ll never hear outside of a Data Scientist cocktail party.  But it’s an important question.

Descriptive Data, or “Why did that happen?” is where the past 2 levels really starts to sing.  Now that your reporting supports business intelligence you can start to dig even deeper

Digging deeper with data sounds intimidating but it doesn’t have to be.  Your BI level information works well for short term understanding and is almost a real time indicator of your business.  Learning WHY it happened factors in variables that would never cross your mind.

So we used the teddy bear example of business intelligence.  You can correlate data points to see beyond the obvious. Descriptive Data takes that even further by looking at the data correlations you might miss and brings them to your attention

For example:  Using our Teddy Bear company, your reporting and BI indicate there is a spike in sales around February and then again around December.  It shouldn’t come as a shock that these are connected to the holidays. If that is a shock, you might want to get out of the teddy bear business.

Descriptive Data looks deeper into the data and notices things that seem obvious but aren’t always clear.  In this case it tells you that dads and husbands buy the bears in February and that mothers buy bears in December.

Not the most groundbreaking discovery either, that is pretty common sense for those holidays.  However, it also tells you that its dads under 40 but over 27 who are buying the bears. Its moms 20-35 who are buying the bears later in the year.  This insight translates to better accuracy with sales and marketing efforts.

Machines don’t find this kind of work tedious or boring, like most of us would qualify it.  Because its a machine you can run these kinds of reports all day every day. To do this by hand would take weeks or months.  A machine can do it in minutes.

Quickly jump back to the previous segments using the buttons below.

“Business Intelligence” as a term always has some interesting visuals in people’s minds.  If you instantly flashed to James Bond or Maxwell Smart…you’re our kind of people. *high five*

Business Intelligence is looking at your viable data and deciphering “what is happening right now?”.  Knowing what is happening helps you repeat success and avoid failure in the future

A great analogy for Business Intelligence (BI for the cool kids) is the car.  Your business is your car, its constantly generating data. The miles per gallon calculation, the miles to your next oil change, tire pressure and so on.  The BI part of your car is the dashboard. You get real time feedback from the car on things like RPM and your speed. You also get neat little lights when you’ve broken something.  Tire gets low? There’s a light. Oil needs a change? There’s a light. Engine detonated at 7000 RPM? There’s a lot of smoke, maybe some fire, and then a light.

Ok, so you understand what BI is on a broad spectrum.  That should be easy to spot with your reporting right?  …not so fast. Somethings are easy to spot in data and others are buried deep in the bowels of your data set.

For example:  If you’re a toy company selling teddy bears you would understand why you see spikes in your data around Valentines Day and Christmas.  People buy more teddy bears around that time of year. Mystery solved. Business Intelligence takes that a step further and shows you the real time data.  Its being broken down and put into a format you can make decisions with. Because you can see your business running to plan there are no “lights” on your dash.  If something is out of the ordinary then your BI solution would let you know instantly you need to fix something.

So now you know you’ve got a correlation in your data that wasn’t what you thought.  Thank you BI for proving I don’t understand my business right? Wrong. Using BI you can now formulate marketing efforts to lengthen that time frame for purchase, hold off on pricing reductions until after your busy time to keep your margin’s high, increase your inventory a week earlier to prevent early outages, and more

You can now see deeper into your business than ever before because you know what just happened.  The more accurate the data you input into your reporting now, the more impressive the results will be as you get further down the line in Data Hierarchy

Quickly jump back to the previous segments using the buttons below.

Reporting is the first step on this journey with Data Hierarchy for several reasons.  Reporting is the “what happened?” part of looking at your business.

If a company doesn’t track and understand “what happened” then how would they implement Artificial Intelligence to give them suggestions on data?  Better yet…how do you file your taxes without some kind of basic reporting? If you’re going to start making data driven decisions, you need to have data…good data.

Reporting takes on all kinds of forms.  The key is to have it in a format that is viable.  Notice the word “viable”? That was intentional, the definition of Viable is as follows:

Viable:  vi·a·ble /ˈvīəb(ə)l / adjective “capable of working successfully; feasible.”

If your reporting is a color coded spreadsheet in excel with abbreviations for fields that only make sense to you and the other person who uses it…you barely have reporting.  Arguably color coding is as effective as a wall covered in handwritten post it notes.  (please don’t be “that guy”)

Remember, the long term goal is to have your data work for you, not you work for your data.  So your reporting data needs to be in a format that is easily understood by others and computers alike.

There is no shortage of software to make reporting work.  The number one problem people run into is cross platform reporting.  Which can be easier to fix than you think.

If you’ve got one platform for sales, another for orders, and yet another for payments…oh and the one for accounting.  You may have given up trying to find a single system that does it all.  Don’t worry, you’re not alone.

You’re actually in the majority with that problem.  So much so other companies saw a need to fix that, and they did!  These platforms take some skill to connect them but that’s part of the setup process.

Once you have connected all your data inputs into a Viable data set you can do amazing things.

Also is your data accurate?  Do you have a system that requires a data field to be entered every single time?  Do you have a (lazy) sales person who is putting “asjkdjkldfshjkfadsjkhasdfjklhsdf” in a vital field out of frustration and lack of understanding?

Educating your team(s) that data, good information or bad information, being accurate is vital to being able to fix problems for everyone.  If your data is all smoke and mirrors you’ll never be able to do anything of value with it.

Going forward you’ll be able to quickly jump back to the previous segments by following the hyperlinks below.