New Monkey Tools Features

We're super excited to let you know that we've just released some new Monkey Tools features!  Let's take a quick look as to what is new...

The Table Monkey

This feature was actually released back in December. However, since we announced it at the KSA meetup (which you can see on YouTube), we decided that it needed a personality of its own.  So now, on the Query Monkey menu you'll find the Table Monkey: a monkey who is dedicated to helping you build queries from Excel tables.The Table Monkey allows creating queries not just from one table, but multiple tables in one shot

Some of the cool features of this Monkey are:

  • It can create multiple "From Table" queries at once.
  • Tables can be excluded with a single click.
  • It can create "Staging" layers for you - as per our Dimensional Modeling course on Skillwave.Training, with custom staging layer names or counts.
  • You can rename the Excel tables by right clicking on the blue boxes that represent the Excel tables.
  • You can rename the Queries by right clicking on the green boxes that represent the data model tables.
  • It allows you to toggle the end query so you can load it to the data model or as a connection.
  • It provides a data typing algorithm that is smarter than Power Query's native algorithm.

Overall, we find this to be super useful. It allows us to create multiple table connections in a few seconds, rather than the minutes it would take us to set things up manually.

This feature is a Pro feature, but is fully functional in our free trial.

Create Query from M Code

The next feature that we included is a nice interface to create a new query from M code.  If you post in forums and need to quickly create a query for testing, you can simply take their code, paste it into the form, give it a name and click create.  Much easier than having to create a new query, edit the code, select everything and then paste:

Using the new Create Query from M Code feature to quickly create a new query

The main benefit of this form is saving you the headache of jumping into the query editor to create your query. Additionally, we also added the ability to indent the code right in the form. So if you're just trying to read it, it can be useful without ever creating a query at all.

We feel that this would be a super useful feature for those helping each other in the community. Thus, this feature falls in to our "Forever Free" category and works at all license levels (include after your trial expires).

Convenience Features - Pivots & Filters

Another one of the new Monkey Tools features that we've added is a Pivots & Filters menu to the Monkey Tools ribbon.  This is purely a convenience feature. It's designed to bring the commands closer to you so that you don't have to do as much tab switching:

The new Pivots & Filters menu allows creating PivotTables, PivotCharts and Slicers and Timelines without leaving the Monkey Tools ribbon

The version on the left is what we are terming the "Classic" view, which shows you the Insert PivotTable button (as well as PivotCharts, Slicers & Timelines).  The view on the right is what your menu will look like once the new Insert PivotTable button rolls out to your Office 365 install.  (If your Monkey Tools menu starts with PivotCharts, then head to our Options screen and uncheck the "Use Legacy PivotTable Menu Buttons" option.)

Bug Fixes

And - of course - like every release we do, we have included a bunch of bug fixes. Fixes that are applicable for all users including Pro, Trial and Free.

How to you get the new Monkey Tools features?

If you already have Monkey Tools installed, then head in to Monkey Tools -> Options.  If you are running 1.0.7678.28973, then you already have them.  And if not, click Check for Updates Now to update.

Don't have Monkey Tools installed?  You can try the full feature set for free for two weeks before the license reverts to a "free" license.  We think you'll be pleasantly surprised with how useful Monkey Tools is on a free license, and yet how much more it does in the Pro version.

 

More free features in Monkey Tools

Wow, it is hard to believe it is already December.  And looking back at my blog, I realized that I forgot to tell you that we released a few more free features in Monkey Tools over the past month!  In fact, November was a busy development month for us, so I though it would be a good time to share what we have done.

GetISOWeek Function

One of my friends saw the ability to create a calendar using the Calendar Monkey.  While he was suitably impressed, he did also ask me if it could do something he badly needed, which was to create a column displaying the ISO week that is commonly used in Europe.  Unfortunately, the Calendar Monkey had not learned enough about ISO weeks at that time, so was unable to help. So, we sent a couple of the Monkeys back to school…!

If you are on a trial or free version of Monkey Tools, you will find that the Query Monkey will now allow you to add a custom Power Query function called GetISOWeek to your file.  From there, you can manually call this function via the Invoke Custom Function button, or via writing a formula in the Custom Column dialog within Power Query.  Simply feed the function any date column to get the ISO Week Number, and include “true” for the final (optional) parameter if you prefer the “precise” text version:

Date\Formula =fnGetISO( [Date] ) =fnGetISO( [Date], true )
Sun 30 Dec 2007 52 2007-W52-7
Mon 31 Dec 2007 1 2008-W01-1
Tue 1 Jan 2008 1 2008-W01-2

Of course, adding a new function in to your workbook is great, but for our Pro users, the Calendar Monkey wanted to make it even easier, and added it as a default column choice.  No fuss, no mess, just choose the ISO date formats you need and let the Calendar Monkey do the rest!

The new ISO Week options displayed on the Calendar Monkey form

Measure Monkey – Basic Explicit Measures

While we are also super proud of our Measure Monkey who will help create Multiple Explicit Measures, we also realize that there are times where you need to create individual measures.  For this reason, we trained another Measure Monkey to do exactly that.

The new Basic Explicit Measures feature shown on the Measure Monkey menu

The Measure Monkey that focuses on Basic Explicit Measures provides you with a no-code experience to create… well… basic explicit aggregations.  (Yes, you could make Implicit versions via drag and drop, but serious modelers far prefer the more customizable and scalable explicit versions.)

This Measure Monkey will help you create these measures without writing a single line of DAX (although it does show you the DAX it has created.)  You will be provided a list of relevant aggregations (go home COUNTA!) and smart default formatting choices.  The Monkey will even capture your preferred defaults to make you even faster next time.

Side by side vide of creating a SUM and LASTDATE aggregation with the Basic Explicit Measure Monkey

And, like its brother who builds Multiple Explicit Measures, this Measure Monkey will work for you for free!

Support for Non-English Queries

Did I mention that my friend whom I referred to above, runs a French version of Excel?  Unfortunately, Monkey Tools had some challenges reading the queries in his model correctly.  While we have always claimed that we only support English versions of Excel, this still bothered us.

One interesting part about being a coder is that MOST coding is written in English. But every now and then, Microsoft localizes something that we did not expect.  So was the case with the underlying Power Query connection name.  To make a long story short, I have now learned that “Query” is “Requête” in French, “Abfrage” in German, and has other localized words among other languages.  And now that we know?  We have retrained our tool to deal with this challenge.

What this means to you if you are a user of a non-English version of Excel is – while we are not quite ready to say we fully support all non-English versions of Excel – we do believe Monkey Tools should work no matter the localization of your Excel install.  (We do still recommend caution here.  Until we say we OFFICIALLY support all languages, please do try the Trial version before you buy, and let us know if Monkey Tools has any issues reading your queries!)

Feedback Mechanisms

Another question we received from time to time was “How do I give you feedback?” or “How do I report a bug?”  It was enough that we realized that we had done a poor job of giving you a mechanism to do so.  So to that end, we have added the following to the Monkey Tools Help menu:

  • Log a Bug
  • QuerySleuth Indenter Issues (for issues specific to QuerySleuth indentation)
  • Feature Suggestions

Each takes you to a form that you can fill out to get in contact with the dev team.  And yes, we are open to hearing your suggestions!

Various Other Bug Fixes

Of course, no release would be complete without a few bug fixes.  There were a half dozen fixes that were included in the various November updates (plus another half dozen published last night.)  Each was minor, and not really worth mentioning on their own, but rest assured that we are trying to fix bugs whenever we find them.

What is the Current Version?

To make sure you have all of the current features, go to Monkey Tools -> Options.  If you are running a version that is less than 1.0.7640.41496, then click Check for Updates Now to update.

And if you don’t have Monkey Tools installed yet… what are you waiting for?  You can try the pro features for free for two weeks, and there are a ton of useful tools even if you don’t elect to purchase a pro license.  Click here to get your copy of Monkey Tools.  And hey… if you decide to upgrade to a Annual Pro license today, you can get 20% off with the code BF20MONKEYTOOLS.

So… What’s Next?

We are working on something cool that will help Excel modelers get started quickly.  And if you want to be one of the first to hear about it and see it in action you should attend the inaugural KSA Excel Power Platform meetup, as I’ll be demoing this new feature.

 

Introducing the Measure Monkey

You know the drill… extract, transform and load your data, relate your tables, then create basic DAX measures.  All work that needs to be done before you can really get started on analyzing your data.  Today we’ve unleashed the Measure Monkey to help speed up that process a bit for you.  (You can think of the Measure Monkey as Quick Measures for Excel.)

If you follow Monkey Tools already, you’ll know that our goal is to help you build better models faster.  We already include helpful functions such as:

  • the ability to inject a query that can automatically switch between local folders and SharePoint folders
  • manage your queries via our Query Sleuth
  • build calendar tables on the fly against your data
  • and so much more...

But while we’ve had a nice tool to trace DAX query chains, we haven’t included a lot of DAX functionality to date.  That is changing today.  And oh… before we dive into it, I want to be clear that this feature will be available to ALL users of Monkey Tools.  Yes, even those of you using a Free license!

The Sample Model

Before we dive into this, let’s take a look at a sample data model:

Notice that everything is nicely created and linked (by the way - we created that calendar in a few seconds with Monkey Tools’ Calendar Monkey…) but that there are no DAX Measures on our Sales and Budget tables.  Date and Category are both foreign keys that link each of the those tables to the Calendar and Categories tables.  However, we really want explicit measures to sum both the Sales[Amount] and Budget[Amount] columns.

Of course, these measures are easy to write, but what if your model is a bit more complicated and there are ton of them to do?

Creating Explicit Measures in Bulk with the Measure Monkey

As of version 1.0.7599.31348, you’ll find a new Measure Monkey menu on the Monkey Tools ribbon for this exact purpose:

Step 1A: Which Tables Host The Columns To Aggregate?

When you launch the new feature, you’ll be taken to a screen that looks like this:

This screen in intended to allow you to tell the Measure Monkey which tables hold the columns you need to aggregate.  Our aim in this screen is to give you the highest possible chance of just clicking "Next". That being said, we realize that this may not work for everyone, so we also allow some flexibility here.

In the top left, we pre-select the tables which we believe have the highest chance of needing aggregation: your fact tables.  (Those tables with only ‘many’ sides of relationships attached to them.)  But if we get this wrong for you, you simply need to check the other boxes to include basic aggregations for other tables.  (Ideally, you shouldn’t be aggregating dimensions, but there are – of course – exceptions to every rule.)  You’ll get immediate feedback in the box in the bottom left, as we show all the tables that will be included based on your checkbox selections.

Step 1B:  Tell the Measure Monkey Where to Store Your New Measures

In the top right, we also allow you to tell us where you want to store the measures.  If you have created a specific “Measures” table, we’ll provide that by default.  If you haven’t, we’ll offer to store the measures on the Host Table.  (In other words, all measures created to aggregate columns from the Budget table will be stored on that table, whereas columns from Sales will be stored on Sales.)

Forgot to set up a new Measures table before doing this?  No worries, click the + to add a measures table on the fly, give it a name, and we’ll create it for you:

There are a couple of Advanced options as well, but we believe most people will want to leave these set based on their defaults.  So let’s click Next, to go to page 2…

Step 2: Choose Your Aggregations

This page contains a ton of info, but again we’re trying to provide you the biggest chance of clicking “Create” right away. Unfortunately, this is something that we can’t do in the image above…

The reason our Create button is disabled is that we have two measures offered with the name “Sum of Amount”.  The blue one is the first instance, and any subsequent measures with the same name will highlight in red.  So let’s fix those, and choose a default data type format:

It’s all good to go now, except that I want to add a “Transactions” measure that counts the rows of the Sales table.  So I’m going to click the “Add another aggregation” button in the Sales table. Then I choose the name of the table from the drop down list:

That will give me a new row with a “Count Rows of Sales” measure, which I can quickly rename to “Transactions” before clicking “Create”.

During this process, the Measure Monkey will create your measures for you. Plus, if you created a Measures table, it gives you some advice on how to make it an “official” measures table.  You can see the results in my data model here:

That was Easy…

The demo above was obviously a fairly simple model. Yet it cuts my explicit measure creation time down to less than a minute to create these two measures.  Now consider the time savings when you get a bit more complicated:

So how do you get the Measure Monkey menu?

This update to Monkey Tools is available in Monkey Tools 1.0.7599.31348 or higher.  And it's will be a “forever free” feature, so you’ll be able to use it on either a Free or Pro license!

To try our free trial, head over to the Monkey Tools product page to download your copy.

If you already have Monkey Tools installed, it will automatically update within a couple of weeks. Alternatively, you can request the update now by going to Monkey Tools -> Options -> Check For Update Now…

Update to Monkey Tools QuerySleuth

We've been kind of quiet here, but we're excited to announce that we've just published an update to Monkey Tools QuerySleuth feature.  It now contains an "tabbed" experience so that you can easily flip back and forth between queries, "pinning" the ones you want to see and compare.

The Updated QuerySleuth Interface

In this case you'll notice that I pinned The ChitDetails and ChitHeaders queries, then selected the Locations query from the left menu.

An image of the update to Monkey Tools QuerySleuth showing the new tabbed interface indicating two pinned queries and two modified queries

Why does this matter?  Did you notice that the ChitDetails and Locations tab names are both red?  That's because I made changes to both of them to update a data type... I can now hold onto those changes as I flip back and forth between JUST the queries I want to keep in focus.

Updating Multiple Queries

But now, of course, I want to commit my changes and force the data model to update to reflect those changes.  In this image, I'm doing just that, with three queries:

An image of the QuerySleuth prompting the user to ask which queries they want to save and refresh

And due to the selection pointed out by the arrow, each of these queries will not only get saved back to the Power Query engine, but a refresh of each query will be triggered as well.

So how do you get this update to Monkey Tools QuerySleuth?

This update to Monkey Tools QuerySleuth is available in Monkey Tools 1.0.7553.5975 or higher.  And it's available in both the free and Pro versions of the tool.  (Of course, you will still need a Pro version in order to actually save your queries.)

To try our free trial, head over to the Monkey Tools product page to download your copy.

If you already have Monkey Tools installed, it will automatically update within a couple of weeks, or you can request the update now by going to Monkey Tools -> Options -> Check For Update Now…

Creating Power Query Based Calendars

We’re super excited to announce that we have given the Monkey Tools calendar creator feature an upgrade.  In fact, it is so much of a power up, that this feature has graduated into its own full grown monkey!  We call it the Calendar Monkey.

In our initial version, the Calendar Creator would create the queries necessary to load a single column calendar of unique dates into the data model and add PeriodID columns for 364-day calendars like 445 and 13 fiscal periods.

The Calendar Monkey adds a couple of pretty powerful features to this original mix.  Let’s walk through the experience quickly to see what I mean.

Step 1:  Define the Calendar Boundaries

Step 1 screen of the Calendar Monkey, allowing you to choose your calendar type, the calendar start/end boundaries and year end

Not much has really changed here.  You can still:

  • Pick your calendar type (12 month, 13 month, 445, 454, 544), and define your custom year end (including a different month for 12 month calendars.)
  • Define a name for your calendar table query
  • Choose the load destination
  • Pick any valid date columns for the Start and End date of your data

These settings allow our monkey to build the calendar to dynamically span the entire range of your data on every refresh.

The only real difference here is that we’ve added a checkbox and a Next button.  (Notice that Create is still available, if you just want to accept the monkey’s default choices for the rest of the options you’re about to see.)

Step 2:  Choose Calendar Columns

One of the things that always bothered us about our original version is that it created the calendar’s Date column, but then left it up to you to add the different date formats that you wanted.  So we decided to improve that, as you can see here:

Step 2 of the Calendar Monkey screen provides a series of check boxes allowing you to specify which date format columns you'd like to add to your model

Our monkey pre-selects the most common date formats, but if you ever check/uncheck one, it will learn your preferred defaults and provide those next time you go to inject a calendar.  The columns shown dynamically react to your choices in Step 1 as well… if you use a Dec 31 year end for a 12 month calendar, you only need the first two columns – so that’s what the Calendar Monkey will show you.  If you have a custom year end (like Jun 30 or Sep 30), you may also want Fiscal columns, so the monkey provides those as options too.  And if you work with a 364-day calendar like a 445 variant or a 13 fiscal periods calendar… there is a final column of PeriodID’s that shows up in that blank spot too.

Step 3: Adding Relationships

Depending on your choices in Step 1, the Calendar Monkey will determine if you will be presented with this page or if it will be skipped.  If you choose to load your calendar to the Data Model (or Data Model & Worksheet), the monkey will list every date column loaded to the data model.  The purpose of this is simple; let you decide if your new calendar table should be linked to any of those columns listed.

For any columns you check, Calendar Monkey will do its best to create those relationships after loading your table to the data model.  (There are some things that can prevent the monkey from accomplishing this, such as creating an inactive relationship.)

Step 3 of the Calendar Monkey provides checkboxes for each date column in the data model, allowing you to declare which tables you'd like to relate your calendar table to.

Step 4: Creating the Calendar

Even though the Calendar Monkey has a lot of work to do when you click create, it also knows the value of good feedback.  For this reason, it will update you as to the progress as it completes all the individual tasks, as you can see here:

This screen shows feedback from the Calendar Monkey, letting you know what has been done, as well as what you need to do need.

Now unfortunately, there are a couple of things that the monkey is unable to do (thanks to a lack of security clearance with the Excel data model).  Rather than just ignore these essential tasks, however, it will tell you what needs to be done, with the exact steps to do so.

As a bit of a pro-tip here… you don’t actually need to close the Calendar Monkey window to take action on those steps… so keep it open until you’ve hopped into the data model and made the advised changes!  Of course, if you understand what’s happening, and know the steps you’ll need to take, there is also an option to automatically close the summary screen upon completion as well.

Looking at the data model with the new calendar table already related to the other tables

What the Calendar Monkey cannot do

There are three things that the Calendar Monkey can’t do at this time:

  1. Automatically hide the keys on the “many” side of the relationship (the foreign keys)
  2. Automatically create the sorting hierarchies to sort Month Name by Month Number and Day Name by Weekday Number
  3. Create the calendar in a Power BI file

Honestly, while we joked earlier that the Calendar Monkey doesn’t have security clearance, the reality is that there is a limitation with the data model’s extensibility model which is preventing us from solving the first two items.  The third… its on our backlog.

So how do you get the new Calendar Monkey?

You need Monkey Tools version 1.0.7493.29574 or higher, and you'll have the Calendar Monkey ready to do your bidding.

If you haven’t already, head over to the Monkey Tools product page to download a copy

If you already have Monkey Tools installed, it will automatically update within a couple of weeks, or you can request the update now by going to Monkey Tools -> Options -> Check For Update Now…

Name Worksheets After Queries

Have you ever loaded a Power Query to a worksheet and then changed the name to match the query? It's a shame that there is no option to name worksheets after queries, as this would be handy.

Well, after seeing this request come up in the forums last week, we thought that this would be a great feature to add to the DestinationSleuth in Monkey Tools.  So as of build 1.0.7433.38066... it's done!

How to Name Worksheets After Queries

We wish that we could add this as an option in the Close & Load dialog, but sadly that's not possible.  So we did the next best thing...  Once you've loaded your queries, you simply need to open our DestinationSleuth and:

  1. Select the queries you're after
  2. Click the Rename Sheets button

Using DestinationSleuth to select queries and change the names of their host worksheets

At that point, we'll quickly loop through the host worksheets and rename them to match the query landed to that sheet.

Name Worksheets After Queries While Changing Load Destinations

You might also notice a new checkbox called "Name Sheets After Queries".  This checkbox allows you to name worksheets after queries while changing a load destination to create a new table.  It's also super easy to use:

  1. Select the query (or queries) you wish to change
  2. Choose to change the load destination to a Table
  3. Check the Name Sheets After Queries checkbox
  4. Click the Update Load Destinations button

Using DestinationSleuth to change a load destination from Connection Only to Table, and update the worksheet to the Query name at the same time.

We'll change the Load Destinations, creating the worksheets AND naming them to match the query in one step.

This is also a "Forever Free" Feature

As mentioned in my last post, while the DestinationSleuth's colour highlighting is only available in the trial and pro versions of Monkey Tools, the ability to Change Multiple Load Destinations at Once is a "forever free" feature.  And so is the ability to rename worksheets after queries!  All you need is Monkey Tools version 1.0.7433.38066 or higher, and you'll have that ability at your disposal.

If you haven’t already, head over to the Monkey Tools product page to download a copy

If you already have Monkey Tools installed, it will automatically update within a couple of weeks, or you can request the update now by going to Monkey Tools -> Options -> Check For Update Now…

 

 

Change Multiple Load Destinations at Once

Have you ever built multiple queries in one Power Query session?  You get to choose one load destination, then have to change each of the incorrect ones... one by one.  Have you ever wished you could change multiple load destinations at once?  Well now you can - you just need Monkey Tools!

Check out the New and Improved DestinationSleuth!

Following on the heels of last week's feature updates, we've added functionality to the DestinationSleuth that allows you to change load destinations of any one or more queries.

Naturally, DestinationSleuth still gives you a very visual view of the different query load destinations all in one place, but do you see that new option just to the left of the Exit button?

DestinationSleuth showing the different load destinations in colour

Assume that you've created 5 new queries in the workbook, and load them all to Connection Only.  But now you want to change 4 of them to the data model.  Rather than click each query, change the load destination, and wait, then move to the next one... Now you can just launch DestinationSleuth and:

  1. Select the load destination
  2. Select each of the queries you want to repoint (hold down your CTRL key)
  3. Click Change

DestinationSleuth being using to change multiple load destinations at once

Sure, it still takes time, but at least you can walk away and let it cook, rather than slowly shepherd it through the entire process.

Delete Host Worksheets

You'll also notice a little checkbox called "Delete Host Worksheets".  You know that issue where you accidentally load a table to the worksheet, then change the query to Connection Only?  It leaves the worksheet behind.  Now true, you can always just delete the worksheet (which will actually set the query to Connection Only automatically). But what if you need to change 5 of them?  It's easy with DestinationSleuth:

  1. Select the load destination
  2. Check the "Delete Host Worksheets"
  3. Select each of the queries you want to repoint (hold down your CTRL key)
  4. Click Change

We will repoint your queries AND remove the worksheets that were holding the query results.  (Naturally, you want to be really sure you acutally want to do this, but it's handy if you do.)

This is a "Forever Free" Feature

While the DestinationSleuth's colour highlighting is only available in the trial and pro versions of Monkey Tools, the ability to Change Multiple Load Destinations at Once is a "forever free" feature.  All you need is Monkey Tools version 1.0.7423.41125 or higher, and you'll have that ability at your disposal.

If you haven’t already, head over to the Monkey Tools product page to download a copy

If you already have Monkey Tools installed, it will automatically update within a couple of weeks, or you can request the update now by going to Monkey Tools -> Options -> Check For Update Now…

Monkey Tools Update Now Available

We’re pleased to announce the first Monkey Tools update is now available for download! This one contains a new feature, some new logic and an update to one of our data connectors. Read on for more information!

I can’t believe it’s been a month since our initial release, but here we are.  Since that release, a couple of notable things happened:

  • We published a blog post on PivotSleuth and Measure Tables. This was a learning experience for me, as I discovered something new about Measure Tables.  I always knew that you could mark your Fact tables as Measure tables by hiding all the unaggregated columns, but I didn’t realize that this also means that disconnected Measure tables will then suppress the “Relationships between tables may be needed error.”
  • One of my friends hit me with an interesting curve ball: he turned on “Store datasets using enhanced metadata format” in the Power BI preview features. And as it turned out, some of the methods we’d been using to analyze the Power BI model disappear when you do that.

Both of these have led to some improvements in the software, which we’re proud to say are finally available to you.

What’s new in version 1.0.7418.29970?

There are three major things that are new in this version:

A New Power BI Connector

This was actually a huge amount of work for us, as not only did we have to build a new connector to read the new Power BI file format, but we also had to analyze the file as it was opening to see if it was in the classic format or not.  And to make it harder, if you have the Enhanced Metadata Format turned on, a legacy file requires using our initial connector, but any refresh must be done with the new connector.  Fun times for a developer and, as you can imagine, it took as a bit of effort to pull it off.  It’s actually this piece that has held us back on the other features, as the connector MUST work and impacted everything.

While most of this work is invisible to the end user, there are two things that hope you do notice:

  • Connecting to the new file format is much faster than using the legacy format.
  • We also took the time to remove the reliance on configuring the version of Power BI that launches for you by default. We now just launch Power BI using your default connector, then bind to it, no matter how many versions you have on your machine, or which they are.

Updated Functionality in the PivotSleuth

As we learned in the PivotSleuth and Measure Tables blog post, hiding all unaggregated columns on a disconnected measure table flags the table as an official Measure table and suppresses the “Relationships between tables may be needed” message.  For that reason, PivotSleuth needed to recognize that this is acceptable.

In other words, when the Measures table is a properly formatted disconnected table, it needed to (and now does) show that there are no issues:

PivotSleuth giving a clean bill of health for a measures from a disconnected measure table

But when that disconnected Measures table has a visible column, not only should it have shown the issues, but also tell you what needs to be done to fix them.  And now it does:

PivotSleuth showing issues for measures used from an improperly formatted measure table

New QueryMonkey Feature: Add Measure Table

As mentioned before, our philosophy is “Build better, faster”.  For that reason we’ve added a new QueryMonkey feature for you:  Add Measure Table.

This feature will prompt you for a name for the table…

Prompting the user to enter a name for the new Measures table

And then create a new empty table in the data model for you:

An empty table in the data model

Unfortunately, the Excel team hasn’t given us a way to programmatically hide columns in the data model (we could REALLY use that ability), so we can’t take that last critical step for you:  Hiding the Measures column to prevent the "Relationships between tables may be needed" message.  But never fear, we do tell you exactly what needs to be done:

Advice from PivotSleuth on what to do to turn the new table into a proper measure table

So while we typically store our measures on the Fact tables, rather than a disconnected Measure table, we totally get that a lot of people like this approach.  Hopefully this make it a lot easier for you!

How do I get the Monkey Tools Update?

The answer to this depends on whether or not you’ve installed Monkey Tools yet.

If you haven’t, then head over to the Monkey Tools product page to download a copy

If you already have Monkey Tools installed, it will automatically update within a couple of weeks, or you can request the update now by going to Monkey Tools -> Options -> Check For Update Now…

Happy sleuthing! 🙂

Appending Columns with Different Names

Will Thompson from the Power BI team threw out a question on Twitter today related to Appending Columns with Different Names:

Let's say I have some CSVs coming in where the column names differ from file to file, but always in the same order. I could use PQ's automatic Column1/Column2 etc. names by skipping the header row and then have them all map to the same columns in AS.  Then build visuals on top of it that'll pick up the fixed column names from AS. Has anyone written a blog or tutorial that covers that sort of scenario? I want to point someone to some guidance…

No worries, Will, I’ve got your back. 🙂

Setting the stage

To be fair, the data source (CSV, AS, Excel or whatever), really isn’t relevant.  It’s all about the process of Appending Columns with Different Names.  What we need to recognize is that we need two types of tables here:

  • Data Tables: These tables have the data, but have different headers.
  • Header Table: This table has the correct headers for the data

And we also need to remember that the Data Table and Header Table columns are always in the same order.

Step 1:  Prep the “Data” tables

Preparing the data tables is pretty easy.  If the different column names are already showing in the headers like this:

A data table with headers

Demote and remove them by going to:

  • Home -> Use First Row as Headers -> Use Headers as First Row.
  • Remove Rows -> Remove Top Rows -> 1

Your table should now have headers called Column1, Column2, etc…

A data table with headers demoted into the first row

Do this for each table, then set each to be a staging query:

  • Excel: Load the query as a Connection Only query
  • PowerBI: Disable the Load of the query

Step 2: Prep the Header Table

Connect to your data, which might look like this:

A data table with headers

Clean it up by going to:

  • Home -> Use Headers as First Row -> Use First Row as Headers.
  • Keep Rows -> Keep Top Rows -> 1

You should now have a 1 row query that shows the names of the column headers.

A single row table with headers demoted to be the only row

It's time to save it:

  • Rename the Query as “Headers”
  • Set it to load as a staging query (as above)

Step 3: Combine the Data

This part is simple:

  • Right click the Header table -> Reference
  • Go to Home -> Append Tables -> Three or more tables
  • Add each of the data tables and click OK
  • Go to Use First Row as Headers
  • Set your data types

And you’re done:

A complete data table with the correct headers

Sample file (Excel) is available here.  But you can import it to Power BI if you prefer that look.

PivotSleuth and Measure Tables

Last week I got an email from one of our readers, with some feedback related to how Monkey Tools’ PivotSleuth works with Measure tables in Power Pivot.

Best practices is to set up a disconnected table to house DAX measures.  Because of this - all of the fields listed in the Pivot Sleuth end up being RED.  So it is not really useful.   Is there away around this?  Other than incorporating your DAX measures in an actual table?

And is this the reason for the " You might need a relationship" annoyance from Excel?

Personally, I challenge the term “best practices” here, as I think it’s borne a bit out of history, and no longer relevant.  But more on that below...

My favourite part about this email was the last line, as this specific problem is actually one of the main reasons we wrote this feature.  Let’s take a look and see what PivotSleuth says about this…

Model Background

The model framework I’m using looks like this:

Framework of a sample data model

As you can see, we have a separate _Measures table in this instance, where all the measures are collected, rather than storing the measures on the Sales & Budget tables.  They’re not complex at all, in fact, they’re about the easiest measures you could create:

  • Sales $: =SUM( Sales[Amount] )
  • Budget $: =SUM( Budgets[Amount] )

And when you drag them on to a PivotTable, they work nicely:

PivotTable using the measures we created

So, if everything is fine, what’s the issue?

It’s all about this nagging little message:

The annoying "Relationship May Be Needed" error

Why is it here?

PivotSleuth and Measure Tables

When you launch PivotSleuth against this PivotTable, you see the following:

PivotSleuth and Measure Tables showing highlighted fields

Why are all the fields red?  The answer is shown when you select one of the measures:  there is no relationship between the Sales, Categories, or Calendar tables.

So, what happens when you store these measures on the Sales and Budget tables?  The irritating message goes away:

Updated PivotTable where the measures are stored on the Sales and Budget tables

(Interested in the other things Monkey Tools does?  Check it out here!)

Why do People Create Measure Tables Anyway?

The practice of storing measures on another table as a recommended practice was born out of Power Pivot instability, back when things crashed a ton.  Sometimes the fix would require removing the table from the data model and re-creating it, at which point you’d lose any measures or relationships built on those tables.  It was frustrating and annoying, and led people to keeping their measures into a separate table to protect themselves from having to do that work.  The challenge, however, was that it caused a “Relationships May be Needed” message every single time you used a measure.  And there was no way to make that go away.

Since 2016, Microsoft focused on fixing bugs related to Power Pivot, with many of them making their way back into the Excel 2016 product, even if they were fixed after 2019 was released.  While they’re certainly not all gone, it’s unusual to see issues that force the need for tables to be deleted and rebuilt now.  To me, this reason for separating your DAX has basically become a non-issue, but the habit still remains.

Some people also argue that this gives you a central place to go to get your measures.  I’d argue that the list can become overwhelming when all your measures are in one folder without any categorization.  (Unlike Power BI, we can’t group measures into folders.)

I far prefer to put my measures on the appropriate tables, then hide all the unaggregated columns on the table.  This offers three benefits:

  1. It groups the measures by table, making them easier to find.  (Sure, you can use the search function, if you like. I’m just saying those tables act like folders.)
  2. It means the “Relationships May be Needed” warning only shows up when a measure can’t be cross filtered by a natural relationship in the model.
  3. It changes the icon of the table to the sigma icon (?), which is synonymous with measures.

The Benefits of Hidden Columns

Let’s take a quick look at the benefits of hiding every unaggregated column in the data model, essentially leaving only measures visible on your table.  This is super easy to do: you just hop into the Power Pivot window, right click every natural column and choose “Hide from Client Tools”.  (I personally prefer to do this in Diagram view, but it works in table view as well.)  The results are pretty impactful when you look at the PivotTable field list:

The results of hiding every unaggregated column

Notice that the tables are now flagged as “Measure Tables”.  They inherit the sigma icon, as well as move to the top of the list.  This is the primary reason I prefer to work this way… the measures are grouped logically based on their tables.  Essentially, they act like Measure Folders.

Full Disclosure on PivotSleuth

As I was writing up this post, I discovered something that I hadn’t tripped on yet.  Look what happens when I hide all unaggregated measures on a Measure table:

Measure table with all unaggregated measures hidden

Notice that there is no “Relationships May be Needed” message.  I didn’t realize that this flag would change that, and as of today Monkey Tools doesn’t check for this, so still shows all read.  So, it looks like we need to update our logic a bit to add some more functionality.  🙂

Conclusion

Hopefully we both learned something here!

Personally, I’m sticking with the individual table approach, and storing my measures as close to the table they summarize.  I like the ability that it categorizes my measures.  But ultimately, it’s cool that we have the ability to work the way we want to work.

And we’ll look at modifying Monkey Tools to avoid showing red when – and only when – you’ve hidden every unaggregated column on your measure table.

PS:  Did you know that you can’t call a table “Measures”?  It’s a reserved word, so will give you a table called “A”.  That’s why I went with “_Measures”