Will Cover BISM Normalizer in SQLBits Session

I’m looking forward to SQLBits!  My published session title and abstract are not totally accurate. In addition to currency conversion/MDX/DAX, I will cover BISM Normalizer. I tried to get the title and abstract changed, but I missed the boat. Here is the corrected version.

BISM Normalizer, MDX vs. DAX & Currency Conversion

This session discusses the benefits of database compare for SSAS tabular models. It covers BISM Normalizer, which is a free tool developed for the SQL community. We will also contrast how multidimensional and tabular often achieve the same end result in quite different ways. Learn where multidimensional decorates the metadata of the model, while tabular offloads responsibility to DAX. Features covered can be used in a broad range of scenarios – not only currency conversion – adding valuable knowledge to your SSAS arsenal.

Deploy Tabular Models and Retain Dynamic Partitions

Kasper de Jonge asked a question on Twitter that I would like to answer here to a) give it more exposure, and b) I pretty quickly ran out of 140 characters in my Twitter reply.

I initially tweeted: Problem when deploy tabular models to Dev server from SSDT & lose dynamic partitions? Deploy with BISM Normalizer & retain partitions

Kasper tweeted: Hey Christian, what do you mean by dynamic partitions?

And here is my response …

Hi Kasper,

We often partition the large tables in a tabular model to speed up processing times (as I’m sure you know).  For example, we might partition by month and then process only the most recent couple of months every night – which is a lot quicker than processing the whole table.  Normally this is done in AMO code that is called from a SSIS package – so it can be managed by the ETL to increment/drop partitions, etc.  In this case, the model is not partitioned when we are developing in SSDT.

While developing in SSDT, I frequently deploy to a dev server (obviously not test/production).  This is necessary because all the dev reports are pointing at the server – not the workspace database on my laptop.  Besides, my laptop happens to not have 50 GB of memory on it (harsh but true).  If the dev ETL has built partitions, they are lost upon deployment from SSDT (using right-click deploy) – and they take a long time to rebuild.

When creating a BISM Normalizer difference comparison, we have the option to “include partitions in table definitions”.  If we leave this unchecked, partitions will not be considered when comparing tables.  BISM Normalizer will treat such tables as equal and will not mess with them – thereby avoiding reprocessing.

Include Partitions Option

The SSAS Deployment Wizard is the only other deployment method I’m aware of for tabular models that supports “retain partitions” functionality.  However, you would not use it to deploy from SSDT to the dev server when making a change to a DAX calc or something.  For Test/Production deployments you may prefer to use the deployment wizard – over maybe using BISM Normalizer to easily create an xmla diff script to associate with a release.

BIDS Helper‘s Deploy MDX Script feature serves a similar purpose for multidimensional.

I would not be surprised (and I have heard nothing from Microsoft to this effect) if the deployment story for SSAS in 5 years time looks more like the deployment story for relational databases with schema compare/dacpacs, etc.

Cheers!

Christian

BISM Normalizer: Version 1.3.8 Released!

Download it from the BISM Normalizer Visual Studio Gallery page.

Enhancements in 1.3.8.0

  • Support for Visual Studio 2012. To run 2010 and 2012 on the same machine, point them both at the same local Addin folder in Visual Studio options.

Update 3/28/2013: Enhancements in 1.3.8.3

  • Allows comparison of tabular projects that are nested within Visual Studio solution folders.

Why not BISM Normalizer work directly with PowerPivot workbooks?

Gerhard Brueckl asked a question on the BISM Normalizer page Q&A section that I would like to answer here to give it more exposure.

Gerhard wrote …

Hi Christian,

first of all – thanks for that awesome tool!
Is it somehow possible to compare a Server/Project to a PowerPivot Workbook and vice-versa?
Office 2013 introduced a new object model for PowerPivot where you can access the PowerPivot – is it possible to extend BISM Normalizer to make use of this?

greets,
gerhard

My response …

Hi Gerhard,

Thanks for the feedback! Others have asked the same question including Grant Paisley.

I would love to have BISM Normalizer work directly with PowerPivot. Not only allow selection of an Excel file to act as a source/target model, but also have an Excel Addin version of BISM Normalizer to support non-Visual Studio users.

I agree there is a strong use case for this. PowerPivot enables users to easily pull data from uncommon sources – and often mash it up with corporate data. So inheriting another tabular model’s metadata would be useful to say the least.

PowerPivot now has the API you mentioned, but it would be much easier if Microsoft provided a single API for both Analysis Services and PowerPivot. The easiest way for me would be if Microsoft allowed access to the PowerPivot AMO interface, which the tabular model designer (I’m assuming even in PowerPivot) uses. Instead, Microsoft has chosen to keep this interface private.

I built an object model on top of AMO for BISM Normalizer. It contains classes for all the objects that BISM Normalizer displays on the differences grid and abstracts calls to AMO. But even so, under the abstraction layer, BISM Normalizer basically just copies AMO objects from source to target. It does not need to rebuild them, which would be required to get them from AMO to DataModel OM in Excel 2013 and vice versa.

I raised this Connect item for this purpose. Feel free to vote for it.

Thanks,
Christian

IF ISCROSSFILTERED performs better than IF HASONEVALUE

See this post for the functional difference between ISCROSSFILTERED and HASONEVALUE.

This DAX query on AdventureWorks runs in less than a second (SP1).

DEFINE
MEASURE 'Date'[Bad Measure] =
   COUNTROWS(GENERATE(ALL('Internet Sales'), VALUES('Date'[Week Of Year])))
MEASURE 'Date'[Conditional Evaluation] =
   IF(ISCROSSFILTERED('Product'[Product Name]), [Bad Measure], 0)
//  IF(HASONEVALUE('Product'[Product Name]), [Bad Measure], 0)
EVALUATE
ADDCOLUMNS(
   VALUES('Date'[Calendar Year])
   ,"Conditional Evaluation", [Conditional Evaluation] //('Product'[Product Name] = "Mountain Pump")
)
ORDER BY 'Date'[Calendar Year]

Uncomment the product filter ['Product'[Product Name] = “Mountain Pump”] to include it. Now it takes about 8 seconds. Looking at the query, this makes sense because it has to call the “Bad Measure”. This suggests that IF ISCROSSFILTERED is optimized to perform conditional logic only when necessary. This behavior is similar to short-circuit evaluation, but not quite the same thing.

Now comment out the line containing ISCROSSFILTERED and uncomment the line containing HASONEVALUE. It takes about 8 seconds whether the product filter is applied or not. This suggests that using IF HASONEVALUE is not optimized; it does perform conditional logic unnecessarily.

Having watched Marco Russo’s Inside DAX Query Plans session from the PASS 2012 Summit, I ran Profiler traces to confirm this theory. For the above query at least, this does seem to be the case.

  • Even when the product filter is not applied, IF HASONEVALUE executes a Vertipaq SE Query on ‘Internet Sales’[RowNumber]. RowNumber is an internal, unique column on every table.
  • The physical DAX query plan includes the following operation, which I believe means it is iterating 19.2 million rows for the formula engine to deal with.
Spool_Iterator<Spool>: IterPhyOp IterCols(0, 108, 136)('Date'[Calendar Year], 'Internet Sales'[RowNumber], 'Date'[Week Of Year]) #Records=19206564 ...

Note: 19,206,564 = 60,398 (number of rows in Internet Sales) * 53 (number of distinct Week of Year values) * 6 (number of rows in the resultset)

Needless to say, none of this happens with IF ISCROSSFILTERED when the product filter is not applied. There are less Vertipaq SE Queries, and the largest Spool_Iterator operation in the physical plan is for 6 rows (the number of rows in the resultset).

BISM Normalizer: Version 1.3.7 Released!

Download it from the BISM Normalizer Visual Studio Gallery page.

Enhancements/Bug Fixes in 1.3.7.1

  • Fix to allow for breaking change in the way SP1 serializes measures to the BIM file. Tested on SP1.
  • No longer disallow creation of relationships where the parent column is not a “unique-value column”.  Will now make it unique if necessary.
  • Supports pasted tables from Excel.
  • Removed equality column (“=”, “≠”, …)
  • Tooltip displayed for items where the action column is disabled due to a parent selection …
Tooltip

Update 11/26/2012: Bug Fixes in 1.3.7.2

  • Fix for when using the option to “merge perspective selections (not replace)”, some selections may be left out.

Update 1/25/2013: Bug Fixes in 1.3.7.5

  • Fix for if a table in a perspective has no columns/hierarchies selected, but it does have measures selected, it may not be displayed in the object definition for the perspective. This did not affect updating/creating perspectives; it was a purely cosmetic bug.

SSAS Tabular SP1: Change in how measures are serialized can cause problems

SP1 changed the way measures are stored in the bim file for tabular models with compatibility level of SP1 (1103).  Instead of having a single command with all the measures, it now creates a command per measure with annotations. This can cause issues with old versions of BISM Normalizer.  This is fixed in Version 1.3.7, which is tested with SP1.

This can also cause problems for other tools that save measures back to the bim file – like the DAX Editor.

Follow

Get every new post delivered to your Inbox.