Will Cover BISM Normalizer in SQLBits Session

I’m looking forward to SQLBits!  My published session title and abstract are not totally accurate. In addition to currency conversion/MDX/DAX, I will cover BISM Normalizer. I tried to get the title and abstract changed, but I missed the boat. Here is the corrected version.

BISM Normalizer, MDX vs. DAX & Currency Conversion

This session discusses the benefits of database compare for SSAS tabular models. It covers BISM Normalizer, which is a free tool developed for the SQL community. We will also contrast how multidimensional and tabular often achieve the same end result in quite different ways. Learn where multidimensional decorates the metadata of the model, while tabular offloads responsibility to DAX. Features covered can be used in a broad range of scenarios – not only currency conversion – adding valuable knowledge to your SSAS arsenal.

IF ISCROSSFILTERED performs better than IF HASONEVALUE

See this post for the functional difference between ISCROSSFILTERED and HASONEVALUE.

This DAX query on AdventureWorks runs in less than a second (SP1).

DEFINE
MEASURE 'Date'[Bad Measure] =
   COUNTROWS(GENERATE(ALL('Internet Sales'), VALUES('Date'[Week Of Year])))
MEASURE 'Date'[Conditional Evaluation] =
   IF(ISCROSSFILTERED('Product'[Product Name]), [Bad Measure], 0)
//  IF(HASONEVALUE('Product'[Product Name]), [Bad Measure], 0)
EVALUATE
ADDCOLUMNS(
   VALUES('Date'[Calendar Year])
   ,"Conditional Evaluation", [Conditional Evaluation] //('Product'[Product Name] = "Mountain Pump")
)
ORDER BY 'Date'[Calendar Year]

Uncomment the product filter ['Product'[Product Name] = “Mountain Pump”] to include it. Now it takes about 8 seconds. Looking at the query, this makes sense because it has to call the “Bad Measure”. This suggests that IF ISCROSSFILTERED is optimized to perform conditional logic only when necessary. This behavior is similar to short-circuit evaluation, but not quite the same thing.

Now comment out the line containing ISCROSSFILTERED and uncomment the line containing HASONEVALUE. It takes about 8 seconds whether the product filter is applied or not. This suggests that using IF HASONEVALUE is not optimized; it does perform conditional logic unnecessarily.

Having watched Marco Russo’s Inside DAX Query Plans session from the PASS 2012 Summit, I ran Profiler traces to confirm this theory. For the above query at least, this does seem to be the case.

  • Even when the product filter is not applied, IF HASONEVALUE executes a Vertipaq SE Query on ‘Internet Sales’[RowNumber]. RowNumber is an internal, unique column on every table.
  • The physical DAX query plan includes the following operation, which I believe means it is iterating 19.2 million rows for the formula engine to deal with.
Spool_Iterator<Spool>: IterPhyOp IterCols(0, 108, 136)('Date'[Calendar Year], 'Internet Sales'[RowNumber], 'Date'[Week Of Year]) #Records=19206564 ...

Note: 19,206,564 = 60,398 (number of rows in Internet Sales) * 53 (number of distinct Week of Year values) * 6 (number of rows in the resultset)

Needless to say, none of this happens with IF ISCROSSFILTERED when the product filter is not applied. There are less Vertipaq SE Queries, and the largest Spool_Iterator operation in the physical plan is for 6 rows (the number of rows in the resultset).

SSAS Tabular SP1: Change in how measures are serialized can cause problems

SP1 changed the way measures are stored in the bim file for tabular models with compatibility level of SP1 (1103).  Instead of having a single command with all the measures, it now creates a command per measure with annotations. This can cause issues with old versions of BISM Normalizer.  This is fixed in Version 1.3.7, which is tested with SP1.

This can also cause problems for other tools that save measures back to the bim file – like the DAX Editor.

Currency Conversion: Formatting Symbols in Tabular

This article on currency conversion was written on my old blog in August 2006.  It still applies today for Multidimensional in 2012.  One of the items covered is how to format the currencies so they look like the following rather than all use one currency’s symbol.

I created a couple of extra columns in my currency dimension table.  My examples use the LCID column for multidimensional and FormatString for tabular.

You can get the LCIDs from here.  An easy way to get the format strings is to select Format=Currency and set the currency symbol in the properties window of SSDT.  This internally sets the FormatString property to the custom string.

Multidimensional

Set the currency attribute’s ValueColumn property to the LCID column.

And then the MemberValue function in MDX script to set the Language property.

Language(([Currency].[Currency].Members, [Measures].[Internet Sales Amount])) = [Currency].[Currency].CurrentMember.MemberValue;

Tabular

I couldn’t find a way to use LCID in tabular, but we can use the FORMAT function and pass in the FormatString value from the currency table.

DEFINE
MEASURE 'Internet Sales'[Internet Sales] =
    SUMX
    (
        'Currency Rate'
        , CALCULATE( Sum('Internet Sales'[Sales Amount]) / Sum('Currency Rate'[End of Day Rate]) )
    )
MEASURE 'Internet Sales'[Internet Sales FORMATTED] =
    FORMAT
    (
        [Internet Sales]
        , VALUES('Currency'[Format String])
    )
MEASURE 'Internet Sales'[Internet Sales ALL MEMBER DOLLARS] =
    IF
    (
        HASONEVALUE('Currency'[Currency Name])
        , [Internet Sales FORMATTED]
        , [Internet Sales FORMATTED]('Currency'[Currency Name] = "US Dollar")
    )
EVALUATE
ADDCOLUMNS
(
    FILTER
    (
        VALUES('Date'[Calendar Year])
        , [Internet Sales] > 0
    )
    ,"US Dollar", [Internet Sales ALL MEMBER DOLLARS]( 'Currency'[Currency Name] = "US Dollar" )
    ,"United Kingdom Pound", [Internet Sales ALL MEMBER DOLLARS]( 'Currency'[Currency Name] = "United Kingdom Pound" )
    ,"Brazilian Real", [Internet Sales ALL MEMBER DOLLARS]( 'Currency'[Currency Name] = "Brazilian Real" )
    ,"Saudi Riyal", [Internet Sales ALL MEMBER DOLLARS]( 'Currency'[Currency Name] = "Saudi Riyal" )
    ,"ALL MEMBER", [Internet Sales ALL MEMBER DOLLARS]
)
ORDER BY 'Date'[Calendar Year]

In multidimensional, we can use the following properties to hide the all member and choose the default currency – summing multiple currencies obviously doesn’t make sense.

  • IsAggregatable=False removes the all member
  • DefaultMember set to the desired default currency

Given that tabular doesn’t have anything equivalent to these features at this time, we need to check if we are at the all level of the currency dimension using HASONEVALUE(‘Currency’[Currency Name]).  The above example sets the all member to US Dollars.  This also covers some clever clogs selecting a multi-value filter on currency.

CurrentMember vs. HASONEVALUE

The closest thing in DAX to the MDX CurrentMember function is the HASONEVALUE function.

One of the most common uses for CurrentMember in MDX is to check if at the all level of a hierarchy. For example …

with member measures.x as
iif([Product].[Product].CurrentMember is [Product].[Product].[All Products],
    'do not apply special calculation',
    'apply special calculation because filtered by a product'
   )
select
measures.x on 0,
[Date].[Calendar].[Calendar Year].Members on 1
from [Adventure Works]
--where [Product].[Product].&[555] //slice by a product - changes evaluation in iif

Here is the DAX equivalent.  Suffice to say that when “x” is returned in the resultset, all products are in filter context (equivalent to the all member in MDX) .  But when “y” is returned, only one product (Thin-Jam Hex Nut 3) is in filter context.

define
measure
    'Date'[x] = if( hasonevalue('Product'[Product Name]),
                    "apply special calculation because filtered by a product",
                    "do not apply special calculation"
                  )
measure
    'Date'[y] = calculate([x], 'Product'[Product Name] = "Thin-Jam Hex Nut 3")
evaluate
addcolumns
(
   values('Date'[Calendar Year])
  ,"x", [x]
  ,"y", [y]
)

Returns

To prove that HASONEVALUE does what it says on the tin …

define
measure
    'Date'[x] = countrows(values('Product'[Product Name]))
measure
    'Date'[y] = calculate([x], 'Product'[Product Name] = "Thin-Jam Hex Nut 3")
measure
    'Date'[z] = calculate([x], 'Product'[Product Name] = "Thin-Jam Hex Nut 3" ||
                               'Product'[Product Name] = "Bearing Ball")
evaluate
addcolumns
(
   values('Date'[Calendar Year])
  ,"x", [x]
  ,"y", [y]
  ,"z", [z]
)

Returns

So “y” indeed has one value.

But what about “z” above? It has 2 values. The MDX equivalent of this is a set of products in the where clause – which CurrentMember can’t handle. And this is one of DAX’s strengths over MDX: we can deal with multi-value filters more easily.

Consider this scenario: apply a conditional calculation only if the user has filtered on one or more products (but not if there is no product filter at all).

HASONEVALUE of course won’t deal with it.

define
measure
    'Date'[x] = if( hasonevalue('Product'[Product Name]),
                    "apply special calculation because filtered by product(s)",
                    "do not apply special calculation"
                  )
measure
    'Date'[y] = calculate([x], 'Product'[Product Name] = "Thin-Jam Hex Nut 3")
measure
    'Date'[z] = calculate([x], 'Product'[Product Name] = "Thin-Jam Hex Nut 3" ||
                               'Product'[Product Name] = "Bearing Ball")
evaluate
addcolumns
(
   values('Date'[Calendar Year])
  ,"x", [x]
  ,"y", [y]
  ,"z", [z]
)

Returns

As “z” has 2 values in filter context, we miss it.

The ISFILTER function would catch it if we only care about filters directly on the ’Product’[Product Name] column.

define
measure
 'Date'[x] = if( isfiltered('Product'[Product Name]),
 "apply special calculation because filtered by product(s)",
 "do not apply special calculation"
 )
measure
 'Date'[y] = calculate([x], 'Product'[Product Name] = "Thin-Jam Hex Nut 3")
measure
 'Date'[z] = calculate([x], 'Product'[Product Name] = "Thin-Jam Hex Nut 3" ||
 'Product'[Product Name] = "Bearing Ball")
evaluate
addcolumns
(
 values('Date'[Calendar Year])
 ,"x", [x]
 ,"y", [y]
 ,"z", [z]
)

Returns

But if the table can be filtered by other columns, we would have to check them all individually with ISFILTERED, or it won’t detect the filter.

define
measure
 'Date'[x] = if( isfiltered('Product'[Product Name]),
 "apply special calculation because filtered by product(s)",
 "do not apply special calculation"
 )
measure
 'Date'[y] = calculate([x], 'Product'[Model Name] = "Chain")
evaluate
addcolumns
(
 values('Date'[Calendar Year])
 ,"x", [x]
 ,"y", [y]
)

Returns

To detect a filter even if applied on another column on the same table, we can use ISCROSSFILTERED.

define
measure
 'Date'[x] = if( iscrossfiltered('Product'[Product Name]),
 "apply special calculation because filtered by product(s)",
 "do not apply special calculation"
 )
measure
 'Date'[y] = calculate([x], 'Product'[Model Name] = "Chain")
evaluate
addcolumns
(
 values('Date'[Calendar Year])
 ,"x", [x]
 ,"y", [y]
)

Even a parent related table’s column filter is detected – that would result in a single or a multi-value filter.  In the following query, the sub-category “Bike Racks” only has one product, whereas “Touring Frames” has 18.

define
measure
 'Date'[x] = if( iscrossfiltered('Product'[Product Name]),
 "apply special calculation because filtered by product(s)",
 "do not apply special calculation"
 )
measure
 'Date'[y] = calculate([x], 'Product Subcategory'[Product SubCategory Name] = "Bike Racks")
measure
 'Date'[z] = calculate([x], 'Product Subcategory'[Product SubCategory Name] = "Touring Frames")
evaluate
addcolumns
(
 values('Date'[Calendar Year])
 ,"x", [x]
 ,"y", [y]
 ,"z", [z]
)

In both cases, the filter is detected.

Follow

Get every new post delivered to your Inbox.