BISM Normalizer: Version 5.0 Released!

Download it from the Visual Studio Marketplace.

Documentation has been updated in the Model Comparison and Merging for AS whitepaper, which discusses use cases.

Enhancements in Version 5.0

Offline metadata file support

Comparisons now work with tabular metadata file such as a BIM file as the source and/or target.

Metadata files are compared offline, without the need for an Analysis Services server instance. All validations are performed on metadata files except dependencies between M expressions because (in the current version) they require an Analysis Services instance.

File Source Target

1500 models

Calculation groups and many-to-many relationships are supported.

1500 diff

Power BI objects

In readiness for XMLA endpoint read/write, objects specific to Power BI are also supported:

  • Incremental refresh policies/partitions
  • Aggregations
  • Table level storage for composite models

Command-line enhancements

The following parameters can be used for command-line execution:

  • Azure AD credentials as command-line parameters to connect to source/target Azure AS servers.
  • Workspace server to be used for automated builds when using projects (SMPRJ files) as the source and/or target of a comparison.

My 2nd Wave

I experienced a breakthrough when I joined Conchango in early 2003. I felt empowered to work with the company’s leadership including Iyas AlQasem, and I quickly rose through the ranks amongst a team of highly talented consultants. I learned about BI and data warehousing from Dan Bulos, Mick Horne, Dan Perrin and Kristian Wedberg, who were seasoned pros I felt lucky to work with. I attended my first SQL PASS conference in 2003 with Mick and Dan P. I clearly remember walking out of our little hotel sensing that one day I would live in the Seattle area and work for Microsoft.

Since then, I spent 10 years in California, created my own company to bill for my services as a consultant, and generated additional revenue selling BISM Normalizer online. I’m grateful for the opportunity to bring my family to the US, ultimately become an US citizen, and transform the environment for my kids to grow up in. I probably became too comfortable in California.

In 2016, I bit the bullet and applied for a job at Microsoft where I joined the BI team. I learn every day from my manager Josh Caplan, the senior leadership team including Amir Netz and Arun Ulag, and architects like Marius Dumutru and Akshai Mirchandani. I believe the best leaders are also the best learners.

I wrote this post 4 months ago, well before my promotion to Principal Program Manager. I meditate every morning, feel gratitude for my amazing family and opportunities, and focus my mind wholeheartedly on my career goals. I believe there is more than enough to go around, especially for those willing to embrace change, and channel their thoughts to align with their aspirations. This is of course relative to one’s current standing and environment, but I believe it applies to all of us.

I have grown immensely at Microsoft. I created one of the few standout demos in the history of Microsoft BI, the “clicky-clicky draggy droppy” trillion-row demo, and established myself as a passionate, authoritative keynote speaker. I drive feature prioritization, planning, design, execution, customer engagement and business results by making Power BI, the most successful BI platform on the planet, even better for the biggest enterprise customers.

I am excited about where Power BI is heading, and look forward to learn every day from a team of highly talented individuals with a genuine desire to create something that is collectively greater than the sum of individuals’ contributions.

BISM Normalizer: Version 4.0 Released!

Download it from the Visual Studio Marketplace.

Prerequisites

Install latest AMO.

Enhancements in Version 4.0

Support for 1400 models

M expressions and structured data sources are now supported and validated.

Expressions

The source model can be 1200 and the target 1400, but not the other way around.


Enhancement to retain partitions

BISM Normalizer has always supported retaining partitions in the target model. This can be achieved by skipping table-update actions. Additionally, if uncheck the “Consider partitions when comparing tables” option, tables with different partitions – but otherwise same definitions – are considered equal and automatically skipped.

In previous versions, if the user needed to change part of a table definition that is not part of its partition definition – for example, specifying a display folder for a column – he/she was forced to update that table, and the source partition definition was pushed to the target as part of the table update.

BISM Normalizer 4.0 introduces a new option “For table updates, retain partitions (not replace)”. When checked, partitions and their data are retained even when making other (non-partition related) updates to a table.

BNorm Options

Table updates with matching partition definitions automatically retain data in the target regardless of these settings.


Support for Visual Studio 2019

Update July 16 2019: BISM Normalizer 4 now works with Visual Studio 2019.

BISM Normalizer: Version 3.4 Released!

Download it from the Visual Studio Marketplace.

Enhancements in Version 3.4

Support for Visual Studio 2017

Like the SSDT for AS extension, BISM Normalizer 3 now works with Visual Studio 2017.

BISM VS 2017

BISM Normalizer 3 continues to work with Visual Studio 2015.


Version bumped TOM references to version 14

This removes the pre-requisite from version 3.3 to install the latest client libraries for Azure AS, which caused compatibility issues for some users.


1400 models not yet supported

1400 models are not supported by this version. If you want BISM Normalizer to work with 1400 models, please vote up this item!

BISM Normalizer: Version 3.3 Released!

Download it from the Visual Studio Gallery.

Enhancements in Version 3.3

Support for Azure Analysis Services!

Azure AD role members work.

aad-members


BISM Normalizer is free and open source!

Removed obfuscation and expiration date. Here is the GitHub page.


Better compatibility with integrated workspace server

In the previous version, if attempt comparison where source/target is a project hosted in integrated workspace mode, and SSDT has just been opened, and the .bim file was not previously left open (so has not been opened yet in the new SSDT session), would fail to connect to workspace server (was necessary to open .bim file and try again). This is now working off the bat; will connect to the integrated workspace server without having to open the .bim file first.

 

BISM Normalizer 3 Overview

BISM Normalizer 3 Use Cases

BNormLogo3_blog

SSAS Compatibility Level 1200

  • Consider BISM Normalizer 3 to deploy tabular models with the awesome new SSAS features
Check

Command-Line Execution

Check

Script Generation

Check

Processing on Deployment (UI Mode)

Check

Retain Partitions & Role Members

  • Retain partitions set up by ETL processes
Check

Visual Studio Integration

Check

Allow Multiple Deployment Strategies

Partial deployment

  • Deploy bug fixes without unfinished features in development
  • Incrementally deploy SSAS objects in isolation, providing flexibility for changing requirements

Traditional all-or-nothing deployment

  • Requires a Production code branch for bug fixes. Use BISM Normalizer to merge bug fixes to the main branch, avoiding manual fix in two different places.
Check

Merging of Tabular Models

  • Pick and choose objects to reuse in other models
  • Merge Power Pivot migrated models into existing corporate models
  • Standardize SSAS objects used by different groups/developers in an enterprise organization
Check

BISM Normalizer: Version 3.2 Released!

Download it from the Visual Studio Gallery.

Enhancements in Version 3.2

Support for SQL Server Analysis Services 2016 General Availability (GA)


Command-line interface supports Visual Studio projects as source/target

Despite Visual Studio not being loaded (or required) in command-line mode, projects can now act as the source/target for comparisons (in addition to databases).

This is useful for automated builds, which get the latest version of files from source control.

The project path is stored in the BSMN file. It can be changed by editing the file.

<ComparisonInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <ConnectionInfoSource>
    <UseProject>true</UseProject>
    <ServerName>localhost</ServerName>
    <DatabaseName>Tabular1200_wade_21527ea8-74fd-4bb6-b2e2-3d1cd3f70063</DatabaseName>
    <ProjectName>Tabular1200</ProjectName>
    <ProjectFile>C:\Users\wade\Documents\Visual Studio 2015\Projects\TabularProject1\TabularProject1.smproj</ProjectFile>
  </ConnectionInfoSource>
  ...

Support for Visual Studio configuration manager

Default values for the target server and database are based on the selected source project’s deployment properties, and the current Visual Studio configuration. Selecting a different source project (or re-selecting the current one) resets the defaults.

VSConfigurations

BISM Normalizer: Version 3.1 Released!

Download it from the Visual Studio Gallery.

Enhancements in Version 3.1

Command-line execution

Run BISM Normalizer from the command line passing the BSMN file as an argument. This allows integration with automated builds.

Syntax
BismNormalizer.exe BsmnFile [/Log:LogFile] [/Script:ScriptFile]
   [/Skip:{MissingInSource | MissingInTarget | DifferentDefinitions}]

 

Arguments

BsmnFile

Full path to the .bsmn file.

/Log:LogFile

All messages are output to LogFile. If the log file already exists, the contents will be replaced.

/Script:ScriptFile

Does not perform actual update to target database; instead, a deployment script is generated and stored to ScriptFile.

/Skip:{MissingInSource | MissingInTarget | DifferentDefinitions}

Skip all objects that are missing in source, missing in target, or with different definitions. This is in addition to the skip actions already defined in the BSMN file.

It is possible to pass a comma-separated list of multiple skip options. For example, “/Skip:MissingInSource,DifferentDefinitions” will skip all objects that are missing in source and those with different definitions.

Automated merging of branches

The /Skip argument can be used for automated merging of branches. For example, by setting /Skip:MissingInSource, it is possible to create/update new/modified objects in a primary branch, without deleting existing ones.

Examples

The following example updates the target database, logging progress and error messages for later review.

BismNormalizer.exe TabularCompare1.bsmn /Log:log.txt

 

The following example does not update the target database. Instead, a script is generated and progress is logged.

BismNormalizer.exe TabularCompare1.bsmn /Log:log.txt /Script:script.xmla

 

The following example updates the target database and progress is logged. None of the objects missing in source are deleted.

BismNormalizer.exe TabularCompare1.bsmn /Log:log.txt /Skip:MissingInSource

 

Passwords and processing

BISM Normalizer in command-line mode does not automatically set passwords for impersonated accounts. They need to be set separately, either manually or securely as part of a build. Processing also needs to be set up separately if required.

Executable location

BISM Normalizer currently only supports standard VSIX deployment from the Visual Studio Gallery. BismNormalizer.exe is located in the extension directory, which looks like “C:\Users\XXX\AppData\Local\Microsoft\VisualStudio\14.0\Extensions\XXX\”, and is shown in the log file produced by the VSIX installer (link available on the installation complete dialog). The executables can be copied to another location using XCOPY deployment.


Translations

To enable this feature, ensure the include cultures check box is checked.

Cultures.png

Comparing cultures works similarly to comparing perspectives. Both have a merge option. This allows existing translations in the target culture to be retained. See this post for more info on merging perspectives and translations.

The JSON order of translations in the source/target text boxes is honored from the BIM file/database definition. In some cases they may have different order, but still considered equal since the JSON order doesn’t affect translation functionality.

Merge Perspectives/Translations in BISM Normalizer

When merging models, it may be useful to create selections from a source perspective without losing existing selections that were already there in the target.

The same applies to translations for a culture.

Merge perspectives

Consider the following comparison taken with the merge perspectives option checked.

Perspective comparison

Note that the differences in the target object definition are highlighted in grey instead of the normal red. This is to indicate that they will not be removed.

After applying the update (assuming FactRate is still in the target database) and re-running the comparison, the following definitions are displayed.

After update

For the purpose of the comparison, the definitions are considered the same because an update would have no effect.

However, if the merge perspectives option is unchecked and the comparison is re-run, the definitions are considered different as shown below.

Different definitions

A subsequent update will remove selections for the FactRate table.

This works basically the same way for cultures. When the merge translations checkbox is checked, existing translations will not be removed. Different translations for an object property that is both in the source and target cultures will be overwritten.