By Moha from MicroStrategy
10.4 is now Generally Available, and while the README has a good overview of what is new in 10.4, this post delves into more detail about what is new in MicroStrategy’s In-memory Analytics functionality. In short, the main Analytical Engine features in 10.4 are the following:
- Attribute Thresholds
- Improved memory usage for data refresh for partitioned Intelligent Cubes
- Improved Flexible Partitioning for multi-table cubes
- Improved In-memory Cube Troubleshooting: Cube Health Check
Read on for more details, and do not forget to comment on these features. All feedback is welcome.
Attribute Thresholds
Now it is possible to add conditional formatting to attribute elements. The general workflow to define thresholds on attributes is similar to that of defining thresholds on metrics. Check out the Attribute Threshold demo video.
Attribute thresholds’ conditions can be based on other attributes or metrics, and these can be defined for attributes present on both the rows and columns of a grid. Plus, when thresholds’ conditions are based on metrics, the conditions can evaluate the metric values and/or their subtotal values. To illustrate these capabilities, it is possible to create threshold such as the following:
- Thresholds based on metrics:
- Highlight categories for which there was a profit margin greater than 10%
- Highlight regions for which’s profit margin subtotals were less than 15%
- Thresholds based on other attributes:
- Highlight all customers that bought a certain product
- Replace all product descriptions with an image if the product’s name starts with “MicroSt” or its category is equal to “analytics”
- Highlight all the years on the columns in which there is least one profit margin value of 15% or less
A particularity about attributes in grids is that an element could span multiple rows or columns when a grid has merged headers on the rows or columns (depending on which axis the attribute is on). In such cases, attribute thresholds are applied as well.
We highlight the merged cell by applying the first defined threshold which’s condition matches any row or column in the cell span. In the example below, the first threshold has priority over the second one (they can be ordered); if any row for the Books category meets the first condition, then we highlight Books as blue.
If we change the first condition to never apply to any row, then we will see Books is highlighted as orange.
Optimized Data Refresh for Partitioned Intelligent Cubes
In 10.4, we are now enabling the Intelligent Server Normalization data population method for incremental refresh reports when refreshing data in partitioned Intelligent Cubes. With this setting, we improve the memory usage while fetching data from the database by normalizing data as soon as it arrives to the Intelligence Server. We avoid holding raw redundant data coming from the database in memory while building the cube.
Generally, improvements in memory usage can be expected if the partitioned Intelligent Cube being refreshed has multiple attributes, and if the different attributes have, for example, text forms.
To enable this data population method, edit the Incremental Refresh Report’s Data Population for reports VLDB setting from MicroStrategy Developer and choose the Normalize report data in the Intelligence Server option.
Note that for 10.4 Normalize data in the Intelligence Server falls back to Do not normalize report data if Parallel Data Fetch is enabled for the partitioned Intelligent Cube (configurable in the cube’s Intelligent Cube Configuration… settings).
Improved Flexible Partitioning for Data Import Cubes
For data import cubes, or multi-table cubes, we have had the possibility to re-partition data in memory without fetching data from the source since 10.2. In 10.4, we optimized significantly the repartitioning mechanism, reducing the overall time to rearrange existent data into a different set of partitions, parallelizing parts of the repartitioning process.
As a context, partitioning cube data improves scalability and performance for in-memory reporting; with partitioniong, several operations can be done in parallel, such as loading into memory or querying cube data, improving performance significantly. For more information about partitioning with In-memory data import cubes, refer to technical note TN221859: PRIME partitioning guidelines for MicroStrategy 10 Secure Enterprise Platform.
Improved In-memory Cube Troubleshooting: Cube Health Check
We introduced a new command from the Intelligent Server console –available when running the Intelligent Server as an application from the command line– only for troubleshooting purposes: cubehealthcheck. This command automatically verifies that the cube is structurally consistent and suggests corrective actions for the cube whenever possible. The command also collects relevant logs about the cube.
The command can perform diagnostics for one particular or all loaded In-memory cubes (both Intelligent and Data Import cubes). Once the server is run as an application (from Windows, run Program Files (x86) /MicroStrategy/Intelligence Server/MSTRSvr2_64; from Linux, run[MSTR_Install_PATH]/bin/mstrsvr), enter the following command:
cubehealthcheck {<cube_id>|allcube}
<cube_id> above represents the GUID of the cube report. If allcube is entered, then diagnostics are done for all the loaded cubes in the Intelligence Server.
After executing the command, the console will output general information about the cube and suggested corrective actions for any problem found. A sample output is like the following:
Project:3 Cube: CubeName...
[BEGIN] Check basic cube information
[INFO] Cube Name: CubeName...
[INFO] Cube ID: CD8C4EC511E5E61004DB0080EF552B41
[INFO] Version ID: F29A767711E637C048E20080EFC5C708
[INFO] The cube is a multi-table EMMA cube.
[INFO] The cube is not partitioned.
[INFO] The largest fact table has 96 row(s).
[INFO] The cube size is 57344 bytes.
[INFO] The report instance size is 431281 Bytes.
[INFO] Detailed information is in memory_usage.log
[END] Check basic cube information
[BEGIN] Dump cube structure logs
[INFO] cubeinfo.log is dumped.
[INFO] memory_usage.log is dumped.
[INFO] cube header is dumped.
[INFO] Server logs are copied from /home/diakite/MSTR/10.4.0023.0049-LIN/BIN/Linux
[END] Dump cube structure logs
[BEGIN] Check Relation tables
[INFO] 1 relation tables in the cube
[INFO] Detailed information is in cubeinfo.log
[END] Check Relation tables
[BEGIN] Check fact tables
[INFO] There are 1 fact tables
[INFO] Detailed information is in cubeinfo.log
[BEGIN] Check fact table 0
[INFO] There are 96 row(s) in the table
[INFO] There are 2 attribute(s) in the table
[INFO] Attributes: State,INSTNM
[WARNING] There are level metrics in the cube: Metric with Nulls, Row Count - CubeName.xlsx
[WARNING] These level metrics have the same rows as main index, but in different order: Metric with Nulls, Row Count - CubeName.xlsx
[SUGGESTION] If the cube is published before MicroStrategy 10.3, you may run cubehealthcheck levelindex <cubeid> to check whether the cube is corrupted and fix it.
[END] Check fact table 0
[END] Check fact tables
[BEGIN] Check internal pointers
[END] Check internal pointers
====== Summary =====
[WARNING] There are level metrics in the cube: Metric with Nulls, Row Count - CubeName.xlsx
[WARNING] These level metrics have the same rows as main index, but in different order: Metric with Nulls, Row Count - CubeName.xlsx
[SUGGESTION] If the cube is published before MicroStrategy 10.3, you may run cubehealthcheck levelindex <cubeid> to check whether the cube is corrupted and fix it.
[INFO] logs are written to cubehealthchecker.log
Moreover, the Intelligence Server generates a set of logs for each cube. The locations for these logs are generally these for two sample Operating Systems:
- Windows: Program Files (x86)/MicroStrategy/Intelligence Server/CubeHealthReport
- Linux: [INSTALL_PATH]/IntelligenceServer/CubeHealthReport
If it is necessary to contact Technical Support for cube-related topics, please collect all the generated logs for the cubes in question and provide this information.
What’s New in 10.4 – In-Memory Analytics: 10.4 is now Generally Available, and while the README has a good overview of what is new in 10.4, this post delves into more detail about what is new in MicroStrategy’s In-memory Analytics functionality. In short, the main Analytical Engine features in 10.4 are the following: Attribute Thresholds I…