The reading pointers below I borrowed from my Google+ microblogs “Data Visualization” ( , 7090+ followers) and “Data Visualization with Tableau” ( , 1010+ followers).

1. Is the Avoidance of 3-D Bar Graphs a Knee-Jerk Reaction?

2. A modern view of Minard’s Map:

3. Averages Aren’t What They Used to Be and Never Were

4. 6 questions with Burn-Murdoch:

5. Visart’s Demos:

6. Can We Trust Salesforce for Business in the Cloud?

7. Pump Up Your Bump with Ranked Bars

8. Dashboard Improvement Opportunities – Surface Observations

9. New in Tableau 9.3:




10. Persuasion?

11. How Tableau Built a $3 Billion Data Empire On Top Of Beautiful Charts:

12. Tableau plus HyPer: “Something up their sleeve”


13. DV and mapping:

14. DV Digest for March 2016:

15. Farewell:

16. Wolfram about himself:

17.David Raab about SAS:

18. Dimensionality Reduction:

19. More about color:

20. New in Spotfire 7.5:



21. 37 QLIK blogs:

22. Text Tables:

23. 167 years ago:

24. QlikSense 3.0:

25. Pareto:

26. Tableau 10:



and cross-DB filtering:

and cross-DB joins:

27. Qonnection 2016:

28. Tableau tips:


29. Directional Lollipops:

30. Oracle DV Desktop:




31. Tile Maps:

32. Trillion Rows:

33. Power BI is trying hard:

34. Advizor Solutions Overview ( ):

As usual, the reading pointers below I borrowed from my Google+ microblogs “Data Visualization” ( , 7090+ followers) and “Data Visualization with Tableau” ( , almost 1000 followers). Again, sometimes the reading is more important then doing or writing.

Map of Scientific collaboration:


MicroStrategy vs. Tableau:

Brain Capacity:


Qlikview 12 finally released:


Amazon QuickSight:



American Panorama:

Statistica 13:

Wolfram Community:

Recreation of Statistical Atlas:


Social Colors:

Pantone’s Language of Color:

Urban Growth:


How many people ever lived:

Stephen Curry:

Free book:

Errol Morris: How Typography Shapes Our Perception of Truth

Animation and Visualization:

Visualizing Sentiment and Inclination

Compare JS libraries:

TabJolt, part 1:

TabJolt, part 2:

Plus 1000 in 2016:

Correlations in Tableau:

Unions in Tableau:

Mapbox and Tableau:



This is the Part 2 of my post about Tableau’s history. Part1 “Tableau self-intro: 2003-7” was published on this blog earlier. The text below is based on Tableau’s attempt to re-write own history, version by version. What is below is said by Tableau, but interpreted by me. Part 1 “Intro” covers 2003-7 from version 1 to 3, Part 2 (this article) “Catching-up” covers 2008-10 from versions 4 to 6. Recent Q3 of 2015  ($171M revenue) financial results showing that Tableau keeps growing faster than anybody in industry, so interest to its history remaining high among visitors of my blog.

In 2010, Tableau reported revenue of $34M, $62M in 2011 (82% YoY), $128M in 2012 (106% YoY). The company’s 2013 revenue reached $232M, an 81% growth over 2012’s $128M.  2014 revenue exceeded $413M (78% YoY) and in 2015 Tableau expected $650M revenue (57% YoY), more than QLIK:


In Multi-line Chart above (data are from Morningstar, for example: the width of the each line reflects the value of Year-over-Year growth for given company for given year (Tableau is blue, Qliktech is green and Microstrategy is orange; unfortunately Spotfire sales data are not available since 2008, thanks to TIBCO). Here is Tableau’s revenue for last 5 quarters:


Tableau’s success has many factors but in my opinion the 5 main contributors are:

  • In 2007 TIBCO bought Spotfire, making it incapable to lead;
  • Both Spotfire and Qliktech left their R&D in Sweden while scattered other offices in US;
  • Release of free Tableau reader in 2008 – brilliant marketing move;
  • Release of free Tableau Public in 2010 – another brilliant marketing move;
  • Gift from Qliktech in 2011-2015 (more about that in Part 3 or 4 of this blog post).

4.0. 2008. Integrated Maps added: “Data elements such as city, state and country are now automatically recognized as mappable dimensions, and users can also assign geospatial rules to selected dimensions. Once maps are created, users can also change the way data is presented and drill down into the underlying information without a need to understand map layers or complex geographic parameters”.

“Other upgrades in Tableau 4.0 include support for embedding visualizations within Web applications, Web sites and portals such as Microsoft SharePoint. Conversely, Web applications can also be embedded into Tableau”.

In 2008 Tableau released the free Tableau Reader and enables server-less distribution of visualization with full Windows UI experience. “Getting an unlimited free trial into the hands of thousands of people raises awareness among people who are interested in analyzing data, while at the same time training them in its use”. Also see old video here:

5.0. 2009. Tableau enables Views and Dashboards to act Visual Filters, which improves tool’s ability to drill-down data. Such actions can be local and global. Tableau Server now is capable of multi-threading and it can be distributed among multiple hardware boxes or virtual machines, greatly improve scalability and performance.

New Data sources and connectors introduced: Postgres 8.3, Oracle 11g, MySQL 5.1, Vertica v3, Teradata 13, DB2 v9.5 ; Tab, Space, Colon and Pipe delimited flat files, custom geocodes.

5.1. 2010. Added reference lines, bands and distributions, added bullet charts and box-and-whisker charts, expanded set of available pallets, enabled the customization of titles, annotations, tooltips, dashboard sizes,


actions and filters. Tableau 5.1 extended the support for Teradata and Essbase.

Tableau Public. 2010. In its 2nd brilliant marketing move (1st was the release of free Tableau Reader in 2008) the free Tableau Public was released and that instantly made Tableau as the leader in Data Visualization field.

6.0. 2010. The evil Data Blending was introduced in version 6 due an inability of Tableau to join tables from multiple databases and datasources. This architectural bug will be partially fixed in 2016 (Tableau 9.2 or later – it was not clear from TC15 announcement), but real solution can be achieved only when Tableau will implement own internal in-memory DBMS (preferably capable to support columnstore).

Data Engine was introduced as the separate process, which in theory is capable to optimize the creation of Data Extracts and the usage of available RAM as well as take advantage of available disk space so Data Extract can be larger than available RAM. Among new features are improved server management; parameters, which can accept user’s input; suite of table calculations; and drag-and-drop UI for creating Ad-hoc hierarchies.

Below is a screenshot of my drill-down dashboard I did originally in Qlikview and then redid in Tableau 6 to prove that Tableau can do as much drill-down as Qlikview can (using Tableau’s Dashboard Actions):


Image above has an interesting “story”: since it was published on this blog more than 4 years ago it was copy-pasted (in many cases “stolen” without credit to me!) and used as the showcase for Tableau by many blogposts, articles and other publications and “authors”, who disrespect the elementary quoting/crediting rules since internet allows copy/paste operations and leaving up to those people to be polite or disrespectful.

The indirect prove of the brilliancy of Tableau’s marketing moves (free Tableau Reader and free Tableau Public) in 2008-2010 is the volume of the internet searches (thanks to for Tableau and its 6 nearest competitors in 2009-14:


In follow-up I am planning the Part 3: Tableau competes, 2011-13 and Part 4: Tableau the leader, 2013-15.

I was accused by many that I like Tableau too much. That is wrong: in fact I love Tableau but I will try to show below that love can be “objective”. Tremendous success of TC15 (with 10000+ attendees, unmatched by any competitor; 1st conference in 2008 attracted only 187 people) convinced me to return to my blog to write about Tableau’s history – it is interesting how it came to be.


Tableau was spun out of Stanford in 2003, from project Polaris, led by professor Pat Hanrahan and Chris Stolte. It was originated at Stanford as a government-sponsored (DoD) research project to investigate new ways for users to interact (including VizQL) with relational and OLAP databases. In 2004 Tableau got $5M from VCs. In 2005, Hyperion (now Oracle owns Hyperion) began to offer a Tableau under the name “Hyperion Visual Explorer“.

By end of 2010 Tableau had 4 products: Tableau Desktop ($1999 for Pro edition), Tableau Server ($10000 for 10 users), Tableau (free) Reader and Tableau (free web service) Public. In 2010 Tableau had about $34M revenue and was one of the fastest growing software companies in the world (123% YoY). Even in Q3 of 2015 Tableau’s revenue was $171M, 64% up from Q3 of 2014 and it was twice more than entire Tableau’s revenue over period of 2003-10. Overall for last 5 years Tableau had explosive (and unsustainable by industry standards) 75% or above growth; that YoY revenue growth (and Tableau expects $650M for entire 2015) presented in bar chart below:


The text below is based on recent Tableau’s attempt to re-write own history, version by version. Also I reused some posts from this blog – I already covered in my blog versions 6.0 (in 2010) and then 6.1, 7.0, 8.0, 8.0 Desktop, 8.0 server8.1, 8.2, Tableau Online, Tableau Reader, Tableau Public.

I will follow this pattern with one exception (and I promise to avoid the marketing BS like “revolutionary innovation”). I will start with something which is still is not here yet at the end of 2015. Noted by me before: No MDI, no re-sharing of workbook infrastructure with other workbooks, no internal DB (ugly data blending instead), no in-memory columnstore, wrong approach to partners etc.

What is below is said by Tableau version by version, but interpreted by me (my blog, my opinions, my interpretation). Part 1 “Intro” covers 2004-7 from version 1 to 3, Part 2 “Catching-up” covers 2008-10 from versions 4 to 6, Part 3 “Competition” covers 2011-13 from version 6 to 8 and Part 4 “Leading the field” covers 2013-15 from version 8.1 to 9.1, including Tableau Online.

1.0. 2004.

Introduction of VizQL allowed less coding (but required a lot of drag-drops, clicks, resizing and other gymnastics with mouse, which seems more acceptable to wider population – Tableau insists on “anyone, anywhere”). Tableau 1.0 can access to Access, Excel, Microsoft Analysis Services Cubes!), MySQL, SQL Server 2000. Data from multiple tables have to be denormalized (this proved overtime to be the weakest part of the tool) into one table before importing into Tableau.

I am not sure why even in 2015 the Tableau insists on its own self-assessment that it works as fast as you can think – that is offensive to thinkers.


Tableau 1.0 was available in three editions. The $999 Standard Edition can connect to Microsoft Excel, Microsoft Access, or plain text files. The $1299 Professional (MySQL) edition adds MySQL to the list of supported data sources, while the $1799 Professional edition extends the list to include Microsoft SQL Server and SQL Server Analysis Services.

2.0. 2006.

Tableau 2.0 added the ability to join tables in the same database. Added the ability to create Data Extracts and work offline without live connection to data. New features: Distinct Counts and Median aggregations, new “Size” shelf (marks will be sized proportionally to the value of the field in that shelf), new “Page” shelf (useful for animations, see example of it I did a while ago):

Here the content of this video as the presentation with 24 Slides:

Tableau 2.0 also added optional trend and reference lines, calculated fields (can be used with formulas, with all functions and with custom SQL and MDX expressions). 3 Screenshots below preserved for us by Stephen Few in his reviews of Tableau 2.0 and 3.0.


Tableau 2.0 is priced at $995 for the standard edition and $1,799 for the professional edition, including one year of software maintenance and unlimited technical support.

3.0. 2007.

Tableau Server introduced so people can see visualizations through browser over intranet or internet. When visualization application is published from Windows Desktop to Tableau Server (which is in fact, application server), it will be converted to web application: no downloads, plugins or coding required and all related data-sources will be published on that server.

Among other new features: new Sorting “shortcuts”,


as well as Ad-hoc grouping, Auto-calculated reference lines, annotations and most importantly, dashboards with global filters. Tableau missed the opportunity to introduce the MDI into multi-view dashboards and this design bug persisted even now in 2015 – tool still using non-MDI containers (panels) instead of MDI child-windows for each chart. Another problem (in Tableau 3.0) was that views in dashboard updated sequentially and not in-parallel.


By 2007 Tableau employed just 50 people but it was just a beginning:


In 2007 the Tableau Software company got lucky, because TIBCO bought Spotfire that year and it greatly restricted the ability of Spotfire to lead Data Visualization field. Another luck for Tableau was a strategic mistake by both Qliktech and Spotfire to leave development teams in Sweden while placing their HQs, sales, marketing etc. elsewhere in multiple US locations. Tableau got lucky one more time later thanks to gift from Qliktech but I will discuss it later in Part 3 or 4 of this blog-post. As mentioned above, I am planning the Part 2 of this post: Tableau is catching-up, 2008-10, then Part 3: Tableau competes, 2011-13 and finally the Part 4: Tableau the leader, 2013-15

Reading pointers below I borrowed from my Google+ microblogs “Data Visualization” ( , 7000+ followers) and “Data Visualization with Tableau” ( , almost 1000 followers). Sometimes the reading is more important then doing or writing. The reading on the beach (like below) can be even more…


  1. How Scalable Do Analytics Solutions Need to Be?
  2. The Data Visualization Catalogue. and
  3. The Evolution of SQL Server BI,
  4. Abela’s Folly – A Thought Confuser.
  5. TIBCO Spotfire Promotes an Insidious Myth.
  6. User Ideas Turned into Product Features:
  7. Is Data Is, or Is Data Ain’t, a Plural? and
  8. Talk: How to Visualize Data,
  9. Pillars Of Mapping Data To Visualizations,
  10. Radar Chart can be useful(?),
  11. Visualization Publication Data Collection,
  12. Visual Representation of SQL Joins, and


  1. Example of stupidity of the crowd:
  2. Reviving the Statistical Atlas of the United States with New Data,
  3. Exploring the 7 Different Types of Data Stories:
  4. Set Your Own Style with Style Templates:
  1. A Look at Choropleth Maps ,
  2. Mountain Chart for different categories (profiles) of web visits:!/vizhome/MountainChart/MountainChart


  1. To the point: 7 reasons you should use dot graphs,
  2. Rant: A Tableau Faithful’s View On Qlik ,
  3. Too Big Data: Coping with Overplotting,
  4. Too much data to visualize? Data densification in Tableau 9 ,
  5. The Architecture of a Data Visualization, , also see
  6. Filter Views using URL Parameters ,


  1. Building a Visualization of Transit System Data Using GTFS ,
  2. A Look At Box Plots ,
  3. Custom Tableau Server Admin Views ,
  4. Circular and Hive Plot Network Graphing in Tableau ,
  5. Hexbins in Tableau ,
  6. Tableau Public Goes Premium for Everyone; Expands Access to 10 Million Rows of Data ,


2 year ago the IPO instantly created almost $3B of market capitalization for Tableau Software Inc. and since then it almost tripled, making Tableau the most “valuable” Data Visualization company (click on image to enlarge):


Tableau more then doubled the number of its Full-Time employees (almost 2200 now, roughly the same (or more?) as QLIK has) and more then doubled its Revenue (again, roughly the same as QLIK has). Tableau’s YoY growth still in range of 77%-100% per year, which is far, far more then any competition:


Combination of that growth with technological progress and new features of Tableau’s products led to huge growth of its share price – it reached in 1st week of June 2015 $115, while Qlik’s share price is hovering around $37 or even below (click on image to enlarge):


Visitors to this blog kept asking me of what is most impressive (for me) about Tableau and what are my concerns. I will list just 3 of each:

  • most impressive: YoY (Year-over-Year growth ratio); migration to 64-bit (finally) and performance improvements; and  increasing capacity of Tableau Public to 10 million rows and 10 GB storage.
  • concerns: rumors that price of Tableau Server will be increased (I heard doubled; that can slow down the growth and the popularity of Tableau); moving CEO to Europe away from HQ (repeating of mistake of Spotfire and Qliktech, who had/have R&D in Europe – away from american HQ);  and limited capacity of Tableau Online (basically it can be good only for small workgroup).

Not all of its huge success can be contributed to Tableau itself:

QLIK for example did not release Qlikview version 12 for last 4 years (but kept updating the last version, recently with release 11 (!) of Qlikview version 11.2). Another help Tableau got from TIBCO, who kept Spotfire inside strict corporate cage and went private with little change for Spotfire to be a spin-off. As a result, competition for Tableau during last 2 years was weaker then before its IPO and we are witnessing a massive migration to Tableau from competitive products.

Don’t assume that Tableau is slowing down: I visualized (using Tableau Public of course, see it here:!/vizhome/Data2Months/TableausMarketCap ) the Tableau’s Market capitalization during last 52 business days and it keeps growing at least as fast as last 2 years:

Tableau's Market Cap

Update 6/7/15: finally, just check the number of Job Openings @Tableau – 344 (as of today 6/7/15), @QLIK – 116 (3 times less then Tableau!), and only 1 (ONE!) opening for Spotfire… If you still think that Microstrategy can compete with Tableau, then please keep this in mind: as of today Microstrategy’s total number of Job Openings is … 50.



My best wishes in 2015 to visitors of this Data Visualization blog!

2014 was very unusual for Data Visualization Community. Most important event was the huge change in market competition where Tableau was a clear winner, QLIK lost it leadership position and Spotfire is slowly declining as TIBCO went private. Pleasant surprise was Microsoft, who is finally trying to package Power BI separately from Office. In addition other competitors like Microstrategy, Panorama and Datawatch were unable to gain bigger share in Data Visualization market.

2014 again was the year of Tableau: market capitalization exceeded $6B, YoY growth was highest again, sales approaching $0.5B/year, number of employees almost the same as @QLIK, LinkedIn index exceeded 90000, number of Job Openings increased again and as of today it is 337! I personally stopped comparing Data Visualization products for last few months, since Tableau is a clear winner overall and it will be difficult for others to catch-up unless Tableau will start making mistakes like QLIK and Spotfire/TIBCO did during last few years.

2014 was very confusing for many members of QLIK community, me included. Qlik.Next project resulted in new Qlik Sense Product (I don’t see too much success for it) and Qlikview 12 is still not released, while prices for both QLIK products are not public anymore. Market Capitalization of QLIK is below $3B despite still solids sales (Over $0.5B/year) and YoY growth is way below of Tableau’s YoY. Qlikview’s LinkedIn index now around 60000 (way below Tableau’s) and Qlik Sense’s LinkedIn index is only 286…  QLIK has only 124 Job opening as of today, almost 3 times less then Tableau!

Curiously, BI Guru Mr. Donald Farmer, who joined QLIK 4 years ago (a few months before the release of Qlikview 11) and who was the largest propagandist of Qlik.Next/Qlik Sense, was moved from VP of Product Management position to new “VP of Innovation” @QLIK just before the release of Qlik Sense and we hear much less from Donald now. Sadly, during these 4 years Qlikview 12 was never released, and QLIK never released anything similar to free Tableau Reader, free Tableau Public and Tableau Online (I am still hoping for Qlikview in Cloud) and all Qlikview prices were unpublished…

As a member of Spotfire community, I was sad to see the failure of Spotfire (and its parent TIBCO) to survive as public company: on December 5, Vista Equity Partners completed the acquisition of TIBX for $4.3 billion. I estimate Spotfire sales around $200M/year (assuming it is 20% of TIBCO sales). LinkedIn index of Spotfire (is way below Tableau’s and Qlikview’s) is around 12000 and number of Job Openings is too small. I hope Vista Equity Partners will spinoff the Spotfire in IPO as soon as possible and move all Spotfire’s Development, Support, Marketing and Sales into one American location, preferably somewhere in Massachusetts (e.g. back to Somerville).

Here is a farewell Line Chart (bottom of Image) to TIBX symbol, which was stopped trading 3 weeks ago (compared to DATA and QLIK Time Series (upper and middle Line Charts) for entire 2014):


While on Cape Cod this summer and when away from beach, I enjoyed some work-unrelated fun with Tableau. My beach reading included this article: by Andrew Ball and I decided to create my own Radar. When I show it to coworkers later, they suggested to me to publish it (at least the fun with Polygons, Path and Radars) on my blog. I may reuse this Radar chart for our internal Web Analytics. CloudsOverAtlantic

Natural Order of Points and Segments in Line.

Many visualization tools will draw the line chart, its datapoints and connecting line segments between datapoints in natural progressing order – repainting them from left to right (horizontal ordering by Axis X) or from bottom to upside
(vertical ordering by Axis Y) or vice versa.

Path as the method to break the Natural Order.

Some demanding visualizations and users wish to break the natural repainting and drawing order and Tableau allows to do that by using the Path as the method to order the datapoints and line segments in Lines and Polygons. A Collection of increasing Ordering Numbers (Pathpoints) for each Datapoint in Line defined a Path for drawing and connecting datapoints and segments of that Line (or Polygon). Each Pathpoint can be predefined or calculated, depends on mplementation and business logic.
Changing the Natural Order can create “artificial” and unusual situations, when two or more datapoints occupying the same pixels on drawing surface but have very different Pathpoints (example can be a Polygon, when Line ends in the same point it starts) or when two or more Line Segments intersecting in the same Pixel on screen (example can be the Center of the letter X ).


Radar Chart has 2 parts: Radar Grid (background) and Radar Polygons (showing repetitive Data Patterns, if linear timeline can be collapsed into circular “timeline”). Radar Grid has Radials (with common Center) and Concentric Rings.
Polygons optionally can be filled with (transparent) color. For future Discussion let’s use the RMax as the maximal possible distance between the Center of Radar Grid (in case of Radar Grid) or the Center of Radar Polygon (in case of Radar
Polygon) and the most remote Datapoint shown in Radar Grid or Polygon respectively. We will use the “normalized” statistics of Visits to typical Website to visualize the hourly and daily (by day of the week) patterns of Web Visitations. By
normalization we mean the removal of insignificant deviations from “normal” hourly and daily amounts of Web Visits. For complete obfuscation we will assume for Demo purposes that RMax = 144.

Radar Radial Grid.

Radial Grid contains a few Radiuses (equidistant from each other) and we will draw each Radius as 3-point line where Starting and Ending points of each line are identical to each other and collocated with the Center of Radar. For Demo Web Visitation Radar we will use Radial Grid with 8 Radiuses, corresponding to the following hours of the complete 24-hours day: 0, 3, 6, 9, 12, 15, 18, 21:
For example see the Radius, corresponding to HOUR = 3 (below in light brown, other Radiuses greyed out on that image):
And for that Radius we are using (redundantly) the following 3 datapoints:

Concentric Rings for Radar Grid.

For Demo Radar we will use 4 Concentric Rings, corresponding to 25%, 50%, 75% and 100% levels of maximum visitation per hour:
Each ring is a line with 25 datapoints, where Starting and Ending Points collocated/equal. For example, dataset for external Ring (red line above) looks like this:
When Radials and Concentric Rings collocated and overlaid they represent the Radar Grid, ready to be a background for Radar Chart:

Radar Polygons.

For Demo purposes we use only 2 Polygons – one (largest) representing average Hourly Visits during Weekday and 2nd Polygon representing average Hourly Visits during Weekend day. For Websites which I observed the minimum number of visits happened around 1 AM, so you will see both Polygons are slightly rotated clockwise and slightly shifted up from the Center of Radar Grid to reflect the fact that the minimum number of visitors (even around 1 AM) is slightly more then 0. Each Radar Polygon (in our Demo) has 25 Data Points with Starting and Ending Points collocated at 1AM. Here is a Weekday Polygon, overlaid with Radar Grid:
Here are the data for Weekday Polygon: 


Here is a Polygon for Weekend day, overlaid with Radar Grid:


Radar Chart.

When Radar Grid and Radar Polygons overlaid (Polygons transparent but on top of
Grid) we will get the Radar Chart. Please note that Centers of Radar Grid and Radar
Polygons can have different locations:



I published Tableau workbook with this Demo Radar Chart and Radar Data here:!/vizhome/radar/Radar

Visitors to this blog keep asking me to estimate Tableau Software prices (including for Tableau Online), even Tableau published all non-server prices on its website here: However this does not include discounts, especially for enterprise volume of buying no pricing for servers of any kind (at least 2 kinds of server licenses exist) and no pricing for consulting and training.

Thanks to website of Tableau Partner “Triad Technology Partners” we have a good estimate of all Tableau prices (they are always subject of negotiations) in form of so called GSA Schedule (General Services Administration, Federal Acquisition Service, Special Items: No. 132-33 Perpetual Software Licenses, No. 132-34 Maintenance of Software as a Service, No. 132-50 Training Courses) for Tableau Software Products and Services, see it here: here (for example it includes prices for IBM Cognos and others): and specific Tableau Prices here:

I grouped Tableau’s Prices (please keep in mind that TRIAD published GSA schedule in April 2013, so it is 1 year old prices, but they are good enough for estimating purposes)  in 5 groups below: Desktop, Server with licensing for Named Users (makes sense if you have less then hundred “registered” users), Core Licenses for Tableau Server (recommended when you have more then 150 “registered” users), Consulting and Training Prices:

Google sheet for spreadsheet above is here:
and image of it – for those who has misbehaved browsers is below:

Again, please keep in mind that above just an estimate for prices (except for Tableau Online), based on 2013 GSA Schedule, and a good negotiator can always get a good discount (I got it each time I tried). You may also wish to review more general article from Boris Evelson here:

Note about choice between Core License and Server License with Named Users: I know organizations who choose to keep Named Users Licensing instead of switching to Core License even with more then 300 registered users, because it allows them to use much more capable hardware (with much more CPU Cores).

Observing and comparing multiple (similar) multidimensional objects over time and visually discovering multiple interconnected trends is the ultimate Data Visualization task, regardless of specific research area – it can be chemistry, biology, economy, sociology, publicly traded companies or even so called “Data Science”.

For purposes of this article I like the dataset, published by World Bank: 1000+ Measures (they called it World Development Indicators) of 250+ countries for over 50+ years – theoretically more then 10 millions of DataPoints:

Of course some DataPoints are missing so I restricted myself to 20 countries, 20 years and 25 measures (more reasonable Dataset with about 10000 DataPoints), so I got 500 Time Series for 20 Objects (Countries) and tried to imitate of how Analysts and Scientists will use Visualizations to “discover” Trends and other Data Patterns in such situation and extrapolate, if possible, this approach to more massive Datasets in practical projects. My visualization of this Dataset can be found here:;:showVizHome=no

In addition to Trends Line Chart (please choose Indicator in Filter at bottom of the Chart, I added (in my Tableau Visualization above) the Motion Chart for any chosen Indicator(s) and the Motion Map Chart for GDP Indicator. Similar Visualization for this Dataset done by Google here: .

As you can see below with samples of just 6 indicators (out of 1000+ published by World Bank), behavior of monitored objects (countries) are vastly different.

GDP trends: clear Leader is USA, with China is the fastest growing among economic leaders and Japan almost stagnant for last 20 years (please note that I use “GDP Colors of each country” for all other 1000+ indicators and Line Charts):


Life Expectancy: Switzerland and Japan provide longest life to its citizens while India and Russian citizens are expected to live less then 70 years. Australia probably improving life expectancy faster than other 20 countries in this subset.


Health Expenditures Per Capita: Group of 4: Switzerland, Norway (fastest growing?), Luxemburg and USA health expenses about $9000 per person per year while India, Indonesia and China spent less then $500:


Consumer Price Index: Prices in Russia, India and Turkey growing faster then elsewhere, while prices in Japan and Switzerland almost unchanged in last 20 years:


Mobile Phones Per 100 Persons: Russia has 182 mobile phones per 100 people(fastest growing in last 10 years) while India has less then 70 cellular phones per 100 people.


Military Expenses as Percentage of Budget (a lot of missing data when it comes to military expenses!):  USA, India and Russia spending more then others – guess why is that:



You can find many examples of Visual Monitoring of multiple objects overtime. One of samples is where over 7000 objects (publicly traded companies) monitored while observing hundreds of indicators (like share prices, Market Capitalization, EBITDA, Income, Debt, Assets etc.). Example (I did for previous blog post):

Data Visualization Readings, Q1 2014, selected from Google+ extensions of this blog: and


Data Visualization Index (using DATA+QLIK+TIBX+MSTR; click on image above to enlarge):
Since 11/1/13 until 3/15/14: DATA stock grew 50%. QLIK 11%, MSTR – 6%, TIBX – lost 1%.
Current Market Capitalization: Tableau – $5.5B, QLIK – $2.6B, TIBCO – 3.5B, Microstrategy – $1.4B
Number of Job Openings Today: Tableau – 231, QLIK – 135, Spotfire (estimate) – 30, Microstrategy – 214
However during last 2 weeks of March of 2014 DATA shares lost 24%, QLIK lost 14%, TIBX and MSTR both lost about 10%

Why use R? Five reasons.

Studying Tableau Performance Characteristics on AWS EC2

Head-to-head comparison of Datawatch and Tableau

Diving into TIBCO Spotfire Professional 6.0

TIBCO beats Q1 2014 estimates but Spotfire falters

Qlik Doesn’t Fear Tableau, Oracle In Data Analytics

Best of the visualisation web… February 2014

Datawatch: ‘Twenty Feet From Stardom’

Tableau plans to raise $345M — more than its IPO — with new stock offering

TIBCO Spotfire Expands Connectivity to Key Big Data Sources

Tableau and Splunk Announce Strategic Technology Alliance

The End of The Data Scientist!?


Data Science Is Dead

Periodic Table of Elements in TIBCO Spotfire

Best of the visualisation web… January 2014

Workbook Tools for Tableau

Tapestry Data Storytelling Conference ReadingLogo

URL Parameters in Tableau

Magic Quadrant 2014 for Business Intelligence and Analytics Platforms

What’s Next in Big Data: Visualization That Works the Way the Eyes and Mind Work

What animated movies can teach you about data analysis

Tableau for Mac is coming, finally

Authenticating an External Tableau Server using SAML & AD FS

Visualize this: Tableau nearly doubled its revenue in 2013

Qlik Announces Fourth Quarter and Full Year 2013 Financial Results


Tableau Mapping – Earthquakes, 300,000,000 marks using Tableau 8.1 64-bit

Data Science: What’s in a Name?

Gapminder World Offline

Advanced Map Visualisation in Tableau using Alteryx

Motion Map Chart

One of Bill Gates’s favorite graphs redesigned

Authentication and Authorization in Qlikview Server

SlopeGraph for QlikView (D3SlopeGraph QlikView Extension)

Revenue Model Comparison: SaaS v. One-Time-Sales

Scientific Data Has Become So Complex, We Have to Invent New Math to Deal With It

Posting data to the web services from QlikView

It’s your round at the bar

Lexical Distance Among the Languages of Europe


For this weekend I got 2 guest bloggers (one yesterday and other today) sharing their thoughts about Cloud Services for BI and DV. I myself published recently  a few articles about this topic, for example here: and here: . My opinions can be different from Guest Bloggers. You can find many providers of DV and BI Cloud Services, including Spotfire Cloud, Tableau Online, GoodData, Microstrategy Cloud, Bime, Yellofin, BellaDati, SpreadsheetWEB etc.

Let me introduce my 2nd guest blogger for this weekend: Ugur Kadakal is the CEO and founder of Pagos, Inc. located in Cambridge, MA. Pagos is the developer of SpreadsheetWEB which transforms Excel spreadsheets into web based Business Intelligence (BI) applications without any programming. SpreadsheetWEB can also convert PowerPivot files into web based dashboards. It provides advanced Data Visualization (DV) to SQL Analysis Services (Tabular) cubes without SharePoint. Mr. Kadakal published a few articles on this blog before with great feedback, so he is a serial Guest Blogger.


Before (or after) you read article of Mr. Kadakal, I suggest to review the article, comparing 5+ scenarios of revenue of Cloud Service vs. Traditional One-Time Sale of software, see it here: . Illustration above is from that article.

Traditional BI versus Cloud BI

Over the past several years, we have been witnessing numerous transformations in the software industry, from a traditional on-premise deployment model to the Cloud. There are some application types for which cloud makes a lot of sense while it doesn’t for some others. BI is somewhere in between.

Before I express my opinion on the subject of Traditional BI versus Cloud BI, I would like to clarify my definitions. I define traditional BI as large enterprise implementations which connect with many data sources in real-time.  These projects have many phases and require large teams to implement. These projects could take years and cost millions of dollars to implement.

Many people define cloud BI as deployments on a proprietary, third-party, multi-tenant environment managed by a vendor. My definition is somewhat different and broader. Cloud BI is more about ease of deployment, use and management. While Cloud BI can be hosted and managed by a vendor, it can also be deployed on a private Cloud infrastructure like Amazon or Microsoft Azure. With the advancement of cloud infrastructure technologies like OpenStack, deploying and managing private cloud infrastructure is becoming easier for many enterprises. As a result, whether Cloud BI is deployed on a multi/single-tenant environment on vendor infrastructure, a third party cloud infrastructure like Amazon, Azure, etc. or on internal private cloud, it becomes more of a business decision rather than a technical limitation.


One main distinction between Traditional BI and Cloud BI is data management. Traditional BI implementations can have real-time data as they can connect to the original data sources directly. I don’t believe that Cloud BI should deal with real-time data, even if implemented on internal private cloud infrastructure. Supporting real-time data is a requirement that makes any BI project complicated and costly. Hence Cloud BI solutions should include simple utilities i.e. ETL, residing on local computers to push internal data into Cloud BI’s data model periodically. Since Cloud BI should not deal with real-time data scenarios, this data synchronization can be configured by the business user accordingly.

Another distinction is the ease of implementation. Regardless of where it is deployed, Cloud BI solutions should take no more than a few hours to implement and configure. Some BI vendors already support images on Amazon cloud to simplify this process.

Traditional BI model typically requires significant upfront investments. Part of this investment is internal while the rest is BI licensing and implementation fees. But the very nature of Cloud BI requires agility from deployment to data management and dashboard creation. Cloud BI project can be deployed easily and it can also be modified and shut down with equal ease. Hence traditional business model of large upfront investments doesn’t make sense here. Cloud BI business model should be subscription based regardless of whether it is implemented on a vendor infrastructure or on an on-premise private cloud infrastructure. Customers should be able to pay what they use and for how long they use it. Such simplicity will also eliminate vendor lock-in risks that most enterprises have to mitigate.


In summary, there are many BI projects that will require traditional BI implementation. These projects typically require real-time data and connectivity to many different data sources. Cloud BI should not attempt to handle these types of projects. But there are many other BI projects that require neither real-time data nor the data which comes from different systems that should be connected. Cloud BI can handle these projects quickly and cost effectively, by empowering business users to manage the whole process without IT or external support. From discovery to data synchronization to dashboard creation and management, every activity can be handled by business users.

For this weekend I got 2 guest bloggers (one today and second tomorrow) sharing their thoughts about Cloud Services for BI and DV. I myself published recently  a few articles about this topic, for example here: and here: . My opinions can be different from Guest Bloggers (see my comment below this article). You can find many providers of DV and BI Cloud Services, including Spotfire Cloud, Tableau Online, GoodData, Microstrategy Cloud, Bime, Yellofin, BellaDati, SpreadsheetWEB etc.

Let me introduce my 1st guest blogger for this weekend: Mark Flaherty is Chief Marketing Officer at InetSoft Technology,  a BI (Business Intelligence) software provider founded in 1996, headquartered in Piscataway, New Jersey with over 150 employees worldwide. InetSoft’s flagship BI application Style Intelligence enables self-service  BI spanning dashboarding, reporting and visual analysis for enterprises and technology providers. The server-based application includes a data mashup engine for combining data from almost any data source and browser-based design tools that power users and developers can use to quickly create interactive DV (Data Visualizations).


Are public BI cloud services really going to overtake the traditional on-premise deployment of BI tools?

(Author: Mark Flaherty. Text below contains Mark’s opinions and they can be different from opinions expressed on this blog).

It’s been six years since public BI cloud services came to be. Originally termed SaaS BI, public BI cloud services refers to commercial service providers who host a BI application in the public cloud that accesses corporate data housed in the corporate private cloud and/or other application providers’ networks. As recently as last month, an industry report from TechNavio said, “the traditional on-premise deployment of BI tools is slowly being taken over by single and multi-tenant hosted SaaS.” I have a feeling this is another one of those projections that copies a historical growth rate forward for the next five years. If you do that with any new offering that starts from zero, you will always project it to dominate a marketplace, right?

I thought it would be interesting to discuss why I think this won’t happen.


In general, there is one legitimate driving force for why companies look to cloud solutions that helps drive the demand for cloud BI services specifically: outsourcing of IT. The types of companies for whom this makes the most sense are small businesses. They have little or no IT staff to set up and support enterprise software, and they also have limited cap-ex budgets so software rentals fit their cash flow structure better. While this is where most of the success for cloud BI has happened, this is only a market segment opportunity. By no means do small companies dominate the IT marketplace.

Another factor for turning to public cloud solutions is expediency. Even at large companies where there is budget for software purchases, the Business sometimes becomes frustrated with the responsiveness of internal IT, and they look outside for a faster solution. This makes sense for domain-specific cases where there is a somewhat narrow scope of need, and the application and the data are self-contained. is the poster child for this case, where it can quickly be set up as a CRM for a sales team. Indeed the fast success of is a big reason why people think cloud solutions will take off in every domain.

But business intelligence is different. A BI tool is meant to span multiple information areas, from finance to sales to support and more. This is where it gets complicated for mid-sized and global enterprises. The expediency factor is nullified because the data that business users want to access with their cloud BI tool is controlled by IT, so they need to be involved. Depending on the organization’s policies and politics, this can either slow down such a move or kill it.

The very valid reason why enterprise IT would kill the idea for a public cloud BI solution is why ultimately I think public BI cloud services has such a limited opportunity in the overall market. One of IT’s responsibilities is ensuring data security, and they will rightly point out the security risks of opening access to sensitive corporate data to a 3rd party. It’s one thing to trust a vendor with one set of data like website visitor traffic, but trusting them with all of a company’s financial and customer data is where almost all companies will draw the line.  This is a concern I don’t see ever going away.

What are some pieces of evidence that public BI cloud services have a limited market opportunity? When BI cloud services first came onto the scene, all of the big BI vendors dabbled in it. Now many no longer champion these hosted offerings, or they have shuttered or demoted them. IBM’s Cognos Express is now only an on-premise option. SAP BusinessObjects BI OnDemand can’t be found from SAP’s main site, but has its own micro site. Tibco’s Spotfire Cloud and Tableau Software’s Tableau Online are two exceptions among the better known BI providers that are still prominently marketed. However, Tibco positions this option for small businesses and workgroups and omits certain functionality.

Our company, too, experimented with a public BI cloud offering years ago. It was first targeted at customers who would want to mash up their CRM data with other enterprise-housed data. We found mostly small, budget challenged companies in their customer base, and the few large enterprises that we found balked at the idea, asking instead, for our software to be installed on-premise where they would connect to any cloud-hosted data on their own. Today the only remaining cloud offering of ours is a free visualization service called Visualize Free which is similar to Tableau Public or IBM’s Many Eyes.

Another observation to make, while there have been a handful of pure-play cloud BI vendors, one named “Lucidera,” came and went quite quickly. Birst is one that seems to have got a successful formula.

In summary, yes, there is a place for public BI cloud services in the small business market, but no, it’s not going to overtake traditional on-premise BI.


For last 6 years every and each February my inbox was bombarded by messages from colleagues, friends and visitors to this blog, containing references, quotes and PDFs to Gartner’s Magic Quadrant (MQ) for Business Intelligence (BI) and Analytics Platforms, latest can be found here: .

Last year I was able to ignore these noises (funny enough I was busy by migrating thousands of users from Business Objects and Microstrategy to Tableau-based Visual Reports for very large company), but in February 2014 I got so many questions about it, that I am basically forced to share my opinion about it.

  • 1st of all, as I said on this blog many times that BI is dead and it replaced by Data Visualization and Visual Analytics. That was finally acknowledged by Gartner itself, by placing Tableau, QLIK and Spotfire in “Leaders Quarter” of MQ for 2nd year in a row.

  • 2ndly last 6 MQs (2009-2014) are suspicious for me because in all of them Gartner (with complete disregard of reality) placed all 6 “Misleading” vendors (IBM, SAP, Oracle, SAS, Microstrategy and Microsoft) of wasteful BI platforms in Leaders Quarter! Those 6 vendors convinced customers to buy (over period of last 6 years) their BI software for over $60B plus much more than that was spent on maintenance, support, development, consulting, upgrades and other IT expenses.

There is nothing magic about these MQs: they are results of Gartner’s 2-dimensional understanding of BI, Analytics and Data Visualization (DV) Platforms, features and usage. 1st Measure (X axis) according to Gartner is the “Completeness of Vision” and 2nd Measure (Y axis) is the “Ability to Execute”, which allows to distribute DV and BI Vendors among 4 “Quarters”: RightTop – “Leaders”, LeftTop -“Challengers”, RightBottom – “Visionaires” and LeftBottom – “Niche Players” (or you can say LeftOvers).


I decided to compare my opinions (expressed on this blog many times) vs. Gartner’s (they wrote 78 pages about it!) by taking TOP 3 Leaders from Gartner, than taking 3 TOP Visionaries from Gartner (Projecting on Axis X all Vendors except TOP 3 Leaders) than taking 3 TOP Challengers from Gartner (Projecting on Axis Y all Vendors except TOP 3 Leaders and TOP 3 Visionaries ) than TOP 3 “Niche Players” from the Rest of Gartner’s List (above) and taking “similar” choices by myself (my list is wider then Gartner’s, because Gartner missed important to me DV Vendors like Visokio and vendors like Datawatch and Advizor Solutions are not included into MQ in order to please Gartner’s favorites), see the comparison of opinions below:

12DVendorsIf you noticed, in order to be able to compare my opinion, I had to use Gartner’s terms like Leader, Challenger etc., which is not exactly how I see it. Basically my opinion overlapping with Gartner’s only in 25% of cases in 2014, which is slightly higher then in previous years – I guess success of Tableau and QLIK is a reason for that.

BI Market in 2013 reached $14B and at least $1B of it spent on Data Visualization tools. Here is the short Summary of the state of each Vendor, mentioned above in “DV Blog” column:

  1. Tableau: $232M in Sales, $6B MarketCap, YoY 82% (fastest in DV market), Leader in DV Mindshare, declared goal is “Data to the People” and the ease of use.

  2. QLIK: $470M in Sales, $2.5B MarketCap, Leader in DV Marketshare, attempts to improve BI, but will remove Qlikview Desktop from Qlik.Next.

  3. Spotfire: sales under $200M, has the most mature Platform for Visual Analytics, the best DV Cloud Services. Spotfire is limited by corporate Parent (TIBCO).

  4. Visokio: private DV Vendor with limited marketing and sales but has one of the richest and mature DV functionality.

  5. SAS: has the most advanced Analytics functionality (not easy to learn and use), targets Data Scientists and Power Users who can afford it instead of free R.

  6. Revolution Analytics: as the provider of commercial version and commercial support of R library is a “cheap” alternative to SAS.

  7. Microsoft: has the most advanced BI and DV technological stack for software developers but has no real DV Product and has no plan to have it in the future.

  8. Datawatch: $33M in sales, $281M MarketCap, has mature DV, BI and real-time visualization functionality, experienced management and sales force.

  9. Microstrategy: $576M in sales, 1.4B MarketCap; BI veteran with complete BI functionality; recently realized that BI Market is not growing and made the desperate attempt to get into DV market.

  10. Panorama: BI Veteran with excellent easy to use front-end to Microsoft BI stack, has good DV functionality, social and collaborative BI features.

  11. Advizor Solutions: private DV Veteran with almost complete set of DV features and ability to do Predictive Analytics interactively, visually and without coding.

  12. RapidMiner: Commercial Provider of open-source-based and easy to use Advanced Analytical Platform, integrated with R.

Similar MQ for “Advanced Analytics Platforms” can be found here: – have fun:


In addition to differences mentioned in table above, I need to say that I do not see that Big Data is defined well enough to be mentioned 30 times in review of “BI and Analytical Platforms” and I do not see that Vendors mentioned by Gartner are ready for that, but may be it is a topic for different blogpost…


Analytics extrapolates Visible Data to the future (“predicts”) and enables us to see more then 6-dimensional subsets of data with mathematical modeling. The ability to do it visually, interactively and without programming … vastly expands the number of potential users for Visual Analytics. I am honored to present the one of the most advanced experts in this area – Mr. Gogswell: he decided to share his thoughts and be the guest blogger here. So the guest-blog-post below is written by Mr. Douglas Cogswell, the Founder, President and CEO of ADVIZOR Solutions Inc.

Formed in 2003, ADVIZOR combines data visualization and in-memory-data-management expertise with usability knowledge and predictive analytics to produce an easy to use, point and click product suite for business analysis. ADVIZOR’s Visual Discovery™ software spun out of a distinguished research heritage at Bell Labs that spans nearly two decades and produced over 20 patents.

Mr. Cogswell is the well known thought leader and he is discussing below the next step in Data Visualization Technology, when limitation of human eye prevents users to comprehend the multidimensional (say more than 6 dimensions) Data Patterns or estimate/predict the future trends with Data from the Past. Such Multidimensional “Comprehension” and Estimations of the Future Trends requires a Mathematical Modeling in form of Predictive Analytics as the natural extension of Data Visualization. This is in turn, requires the Integration of Predictive Analytics and Interactive Data Visualization. Such Integration will be accepted much easier by business and analysts , if it will require no coding.

Mr. Cogswell discussing the need and possibility of that in his article (Copyright ADVIZOR Solutions, 2014) below.


Integrating Predictive Analytics and Interactive Data Visualization WITHOUT any Coding!

It’s a new year, and many organizations are mulling over how and where they will make new investments. One area  getting a lot of attention these days is predictive analytics tools. The need  to better understand the present and predict what might happen in the future for competitive advantage is enticing many to look at what these tools can do. TechRadar spoke with James Fisher, who said 85 percent of the organizations that have adopted these tools believe they have positively impacted their business.

Fast Fact Based Decision Making is Critical.

“Businesses are collecting information on their customers’ mobile habits, buying habits, web-browsing habits… The list really does go on,” he said. “However, it is what businesses do with that data that counts. Analytics technology allows organizations to analyze their customer data and turn it into actionable insights, in a way that benefits business.”

Interest in predictive analytics by businesses is expected to continue to grow well beyond this year, with Gartner reporting in early 2013 that approximately 70 percent of the best performing enterprises will either manage or have a view of their processes with predictive analytics tools by 2016. By doing this, businesses will gain a better sense of what is happening within their own networks and corporate walls, which actions could have the best impact and give increased visibility across their industries. This will give situational awareness across the business, making operating much easier than it has been in past years.

Simplicity and Ease of Use are Key.

Analytics is something every business should be figuring out.  There are more software options than ever, so executives will need to figure out which solution will work best for them and their teams. According to InformationWeek’s Doug Henschen, the “2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey” found that business users and salespeople need easy-to-use, visual data analytics that is intuitive and easily accessible from anywhere, any time. . These data visualization business intelligence tools can give a competitive edge to the companies adopting them.

“The demand for these more visual analytics tools leads to one of the biggest complaints about analytics,” he said. Ease-of-use challenges have crippled the utilization rate of this software.  But that is changing.  “Analytics and BI vendors know that IT groups are overwhelmed with requests for new data sources and new dimensions of data that require changes to reports and dashboards or, worse, changes to applications and data warehouses,” he wrote. “It’s no wonder that ‘self-service’ capabilities seem to be showing up in every BI software upgrade.”

A recent TDWI research report titled “Data Visualization and Discovery for Better Business Decisions” found that companies do have their future plans focused on these analytics and how they can use them. In fact, 60 percent said their organizations are currently using business visualization for snapshot reports, scorecards, or display. About one-third are using it for discovery and analysis and 26 percent for operational alerting. However, companies are looking to expand how they use the technology, as 45 percent are looking to adopt it for discovery and analysis, and 39 percent for alerts.

“Visualization is exciting, but organizations have to avoid the impulse to clutter users’ screens with nothing more than confusing ‘eye candy’,” Stodder wrote. “One important way to do this is to evaluate closely who needs what kind of visualizations. Not all users may need interactive, self-directed visual discovery and analysis; not all need real-time operational alerting.”

Data Visualization & Predictive Analytics Naturally Complement Each Other.

Effective data visualizations are designed to complement human perception and our innate ability to see and respond to patterns.  We are wired as humans to perceive meaningful patterns, structure, and outliers in what we see.  This is critical to making smarter decisions and improving productivity, and essential to the broader trend towards self-directed analysis and BI reporting, and tapping into new sources of data.

Visualization also encourages “storytelling” and new forms of collaboration.  It makes it really easy to not only “see” stories in data, but also to highlight what is actionable to colleagues. 

On the other hand, the human mind is limited in its ability to “see” very many correlations at once.  While visualization is great for seeing patterns across 2, or 4 or maybe 6 criteria at a time, it breaks down when there are many more variables than that.  Very few people are able to untangle correlations and patterns across, say, 15 or 25 or 75 or in some cases 300+ criteria that exist in many corporate datasets.

Predictive Analytics, on the other hand, is not capacity constrained!!  It uses mathematical tools and statistical algorithms to examine and determine patterns in one set of data . . . in order to predict behavior in another set of data.  It integrates well with in-memory-data and data visualization, and leads to faster and better decision making.

Making it Simple & Delivering Results.

The challenge is that most of the predictive analytics software tools on the market require the end-user to be able to program in SQL in order to prep data, and have some amount of statistics background to build models in R … or SPSS … or SAS.  At ADVIZOR Solutions our vision has been to empower business analysts and users to build predictive models without any code or statistics background.


The results have been extremely promising — inquisitive and curious-minded end-users with a sense for causality in their data can easily do this — and are turning around models in just a few hours.  The result is they are using data in new and powerful ways to make better business decisions.

Three Key Enablers to a Simple End-User Process.

The three keys to making this happen are:  (1) having all the relevant data offloaded from the database or datamart into RAM, (2) allowing the business user to explore it visually, and (3) providing a really simple modeling interface.

Putting the data in RAM is key to making it easy to condition so that the business user can create modeling factors (such as time lags, factors from data in multiple tables, etc.) without having to go back and condition data in the underlying databases — which is usually a time consuming process that involves coordinating with IT and/or DBAs. 

Allowing the business user to explore it visually is key to hypothesis generation and vetting about what really matters, before building and running models.

Providing really simple interfaces that automate the actual statistics part of the process lets the business user focus on their data, not the statistics of the model.  That simple modeling process includes:

  • Select the Target & Base Populations
    • The “target” is the group you want to study (e.g., people who responded to your campaign)
    • The “base” is the group you want to compare the target to (e.g., everybody who received the campaign)
  • Visually Explore the data and develop Hypotheses
    • This helps set up which explanatory fields to include …
    • … and which additional ones may need to be added
  • Select list of Explanatory Fields
    • The “explanatory fields” are the factors in your data that might explain what makes the target different from other entities in your data
  • Build Model
  • Iterate
  • Understand and Communicate what the model is telling you
  • Predict / Score Base Population
  • Get lists of Scored potential targets

Check out how you can do this with no code in this 8 min YouTube video.

Best Done In-house with Your Team.

In our experience this type of work is best done in-house with your team.  That’s because it’s not a “black box”, it’s a process.  And since your team knows the data and its context better than anybody else, they are the ones best suited to discuss, interpret, and apply the results.  In our experience, over and over again it has been proven that knowing the data and context is the key factor  … and that you don’t need a statistics degree to do this.


Quick Example: Consumer Packaged Goods Sales.

In recent client work a well known consumer packaged goods company was trying to untangle what was driving sales.  They had several key questions they were attempting to answer:

  • What factors drive sales?
  • How do peaks in incremental sales relate to the Social Media spikes?
    • For all brands
    • By each brand
  • How does it vary by media provider?  By type of post?
  • Can we use this data to forecast incremental sales? Which factors have the biggest impact?

They had lots of data, which included sales by brand by week, and a variety of potential influences which included:  a variety of their own promotions, call center stats, social media posts, and mined sentiment from those social media posts (e.g., was the post “positive”, “neutral”, or “negative”).   The key step in creating the right explanatory fields was developing time lags for each of these potential influences since the impact on sales was not necessarily immediate — for example, positive Twitter posts this week may have some impact on sales, but more likely the impact will be on sales +1 week, or maybe +2 weeks, or +4 weeks, etc. 

Powerful Results.

What we learned was that there were multiple influences and their intensity varied by brand. Seasonality was no longer the major driver.  New influences — including social media posts and online promotions — were now in the top spot.  We also learned that the key influences can and should be managed.  This was critical — there are lags between the impact of, for example, a negative Twitter post and when it hits sales. As a result, a quick positive response to a negative post can heavily offset that negative post.

In Summary.

An easy to use data discovery and analysis tool that integrates predictive analytics with interactive data visualization and which is then placed in the hands of business analysts and end-users can make huge differences in how data is analyzed, how fast that can happen, and how it is then communicate to and accepted by the decision makers in an organization.

And, stay tuned.  We’ll next be talking about the people side of predictive analytics — if there is now technology that lets you create and use models without writing any code, then what are the people skills and processes required to do this well?

This is a repost from Data Visualization Consulting Page.

Visitors of this blog generated a lot of requests for my Data Visualization “Advice” (small projects for a few hours or days, no NDA [Non-Disclosure Agreement] involved), for Data Visualization Consulting projects (a few weeks or months; I tend to avoid the NDAs as they can interfere with my blogging activities) and even for Full-time work (for example my latest full-time job I got because my employer often visited and read my blog; NDA needed).

Additionally, sometimes I am doing free-of-charge work, if involved projects are short, extremely interesting for me and beneficial for my Data Visualization Blog, like this project:

Obviously all these projects can be done only when I have spare time either from full-time work and/or other projects, duties and activities.

I also cannot relocate or travel, so I can do it mostly from my home office – telecommuting (RDP, Skype, phone, WebEx, GoToMeeting etc.) or if client is local to Massachusetts, then sometime I can visit Client’s site, see below the Map of my Local “Service Area” – part of Middlesex County between Routes 495, 3 and 20 – where I can commute to Client’s Location (please click on map below to enlarge the image) :


If I do have time for short-term advisory projects (from 2 hours to 2 weeks), clients usually pay by the highest rate, similar to what Qliktech, Spotfire, Tableau or IBM charging for their Consulting Services (I consider my consulting as better service than theirs…). If you will go to this thread on Tableau Community: then you will find these Indicative Rates for Consulting Tableau Work (Qlikview and Spotfire Rates are very similar):

Low $125,  Max $300,  Average around $175 per hour.

Here are the most popular requests for my Advisory work:

  • Visual Design and Architectural Advice for Monitoring or Operational Dashboard(s);
  • Review of Data Visualization Work done by my Clients;
  • Prototyping of Data Visualizations (most requested by my visitors);
  • My opinion on Strengths and Weaknesses of Data Visualization Vendor/Product, requested by trader, portfolio or hedge fund manager(s)
  • Advice about what Hardware to buy (say to get the most from Tableau License client has);
  • Advice what Charts and Filters to use for given Dataset and Business Logic;
  • Technical Due Diligence on Data Visualization Startup for Venture Capitalists investing into that Start-up.
  • Etc…


For mid-size projects (from 2 weeks to 6 months) clients getting a “Progressive” discount – the longer the project then the larger the discount. Here are the most popular requests for my Consulting Data Visualization Work:

  • Comparing Data Visualization Product vs. Other Visualization Product for specific Client’s needs and projects;
  • Comparing Clients’s Visualization Product vs. Competitor(s) Visualization Product (most requested);
  • Benchmarking one or more Visualization Product(s) vs. specific data and application logic.
  • Managed Clients migration of their Reporting and Analytical IT Infrastructure from obsolete BI Platforms like Business Objects, Cognos and Microstrategy to modern Data Visualization Environments like Tableau, Qlikview and Spotfire.
  • Etc.


Full-time work (1 year or more engagements) is not exactly a Consulting but Full-time job when clients asking me to join their company. These jobs are similar to what I had in the past: Director of Visual Analytics, Data Visualization Director, VP of Data Visualization, Principal Data Visualization Consultant, Tableau Architect etc. Here are samples of full-time projects:

  • Created, Maintained and Managed the Data Visualization Consulting Practices for my company/employer;
  • Led the growth of Data Visualization Community (the latest example – 4000 strong Tableau Community) with own Blog, Portal and User Group behind the corporate firewall, created Dozens of near-real-time Monitoring Dashboards for Analytical and Data Visualization Communities;
  • Designed and Implemented myself hundreds of Practical Data Visualizations and Visual Reports, which led to discovery of trends, outliers, clusters and other Data Patterns, Insights and Actions;
  • Created hundreds of Demos, Prototypes and Presentations for Business Users;
  • Designed Data Visualization Architecture and Best Practices for Dozen of Analytical Projects;
  • Significantly improved the Mindshare and increased the Web Traffic to website of my company, Created and Maintained the Data Visualization blog for it.

You can find more observations about relationship between Full-Time salary and Hourly Rate for consulting in my previous post (from 6 months ago) here:

Data Visualization readings – last 4 months of 2013.

(time to read is shrinking…)

0. The Once and Future Prototyping Tool of Choice

1. Block by Block, Brooklyn’s Past and Present

2. Data Visualization and the Blind


4. Old Charts

5. Back To Basics

6. In-Memory Data Grid Key to TIBCO’s Strategy

7. Submarine Cable Map

8. Interview with Nate Silver:

9. Qlikview.Next will be available in 2014

10. Importance of color?

11. Qlikview.Next has a gift for Tableau and Datawatch

12. (October 2013) Tableau posts 90% revenue gain and tops 1,000 staffers, files for $450 million secondary offering

13. The Science Of A Great Subway Map

14. SEO Data Visualization with Tableau

15. John Tukey “Badmandments”

Supplementary BADMANDMENTS:

  • 91. NEVER plan any analysis before seeing the DATA.
  • 92. DON’T consult with a statistician until after collecting your data.
  • 94. LARGE enough samples always tell the truth

16. Thinking about proper uses of data visualization.

17. Big BI is Stuck: Illustrated by SAP BusinessObjects Explorer

18. IBM (trying to catch up?) bets on big data visualization

19. Site features draft designs and full views of the Treemap Art project (By Ben Shneiderman)

20. A Guide to the Quality of Different Visualization Venues

21. Short History of (Nothing) Data Science

22. Storytelling: Hans Rosling at Global Health – beyond 2015

23. DataWatch Quarterly Review: Rapid Growth Finally Materializing

24. QlikView Extension – D3 Animated Scatter Chart


25. SlopeGraph for QlikView (D3SlopeGraph QlikView Extension)

26. Recipe for a Pareto Analysis

27. Color has meaning

28. TIBCO’s Return To License Growth Frustratingly Inconsistent

29. Automated Semantics and BI

30. What is wrong with definition of Data Science?

31. Scientific data became so complex, we have to Invent new Math to deal with it

32. Samples

Selected Tableau Readings after TCC13 (since September 18, 2013)

sometimes reading is better then doing or writing…

0. Top 10 sessions from TCC13:

1. Dual Color Axis:

2. Evaluate models with fresh data using Tableau heat maps:

3. Tableau Throws a Brick at Traditional BI:

4. Easy Empty Local Extracts:

5. Tableau 8.1: Sophisticated Analytics for Sophisticated People:

6. Tableau 8.1 and R (can be interesting for at least 5% of Tableau users):
also see:
and here:

7. Tableau, When Are You Going to Fix This?

8. Automated PDF Email Distribution of Tableau Views Using PowerShell and Tabcmd:

9. Geocoding Addresses Directly in Tableau 8.1 Using Integration with R:

10. Best Practices for Designing Efficient Workbooks (and white Paper about it):

11. Tableau Mapping Architecture:

12. Story Points in Tableau 8.2 presentation mode:

13. Truly Global: Filtering Across Multiple Tableau Workbooks with the JavaScript API:

14. Tableau 8.1 Worksheet, Dashboard menus improved, still room for more:

15. Lollipops Charts in Tableau:

16. Was Stephen Few Right?

17. Precision Inputs Required In Addition To Analog Controls:

18. Google Spreadsheets to Tableau connector – a working driver:

19. Leveraging Color to Improve Your Data Visualization:

20. Workbook acts as a container for multiple Tableau-based Charts – 114
Samples and Visualization Types:

21. The New Box-and-Whisker Plot:

22. The Tableau Workbook Library:

23. Customizing Tableau Server Experience (Parts 1, 1.5, 2):

24. SAML Integration in Tableau 8.1:

25. Tableau file types and extensions:

26. Tableau Server XML Information Files: The Master Class:

27. Is it Transparency? Is it Opacity? Labeled one, works like the other:

28. Viz Hall of Fame:

29. Tableau Weekly Archive:

30. 2013 Winners:
Happy New Year!

My Best Wishes for 2014 to all visitors of this Blog!


2013 was very successful year for Data Visualization (DV) community, Data Visualization vendors and for this Data Visualization Blog (number of visitors per grew from average 16000 to 25000+ per month).

From certain point of view 2013 was the year of Tableau – it went public, Tableau has now the largest Market Capitalization among DV Vendors (more than $4B as of Today) and its strategy (Data to the People!) became the most popular among DV users and it had (again) largest YoY revenue growth (almost 75% !) among DV Vendors. Tableau already employed more than 1100 people and still has 169+ job openings as of today. I wish Tableau to stay the Leader of our community and to keep their YoY above 50% – this will not be easy.

Qliktech is the largest DV Vendor and it will exceed in 2014 the half-billion dollars benchmark in revenue (probably closer to $600M by end of 2014) and will employ almost 2000 employees. Qlikview is one of the best DV product on market. I wish in 2014 Qlikview will create Cloud Services, similar to Tableau Online and Tableau Public and I wish Qlikview.Next will keep Qlikview Desktop Professional (in addition to HTML5 client).

I wish TIBCO will stop trying to improve BI or make it better – you cannot reanimate a dead horse; instead I wish Spotfire will embrace the approach “Data to the People” and act accordingly. For Spotfire my biggest wish is that TIBCO will spin it off the same way EMC did with VMWare. And yes, I wish Spofire Cloud Personal will be free and enabled to read at least local flat files and local DBs like Access.

2014 (or may be 2015?) can witness new, 4th DV player coming to competition: Datawatch bought recently Panopticon and if it will complete integration of all products correctly and add features which other DV vendors above already have (like Cloud Services), it can be very competitive player. I wish them luck!


Microsoft released in 2013 a lot of advanced and useful DV-related functionality and I wish (I recycling this wish for many years now) that Microsoft finally will package the most its Data Visualization Functionality in one DV product and add it to Office 20XX (like they did with Visio) and Office 365 instead of bunch of plug-ins to Excel and SharePoint.

It is a mystery for me why Panorama, Visokio and Advizor Solutions still relatively small players, despite all 3 of them having an excellent DV features and products. Based on 2013 IPO experience with Tableau may be the best way for them to go public and get new blood? I wish to them to learn from Tableau and Qlikview success and try this path in 2014-15…

For Microstrategy my wish is very simple – they are only traditional BI player who realised that BI is dead and they started in 2013 (actually before then 2013) a transition into DV market and I wish them all success they can handle!

I also think that a few thousands of Tableau, Qlikview and Spotfire customers (say 5% of customer base) will need (in 2014 and beyond) more deep Analytics and they will try to complement their Data Visualizations with Advanced Visualization technologies they can get from vendors like

My best wishes to everyone! Happy New Year!


2 months ago TIBCO (Symbol TIBX on NASDAQ) announced Spotfire 6 at TUCON 2013 user conference. This as well a follow-up release  (around 12/7/13) of Spotfire Cloud supposed to be good for TIBX prices. Instead since then TIBX lost more then 8%, while NASDAQ as whole grew more then 5%:


For example, at TUCON 2013 TIBCO’s CEO re-declared “5 primary forces for 21st century“(IMHO all 5 “drivers” sounds to me like obsolete IBM-ish Sales pitches) – I guess to underscore the relevance of TIBCO’s strategy and products to 21st century:

  1. Explosion of data (sounds like Sun rises in the East);

  2. Rise of mobility (any kid with smartphone will say the same);

  3. Emergence of Platforms (not sure if this a good pitch, at least it was not clear from TIBCO’s presentation);

  4. Emergence of Asian Economies (what else you expect? This is the side effect of the greedy offshoring for more then decade);

  5. Math trumping Science  (Mr. Ranadive and various other TUCON speakers kept repeating this mantra, showing that they think that statistics and “math” are the same thing and they do not know how valuable science can be. I personally think that recycling this pitch is dangerous for TIBCO sales and I suggest to replace this statement with something more appealing and more mature).

Somehow TUCON 2013 propaganda and introduction of new and more capable version 6 of Spotfire and Spotfire Cloud did not help TIBCO’s stock. For example In trading on Thursday, 12/12/13 the shares of TIBCO Software, Inc. (NASD: TIBX) crossed below their 200 day moving average of $22.86, changing hands as low as $22.39 per share while Market Capitalization was oscillating around $3.9B, basically the same as the capitalization of 3 times smaller (in terms of employees) competitor Tableau Software.

As I said above, just a few days before this low TIBX price, on 12/7/13, as promised on TUCON 2013, TIBCO launched Spotfire Cloud and published licensing and pricing for it.

Most disappointing news is that in reality TIBCO withdrew itself from the competition for mindshare with Tableau Public (more then 100 millions of users, more then 40000 active publishers and Visualization Authors with Tableau Public Profile), because TIBCO no longer offers free annual evaluations. In addition, new Spotfire Cloud Personal service ($300/year, 100GB storage, 1 business author seat) became less useful under new license since its Desktop Client has limited connectivity to local data and can upload only local DXP files.

The 2nd Cloud option called Spotfire Cloud Work Group ($2000/year, 250GB storage, 1 business author/1 analyst/5 consumer seats) and gives to one author almost complete TIBCO Spotfire Analyst with ability to read 17 different types of local files (dxp, stdf, sbdf, sfs, xls, xlsx, xlsm, xlsb, csv, txt, mdb, mde, accdb, accde, sas7bdat,udl, log, shp), connectivity to standard Data Sources (ODBC, OleDb, Oracle, Microsoft SQL Server Compact Data Provider 4.0, .NET Data Provider for Teradata, ADS Composite Information Server Connection, Microsoft SQL Server (including Analysis Services), Teradata and TIBCO Spotfire Maps. It also enables author  to do predictive analytics, forecasting, and local R language scripting).

This 2nd Spotfire’s Cloud option does not reduce Spotfire chances to compete with Tableau Online, which costs 4 times less ($500/year). However (thanks to 2 Blog Visitors – both with name Steve – for help), you cannot use Tableau online without licensed version of Tableau Desktop ($1999 perpetual non-expiring desktop license with 1st year maintenance included and each following year 20% $400 per year maintenance) and Online License (additional $500/year for access to the same site, but extra storage will not be added to that site!) for each consumer. Let’s compare Spotfire Workgroup Edition and Tableau Online cumulative cost for 1, 2, 3 and 4 years for 1 developer/analyst and 5 consumer seats :


Cumulative cost for 1, 2, 3 and 4 years of usage/subscription, 1 developer/analyst and 5 consumer seats:


Spotfire Cloud Work Group, 250GB storage

Tableau Online (with Desktop), 100GB storage

Cost Difference (negative if Spotfire cheaper)

















UPDATE: You may need to consider some other properties, like available storage and number of users who can consume/review visualizations, published in cloud. In sample above:

  • Spotfire giving to Work Group total 250 GB storage, while Tableau giving total 100 GB to the site. 2 or more subscriptions can be associated with the same site, but it will not increase the size of storage for the site from 100 GB to more (e.g. 200 GB for 2 subscribers). 
  • Spotfire costs less than Tableau Online for similar configuration (almost twice less!)

Overall, Spotfire giving more for your $$$ and as such can be a front-runner in Cloud Data Visualization race, considering that Qlikview does not have any comparable cloud options (yet) and Qliktech relying on its partners (I doubt it can be competitive) to offer Qlikview-based services in the cloud. Gere is the same table as above but as IMage (to make sure all web browsers can see it):


It is important to consider another advantage of Spotfire Cloud: ability to share visualizations with everybody on internet by publishing them into Public Folder(s). By contrast, Tableau has limited licensing for this: in order to access to published workbooks on Tableau Online site, the Tableau Software by default requires the extra subscription, which is wrong from my point of view, because you can just publish it on Public Folder of such site (if such option allowed). By default (and without additional negotiations) Tableau Online does not allow the usage of Public Folder.

3rd Spotfire’s Cloud option called Spotfire Cloud Enterprise, it has customizable seating options and storage, more advanced visualization, security and scalability and connects to 40+ additional data sources. It requires an annoying negotiations with TIBCO sales, which may result to even larger pricing. Existence of 3rd Spotfire Cloud option decreases the value of its 2nd Cloud Option, because it saying to customer that Spotfire Cloud Work Group is not best and does not include many features. Opposite to that is Tableau’s Cloud approach: you will get everything (with one exception: Multidimensional (cube) data sources are not supported by Tableau Online) with Tableau Online, which is only the option.

Update 12/20/13:  TIBCO announced results for last quarter, ending 11/30/13 with Quarterly revenue $315.5M (only 6.4% growth compare with the same Quarter of 2012) and $1070M Revenue for 12 months ended 11/30/13 (only 4.4% growth compare with the same period of 2012). Wall Street people do not like it today and TIBX lost today 10% of its value, with Share Price ending $22 and Market Capitalization went down to less then $3.6B. At the same time Tableau’s Share Price went up $1 to $66 and Market Capitalization of Tableau Software (symbol DATA) went above $3.9B). As always I think it is relevant to compare the number of job openings today: Spotfire – 28, Tableau – 176, Qliktech – 71

Since we approaching (in USA that is) a Thanksgiving Day for 2013 and shopping is not a sin for few days, multiple blog visitors asked me what hardware advise I can share for their Data Science and Visualization Lab(s). First of all I wish you will get a good Turkey for Thanksgiving (below is what I got last year):


I cannot answer DV Lab questions individually – everybody has own needs, specifics and budget, but I can share my shopping thoughts about needs for Data Visualization Lab (DV Lab). I think DV Lab needs many different types of devices: smartphones, tablets, projector (at least 1), may be a couple of Large Touchscreen Monitors (or LED TVs connectable to PCs), multiple mobile workstations (depends on size of DV Lab team), at least one or two super-workstation/server(S) residing within DV Lab etc.

Smartphones and Tablets

I use Samsung Galaxy S4 as of now, but for DV Lab needs I will consider either Sony Xperia Z Ultra or Nokia 1520 with hope that Samsung Galaxy S5 will be released soon (and may be it will be the most appropriate for DV Lab):


My preference for Tablet will be upcoming Google Nexus 10 (2013 or 2014 edition – it is not clear, because Google is very secritive about it) and in certain cases Google Nexus 7 (2013 edition). Until Nexus 10 ( next generation) will be released, I guess that two leading choices will be ASUS Transformer Pad TF701T


and Samsung Galaxy Note 10.1 2014 edition (below is a relative comparison of the size of these 2 excellent tablets):


Projectors, Monitors and may be Cameras.

Next piece of hardware in my mind is a projector with support for full HD resolution and large screens. I think there are many good choices here, but my preference will be BENQ W1080ST for $920 (please advise if you have a better projector in mind in the same price range):


So far you cannot find too many Touchscreen Monitors for reasonable price, so may be these two 27″ touchscreen monitors (DELL P2714T for $620 or Acer T272HL bmidz for $560) are good choices for now:


I also think that a good digital camera can help to Data Visualization Lab and considering something like this (can be bought for $300): Panasonic Lumix DMC FZ72 with 60X optical zoom and ability to do a Motion Picture Recording as HD Video in 1,920 x 1,080 pixels – for myself:


Mobile and Stationary Workstations and Servers.

If you need to choose CPU, I suggest to start with Intel’s Processor Feature Filter here: . In terms of mobile workstations you can get quad-core notebook (like Dell 4700 for $2400 or Dell Precison 4800 or HP ZBook 15 for $3500) with 32 GB RAM and decent configuration with multiple ports, see sample here:


If you are OK with 16GB of RAM for your workstation, you may prefer Dell M3800 with excellent touchscreen monitor (3200×1800 resolution) and only 2 kg of weight. For a stationary workstation (or rather server) good choices are Dell Precision T7600 or T7610 or HP Z820 workstation. Either of these workstations (it will cost you!) can support up to 256GB RAM, up to 16 or even 24 cores in case of HP Z820), multiple high-capacity hard disks and SSD, excellent Video Controllers and multiple monitors (4 or even 6!) Here is an example of backplane for HP Z820 workstation:


I wish to visitors of this blog a Happy Holidays and good luck with their DV Lab Shopping!

Famous Traditional BI vendor got sick and tired to be out of Data Visualization market and decided to insert itself into it by force by releasing today 2 Free (for all users) Data Visualization Products:

  • MicroStrategy Analytics Desktop™ (Free self-service visual analytics tool)

  • MicroStrategy Analytics Express™ (Free Cloud-based self-service visual analytics)

That looks to me as the huge Disruption of Data Visualization Market: For example similar Desktop Product from Tableau costs $1999 and Cloud Product called Tableau Online costs $500/year/user. It puts Tableau, Qlikview and Spotfire to a very tough position price-wise. However only Tableau stock went down almost $3 (more then %4) today, but MSTR, TIBX an QLIK basically did not react on Microstrategy announcement):


And don’t think that only MIcrostrategy trying to get into DV market. For example SAP did similar (in less-dramatic and non-disruptive fashion) a few months ago with SAP Lumira (Personal Edition is free), also SAP Cloud and Standard edition available too, see it here and here . SAP senior vice president and platform head Steve Lucas 10 weeks ago was asked if SAP would consider buying Tableau, Lucas went in the opposite direction. “We aren’t going to buy Tableau,” Lucas said with a smile on his face. There’s no need to buy an overvalued software company.” Rather, SAP wants to crush companies like Tableau (I doubt it is possible, but SAP is free to try) and build own Data Visualization product line out of Lumira, read more at

If I will be Tableau, Qlikview or Spotfire I will not worry yet about Microstrategy competition yet, because it is unclear how the future R&D for free Analytics Desktop and Express will be funded – out of MicroStrategy Analytics Enterprise™ R&D budget? That can be tricky, considering as of right now Tableau hiring hard (163 open job positions as of yesterday!) and Qliktech is very active too (about 93 openings as of yesterday) and even TIBCO has 36 open positions just for Spotfire alone.

But I may start to worry about other DV Vendor – Datawatch, who recently completed the acquisition of Panopticon. Datawatch grew 45% YoY (2012-over-2011), has only 124 employees but $27.5M in sales, very experienced leadership, 40000+ customers worldwide and mature product line. May be another evidence of it here:

The three MicroStrategy Analytics Platform products also share a common user experience—making it easy to start small with self-service analytics and grow into the production-grade features of Enterprise. Desktop and Express from Microstrategy can be naturally extended (for fee)  to a new enterprise-grade BI&DV Suite, also released today and called MicroStrategy Analytics Enterprise™ (known under other name as MIcrostrategy Suite 9.4). 

New MicroStrategy Analytics Enterprise 9.4 includes data blending, which allows users to combine data from more than one source; the software stores the data in working memory without the need for a separate data integration product.  9.4 can connect with the MongoDB NoSQL data store as well as Hadoop distributions from Hortonworks, Intel and Pivotal. It comes with the R, adds better ESRI integration. The application can now fit 10 times as much data in memory as the previous version could, and the self-service querying now runs up to 40 percent faster.

MicroStrategy Analytics Enterprise™ Suite is also available starting today for free for developers and non-production use: 10 named user licenses of MicroStrategy Intelligence Server, MicroStrategy Web Reporter and Analyst, MicroStrategy Mobile, MicroStrategy Report Services, MicroStrategy Transaction Services, MicroStrategy OLAP Services, MicroStrategy Distribution Services, and MultiSource Option. 1 named user license of development software, MicroStrategy Web Professional, MicroStrategy Developer, and MicroStrategy Architect The server components have a 1 CPU limit).

Quote from Wayne Eckerson, President of  BI Leader Consulting: “The new MicroStrategy Analytics Desktop makes MicroStrategy a top-tier competitor in the red-hot visual discovery market. The company was one of the first traditional enterprise BI vendors to ship a visual discovery tool, so its offering is mature compared to others in its peer group, but it was locked away inside its existing platform. By offering a stand-alone desktop visual discovery tool and making it freely available, MicroStrategy places itself among” Data Visualization Leaders.

You also can read today’s article from very frequent visitor to my blog (his name Akram), who is the Portfolio and Hedge Manager, Daily Trader and excellent investigator of all Data Visualization Stocks, DV Market and DV Vendors. His article “Tableau: The DV Market Just Got More Crowded”  can be found here (cannot resist to quote: “Microstrategy is priced like it has nothing to do with this space, and Tableau is priced like it will own the whole thing.”):

Heatmap generated by Microstrategy Analytic Desktop

Heatmap generated by Microstrategy Analytic Desktop

MicroStrategy Analytics Desktop.

It’s free visual analytics: Free Visual Insight, 100M per file, 1GB total storage, 1 of user, Free e-mail support for 30 days. Free access to online training, forum, and knowledge base.
Data Sources: xls, csv, RDBMSes, Multidimensional Cubes, MapReduce, Columnar DBs, Access with Web Browser, export to Excel, PDF, flash and images, email distribution. The product is freely available to all and can be downloaded instantly at: .

TRellis of Bar Charts generated by Microstrategy Analytics Desktop

TRellis of Bar Charts generated by Microstrategy Analytics Desktop

Kevin Spurway, MicroStrategy’s vice president of industry and mobile marketing said: “The new desktop software was designed to compete with other increasingly popular self-serve, data-discovery desktop visualization tools offered by Tableau and others”. To work with larger data sets, a user should have 2GB or more of working memory on the computer, Spurway said. See more here:

MicroStrategy Analytics Express.

MicroStrategy Analytics Express is a software-as-a-service (SaaS)-based application that delivers all the rapid-fire self-service analytical capabilities of Desktop, plus reports and dashboards, native mobile applications, and secure team-based collaboration – all instantly accessible in the Cloud. Today, the Express community includes over 32,000 users across the globe.

In this release, Express inherits all the major functional upgrades of the MicroStrategy Analytics Platform, including new data blending features, improved performance, new map analytics, and much more. For a limited time, MicroStrategy is also making Express available to all users free for a year. With this valuable offer, users will be able to establish an account, invite tens, hundreds, or even thousands of colleagues to connect, analyze and share their data and insight, and do it all at no charge. For some organizations, the potential value of this offer can be $1 million or more. Users can sign up, access the service, and take advantage of this offer instantly at

MicroStrategy Analytics Express includes Free Visual Insight, Free web browser and iPad access, Free SaaS for one year, 1GB upload per file, unlimited number of users, Free e-mail support for 30 days. Free access to online training, forum, and knowledge base. Data Sources: xls, csv, RDBMSes Columnar DBs, Drobbox, Google Drive Connector, Visual Insight, a lot of security and a lot more, see

All tools from Microstrategy Analytics Platform (Desktop, Express and Entereprise Suite) support standard list of Chart Styles and Types: Bar (Vertical/Horizontal Clustered/Stacked/100% Stacked), Line (Vertical/Horizontal Absolute/Stacked/100% Stacked), Combo Chart (of Bar and Area)Area (Vertical/Horizontal Absolute/Stacked/100% Stacked)

Area Chart Generated by Microstrategy Analytics Express

Area Chart Generated by Microstrategy Analytics Express

Dual Axis ( Bar/Line/Area Vertical/Horizontal), HeatMap, Scatter, Scatter Grid, Bubble, Bubble Grid, Grid,

Data Grid generated by Microstrategy Analytics Express

Data Grid generated by Microstrategy Analytics Express

Pie, Ring, ESRI Maps,

Microstrategy Analytics Desktop and Express integrate and generate ESRI Map Visualizations

Microstrategy Analytics Desktop and Express integrate and generate ESRI Map Visualizations

Network of Nodes, with lines representing links/connections/relationship,

Network Graph Generated by Microstrategy Analytics Express

Network Graph Generated by Microstrategy Analytics Express

Microcharts and Sparklines,


Data and Word Clouds,


and of course any kind of interactive Dashboards as combination of all of the above Charts, Graphs, and Marks:

Interactive Dashboard Generated by Microstrategy Analytics Express

Interactive Dashboard Generated by Microstrategy Analytics Express

Yesterday TIBCO announced Spotfire 6 with features, competitive with Tableau 8.1 and Qlikview.Next (a.k.a Qlikview 12). Some new features will be showcased at TUCON® 2013, TIBCO’s annual user conference, October 14-17, 2013 (2100 attendees). Livestream Video is here: , tune in October 15th and 16th from 11:30am – 3:30pm EST.

More details will be shown in webcasts and webinars (I personally prefer detailed articles, blogposts, slides, PDFs and Demos, but TIBCO’s corporate culture ignores my preferences for years) on 10/30/13 by Steve Farr

Spotfire 6.0 will be available in mid-November, presumably the same time as Tableau 8.1 and before then Qlikview.Next so TIBCO is not a loser in Leap-frogging game for sure…

TIBCO bought the Extended Results and will presumably will show the integration with PSUHBI product, see it here: ; TIBCO called it as Delivery of  personal KPIs and Metrics on any mobile phone, tablet or laptop, online or offline (new name for it will be TIBCO Spotfire® Consumer):

ipadiphoneAnother TIBCO’s Purchase is MAPORAMA and integration with it TIBCO called (very appropriately) as the Location Analytics with promise to

  • Visualize, explore and analyze data in the context of location

  • Expand situational understanding with multi-layered geo-analytics

  • Mashup new data sources to provide precise geo-coding across the enterprise

la1Spotfire Location Services is the Agnostic Platform and supports (I guess this needs to be verified, because sounds too good to be true) any map service, including own TIBCO, ESRI (Spotfire integrates with ESRI previously), Google:

GeoCodingSP6TIBCO has Event processing capabilities (e.g BusinessEvents (5.1.2. now), ActiveSpaces ( currently v. 2.2) and realtime streaming of “Big Data” StreamBase (7.3.7) , they bought (StreamBase that is) a few months ago, see it here: and it will be interesting to see the new Spotfire Events Analytics (to Spot Event patterns)  product (see also: ) integrated with Spotfire 6:

  • Identify new trends and outliers with continuous process monitoring

  • Automate the delivery of analytics applications based on trends

  • Operationalize analytics to support continuous process improvement:


One more capability in Spotfire mentioned (this claim needs to be verified) in recent TIBCO blogpost as the ability to overlap 2 related but separated in real-life processes: the processes of analysis (discovery of insights in data) and execution (deciding and actions) could be separated by days, but with Spotfire 6.0 the entire decision process can happen in real time:


For business user Spotfire 6 has new web-based authoring (Spotfire has a few “Clients”, one called Web Player and another called Enterprise Player, both are not free unlike Tableau Free Reader or Tableau Public). Bridging the gap between simple dashboards and advanced analytic applications, Spotfire 6.0 provides a new client “tailored to meet the needs of the everyday business user, who typically has struggled to manipulate pivot tables and charts to address their data discovery needs”.

With this new web application, known as TIBCO Spotfire® Business Author, business users can visually explore and interact with data, whether residing in a simple spreadsheet or dashboard, a database, or a predefined analytic application. It will definitely compete with Web Authoring in Tableau 8.1 and incoming Qlikview.Next.

For me personally the most interesting new feature is new Spotfire Cloud Services (supposedly the continuation of Spotfire SIlver, which I like but it is overpriced and non-competitive storage-wise vs. Tableau Public and Tableau Online cloud services). Here is the quote from yesterday’s Press Release: “TIBCO Spotfire® Cloud is a new set of cloud services for enterprises, work groups, and personal use (see some preliminary info here: :

  • Personal: Web-based product, Upload Excel, .csv and .txt data, 12 visualization types, 100 GB of data storage. However, Spofire making a big mistake by denying access to Spotfire Analyst desktop product and making it as not free but only as “free trial for 30 days”, after which you have to pay a fee. That will benefit Tableau for sure and may be even Datawatch. As of 11/13/13, Spotfire still did not posted prices and fees for Spotfire Cloud Personal etc. and suggested to contact them over email, which I did but they never replied…
  • Workgroup: Web-based and desktop product, Connect and integrate multiple data sources, All visualization types, 250 GB of data storage.
  • Enterprise: Web-based and desktop product, Connect to 40+ data sources, All visualization types, Advanced statistics services, 500 GB of data storage

TIBCO Spotfire® Cloud Enterprise provides a secure full-featured version of Spotfire in the cloud to analyze and collaborate on business insights, whether or not the data is hosted. For project teams seeking data discovery as a service, TIBCO Spotfire® Cloud Work Group provides a wealth of application-building tools so distributed teams can visually explore data quickly and easily and deploy analytic applications at a very low cost. For individuals looking for a single step to actionable insight, TIBCO Spotfire®Personal is a cost-effective web-based client for quick data discovery needs.”

Please don’t forget that Spotfire 6 has TIBBR v.5 as of now: (social computing platform built for the workplace and integrated with Spotfire; Ram Menon, President of Social Computing at tibbr, says, “We now have 6.5 million users for tibbr as of October [2013]” and accessed from 7,000 cities, and 2,100 different device models. “A typical tibbr post is now seen by 100 users in the span of 24 hours, in 7 countries and over 50 mobile devices.” This fulfills TIBCO’s mission of getting the right information to the right people, at the right time. Related: integration between TIBBR and HUDDLE: )

And finally – enterprise-class, R-compatible statistical engine: TIBCO Enterprise Runtime for R (TERR) which is  the part of excellent TIBCO Spotfire Statistics Services (TSSS). TSSS allows Integration of R (including TERR), Spotfire’s own S+ (SPlus is Spotfire’s commercial version of R), SAS® and MATLAB® into Spotfire and custom applications. TERR, see supports:

  • Support for paralelized R-language scripts in TERR

  • Support for call outs to open source R from TERR

  • Use RStudio – the most popular IDE in the R Community-to develop your TERR scripts

  • Over a thousand TERR compatible CRAN packages

Among other news is support for new Data Sources: including SAP NetWeaver Business Warehouse v.7.0.1 (required TIBCO Connector Link).

General notes:

  1. I maintain my opinion that the best way for TIBCO to capitalize on tremendous hidden market value of Spotfire is to spin-it off as EMC did with VMWare.

  2. My other concern is too many offices involved with Spotfire: (Parental) TIBCO’s HQ in California, Swedish HQ (mostly R&D) office in Sweden and Large Marketing, Sales, Support and Consulting office in Newton, Massachusetts. My advise to have only one main office in MA, which is compatible with spin-off idea. Tableau has advantage here with concentrating their main office in Seattle.

  3. Update 11/13/13: TIBCO’s Spotfire propaganda so far did not help TIBCO stock shares at all, but seems to me that it helps a lot to Datawatch stock prices (Datawatch bought recently a very capable (technically) DV Vendor Panopticon and integrated its own software with Panopticon Software; Datawatch has 40000+ customers with 500000+ end users)

Last month Tableau and Qliktech both declared that Traditional BI is too slow (I am saying this for many years) for development and their new Data Visualization (DV software) is going to replace it. Quote from Tableau’s CEO: Christian Chabot: “Traditional BI software is obsolete and dying and this is very direct challenge and threat to BI vendors: your (BI that is) time is over and now it is time for Tableau.” Similar quote from Anthony Deighton, Qliktech’s CTO & Senior VP, Products: “More and more customers are looking at QlikView not just to supplement traditional BI, but to replace it“.

One of my clients – large corporation (obviously cannot say the name of it due NDA) asked me to advise of what to choose between Traditional BI tools with long Development Cycle (like Cognos, Business Objects or Microstrategy), modern BI tools (like JavaScript and D3 toolkit) which is attempt to modernize traditional BI but still having  sizable development time and leading Data Visualization tools with minimal development time (like Tableau, Qlikview or Spotfire).

Since main criterias for client were

  • minimize IT personnel involved and increase its productivity;

  • minimize the off-shoring and outsourcing as it limits interactions with end users;

  • increase end users’s involvement, feedback and action discovery.

So I advised to client to take some typical Visual Report project from the most productive Traditional  BI Platform (Microstrategy), use its prepared Data and clone it with D3 and Tableau (using experts for both). Results in form of Development time in hours) I put below; all three projects include the same time (16 hours) for Data Preparation & ETL, the same time for Deployment (2 hours) and the same number (8) of Repeated Development Cycles (due 8 consecutive feedback from End Users):


It is clear that Traditional BI requires too much time, that D3 tools just trying to prolongate old/dead BI traditions by modernizing and beautifying BI approach, so my client choose Tableau as a replacement for Microstrategy, Cognos, SAS and Business Objects and better option then D3 (which require smart developers and too much development). This movement to leading Data Visualization platforms is going on right now in most of corporate America, despite IT inertia and existing skillset. Basically it is the application of the simple known principle that “Faster is better then Shorter“, known in science as Fermat’s Principle of least time.

This changes made me wonder (again) if Gartner’s recent marketshare estimate and trends for Dead Horse sales (old traditional BI) will stay for long. Gartner estimates the size of BI market as $13B which is drastically different from TBR estimate ($30B).

BIDeadHorseTheoryTBR predicts that it will keep growing at least until 2018 with yearly rate 4% and BI Software Market to Exceed $40 Billion by 2018 (They estimate BI Market as $30B in 2012 and include more wider category of Business Analytics Software as opposed to strictly BI tools). I added estimates for Microstrategy, Qliktech, Tableau and Spotfire to Gartner’s MarketShare estimates for 2012 here:


However, when Forrester asked people what BI Tools they used, it’s survey results were very different from Gartner’s estimate of “market share:


“Traditional BI is like a pencil with a brick attached to it” said Chris Stolte at recent TCC13 conference and Qliktech said very similar in its recent announcement of Qlikview.Next. I expect TIBCO will say similar about upcoming new release of Spotfire (next week at TUCON 2013 conference in Las Vegas?)


These bold predictions by leading Data Visualization vendors are just simple application of Fermat’s Principle of Least Time: this principle stated that the path taken between two points by a ray of light (or development path in our context) is the path that can be traversed in the least time.

Pierre_de_Fermat2Fermat’s principle can be easily applied to “PATH” estimates to multiple situations like in video below, where path from initial position of the Life Guard on beach to the Swimmer in Distress (Path through Sand, Shoreline and Water) explained: 

Even Ants following the Fermat’s Principle (as described in article at Public Library of Science here: ) so my interpretation of this Law of Nature (“Faster is better then Shorter“) that  traditional BI is a dying horse and I advise everybody to obey the Laws of Nature.

AntsOn2SurfacesIf you like to watch another video about Fermat’s principle of Least Time and related Snell’s law, you can watch this: 

Qlikview 10 was released around 10/10/10, Qlikview 11 – around 11/11/11, so I expected Qlikview 12 to be released on 12/12/12. Qliktech press release said today that the next (after 11.2) version of Qlikview will be delivered under the new nickname Qlikview.Next in 2014 but “for  early adopter customers in a production environment in 2013”. I hope I can get my hands on it ASAP!

The new buzzword is Natural Analytics: “QlikView.Next’s key value as an alternative BI platform is in its use of Natural Analytics“. The new Qliktech motto that “Qlikview is a Replacement of Traditional BI” is similar to what we heard from Tableau leaders just 2 weeks ago on Tableau Customer Conference in Washington, DC.  Another themes I hear from Qliktech about Qliview.Next are sounds familiar too: Gorgeous, Genius, Visually Beautiful, Associative Experience, Comparative Analysis, Anticipatory, Drag and Drop Analytics.

Qlikview.Next will introduce “Data Dialogs” as live discussions between multiple users about Data they see and explore collectively, enabling “Social BI”. This reminds me the integration between TIBBR (TIBCO’s collaboration platform) and Spotfire, which existed since Spotfire 4.0.

Details about new features in Qlikview.Next will be released later, but at least we know now when Qlikview 12 (sorry, Qlikview.Next that is) will be available. Some features actually unveiled in generic terms::

  • Unified, Browser-Based HTML5 Client, which will automatically optimize itself for user’ device;

  • Automatic and Intelligent re-sizing of objects to fit user’s screen;

  • Server-side Analysis and Development, Web-based creation and delivery of content, Browser-based Development;

  • Data Storytelling, narrative and social with Data Dialogs;

  • Library and Repository for UI objects;

  • Multi-source Data Integration and new web-based scripting;

  • QlikView Expressor for advanced graphical Data Integration and Metadata Management;

  • Improved Data Discovery with associative experience across all the data, both in memory and on disks;

  • Open API: JSON, .NET SDK and as JavaScript API;

  • All UI Objects can be treated as extension Objects, customizable with their source files available to developers;

  • New Managment Console with Qlikview on Qlikview Monitor;

  • New visualization capabilities, based on advanced data visualization suite from NComVA (bought by Qliktech a few months ago), potential samples see here:


In addition Qliktech is launching the “Qlik Customer Success Framework” , which includes:

  • Qonnect Partner Program: An extensive global network of 1500+ partners, including resellers, (OEMs), technology companies, and system integrators.

  • Qlik Community: An online community with nearly 100,000 members comprised of customers, partners, developers and enthusiasts.

  • Qlik Market: An online showcase of applications, extensions and connectors.

  • Qoncierge: A single point of contact service offering for customers to help them access the resources they need.

  • Comprehensive Services: A wide range of consulting services, training and support.


Also see Ted Cuzzillo blogpost about it here: and Cindi Howson’s old post here: and new article here:!

Today Tableau Customer Conference 2013 started with 3200+ attendees from 40+ countries and 100+ industries, with 700 employees of Tableau, 240 sessions. Tableau 8.1 pre-announced today for release in fall of 2013, also version 8.2 planned for winter 2014, and Tableau 9.0 for later in 2014.

Update 9/10/13: keynote now is available recorded and online:
(Recorded Monday Sept 9, 2013 Christian Chabot, Chris Stolte and the developers LIVE)

New in 8.1: 64-bit, Integration with R, support for SAML, IPV6 and External Load Balancers, Copy/Paste Dashboards and worksheets between workbooks, new Calendar Control, own visual style, including customizing even filters, Tukey’s Box-and-Whisker Box-plot, prediction bands, ranking, visual analytics for everyone and everywhere (in the cloud now)

Planned and new for 8.2: Tableau for MAC, Story Points (new type of worksheet/dashboard with mini-slides as story-points), seamless access to data via data connection interface to visually build a data schema, including inner/left/right/outer visual joins, beautifying columns names, easier metadata etc, Web authoring enhancements (it may get into 8.1: moving quick filters, improvement for Tablets, color encoding.) etc.

8.1:  Francois Ajenstat announced: 64-bit finally (I asked for that for many years) for server processes and for Desktop, support for SAML (single-sign-ON on Server and Desktop), IPV6, External Load Balancers:


SAML8.1: Dave Lion announced R integration with Tableau:


r8.1: Mike Arvold announced “Visual Analytics for everyone”, including implemention of famous Tukey’s Box-and-Whisker Box-plot (Spotfire has it for a while, see it here: ),

better forecasting, prediction bands, ranking, better heatmaps:

MikeArvold8.1: Melinda Minch announced “fast, easy, beautiful”, most importantly copy/paste dashboards and worksheets between workbooks, customizing everything, including quick filters, new calendar control, own visual style, folders in Data Window etc…

MelindaMinch28.2: Jason King pre-announced the Seamless access to data via data connection interface to visually build a data schema, including inner/left/right/outer “visual” joins, beautifying columns names, default formats, new functions like DATEPARSE, appending data-set with new tables, beautifying columns names, easier metadata etc.

JasonKingSeamlessAccess2data28.2: Robert Kosara introduced Story Points (using new type of worksheet/dashboard with mini-slides as story-points) for new Storytelling functionality:


Here is an example of Story Points, done by Robert:


8.2: Andrew Beers pre-announced Tableau 8.2 on MAC and he got a very warm reception from audience for that:

AndrewBeers3Chris Stolte proudly mentioned his 275-strong development team, pre-announced upcoming Tableau Releases 8.1 (this fall), 8.2 (winter 2014) and 9.0 (later in 2014) and introduced 7 “developers” who (see above Francois, Mike, Dave, Melinda, Jason, Robert and Andrew) discussed during this keynote new features (feature list is definitely longer and wider that recent “innovations” we saw from Qlikview 11.2 and even from Spotfire 5.5):

ChrisStolte2Christian Chabot opening keynote today… He said something important: current BI Platforms are not fast, nor easy, they are not beautiful and not for anyone and they are definitely not “anywhere” but only in designated places with appropriate IT personnel (compare with Tableau Public, Tableau Online, Tableau free Reader etc.) and it is only capable to produce a bunch of change requests from one Enterprise’s department to another, which will take long time to implement with any SDLC framework.

CEOChristian basically repeated what I am saying on this blog for many years, check it here : traditional BI software (from SAP, IBM, Oracle, Microstrategy and even Microsoft cannot compete with Tableau, Qlikview and Spotfire) is obsolete and dying and this is very direct challenge and threat to BI vendors (I am not sure if they understand that): your (BI that is) time is over and now it is time for Tableau (also for Qlikview and Spotfire but they are slightly behind now…).

Update on 11/21/13: Tableau 8.1 is available today, see it here: and Tableau Public 8.1 is available as well, see it here:

While blog preserving my observations and thoughts, it preventing me to spend enough time to read what other people thinking and saying, so I created almost 2 years ago the extension of this blog in the form of 2 Google+ pages and , where I accumulated all reading pointers for myself and gradually reading those materials when I have time.

Those 2 pages magically became extremely popular (this is unintended result) with total more than 5000 Google+ followers as of today. For example here is a Chart showing monthly growth of the  number of followers for the main extension of this blog :


So please see below some samples of Reading Pointers accumulated over last 3 months of summer by my Google+ pages:

Author trying to simplify BigData Definition as following: “BigData Simplified: Too much data to fit into a single server”:

Recent talk from Donald Farmer:

Dmitry pointing to implementation Disaster of Direct Discovery in Qlikview 11.2:

Specs for Tableau in Cloud:

The DB-Engines Monthly Ranking ranks database management systems according to their popularity. Turned out that only 3 DBMSes are popular: Oracle, SQL Server and MySQL:

According to Dr. Andrew Jennings, chief analytics officer at FICO and head of FICO Labs, three main skills of data scientist are the same 3 skills I tried to find when hiring programmers for my teams 5, 10, 20 and more years ago: 1. Problem-Solving Skills. 2. Communications Skills. 3. Open-Mindedness. This makes all my hires for last 20+ years Data Scientists, right? See it here:

A study finds the odds of rising to another income level are notably low in certain cities, like Atlanta and Charlotte, and much higher in New York and Boston:

Tableau is a prototyping tool:

Why More Data and Simple Algorithms Beat Complex Analytics Models:

New Census Bureau Interactive Map Shows Languages Spoken in America:

Google silently open sourced a tool called word2vec, prepackaged deep-learning software designed to understand the relationships between words with no human guidance. It actually similar to known for a decade methods called PLSI and PLSA:

“Money is not the only reward of education, yet it is surely the primary selling point used to market data science programs, and the primary motivator for students. But there’s no clear definition of data science and no clear understanding of what knowledge employers are willing to pay for, or how much they will pay, now or in the future. Already I know many competent, diligent data analysts who are unemployed or underemployed. So, I am highly skeptical that the students who will invest their time and money in data science programs will reap the rewards they have been led to expect.”:

Some good blog-posts from InterWorks:

Technique for using Tableau data blending to create a dynamic, data-driven “parameter”:

More about Colors:

Russian Postcodes are collected and partially visualized:

EXASolution claims to be up to 1000 times faster than traditional databases and the fastest database in the world – based on in memory computing.

web interest to Tableau and Qlikview:

With releases of Spotfire Silver (soon to to be a Spotfire Cloud), Tableau Online and attempts of a few Qlikview Partners (but not Qliktech itself yet) to the Cloud and providing their Data Visualization Platforms and Software as a Service, the Attributes, Parameters and Concerns of such VaaS or DVaaS ( Visualization as a Service) are important to understand. Below is attempt to review those “Cloud” details at least on a high level (with natural limitation of space and time applied to review).

But before that let’s underscore that Clouds are not in the skies but rather in huge weird buildings with special Physical and Infrastructure security likes this Data Center in Georgia:


You can see some real old fashion clouds above the building but they are not what we are talking about. Inside Data Center you can see a lot of Racks, each with 20+ servers which are, together with all secure network and application infrastructure contain these modern “Clouds”:


Attributes and Parameters of mature SaaS (and VaaS as well) include:

  • Multitenant and Scalable Architecture (this topic is too big and needs own blogpost or article). You can review Tableau’s whitepaper about Tableau Server scalability here:
  • SLA – service level agreement with up-time, performance, security-related and disaster recovery metrics and certifications like SSAE16.
  • UI and Management tools for User Privileges, Credentials and Policies.
  • System-wide Security: SLA-enforced and monitored Physical, Network, Application, OS and Data Security.
  • Protection or/and Encryption of all or at least sensitive (like SSN) fields/columns.
  • Application Performance: Transaction processing speed, Network Latency, Transaction Volume, Webpage delivery times, Query response times
  • 24/7 high availability: Facilities with reliable and backup power and cooling, Certified Network Infrastructure, N+1 Redundancy, 99.9% (or 99.99% or whatever your SLA with clients promised) up-time
  • Detailed historical availability, performance and planned maintenance data with Monitoring and Operational Dashboards, Alerts and Root Cause Analysis
  • Disaster recovery plan with multiple backup copies of customers’ data in near real time at the disk level, a 

    multilevel backup strategy that includes disk-to-disk-to-tape data backup where tape backups serve as a secondary level of backup, not as their primary disaster recovery data source.

  • Fail-over that cascades from server to server and from data center to data center in the event of a regional disaster, such as a hurricane or flood.

While Security, Privacy, Latency and Hidden Cost usually are biggest concerns when considering SaaS/VaaS, other Cloud Concerns surveyed and visualized below. Recent survey and diagram are published by Charlie Burns this month:


Other survey and diagram are published by Shane Schick in October 2011 and in February of 2013 by KPMG. Here are concerns, captured by KPMG survey:


As you see above, Rack in Data Center can contain multiple Servers and other devices (like Routers and Switches, often redundant (at least 2 or sometimes N+1). Recently I designed the Hosting Data VaaS Center for Data Visualization and Business Intelligence Cloud Services and here are simplified version of it just for one Rack as a Sample.

You can see redundant network, redundant Firewalls, redundant Switches for DMZ (so called “Demilitarized Zone” where users from outside of firewall can access servers like WEB or FTP), redundant main Switches and Redundant Load Balancers, Redundant Tableau Servers, Redundant Teradata Servers, Redundant Hadoop Servers, Redundant NAS servers etc. (not all devices shown on Diagram of this Rack):


20 months ago I checked how many job openings leading DV Vendors have. On 12/5/11 Tableau had 56, Qliktech had 46 and Spotfire had 21 openings. Today morning I checked their career sites again and noticed that both Tableau and Qliktech almost double their thirst for new talents, while Spotfire basically staying on the same level of hiring needs:

  • Tableau has 102(!) openings, 43 of them are engineering positions (I counted their R&D positions and openings in Operation department too) – that is huge! Update as of 9/18/13 has exactly 1000 employees. 1000th employee can be found on this pictureTableau1000Employees091813

  • Qliktech has 87 openings, 29 of them are engineering positions (I included R&D, IT, Tech Support and Consulting).

  • TIBCO/Spotfire has 24 openings, 16 of them are engineering positions (R&D, IT, Tech.Support).


All 3 companies are Public now, so I decided to include their Market Capitalization as well. Since Spofire is hidden inside its corporate parent TIBCO, I used my estimate that Spotfire’s Capitalization is about 20% of TIBCO’s capitalization (which is $3.81B as of 8/23/13, see ). As a result I have this Market Capitalization numbers for 8/23/13 as closing day:

Those 3 DV Vendors above together have almost $8B market capitalization as of evening of 8/23/13 !

Market Capitalization update as of 8/31/13: Tableau: $4.3B, Qliktech $2.9B, Spotfire (as 20% of TIBCO) – $0.72B

Market Capitalization update as of 9/4/13 11pm: Tableau: $4.39B, Qliktech $3B, Spotfire (as 20% of TIBCO) – $0.75B . Also as of today Qliktech employed 1500+ (approx. $300K revenue per year per employee), Tableau about 1000 (approx. $200K revenue per year per employee) and Spotfire about 500+ (very rough estimate, also approx. $350K revenue per year per employee)


Last week Tableau increased by 10-fold the capacity of Data Visualizations published with Tableau Public to a cool 1 Million rows of Data, basically to the same amount of rows, which Excel 2007, 2010 and 2013 (often used as data sources for Tableau Public) can handle these days and increased by 20-fold the storage capacity (to 1GB of free storage) of each free Tableau Public Account, see it here:

It means that free Tableau Public Account will have the storage twice larger than Spotfire Silver’s the most expensive Analyst Account (that one will cost you $4500/year). Tableau said: “Consider it a gift from us to you.”. I have to admit that even kids in this country know that there is nothing free here, so please kid me not – we are all witnessing of some kind of investment here – this type of investment worked brilliantly in the past… And all users of Tableau Public are investing too – with their time and learning efforts.

And this is not all: “For customers of Tableau Public Premium, which allows users to save locally and disable download of their workbooks, the limits have been increased to 10 million rows of data at 10GB of storage space” see it here: without changing the price of service (of course in Tableau Public Premium price is not fixed and depends on the number of impressions).

Out of 100+ millions of Tableau users only 40000 qualified to be called Tableau Authors, see it here so they are consuming Tableau Public’s Storage more actively then others. As an example you can see my Tableau’s Author Profile here: .

I will assume those Authors will consume 40000GB of online storage, which will cost to Tableau Software less then (my guess, I am open to correction from blog visitors) $20K/year just for the storage part of Tableau Public Service.

During the last week the other important announcement on 8/8/13 – Quarterly Revenue – came from Tableau: it reported the Q2 revenue of $49.9 million, up 71% year-over-year: .

Please note that 71% is extremely good YoY growth compare with the entire anemic “BI industry”, but less then 100% YoY which Tableau grew in its private past.

All these announcements above happened simultaneously with some magical (I have no theory why this happened; one weak theory is the investors madness and over-excitement about Q2 revenue of $49.9M announced on 8/8/13?) and sudden increase of the nominal price of Tableau Stock (under the DATA name on NYSE) from $56 (which is already high) on August 1st 2013 (announcement of 1 millions of rows/1GB storage for Tableau public Accounts) to $72+ today:


It means that the Market Capitalization of Tableau Software may be approaching $4B and sales may be $200M/year. For comparison, Tableau’s direct and more mature competitor Qliktech has now the Capitalization below $3B while its sales approaching almost $500M/year. From Market Capitalization point of view in 3 moths Tableau went from a private company to the largest Data Visualization publicly-traded software company on market!

Competition in Data Visualization market is not only on features, market share and mindshare but also on pricing and lisensing. For example the Qlikview licensing and pricing is public for a while here: and Spotfire Silver pricing public for a while too: .

Tableau Desktop has 3 editions: Public (Free), Personal ($999) and Professional ($1999), see it here: ; in addition you can have full Desktop (read-only) experience with free Tableau Reader (neither Qlikview nor Spotfire have free readers for server-less, unlimited distribution of Visualizations, which is making Tableau a mind-share leader right away…)

The release of Tableau Server online hosting this month: heated the licensing competition and may force the large changes in licencing landscape for Data Visualization vendors. Tableau Server existed in the cloud for a while with tremendous success as Tableau Public (free) and Tableau Public Premium (former Tableau Digital with its weird pricing based on “impressions”).

But Tableau Online is much more disruptive for BI market: for $500/year you can get the complete Tableau Server site (administered by you!) in the cloud with (initially) 25 (it can grow) authenticated by you users and 100GB of cloud storage for your visualizations, which is 200 times more then you can get for $4500/year top-of-the line Spotfire Silver “Analyst account”. This Tableau Server site will be managed in the cloud by Tableau Software own experts and require nor IT personnel from your side! You may also compare it with .

A hosted by Tableau Software solution is particularly useful when sharing dashboards with customers and partners because the solution is secure but outside a company’s firewall. In the case of Tableau Online users can publish interactive dashboards to the web and share them with clients or partners without granting behind-the-firewall access.

Since Tableau 8 has new Data Extract API, you can do all data refreshes behind your own firewall and republish your TDE files in the cloud anytime (even automatically, on demand or on schedule) you need. Tableau Online has no minimum number of users and can scale as a company grows. At any point, a company can migrate to Tableau Server to manage it in-house. Here is some introductionla video about Tableau Online: Get started with Tableau Online.

Tableau Server in the cloud provides at least 3 ways to update your data (more details see here: )


Here is another, more lengthy intro into Tableau BI in Cloud:

Tableau as a Service is a step in right direction, but be cautious:  in practice, the architecture of the hosted version could impact performance. Plus, the nature of the product means that Tableau isn’t really able to offer features like pay-as-you-go that have made cloud-based software popular with workers. By their nature, data visualization products require access to data. For businesses that store their data internally, they must publish their data to Tableau’s servers. That can be a problem for businesses that have large amounts of data or that are prevented from shifting their data off premises for legal or security reasons. It could also create a synchronization nightmare, as workers play with data hosted at Tableau that may not be as up-to-date as internally stored data. Depending on the location of the customer relative to Tableau’s data center, data access could be slow.

And finally, the online version requires the desktop client, which costs $2,000. Tableau may implement Tableau desktop analytical features in a browser in the future while continue to support the desktop and on-premise model to meet security and regulations facing some customers.


I got many questions from Data Visualization Blog’s visitors about differences between compensation for full-time employees and contractors. It turned out that many visitors are actually contractors, hired because of their Tableau or Qlikview or Spotfire skills and also some visitors consider a possibility to convert to consulting or vice versa: from consulting to FullTimers. I am not expert in all these compensation and especially benefits-related questions, but I promised myself that my blog will be driven by vistors’s requests, so I google a little about Contractor vs. Full-Time worker compensation and below is brief description of what I got:

Federal Insurance Contribution Act mandates Payroll Tax splitted between employer (6.2% Social Security with max $$7049.40 and 1.45% Medicare on all income) and employee, with total (2013) as 15.3% of gross compensation.


In addition you have to take in account employer’s contribution (for family it is about $1000/per month) to medical benefits of employee, Unemployment Taxes, employer’s contribution to 401(k), STD and LTD (short and long term disability insurances), pension plans etc.

I also added into my estimate of contractor rate the “protection” for at least 1 month GAP between contracts and 1 month of salary as bonus for full-time employees.


Basically the result of my minimal estimate as following: you need to get as a contractor the rate at least 50% more than base hourly rate of the full-time employee. This  base hourly rate of full-time employee I calculate as employee’s base salary divided on 1872 hours: 1872 = (52 weeks*40 hours – 3 weeks of vacation – 5 sick days – 6 holidays) = 2080 hours – 208 hours (Minimum for a reasonable PTO, Personal Time Off) = 1872 working hours per year.

I did not get into account any variations related to the usage of W2 or 1099 forms or Corp-To-Corp arrangements and many other fine details (like relocation requirements and overhead associated with involvement of middlemen like headhunters and recruiters) and differences between compensation of full-time employee and consultant working on contract – this is just a my rough estimate – please consult with experts and do not ask me any questions related to MY estimate, which is this:

  • Contractor Rate should be 150% of the base rate of a FullTimer

RS-COLLEGE LOAN SCAMS low resIn general, using Contractors (especially for business analytics) instead of Full-timers is basically the same mistake as outsourcing and off-shoring: companies doing that do not understand that their main assets are full-time people. Contractors are usually not engaged and they are not in business to preserve intellectual property of company.

For reference see Results of Dr. Dobbs 2013 Salary Survey for Software Developers which are very comparable with salary of Qlikview, Tableau and Spotfire developers and consultants (only in my experience salary of Data Visualization Consultants are 10-15% higher then salaries of software developers):


This means that for 2013 the Average rate for Qlikview, Tableau and Spotfire developers and consultants should be around 160% of the base rate of a average FullTimer, which ESTIMATES to Effective Equivalent Pay to Contractor for 1872 hours per Year as $155,200 and this is only for average consultant... If you take less then somebody tricked you, but if you read above you already know that.

2400 years ago the concept of Data Visualization was less known, but even than Plato said “Those who tell stories rule society“.


I witnessed multiple times how storytelling triggered the Venture Capitalists (VCs) to invest. Usually my CEO (biggest BS master on our team) will start with a “60-seconds-long” short Story (VCs called them “Elevator Pitch”) and then (if interested) VCs will do a long Due Diligence Research of Data (and Specs, Docs and Code) presented by our team and after that they will spend comparable time analyzing Data Visualizations (Charts, Diagrams, Slides etc.) of our Data, trying to prove or disprove the original Story.

Some of conclusions from all these startup storytelling activity were:

  • Data: without Data nothing can be proved or disproved (Action needs Data!)

  • View: best way to analyze Data and trust it is to Visualize it (Seeing is Believing!)

  • Discovery of Patterns: visually discoverable trends, outliers, clusters etc. which form the basis of the Story and follow-up actions

  • Story: the Story (based on that Data) is the Trigger for the Actions (Story shows the Value!),

  • Action(s): start with drilldown to a needle in haystack, embed Data Visualization into business, it is not an Eye Candy but a practical way to improve the business

  • Data Visualization has 5 parts: Data (main), View (enabler), Discovery (visually discoverable Patterns), Story (trigger for Actions) and finally the 5th Element – Action!

  • Life is not fair: Storytellers were there people who benefited the most in the end… (no Story no Glory!).


And yes, Plato was correct – at least partially and for his time. Diagram above uses analogy with 5 Classical Greek Elements. Plato wrote about four classical elements (earth, air, water, and fire) almost 2400 years ago (citing even more ancient philosopher) and his student Aristotle added a fifth element, aithêr (aether in Latin, “ether” in English) – both men are in the center of 1st picture above.

Back to our time: the Storytelling is a hot topic; enthusiasts saying that “Data is easy, good storytelling is the challenge” or even that “Data Science is a Storytelling”: . Nothing can be further from the truth: my observation is that most Storytellers (with a few known exceptions like Hans Rosling or Tableau founder Pat Hanrahan) ARE NOT GOOD at visualizing but they still wish to participate in our hot Data Visualization party. All I can say is “Welcome to the party!”

It may be a challenge for me and you but not for people who had a conference about storytelling: this winter, 2/27/13 in Nashville, KY: :

Some more reasonable  people referring to storytelling as a data journalism and narrative visualization:

Tableau founder Pat Hanrahan recently talked about “Showing is Not Explaining”. In parallel, Tableau is planning (after version 8.0) to add features that support storytelling by constructing visual narratives and effective communication of ideas, see it here:

Collection of resources on storytelling topic can be found here:

You may also to check what Stephen Few thinks about it here:

Storytelling as an important part (using Greek Analogy – 4th Classical Element (Air) after Data (Earth), View (Water) and Discovery (Fire) and before Action (Aether) ) of Data Visualization has a practical effect on Visualization itself, for example:

  • if Data View is not needed for Story or for further Actions, then it can be hidden or removed,

  • if number of Data Views in Dashboard is affecting impact of (preferably short Data Story), then number of Views should be reduced (usually to 2 or 3 per dashboard),

  • If number of DataPoints is too large per View and affecting the triggering power of the story, then it can be reduced too (in conversations with Tableau they even recommending 5000 Datapoints per View as a threshold between Local and Server-based rendering).




Below you can find samples of Guidelines and Good Practices for Data Visualization (mostly with Tableau), which I used recently.

best-practiceSome of this samples are Tableau-specific, but others (may be with modifications) can be reused for other Data Visualization Platform and tools. I will appreciate feedback, comments and suggestions.

Naming Convention for Tableau Objects

  • Use CamelCase Identifiers: Capitalize the 1st letter of each concatenated word

  • Use Suffix for Identifiers with preceding underscore to indicate the type (example: _tw for workbooks).

Workbook Sizing Guidelines

  • Use Less than 5 Charts per Dashboard, Minimize the number of Visible TABs/Worksheets

  • Move Calculations and Functions from Workbook to the Data.

  • Use less than 5000 Data-points per Chart/Dashboard to enable Client-side rendering.

  • To enable Shared Sessions, don’t use filters and interactivity if it is not needed.

Guidelines for Colors, Fonts, Sizes

  • To express desirable/undesirable points, use green for good, red for bad, yellow for warning.

  • When you are not describing “Good-Bad situation” (thanks to feedback of visitor under alias “SF”) , try to use pastel, neutral and blind colors, e.g. similar to “Color Blind 10” Palette from Tableau.

  • Use “web-safe” fonts, to approximate what users can see from Tableau Server.

  • Use either auto-resize or standard (target smaller screen) sizes for Dashboards

Data and Data Connections used with Tableau

  • Try to avoid pulling more than 15000 rows for Live Data Connections.

  • For Data Extract-based connections 10M rows is the recommended maximum.

  • For widely distributed Workbooks use of Application IDs instead of Personal Credentials.

  • Job failure due expired credentials leads to suspension from Schedule, so try to keep embedded credentials up to date


Tableau Data Extracts (TDE)

  • If Refresh of TDE takes more than 2 hours, consider to redesign it.

  • Reuse and share TDEs and Data Sources as much as possible.

  • Use of Incremental Data Refresh instead of Full Refresh when possible.

  • Designate Unique ID for each row when Incremental Data Refresh is used.

  • Try to use free Tableau Data Exract API instead of licensed Tableau Server to create Data Extracts

Scheduling of Background Tasks with Tableau

  • Serial Schedules is recommended; avoid the usage of hourly Schedules.

  • Avoid scheduling during peak hours (8am-6pm), consider weekly instead of daily schedules.

  • Optimize Schedule Size, group tasks related to the same project into one Schedule, if total tasks execution exceeds 8 hours, split Schedule on a few with similar Name but preferably with different starting time.

  • Maximize the usage of Monthly and Weekly Schedules (as oppose to Daily Schedules) and usage of weekends and nights.

Guidelines for using Charts

  • Use Bars to compare across categories, use Colors with Stacked or Side-by-Side Bars for deeper Analysis

  • Use Line for Viewing Trends over time, consider Area Charts for Multi-lines

  • Minimize the usage of Pie Charts; when appropriate – use it for showing proportions. It is recommended to limit pie wedges to six.

  • Use Map to show geocoded data, consider use maps as interactive filters

  • Use Scatter to analyze outliers, clusters and construct regressions


You can find more about Guidelines and Good Practices for Data Visualization here:

Tableau Software filed for IPO, on the New York Stock Exchange under the symbol “DATA”. In sharp contrast to other business-software makers that have gone public in the past year, Tableau is profitable, despite hiring huge number of new employees. For the years ended December 31, 2010, 2011 and 2012,  Tableau’s total revenue were $34.2 million, $62.4 million and $127.7 million for 2012. Number of full-time employees increased from 188 as of December 31, 2010 to 749 as of December 31, 2012.

Tableau’s biggest shareholder is venture capital firm New Enterprise Associates, with a 38 percent stake. Founder Pat Hanrahan owns 18 percent, while co-founders Christopher Stolte and Christian Chabot, who is also chief executive officer, each own more than 15 percent. Meritech Capital Partners controls 6.4 percent. Tableau recognized three categories of Primary Competitors:

  • large suppliers of traditional business intelligence products, like IBM, Microsoft, Oracle and SAP AG;

  • spreadsheet software providers, such as Microsoft Corporation

  • business analytics software companies: Qlik Technologies Inc. and TIBCO Spotfire.


Update 4/29/13: This news maybe related to Tableau IPO: I understand that Microstrategy’s growth cannot be compared with growth of Tableau or even Qliktech. But to go below of the average “BI market” growth? Or even 6% or 24% decrease? What is going on (?) here : “First quarter 2013 revenues were $130.2 million versus $138.3 million for the first quarter of 2012, a 6% decrease.  Product licenses revenues for the first quarter of 2013 were $28.4 million versus $37.5 million for the first quarter of 2012, a 24% decrease.”

Update 5/6/13: Tableau Software Inc. will sell 5 million shares, while shareholders will sell 2.2 million shares, Tableau said in an amended filing with the U.S. Securities and Exchange Commission. The underwriters have the option to purchase up to an additional 1,080,000 shares. It means total can be 8+ millions of shares for sale.

The company expects its initial public offer to raise up to $215.3 million at a price of $23 to $26 per share. If this happened, that will create public company with large capitalization, so Qliktech and Spotfire will have an additional problem to worry about. This is how QLIK (blue line), TIBX (red) and MSTR (orange line) stock behaved during last 6 weeks after release of Tableau 8 and official Tableau IPO announcement:


Update 5/16/13: According to this article  at Seeking Alpha (also see S-1 Form) Tableau Software Inc. (symbol “DATA”) is scheduled a $176 million IPO with a market capitalization of $1.4 billion for Friday, May 17, 2013. Tableau’s March Quarter sales were up 60% from the March ’12 quarter. Qliktech’s sales were up only 23% on a similar comparative basis.


According to other article, Tableau raised it IPO price and it may reach capitalization of $2B by end of Friday, 5/17/13. That is almost comparable with capitalization of Qliktech…

Update 5/17/13: Tableau’s IPO offer price was $31 per share, but it started today

with price $47 and finished day with $50.75 (raising $400M in one day) with estimated Market Cap around $3B (or more?). It is hard to understand the market: Tableau Stock (symbol: DATA) finished its first day above $50 with Market Capitalization higher than QLIK, which today has Cap = $2.7B but Qliktech has almost 3 times more of sales then Tableau!

For comparison MSTR today has Cap = $1.08B and TIBX today has Cap = $3.59B. While I like Tableau, today proved that most investors are crazy, if you compare numbers in this simple table:

Symbol  : Market Cap, $B, as of 5/17/13 Revenue, $M, as of 3/31/13 (trailing 12 months) FTE (Full Time Employees)
TIBX 3.59 1040 3646
MSTR 1.08 586 3172
QLIK 2.67 406 1425
DATA between $2B and $3B? 143 834

See interview with Co-Founder of Tableau Software Christian Chabot  – he discusses taking the company’s IPO with Emily Chang on Bloomberg Television’s “Bloomberg West.” However it makes me sad when Tableau’s CEO is implying that Tableau is ready for big data, which is not true.

TableauCEOaboutIPOHere are some pictures of the Tableau team at the NYSE: and here is the announcement about “closing IPO”.

Initial public offering gave to Tableau $254 million (preliminary estimate)

Today Tableau 8 was released with 90+ new features (actually it may be more than 130) after exhausting 6+ months of Alpha and Beta Testing with 3900+ customers as Beta Testers! I personally expected it it 2 months ago, but I rather will have it with less bugs and this is why I have no problem with delay. During this “delay” Tableau Public achieved the phenomenal Milestone: 100 millions of users…

Tableau 8 introduced:

  • web and mobile authoring,
  • added access to new data sources: Google Analytics,, Cloudera Impala, DataStax Enterprise, Hadapt, Hortonworks Hadoop Hive, SAP HANA, and Amazon Redshift.
  • New Data Extract API that allows programmers to load data from anywhere into Tableau and make certain parts of Tableau Licensing ridiculous, because consuming part of licensing (for example core licensing) for background tasks should be set free now.
  • New JavaScript API enables integration with business (and other web-) applications.
  • Local Rendering: leveraging the graphics hardware acceleration available on ordinary computers. Tableau 8 Server dynamically determines where rendering will complete faster – on the server or in the browser. Also – and acts accordingly. Also Dashboards now render views in parallel when possible.

Tableau Software plans to add in next versions (after 8.0) some very interesting and competitive features, like:

  • Direct query of large databases, quick and easy ETL and data integration.
  • Tableau on a Mac and Tableau as a pure Cloud offering.
  • Make statistical & analytical techniques accessible (I wonder if it means integration with R?).
  • Tableau founder Pat Hanrahan recently talked about “Showing is Not Explaining”, so Tableau planned to add features that support storytelling by constructing visual narratives and effective communication of ideas.

I did not see on Tableau’s roadmap some very long overdue features like 64-bit implementation (currently even all Tableau Server processes, except one, are 32-bit!), Server implementation on Linux (we do not want to pay Windows 2012 Server CAL taxes to Bill Gates) and direct mentioning of integration with R like Spotfire does – I how those planning and strategic mistakes will not impact upcoming IPO.

I personally think that Tableau has to stop using its ridiculous practice when 1 core license used per each 1 Backgrounder server process and since Tableau Data Extract API is free so all Tableau Backgrounder Processes should be free and have to be able to run on any hardware and even any OS.

Tableau 8 managed to get the negative feedback from famous Stephen Few, who questioned Tableau’s ability to stay on course. His unusually long blog-post “Tableau Veers from the Path” attracted enormous amount of comments from all kind of Tableau experts. I will be cynical here and notice that there is no such thing as negative publicity and more publicity is better for upcoming Tableau IPO.


The most popular (among business users) approach to visualization is to use a Data Visualization (DV) tool like Tableau (or Qlikview or Spotfire), where a lot of features already implemented for you. Recent prove of this amazing popularity is that at least 100 million people (as of February 2013),  used Tableau Public as their Data Visualization tool of choice, see

However, to make your documents and stories (and not just your data visualization applications) driven by your data, you may need the other approach – to code visualization of your data into your story and visualization libraries like  popular D3 toolkit can help you. D3 stands for “Data-Driven Documents”. The Author of D3 Mr. Mike Bostock designs interactive graphics for New York Times – one of latest samples is here:

and NYT allows him to do a lot of Open Source work which he demonstartes at his website here: .


Mike was a “visualization scientist” and a computer science PhD student at #Stanford University and member of famous group of people, now called “Stanford Visualization Group”:

This Visualization Group was a birthplace of Tableau’s prototype – sometimes they called it  “a Visual Interface” for exploring data and other name for it is Polaris:

and we know that creators of Polaris started Tableau Software. One of other Group’s popular “products” was a graphical toolkit (mostly in JavaScript, as oppose to Polaris, written in C++) for Visualization, called ProtoVis:

– and Mike Bostock was one of ProtoViz’s main co-authors. Less then 2 years ago Visualization Group suddenly stopped developing ProtoViz and recommended to everybody to switch to D3 library,

authored by Mike. This library is Open Source (only 100KB in ZIP format) and can be downloaded from here:


In order to use D3, you need to be comfortable with HTML, CSS, SVG, Javascript programming, DOM (and other Web Standards); understanding of jQuery paradigm will be useful too. Basically if you want to be at least partially as good as Mike Bostock, you need to have a mindset of a programmer (I guess in addition to business user mindset), like this D3 expert:

Most of successful early D3 adopters combining even 3+ mindsets: programmer, business analyst, data artist and even sometimes data storyteller. For your programmer’s mindset you may be interested to know that D3 has a large set of Plugins, see:

and rich #API, see

You can find hundreds of D3 demos, samples, examples, tools, products and even a few companies using D3 here:


Best of the Tableau Web… December 2012:
Top 100 Q4 2012 from Tableau Public:
eBay’s usage of Tableau as the front-end for big data, Teradata and Hadoop with 52 petabytes of
data on everything from user behavior to online transactions to customer shipments and much more:
Why The Information Lab recommends Tableau Software:
Fun with #Tableau Treemap Visualizations
Talk slides: Tableau, SeaVis meetup & Facebook, Andy Kirk’s Facebook Talk from Andy Kirk
Usage of RAM, Disk and Data Extracts with Tableau Data Engine:
Migrating Tableau Server to a New Domain
SAS/Tableau Integration
IFNULL – is not “IF NULL”, is “IF NOT NULL”
Worksheet and Dashboard Menu Improvements in Tableau 8:
Jittery Charts – Why They Dance and How to Stop Them:
Tableau Forums Digest #8
Tableau Forums Digest #9
Tableau Forums Digest #10
Tableau Forums Digest #11
implementation of bandlines in Tableau by Jim Wahl (+ Workbook):

This is the Part 2 of the guest blog post: the Review of Visual Discovery products from Advizor Solutions, Inc., written by my guest blogger Mr. Srini Bezwada (his profile is here: ), who is the Director of Smart Analytics, a Sydney based professional BI consulting firm that specializes in Data Visualization solutions. Opinions below belong to Mr. Srini Bezwada.

ADVIZOR Technology

ADVIZOR’s Visual Discovery™ software is built upon strong data visualization tools technology spun out of a distinguished research heritage at Bell Labs that spans nearly two decades and produced over 20 patents. Formed in 2003, ADVIZOR has succeeded in combining its world-leading data visualization and in-memory-data-management expertise with extensive usability knowledge and cutting-edge predictive analytics to produce an easy to use, point and click product suite for business analysis.

ADVIZOR readily adapts to business needs without programming and without implementing a new BI platform, leverages existing databases and warehouses, and does not force customers to build a difficult, time consuming, and resource intensive custom application. Time to deployment is fast, and value is high.

With ADVIZOR data is loaded into a “Data Pool” in main memory on a desktop or laptop computer, or server. This enables sub-second response time on any query against any attribute in any table, and instantaneously update all visualizations. Multiple tables of data are easily imported from a variety of sources.

With ADVIZOR, there is no need to pre-configure data. ADVIZOR accesses data “as is” from various data sources, and links and joins the necessary tables within the software application itself. In addition, ADVIZOR includes an Expression Builder that can perform a variety of numeric, string, and logical calculations as well as parse dates and roll-up tables – all in-memory. In essence, ADVIZOR acts like a data warehouse, without the complexity, time, or expense required to implement a data warehouse! If a data warehouse already exists, ADVIZOR will provide the front-end interface to leverage the investment and turn data into insight.
Data in the memory pool can be refreshed from the core databases / data sources “on demand”, or at specific time intervals, or by an event trigger. In most production deployments data is refreshed daily from the source systems.

Data Visualization

ADVIZOR’s Visual Discovery™ is a full visual query and analysis system that combines the excitement of presentation graphics – used to see patterns and trends and identify anomalies in order to understand “what” is happening – with the ability to probe, drill-down, filter, and manipulate the displayed data in order to answer the “why” questions. Conventional BI approaches (pre-dating the era of interactive Data Visualization) to making sense of data have involved manipulating text displays such as cross tabs, running complex statistical packages, and assembling the results into reports.

ADVIZOR’s Visual Discovery™ making the text and graphics interactive. Not only can the user gain insight from the visual representation of the data, but now additional insight can be obtained by interacting with the data in any of ADVIZOR’s fifteen (15) interactive charts, using color, selection, filtering, focus, viewpoint (panning, zooming), labeling, highlighting, drill-down, re-ordering, and aggregation.

Visual Discovery empowers the user to leverage his or her own knowledge and intuition to search for patterns, identify outliers, pose questions and find answers, all at the click of a mouse.

Flight Recorder – Track, Save, Replay your Analysis Steps

The Flight Recorder tracks each step in a selection and analysis process. It provides a record of those steps, and be used to repeat previous actions. This is critical for providing context to what and end-user has done and where they are in their data. Flight records also allow setting bookmarks, and can be saved and shared with other ADVIZOR users.
The Flight Recorder is unique to ADVIZOR. It provides:
• A record of what a user has done. Actions taken and selections from charts are listed. Small images of charts that have been used for selection show the selections that were made.
• A place to collect observations by adding notes and capturing images of other charts that illustrate observations.
• A tool that can repeat previous actions, in the same session on the same data or in a later session with updated data.
• The ability to save and name bookmarks, and share them with other users.

Predictive Analytics Capability

The ADVIZOR Analyst/X is a predictive analytic solution based on a robust multivariate regression algorithm developed by KXEN – a leading-edge advanced data mining tool that models data easily and rapidly while maintaining relevant and readily interpretable results.
Visualization empowers the analyst to discover patterns and anomalies in data by noticing unexpected relationships or by actively searching. Predictive analytics (sometimes called “data mining”) provides a powerful adjunct to this: algorithms are used to find relationships in data, and these relationships can be used with new data to “score” or “predict” results.


Predictive analytics software from ADVIZOR don’t require enterprises to purchase platforms. And, since all the data is in-memory, the Business Analyst can quickly and easily condition data and flag fields across multiple tables without having to go back to IT or a DBA to prep database tables. The interface is entirely point-and-click, there are no scripts to write. The biggest benefit from the multi-dimensional visual solution is how quickly it delivers analysis, solving critical business questions, facilitating intelligence-driven decision making, providing instant answers to “what if?” questions.

Advantages over Competitors:

• The only product in the market offering a combination of Predictive Analytics + Data Visualisation + In memory data management within one Application.
• The cost of entry is lower than the market leading data visualization vendors for desktop and server deployments.
• Advanced Visualizations like Parabox, Network Constellation in addition to normal bar charts, scatter plots, line charts, Pie charts…
• Integration with leading CRM vendors like, Blackbaud, Ellucian, Information Builder
• Ability to provide sub-second response time on query against any attribute in any table, and instantaneously update all visualizations.
• Flight recorder that lets you track, replay, and save your analysis steps for reuse by yourself or others.

Update on 5/1/13 (by Andrei): Avizor 6.0 is available now with substantial enhancements:

If you visited my blog before, you know that my classification of Data Visualization and BI vendors are different from researchers like Gartner. In addition to 3 DV Leaders – Qlikview, Tableau, Spotfire – I rarely have time to talk about other “me too” vendors.

However, sometimes products like Omniscope, Microstrategy’s Visual Insight, Microsoft BI Stack (Power View, PowerPivot, Excel 2013, SQL Server 2012, SSAS etc.), Advizor, SpreadshetWEB etc. deserve attention too. However, it takes so much time, so I am trying to find guest bloggers to cover topics like that. 7 months ago I invited volunteers to do some guest blogging about Advizor Visual Discovery Products:

So far nobody in  USA or Europe committed to do so, but recently Mr. Srini Bezwada, Certified Tableau Consultant and Advizor-trained expert from Australia contacted me and submitted the article about it.  He also provided me with info about how Advizor can be compared with Tableau, so I will do it briefly, using his data and opinions. Mr. Bezwada can be reached at , where he is a director at

Below is quick comparison of Advizor with Tableau. Opinions below belong to Mr. Srini Bezwada. Next blog post will be a continuation of this article about Advizor Solutions Products, see also Advizor’s website here:

Criteria Tableau ADVIZOR Comment
Time to implement Very Fast Fast, ADVIZOR can be implemented within Days Tableau Leads
Scalability Very Good Very Good Tableau: virtual RAM
Desktop License $1,999 $ 1,999 $3,999 for AnalystX with Predictive modeling
Server License/user $1K, min 10 users, 299 K for Enterprise Deployment license for up to 10 named users $8 K ADVIZOR is a lot cheaper for Enterprise Deployment $75 K for 500 Users
Support fees / year



1st year included
SaaS Platform Core or Digital Offers Managed Hosting ADVIZOR Leads
Overall Cost Above Average Competitive ADVIZOR Costs Less
Enterprise Ready Good for SMB Cheaper cost model for SMB Tableau is expensive for Enterprise Deployment
Long-term viability Fastest growth Private company since 2003. Tableau is going IPO in 2013
Mindshare Tableau Public Growing Fast Tableau stands out
Big Data Support Good Good Tableau is 32-bit
Partner Network Good Limited Partnerships Tableau Leads
Data Interactivity Excellent Excellent
Visual Drilldown Very Good Very Good
Offline Viewer Free Reader None Tableau stands out
Analyst’s Desktop Tableau Professional Advizor has Predictive Modeling ADVIZOR is a Value for Money
Dashboard Support Excellent Very Good Tableau Leads
Web Client Very Good Good Tableau Leads
64-bit Desktop None Very Good Tableau still a 32-bit app
Mobile Clients Very Good Very Good
Visual Controls Very Good Very Good
Data Integration Excellent Very Good Tableau Leads
Development Tableau Pro ADVIZOR Analyst
64-bit in-RAM DB Good Excellent Advizor Leads
Mapping support Excellent Average Tableau stands out
Modeling, Analytics Below Average Advanced Predictive Modelling ADVIZOR stands out
Predictive Modeling None Advanced Predictive Modeling Capability with Built in KXEN algorithms ADVIZOR stands out
Flight Recorder None Flight recorder lets you track, replay, save your analysis steps for reuse by yourself or others. ADVIZOR stands out
Visualization 22 Chart types All common charts like  bar charts, scatter plots, line charts, Pie charts are supported Advizor has Advanced Visualizations like Parabox, Network Constellation
Third party integration Many Data Connectors, see Tableau’s drivers page ADVIZOR integrates well with CRM software:, Ellucian, Blackbaud and others. ADVIZOR leads in CRM area
Training Free Online and paid Classroom Free Online and paid via company trainers & Partners Tableau Leads

In my previous post (this post is the continuation of it) , I said that Tableau 8 introduced 130+ new features, 3 times more then Tableau 7 did. Many of these new features are in Tableau 8 Server and this post about those new Server features (this is a repost from my Tableau blog: ).

The Admin and Server pages have been redesigned to show more info quicker. In list view the columns can be resized. In thumbnail view the grid dynamically resizes. You can hover over a thumbnail to see more info about visualization. The content search is better too:


Web authoring (even mobile) introduced by Tableau 8 Server. Change dimensions, measures, mark types, add filters, and use Show Me are all directly in a web browser and can be saved back to the server as a  new workbook or if individual permissions allow, to the original workbook:


Subscribing to a workbook or worksheet will automatically notify about the dashboard or view updates to your email inbox. Subscriptions deliver image and link.

Tableau 8 Data Engine is more scalable now, it can be distributed between 2 nodes, 2nd instance of it now can be configured as Active, Synced and Available for reading if  Tableau Router decided to use it (in addition Fail-over function as before)server2sTableau 8 Server now supports Local Rendering, using graphic ability of local devices, modern browsers and HTML5. No-round-trip to server while rendering using latest versions of chrome 23+, Firefox 17+, Safari , IE 9+. Tableau 8 (both Server and Desktop, computing each view in Parallel. PDF files, generated by Tableau 8 up to 90% smaller and searchable. And Performance Recorder works on both Server and Desktop.

Tableau 8 Server introducing Shared sessions allows more concurrency, more caching. Tableau 7 uses 1 session per viewer. Tableau 8 using one session per many viewers, as long as they do no change state of filters and don’t do other altering interaction. If interaction happened, Tableau 8 will clone the session for appropriate Interactor and apply his/her changes to new session:server3sIFinally Tableau getting API, 1st part of it I described in previous blog post about TDesktop – TDE API (C/C++, Python, Java on both Windows AND Linux!).

For Web Development Tableau has now brand new JavaScript API to customize selection, filtering, triggers to events, custom toolbar, etc. Tableau 8 has own JavaScript API WorkBench, which can be used right from you browser:server4w

TDE API allows to build own TDE on any machine with Python, C/C++ and Java (see 24:53 at ). Additionally Server API (REST API) allows programmatically create/enable/suspend sites and add/remove users to sites.

In addition to Faster Uploads andPublishing Data Sources, users can Publish Filters as Set and User Filters. Data Sources can be Refreshed or Appended instead of republishing – all from Local Sources. Such Refreshes can scheduled using Windows Task Scheduler or other task scheduling software on client devices – this is a real TDE proliferation!

My wishlist for Tableau 8 Server: all Tableau Server processes needs to be 64-bit (and they still 32-bit, see it here: ; they are way overdue to be the 64-bit; Linux version of Tableau Server (Microsoft recently changed very unfavorably the way they charge users for each Client Access) is needed, I wish integration with R Library (Spotfire has it for years), I want Backgrounder Processes (mostly doing data extracts on server) will not consume core licenses etc…

And yes, I found in San Diego even more individuals who found the better way to spend their time compare with attending Tableau 2012 Customer Conference and I am not here to judge:


I left Tableau 2012 conference in San Diego (where Tableau 8 was announced) a while ago with enthusiasm which you can feel from this real-life picture of 11 excellent announcers:


Conference was attended by 2200+ people and 600+ Tableau Software employees (Tableau almost doubled the number of employees in a year) and it felt like a great effort toward IPO (see also article here: ).  See some video here: TCC12 Keynote . Tableau 8 introduce 130+ new features, 3 times more then Tableau 7 did. Almost half of these new features are in Tableau 8 Desktop and this post about those new Desktop features (this is a repost from my Tableau Blog: New Tableau 8 Server features deserved a separate blog post which I will publish a little later after playing with Beta 1 and may be Beta 2.

A few days after conference the Tableau 8 Beta Program started with 2000+ participants. One of the most promising features is new rendering engine and I build special Tableau 7 visualization (and its port to Tableau 8) with 42000 datapoints:  to compare the speed of rendering between versions 7 and 8:


Among new features are new (for Tableau) visualization types: Heatmap, “Packed” Bubble Chart and Word Cloud, and I build simple Tableau 8 Dashboard to test it (all 3 are visualizing the 3-dimensional set where 1 dimension used as list of items, 1 measure used for size and 2nd measure used for color of items):


List of new features includes improved Sets (comparing members vs. non-members, adding/removing members, combining Sets: all-in-both, shared-by-both, left-except-right, right-except-left), Custom SQL with parameters, Freeform Dashboards (I still prefer MDI UI where each Chart/View Sheet has own Child Window as oppose to Pane), ability to add multiple fields to Labels, optimized label placement, built-in statistical models for visual Forecasting, Visual Grouping based on your data selection, Redesigned Mark Card (for Color, Size, Label, Detail and Tooltip Shelves).

New Data features include data blending without mandatory linked field in a view and with ability to filter data in secondary data sources; refreshing server-based Data Extracts can be done from local data sources; Data Filters (in addition be either local or global) can be shared now among selected set of worksheets and dashboards. Refresh of Data Extract can be done using command prompt for Tableau Desktop, for example

>tableau.exe refreshremoteextract

Tableau 8 has (finally) API (C/C++, Python, Java) to directly create a Tableau Data Extract (TDE) file, see example here:

Tableau 8 (both Desktop and Server) can then connect to this extract file natively! Tableau provides new native connection for Google Analytics and TDE files now much smaller (especially with text values) – up to 40% smaller compare with Tableau 7.

Tableau 8 has performance enhancements, such as the new ability to use hardware accelerators (of modern graphics cards), computing views within dashboard in parallel (in Tableau 7 it was consecutive computations) and new  performance recorder allows to estimate and tune a workload of various activities and functions and optimize the behavior of workbook.

I still have a wishlist of features which are not implemented in Tableau and I hope some them will be implemented later: all Tableau processes are 32-bit (except 64-bit version of data engine for server running on 64-bit OS) and they are way overdue to be the 64-bit; many users demand MAC version of Tableau Desktop and Linux version of Tableau Server (Microsoft recently changed very unfavorably the way they charge users for each Client Access), I wish MDI UI for Dashboards where each view of each worksheet has own Window as oppose to own pane (Qlikview does it from the beginning of the time), I wish integration with R Library (Spotfire has it for years), scripting languages and IDE (preferably Visual Studio), I want Backgrounder Processes (mostly doing data extracts on server) will not consume core licenses etc…

Despite the great success of the conference, I found somebody in San Diego who did not pay attention to it (outside was 88F, sunny and beautiful):


On May 3rd of 2012 the Google+ extension of this Data Visualization blog reached 500+ followers, on July 9 it got 1000+ users, on October 11 it had already 2000+ users, 11/27/12 my G+ Data Visualization Page has 2190+ followers and still growing every day (updated as of 12/01/12: 2500+ followers.

One of reasons of course is just a popularity of Data Visualization related topics and other reason covered in interesting article here: .

In any case, it helped me to create a reading list for myself and other people, base on feedback I got. According to CicleCount, as of 11/13/12 update, my Data Visualization Google+ Page ranked as #178 most popular page in USA. Thank you G+ ! Updates:

5/25/13: G+ extension of this blog now has 3873+ followers,
and as of  7/15/13 as of 4277+ followers),
and as of 11/11/13 it has 5013+ followers:


I also have 2nd G+ extension of this blog, see it here: with 375 followers as of 11/11/13:



Qlikview 10 was released around 10/10/10, Qlikview 11 – around 11/11/11, so I expected Qlikview 12 to be released on 12/12/12 but “instead” we are getting Qlikview 11.2 with Direct Discovery in December 2012, which supposedly provides a “hybrid approach so business users can get the QlikView associative experience even with data that is not stored in memory”

This feature demanded by users (me included) for a long time, but I think noise around so called Big Data and competition forced Qliktech to do it. Spotfire has it for a longtime (as well as 64-bit implementation) and Tableau has something like that for a while (unfortunately Tableau still 32-bit) . You can test Beta of it, if you have time:

Just 8 months ago Qliktech estimated its sales for 2012 as $410M and suddenly 3 months ago it changed its estimates down to $381M, just 19% over 2011, which is in huge contrast with Qliktech’s previous speed of growth and way behind the current speed of growth of Tableau and even less then current speed of growth of Spotfire. During last 2 years QLIK stock unable to grow significantly:

and all of the above forcing Qliktech to do something outside of gradual improvements – new and exciting functionality needed and Direct Discovery may help!

QlikView Direct Discovery enables users to perform visual analysis against “any amount of data, regardless of size”. With the introduction of this unique hybrid approach, users can associate data stored within big data sources directly alongside additional data sources stored within the QlikView in-memory model. QlikView can “seamlessly connect to multiple data sources together within the same interface”, e.g. Teradata to SAP to Facebook allowing the business user to associate data across the data silos. Data outside of RAM can be joined with the in-memory data with the common field names. This allows the user associatively navigate both on the direct discovery and in memory data sets.

QlikView developer should setup the Direct Discovery table on the QlikView application load script to allow the business users to query the desired big data source. Within the script editor a new syntax is introduced to connect to data in direct discovery form. Traditionally the following syntax is required to load data from a database table:

To invoke the direct discovery method, the keyword “SQL” is replaced with “DIRECT”.

In the example above only column CarrierTrackingNumber and ProductID are loaded into QlikView in the traditional manner, other columns exist in the data table within the Database including columns OrderQty and Price. OrderQty and Price fields are referred as “IMPLICIT” fields. An implicit field is a field that QlikView is aware of on a “meta level”. The actual data of an implicit field resides only in the database but the field may be used in QlikView expressions. Looking at the table view and data model of the direct discovery columns are not within the model (on the OrderFact table):

Once the direct discovery structure is established, the direct discovery data can be joined with the in-memory data with the common field names (Figure 3). In this example, “ProductDescription” table is loaded in-memory and joined to direct discovery data with the ProductID field. This allows the user to associatively navigate both on the “direct discovery” and in memory data sets.

Direct Discovery will be much slow then in-memory processing and this is is expected, but it will take away from Qlikview its usual claim that is is faster then competitors. QlikView Direct Discovery can only be used against SQL compliant data sources. The following data sources are supported;

• ODBC/OLEDB data sources – All ODBC/OLEDB sources are supported, including SQL Server, Teradata and Oracle.
• Custom connectors which support SQL –, SAP SQL Connector, Custom QVX connectors for SQL compliant data stores.

Due to the interactive and SQL syntax specific nature of the Direct Discovery approaches a number of limitations exist. The following chart types are not supported;
• Pivot tables
• Mini charts
And the following QlikView features are not supported:
• Advanced aggregation
• Calculated dimensions
• Comparative Analysis (Alternate State) on the QlikView objects that use Direct
Discovery fields
• Direct Discovery fields are not supported on Global Search
• Binary load from a QlikView application with Direct Discovery table

Here is a some preliminary video about Direct Discovery, published by Qliktech:

It was interesting to me that just 2 days after Qliktech pre-anounced Direct Discovery it also partners with Teradata. Tableau partners with Teradata for a while and Spotfire did it a month ago, so I guess Qliktech trying to catchup in this regard as well. I mentioned it only to underscore the point of this blog post: Qliktech realized that it behind its competitors in some areas and it has to follow ASAP.

Today TIBCO announced Spotfire 5, which will be released in November 2012. Two biggest news are the access to SQL Server Analysis Services cubes and the integration with Teradata “by pushing all aggregations, filtering and complex calculations used for interactive visualization into the (Teradata) database”.

Spotfire team “rewrote” its in-memory engine for v. 5.0 to take advantage of high-capacity, multi-core servers. “Spotfire 5 is capable of handling in-memory data volumes orders of magnitude greater than the previous version of the Spotfire analytics platform” said Lars Bauerle, vice president of product strategy at TIBCO Spotfire.

Another addition is “in-database analysis” which allows to apply analytics within the database platforms (such as Oracle, Microsoft SQL Server and Teradata) without  extracting and moving data, while handling analyses on Spotfire server and returning result sets back to the database platform.

Spotfire added new Tibco Enterprise Runtime for R, which embeds R runtime engine into the Spotfire statistical server. TIBCO claims that Spotfire 5.0 scales to tens of thousands of users! Spotfire 5 is designed to leverage the full family of TIBCO business optimization and big data solutions, including TIBCO LogLogic®, TIBCO Silver Fabric, TIBCO Silver® Mobile, TIBCOBusinessEvents®, tibbr® and TIBCO ActiveSpaces®.

The Mass Technology Leadership Council (MassTLC) organized today the Data Visualization Panel in their series of “Big Data Seminars”:

and they invited me to be a Speaker and Panelist together with Irene Greif (Fellow @IBM) and Martin Leach (CIO @Broad Institute). Most interesting about this event was that it was sold out and about 150 people came to participate, even it was most productive time of the day (from 8:30am until 10:30am). Compare with what I observed just a few years ago, I sensed the huge interest to Data Visualization, base on multiple, very interesting and relevant questions I got from event participants.

I doubt that Microsoft is paying attention to my blog, but recently they declared that Power View now has 2 versions: one  for SharePoint (thanks, but no thanks) and one for Excel 2013. In other words, Microsoft decided to have own Desktop Visualization tool. In combination with PowerPivot and SQL Server 2012 it can be attractive for some Microsoft-oriented users but I doubt it can compete with Data Visualization Leaders – too late.

Most interesting is the note about Power View 2013 on Microsoft site: “Power View reports in SharePoint are RDLX files. In Excel, Power View sheets are part of an Excel XLSX workbook. You can’t open a Power View RDLX file in Excel, and vice versa. You also can’t copy charts or other visualizations from the RDLX file into the Excel workbook.

But most amazing is that Microsoft decided to use the dead Silverlight for Powerview: “Both versions of Power View need Silverlight installed on the machine.” And we know that Microsoft switched to HTML5 from Silverlight and no new development planned for Silverlight! Good luck with that…

And yes, you can add now maps (Bing of course), see it here:

(this is a repost from my other blog: )

Big Data can be useless without multi-layer data aggregations, hierarchical or cube-like intermediary Data Structures, when ONLY a few dozens, hundreds or thousands data-points exposed visually and dynamically every single viewing moment to analytical eyes for interactive drill-down-or-up hunting for business value(s) and actionable datum (or “datums” – if plural means data). One of best expression of this concept (at least how I interpreted it) I heard from my new colleague who flatly said:

“Move the function to the data!”

I got recently involved with multiple projects using large data-sets for Tableau-based Data Visualizations (100+ millions of rows and even Billions of records!). Some of largest examples of their sizes I used were: 800+ millions of records and other was 2+ billions of rows.

So this blog post is to express my thoughts about such Big Data (in average examples above have about 1+ KB per CSV record before compression and other advanced DB tricks, like columnar Databases used by Data Engine of Tableau) as back-end for Tableau. But please keep in mind that as a 32-bit tool, Tableau itself is not ready for Big Data. In addition, I think Big Data is mostly a buzzword and BS and we are  forced by marketing BS masters to use sometimes this stupid term.

Here are some Factors involved into Data Delivery from main and designated Database (Back-ends like Teradata, DB2, SQL Server or Oracle) for Tableau-based Big Data Visualizations) into “local” Tableau Visualizations (many people still trying to use Tableau as a Reporting tool as oppose to (Visual) Analytical Tool:

  • Queuing thousands of Queries to Database Server. There is no guarantee your Tableau query will be executed immediately; in fact it WILL be delayed.

  • Speed of Tableau Query when it will start to be executed depends on sharing CPU cycles, RAM and other resources with other queries executed SIMULTANEOSLY with your query.

  • Buffers, pools and other resources available for particular user(s) and queries at your Database Server are different and depends on privileges and settings given to you as a Database User

  • Network speed: between some servers it can be 10Gbits (or even more), in most cases it is 1Gbit inside server rooms, outside of server rooms I observed in many old buildings (over wired Ethernet) max 100Mbits coming into user’s PC; in case if you using Wi-Fi it can be even less (say 54 Mbits?). If you are using internet it can be even less (I observed speed in some remote offices as 1 Mbit or so over old T-1 lines); if you using VPN it will max out at 4Mbits or less (I observed it in my home office).

  • Utilization of network. I use Remote Desktop Protocol – RDP to VM (from my workstation or notebook; (VM or VDI Virtual Machine, sitting in server room) and connected to servers with network speed of 1Gbit, but it still using maximum 3% of network speed (about 30 MBits, which is about 3 Megabytes of data per second, which is probably about few thousands of records per seconds.

That means that network may have a problem to deliver 100 millions of records to “local” report overnight (say 10 hours, 10 millions of records per hour, 3000 records per second) – partially and probably because of factors 4 above.

On top of those factors please keep in mind that Tableau is a set of 32-bit applications (with exception of one out of 7 processes on Server side), which is restricted to 2GB of RAM; if data-set cannot fit into RAM, than Tableau Data Engine will use the disk as Virtual RAM, which is much, much slower and for some users such disk space actually not local to his/her workstation and mapped to some “remote” network file server.

Tableau desktop is using in many cases 32-bit ODBC drivers, which may even add more delay into data delivery into local “Visual Report”. As we learned from Tableau support itself, even with latest Tableau Server 7.0.X, the RAM allocated for one user session restricted to 3GB anyway.

Unfortunate Update: Tableau 8.0 will be 32-bit application again, but may be follow up version 8.x or 9 (I hope) will be ported to 64-bits… It means that Spotfire, Qlikview and even PowerPivot will keep some advantages over Tableau for a while…

(this is a repost from my other Data Visualization blog: )

Often I used small Tableau (or Spotfire or Qlikview) workbooks instead of PowerPoint, which are proving at least 2 concepts:

  • Good Data Visualization tool can be used as the Web or Desktop Container for Multiple Data Visualizations (it can be used to build a hierarchical Container Structures with more then 3 levels; currently 3: Container-Workbooks-Views)

  • It can be used as the replacement for PowerPoint; in example below I embedded into this Container 2 Tableau Workbooks, one Google-based Data Visualization, 3 image-based Slides and Textual Slide:

  • Tableau (or Spotfire or Qlikview) is better then PowerPoint for Presentations and Slides

  • Tableau (or Spotfire or Qlikview) is the Desktop and the Web Container for Web Pages, Slides, Images, Texts

  • Good Visualization Tool can be a Container for other Data Visualizations

  • Sample Tableau Presentation above contains the Introductory Textual Slide

  • Sample Tableau Presentation above  contains a few Tableau Visualization:This Tableau Presentation contains a Web Page with the Google-based Motion Chart Demo

    1. The Drill-down Demo

    2. The Motion Chart Demo ( 6 dimensions: X,Y, Shape, Color, Size, Motion in Time)

  • This Tableau Presentation contains a few Image-based Slides:

    1. The Quick Description of Origins and Evolution of Software and Tools used for Data Visualizations during last 30+ years

    2. The Description of Multi-level Projection from Multidimensional Data Cloud to Datasets, Multidimensional Cubes and to Chart

    3. The Description of 6 stages of Software Development Life Cycle for Data Visualizations

Next Page »