DV News


Data Visualization Readings, Q1 2014, selected from Google+ extensions of this blog:
http://tinyurl.com/VisibleData and
http://tinyurl.com/VisualizationWithTableau

dvi032914

Data Visualization Index (using DATA+QLIK+TIBX+MSTR; click on image above to enlarge):
Since 11/1/13 until 3/15/14: DATA stock grew 50%. QLIK 11%, MSTR – 6%, TIBX – lost 1%.
Current Market Capitalization: Tableau – $5.5B, QLIK – $2.6B, TIBCO – 3.5B, Microstrategy – $1.4B
Number of Job Openings Today: Tableau – 231, QLIK – 135, Spotfire (estimate) – 30, Microstrategy – 214
However during last 2 weeks of March of 2014 DATA shares lost 24%, QLIK lost 14%, TIBX and MSTR both lost about 10%

Why use R? Five reasons.
http://www.econometricsbysimulation.com/2014/03/why-use-r-five-reasons.html

Studying Tableau Performance Characteristics on AWS EC2
http://tableaulove.tumblr.com/post/80571148718/studying-tableau-performance-characteristics-on-aws-ec2

Head-to-head comparison of Datawatch and Tableau
http://datawatch.com/datawatch-vs-tableau

Diving into TIBCO Spotfire Professional 6.0
http://www.jenunderwood.com/2014/03/25/diving-into-tibco-spotfire-professional-6-0/

TIBCO beats Q1 2014 estimates but Spotfire falters
http://diginomica.com/2014/03/20/tibco-beats-estimates-spotfire-falters/

Qlik Doesn’t Fear Tableau, Oracle In Data Analytics
http://news.investors.com/031314-693154-qlik-focuses-on-easy-to-use-data-analytics.htm?p=full

Best of the visualisation web… February 2014
http://www.visualisingdata.com/index.php/2014/04/best-of-the-visualisation-web-february-2014/

Datawatch: ‘Twenty Feet From Stardom’
http://seekingalpha.com/article/2101513-datawatch-twenty-feet-from-stardom

Tableau plans to raise $345M — more than its IPO — with new stock offering
http://venturebeat.com/2014/03/16/tableau-plans-to-raise-345m-more-than-its-ipo-with-new-stock-offering/

TIBCO Spotfire Expands Connectivity to Key Big Data Sources
http://www.marketwatch.com/story/tibco-expands-connectivity-to-key-big-data-sources-2014-03-11

Tableau and Splunk Announce Strategic Technology Alliance
http://www.splunk.com/view/SP-CAAAKH5?awesm=splk.it_hQ

The End of The Data Scientist!?
http://alpinenow.com/blog/the-end-of-the-data-scientist/

bigData

Data Science Is Dead
http://slashdot.org/topic/bi/data-science-is-dead/

Periodic Table of Elements in TIBCO Spotfire
http://insideinformatics.cambridgesoft.com/InteractiveDemos/LaunchDemo/?InteractiveDemoID=1

Best of the visualisation web… January 2014
http://www.visualisingdata.com/index.php/2014/03/best-of-the-visualisation-web-january-2014/

Workbook Tools for Tableau
http://powertoolsfortableau.com/tableau-workbooks/workbook-tools/

Tapestry Data Storytelling Conference
http://www.tapestryconference.com/attendees
http://www.visualisingdata.com/index.php/2014/03/a-short-reflection-about-tapestry-conference/ ReadingLogo

URL Parameters in Tableau
http://interworks.co.uk/business-intelligence/url-parameters-tableau/

Magic Quadrant 2014 for Business Intelligence and Analytics Platforms
http://www.gartner.com/technology/reprints.do?id=1-1QLGACN&ct=140210&st=sb

What’s Next in Big Data: Visualization That Works the Way the Eyes and Mind Work
http://insights.wired.com/profiles/blogs/what-s-next-in-big-data-visualization-that-works-the-way-the-eyes#axzz2wPWAYEuY

What animated movies can teach you about data analysis
http://www.cio.com.au/article/539220/whatanimatedmoviescanteachaboutdata_analysis/

Tableau for Mac is coming, finally
http://www.geekwire.com/2014/tableau-mac-coming-finally/

Authenticating an External Tableau Server using SAML & AD FS
http://www.theinformationlab.co.uk/2014/02/04/authenticating-external-tableau-server-using-internal-ad/

Visualize this: Tableau nearly doubled its revenue in 2013
http://gigaom.com/2014/02/04/visualize-this-tableau-nearly-doubled-its-revenue-in-2013/

Qlik Announces Fourth Quarter and Full Year 2013 Financial Results
http://investor.qlik.com/releasedetail.cfm?ReleaseID=827231

InTheMiddleOfWinter2

Tableau Mapping – Earthquakes, 300,000,000 marks using Tableau 8.1 64-bit
http://theywalkedtogether.blogspot.com/2014/01/tableaumapping-earthquakes-300000000.html

Data Science: What’s in a Name?
http://www.linkedin.com/today/post/article/20130215205002-50510-the-data-scientific-method

Gapminder World Offline
http://www.gapminder.org/world-offline/

Advanced Map Visualisation in Tableau using Alteryx
http://www.theinformationlab.co.uk/2014/01/15/DrawingArrowsinTableau

Motion Map Chart
http://apandre.wordpress.com/2014/01/12/motion-map-chart/

One of Bill Gates’s favorite graphs redesigned
http://www.perceptualedge.com/blog/?p=1829

Authentication and Authorization in Qlikview Server
http://community.qlik.com/blogs/qlikviewdesignblog/2014/01/07/authentication-and-authorization

SlopeGraph for QlikView (D3SlopeGraph QlikView Extension)
http://www.qlikblog.at/3093/slopegraph-for-qlikview-d3slopegraph-qlikview-extension/

Revenue Model Comparison: SaaS v. One-Time-Sales
http://www.wovenware.com/blog/2013/12/revenue-model-comparison-saas-v-one-time-sales#.UyimffmwIUo

Scientific Data Has Become So Complex, We Have to Invent New Math to Deal With It
http://www.wired.com/wiredscience/2013/10/topology-data-sets/all/

Posting data to the web services from QlikView
http://community.qlik.com/docs/DOC-5530

It’s your round at the bar
http://interworks.co.uk/tableau/radial-bar-chart/

Lexical Distance Among the Languages of Europe
http://elms.wordpress.com/2008/03/04/lexical-distance-among-languages-of-europe/

SnowInsteadOfRainJan2014-SNOW

Data Visualization readings – last 4 months of 2013.

(time to read is shrinking…)

0. The Once and Future Prototyping Tool of Choice
http://tableaufriction.blogspot.com/2013/07/the-once-and-future-prototyping-tool-of.html

1. Block by Block, Brooklyn’s Past and Present
http://bklynr.com/block-by-block-brooklyns-past-and-present/

2. Data Visualization and the Blind
http://www.perceptualedge.com/blog/?p=1756

3. WHY ABRAHAM LINCOLN LOVED INFOGRAPHICS
http://www.newyorker.com/online/blogs/elements/2013/10/why-abraham-lincoln-loved-infographics.html#

4. Old Charts

5. Back To Basics
http://www.quickintelligence.co.uk/back-to-basics/

6. In-Memory Data Grid Key to TIBCO’s Strategy
http://www.datanami.com/datanami/2013-10-21/in-memory_data_grid_key_to_tibco_s_strategy.html

7. Submarine Cable Map
http://visual.ly/submarine-cable-map?view=true

8. Interview with Nate Silver:
http://blogs.hbr.org/2013/09/nate-silver-on-finding-a-mentor-teaching-yourself-statistics-and-not-settling-in-your-career/

9. Qlikview.Next will be available in 2014
http://apandre.wordpress.com/2013/09/25/qlikview-next/

10. Importance of color?
http://www.qualia.hr/why-is-color-so-important-in-data-visualization/#

11. Qlikview.Next has a gift for Tableau and Datawatch
http://apandre.wordpress.com/2013/10/24/qlik-next-has-gift/

12. (October 2013) Tableau posts 90% revenue gain and tops 1,000 staffers, files for $450 million secondary offering
http://www.geekwire.com/2013/tableau-software/#

13. The Science Of A Great Subway Map
http://www.fastcodesign.com/3020708/evidence/the-science-of-a-great-subway-map

14. SEO Data Visualization with Tableau
http://www.blastam.com/blog/index.php/2013/10/how-to-create-awesome-seo-data-visualization-with-tableau/

15. John Tukey “Badmandments”
http://www.kdnuggets.com/2013/11/john-tukey-badmandments-lessons-from-great-statistician.html#
Tukey

Supplementary BADMANDMENTS:

  • 91. NEVER plan any analysis before seeing the DATA.
  • 92. DON’T consult with a statistician until after collecting your data.
  • 94. LARGE enough samples always tell the truth

16. Thinking about proper uses of data visualization.
http://data-visualization-software.com/finally-some-clear-thinking-about-proper-uses-of-data-visualization/

17. Big BI is Stuck: Illustrated by SAP BusinessObjects Explorer
http://www.perceptualedge.com/blog/?p=727

18. IBM (trying to catch up?) bets on big data visualization
http://www.zdnet.com/ibm-bets-on-big-data-visualization-7000022741/

19. Site features draft designs and full views of the Treemap Art project (By Ben Shneiderman)
http://treemapart.wordpress.com/
http://www.cs.umd.edu/hcil/treemap-history/
http://www.cs.umd.edu/hcil/treemap/
http://treemapart.wordpress.com/full-views/
http://treemapart.wordpress.com/category/draft-designs/
img_6560

20. A Guide to the Quality of Different Visualization Venues
http://eagereyes.org/blog/2013/a-guide-to-the-quality-of-different-visualization-venues

21. Short History of (Nothing) Data Science
http://www.forbes.com/sites/gilpress/2013/05/28/a-very-short-history-of-data-science/

22. Storytelling: Hans Rosling at Global Health – beyond 2015

23. DataWatch Quarterly Review: Rapid Growth Finally Materializing
http://seekingalpha.com/article/1872591-datawatch-quarterly-review-rapid-growth-finally-materializing

24. QlikView Extension – D3 Animated Scatter Chart
http://www.qlikblog.at/2574/qlikview-extension-animated-scatter-chart/

AnimatedScatterChart-500x328

25. SlopeGraph for QlikView (D3SlopeGraph QlikView Extension)
http://www.qlikblog.at/3093/slopegraph-for-qlikview-d3slopegraph-qlikview-extension/

26. Recipe for a Pareto Analysis
http://community.qlikview.com/blogs/qlikviewdesignblog/2013/12/09/pareto-analysis

27. Color has meaning
http://www.juiceanalytics.com/design-principles/color-has-meaning/#
Meaning-in-color-e1328906744180

28. TIBCO’s Return To License Growth Frustratingly Inconsistent
http://seekingalpha.com/article/1909571-tibcos-return-to-license-growth-frustratingly-inconsistent

29. Automated Semantics and BI
http://www.forbes.com/sites/danwoods/2013/12/30/why-automated-semantics-will-solve-the-bi-dashboard-crisis/

30. What is wrong with definition of Data Science?
http://www.kdnuggets.com/2013/12/what-is-wrong-with-definition-data-science.html
mout-stats-cs-database

31. Scientific data became so complex, we have to Invent new Math to deal with it
http://www.wired.com/wiredscience/2013/10/topology-data-sets/

32. Samples

Selected Tableau Readings after TCC13 (since September 18, 2013)

sometimes reading is better then doing or writing…

0. Top 10 sessions from TCC13:
http://www.tableausoftware.com/about/blog/2013/12/top-10-sessions-tcc13-27292

1. Dual Color Axis:
https://www.interworks.com/blogs/wjones/2013/09/18/create-dual-color-axis-tableau

2. Evaluate models with fresh data using Tableau heat maps:
http://cooldata.wordpress.com/2012/07/12/evaluate-models-with-fresh-data-using-tableau-heat-maps/

3. Tableau Throws a Brick at Traditional BI:
http://www.datanami.com/datanami/2013-09-11/tableau_throws_a_brick_at_traditional_bi.html

4. Easy Empty Local Extracts:
http://www.tableausoftware.com/about/blog/2013/9/easy-empty-local-extracts-25152

5. Tableau 8.1: Sophisticated Analytics for Sophisticated People:
http://www.tableausoftware.com/about/blog/2013/9/tableau-81-sophisticated-analytics-sophisticated-people-25177

6. Tableau 8.1 and R (can be interesting for at least 5% of Tableau users):
http://www.tableausoftware.com/about/blog/2013/10/tableau-81-and-r-25327
also see:
https://www.interworks.com/blogs/trobeson/2013/11/27/using-r-tableau-81-getting-started
and here:
http://www.tableausoftware.com/about/blog/r-integration

7. Tableau, When Are You Going to Fix This?
http://www.datarevelations.com/tableau-when-are-you-going-to-fix-this.html

8. Automated PDF Email Distribution of Tableau Views Using PowerShell and Tabcmd:
http://www.interworks.com/blogs/tladd/2013/08/22/automated-pdf-email-distribution-tableau-views-using-powershell-and-tabcmd

9. Geocoding Addresses Directly in Tableau 8.1 Using Integration with R:
http://www.dataplusscience.com/Geocoding%20in%20Tableau%20using%20R.html

10. Best Practices for Designing Efficient Workbooks (and white Paper about it):
http://www.tableausoftware.com/about/blog/2013/10/best-practices-designing-efficient-workbooks-25391

11. Tableau Mapping Architecture:
http://urbanmapping.com/tableau/mapping-architecture.html

12. Story Points in Tableau 8.2 presentation mode:
http://eagereyes.org/blog/2013/story-points

13. Truly Global: Filtering Across Multiple Tableau Workbooks with the JavaScript API:
https://www.interworks.com/blogs/tladd/2013/10/24/truly-global-filtering-across-multiple-tableau-workbooks-javascript-api

14. Tableau 8.1 Worksheet, Dashboard menus improved, still room for more:
http://tableaufriction.blogspot.com/2013/10/tv81-beta-3-worksheet-dashboard-menus.html

15. Lollipops Charts in Tableau:
http://drawingwithnumbers.artisart.org/lollipops-for-quality-improvement/

16. Was Stephen Few Right?
http://www.datarevelations.com/was-stephen-few-right-my-problems-with-a-companys-iron-viz-competition.html

17. Precision Inputs Required In Addition To Analog Controls:
http://tableaufriction.blogspot.com/2013/11/precision-inputs-required-in-addition.html

18. Google Spreadsheets to Tableau connector – a working driver:
http://community.tableausoftware.com/thread/135281

19. Leveraging Color to Improve Your Data Visualization:
http://www.tableausoftware.com/public/blog/2013/10/leveraging-color-improve-your-data-visualization-2174

20. Workbook acts as a container for multiple Tableau-based Charts – 114
Samples and Visualization Types:
http://www.alansmitheepresents.org/2013/07/team-geiger-rides-again.html

21. The New Box-and-Whisker Plot:
http://www.tableausoftware.com/public/blog/2013/11/box-and-whisker-plots-2231

22. The Tableau Workbook Library:
http://www.tableausoftware.com/about/blog/2013/11/tableau-workbook-library-27004

23. Customizing Tableau Server Experience (Parts 1, 1.5, 2):
http://ugamarkj.blogspot.com/2013/11/customizing-tableau-server-experience.html
http://ugamarkj.blogspot.com/2013/12/customizing-tableau-server-experience.html
http://ugamarkj.blogspot.com/2013/12/customizing-tableau-server-experience_15.html

24. SAML Integration in Tableau 8.1:
https://www.interworks.com/blogs/daustin/2013/11/27/saml-integration-tableau-81

25. Tableau file types and extensions:
http://www.theinformationlab.co.uk/2013/12/02/tableau-file-types-and-extensions/

26. Tableau Server XML Information Files: The Master Class:
http://tableaulove.tumblr.com/post/69383091006/tableau-server-xml-information-files-the-master-class

27. Is it Transparency? Is it Opacity? Labeled one, works like the other:
http://tableaufriction.blogspot.com/2013/12/is-it-transparency-is-it-opacity.html

28. Viz Hall of Fame:
http://www.tableausoftware.com/about/blog/2013/12/viz-hall-fame-27270

29. Tableau Weekly Archive:
http://us7.campaign-archive1.com/home/?u=f3dd94f15b41de877be6b0d4b&id=d23712a896

30. 2013 Winners:
http://www.tableausoftware.com/public/blog/2013/12/2013-award-winners-2272
Happy New Year!
2014Cubes

2 months ago TIBCO (Symbol TIBX on NASDAQ) announced Spotfire 6 at TUCON 2013 user conference. This as well a follow-up release  (around 12/7/13) of Spotfire Cloud supposed to be good for TIBX prices. Instead since then TIBX lost more then 8%, while NASDAQ as whole grew more then 5%:

TIBXvsNasdaqFrom1014To121313

For example, at TUCON 2013 TIBCO’s CEO re-declared “5 primary forces for 21st century“(IMHO all 5 “drivers” sounds to me like obsolete IBM-ish Sales pitches) – I guess to underscore the relevance of TIBCO’s strategy and products to 21st century:

  1. Explosion of data (sounds like Sun rises in the East);

  2. Rise of mobility (any kid with smartphone will say the same);

  3. Emergence of Platforms (not sure if this a good pitch, at least it was not clear from TIBCO’s presentation);

  4. Emergence of Asian Economies (what else you expect? This is the side effect of the greedy offshoring for more then decade);

  5. Math trumping Science  (Mr. Ranadive and various other TUCON speakers kept repeating this mantra, showing that they think that statistics and “math” are the same thing and they do not know how valuable science can be. I personally think that recycling this pitch is dangerous for TIBCO sales and I suggest to replace this statement with something more appealing and more mature).

Somehow TUCON 2013 propaganda and introduction of new and more capable version 6 of Spotfire and Spotfire Cloud did not help TIBCO’s stock. For example In trading on Thursday, 12/12/13 the shares of TIBCO Software, Inc. (NASD: TIBX) crossed below their 200 day moving average of $22.86, changing hands as low as $22.39 per share while Market Capitalization was oscillating around $3.9B, basically the same as the capitalization of 3 times smaller (in terms of employees) competitor Tableau Software.

As I said above, just a few days before this low TIBX price, on 12/7/13, as promised on TUCON 2013, TIBCO launched Spotfire Cloud and published licensing and pricing for it.

Most disappointing news is that in reality TIBCO withdrew itself from the competition for mindshare with Tableau Public (more then 100 millions of users, more then 40000 active publishers and Visualization Authors with Tableau Public Profile), because TIBCO no longer offers free annual evaluations. In addition, new Spotfire Cloud Personal service ($300/year, 100GB storage, 1 business author seat) became less useful under new license since its Desktop Client has limited connectivity to local data and can upload only local DXP files.

The 2nd Cloud option called Spotfire Cloud Work Group ($2000/year, 250GB storage, 1 business author/1 analyst/5 consumer seats) and gives to one author almost complete TIBCO Spotfire Analyst with ability to read 17 different types of local files (dxp, stdf, sbdf, sfs, xls, xlsx, xlsm, xlsb, csv, txt, mdb, mde, accdb, accde, sas7bdat,udl, log, shp), connectivity to standard Data Sources (ODBC, OleDb, Oracle, Microsoft SQL Server Compact Data Provider 4.0, .NET Data Provider for Teradata, ADS Composite Information Server Connection, Microsoft SQL Server (including Analysis Services), Teradata and TIBCO Spotfire Maps. It also enables author  to do predictive analytics, forecasting, and local R language scripting).

This 2nd Spotfire’s Cloud option does not reduce Spotfire chances to compete with Tableau Online, which costs 4 times less ($500/year). However (thanks to 2 Blog Visitors – both with name Steve – for help), you cannot use Tableau online without licensed version of Tableau Desktop ($1999 perpetual non-expiring desktop license with 1st year maintenance included and each following year 20% $400 per year maintenance) and Online License (additional $500/year for access to the same site, but extra storage will not be added to that site!) for each consumer. Let’s compare Spotfire Workgroup Edition and Tableau Online cumulative cost for 1, 2, 3 and 4 years for 1 developer/analyst and 5 consumer seats :

 

Cumulative cost for 1, 2, 3 and 4 years of usage/subscription, 1 developer/analyst and 5 consumer seats:

Year

Spotfire Cloud Work Group, 250GB storage

Tableau Online (with Desktop), 100GB storage

Cost Difference (negative if Spotfire cheaper)

1

$2000

$4999

-$2999

2

$4000

$8399

-$4399

3

$6000

$11799

-$5799

4

$8000

$15199

-$7199

UPDATE: You may need to consider some other properties, like available storage and number of users who can consume/review visualizations, published in cloud. In sample above:

  • Spotfire giving to Work Group total 250 GB storage, while Tableau giving total 100 GB to the site. 2 or more subscriptions can be associated with the same site, but it will not increase the size of storage for the site from 100 GB to more (e.g. 200 GB for 2 subscribers). 
  • Spotfire costs less than Tableau Online for similar configuration (almost twice less!)

Overall, Spotfire giving more for your $$$ and as such can be a front-runner in Cloud Data Visualization race, considering that Qlikview does not have any comparable cloud options (yet) and Qliktech relying on its partners (I doubt it can be competitive) to offer Qlikview-based services in the cloud. Gere is the same table as above but as IMage (to make sure all web browsers can see it):

SFvsTBCloudPrice

It is important to consider another advantage of Spotfire Cloud: ability to share visualizations with everybody on internet by publishing them into Public Folder(s). By contrast, Tableau has limited licensing for this: in order to access to published workbooks on Tableau Online site, the Tableau Software by default requires the extra subscription, which is wrong from my point of view, because you can just publish it on Public Folder of such site (if such option allowed). By default (and without additional negotiations) Tableau Online does not allow the usage of Public Folder.

3rd Spotfire’s Cloud option called Spotfire Cloud Enterprise, it has customizable seating options and storage, more advanced visualization, security and scalability and connects to 40+ additional data sources. It requires an annoying negotiations with TIBCO sales, which may result to even larger pricing. Existence of 3rd Spotfire Cloud option decreases the value of its 2nd Cloud Option, because it saying to customer that Spotfire Cloud Work Group is not best and does not include many features. Opposite to that is Tableau’s Cloud approach: you will get everything (with one exception: Multidimensional (cube) data sources are not supported by Tableau Online) with Tableau Online, which is only the option.

Update 12/20/13:  TIBCO announced results for last quarter, ending 11/30/13 with Quarterly revenue $315.5M (only 6.4% growth compare with the same Quarter of 2012) and $1070M Revenue for 12 months ended 11/30/13 (only 4.4% growth compare with the same period of 2012). Wall Street people do not like it today and TIBX lost today 10% of its value, with Share Price ending $22 and Market Capitalization went down to less then $3.6B. At the same time Tableau’s Share Price went up $1 to $66 and Market Capitalization of Tableau Software (symbol DATA) went above $3.9B). As always I think it is relevant to compare the number of job openings today: Spotfire – 28, Tableau – 176, Qliktech – 71

Famous Traditional BI vendor got sick and tired to be out of Data Visualization market and decided to insert itself into it by force by releasing today 2 Free (for all users) Data Visualization Products:

  • MicroStrategy Analytics Desktop™ (Free self-service visual analytics tool)

  • MicroStrategy Analytics Express™ (Free Cloud-based self-service visual analytics)

That looks to me as the huge Disruption of Data Visualization Market: For example similar Desktop Product from Tableau costs $1999 and Cloud Product called Tableau Online costs $500/year/user. It puts Tableau, Qlikview and Spotfire to a very tough position price-wise. However only Tableau stock went down almost $3 (more then %4) today, but MSTR, TIBX an QLIK basically did not react on Microstrategy announcement):

DataMstrQlikTibx

And don’t think that only MIcrostrategy trying to get into DV market. For example SAP did similar (in less-dramatic and non-disruptive fashion) a few months ago with SAP Lumira (Personal Edition is free), also SAP Cloud and Standard edition available too, see it here http://www.saplumira.com/index.php and here http://store.businessobjects.com/store/bobjamer/en_US/Content/pbPage.sap-lumira . SAP senior vice president and platform head Steve Lucas 10 weeks ago was asked if SAP would consider buying Tableau, Lucas went in the opposite direction. “We aren’t going to buy Tableau,” Lucas said with a smile on his face. There’s no need to buy an overvalued software company.” Rather, SAP wants to crush companies like Tableau (I doubt it is possible, but SAP is free to try) and build own Data Visualization product line out of Lumira, read more at

http://venturebeat.com/2013/07/30/sap-platform-head-tableau-overvalued/#yFzUpzOh6ivMYvqP.99

If I will be Tableau, Qlikview or Spotfire I will not worry yet about Microstrategy competition yet, because it is unclear how the future R&D for free Analytics Desktop and Express will be funded – out of MicroStrategy Analytics Enterprise™ R&D budget? That can be tricky, considering as of right now Tableau hiring hard (163 open job positions as of yesterday!) and Qliktech is very active too (about 93 openings as of yesterday) and even TIBCO has 36 open positions just for Spotfire alone.

But I may start to worry about other DV Vendor - Datawatch, who recently completed the acquisition of Panopticon. Datawatch grew 45% YoY (2012-over-2011), has only 124 employees but $27.5M in sales, very experienced leadership, 40000+ customers worldwide and mature product line. May be another evidence of it here:

http://online.wsj.com/article/PR-CO-20131023-907942.html

The three MicroStrategy Analytics Platform products also share a common user experience—making it easy to start small with self-service analytics and grow into the production-grade features of Enterprise. Desktop and Express from Microstrategy can be naturally extended (for fee)  to a new enterprise-grade BI&DV Suite, also released today and called MicroStrategy Analytics Enterprise™ (known under other name as MIcrostrategy Suite 9.4). 

New MicroStrategy Analytics Enterprise 9.4 includes data blending, which allows users to combine data from more than one source; the software stores the data in working memory without the need for a separate data integration product.  9.4 can connect with the MongoDB NoSQL data store as well as Hadoop distributions from Hortonworks, Intel and Pivotal. It comes with the R, adds better ESRI integration. The application can now fit 10 times as much data in memory as the previous version could, and the self-service querying now runs up to 40 percent faster.

MicroStrategy Analytics Enterprise™ Suite is also available starting today for free for developers and non-production use: 10 named user licenses of MicroStrategy Intelligence Server, MicroStrategy Web Reporter and Analyst, MicroStrategy Mobile, MicroStrategy Report Services, MicroStrategy Transaction Services, MicroStrategy OLAP Services, MicroStrategy Distribution Services, and MultiSource Option. 1 named user license of development software, MicroStrategy Web Professional, MicroStrategy Developer, and MicroStrategy Architect The server components have a 1 CPU limit).

Quote from Wayne Eckerson, President of  BI Leader Consulting: “The new MicroStrategy Analytics Desktop makes MicroStrategy a top-tier competitor in the red-hot visual discovery market. The company was one of the first traditional enterprise BI vendors to ship a visual discovery tool, so its offering is mature compared to others in its peer group, but it was locked away inside its existing platform. By offering a stand-alone desktop visual discovery tool and making it freely available, MicroStrategy places itself among” Data Visualization Leaders.

You also can read today’s article from very frequent visitor to my blog (his name Akram), who is the Portfolio and Hedge Manager, Daily Trader and excellent investigator of all Data Visualization Stocks, DV Market and DV Vendors. His article “Tableau: The DV Market Just Got More Crowded”  can be found here (cannot resist to quote: “Microstrategy is priced like it has nothing to do with this space, and Tableau is priced like it will own the whole thing.”):

http://seekingalpha.com/article/1760432-tableau-the-dv-market-just-got-more-crowded?source=yahoo

Heatmap generated by Microstrategy Analytic Desktop

Heatmap generated by Microstrategy Analytic Desktop

MicroStrategy Analytics Desktop.

It’s free visual analytics: Free Visual Insight, 100M per file, 1GB total storage, 1 of user, Free e-mail support for 30 days. Free access to online training, forum, and knowledge base.
Data Sources: xls, csv, RDBMSes, Multidimensional Cubes, MapReduce, Columnar DBs, Access with Web Browser, export to Excel, PDF, flash and images, email distribution. The product is freely available to all and can be downloaded instantly at:http://www.microstrategy.com/free/desktop .

TRellis of Bar Charts generated by Microstrategy Analytics Desktop

TRellis of Bar Charts generated by Microstrategy Analytics Desktop

Kevin Spurway, MicroStrategy’s vice president of industry and mobile marketing said: “The new desktop software was designed to compete with other increasingly popular self-serve, data-discovery desktop visualization tools offered by Tableau and others”. To work with larger data sets, a user should have 2GB or more of working memory on the computer, Spurway said. See more here:

http://www.microstrategy.com/Strategy/media/downloads/free/analytics-desktop_quick-start-guide.pdf

MicroStrategy Analytics Express.

MicroStrategy Analytics Express is a software-as-a-service (SaaS)-based application that delivers all the rapid-fire self-service analytical capabilities of Desktop, plus reports and dashboards, native mobile applications, and secure team-based collaboration – all instantly accessible in the Cloud. Today, the Express community includes over 32,000 users across the globe.

In this release, Express inherits all the major functional upgrades of the MicroStrategy Analytics Platform, including new data blending features, improved performance, new map analytics, and much more. For a limited time, MicroStrategy is also making Express available to all users free for a year. With this valuable offer, users will be able to establish an account, invite tens, hundreds, or even thousands of colleagues to connect, analyze and share their data and insight, and do it all at no charge. For some organizations, the potential value of this offer can be $1 million or more. Users can sign up, access the service, and take advantage of this offer instantly at

www.microstrategy.com/free/express

MicroStrategy Analytics Express includes Free Visual Insight, Free web browser and iPad access, Free SaaS for one year, 1GB upload per file, unlimited number of users, Free e-mail support for 30 days. Free access to online training, forum, and knowledge base. Data Sources: xls, csv, RDBMSes Columnar DBs, Drobbox, Google Drive Connector, Visual Insight, a lot of security and a lot more, see http://www.microstrategy.com/Strategy/media/downloads/free/analytics-express_user-guide.pdf

All tools from Microstrategy Analytics Platform (Desktop, Express and Entereprise Suite) support standard list of Chart Styles and Types: Bar (Vertical/Horizontal Clustered/Stacked/100% Stacked), Line (Vertical/Horizontal Absolute/Stacked/100% Stacked), Combo Chart (of Bar and Area)Area (Vertical/Horizontal Absolute/Stacked/100% Stacked)

Area Chart Generated by Microstrategy Analytics Express

Area Chart Generated by Microstrategy Analytics Express

Dual Axis ( Bar/Line/Area Vertical/Horizontal), HeatMap, Scatter, Scatter Grid, Bubble, Bubble Grid, Grid,

Data Grid generated by Microstrategy Analytics Express

Data Grid generated by Microstrategy Analytics Express

Pie, Ring, ESRI Maps,

Microstrategy Analytics Desktop and Express integrate and generate ESRI Map Visualizations

Microstrategy Analytics Desktop and Express integrate and generate ESRI Map Visualizations

Network of Nodes, with lines representing links/connections/relationship,

Network Graph Generated by Microstrategy Analytics Express

Network Graph Generated by Microstrategy Analytics Express

Microcharts and Sparklines,

e4Microcharts

Data and Word Clouds,

DataCloud

and of course any kind of interactive Dashboards as combination of all of the above Charts, Graphs, and Marks:

Interactive Dashboard Generated by Microstrategy Analytics Express

Interactive Dashboard Generated by Microstrategy Analytics Express

Yesterday TIBCO announced Spotfire 6 with features, competitive with Tableau 8.1 and Qlikview.Next (a.k.a Qlikview 12). Some new features will be showcased at TUCON® 2013, TIBCO’s annual user conference, October 14-17, 2013 (2100 attendees). Livestream Video is here: http://tucon.tibco.com/video/index.html , tune in October 15th and 16th from 11:30am – 3:30pm EST.

More details will be shown in webcasts and webinars (I personally prefer detailed articles, blogposts, slides, PDFs and Demos, but TIBCO’s corporate culture ignores my preferences for years) on 10/30/13 by Steve Farr

Spotfire 6.0 will be available in mid-November, presumably the same time as Tableau 8.1 and before then Qlikview.Next so TIBCO is not a loser in Leap-frogging game for sure…

TIBCO bought the Extended Results and will presumably will show the integration with PSUHBI product, see it here: http://www.pushbi.com/ ; TIBCO called it as Delivery of  personal KPIs and Metrics on any mobile phone, tablet or laptop, online or offline (new name for it will be TIBCO Spotfire® Consumer):

ipadiphoneAnother TIBCO’s Purchase is MAPORAMA and integration with it TIBCO called (very appropriately) as the Location Analytics with promise to

  • Visualize, explore and analyze data in the context of location

  • Expand situational understanding with multi-layered geo-analytics

  • Mashup new data sources to provide precise geo-coding across the enterprise

la1Spotfire Location Services is the Agnostic Platform and supports (I guess this needs to be verified, because sounds too good to be true) any map service, including own TIBCO, ESRI (Spotfire integrates with ESRI previously), Google:

GeoCodingSP6TIBCO has Event processing capabilities (e.g BusinessEvents (5.1.2. now), ActiveSpaces ( currently v. 2.2) and realtime streaming of “Big Data” StreamBase (7.3.7) , they bought (StreamBase that is) a few months ago, see it here: http://www.streambase.com/news-and-events/press-releases/pr-2013/tibco-software-acquires-streambase-systems/#axzz2hiEjnr9X) and it will be interesting to see the new Spotfire Events Analytics (to Spot Event patterns)  product (see also: http://www.streambase.com/products/streambasecep ) integrated with Spotfire 6:

  • Identify new trends and outliers with continuous process monitoring

  • Automate the delivery of analytics applications based on trends

  • Operationalize analytics to support continuous process improvement:

ea1

One more capability in Spotfire mentioned (this claim needs to be verified) in recent TIBCO blogpost http://www.tibco.com/blog/2013/10/11/connecting-the-loops-the-next-step-in-decision-management/ as the ability to overlap 2 related but separated in real-life processes: the processes of analysis (discovery of insights in data) and execution (deciding and actions) could be separated by days, but with Spotfire 6.0 the entire decision process can happen in real time:

spotfireloops

For business user Spotfire 6 has new web-based authoring (Spotfire has a few “Clients”, one called Web Player and another called Enterprise Player, both are not free unlike Tableau Free Reader or Tableau Public). Bridging the gap between simple dashboards and advanced analytic applications, Spotfire 6.0 provides a new client “tailored to meet the needs of the everyday business user, who typically has struggled to manipulate pivot tables and charts to address their data discovery needs”.

With this new web application, known as TIBCO Spotfire® Business Author, business users can visually explore and interact with data, whether residing in a simple spreadsheet or dashboard, a database, or a predefined analytic application. It will definitely compete with Web Authoring in Tableau 8.1 and incoming Qlikview.Next.

For me personally the most interesting new feature is new Spotfire Cloud Services (supposedly the continuation of Spotfire SIlver, which I like but it is overpriced and non-competitive storage-wise vs. Tableau Public and Tableau Online cloud services). Here is the quote from yesterday’s Press Release: “TIBCO Spotfire® Cloud is a new set of cloud services for enterprises, work groups, and personal use (see some preliminary info here: https://marketplace.cloud.tibco.com/marketplace/marketplace/apps#/sc :

  • Personal: Web-based product, Upload Excel, .csv and .txt data, 12 visualization types, 100 GB of data storage. However, Spofire making a big mistake by denying access to Spotfire Analyst desktop product and making it as not free but only as “free trial for 30 days”, after which you have to pay a fee. That will benefit Tableau for sure and may be even Datawatch. As of 11/13/13, Spotfire still did not posted prices and fees for Spotfire Cloud Personal etc. and suggested to contact them over email, which I did but they never replied…
  • Workgroup: Web-based and desktop product, Connect and integrate multiple data sources, All visualization types, 250 GB of data storage.
  • Enterprise: Web-based and desktop product, Connect to 40+ data sources, All visualization types, Advanced statistics services, 500 GB of data storage

TIBCO Spotfire® Cloud Enterprise provides a secure full-featured version of Spotfire in the cloud to analyze and collaborate on business insights, whether or not the data is hosted. For project teams seeking data discovery as a service, TIBCO Spotfire® Cloud Work Group provides a wealth of application-building tools so distributed teams can visually explore data quickly and easily and deploy analytic applications at a very low cost. For individuals looking for a single step to actionable insight, TIBCO Spotfire®Personal is a cost-effective web-based client for quick data discovery needs.”

Please don’t forget that Spotfire 6 has TIBBR - v.5 as of now: https://tibco.tibbr.com/tibbr/web/login (social computing platform built for the workplace and integrated with Spotfire; Ram Menon, President of Social Computing at tibbr, says, “We now have 6.5 million users for tibbr as of October [2013]” and accessed from 7,000 cities, and 2,100 different device models. “A typical tibbr post is now seen by 100 users in the span of 24 hours, in 7 countries and over 50 mobile devices.” This fulfills TIBCO’s mission of getting the right information to the right people, at the right time. Related: integration between TIBBR and HUDDLE: http://www.huddle.com/blog/huddle-and-tibbr-unite-to-bring-powerful-collaboration-to-enterprise-social-networking/ )

And finally – enterprise-class, R-compatible statistical engine: TIBCO Enterprise Runtime for R (TERR) which is  the part of excellent TIBCO Spotfire Statistics Services (TSSS). TSSS allows Integration of R (including TERR), Spotfire’s own S+ (SPlus is Spotfire’s commercial version of R), SAS® and MATLAB® into Spotfire and custom applications. TERR, see http://spotfire.tibco.com/en/discover-spotfire/what-does-spotfire-do/predictive-analytics/tibco-enterprise-runtime-for-r-terr.aspx supports:

  • Support for paralelized R-language scripts in TERR

  • Support for call outs to open source R from TERR

  • Use RStudio – the most popular IDE in the R Community-to develop your TERR scripts

  • Over a thousand TERR compatible CRAN packages

Among other news is support for new Data Sources: http://spotfire.tibco.com/en/resources/support/spotfire-data-sources.aspx including SAP NetWeaver Business Warehouse v.7.0.1 (required TIBCO Connector Link).

General notes:

  1. I maintain my opinion that the best way for TIBCO to capitalize on tremendous hidden market value of Spotfire is to spin-it off as EMC did with VMWare.

  2. My other concern is too many offices involved with Spotfire: (Parental) TIBCO’s HQ in California, Swedish HQ (mostly R&D) office in Sweden and Large Marketing, Sales, Support and Consulting office in Newton, Massachusetts. My advise to have only one main office in MA, which is compatible with spin-off idea. Tableau has advantage here with concentrating their main office in Seattle.

  3. Update 11/13/13: TIBCO’s Spotfire propaganda so far did not help TIBCO stock shares at all, but seems to me that it helps a lot to Datawatch stock prices (Datawatch bought recently a very capable (technically) DV Vendor Panopticon and integrated its own software with Panopticon Software; Datawatch has 40000+ customers with 500000+ end users)

Last month Tableau and Qliktech both declared that Traditional BI is too slow (I am saying this for many years) for development and their new Data Visualization (DV software) is going to replace it. Quote from Tableau’s CEO: Christian Chabot: “Traditional BI software is obsolete and dying and this is very direct challenge and threat to BI vendors: your (BI that is) time is over and now it is time for Tableau.” Similar quote from Anthony Deighton, Qliktech’s CTO & Senior VP, Products: “More and more customers are looking at QlikView not just to supplement traditional BI, but to replace it“.

One of my clients – large corporation (obviously cannot say the name of it due NDA) asked me to advise of what to choose between Traditional BI tools with long Development Cycle (like Cognos, Business Objects or Microstrategy), modern BI tools (like JavaScript and D3 toolkit) which is attempt to modernize traditional BI but still having  sizable development time and leading Data Visualization tools with minimal development time (like Tableau, Qlikview or Spotfire).

Since main criterias for client were

  • minimize IT personnel involved and increase its productivity;

  • minimize the off-shoring and outsourcing as it limits interactions with end users;

  • increase end users’s involvement, feedback and action discovery.

So I advised to client to take some typical Visual Report project from the most productive Traditional  BI Platform (Microstrategy), use its prepared Data and clone it with D3 and Tableau (using experts for both). Results in form of Development time in hours) I put below; all three projects include the same time (16 hours) for Data Preparation & ETL, the same time for Deployment (2 hours) and the same number (8) of Repeated Development Cycles (due 8 consecutive feedback from End Users):

DVvsD3vsBI

It is clear that Traditional BI requires too much time, that D3 tools just trying to prolongate old/dead BI traditions by modernizing and beautifying BI approach, so my client choose Tableau as a replacement for Microstrategy, Cognos, SAS and Business Objects and better option then D3 (which require smart developers and too much development). This movement to leading Data Visualization platforms is going on right now in most of corporate America, despite IT inertia and existing skillset. Basically it is the application of the simple known principle that “Faster is better then Shorter“, known in science as Fermat’s Principle of least time.

This changes made me wonder (again) if Gartner’s recent marketshare estimate and trends for Dead Horse sales (old traditional BI) will stay for long. Gartner estimates the size of BI market as $13B which is drastically different from TBR estimate ($30B).

BIDeadHorseTheoryTBR predicts that it will keep growing at least until 2018 with yearly rate 4% and BI Software Market to Exceed $40 Billion by 2018 (They estimate BI Market as $30B in 2012 and include more wider category of Business Analytics Software as opposed to strictly BI tools). I added estimates for Microstrategy, Qliktech, Tableau and Spotfire to Gartner’s MarketShare estimates for 2012 here:

9Vendors

However, when Forrester asked people what BI Tools they used, it’s survey results were very different from Gartner’s estimate of “market share:

BIToolsInUse

“Traditional BI is like a pencil with a brick attached to it” said Chris Stolte at recent TCC13 conference and Qliktech said very similar in its recent announcement of Qlikview.Next. I expect TIBCO will say similar about upcoming new release of Spotfire (next week at TUCON 2013 conference in Las Vegas?)

Tableau_brick2

These bold predictions by leading Data Visualization vendors are just simple application of Fermat’s Principle of Least Time: this principle stated that the path taken between two points by a ray of light (or development path in our context) is the path that can be traversed in the least time.

Pierre_de_Fermat2Fermat’s principle can be easily applied to “PATH” estimates to multiple situations like in video below, where path from initial position of the Life Guard on beach to the Swimmer in Distress (Path through Sand, Shoreline and Water) explained: 

Even Ants following the Fermat’s Principle (as described in article at Public Library of Science here: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0059739 ) so my interpretation of this Law of Nature (“Faster is better then Shorter“) that  traditional BI is a dying horse and I advise everybody to obey the Laws of Nature.

AntsOn2SurfacesIf you like to watch another video about Fermat’s principle of Least Time and related Snell’s law, you can watch this: 
Google+

Qlikview 10 was released around 10/10/10, Qlikview 11 – around 11/11/11, so I expected Qlikview 12 to be released on 12/12/12. Qliktech press release said today that the next (after 11.2) version of Qlikview will be delivered under the new nickname Qlikview.Next in 2014 but “for  early adopter customers in a production environment in 2013″. I hope I can get my hands on it ASAP!

The new buzzword is Natural Analytics: “QlikView.Next’s key value as an alternative BI platform is in its use of Natural Analytics“. The new Qliktech motto that “Qlikview is a Replacement of Traditional BI” is similar to what we heard from Tableau leaders just 2 weeks ago on Tableau Customer Conference in Washington, DC.  Another themes I hear from Qliktech about Qliview.Next are sounds familiar too: Gorgeous, Genius, Visually Beautiful, Associative Experience, Comparative Analysis, Anticipatory, Drag and Drop Analytics.

Qlikview.Next will introduce “Data Dialogs” as live discussions between multiple users about Data they see and explore collectively, enabling “Social BI”. This reminds me the integration between TIBBR (TIBCO’s collaboration platform) and Spotfire, which existed since Spotfire 4.0.

Details about new features in Qlikview.Next will be released later, but at least we know now when Qlikview 12 (sorry, Qlikview.Next that is) will be available. Some features actually unveiled in generic terms::

  • Unified, Browser-Based HTML5 Client, which will automatically optimize itself for user’ device;

  • Automatic and Intelligent re-sizing of objects to fit user’s screen;

  • Server-side Analysis and Development, Web-based creation and delivery of content, Browser-based Development;

  • Data Storytelling, narrative and social with Data Dialogs;

  • Library and Repository for UI objects;

  • Multi-source Data Integration and new web-based scripting;

  • QlikView Expressor for advanced graphical Data Integration and Metadata Management;

  • Improved Data Discovery with associative experience across all the data, both in memory and on disks;

  • Open API: JSON, .NET SDK and as JavaScript API;

  • All UI Objects can be treated as extension Objects, customizable with their source files available to developers;

  • New Managment Console with Qlikview on Qlikview Monitor;

  • New visualization capabilities, based on advanced data visualization suite from NComVA (bought by Qliktech a few months ago), potential samples see here: http://www.ncomva.se/guide/?chapter=Visualizations

NComVAVisualizations11

In addition Qliktech is launching the “Qlik Customer Success Framework” , which includes:

  • Qonnect Partner Program: An extensive global network of 1500+ partners, including resellers, (OEMs), technology companies, and system integrators.

  • Qlik Community: An online community with nearly 100,000 members comprised of customers, partners, developers and enthusiasts.

  • Qlik Market: An online showcase of applications, extensions and connectors.

  • Qoncierge: A single point of contact service offering for customers to help them access the resources they need.

  • Comprehensive Services: A wide range of consulting services, training and support.

QlikFramework

Also see Ted Cuzzillo blogpost about it here: http://datadoodle.com/2013/10/09/next-for-qlik/# and Cindi Howson’s old post here: http://biscorecard.typepad.com/biscorecard/2012/05/qliktech-shares-future-product-plans-for-qlikview.html and new article here: http://www.informationweek.com/software/business-intelligence/qliktech-aims-to-disrupt-bi-again/240162403#!

Today Tableau Customer Conference 2013 started with 3200+ attendees from 40+ countries and 100+ industries, with 700 employees of Tableau, 240 sessions. Tableau 8.1 pre-announced today for release in fall of 2013, also version 8.2 planned for winter 2014, and Tableau 9.0 for later in 2014.

Update 9/10/13: keynote now is available recorded and online:  http://www.tableausoftware.com/keynote
(Recorded Monday Sept 9, 2013 Christian Chabot, Chris Stolte and the developers LIVE)

New in 8.1: 64-bit, Integration with R, support for SAML, IPV6 and External Load Balancers, Copy/Paste Dashboards and worksheets between workbooks, new Calendar Control, own visual style, including customizing even filters, Tukey’s Box-and-Whisker Box-plot, prediction bands, ranking, visual analytics for everyone and everywhere (in the cloud now)

Planned and new for 8.2: Tableau for MAC, Story Points (new type of worksheet/dashboard with mini-slides as story-points), seamless access to data via data connection interface to visually build a data schema, including inner/left/right/outer visual joins, beautifying columns names, easier metadata etc, Web authoring enhancements (it may get into 8.1: moving quick filters, improvement for Tablets, color encoding.) etc.

8.1:  Francois Ajenstat announced: 64-bit finally (I asked for that for many years) for server processes and for Desktop, support for SAML (single-sign-ON on Server and Desktop), IPV6, External Load Balancers:

Francois

SAML8.1: Dave Lion announced R integration with Tableau:

DaveLion

r8.1: Mike Arvold announced “Visual Analytics for everyone”, including implemention of famous Tukey’s Box-and-Whisker Box-plot (Spotfire has it for a while, see it here: http://stn.spotfire.com/stn/UserDoc.aspx?UserDoc=spotfire_client_help%2fbox%2fbox_what_is_a_box_plot.htm&Article=%2fstn%2fConfigure%2fVisualizationTypes.aspx ),

better forecasting, prediction bands, ranking, better heatmaps:

MikeArvold8.1: Melinda Minch announced “fast, easy, beautiful”, most importantly copy/paste dashboards and worksheets between workbooks, customizing everything, including quick filters, new calendar control, own visual style, folders in Data Window etc…

MelindaMinch28.2: Jason King pre-announced the Seamless access to data via data connection interface to visually build a data schema, including inner/left/right/outer “visual” joins, beautifying columns names, default formats, new functions like DATEPARSE, appending data-set with new tables, beautifying columns names, easier metadata etc.

JasonKingSeamlessAccess2data28.2: Robert Kosara introduced Story Points (using new type of worksheet/dashboard with mini-slides as story-points) for new Storytelling functionality:

RobertKosara2

Here is an example of Story Points, done by Robert:

storypoints-4

8.2: Andrew Beers pre-announced Tableau 8.2 on MAC and he got a very warm reception from audience for that:

AndrewBeers3Chris Stolte proudly mentioned his 275-strong development team, pre-announced upcoming Tableau Releases 8.1 (this fall), 8.2 (winter 2014) and 9.0 (later in 2014) and introduced 7 “developers” who (see above Francois, Mike, Dave, Melinda, Jason, Robert and Andrew) discussed during this keynote new features (feature list is definitely longer and wider that recent “innovations” we saw from Qlikview 11.2 and even from Spotfire 5.5):

ChrisStolte2Christian Chabot opening keynote today… He said something important: current BI Platforms are not fast, nor easy, they are not beautiful and not for anyone and they are definitely not “anywhere” but only in designated places with appropriate IT personnel (compare with Tableau Public, Tableau Online, Tableau free Reader etc.) and it is only capable to produce a bunch of change requests from one Enterprise’s department to another, which will take long time to implement with any SDLC framework.

CEOChristian basically repeated what I am saying on this blog for many years, check it here http://apandre.wordpress.com/market/competitors/ : traditional BI software (from SAP, IBM, Oracle, Microstrategy and even Microsoft cannot compete with Tableau, Qlikview and Spotfire) is obsolete and dying and this is very direct challenge and threat to BI vendors (I am not sure if they understand that): your (BI that is) time is over and now it is time for Tableau (also for Qlikview and Spotfire but they are slightly behind now…).

Update on 11/21/13: Tableau 8.1 is available today, see it here: http://www.tableausoftware.com/new-features/8.1 and Tableau Public 8.1 is available as well, see it here: http://www.tableausoftware.com/public/blog/2013/11/tableau-public-81-launches-2226

While blog preserving my observations and thoughts, it preventing me to spend enough time to read what other people thinking and saying, so I created almost 2 years ago the extension of this blog in the form of 2 Google+ pages http://tinyurl.com/VisibleData and http://tinyurl.com/VisualizationWithTableau , where I accumulated all reading pointers for myself and gradually reading those materials when I have time.

Those 2 pages magically became extremely popular (this is unintended result) with total more than 5000 Google+ followers as of today. For example here is a Chart showing monthly growth of the  number of followers for the main extension of this blog http://tinyurl.com/VisibleData :

GPFollowersMonthly

So please see below some samples of Reading Pointers accumulated over last 3 months of summer by my Google+ pages:

Author trying to simplify BigData Definition as following: “BigData Simplified: Too much data to fit into a single server”: http://yottascale.com/entry/the-colorful-secrets-of-bigdata-platforms

Recent talk from Donald Farmer: http://www.wired.com/insights/2013/06/touch-the-next-frontier-of-business-intelligence/

Dmitry pointing to implementation Disaster of Direct Discovery in Qlikview 11.2: http://bi-review.blogspot.com/2013/04/first-look-at-qlikview-direct-discovery.html

Specs for Tableau in Cloud: https://www.tableausoftware.com/products/online/specs

The DB-Engines Monthly Ranking ranks database management systems according to their popularity. Turned out that only 3 DBMSes are popular: Oracle, SQL Server and MySQL:

According to Dr. Andrew Jennings, chief analytics officer at FICO and head of FICO Labs, three main skills of data scientist are the same 3 skills I tried to find when hiring programmers for my teams 5, 10, 20 and more years ago: 1. Problem-Solving Skills. 2. Communications Skills. 3. Open-Mindedness. This makes all my hires for last 20+ years Data Scientists, right? See it here: http://www.informationweek.com/big-data/news/big-data-analytics/3-key-skills-of-successful-data-scientis/240159803

A study finds the odds of rising to another income level are notably low in certain cities, like Atlanta and Charlotte, and much higher in New York and Boston: http://www.nytimes.com/2013/07/22/business/in-climbing-income-ladder-location-matters.html

Tableau is a prototyping tool: http://tableaufriction.blogspot.com/2013/07/the-once-and-future-prototyping-tool-of.html

Why More Data and Simple Algorithms Beat Complex Analytics Models: http://data-informed.com/why-more-data-and-simple-algorithms-beat-complex-analytics-models/

New Census Bureau Interactive Map Shows Languages Spoken in America: http://www.census.gov/newsroom/releases/archives/education/cb13-143.html

Google silently open sourced a tool called word2vec, prepackaged deep-learning software designed to understand the relationships between words with no human guidance. It actually similar to known for a decade methods called PLSI and PLSA:

“Money is not the only reward of education, yet it is surely the primary selling point used to market data science programs, and the primary motivator for students. But there’s no clear definition of data science and no clear understanding of what knowledge employers are willing to pay for, or how much they will pay, now or in the future. Already I know many competent, diligent data analysts who are unemployed or underemployed. So, I am highly skeptical that the students who will invest their time and money in data science programs will reap the rewards they have been led to expect.”: http://www.forbes.com/sites/gilpress/2013/08/19/data-science-whats-the-half-life-of-a-buzzword/

Some good blog-posts from InterWorks:

Technique for using Tableau data blending to create a dynamic, data-driven “parameter”: http://drawingwithnumbers.artisart.org/creating-a-dynamic-parameter-with-a-tableau-data-blend/

More about Colors:

Russian Postcodes are collected and partially visualized:

http://acuitybusiness.com/blog/bid/175066/Three-Reasons-Why-Companies-Should-Outlaw-Excel

EXASolution claims to be up to 1000 times faster than traditional databases and the fastest database in the world – based on in memory computing.

http://www.exasol.com/en/exasolution/technical-details.html

web interest to Tableau and Qlikview:

http://www.google.com/trends/explore?q=qlikview%2C+tableau%2C+spotfire%2C+microstrategy#q=tableau%2C%20microstrategy%2C%20qlikview%2C%20spotfire&geo=US&date=9%2F2008%2061m&cmpt=q

20 months ago I checked how many job openings leading DV Vendors have. On 12/5/11 Tableau had 56, Qliktech had 46 and Spotfire had 21 openings. Today morning I checked their career sites again and noticed that both Tableau and Qliktech almost double their thirst for new talents, while Spotfire basically staying on the same level of hiring needs:

  • Tableau has 102(!) openings, 43 of them are engineering positions (I counted their R&D positions and openings in Operation department too) – that is huge! Update as of 9/18/13 has exactly 1000 employees. 1000th employee can be found on this pictureTableau1000Employees091813

  • Qliktech has 87 openings, 29 of them are engineering positions (I included R&D, IT, Tech Support and Consulting).

  • TIBCO/Spotfire has 24 openings, 16 of them are engineering positions (R&D, IT, Tech.Support).

BostonSkylineFromWindow

All 3 companies are Public now, so I decided to include their Market Capitalization as well. Since Spofire is hidden inside its corporate parent TIBCO, I used my estimate that Spotfire’s Capitalization is about 20% of TIBCO’s capitalization (which is $3.81B as of 8/23/13, see https://www.google.com/finance?q=TIBX ). As a result I have this Market Capitalization numbers for 8/23/13 as closing day:

Those 3 DV Vendors above together have almost $8B market capitalization as of evening of 8/23/13 !

Market Capitalization update as of 8/31/13: Tableau: $4.3B, Qliktech $2.9B, Spotfire (as 20% of TIBCO) – $0.72B

Market Capitalization update as of 9/4/13 11pm: Tableau: $4.39B, Qliktech $3B, Spotfire (as 20% of TIBCO) – $0.75B . Also as of today Qliktech employed 1500+ (approx. $300K revenue per year per employee), Tableau about 1000 (approx. $200K revenue per year per employee) and Spotfire about 500+ (very rough estimate, also approx. $350K revenue per year per employee)

Google+

Last week Tableau increased by 10-fold the capacity of Data Visualizations published with Tableau Public to a cool 1 Million rows of Data, basically to the same amount of rows, which Excel 2007, 2010 and 2013 (often used as data sources for Tableau Public) can handle these days and increased by 20-fold the storage capacity (to 1GB of free storage) of each free Tableau Public Account, see it here:

http://www.tableausoftware.com/public/blog/2013/08/one-million-rows-2072

It means that free Tableau Public Account will have the storage twice larger than Spotfire Silver’s the most expensive Analyst Account (that one will cost you $4500/year). Tableau said: “Consider it a gift from us to you.”. I have to admit that even kids in this country know that there is nothing free here, so please kid me not – we are all witnessing of some kind of investment here – this type of investment worked brilliantly in the past… And all users of Tableau Public are investing too – with their time and learning efforts.

And this is not all: “For customers of Tableau Public Premium, which allows users to save locally and disable download of their workbooks, the limits have been increased to 10 million rows of data at 10GB of storage space” see it here:

http://www.tableausoftware.com/about/press-releases/2013/tableau-software-extends-tableau-public-1-million-rows-data without changing the price of service (of course in Tableau Public Premium price is not fixed and depends on the number of impressions).

Out of 100+ millions of Tableau users only 40000 qualified to be called Tableau Authors, see it here  http://www.tableausoftware.com/about/press-releases/2013/tableau-software-launches-tableau-public-author-profiles so they are consuming Tableau Public’s Storage more actively then others. As an example you can see my Tableau’s Author Profile here: http://public.tableausoftware.com/profile/andrei5435#/ .

I will assume those Authors will consume 40000GB of online storage, which will cost to Tableau Software less then (my guess, I am open to correction from blog visitors) $20K/year just for the storage part of Tableau Public Service.

During the last week the other important announcement on 8/8/13 – Quarterly Revenue – came from Tableau: it reported the Q2 revenue of $49.9 million, up 71% year-over-year: http://investors.tableausoftware.com/investor-news/investor-news-details/2013/Tableau-Announces-Second-Quarter-2013-Financial-Results/default.aspx .

Please note that 71% is extremely good YoY growth compare with the entire anemic “BI industry”, but less then 100% YoY which Tableau grew in its private past.

All these announcements above happened simultaneously with some magical (I have no theory why this happened; one weak theory is the investors madness and over-excitement about Q2 revenue of $49.9M announced on 8/8/13?) and sudden increase of the nominal price of Tableau Stock (under the DATA name on NYSE) from $56 (which is already high) on August 1st 2013 (announcement of 1 millions of rows/1GB storage for Tableau public Accounts) to $72+ today:

DATAstock812Area2

It means that the Market Capitalization of Tableau Software may be approaching $4B and sales may be $200M/year. For comparison, Tableau’s direct and more mature competitor Qliktech has now the Capitalization below $3B while its sales approaching almost $500M/year. From Market Capitalization point of view in 3 moths Tableau went from a private company to the largest Data Visualization publicly-traded software company on market!

Competition in Data Visualization market is not only on features, market share and mindshare but also on pricing and lisensing. For example the Qlikview licensing and pricing is public for a while here: http://www.qlikview.com/us/explore/pricing and Spotfire Silver pricing public for a while too:  https://silverspotfire.tibco.com/us/silver-spotfire-version-comparison .

Tableau Desktop has 3 editions: Public (Free), Personal ($999) and Professional ($1999), see it here: http://www.tableausoftware.com/public/comparison ; in addition you can have full Desktop (read-only) experience with free Tableau Reader (neither Qlikview nor Spotfire have free readers for server-less, unlimited distribution of Visualizations, which is making Tableau a mind-share leader right away…)

The release of Tableau Server online hosting this month:  http://www.tableausoftware.com/about/press-releases/2013/tableau-unveils-cloud-business-intelligence-product-tableau-online heated the licensing competition and may force the large changes in licencing landscape for Data Visualization vendors. Tableau Server existed in the cloud for a while with tremendous success as Tableau Public (free) and Tableau Public Premium (former Tableau Digital with its weird pricing based on “impressions”).

But Tableau Online is much more disruptive for BI market: for $500/year you can get the complete Tableau Server site (administered by you!) in the cloud with (initially) 25 (it can grow) authenticated by you users and 100GB of cloud storage for your visualizations, which is 200 times more then you can get for $4500/year top-of-the line Spotfire Silver “Analyst account”. This Tableau Server site will be managed in the cloud by Tableau Software own experts and require nor IT personnel from your side! You may also compare it with http://www.rosslynanalytics.com/rapid-analytics-platform/applications/qlikview-ondemand .

A hosted by Tableau Software solution is particularly useful when sharing dashboards with customers and partners because the solution is secure but outside a company’s firewall. In the case of Tableau Online users can publish interactive dashboards to the web and share them with clients or partners without granting behind-the-firewall access.

Since Tableau 8 has new Data Extract API, you can do all data refreshes behind your own firewall and republish your TDE files in the cloud anytime (even automatically, on demand or on schedule) you need. Tableau Online has no minimum number of users and can scale as a company grows. At any point, a company can migrate to Tableau Server to manage it in-house. Here is some introductionla video about Tableau Online: Get started with Tableau Online.

Tableau Server in the cloud provides at least 3 ways to update your data (more details see here: http://www.tableausoftware.com/learn/whitepapers/tableau-online-understanding-data-updates )

TableauDesktopAsProxyForTableauServer

Here is another, more lengthy intro into Tableau BI in Cloud:

Tableau as a Service is a step in right direction, but be cautious:  in practice, the architecture of the hosted version could impact performance. Plus, the nature of the product means that Tableau isn’t really able to offer features like pay-as-you-go that have made cloud-based software popular with workers. By their nature, data visualization products require access to data. For businesses that store their data internally, they must publish their data to Tableau’s servers. That can be a problem for businesses that have large amounts of data or that are prevented from shifting their data off premises for legal or security reasons. It could also create a synchronization nightmare, as workers play with data hosted at Tableau that may not be as up-to-date as internally stored data. Depending on the location of the customer relative to Tableau’s data center, data access could be slow.

And finally, the online version requires the desktop client, which costs $2,000. Tableau may implement Tableau desktop analytical features in a browser in the future while continue to support the desktop and on-premise model to meet security and regulations facing some customers.

Tableau_Online

I got many questions from Data Visualization Blog’s visitors about differences between compensation for full-time employees and contractors. It turned out that many visitors are actually contractors, hired because of their Tableau or Qlikview or Spotfire skills and also some visitors consider a possibility to convert to consulting or vice versa: from consulting to FullTimers. I am not expert in all these compensation and especially benefits-related questions, but I promised myself that my blog will be driven by vistors’s requests, so I google a little about Contractor vs. Full-Time worker compensation and below is brief description of what I got:

Federal Insurance Contribution Act mandates Payroll Tax splitted between employer (6.2% Social Security with max $$7049.40 and 1.45% Medicare on all income) and employee, with total (2013) as 15.3% of gross compensation.

Historical_Payroll_Tax_Rates

In addition you have to take in account employer’s contribution (for family it is about $1000/per month) to medical benefits of employee, Unemployment Taxes, employer’s contribution to 401(k), STD and LTD (short and long term disability insurances), pension plans etc.

I also added into my estimate of contractor rate the “protection” for at least 1 month GAP between contracts and 1 month of salary as bonus for full-time employees.

RR20120507-BCC-2

Basically the result of my minimal estimate as following: you need to get as a contractor the rate at least 50% more than base hourly rate of the full-time employee. This  base hourly rate of full-time employee I calculate as employee’s base salary divided on 1872 hours: 1872 = (52 weeks*40 hours – 3 weeks of vacation – 5 sick days – 6 holidays) = 2080 hours – 208 hours (Minimum for a reasonable PTO, Personal Time Off) = 1872 working hours per year.

I did not get into account any variations related to the usage of W2 or 1099 forms or Corp-To-Corp arrangements and many other fine details (like relocation requirements and overhead associated with involvement of middlemen like headhunters and recruiters) and differences between compensation of full-time employee and consultant working on contract – this is just a my rough estimate – please consult with experts and do not ask me any questions related to MY estimate, which is this:

  • Contractor Rate should be 150% of the base rate of a FullTimer

RS-COLLEGE LOAN SCAMS low resIn general, using Contractors (especially for business analytics) instead of Full-timers is basically the same mistake as outsourcing and off-shoring: companies doing that do not understand that their main assets are full-time people. Contractors are usually not engaged and they are not in business to preserve intellectual property of company.

Capitalist
For reference see Results of Dr. Dobbs 2013 Salary Survey for Software Developers which are very comparable with salary of Qlikview, Tableau and Spotfire developers and consultants (only in my experience salary of Data Visualization Consultants are 10-15% higher then salaries of software developers):

Fig01SalaryByTitle_full

This means that for 2013 the Average rate for Qlikview, Tableau and Spotfire developers and consultants should be around 160% of the base rate of a average FullTimer, which ESTIMATES to Effective Equivalent Pay to Contractor for 1872 hours per Year as $155,200 and this is only for average consultant... If you take less then somebody tricked you, but if you read above you already know that.

RedBinaries

NearBy1920

Best of the Tableau Web… December 2012:

http://www.tableausoftware.com/about/blog/2013/1/best-tableau-web-december-2012-20758

Top 100 Q4 2012 from Tableau Public:

http://www.tableausoftware.com/public/blog/2013/01/top-100-q4-2012-1765

eBay’s usage of Tableau as the front-end for big data, Teradata and Hadoop with 52 petabytes of
data on everything from user behavior to online transactions to customer shipments and much more:

http://www.infoworld.com/d/big-data/big-data-visualization-big-deal-ebay-208589

Why The Information Lab recommends Tableau Software:

http://www.theinformationlab.co.uk/2013/01/04/recommend-tableau-software/

Fun with #Tableau Treemap Visualizations

http://tableaulove.tumblr.com/post/40257187402/fun-with-tableau-treemap-visualizations

Talk slides: Tableau, SeaVis meetup & Facebook, Andy Kirk’s Facebook Talk from Andy Kirk

http://www.visualisingdata.com/index.php/2013/01/talk-slides-tableau-seavis-meetup-facebook/

Usage of RAM, Disk and Data Extracts with Tableau Data Engine:

http://www.tableausoftware.com/about/blog/2013/1/what%E2%80%99s-better-big-data-analytics-

memory-or-disk-20904
Migrating Tableau Server to a New Domain

https://www.interworks.com/blogs/bsullins/2013/01/11/migrating-tableau-server-new-domain

SAS/Tableau Integration

http://www.see-change.co/services/sastableau-integration/

IFNULL – is not “IF NULL”, is “IF NOT NULL”

http://tableaufriction.blogspot.com/2012/09/isnull-is-not-is-null-is-is-not-null.html

Worksheet and Dashboard Menu Improvements in Tableau 8:

http://tableaufriction.blogspot.com/2013/01/tv8-worksheet-and-dashboard-menu.html

Jittery Charts – Why They Dance and How to Stop Them:

http://tableaufriction.blogspot.com/2013/01/jittery-charts-and-how-to-fix-them.html

Tableau Forums Digest #8

http://shawnwallwork.wordpress.com/2013/01/06/67/

Tableau Forums Digest #9

http://shawnwallwork.wordpress.com/2013/01/14/tableau-forums-digest-9/

Tableau Forums Digest #10

http://shawnwallwork.wordpress.com/2013/01/19/tableau-forums-digest-10/

Tableau Forums Digest #11

http://shawnwallwork.wordpress.com/2013/01/26/tableau-forums-digest-11/

implementation of bandlines in Tableau by Jim Wahl (+ Workbook):

http://community.tableausoftware.com/message/198511

In my previous post http://apandre.wordpress.com/2012/11/16/new-tableau-8-desktop-features/ (this post is the continuation of it) , I said that Tableau 8 introduced 130+ new features, 3 times more then Tableau 7 did. Many of these new features are in Tableau 8 Server and this post about those new Server features (this is a repost from my Tableau blog: http://tableau7.wordpress.com/2012/11/30/new-tableau-8-server-features/ ).

The Admin and Server pages have been redesigned to show more info quicker. In list view the columns can be resized. In thumbnail view the grid dynamically resizes. You can hover over a thumbnail to see more info about visualization. The content search is better too:

ThumbnailView

Web authoring (even mobile) introduced by Tableau 8 Server. Change dimensions, measures, mark types, add filters, and use Show Me are all directly in a web browser and can be saved back to the server as a  new workbook or if individual permissions allow, to the original workbook:

webAuthoring

Subscribing to a workbook or worksheet will automatically notify about the dashboard or view updates to your email inbox. Subscriptions deliver image and link.

Tableau 8 Data Engine is more scalable now, it can be distributed between 2 nodes, 2nd instance of it now can be configured as Active, Synced and Available for reading if  Tableau Router decided to use it (in addition Fail-over function as before)server2sTableau 8 Server now supports Local Rendering, using graphic ability of local devices, modern browsers and HTML5. No-round-trip to server while rendering using latest versions of chrome 23+, Firefox 17+, Safari , IE 9+. Tableau 8 (both Server and Desktop, computing each view in Parallel. PDF files, generated by Tableau 8 up to 90% smaller and searchable. And Performance Recorder works on both Server and Desktop.

Tableau 8 Server introducing Shared sessions allows more concurrency, more caching. Tableau 7 uses 1 session per viewer. Tableau 8 using one session per many viewers, as long as they do no change state of filters and don’t do other altering interaction. If interaction happened, Tableau 8 will clone the session for appropriate Interactor and apply his/her changes to new session:server3sIFinally Tableau getting API, 1st part of it I described in previous blog post about TDesktop – TDE API (C/C++, Python, Java on both Windows AND Linux!).

For Web Development Tableau has now brand new JavaScript API to customize selection, filtering, triggers to events, custom toolbar, etc. Tableau 8 has own JavaScript API WorkBench, which can be used right from you browser:server4w

TDE API allows to build own TDE on any machine with Python, C/C++ and Java (see 24:53 at http://www.tableausoftware.com/tcc12conf/videos/new-tableau-server-8 ). Additionally Server API (REST API) allows programmatically create/enable/suspend sites and add/remove users to sites.

In addition to Faster Uploads andPublishing Data Sources, users can Publish Filters as Set and User Filters. Data Sources can be Refreshed or Appended instead of republishing – all from Local Sources. Such Refreshes can scheduled using Windows Task Scheduler or other task scheduling software on client devices – this is a real TDE proliferation!

My wishlist for Tableau 8 Server: all Tableau Server processes needs to be 64-bit (and they still 32-bit, see it here: http://onlinehelp.tableausoftware.com/v7.0/server/en-us/processes.htm ; they are way overdue to be the 64-bit; Linux version of Tableau Server (Microsoft recently changed very unfavorably the way they charge users for each Client Access) is needed, I wish integration with R Library (Spotfire has it for years), I want Backgrounder Processes (mostly doing data extracts on server) will not consume core licenses etc…

And yes, I found in San Diego even more individuals who found the better way to spend their time compare with attending Tableau 2012 Customer Conference and I am not here to judge:

SealsInLaJolla

I left Tableau 2012 conference in San Diego (where Tableau 8 was announced) a while ago with enthusiasm which you can feel from this real-life picture of 11 excellent announcers:

Tableau8IntroducedInSanDiego

Conference was attended by 2200+ people and 600+ Tableau Software employees (Tableau almost doubled the number of employees in a year) and it felt like a great effort toward IPO (see also article here: http://www.bloomberg.com/news/2012-12-12/tableau-software-plans-ipo-to-drive-sales-expansion.html ).  See some video here: TCC12 Keynote . Tableau 8 introduce 130+ new features, 3 times more then Tableau 7 did. Almost half of these new features are in Tableau 8 Desktop and this post about those new Desktop features (this is a repost from my Tableau Blog: http://tableau7.wordpress.com/2012/11/16/new-tableau-8-desktop-features/). New Tableau 8 Server features deserved a separate blog post which I will publish a little later after playing with Beta 1 and may be Beta 2.

A few days after conference the Tableau 8 Beta Program started with 2000+ participants. One of the most promising features is new rendering engine and I build special Tableau 7 visualization (and its port to Tableau 8) with 42000 datapoints: http://public.tableausoftware.com/views/Zips_0/Intro?:embed=y  to compare the speed of rendering between versions 7 and 8:

ZipColors

Among new features are new (for Tableau) visualization types: Heatmap, “Packed” Bubble Chart and Word Cloud, and I build simple Tableau 8 Dashboard to test it (all 3 are visualizing the 3-dimensional set where 1 dimension used as list of items, 1 measure used for size and 2nd measure used for color of items):

3NewTypesOfVisualizationsInTableau

List of new features includes improved Sets (comparing members vs. non-members, adding/removing members, combining Sets: all-in-both, shared-by-both, left-except-right, right-except-left), Custom SQL with parameters, Freeform Dashboards (I still prefer MDI UI where each Chart/View Sheet has own Child Window as oppose to Pane), ability to add multiple fields to Labels, optimized label placement, built-in statistical models for visual Forecasting, Visual Grouping based on your data selection, Redesigned Mark Card (for Color, Size, Label, Detail and Tooltip Shelves).

New Data features include data blending without mandatory linked field in a view and with ability to filter data in secondary data sources; refreshing server-based Data Extracts can be done from local data sources; Data Filters (in addition be either local or global) can be shared now among selected set of worksheets and dashboards. Refresh of Data Extract can be done using command prompt for Tableau Desktop, for example

>tableau.exe refreshremoteextract

Tableau 8 has (finally) API (C/C++, Python, Java) to directly create a Tableau Data Extract (TDE) file, see example here: http://ryrobes.com/python/building-tableau-data-extract-files-with-python-in-tableau-8-sample-usage/

Tableau 8 (both Desktop and Server) can then connect to this extract file natively! Tableau provides new native connection for Google Analytics and Saleforce.com. TDE files now much smaller (especially with text values) – up to 40% smaller compare with Tableau 7.

Tableau 8 has performance enhancements, such as the new ability to use hardware accelerators (of modern graphics cards), computing views within dashboard in parallel (in Tableau 7 it was consecutive computations) and new  performance recorder allows to estimate and tune a workload of various activities and functions and optimize the behavior of workbook.

I still have a wishlist of features which are not implemented in Tableau and I hope some them will be implemented later: all Tableau processes are 32-bit (except 64-bit version of data engine for server running on 64-bit OS) and they are way overdue to be the 64-bit; many users demand MAC version of Tableau Desktop and Linux version of Tableau Server (Microsoft recently changed very unfavorably the way they charge users for each Client Access), I wish MDI UI for Dashboards where each view of each worksheet has own Window as oppose to own pane (Qlikview does it from the beginning of the time), I wish integration with R Library (Spotfire has it for years), scripting languages and IDE (preferably Visual Studio), I want Backgrounder Processes (mostly doing data extracts on server) will not consume core licenses etc…

Despite the great success of the conference, I found somebody in San Diego who did not pay attention to it (outside was 88F, sunny and beautiful):

HummingbirdInLaJolla

Qlikview 10 was released around 10/10/10, Qlikview 11 – around 11/11/11, so I expected Qlikview 12 to be released on 12/12/12 but “instead” we are getting Qlikview 11.2 with Direct Discovery in December 2012, which supposedly provides a “hybrid approach so business users can get the QlikView associative experience even with data that is not stored in memory”

This feature demanded by users (me included) for a long time, but I think noise around so called Big Data and competition forced Qliktech to do it. Spotfire has it for a longtime (as well as 64-bit implementation) and Tableau has something like that for a while (unfortunately Tableau still 32-bit) . You can test Beta of it, if you have time: http://community.qlikview.com/blogs/technicalbulletin/2012/10/22/qlikview-direct-discovery-beta-registration-is-open

Just 8 months ago Qliktech estimated its sales for 2012 as $410M and suddenly 3 months ago it changed its estimates down to $381M, just 19% over 2011, which is in huge contrast with Qliktech’s previous speed of growth and way behind the current speed of growth of Tableau and even less then current speed of growth of Spotfire. During last 2 years QLIK stock unable to grow significantly:

and all of the above forcing Qliktech to do something outside of gradual improvements – new and exciting functionality needed and Direct Discovery may help!

QlikView Direct Discovery enables users to perform visual analysis against “any amount of data, regardless of size”. With the introduction of this unique hybrid approach, users can associate data stored within big data sources directly alongside additional data sources stored within the QlikView in-memory model. QlikView can “seamlessly connect to multiple data sources together within the same interface”, e.g. Teradata to SAP to Facebook allowing the business user to associate data across the data silos. Data outside of RAM can be joined with the in-memory data with the common field names. This allows the user associatively navigate both on the direct discovery and in memory data sets.

QlikView developer should setup the Direct Discovery table on the QlikView application load script to allow the business users to query the desired big data source. Within the script editor a new syntax is introduced to connect to data in direct discovery form. Traditionally the following syntax is required to load data from a database table:

To invoke the direct discovery method, the keyword “SQL” is replaced with “DIRECT”.

In the example above only column CarrierTrackingNumber and ProductID are loaded into QlikView in the traditional manner, other columns exist in the data table within the Database including columns OrderQty and Price. OrderQty and Price fields are referred as “IMPLICIT” fields. An implicit field is a field that QlikView is aware of on a “meta level”. The actual data of an implicit field resides only in the database but the field may be used in QlikView expressions. Looking at the table view and data model of the direct discovery columns are not within the model (on the OrderFact table):

Once the direct discovery structure is established, the direct discovery data can be joined with the in-memory data with the common field names (Figure 3). In this example, “ProductDescription” table is loaded in-memory and joined to direct discovery data with the ProductID field. This allows the user to associatively navigate both on the “direct discovery” and in memory data sets.

Direct Discovery will be much slow then in-memory processing and this is is expected, but it will take away from Qlikview its usual claim that is is faster then competitors. QlikView Direct Discovery can only be used against SQL compliant data sources. The following data sources are supported;

• ODBC/OLEDB data sources – All ODBC/OLEDB sources are supported, including SQL Server, Teradata and Oracle.
• Custom connectors which support SQL – Salesforce.com, SAP SQL Connector, Custom QVX connectors for SQL compliant data stores.

Due to the interactive and SQL syntax specific nature of the Direct Discovery approaches a number of limitations exist. The following chart types are not supported;
• Pivot tables
• Mini charts
And the following QlikView features are not supported:
• Advanced aggregation
• Calculated dimensions
• Comparative Analysis (Alternate State) on the QlikView objects that use Direct
Discovery fields
• Direct Discovery fields are not supported on Global Search
• Binary load from a QlikView application with Direct Discovery table

Here is a some preliminary video about Direct Discovery, published by Qliktech:

It was interesting to me that just 2 days after Qliktech pre-anounced Direct Discovery it also partners with Teradata. Tableau partners with Teradata for a while and Spotfire did it a month ago, so I guess Qliktech trying to catchup in this regard as well. I mentioned it only to underscore the point of this blog post: Qliktech realized that it behind its competitors in some areas and it has to follow ASAP.

Today TIBCO announced Spotfire 5, which will be released in November 2012. Two biggest news are the access to SQL Server Analysis Services cubes and the integration with Teradata “by pushing all aggregations, filtering and complex calculations used for interactive visualization into the (Teradata) database”.

Spotfire team “rewrote” its in-memory engine for v. 5.0 to take advantage of high-capacity, multi-core servers. “Spotfire 5 is capable of handling in-memory data volumes orders of magnitude greater than the previous version of the Spotfire analytics platform” said Lars Bauerle, vice president of product strategy at TIBCO Spotfire.

Another addition is “in-database analysis” which allows to apply analytics within the database platforms (such as Oracle, Microsoft SQL Server and Teradata) without  extracting and moving data, while handling analyses on Spotfire server and returning result sets back to the database platform.

Spotfire added new Tibco Enterprise Runtime for R, which embeds R runtime engine into the Spotfire statistical server. TIBCO claims that Spotfire 5.0 scales to tens of thousands of users! Spotfire 5 is designed to leverage the full family of TIBCO business optimization and big data solutions, including TIBCO LogLogic®, TIBCO Silver Fabric, TIBCO Silver® Mobile, TIBCOBusinessEvents®, tibbr® and TIBCO ActiveSpaces®.

The Mass Technology Leadership Council (MassTLC) organized today the Data Visualization Panel in their series of “Big Data Seminars”:

http://www.masstlc.org/events/event_details.asp?id=243502

and they invited me to be a Speaker and Panelist together with Irene Greif (Fellow @IBM) and Martin Leach (CIO @Broad Institute). Most interesting about this event was that it was sold out and about 150 people came to participate, even it was most productive time of the day (from 8:30am until 10:30am). Compare with what I observed just a few years ago, I sensed the huge interest to Data Visualization, base on multiple, very interesting and relevant questions I got from event participants.

TIBCO said Spotfire 4.5 will be available later this month (May 2012).

Among news and additions to Spotfire: it will include ADS connector to Hadoop, integration with SAS, Mathworks and Attivio engines and new deployment kit for iPad.

In unusual, interesting (what it means? is it promising or what?) move the two Data Visualization leaders (Panopticon and Qliktech) partners today, see

http://panopticon.com/Panopticon-Software-Partners-with-QlikTech-to-Provide-Real-Time-Visual-Data-Monitoring-and-Analysis-Dashboards

“to offer enhanced, real-time visualization capabilities for the QlikView Business Discovery platform”.

Panopticon’s press-release looks overly submissive to me:

“As a member of QlikTech’s Qonnect Partner Program for Technology Partners, Panopticon supports QlikView desktop, web, and mobile interactive dashboards and allows users to filter and interact directly with real-time data. By integrating Panopticon into their systems, QlikView users can:

The combined Panopticon-QlikView platform is now available for immediate installation.”

Panopticon integration into QlikView dashboards utilizes QlikView UI extension objects within the web browser. The extension object calls Panopticon “web parts” and creates a Panopticon extension object with a number of pre-defined properties. The defined context/data is passed into the Panopticon extension object. The Panopticon “web parts” call a Panopticon EX Java applet and renders the requested Panopticon visualization workbook within the context defined by the QlikView user. The Panopticon component executes parameterized URL calls and parameterized JavaScripts to update the parent QlikView display.

Qliktech is trying to be politically correct and its Michael Saliter, Senior Director Global Market Development – Financial Services at QlikTech said, “Our partnership with Panopticon allows us to incorporate leading real-time visualization capabilities into our QlikView implementations. We recognize the importance of providing our clients with truly up-to-date information, and this new approach supports that initiative. Our teams share a common philosophy about proper data visualization design. This made it easy to develop a unified approach to the presentation of real-time, time series, and static data in ways that people can understand in seconds.”

While I like when competitors are cooperating (it benefits users and hopefully improve sales for both vendors), I still have a question: Qliktech got a lot of money from IPO, had a lot of sales and hired a lot of people lately; why they (Qlikview Developers) was not able to develop real-time functionality themselves?

Hugh Heinsohn, VP of Panopticon, said to me: “we (Panopticon) don’t see ourselves as competitors – and neither do they (Qliktech). When you get into the details, we do different things and we’re working together closely now”

Another indirect sign of relationship between Panopticon and Qliktech is the recent inclusion of Måns Hultman, former CEO of QlikTech into the list of advisors for Panopticon’s Board of Directors.

Other questions are rising too: if Qliktech suddenly is open to integration with Panopticon, why not to integrate with Quantrix and R Library (I proposed integration with R a while ago). Similar questions applicable to Tableau Software…

I was silent for a while for a reason: I owe myself to read a big pile of books, articles and blog posts by many authors – I have to read it before I can write something myself. List is huge and it goes many weeks back! I will sample a sublist  here with some relatively fresh reading materials in no particular order:

1. Excellent “Clearly and Simply” blog by Robert Mundigl, here are just 2 samples:

2. Interesting site dedicated to  The Traveling Salesman Problem:

3. Excellent QV Design blog by Matthew Crowther, here are a few examples:

4. Good article by James Cheshire here:

5. Interesting blog by Josh Tapley: http://data-ink.com/

6. A must read blog of Stephen Wolfram, just take a look on his 2 last posts:

7. Nice post by my friend John Callan: http://community.qlikview.com/blogs/theqlikviewblog/2012/03/09/why-discovery-really-matters

8. I am trying to follow David Raab as much as I can:

9. As always, interesting articles from Timo Elliott:

10. Huge set of articles from variety of Sources about newly released or about to be released xVelocity, PowerPivot2, SQL Server 2012, SSDT (SQl Server Data Tools), VS11 etc.

11. Here is a sample of article with which I disagree (I think OBIEE is TWO generations behind of Qlikview, Tableau and Spotfire), but still need to read it:

http://www.projectedconsulting.com/index.php/component/wordpress/2012/03/qlikview-versus-bi-applications-and-obiee

this list is go on and on and on, so answer on my own question is: to read!

Below is a prove (unrelated to Data Visualization, but cannot resist to publishing it – I did the spreadsheet below by myself) – rather for myself, that reading can help to avoid mistakes (sounds funny, I know). For example if you will listen last week’s iPropaganda from iChurch, you will think that new iPad 2012 is the best tablet on market. But if you read carefully specification of new iPad 2012 and compare it (after careful reading) with specifications of new Asus Transformer Pad Infinity, you will have a different choice:

Dan Primack, Senior Editor at Fortune, posted today at http://finance.fortune.cnn.com/2012/02/22/tableau-to-ipo-next-year/ a suggestion that Tableau can go public next year and I quote:

“Scott Sandell, a partner with New Enterprise Associates  (the venture capital firm that is Tableau’s largest outside shareholder) told Dan “that the “board-level discussions” are about taking the company public next year, even though it has the numbers to go out now if it so chose. Sandell added that the company has been very efficient with $15 million or so it has raised in VC funding, and that it shouldn’t need additional pre-IPO financing”.

Mr. Primack also mentioned an “unsolicited email, from outside spokesman: “Next week Tableau Software will announce its plans to go IPO“…

I do not have comments, but I will not be surprised if somebody will buy Tableau before IPO… Among potential buyers I can imagine:

  • Microsoft (Seattle, Multidimensional Cubes, integration with Excel),
  • Teradata (Aster Data is in, front-end for “big data” is needed),
  • IBM (if you cannot win against the innovator, how about buying it),
  • and even Oracle (everything moving is the target?)…

This is a repost from my Tableau-dedicated blog: http://tableau7.wordpress.com/2012/01/17/tableau-7/

2011 was the Year of Tableau with almost 100% (again!) Year-over-Year growth ($72M in sales in 2011, see interview with Christian Chabot here: http://www.xconomy.com/seattle/2012/01/27/tableaus-10th-year/ ), with 163+ new employees (total 350 employees as of the end of 2011) – below is the column chart I found on Tableau’s website:

and with tremendous popularity of Tableau Public and Tableau Free Desktop Reader. In January 2012 Tableau Software disclosed the new plan to hire 300 more people in 2012, basically doubling its size in 2012 and all of these are great news!

Tableau 7.0 is released in January 2012 with 40+ new cool features, I like them, but I wish 4+ more “features”. Mostly I am puzzled what wizards from Seattle are thinking when they released (in 2012!) their Professional Desktop Client only as a 32-bit program?

Most interesting for me is the doubling of the performance and the scalability of Tableau Server with 100+ users deployments (while adding multi-tenancy, which is the sign of the maturing toward large enterprise customers):

and adding “Data Server” features, like sharing data extracts (Tableau-optimized DB-independent file containers for datasets) and metadata across visualizations (Tableau applications called workbooks), automatic (through proxies) live reconnection to datasources, support for new datasources like Hadoop (since 6.1.4) and Vectorwise and new “Connect to Data” Tab:

Tableau’s target operating system is Windows 7 (both 64-bit and 32-bit but for Data Visualization purposes 64-bit is the most important), Tableau rightfully claims to complement Excel 2010 and PowerPivot (64-bit again), Access 2010 (64-bit), SQL Server 2012 (64-bit) and their competitors are supporting 64-bit for a while (e.g. Qlikview Professional has both 64-bit and 32-bit client for years).

Even Tableau’s own in-memory Data Engine (required to be used with Tableau Professional) is the 64-bit executable (if running under 64-bit Windows). I am confused and hope that Tableau will have 64-bit client as soon as possible (what is a big deal here? don’t explain, don’t justify, just do it! On Tableau site you can find attempts to explain/justify, like this: “There is no benefit to Tableau supporting 64-bit for our processing. The amount of data that is useful to display is well within the reach of 32 bit systems” but it was not my (Andrei’s) experience with competitive tools). I also noticed that under 64-bit Windows 7 the Tableau Professional client is  using at least 4 executables: 32-bit tableau.exe (main Tableau program), 64-bit tdeserver64.exe (Tableau Data Engine) and two 32-bit instances of Tableau Protocol Server (tabprotosrv.exe ) – looks strange (at least) to me…

You also can find on Tableau’s site users are reporting that Tableau 6.X underuses multi-core processors: “Tableau isn’t really exploiting the capabilities of a multi-core architecture, so speed was more determined by relative speeds of one core of a core 2 duo vs 1 core of an i7 – which weren’t that different, plus any differences in disk and memory speed“. Good news: I tested Tableau 7.0 and it uses multi-core CPUs much better then 6.X !

Of course, most appealing and sexy new features in Tableau 7.0 are related to mapping. For example I was able quickly create Filled Map, showing the income differences between states of USA:

Other mapping features include wrapped maps, more synonyms and mixed mark types on maps (e.g. PIE instead of BUBBLE), the ability to edit  locations and add new locationsas well as using Geography as Mark(s), like I did below:

etc.

Tableau 7.0 supports new types of Charts (e.g. finally Area Charts) and has new Main Menu, which actually causes a lot of changes where user can find menu items, see it here: http://kb.tableausoftware.com/articles/knowledgebase/new-locations

Tableau added many analytical and convenience features for users, like parameter-based Ref.lines, Top N filtering and Bins, Enhanced Summary Statistics (e.g. median, deviation, quartiles, kurtosis and skewness are added):

Trend models are greatly improved (added t-value, p-value, confidence bands, exponential trends, exporting of trends etc.). Tableau 7.0 has now 1-click and dynamic sorting, much better support for tooltips and colors.

I hope Tableau will implement my other 3+ wishes (in addition to my wish to have 64-bit Tableau Professional “client”) and will release API, will support the scripting (Python, JavaScript, VBScript, PowerShell, whatever) and will integrate with R Library as well.

I never liked pre-announcements of “new” products, especially if they are in state which will screw my PCs. But almost everybody doing it to us, starting with Microsoft SQL Server 2012 (Denali can be downloaded as “CTP3″), Tableau 7.0, Qlikview 11 (Qliktech partners and customers can download “Release candidate”) etc. Just a few months after releasing Spotfire 3.3, TIBCO announced that Spotfire 4.0 will be available in November 2011 with a lot of new features.

.

Some of them sound like buzzwords: “”free dimensional” analytics, collective intelligence, visual and social data discovery etc.” (we need that marketing will brainwash us, right?), but some of them can be very useful, like integration with TIBBR (that I like; in fact TIBCO has many other good products and they should be integrated with Spotfire) and SharePoint (sounds like a M$ bending to me, I don’t see too much DV money coming from SharePoint hole), support for dynamic icons, sparklines,

stepped linecharts, pop-over filters and legends, better font management, embedded actions and more. Some features I wish will be added, but I guess we need to wait more: I wish to be able to read with Spotfire the SSAS and PowerPivot multidimensional Cubes and support for some other Data Sources, like Tableau 6.1 does…

.

Spotfire and its Web Player Server support  now the latest web browsers, .NET 4.0 and it dropped support for obsolete stuff like Internet Explorer 6 and Windows 2003 Server. I mentioned on this blog earlier that I like Spotfire Silver 2.0 and the wealth and depth of Spotfire Analytical Platform (S-Plus, Miner, S+FinMetrics, Spotfire Developer/API, Statistics, Data and Automation Services, Metrics, Network Analysis, Decision Site, Clinical Graphics and more, this list should make Qliktech and Tableau worry or at least try to add similar features…).

.

Spotfire updated their set of Demos to reflect Spotfire 4.0 features: Spotfire Demos and Templates. More to come later, especially when Spotfire 4.0 will be Released (as oppose to be announced).

Qlikview 11

is announced on 10/11/11 – one year after 10/10/10, the release date of Qlikview 10! Qliktech also lunched new demo site with 12 demos of Qlikview 11 Data Visualizations: http://demo11.qlikview.com/ . Real release happened (hopefully) before end of 2011, my personal preference for release date will be 11/11/11 but it may be too much to ask…

QlikView 11 introduces the comparative analysis by enabling the interactive comparison of user-defined groupings. Also now with comparative analysis business users have the power of creating any (own) data (sub)sets and decide which dimensions and values would define the data sets. Users can then view the data sets they have created side by side in a single chart or in different charts:

Collaborative Data Visualization and Discovery.

Also Qlikview 11 enables Collaborative Workspaces – QlikView users can invite others – even those who do not have a license – to participate in live, interactive, shared sessions. All participants in a collaborative session interact with the same analytic app and can see others’ interactions live, see

QlikView users can engage each other in discussions about QlikView content. A user can create notes associated with any QlikView object. Other users can then add their own commentary to create a threaded discussion. Users can capture snapshots of their selections and include them in the discussion so others can get back to the same place in the analysis when reviewing notes and comments. QlikView captures the state of the object (the user’s selections), as well as who made each note and comment and when. Qliktech’s press release is here:

http://www.qlikview.com/us/company/press-room/press-releases/2011/en/1011-qliktech-introduces-social-business-discovery-in-launch-of-qlikview-11

“Our vision for QlikView 11 builds on the fact that decisions aren’t made in isolation, but through social exchanges driven by real-time debate, dialog, and shared insight,” says Anthony Deighton, CTO and senior Vice President, Products at QlikTech. “QlikView 11’s social business discovery approach allows workgroups and teams to collaborate and make decisions faster by collectively exploring data, anywhere, anytime, on any device. Business users are further empowered with new collaborative and mobile capabilities, and IT managers will appreciate the unified management functionality that allows them to keep control and governance at the core while pushing usage out to the edges of the organization.”

New Features in Qlikview 11

Qlikview now is integrated (I think it is a big deal) with TFS – source control system from Microsoft. This makes me think that may be Donald Farmer (he left Microsoft in January 2011 and joined Qliktech) has an additional assignment to make it possible for Microsoft to buy Qliktech? [Dear Donald - please be careful: Microsoft already ruined ProClarity and some others after buying them]. Free QlikView 11 Personal Edition will be available for free download by the end of year at www.qlikview.com/download.

Also if you will check Demo “What is new in Qlikview 11″ here:
http://us.demo11.qlikview.com/QvAJAXZfc/opendoc.htm?document=Whats%20New%20in%20QlikView11.qvw&host=demo11&anonymous=true , you can find the following new features:

  • mentioned above Comparative Analysis
  • Collaborative Data Visualization
  • integration with TFS
  • granular chart dimension control.
  • Conditional Enabling (dynamic add/remove) dimensions and/or expressions/metrics
  • Grid Container to show multiple objects, including another containers
  • Metadata for Charts: annotations, tips, labels/keywords, comments, mouse-over pop-up labels
  • some new actions (including Clear Field)

Do you want the 1st class Data Visualization on your cool Mac without any Virtual Machine with Windows? If so, your best choice will be the Omniscope 2.6 which is finally about to be released (after more then 2 years of delays) by Visokio, located in UK. Of course the Omniscope will run on Windows (most customers use it on Windows anyway) too: all it needs is Java (if needed, a private copy of Java will be installed on your computer as part of Omniscope package). You can get Omniscope Viewer on Linux workstation as well but if you need a full Omniscope 2.6 on Linux, you will have to ask Visokio about special license for you.

Java  was the problem for me, when I first heard about Omniscope, but more about that in a Special note at the end of this post. Visokio is a tiny company, started in 2002. Because of its size and private funding it took 3 years to release Omniscope 1.0 in 2005 and another 4 years to release Omniscope 2.5 in 2009,

which is what Visokio currently is still shipping. Visokio obviously have rich customers in financial (13+ clients), publishing and marketing(10+), and many other  industries and some of them in love with Apple’s Macs, but most customers prefer Windows. Omniscope is a Desktop Java application but completely integrated with internet. It has 4 editions (in both 32-bit and 64-bits versions), which are identical as far a deployment file-set concern, so all you need is buy an appropriate license. The installation process requires about 5 clicks, and user can get started by simply dragging in an Excel file and data will immediately appear and can be explored organically.


Omniscope Editions: Viewer, Desktop, Server, Server Plus.

Free Viewer allows server-less distribution of all Data Visualizations and interact fully (explore, select, filter and drill-down among other interactions) with all data, charts and reports, which are all can be easily exported to PDF, PPT, XLS and JPG files. Omniscope has zero-install “Web Start online version of free Viewer.

Omniscope Desktop/Professional ($4000 with discount for volume orders) in addition to all Viewer functionality, acts as a Development Studio for Data Visualizations (so called IOK applications are secure and compressed files, ready for easy internet delivery) and as a ETL wizard (using Drag-and-Drop Data Manager) for data:

Omniscope Desktop creates, edits and continuously refreshes all involved datasets, formulas, filters, views, layouts, even assumption-driven models, designs and export interactive Flash Data Players, embeddable into websites and into documents. Desktop able to read multidimensional cubes, just like Tableau and PowerPivot, which is a big advantage over Qlikview and Spotfire.

Omniscope Server (about $16000) adds to Desktop functionality: enables 64-bit IOK files behave (even remotely) as Central Datamarts (multi-source data assembly), as Timeslices (auto-refreshable proxies for datasources: one per each datasource), as Master Report IOK (automatically refreshed from Central Datamart IOK) and as Distributed Report IOK(s) (automatically distributed and live-refreshed from Master Report IOK), automates the refreshing of data, enables batch and scheduled distribution of customized IOK files.

Server Plus (about $24000) includes all Server functionality and adds ability to empower selected actions in free Omniscope Viewers (e.g. continuous data refreshing from Datamart IOK files, export to XLS, PPT, PDF, add/edit/save comments and queries etc.), permits unrestricted publishing of IOK visualizations, enables white labeling and branding Viewers and IOK files to customers specifications, allows multiple servers work as one.

Data Engine.

Omniscope is using in-memory Columnar Database, as all best Data Visualizers do but its architecture is different. For example, all datasets are collection of Cells (organized in column, rows and tables). Each Cell with String or Text is a separate Java Object and it leads to a large overhead in terms of memory usage (I always blame Java, which allows only 1.2GB of addressable memory for 32-bit Windows). Some usage statistics prompting that 32-bit Omniscope Desktop/Professional thinks that 5 millions cells is a large dataset and 15 millions cells is a very large dataset. According to Visokio, average client data file is around 40 fields and 50,000 records (2 million cells).

With Omniscope 2.6, experts from Visokio was able to run on 32-bit Windows PC (with 2GB of RAM) the Data Visualization with 70 millions of cells. For comparison with Qlikview I was able to fit 600+ millions of (data) cells into the same 32-bit PC, basically 9 times more data then with Omniscope and overall Omniscope is slower then competitors. As of now, Omniscope will try to use as much memory as possible in order to accelerate performance. I expect in near future the version of Omniscope with large performance and memory management improvements.

64-bit Installations of Omniscope are far more scalable, for example with 8GB of RAM 120 millions of cells was not a problem; largest known installation of Omniscope has 34 million Rows (about half of billion of cells) running on 64-bit Windows/Java PC with 16GB of RAM

In Omniscope 2.6, the DataManager can be used as an entirely new and independent application, allowing you to create and automate ETL workflows, without even loading data into the classic Omniscope interface.  You can visually drag sources in, append and merge, and transform with a variety of powerful operations such as Field Organiser which allows you to add formulas.  You can then publish, including a Batch Publisher which allows you to specify commands in another IOK file, such as “Publish [this subset] to [email] using [this view template]“, etc.

For full list of Omniscope features please check this: http://www.visokio.com/omniscope-features and for new features in version 2.6 please review this: http://www.visokio.com/omniscope-new-in-2-6 .

The original foundation of exportable Flash DataPlayer “generation” was totally re-written (for Omniscope 2.6) in ActionScript 3, which increased the scalability of DataPlayer  and added new view types/features. DataPlayers available as an experimental feature in Omniscope 2.6, and fully feature-complete in Omniscope 2.7 (I personally think that the time for Flash is gone/over and it is time to port DataPlayers into HTML5).

Visokio is confident that Omniscope 2.7 will come soon after release of Omniscope 2.6 and it will be integrated with super-popular Open Source Statistical R Library, and hopefully will contain HTML5-based DataPlayer, integration with Salesforce etc. If customers will demand, I also expect the Linux version of Omniscope at some future point.

By the way, my recent Poll is confirming that Omniscope is among Data Visualization Leaders and it got respectable 6% of votes so far! You can vote on this poll, just click here!

Special Note about Java.

While Java gave Omniscope the unique ability to run everywhere, it also gave a performance disadvantage to it, compare with my favorites Qlikview, Spotfire, Tableau and PowerPivot (all 4 written as native Windows applications).

Spotfire Silver version 2.0 is available now on https://silverspotfire.tibco.com/us/home and it will be officially announced at TIBCO User Conference 2011 (9/27-9/29/11) at http://tucon.tibco.com/

Spotfire Silver available in 4 Editions, see Product Comparison Chart here: https://silverspotfire.tibco.com/us/product-comparison-chart and Feature List at Feature Matrix here: https://silverspotfire.tibco.com/us/get-spotfire/feature-matrix

Update 9/27/11: TIBCO officially released Silver 2.0, see http://www.marketwatch.com/story/tibco-unveils-silver-spotfire-20-to-meet-growing-demand-for-easy-to-use-cloud-based-analytics-solutions-2011-09-27 “TIBCO Silver Spotfire 2.0 gives users the ability to embed live dashboards into their social media applications, including business blogs, online articles, tweets, and live feeds, all without complex development or corporate IT resources… Overall, the software’s capabilities foster collaboration, which allows users to showcase and exchange ideas and insights — either internally or publicly. In addition, it allows users to share solutions and application templates with customers, prospects, and other members of the community.”

Spotfire Silver Personal Edition is Free (Trial for one year, can be “renewed” with other email address for free) and allows 50MB (exactly the same amount as Tableau Public) and allows 10 concurrent read-only web users of your content. If you wish more then Personal Edition you can buy Personal Plus ($99/year) or Publisher ($99/month or $1000/year) or Analyst ($399/month) Account.

In any case you will GET for your Account needs a real Spotfire Desktop Client and worry-free and hassle-free web hosting (by TIBCO) of your Data Visualization applications – you do not need to buy any hardware,  software or services for web hosting, it is all part of your Spotfire Silver account.

To test Spotfire Silver 2.0 Personal Edition I took Adventure Works dataset from Microsoft (60398 rows, which is 6 times more than Spotfire’s own estimate of 10000 rows for 50MB Web storage). Adventure Works dataset  requires 42MB as Excel XLS file (or 16M as XLSX with data compression) and only 5.6MB as Spotfire DXP file (Tableau file took approximately the same disk space, because both Spotfire and Tableau are doing a good data compression job). This 5.6MB size of DXP file for Adventure Works is just 11% of web storage allowed by Spotfire (50MB for Personal Edition) to each user of free Spotfire Silver 2.0 Personal Edition.

Spotfire Silver 2.0 is a very good and mature Data Visualization product with excellent Web Client, with Desktop Client development tool and with tutorials online here: https://silverspotfire.tibco.com/us/tutorials . Functionally (and Data Visualization-wise) Spotfire Silver 2.0 has more to offer then Tableau Public. However Tableau Public account will not expire after 1 year of “trial” and will not restrict number of simultaneous users to 10.

Spotfire Silver 2.0 Publisher and Analyst Accounts can compete successfully with Tableau Digital and they have much clear licensing then Tableau Digital (see http://www.tableausoftware.com/products/digital#top-10-features-of-tableau-digital ), which is based on number of “impressions” and can be confusing and more expensive then Spotfire Silver Analyst Edition.

Today Tableau 6.1 is released (and client for iPad and Tableau Public for iPad), that includes the full support for incremental Data updates whether they are scheduled or on demand:

New in Tableau 6.1

  • Incremental Data updates scheduled or on demand
  • Text parser faster, can parse any text files as data source (no 4GB limit)
  • Files larger than 2GB can now be published to Tableau Server (more “big data” support)
  • Impersonation for SQL Server and Teradata; 4 times faster Teradata reading
  • Tableau Server auto-enables touch, pinch, zoom, gesture UI for Data Views
  • Tableau iPad app is released, it browses and filters a content on Server
  • Any Tableau Client sees Server-Published View: web browser, mobile Safari, iPad
  • Server enforces the same (data and user) security on desktop, browser, iPad
  • Straight links from an image on a dashboard, Control of Legend Layout etc.

Here is a Quick demo of how to create Data Visualization with Tableau 6.1 Desktop, how easy to publish it on Tableau server 6.1 and how it is instantly visible, accessible  and touch optimized on the iPad:

 

New since Tableau 6.0, more then 60 features, including:

  • Tableau now has in-memory Data Engine, which greatly improves I/O speed
  • Support for “big” data
  • Data blending from multiple sources
  • Unique support for local PowerPivot Multidimensional Cubes as Data Source
  • Support for Azure Datamarket and OData (Open Data Protocol) as Data Sources
  • Support for parameters in Calculations
  • Motion Charts and Traces (Mark History)
  • In average 8 times faster of rendering of Data Views (compare with previous version)

Tableau Product Family

  • Desktop: Personal ($999), Professional ($1999), Digital, Public.
  • Server: Standard, Core Edition, Digital, Public Edition.
  • Free Client: Web Browser, Desktop/Offline Tableau Reader.
  • Free Tableau Reader enables Server-less distribution of Visualizations!
  • Free Tableau Public served 20+ millions visitors since inception

Tableau Server

  • Easy to install: 13 minutes + optional 10 minutes for firewall configuration
  • Tableau has useful command line tools for administration and remote management
  • Scalability: Tableau Server can run (while load balancing) on multiple machines
  • Straightforward licensing for Standard Server (min 10 users, $1000/user)
  • With Core Edition Server License: unlimited number of users, no need for User Login
  • Digital Server Licensing based on impressions/month, allows unlimited data, Tableau-hosted.
  • Public Server License: Free, limited (100000 rows from flat files) data, hosted by Tableau.

Widest (and Tableau optimized) Native Support for data sources

  • Microsoft SSAS and PowerPivot: Excel Add-in for PowerPivot, native SSAS support
  • Native support for Microsoft SQL Server, Access, Excel, Azure Marketplace DataMarket
  • Other Enterprise DBMSes: Oracle, IBM DB2, Oracle Essbase
  • Analytical DBMSes: Vertica, Sybase IQ, ParAccel, Teradata, Aster Data nCluster
  • Database appliances: EMC/GreenPlum, IBM/Netezza
  • Many Popular Data Sources: MySQL, PostgreSQL, Firebird, ODBC, OData, Text files etc.

Some old problems I still have with Tableau

  • No MDI support in Dashboards, all charts share the same window and paint area
  • Wrong User Interface (compare with Qlikview UI) for Drilldown Functionality
  • Tableau’s approach to Partners is from stone ages
  • Tableau is 2 generations behind Spotfire in terms of API, Modeling and Analytics

Microsoft finally released SQL Server 11 “Denali” as CTP3 (Community Technology Preview) for public … Preview. Microsoft is (these are politeness words I can type) stubbornly refusing to have/build own Data Visualization Product. I doubt Crescent “experience” can be considered as a product, especially because it is Silverlight-base, while world already moved to HTML5.

If you have 7 minutes, you can watch Crescent Demo from WPC11, which is showing that while trailing a few years behind DV Leaders and Google, Microsoft is giving to its die hard followers something to cheer about:

I have to admit, that while there is nothing new (for DV expert) in video above, it is a huge progress compare with Excel-based Data Visualizations, which Microsoft tried to promote as a replacement of ProClarity and PerformancePoint Server. Even Microsoft itself positions Crescent (which is 32-bit only!) as a replacement for SSRS Report Builder, so DV Leaders can sleep well another night.

However, Microsoft’s BI Stack is the number 4 or 5 on my list of DV Leaders and CTP3 is so rich with new cool functionality, that it deserves to be covered on this blog.

Of course major news is availability of Tabular Data Model, which means VertiPaq in-memory columnar Engine, similar to PowerPivot Engine but running on Server without any SharePoint (which is a slow virus, as far as I am concerned) and without stupid SharePoint UI and limitations and I quote Microsoft: ” In contrast with the previous release, where VertiPaq was only available via in PowerPivot for SharePoint, you can now use VertiPaq on a standalone Analysis Services instance with no dependency on SharePoint.“!

SSAS (SQL Server Analysis Services) has new (they may existed before, but before CTP3 – ALL who knew that were under NDA) features like memory paging (allows models to be larger than the physical memory of the server, means unlimited scalability and BIG Data support), row level security (user identity used to hide/show visible data), KPI, Partitions; CTP3 removes the maximum 4GB file size limit for string storage file, removes the limit of 2 billion rows per table (each column is still limited to a maximum of 2 billion distinct values, but in columnar database it is much more tolerable restriction!).

New version of PowerPivot is released with support of  Tabular Model and I quote: “You can use this version of the add-in to author and publish PowerPivot workbooks from Excel 2010 to Microsoft SQL Server” and it means no SharePoint involvement again! As Marco Russo put it: “Import your existing PowerPivot workbooks in a Tabular project (yes, you can!)” and I agreed 100% with Marco when he said 4 times: Learn DAX!

After 3 years of delays, Microsoft is finally has BIDS for Visual Studio 2010  and that is huge too, I quote again: “The Tabular Model Designer … is now integrated with Microsoft SQL Server “Denali” (CTP 3) Business Intelligence Development Studio.” It means that BIDS now is not just available but is the main unified development interface for both Multidimensional and Tabular Data Models. Now we can forget about Visual Studio 2008 and finally use more modern VS2010!

Another extremely important for Data Visualization feature is not in SSAS but in SQL Server itself: Columnstore index is finally released and I a quote 1 more time again: “The … SQL Server (CTP 3) introduces a new data warehouse query acceleration feature based on a new type of index called the columnstore. This new index … improves DW query performance by hundreds to thousands of times in some cases, and can routinely give a tenfold speedup for a broad range of decision support queries… columnstore indexes limit or eliminate the need to rely on pre-built aggregates, including user-defined summary tables, and indexed (materialized) views. Furthermore, columnstore indexes can greatly improve ROLAP performance” (ROLAP can be used for real-time Cubes and real-time Data Visualizations).

All these cool SQL Server 11 new stuff is coming soon into Azure Cloud and this can be scary for any DV vendor, unless it knows (Tableau does; Qliktech and Spotfire still ignore SSAS) how to be friendly with Microsoft.

As we know now the newly coined by Microsoft term BISM (Business  Intelligence  Semantic Model) was a marketing attempt to have a “unified” umbrella

for 2 different Data Models and Data Engines: Multidimensional Cubes (invented by Mosha Pasumansky 15 years ago and the foundation for SSAS and MDX – SQL Server Analysis Services) and Tabular Model (used in PowerPivot and VertiPaq in-memory columnar Database with new DAX Language which is going to be very important for future Data Visualization projects).

New CTP3-released BIDS 2010 (finally almighty Visual Studio 2010 will have a “Business Intelligence Development Studio” after 3+ years of unjustified delays!) UI-wise will able to handle these 2 Data Models, but it is giving me a clue why Mosha left Microsoft for Google. And lack of DV product is a clue for me why Donald Farmer (face of Microsoft BI) left Microsoft for Qliktech.

Even more: if you need both Data Models to be present, you need to install 2 (TWO!) different instances of “Analysis Services”: one with Multidimensional Engine and one with new Tabular (VertiPaq/PowerPivot) Engine. It seems to me not as ONE “BI” architecture but TWO “BI” Architectures, interface-glued on Surface by BIDS 2010 and on back-end by all kind of Data Connectors. Basically Microsoft is in confused BI state now because financially it can afford 2 BI Architectures and NO Data Visualization Product!

I cannot believe I am saying this, but I wish Bill Gates back from retirement (it will be good for Microsoft shares and good for Microsoft market capitalization too – just ask Apple’s shareholders about Steve and they will say he is a god)!

Permalink: http://apandre.wordpress.com/2011/07/14/tabular-model/

TIBCO released Spotfire 3.3 and first (see what is new here) that jumped to my eyes was how mature this product is. For example, among new features is improved scalability – each additional simultaneous user of a web analysis initially claims very little additional system memory:

Many Spotfire customers will be able to support a greater number of web users on their existing hardware by upgrading to 3.3. Spotfire Web Player 3.3 includes significant improvements in memory consumption (as shown above for certain scenarios). Theoretically goal is to minimize the amount of system memory needed to support larger numbers of simultaneous users on the same analysis file. Main use case here: the larger the file and the greater the number of simultaneous web users on that file, then less initial system memory required to support each additional user: it is greatly reduced compared to version 3.2.1 and earlier.

Comparison with competition and thorough testing of new Spotfire scalability has to be done (similar to what Qliktech done with Qlikview here), but my initial reaction is as I said in a Title: we are witnessing a very mature software. Apparently the Defense Intelligence Agency (DIA) agrees with me and Defense Intelligence Agency Selects TIBCO Spotfire Analytics Solutions for Department of Defense Intelligence Information System Community. “With more than 16,500 military and civilian employees worldwide, DIA is a major producer and manager of foreign military intelligence”

Spotfire 3.3 also includes collaborative bookmarking, which enables all Spotfire users  to capture a dashboard – its complete configuration, including markings, drop down selections, and filter settings and share that visualization immediately with other users of that same dashboard, regardless of client in use. Spotfire actually not just a piece of Data Visualization Software, but a real Analytical Platform with large portfolio of products, including completely integrated S-PLUS (commercial version of R Library which has million of users), best Web Client (you can go Zero-footprint with Spotfire Web Player or/and partially free Spotfire Silver), free iPad Client version 1.1.1 (requires iTunes, so be prepared for Apple intrusion), very rich API, SDK, integration with Visual Studio, support of IronPython and JavaScript , well-thought Web Architecture, set of Extension Points etc.

System requirements for Spotfire 3.3 can be found here. Coincidentally with 3.3 Release Spotfire VAR Program got expansion too. Spotfire has a very rich set of training options, see it here. You can also find set of good Spotfire videos from Colin White’s Screencast Library, especially 2011 Webcasts.

My only and large concern with Spotfire is its focus, since it is part of a large corporation TIBCO, which has 50+ products and 50+ reasons to focus on something else. Indirectly it can be confirmed with sales: my estimate that Tableau is growing much faster than Spotfire (sales-wise) and Qlikview Sales probably 3 times larger (dollar-wise) than Spotfire sales. Since TIBCO bought Spotfire in 2007, I expected Spotfire will be integrated with other great TIBCO products, but after 4 years it is still not a case… And TIBCO has no reason to change its corporate policies, since its busines is good and stock is doing well:

(at least 500% increase of share price since end of 2008!). Also see article written by Ted Stamas for SeekingAlpha and comparison of TIBX vs. ETF here:

I think it is interesting to notice that TIBCO recently rejected a buyout offer from HP!

Last week of April 2011 was good for Qliktech. It released the results for a First Quarter 2011 and they are very positive.

Revenue is up (does not look like it is slowing down) 44% YoY, if compared with 1Q2010 with revenue $63M and projection for total 2011 now about $300M (up from preliminary projection of $280M before Q1 happened). Ended the first quarter of 2011 with an active customer count of approximately 19,000 (means about 900000 licensed, paying Data Visualization and BI users now and number of Qlikview users may exceed 1 million in 2011!), up from approximately 14,000 active customers at the end of the first quarter of 2010! Among other news:

  • Qliktech hired 103 new employees in Q1 of 2011 and currently employed 883 people (a 43% increase year-over-year).
  • Qliktech signed a strategic alliance with Deloitte, starting with Netherlands and planning expansion of alliance to Deloitte worldwide.
  •  About 2 weeks ago Qliktech unveils one of the first HTML5-based full client application: Qlikview on iPad (free [user will need license to access Qlikview Server anyway] – and delivered it through the Safari mobile Web browser) – Qliktech claims that it is “every bit as rich as a native app.”

I guess most of DV Client applications should have HTML5 reincarnation soon… As a result of all these positive sound bites, Qliktech shares ended this week above $32, more than tripled in 9 months:


and I compared Qliktech’s relative growth in above Annotated Timeline chart with Microstrategy, TIBCO and Apple (yes, Qliktech is growing at least twice faster than … Apple). I cannot include Tableau in comparison, because Tableau Software is still … a private company.

Qliktech’s capitalization as of today, 4/30/11 is $2.5B, $1B more than Microstrategy and only twice less than TIBCO’s capitalization. I know at least 3 software vendors, who are focused only on BI and DV: Tableau (it is still a private company; BTW, Tableau 6.1 will be released soon) – growing faster (114% YoY- see it here) than anybody, Qliktech (share price has tripled in last 9 months) and Microstrategy (it’s share price almost doubled in last 9 months). I consider the dedication to DV and BI as very important for future success in DV market; for example TIBCO’s Spotfire is only one of 50+ TIBCO’s products… and it dangers the future of one of the most advanced and mature DV products – Spotfire (version 3.3 is coming soon) .

One of reasons for Qliktech growth is its 1000+ partners and extensive Partner Programs for OEM Partners, Solution Providers, Business Consultants and System Integrators. Those overdeveloped Partner Programs required mandatory commitments from Partners in terms of Revenue Targets, Membership Fees, Qlikview Certifications and Minimum number of Trained employees. Lately Qliktech unreasonably raised those requirements and it may backfire and slowdown Qliktech growth and will help competitors like Tableau (Tableau actually opposite to Qliktech: their partnership program is underdeveloped – in my opinion – and requires big improvements) and recently Microstrategy (which seems learning from own and competitors mistakes and catching up lately).

Update 3 months later:

in Q2 of 2011 Qliktech reached 21000 customers worldwide (it means almost 1 million licensed users), $74 Millions in revenue (45% over Q2 2o1o); 1000 full time employes (400+ more compare with Q2 2010), $2.4B Market Capitalization and guess what – $2.2  Million of lost!

Permalink: http://apandre.wordpress.com/2011/04/30/good-week-for-qliktech/

Microstrategy is a famous and BI-dedicated company, operating for 22+ years, recently released Visual Insight (as part of the release of Microstrategy 9.2 this week) and joint the DV race. A couple of years ago, I advised to some local company in terms of choosing Data Visualization Partner and final 3 choices were Qlikview, Spotfire and Microstrategy. Microstrategy was most competitive pricing-wise, but their Data Visualization functionality was not ready yet. They are ready now, see it here (from webcast this week):

Visual Insight as part of Microstrategy 9.2 targets so called “self-service BI”, and transition (they acknowledged that) from “old BI” (tabular reports: published static and OLAP reports) to “new BI” (Data Visualization and Dashboards), from Desktop to Mobile Clients (that is a forward looking statement for sure), from Physical to Cloud.

Microstrategy is claiming that Visual Insight allows to visualize Data in 30 minutes (that is good to know, but DV Leaders already have it for a while, welcome to the club!) compare with 30 days for the same process with “traditional BI”:

(I am saying this for 6 years now and on this blog since inception of it; does it mean that old BI is useless now and too pricey? Microstrategy presenters saying that answer is yes! and I want to thank Microstrategy for the validation of my 6-years old conclusion). For full set of Microstrategy 9.2 slides click here.

Microstrategy 9.2 has a full BI product portfolio, fast in-memory Data Engine, free mobile and tablet clients, has even Free Reporting Suite . Microstrategy (like Qliktech, Tableau and Visokio) is completely focused on Business Intelligence and Data Visualization functionality unlike its giant competitors like SAP, IBM, Oracle and Microsoft.

Update 9/27/11. MIcrostrategy released free Cloud Personal edition, based on Visual Insight, see it for yourself here:

Pagos released this week SpreadsheetWEB 3.2 (PSW for short) with new Data Visualization features (Pagos Data Visualizer or PDV for short). Among those features is an ability to drill-down any Visible Data through synchronized filters, which immediately made the SpreadsheetWEB a player in Data Visualization Market.

Tools like Tableau, Qlikview or Spotfire allow people to visualize data, but have very limited ability to collect and update data. PSW (Pagos SpreadsheetWEB), on other hand, since versions 1.X was able to convert any Excel Spreadsheet into Web Application and Web-based Data Collector, to save collected data into SQL Server (including latest SQL Server 2008 R2) Database, and to Report or Visualize the Data online through SaaS web-based spreadsheet, which looks and behaves as Excel Spreadsheet! SpreadsheetWEB has unique ability to collect data in a Batch Process and run large datasets against SpreadsheetWEB application. This video demonstrates data collection and data management and collaborations utilizing workflow capabilities and SpreadsheetWEB Control Panel interface. SpreadsheetWEB can use Web-Service as Data Source (like Excel does) and allows web-based spreadsheets to function as Web Service too:

One of the reasons why most people still use and like Excel as a BI tool is that they can use many of the built-in worksheet formulas to process data in real-time while filtering the dashboard. SpreadsheetWEB converts those formulas and can execute them on the server. Database-driven SpreadsheetWEB applications support most features in Excel, including worksheet formulas, 333+ Excel functions, formatting, 33+ types of Excel charts as well as Sparklines,

also see video here:

as well as pivot tables, validation, comments, filters and hyperlinks, while almost completely eliminating the need for application and database developers, as well as need for IT services. Basically if person knows Excel, than he knows how to use SpreadsheetWEB. SpreadsheetWEB (both 64-bit and 32-bit) has HTML Editor and Scripting Support (JavaScript), similar to what macros do for Excel (be aware that it is not port of VBA):

Among 3 DV Leaders only Tableau is able to read Microsoft SQL Server Analysis Services (SSAS) data sources, which is a must for long-term success in Visual Analytics market. SpreadhseetWEB has this functionality the same way as Excel does and therefore ahead of Qlikview and Spotfire in this extremely important department. Among other advanced Data Visualization Features SpreadsheetWEB supports Maps in Dashboards

and multi-page Dashboard reports. I like Version Control for applications and Server Monitoring features – they can be very attractive for enterprise users. SpreadsheetWEB does not require SharePoint Server to execute Excel workbooks on the server. Pagos developed proprietary spreadsheet technology to achieve that independence from SharePoint Server (I personally consider SharePoint as a Virus). This makes Pagos very attractive to cost conscious small to medium size organizations. Installing SpreadsheetWEB only requires Windows Server and Microsoft SQL Server. In addition, SpreadsheetWEB works with free SQL Express Edition, which is an additional savings for Customers with small datasets.

For advanced Data Visualization functionality, Pagos established the OEM partnership with TIBCO and integrates SpreadsheetWEB with TIBCO Spotfire Analytic Platform. For advanced SaaS features, including strictest security and hosting requirements and SAS70 Compliance, Pagos partners with Rackspace.

SpreadsheetWEB is one of the few players in the market that offer Software-as-a-Service (SaaS) licensing along with traditional server licensing. Pagos has very attractive SaaS fees and extremely competitive pricing for those who want to buy own SpreadsheetWEB server: $4900 per SpreadsheetWEB server for 50 named users and 25 web applications and dashboards; that price at least 10 times better than prices from Qlikview, Spotfire and Tableau. Pagos provides 44+ Video Tutorials, 53+ online Demos, free non-expiring trial and Wiki-based full Documentation for SpreadsheetWEB, so people can review, browse and evaluate SpreadsheetWEB way before they will buy it.

Pagos is in BI business since 2002, profitable and fully self-funded since inception, with hundreds of customers. Pagos has other advanced BI-related products, like SpreadsheetLIVE (it offers a fully featured spreadsheet application environment within a web browser) and Pagos Spreadsheet Component (allows software developers to create web and desktop applications that can read, execute, and create Excel spreadsheets without requring Microsoft Excel). If you will compare SpreadsheetWEB with Microsoft’s own attempt to webify Excel and Microsoft’s own Long List of Unsupported Excel features, you can easily appreciate the significance of what Pagos achieved!

Permalink: http://apandre.wordpress.com/2011/03/13/spreadsheetweb/

I never saw before when one man moved from one company to another, then 46+ people will almost immediately comment on it. But this is what happened during last few days, when Donald Farmer, the Principal Program Manager for Microsoft BI Platform for 10 years, left Microsoft for Qliktech. Less than one year ago, Donald compared Qlikview and PowerPivot and while he was respectful to Qlikview, his comparison favored PowerPivot and Microsoft BI stack. I can think/guess about multiple reasons why (and I quote him: “I look forward to telling you more about this role and what promises to be a thrilling new direction for me with the most exciting company I have seen in years”) he did it, for example:

  • Microsoft does not have a DV Product (and one can guess that Donald wants to be the “face” of the product),
  • Qliktech had a successful IPO and secondary offering (money talks, especially when 700-strong company has $2B market capitalization and growing),
  • lack of confidence in Microsoft BI Vision (one can guess that Donald has a different “vision”),
  • SharePoint is a virus (SharePoint created a billion dollar industry, which one can consider wasted),
  • Qlikview making a DV Developer much more productive (a cool 30 to 50 times more productive) than Microsoft’s toolset (Microsoft even did not migrate the BIDS 2008 to Visual Studio 2010!),
  • and many others (Donald said that for him it is mostly user empowerment and user inspiration by Qlikview – sounds like he was underinspired with Microsoft BI stack so is it just a move from Microsoft rather then move  to Qliktech? – I guess I need a better explanation),

but Donald did explain it in his next blog post: “QlikView stands out for me, because it not only enables and empowers users; QlikView users are also inspired. This is, in a way, beyond our control. BI vendors and analysts cannot prescribe inspiration“. I have to be honest – and I repeat it again – I wish a better explanation… For  example, one my friend made a “ridiculous guess” that Microsoft sent Donald inside Qliktech to figure out if it does make sense to buy Qliktech and when (I think it is too late for that, but at least it is an interesting thought: good/evil  buyer/VC/investor will do a “due diligence” first, preferably internal and “technical due diligence” too) to buy it and who should stay and who should go.

I actually know other people recently moved to Qliktech (e.g. from Spotfire), but I have a question for Donald about his new title: “QlikView Product Advocate”. According to http://dictionary.reference.com/ the Advocate is a person who defends, supports and promotes a cause. I will argue that Qlikview does not need any of that (no need to defend it for sure, Qlikview has plenty of Supporters and Promoters); instead Qlikview needs a strong strategist and visionary

(and Donald is the best at it) who can lead and convince Qliktech to add new functionality in order to stay ahead of competition with at least Tableau, Spotfire and Microsoft included. One of many examples will be an ability to read … Microsoft’s SSAS multidimensional cubes, like Tableau 6.0 and Omniscope 2.6 have now.

Almost unrelated – I updated this page:  http://apandre.wordpress.com/market/competitors/qliktech/

Permalink: http://apandre.wordpress.com/2011/01/09/farmer_goes_2_qlikview/

It looks like honeymoon for Qlikview after Qliktech’s IPO is over. In addition to Spotfire 3.2/Silver, now we have the 3rd great piece of software in form of Tableau 6. Tableau 6.0 released today (both 32-bit and 64-bit) with new in-memory data engine (very fast, say 67 millions of rows in 2 seconds) and quick data blending from multiple data sources while normalizing across them. Data Visualization Software available as a Server (with web browsers as free Clients) and as a Desktop (Pro for $1999, Personal for $999, Reader for free).

New Data Sources include local PowerPivot files(!),  Aster Data ; new Data Connections include OData , (recently released) Windows Azure Marketplace Datamarket; Data Connection can be Direct/Live or to in-memory data engine. Tableau 6 does full or partial automatic data updates; supports parameters for calculations, what-if modeling, and selectability of Displaying fields in Chart’s axis; combo charts of any pair of charts; has new project views, supports Motion Charts

tumblr_mssaaxhajz1stz40uo1_500

(a la Hans Rosling) etc. Also see Ventana Research and comments by Tableau followers. This post can be expanded, since it is officially 1st day of release.

n009: http://wp.me/sCJUg-tableau6

BI and DV vendors do not want me to relax and keep releasing new stuff too often. I feel guilty now and I will (3+ months after it was released) comment on Spotfire 3.2 release soon. But today I have to comment on Cognos 10 Release (which will be available Oct. 30; everybody now does pre-announcement: 2 weeks ago Qlikview 10, yesterday BO4, today Cognos 10). I quote: “IBM acquired Cognos in early 2008 during a five year buying spree that saw it swallow over 24 analytics companies in five years for a total bill of US$14 billion”. Rob Ashe, general manager for BI at IBM, said: ““Analytics is a key part of our 2015 roadmap. Last year, analytics contributed $9 billion to our revenues, and we expect to see that grow to $16 billion in 2015.”

The Cognos 10 embeds SSPS and Lotus Connections, supports SaaS, active/interactive reports via email (no need to install anything), mobile devices such as iPhones, iPads and BlackBerrys (as well as Symbian phones, and Windows Mobile devices), real-time updates, has “modern” Web 2.0 user interface. Cognos TM1 (from Applix) is a multidimensional, 64-bit, in-memory OLAP engine which provides fast performance for analyzing complex and sophisticated models, large data sets and even streamed data.

Personally I think Cognos 10 compares favorably against BO4, SAS 9.2, OBIEE 11g , but all 4 have at least 2 common problems: they are all engaged too much with Java and they are far (of Qlikview, Spotfire, Omniscope, Tableau etc.) behind in Data Visualization

n006: http://wp.me/pCJUg-4Z


Tableau added 1500 new customers during last year (5500 total, also it is used by Oracle on OEM basis as Oracle Hyperion Visual Explorer), had $20M in sales in 2009, Q3 of 2010 showing 123% growth over the same period a year ago, claiming to be a fastest growing software company in BI market (faster than Qliktech), see http://www.tableausoftware.com/press_release/tableau-massive-growth-hiring-q3-2010

Tableau 6.0 will be released next month, they claiming it is 100 times faster than previous version (5.2) with in-memory columnar DB, 64-bit support and optional data compression. They are so confident (due increasing sales) so they put 40 job openings last week (they had 99 employees in 2009, 180 now and plan to have 200 by end of 2010). Tableau is raising (!) prices for their Tableau Desktop Professional from $1800 to $1999 in November 2010, while Personal will stay at $999. They aim directly at Qliktech saying (through their loyal customer) this: “Competitive BI software like QlikView from QlikTech is difficult to use without a consultant or IT manager by your side, a less than optimal allocation of our team’s time and energy. Tableau is a powerful tool that’s easy to use, built to last, and continues to impress my customers.”

In Tableau’s new sales pitch they claiming (among other 60 new features):

  • New super-fast data engine that can cross-tab 10 million rows in under 1 second
  • The ability to blend data from multiple sources in just a click
  • Create endless combination graphs such as bars with lines, circles with bars, etc.

n004: http://wp.me/pCJUg-3Z

Qliktech released as planned the new version 10 of Qlikview last week, see http://www.qlikview.com/us/company/press-room/press-releases/2010/us/1012-qlikview-10-delivers-consumer-bi-software and delivered a lot of new functionality, see

http://apandre.files.wordpress.com/2010/10/ds-whats-new-in-qlikview-10-en.pdf

to its already impressive list, like in-memory columnar database, the leading set of visual controls (pie/10, bar/7, column/7, line/6, combo/6, area/4, radar/4, scatter/5, bubble/3, heat-map/block/5, gauge/7, pivot/12, table/12, funnel/2, mekko, sparkline, motion charts etc.) totaling more than 80 different charts (almost comparable with Excel 2010 diversity-wise). Qlikview enjoying the position of the DV Leader in Data Visualization market for last few years, thanks to above functionality and to its charts, functioning as visual filters with interactive drill-down functionality, with best productivity for developers, with easiest UI and with multitude of clients (desktop, IE plugin, Java, ajax, most smartphones). Also take a look on this: http://www.ventanaresearch.com/blog/commentblog.aspx?id=4006 and this: http://customerexperiencematrix.blogspot.com/2010/12/qlikviews-new-release-focuses-on.html

Qliktech recently had a successful IPO and secondary offering,  see http://www.google.com/finance?q=Qlik which made capitalization of the Qliktech approaching $2B. DV competition is far from over: recently Qlikview got very strong competition from Spotfire 3.2, PowerPivot and upcoming (this or next month) releases of Tableau 6 and Omniscope 2.6. And don’t forget DV misleaders with a bunch of money, trying to catch-up: SAP, IBM, Oracle, Microsoft, Microstrategy, even Google and others trying very hard to be a DV contenders                                                                                (n002: http://apandre.wordpress.com/2010/10/19/qlikview10/)

Qliktech uses this Diagram to present its current set of Components and DataFlow between them:

QV10 Components and DataFlow.

Since I commented on recent releases of competing DV products (Qlikview, Tableau, Cognos, Business Objects etc.) I feel the need to post about Spotfire 3.2. For me the most important new feature in 3.2 is the  availability of all functionality of Spotfire THICK client in Spotfire 3.2 WebPlayer, specfically Spotfire WebPlayer now can do the same visual drill-down as Qlikview does for a while. Overall the 3.2  Release enabled Spotfire to catch-up with Qlikview and become a co-leader in DV market. Also SPotfire Clinical 3.2 was released, which enables Spotfire to connect with Oracle Clinical Databases. TIBCO Spotfire offers a unique memory-swapping or paging feature, which lets it analyze models that are larger than a single available memory space.

Among new features ability to export any Pages and Visualizations to PDF, improved integration with S-Plus and IronPython, ability to embed more than 4GB (actually unlimited) of application’s data into application file (and TIBCO Spotfire Binary Data Format file) and other improvements, like subtotals in Cross Table, SSO with NTLMv2 (Vista, Win7), Lists Tools and LDAP synchronization, Multiple Localizations for major Asian and European languages. Update on 11/2/10: TIBCO released Spotfire WebPlayer 3.2.1, which now fully supports iPad and its native multi-touch interface.

A few days later on 7/14/10, TIBCO released Spotfire Silver as a fully SaaS/ZFC version of Spotfire 3.2, designated for Self-Serviced BI users, who prefer to minimize their interactions with own IT/MIS departments. Spotfire Silver ahead of all DV competitors in terms of fully web-based but fully functional DV environment.

In case if users prefer behind-firewall Clustering and Fail-over configuration for Spotfire deployment, it may look like this:

n007=http://wp.me/pCJUg-5n

Follow

Get every new post delivered to your Inbox.

Join 288 other followers