Tableau in the Modern Data Stack

Tableau has been a leader in the business intelligence space in the last decade. It entered Gartner’s leaders quadrant in 2013 and remains one of the platforms with the greatest ability to execute and completeness of vision today.

The simplicity of the drag and drop functionality, combined with the lightning speed of the extracts (at a time when Redshift was a newcomer to the space) enticed analysts and decision-makers who finally had a modern, intuitive, fast, and trustworthy tool to better understand their business.

Tableau has continued to evolve, but so has the whole environment. The traditional ETL (Extract, Transform, Load) has been at the forefront of a transformation fueled by microservices, APIs and cloud computing. The paradigm has shifted to ELT, with a cloud data warehouse enabling the transformation engine to be the same as the BI data backend, and there have been countless new solutions in different stages of the data ecosystem. The modern data stack has been born, and, with it, a myriad of players offering specialised services. It is time to consider how well Tableau is keeping up with modern times and where it fits in the new stack.

What is the MDS

 

But first, what is the modern data stack (MDS)?

The modern data stack is more than a buzzword. It is a term coined to encompass a new way of working with data. Essentially, it is a combination of tools, technologies and principles applied to the whole data journey to facilitate analytics. It is:

  • Cloud-first;
  • Designed with the idea of facilitating analysis in mind, simplifying the process of ingesting data and favouring buy-over-build while remaining cost-effective;
  • Scalable and flexible, contributing to a modular design. Modular design allows you to choose the best-in-class at each step and avoid vendor lock-in while simultaneously facilitating iteration;
  • Designed to cater for both analysts and business users;
  • Born with governance in mind.

The MDS empowers organisations to become agile and cost-effective and encourages them to develop a data culture. Its modularity has enabled the rise of multiple players for each of the functions of the modern data stack:

  • Sources: databases, ERP, 3rd party APIs, logs;
  • Ingestion: using connectors in Fivetran, Stitch, Matilion, Xplenty;
  • Transformation: dbt, Dataform;
  • Orchestration: Airflow, Prefect, Dagster, Argo;
  • Storage and query: analytical databases like Snowflake, Bigquery, Redshift, Azure Synapse and even further disruptors like Firebolt on the horizon;
  • Output: Tableau, Looker, Mode;

We finally have the technology to enable an integrated pipeline that powers data-driven companies. Data analytics is ready to be democratised and yet the vast proportion of companies remain non-data-driven while self-service analytics remain aspirational. According to the Harvard Business Review, “cultural challenges — not technological ones — represent the biggest impediment around data initiatives”. These might include “organisational alignment, business processes, change management, communication, people skill sets, and resistance or lack of understanding”.

How tableau fits in the MDS

 

How does Tableau fit in the modern data stack?

First and foremost, the visual analytics capabilities of Tableau are without equal in the current landscape. When it comes to ad-hoc, iterative exploratory analysis (asking questions and getting quick answers that lead to new questions), Tableau is unparalleled.

Despite the MDS, probably not all of your data is going to be available in your data warehouse—and an even smaller segment of it is going to be modelled. Tableau’s agnosticism about data sources and the impressive amount of connectors available will make quick work of those analyses. This is especially useful in the initial stage of discovery and quick iteration or for facilitating real-time input from business users attached to spreadsheets.

Tableau as a company is well aware of the cultural change needed with the new data strategy and has invested heavily in different fronts to help companies implement it successfully. Their blueprint, designed to help guide companies who are at different stages in their data journey, stresses this:

“Realizing the full value of your data means empowering everyone to make better decisions with it, and this cannot be done simply by choosing the right technology. As you chart your course to becoming a data-driven organization with Tableau, it’s important to remember that you are not just deploying software—you are driving organizational transformation by prioritizing facts over intuition with data at the centre of every conversation.”

“Trust is built in drops and lost in buckets” K. Plank

Data and content governance are paramount for a successful data strategy. This is so for the whole MDS, but it is especially true for the self-service promise to business users. Business users, who may not fully understand the whole data pipeline, need to identify which data source to query in order to answer a given question. Are there any problems with that data source? When was the last time it was updated? Having certified data sources with data quality alerts and a clear lineage will help business users choose the right data and understand the limitations of the insights derived.

Integrating data governance with Tableau

 

Integrating data governance

Data is a commodity: regardless of your role and level, everyone needs to be able to transform data into information and, eventually, into knowledge. Agility in decision-making processes is crucial. Governance and lineage help companies to better fulfil their GDPR duties through traceback PII. They also empower employees from different departments to have a more self-service approach in answering business questions, such as How is that KPI built? Who defined it? What are the source systems behind it?

All of this requires having a group of data owners that maintain and curate the selected data sources: adding that visual checkmark and description that will guide business users or adding the data quality alert that will pop up in a dashboard in case of any issue.

Being able to tie this with the modelling layer can help the advanced user, usually the analytics engineer. For example, understanding the true lineage from the raw data source and system to the one published in Tableau Server; being aware of what dashboards and data sources will be impacted if the pipeline fails; and combining this knowledge with the use of an orchestration tool can update the alerts in real-time, i.e. automating data governance).

Tableau’s granular permission structure allows for different governance approaches at the same time: from keeping tight control of core projects with centralised governance to delegating department-owned projects or having self-governing spaces either for individual users or for mature organisations. Combined with the different APIs, Tableau Server makes using multiple environments (e.g. DEV, UA, PROD) a breeze. This is also possible, albeit slightly more convoluted, with Tableau Online, their hosted solution.

Modifying culture

 

Modifying culture

Data literacy is at the heart of Tableau’s mission and they have been very successful at recruiting the help of the massive community of users. Initiatives to use Tableau Public, their free product, with different datasets and objectives abound. Piggybacking on those initiatives is a sure way of finding internal champions for the product and eventually increasing adoption and engagement.

The modularity of the MDS is an advantage as it caters for progressive adoption. Finding champions inside different departments can boost the uptake. The combination with new tools like dbt makes it easier to handle complex modelling requests that would put off less savvy users. With a well-thought report and architecture design, on the one hand, Tableau is a much more manageable solution, and on the other, the fundamental model is available for other tools and applications downstream.

DataOps with Tableau

 

DataOps with Tableau

DataOps is a recurring theme that goes hand in hand with the MDS. Essentially, it is an automated, process-oriented methodology used by analytics and data teams to improve data analytics quality and reduce cycle time. It draws from DevOps principles and is applied to the whole lifecycle from data ingestion and transformation to analysis. It is analytics as code and where data and processes can be tested, reused and automated.

Tableau’s approach is to offer a simple, straightforward interface where permissions, settings, alerts, etc. can be set up with a click. They have put great effort into getting an intuitive UI that will resonate with the vast majority of users. The user experience is consistent whether you are designing a dashboard or setting up permissions on a project. They have built part of the components of the DataOps inside the tool by making sure that:

  • Different versions of the dashboard are stored inside Tableau Server’s database and can be restored with a single click;
  • Users can be assigned to groups with a single click;
  • Users can subscribe to a view and receive it every day in their emails with a single click.

As useful as single-click functionalities are, they are not well aligned with the core principles of DataOps. Fortunately, Tableau has also developed highly specialised APIs that will enable power users to automate and make processes reproducible. This opens new ways to collaborate and maintain governance in data and content where we can:

  • Promote content from a dev environment to UAT and prod, changing the underlying data accordingly along the way
  • Combine webhooks or commits with a validation step to make sure that new workbooks conform to the style guide and reject/alert if not;
  • Derive the data quality alerts if there are any errors in our series of data transformations (e.g. dbt run), which will be extremely helpful for business users.

Tableau advantages

 

Prospects

Tableau continues to be in a favourable position in the BI space. Its sleek UI appeals to business users while the versatility it offers through its dedicated set of APIs makes it a go-to solution among power users. Keeping the enterprise focus on governance, Tableau fully adheres to the spirit of MDS.

The acquisition last year by Salesforce has been followed by a tighter integration with predictive tools, and the recent acquisition of Slack promises increased collaboration and ease of distribution. Truly integrating the platform with the upstream MDS to fully apply the governance model requires a set of particular skills combining DevOps, Data engineering and analytics that is not immediately available to many companies but the benefits are definitely worth the effort.

We’d love to learn how you are using Tableau at your organisation and hear about any limitations you might be facing related to the tool or the use of DataOps in your data life cycle.

Get in touch, so we can talk about it now.

Don’t forget to check out the rest of the content on the Infinite Lambda blog too.

Share on facebook
Share on twitter
Share on linkedin

More on the topic