How to build a modern data stack

Being data-driven is essential to staying competitive. When your team can access the data they need, when they need it, better decisions are made, goals are clear and measurable, and a greater understanding of your customer, product, and processes is achieved. 

Once upon a time, getting access to the data you needed required filing a request, sending an SQL query, waiting days (or weeks), and finally receiving tangled data that was often incomplete or impossible to work with. 

All that has changed, thanks to the modern data stack. In this blog, we’ll explore what the modern data stack is, the problems it sets out to solve, and how you can build a modern data stack for your business. 

Modern data stack definition

A data stack is a suite of apps and technologies that are specifically built to store and organize data, and funnel it into a business with the aim of making that data actionable. While a traditional data stack typically included an on-premise data ecosystem and an SQL warehouse, a modern data stack includes cloud-based services and low or no code tools that turn all that data into actionable insights.

When a data stack is truly modern—more on what that means below—it will become a critical component of your business, with every team across the company benefitting. 

The impact of a modern data stack 

It’s highly likely you’re not starting from zero. You may have some data SaaS tools including a CRM, Google Analytics, and Google Sheets working away, but still, your business struggles to do things like build insights, report on attribution, access data in real-time for marketing personalization, or draw the data needed to make a case for innovations and experiments. 

Alex Halliday, the CEO and co-founder of AirOps, has seen first-hand how this kind of disjointed data setup can quickly lead to problems including: 

• Data is siloed in various tools and documents, so it can’t be easily accessed, analyzed, and used to advance the business.

• Without a unified data infrastructure, there are more opportunities for errors, like people working from an outdated Google Sheet with financial metrics that don't match the most recent data collected in Stripe.

• Employees don’t trust data, because it’s hard to tell what data is accurate and reliable.

• A disjointed, siloed data system isn’t scalable, so your technology and processes can’t grow as data volume, analyses, and overall complexity increase.

From sales to R&D, finance to human resources, a modern data stack has the potential to eliminate these issues in every department of your business, but none more so than marketing. With a data stack, marketers will be able to automate more, identify ways to eliminate advertising wastage, identify ideas and initiatives that are more likely to be successful, and prove ROI on new initiatives.

Marketing Director of Green Building Elements, Sarah Jameson, agrees, “In marketing, the data stack helps set the parameters for a model that fits with the business and is bound to work for customers. There are forms to collect meaningful data that could help with marketing: the buyer's journey, customer interactions, or your customer personas. These data, when properly managed and sorted, can help marketing teams identify new opportunities to launch the product or service into the market.”

Nik TeBockhorst Friedman, the VP of Solutions at McGaw.io adds, "A modern data stack helps you stay on top of the trends and regulations. Integrated tools even give you ways to track your users while respecting Apple's ITP and regulations such as GDPR.

The impact was clear in a recently published report from CMO Alliance, Data-Driven CMO Report 2022 survey respondents were asked to cite the most important components of their martech stack, and the top three all happened to be part of a modern data stack: 

• An effective CRM or a platform with inbuilt CRM processes

• Website analytics tool 

• Data visualization tool, to make it easier to share and compare data

In short, a modern data stack is essential to the day-to-day running of a business in 2022 and beyond.   

Four steps to building a modern data stack 

Follow these four steps to take the headache out of building a modern data stack: 

1. Data collection
   Solutions: May include a CRM, CDP, HelpDesk, payment tools, marketing automation and other sales/martech tools
   
   It’s likely your business is already using a wide range of SaaS tools and have multiple data sources that will need to be unified and organized. Take stock of everything you’re currently using and keep them top of mind as you weigh up your storage, ingestion, analytics, and activation tools.
   

2. Data loading and storage
   Solutions: Cloud-based data warehouse or data lake
   
   A cloud-based data warehouse will be the central storage place for all of your data, meaning it will be responsible for collecting, structuring, and unifying your data and sending it to the other tools in your data stack. Redshift, BigQuery and Snowflake are some of the most popular solutions out there as they are SQL-first platforms that offer flexible billing, efficient storage, and a wide ecosystem of integrations. 
   
   A data lake is a repository of data stored in its raw state. This means data is both structured and unstructured. Data lakes can store a huge volume of data efficiently (both cost and time), but given the nature of the data, you will require more maintenance.
   
   When assessing options, be realistic about the maintenance involved and how much time your engineers can invest, the costs involved, and the type of work you will be doing with your data. Many businesses will choose to use both a data lake and a data warehouse, especially if they have large volumes of data. The table below outlines the key differences between each.
   

   

3. Data transformation
   Solutions: ETL or ELT data ingestion tools
   
   A data ingestion tool will allow you to automatically extract data from various sources and stream it into a single tool to unify your data quickly. As a part of this process, raw data will be cleaned and business logic will be applied to ensure your data is usable.
   
   There are two main methods of ingestion — ETL (extract, transform, load) and ELT (extract, load, transform). The process you choose will depend on whether you require data lake support, the size, and type of your data set, and the technical skills available. In short, ELT is a simpler, more flexible, and low-maintenance solution. Check out the table below to see the key differences.
   
   

   

   Thankfully, most data ingestion tools, whether standalone or offered as part of a storage or other solution, are designed to be easy to operate — you should not need a lot of technical knowledge.
   
   It is also common for data ingestion tools to do the work of data transformation or modeling, that is converting your data into a consistent, clean model that does not require your team to have to sift through raw sets of data.
   
   While weighing up your options, consider how your data will be protected, whether the tool is open-source and real-time to allow for flexibility and speed, and — as always — how it plays with the other tools in your data stack.

4. Data analysis and action
   Solutions: BI and analytics tools, marketing automation tools, CDP
   
   At this stage, your data is turning into something you can really use. Tools like Ortto will not only unify data from all of your sources but will allow you to create visual reports and dashboards that turn all that data into insights.
   
   Modern BI and analytics tools, like Ortto, will allow any team member, with any level of technical experience to create and share reports and dashboards when they need them. 
   
   Consider, too, how your data will be operationalized across the entire organization. What is the single source of truth? A platform like Ortto will bring all of your data into one place and provide every team, from sales and marketing to finance and support, with a single source of truth. Another advantage of Ortto is that we offer built-in marketing automation including customer journeys with email, SMS and capture widgets. This means your data is not just viewable, but usable, empowering the marketing team to personalize every interaction and make better, more data-driven decisions.

What to look for in a modern data stack 

As you’re building or reassessing your data stack, consider these three components that set the ‘modern’ apart from the ‘traditional’: 

1. Flexibility and adaptability
As mentioned, a modern data stack exists in the cloud making it, by default, a more flexible option. In addition, a modern data stack should allow for integrations, with data flowing in and out of the tool, and for the ingestion of multiple different types of data streams. 

Dan McGaw, CEO at McGaw.io puts it: "Ensure all of the tools in your product marketing tech stack are integrated. The heartbeat of unified data is managed through a CDP. This way, you can be confident that you’re getting a complete picture of your customer’s journey."

When assessing new tools, ask how the product’s APIs work. Jerome Choo, Director of Growth Marketing at Diffbot shares, “It’s quite common for SaaS companies to lock their data behind their app, believing that this will force users to use their app to get data. This is short-sighted. Data-driven decisions rarely happen in apps. By exposing an API or data warehouse integration, customers are much more easily able to run analyses wherever they like and are thus more likely to actually use your data.” 

Make note of all the existing integrations available, and consider how possible it is for you to integrate data from other sources or build automation with tools like Zapier, how quickly data will be updated, and whether data is dynamic. 

2. Scalability 

Modern tools are elastic, stretching to accommodate trillions of records as required. This scalability sets the ‘modern’ apart from the ‘traditional’ and will ensure that your data stack is set up to grow with your business.

In addition to data points, look for scalability around use cases, activities or objects, and user numbers. Often growth happens fast, and if your data stack cannot accommodate your velocity, you’ll be left scrambling. 

Contract terms can be a great way to assess both scalability and flexibility. Many SaaS tools offer flexible month-to-month contracts, with data and usage limits expanding as your business does. For many SaaS companies, this is the best-case scenario as you will not be overpaying for features or limits you do not use, but you can rest assured that the platform will grow with you. 

3. Simplicity 

The idea here is to democratize access to data, not put more barriers up. A modern data solution will be easy to implement and use. Look for solutions that even a junior member of a marketing team can figure out and use without hours of specialized training or prior experience. 

One thing to look for is a free trial or freemium tier. Not only will this give you a chance to test the features and functionality yourself, but a trial period (especially a 14-day or less trial period) usually indicates that the user can experience value realization in that amount of time. 

Final word 

To build a data-driven culture, your team needs an organized, effective, modern data stack that prioritizes access, ease of use, and flexibility. But it doesn't end there—the importance of building a data-driven culture alongside your data stack cannot be overstated.

In a 2021 article from Towards Data Science, Co-founder of Atlan, Prukalpa identifies the importance of building a ‘Modern Data Culture Stack’ — this concept refers to the “practices, values, and cultural rituals that will help us diverse humans of data come together and collaborate effectively.”