3 Reasons Engineers Struggle with Snowflake for Multi-Tenant Analytics

Snowflake is popular for a reason. They made it super simple to start with pay-as-you-go pricing. Great.

But now you have gone far and that volume is getting expensive. How are you going to make this work in a multi-tenant SaaS application? Concurrency alone is going to dramatically increase your Snowflake costs.

This is a trend we’re seeing from customers and prospects on a regular basis. They use Snowflake for internal analytics and thought they could use it for multi-tenant analytics. In every case, their Snowflake bill gets to be too much.

We took a look at the Snowflake design patterns document for creating a multi-tenant analytics backend. Here are three potential challenges when using Snowflake for a multi-tenant analytics database, based on its design patterns.

This is particularly relevant for software engineering leaders tasked with building software components to connect Snowflake to the front-end of a web application.

3 Challenges with Snowflake for Multi-Tenant Analytics

1) Maintaining the Entitlement Table:

Why is Snowflake Complex?

The entitlement table is a critical component governing each tenant’s data access. The access control model needs regular updates so it’s in sync with the constantly changing tenant structure. Things like new tenants added, permissions updated, etc. As the number of tenants grows, managing this table manually becomes increasingly error-prone and time-consuming.

Primary Risks:

• Data security: Errors in the entitlement table could lead to unauthorized access to sensitive customer data. This is bad news when sensitive information is in play that could result in data breaches and compliance violations.
• Application downtime: Issues with the entitlement table could break the application’s ability to serve data correctly. Disruptions for end-users are a sure-fire way to decrease CSAT scores.
• Scalability limitations: The entitlement table management can become a bottleneck without proper automation and processes. You want your business to grow, but if the analytics feature can’t support more tenants, it will be a problem.

2) Authenticating and Routing Users:

Why is Snowflake Complex?

Implementing a robust login and authorization management system that sits between the front-end and Snowflake can be complex. It involves managing user sessions, secrets, RBAC configurations, and routing logic. All this ensures users get data from the correct databases, schemas, and warehouses based on their roles and permissions.

Primary Risks

• Security risk: Improper implementation of authentication and authorization logic can lead to security vulnerabilities, allowing unauthorized access to sensitive data.
• User experience issues: Incorrect routing of users can result in poor user experiences. If users see wrong data or can’t access what they need, it’s up to you to fix it….yesterday.
• Compliance violations: Failure to enforce proper access controls can lead to compliance violations. This is especially true in regulated industries with strict data privacy and access privilege requirements like healthcare analytics. Imagine being responsible for leaking social security numbers.

3) Isolating Workloads and Managing Compute Resources:

Why is Snowflake Complex

Deciding on the appropriate level of isolation for different workloads (development, ingestion, transformation, serving) is hard enough. Add in having to determine the optimal compute resource allocation (shared vs. dedicated) for tenants can be complex.

This often goes beyond the skill set of a typical software engineer. It requires a deep understanding of the embedded analytics requirements, performance characteristics, and cost considerations.

Primary Risks:

• Performance issues: Improper workload isolation or resource allocation can lead to performance bottlenecks, slow query responses, and poor user experiences.
• Wasting Money: Over-provisioning compute resources can lead to unnecessary costs. While under-provisioning can result in performance degradation and customer frustration.
• Scalability limitations: Failure to design a scalable architecture can limit your ability to grow. You want more tenants and data volumes to grow your business. Multi-tenant analytics software needs to scale too.

These challenges highlight the importance of careful planning and robust software engineering practices.

Challenge	Description	Risks
Maintaining the Entitlement Table	Managing the critical entitlement table that governs each tenant’s data access becomes increasingly complex as the number of tenants grows. Updates for new tenants, permission changes, etc., need to be managed.	– Data security breaches – Application downtime – Scalability limitations
Authenticating and Routing Users	Implementing a robust login, authorization, and user routing system that ensures users access the correct databases, schemas, and warehouses based on their roles and permissions can be complex.	– Security vulnerabilities – Poor user experience – Compliance violations
Isolating Workloads and Managing Compute Resources	Deciding on the appropriate level of workload isolation (development, ingestion, transformation, serving) and determining optimal compute resource allocation (shared vs. dedicated) for tenants requires deep expertise.	– Performance bottlenecks – Wasted costs (over-provisioning) – Scalability limitations

Or, you could not build this yourself at all.

This is Qrvey’s sweet spot. We take care of the hard work for multi-tenant analytics so you don’t have to build it yourself.

Here’s the with-and-without Qrvey in a nutshell.

WITH QRVEY

WITHOUT QRVEY

Basically, our solution gives you the semantic layer, the multi-tenant data lake and the unified data pipeline. Together this means your embedded analytics function scales now and into the future.

Better experience for your end-users and you save your engineering team valuable time. Who doesn’t want that?

First, watch our video below that goes a bit deeper.

Second, check out our Snowflake cost calculator to estimate your potential savings.

Then let’s chat about how we can work together to make your engineering team more efficient.