Agile Isn't Dead, It Just Needs the Right Support
How Latttice makes Agile work for data teams by solving the real problems that derail collaboration and delivery.
Agile isn't broken, how we apply it is
Recently I read a LinkedIn post: "Agile is dead."
You've probably also heard the phrase thrown around, but the truth is more nuanced. Agile itself isn't broken, how we've applied it is. Originally designed to help teams collaborate and deliver value quickly, Agile often derails in execution. Standups become rituals, sprints feel rushed, and teams find themselves delivering "something" instead of the right thing.
Where the dysfunction shows up most
Nowhere is this dysfunction more evident than in data projects. The nature of data work, complex pipelines, siloed access, and misaligned priorities between business and data teams, exposes Agile's weaknesses when applied poorly.
When used correctly, Agile solves these challenges by fostering communication, alignment and value-driven delivery. The problem is that data teams often lack the tools and environment to make Agile truly effective.
Latttice is not a replacement for Agile, but a data solution that makes Agile easier, especially in the data space.
Four hurdles that break the rhythm
Data projects are uniquely complex. Business teams want answers yesterday, but data teams face significant hurdles:
Data Silos
Critical data is locked away, slowing progress. According to Gartner (2022), nearly 80% of organizations struggle with siloed data, which hinders agility in data projects.
Disconnect Between Business and Data Teams
Business owners expect results without understanding data complexities. Data teams work in isolation, often solving the wrong problems. Forrester (2022) notes that 56% of data teams are tasked with solving problems not fully understood by business stakeholders.
Repetitive Manual Work
Much of data work involves cleaning and preparing datasets, leaving little time for value-added analysis. DataIQ (2021) found that 60% of data professionals spend most of their time on manual data preparation tasks.
Misaligned Priorities
Agile aims to deliver incremental value, but in data projects "value" often isn't clearly defined or agreed upon. The result: wasted resources and unmet expectations (McKinsey, 2021).
Agile becomes performative. Standups turn into status updates. Sprints become cycles of frustration. Instead of collaboration, teams experience burnout and growing distrust.
, Deloitte, 2023
How Agile can work for data teams
At its core, Agile is about three things:
For data projects, when Agile is done right it eliminates the chaos. Standups clarify blockers and priorities. Sprints deliver incremental insights the business can act on. Retrospectives ensure teams learn and adapt. Most importantly, Agile keeps data teams and business teams aligned on the problem being solved and the outcomes being delivered.
Organizations using Agile effectively in data projects achieved a 30% improvement in time-to-insight, directly improving business outcomes.
, Harvard Business Review, 2022
But even when Agile works as intended, data projects can still hit roadblocks, particularly around data access, manual work, and misaligned expectations.
Data products: the unit Agile was missing
1. Break down the work, epics, stories, tasks
In Agile you break a large goal into smaller, manageable pieces. You don't build the whole app in one go; you tackle one feature at a time. A data product behaves the same way: a self-contained, high-quality dataset designed to serve a specific purpose, delivered as schemas, transformations and APIs, incrementally.
2. Deliver incrementally and iteratively
Agile delivers value early and often, sprint by sprint. For data, that means starting with a Minimum Viable Data Product (MVDP), a basic dataset with essential fields, pipelines and access, then layering in the rest based on feedback.
3. Collaboration and stakeholder involvement
Agile thrives on close collaboration between builders and stakeholders. In a data mesh, the domain team that owns the data collaborates with the consumers who use it, and by involving consumers early, you ensure the data product solves a real problem.
4. Focus on value and feedback loops
Data products are designed for a specific business need. Gather feedback from consumers on usability, quality and structure, and adapt in the next iteration.
5. Continuous improvement
As you build and maintain a data product, you continuously monitor its performance, quality, reliability, usability, and iterate to make it better over time.
Why Agile works for data products
Flexibility: requirements for data products shift constantly as business priorities move and users explore new questions. Value-driven: Agile keeps the focus on data that drives business impact, not features no one uses. Efficiency: incremental delivery means quicker time-to-value and avoids the risks of "big bang" data product launches.
Agile isn't dead, it just needs the right support
Agile isn't the problem; it's how we apply it, especially in data projects, where misalignment and access bottlenecks create chaos.
By empowering teams with access, automation and alignment, Latttice makes Agile easier, enabling data teams to deliver real, measurable value with every sprint.
Ready to transform your data projects? Discover how Latttice can support your Agile workflows and keep your teams focused on what matters.
Join a Data Conversation
Jessie Moelzer.
Jessie Moelzer
Data Tiles
Jessie writes on the practice of data, the rituals, the realities, and the tools that finally let teams ship value instead of status.
Data Conversation with Jessie Moelzer
References
- McKinsey & Company. (2021). Why Data Projects Fail, and How to Make Them Succeed.
- Gartner. (2022). Data Management Challenges in the Enterprise.
- Forrester. (2022). The Disconnect Between Data Teams and Business Needs.
- DataIQ. (2021). Survey on Manual Data Preparation in Organizations.
- Harvard Business Review. (2022). Agile in Data Projects: Lessons from High-Performing Teams.
- Deloitte. (2023). Lessons from Failed Agile Transformations.