Datafold Blog

The volume of data streaming in from different sources makes data quality hard to maintain during analysis. A survey of data professionals carried out by Dimensional Research [https://www.businesswire.com/news/home/20200611005048/en/Survey-Says-Data-Analytics-Plagued-with-Inefficiencies#:~:text=Nearly%20all%20respondents%20(90%20percent)%20said%20their%20work%20was%20slowed%20by%20numerous%20unreliable%20data%20sources%20over%20the%20last%2012%20months.] showed that 90 percent admitted: “numero

Lightdash [https://github.com/lightdash/lightdash] is an open-source alternative to Looker [https://looker.com] that natively integrates with dbt [https://github.com/dbt-labs/dbt]. It may not be as mature as other open-source products like Metabase, Querybook, or Superset, but it is different in a few essential ways. To explain why Lightdash matters, it's helpful to contemplate what's so special about Looker (besides it being acquired by Google for $2.6B) and how it managed to become the most

Learn what steps your team needs to take to improve data quality and get the most out of your data.

Data quality testing: we all recognize its importance, but sometimes have trouble figuring out exactly how it manifests itself in data engineering workflows. Ingraining data quality within CI/CD and ongoing data modeling will ensure notification upon data quality drift.

Lyft vs. Shopify in testing ETL at scale, using fake data to align your stakeholders, and how to avoid nuclear meltdowns in your data platform.

We are thrilled to announce we are finally on Clubhouse! Our latest conversation of data, engineering, and data quality featured co-host George Xing, Zeeshan Qureshi, and Josh Temple. They joined us from Lyft, Shopify, and Spotify, respectively

Why implement regression testing for ETL code changes, how to align data producers and consumers, and what Data teams at Carta, Thumbtack, Shopify & Clari do to solve data quality.

Take your ETL workflow to the next level with Datafold and dbt integration that automates data testing and provides column-level data lineage

The more people that are looking at the data, and the more apps that are using the data, the faster data quality issues will be identified and resolved.

On the second Data Quality Meetup, we discussed three types of data testing and when to apply them, new-generation ETL frameworks and ROI of open-source data catalogs.

Over the past 10 years, we've seen a great advancement in technologies and tools for analytics and machine learning: with today’s modern analytics stack, we have fast and scalable data warehouses, dirt-cheap data storage, capable ETL orchestrators, and powerful BI tools.