List: Data Engineering | Curated by João Vazao Vasques

Dec 17, 2024
43 stories
Data EngineeringAwesome data engineering articles
In
Data Engineer Things
by
Dani
Apache Iceberg: The Hadoop of the Modern Data Stack?The bigger they are the harder they fall.
Dec 12, 2024
6
Dec 12, 2024
6
In
TDS Archive
by
Jesus L. Lobo
Detecting real-time and unsupervised anomalies in streaming data: a starting pointSensors enable the Internet of Things (IoT) by collecting the data for smarter decisions in all kinds of systems. Data are usually…
Jan 10, 2020
2
Jan 10, 2020
2
Analytics at Meta
The future of the data engineer — Part IIntroduction
Apr 3, 2023
26
Apr 3, 2023
26
In
Qodea Google Cloud Tech Blog
by
Keven Pinto
Running pyspark jobs on Google Cloud using Serverless DataprocRun Spark batch workloads without having to bother with the provisioning and management of clusters!.
Mar 24, 2022
1
Mar 24, 2022
1
Andy Bryant
Processing guarantees in KafkaEach of the projects I’ve worked on in the last few years has involved a distributed message system such as AWS SQS, AWS Kinesis and more…
Nov 16, 2019
4
Nov 16, 2019
4
In
TDS Archive
by
💡Mike Shakhomirov
Data pipeline design patternsChoosing the right architecture with examples
Jan 2, 2023
8
Jan 2, 2023
8
In
TDS Archive
by
Chengzhi Zhao
10 Fantastic Classic Books For Data EngineeringThe Books Make You Success For Data Engineers
Dec 6, 2022
2
Dec 6, 2022
2
Apache DolphinScheduler
Apache Open-source Projects in Modern Data StacksEditor: Detong, github.com/mischaZhang
Nov 23, 2022
Nov 23, 2022
Darth Data
Data is not the new oil! Lessons in evolving a Data PlatformOur data platform has been running for years. We hired a bunch of data scientist and engineers, too many unused tables which are running…
Oct 27, 2022
Oct 27, 2022
Ahmed Omrane
BQ+DBT: 5 proven practices to scale you analytics infrastructure effectively without exploding your…At The Fabulous, we have been using BQ and DBT as the core of our Data Analytics for the past 2 years. As of end of 2022, we have fully…
Nov 13, 2022
1
Nov 13, 2022
1
The Data Observer
Data Engineering Best Practices: How Big Tech & FAANG Firms Manage and Optimize Apache KafkaThe popular open-source messaging/streaming system, Apache Kafka, is a key enabler for some of the most data-driven and disruptive…
Jun 22, 2022
Jun 22, 2022
Alexandre Beauvois
Data Platforms: The PresentThe Evolution of Data Platforms
Jun 10, 2022
1
Jun 10, 2022
1
Dagster Blog
Postgres: a better message queue than Kafka? | Dagster BlogWe shipped Dagster Cloud 1.0 in August. It’s been pretty successful so far, and this is the first in a series of blog posts talking about…
Oct 5, 2022
8
Oct 5, 2022
8
In
Artefact Engineering and Data Science
by
Benoît Goujon
dbt coalesce 2022 recapThis year’s edition was taking place in New Orleans. And as in the past editions, we learned a ton about the analytics engineering…
Oct 21, 2022
Oct 21, 2022
In
Creandum
by
Staffan Helgesson
A New Era Of Data Analysis — Enter MasonAfter a decade of quasi-code tools and pure no-code tools for data analysis and visualization, users of all sorts still return to SQL.
Oct 18, 2022
1
Oct 18, 2022
1
In
Data Engineer Things
by
Xinran Waibel
Data Engineering Excellency at NetflixTakeaways from two years as a data engineer at Netflix.
Oct 7, 2022
8
Oct 7, 2022
8
Alexandre Beauvois
Data Platforms: The FutureThe Evolution of Data Platforms
Jul 5, 2022
1
Jul 5, 2022
1
In
McDonald’s Technical Blog
by
Global Technology
McDonald’s event-driven architecture: The data journey and how it worksPart two of event-driven architecture post.
Aug 30, 2022
5
Aug 30, 2022
5
Asu Rana Yol
Future of Data EngineeringWhat is data engineering?
Aug 10, 2022
Aug 10, 2022
In
Geek Culture
by
Paul Lee
I Got Rejected From Meta’s Data Engineer Interview— Here’s What I LearnedThrough every success… every failure… there’s a lesson to learn in every endeavor  — Paul Lee (Author)
Jul 1, 2022
21
Jul 1, 2022
21