Change log

2024-09-07

Adding Week 7 Data Quality
This is the essential part in any data engineering project as the MUST HAVE item. Good Data is Better Than Big Data.
How to use data quality in data warehouse and data pipeline; e.g: Snowflake and dbt. I experience with Deequ and implement that into Spark Job.
Adding Common Components for Analytics Platform and how Data Architecture looks in real-world. That helps you understand the daily works of Engineers who works on data industry.

Add Week6 of Data Engineering: Streaming with Kafka, Kafka use cases, Simple Kafka EcoSystem, etc.
I mention about thinking process to resolve the problem of streaming data with Kafka.
Add the Deep Work section in How to Read.
Add Scope of Data Engineer & Better Data Engineer

Update Basic content, Updated Analytics Platform, Updated Data Achitecture Design
I have added a subscription option for half of part two and whole part three. This was a difficult decision for me, as I genuinely want to share everything I write with all of you, you can check the blogs site. However, I must acknowledge the significant amount of time and effort I have invested in compiling and distilling my free resources into this comprehensive book.
If you find value in this book, I believe the small subscription fee is justified to ensure you receive the latest, up-to-date content in data engineering. I apologize if this decision disappoints anyone. This also allows me to dedicate more time to this book and my writing in general.
For more details, please visit the subscription page. Additionally, I have updated the copyright and legal notice, as well as the privacy policy, to reflect the technology used for managing subscriptions.
Furthermore, I am going to pack all the ebook into single package so that you can checkout and download it. Your support always meaningful with me. Thanks